Skip to main content

6.6.0-1/2 Hotfix - Pending CID Report data and Hub/Collector Filesystems at 100% usage

Ryan Wilp
  • Updated

Product: Stratusphere FIT/UX

Product Version: 6.6.0-1, 6.6.0-2

Expires on: 365 days from publish date

Updated: Jan 31, 2023

 

Problem:

  • Report data stops coming in and pending reports keep increasing
  • /var/log/ and possibly /var/lib/jakarta-tomcat/ fill up to 100% on hub and/or collector
  • This error may be in tnt-backend.log, you can run grep -i 'rollback only' /var/log/tnt/tnt-backend.log as root to check:
2022-04-22 18:10:31,253 ERROR c.t.i.i.b.s.ServiceThrowsAdvice [http-nio2-5501-exec-7] Service threw exception during execution                                            
org.springframework.transaction.UnexpectedRollbackException: JTA transaction unexpectedly rolled back (maybe due to a timeout); nested exception is javax.transaction.Roll
backException: Transaction set to rollback only

Possible resolution:

First, make sure Local Directory auto-registration is enabled.

mceclip4.png

Putty into the hub and CID collectors and run the following commands. For on-premises appliances, the user is friend, AWS is ec2-user, and Azure is azureuser or whatever unique username was chosen.  If the appliance does not have internet access, you can download the rpm below and WinSCP it over (skip the wget command):

  • sudo bash
  • wget https://cdn.liquidware.com/stratusphere/hotfix/lwl-collector-backend-1.0.0-80358.noarch.rpm
  • rm -f /var/log/tnt/*log* /var/log/httpd/*log* /var/log/messages* /var/lib/jakarta-tomcat/default/logs/*log*
  • rpm -Uvh --nosignature lwl-collector-backend-1.0.0-80358.noarch.rpm
  • systemctl restart lwl-backend lwl-smx httpd

NOTE: The Database may take anywhere from 10 minutes to 30 minutes to get back up to working again. If you experience a 503 Error on the browser, please be patient as the db is cleaning up some outstanding processing after patching.

Related articles