nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Cache Corruption error message

Natasha

Hi All,

We are getting the following error messages in SystemOut.log of LiveSite runtime server :-

[HTML]
[5/9/09 19:17:25:459 BST] 00000067 SystemOut O ---sitemapUtils Smiley Tongue

arseLeafNodeHierarchy--- looped!
[5/9/09 19:17:25:460 BST] 00000067 SystemOut O ---sitemapUtils Smiley Tongue

arseLeafNodeHierarchy--- looped!
[5/9/09 19:17:53:581 BST] 0000006e CoordinatorCo W HMGR0152W: CPU Starvation detected. Current thread scheduling delay is 14 seconds.
[5/9/09 19:18:13:571 BST] 0000000f ApplicationMo W DCSV0004W: DCS Stack DefaultCoreGroup at Member nd_cellname\snt5k024\server1: Did not rec
eive adequate CPU time slice. Last known CPU usage time at 19:16:26:965 BST. Inactivity duration was 106 seconds.
[5/9/09 19:18:13:860 BST] 00000039 SystemOut O ---sitemapUtils Smiley Tongue

arseLeafNodeHierarchy--- looped!
[5/9/09 19:18:13:861 BST] 00000039 SystemOut O ---sitemapUtils Smiley Tongue

arseLeafNodeHierarchy--- looped!
[5/9/09 19:18:13:862 BST] 00000039 SystemOut O ---sitemapUtils Smiley Tongue

arseLeafNodeHierarchy--- looped!
[5/9/09 19:18:13:863 BST] 00000039 SystemOut O ---sitemapUtils Smiley Tongue

arseLeafNodeHierarchy--- looped!
[5/9/09 19:18:29:562 BST] 00000035 SystemOut O 19:18:29,562 ERROR [URLChecker] The status of the response was:404 when calling: http://www
.fundslibrary.co.uk/fundslibrary.dataretrieval/Documents.aspx?ISIN=GB0032310129&type=packet_fund_doc_factsheet_s_and_p&user=fidelity&TRACK1=ad
viser&TRACK2=GB0032310129
[5/9/09 19:18:37:635 BST] 00000074 SystemOut O 19:18:37,635 ERROR [SQLUtils] java.sql.SQLException: Non supported SQL92 token at position:
356: Sort
[5/9/09 19:19:27:714 BST] 0000007a SystemOut O 19:19:27,713 ERROR [URLChecker] The status of the response was:404 when calling: http://www
.fundslibrary.co.uk/fundslibrary.dataretrieval/Documents.aspx?ISIN=GB00B2Q4RZ35&type=packet_fund_doc_factsheet_s_and_p&user=fidelity&TRACK1=ad
viser&TRACK2=GB00B2Q4RZ35
[5/9/09 19:19:30:013 BST] 0000007a SystemOut O 19:19:30,012 ERROR [URLChecker] The status of the response was:404 when calling: http://www
.fundslibrary.co.uk/fundslibrary.dataretrieval/Documents.aspx?ISIN=GB00B2Q4RZ35&type=packet_fund_doc_reports_and_accounts&user=fidelity&TRACK1
=adviser&TRACK2=GB00B2Q4RZ35
[5/9/09 19:19:35:540 BST] 0000007d SystemOut O 19:19:35,539 ERROR [JCSCache] cache corruption!!! [580]
[5/9/09 19:19:35:540 BST] 0000007d SystemOut O 19:19:35,540 ERROR [JCSCache] purging adviserservices:default Smiley Tongue

rimary:en:1233402198635:1
[5/9/09 19:19:35:555 BST] 0000007d SystemOut O 19:19:35,555 ERROR [JCSCache] purging adviserservices:default Smiley Tongue

rimary:en:1233402198638:1
[5/9/09 19:19:35:636 BST] 0000007d SystemOut O 19:19:35,635 ERROR [JCSCache] cache corruption!!! [581]
[5/9/09 19:19:35:636 BST] 0000007d SystemOut O 19:19:35,636 ERROR [JCSCache] purging adviserservices:default Smiley Tongue

rimary:en:1236327520443:1
[5/9/09 19:19:35:651 BST] 0000007d SystemOut O 19:19:35,651 ERROR [JCSCache] purging adviserservices:default Smiley Tongue

rimary:en:1236327520442:1
[5/9/09 19:19:35:803 BST] 0000007d SystemOut O 19:19:35,803 ERROR [JCSCache] cache corruption!!! [582]
[5/9/09 19:19:35:803 BST] 0000007d SystemOut O 19:19:35,803 ERROR [JCSCache] purging adviserservices:default Smiley Tongue

rimary:en:1233402198636:1
[5/9/09 19:19:35:818 BST] 0000007d SystemOut O 19:19:35,818 ERROR [JCSCache] purging adviserservices:default Smiley Tongue

rimary:en:1233402198639:1

[/HTML]

Because of this issue, the web site went down so we had to do a stack switch in order to resolve the issue on LIVE. On investigation, we noticed the above errors in log. We deleted the cache.WebRoot* and .jar files present under "sites" folder and have been monitoring it continously over a period of 4 hours, the problem hasn't reoccurred.
Does anyone know about the exact cause of this issue particulary cache corruption?

Environment Details - LiveSite 3.0,TeamSite 6.7.1 SP1, Solaris 10.

Thanks,
Natasha.

Find more posts tagged with

Comments

Natasha

We have again started facing the CPU starvation exception messages in the logs, could anyone point out what can been done to resolve it?

vpatel

Seems like cache is corrupted. Have you tried resetting the cache?

StringInCrosstabSummary.zip

tanguyr

We have again started facing the CPU starvation exception messages in the logs, could anyone point out what can been done to resolve it?

We had a problem with LSRT cache corruption last year (we were on LSRT 3.0 at the time), we contacted support and they sent us a patch labled "cache_patch_LST3096.zip". According to them the issue was fixed on TS 6.7.2 / LSRT 3.1

I recommend you contact support and ask them to send you this patch

/t

Natasha

We reset the cache yesterday, got the servers restarted and after that we didn't get this "Cache Corruption" message again. But CPU Starvation messages have become a quite frequent now .We have also monitored the GC levels which shoot up at times resulting in slow response time and eventually leads to server crashing because of which we had to get the stack switched couple of times.
Does anyone know the reason for such a behavior?

Thanks,
Natasha.

vpatel

GC issue could be (and it usually is) totally dependent on your implementation. There are some settings to tune the caching and GC and all that stuff. I'd say get someone who knows what he/she is doing before adjusting those or do it at your own risk. Better yet, try it on a test machine first and then perform some sort of load testing.

tec_iwov

the cache corruption errors you are seeing means you have the patched code.

you should try to do a thread dump. this may shed light on what is thread bound.

If your doing DB queries make sure to explicitly close the connection statement and ruleset. some DB's such as oracle will lock up if these objects are closed explicitly.

You might want to do some GC tuning.

this is for 1.5 but i think it's still relevant for later jdk releases

http://java.sun.com/docs/hotspot/gc1.4.2/

Natasha

Thank you for your responses, I will keep you posted on the resolution approach we adopt in order to fix it.