Servletd - Thread Full Error

This error was spotted in servletd.log file after which the GUI stopped responding

org.apache.tomcat.util.threads.ThreadPool logFull
SEVERE: All threads (250) are currently busy, waiting. Increase maxThreads (250) or check the servlet status

Currently the max threads is set to 250.

When would this number be reached? Is it when the number of users increase?

Missed to mention - After restarting Servletd, GUI was fine

Thanks

Find more posts tagged with

Comments

mdang

Version of TS? Platform?
Assuming Solaris, run jstack on the servletd pid to see what all the threads are doing.

JulesFromAUS

Thanks for your response.
Teamsite version is 6.5 SP2 and it is running on windows

mdang

General recommendation is to upgrade. TS 6.5 SP2 is not even the latest SP for 6.5. Preferably, upgrade to TS 6.7.2. There are quite a few thread leaking issues fixed after 6.5 SP2.

To actually debug the issue on hand (Windows and TS 6.5 really makes it tough), you would need to start servletd in debug mode and attach to it to issue thread dump. But if you are on product, then this is usually not an option. Smiley Sad

JulesFromAUS

Oh! Alright! Then, the only solution I have is to restart servletd.
One question about these threads - are they because of an increase in the number of users/requests?

Thanks

mdang

Generally, more concurrent users require more threads.
But 250 threads is large enough thread pool that can serve ~80 concurrent users that are clicking on things simultaneously. Does your server has that high frequent of use? If so, increasing the thread limit may help a bit. But I suspect more of a thread leak than a heavy load.

JulesFromAUS

The load was not that high and defn we had under 80 users at that time.
Can not having a session timeout for the content center website cause this prob?
Which would mean there would be some idle sessions as well.

Thanks!

mdang

No, have not seen a session timeout caused a thread leak.

The whole story can be told by a thread dump. I would recommend keeping an eye on this error. If it happens frequent enough, there is a way to get the dump without affecting production.

The idea is to run servletd in console mode. This won't affect performance. When the problem happens again, press Control-Break on the console to force a thread dump. The output would be the key.

Frederik

suggest you monitor your JVM heap size usage and threads by enabling this at
http://your_server/iw-cc/command/iw.admin.teamsite.jvm_performance (master login required, IIRC)