Teamsite Performance Issues

We have teamsite 6.5 on solaris .

Recently we observe teamsite performance issues . July 7th and 9th , teamsite became very slow and we had to restart the server.

We have mainly observed

1. java.lang.OutOfMemoryError in serveltd.log

2 .Invalid workflow errors

Workflow object 2336743 does not exist.
Workflow object 2336768 does not exist.
ID 2336768 is invalid
Workflow object 2336767 does not exist.
Workflow object 2336784 does not exist.

3. SSL Handshake errors

[09/Jul/2008 11:23:49 06657] [error] SSL handshake failed (server qtcfs012.ca.nortel.com:443, client 47.129.93.35) (OpenSSL library error follows)
[09/Jul/2008 11:23:49 06657] [error] OpenSSL: error:14094415 Smiley Frustrated

SL routines Smiley Frustrated

SL3_READ_BYTES:sslv3 alert certificate expired

We are planning to go ahead with increasing java memory. These are the steps as per the document:-

1. Edit the /iw-home/servletd/bin/servletd script:
JAVA_OPTS="-server -Xms64m -Xmx512m -Djava.library.path=$IW_HOME/cssdk:$IW_HOME/
lib -Dcssdk.cfg.path=$IW_HOME/cssdk/cssdk.cfg"
2. Increase to -Xmx1024m (3GB max)
3. Run iwreset -ui

Anybody has tried this? Is there any risks associated with this? Do we have to stop the teamsite server for this?

2. Invalid workflow errors and SSL handshake errors , we see frequently. Can they also be the reason for performance issues?Any suggestions on these two?

Find more posts tagged with

Comments

John742

1) Increasing JVM for Tomcat is a normal and IWOV recommended practice for the error message you are receiving(java.lang.OutOfMemoryError in serveltd.log). An iwreset -ui is all that is needed since you are only impacting the serveltd process. But instead of just increasing this, I would encourage you to find RC on why this started to occur.

2) What log are you seeing the workflow does not exist errors? This seems like a tsreport.log error? If so, it means the reporting is trying to look up a workflow reference to pull data,but it has already been deleted. there is an iw.cfg setting that you should use in this case(delete_jobs_on_completion=false), otherwise please provide info on where this error is logging.

3) Where is the ssl error logging? Is this your teamsite server, qtcfs012.ca.nortel.com? Over 443? What type of cert are you using for this? and finally is it expired?

All 3 issues as described are not related to each other.

Antharix

Check to see if there are any hanging tasks on the system. such tasks often consume lot of TS resources and block them. Deleting such tasks will definitely improve system performances.

moonkhanna

We have teamsite 6.5 on solaris .

Recently we observe teamsite performance issues . July 7th and 9th , teamsite became very slow and we had to restart the server.

We have mainly observed

1. java.lang.OutOfMemoryError in serveltd.log

2 .Invalid workflow errors

Workflow object 2336743 does not exist.
Workflow object 2336768 does not exist.
ID 2336768 is invalid
Workflow object 2336767 does not exist.
Workflow object 2336784 does not exist.

3. SSL Handshake errors

[09/Jul/2008 11:23:49 06657] [error] SSL handshake failed (server qtcfs012.ca.nortel.com:443, client 47.129.93.35) (OpenSSL library error follows)
[09/Jul/2008 11:23:49 06657] [error] OpenSSL: error:14094415SL routinesSL3_READ_BYTES:sslv3 alert certificate expired

We are planning to go ahead with increasing java memory. These are the steps as per the document:-

1. Edit the /iw-home/servletd/bin/servletd script:
JAVA_OPTS="-server -Xms64m -Xmx512m -Djava.library.path=$IW_HOME/cssdk:$IW_HOME/
lib -Dcssdk.cfg.path=$IW_HOME/cssdk/cssdk.cfg"
2. Increase to -Xmx1024m (3GB max)
3. Run iwreset -ui

Anybody has tried this? Is there any risks associated with this? Do we have to stop the teamsite server for this?

2. Invalid workflow errors and SSL handshake errors , we see frequently. Can they also be the reason for performance issues?Any suggestions on these two?

Here is what you could do as thumb rule for performance issues -

1. Run iwreset on your server.
2. Check to see if there are stuck jobs .. If yes, ensure you have a backup of these and kill these jobs..
3. Get the list stale locks script from IWOV Support and see if you have stale locks on your branches.. These often stay unobserved and keep on accumulating. If you find any, remove these using unlock clt.