TeamSite hangs on delete / rename file

Hi Team,

We are facing an issue in TeamSite 7.1. Sometimes, when we try to delete / rename a file, it just hangs. After abot 10 - 15 minutes, it shows up the error:

"Interwoven web daemon was unable to contact Interwoven Servlet Engine. Interwoven Servlet Engine may be down or running on a unexpected host or port".

Then we need to restart TeamSite and it works fine. This has happened about 3-4 times in our environment.

We also enabled the debuggers, but did not get *anything* in the content_center.log / servletd_out.log. Error.log says:

[Thu Aug 30 02:14:03 2012] [error] [client 10.160.8.5] proxy: error reading status line from remote server localhost, referer: http:///iw-cc/command/iw.ui
[Thu Aug 30 02:14:03 2012] [error] [client 10.160.8.5] proxy: Error reading from remote server returned by /iw-cc/command/iw.group.delete, referer: http:///iw-cc/command/iw.ui

Has anyone faced any issue like this? Any pointers on this is appreciated.

Thanks
Payal

Env: RHEL 5.5
iwserver: 7.1.0 Build 1255 Interwoven

Find more posts tagged with

Comments

nipper

when it happens do an iwstat and df -h

see if anything is out of ordinary. Also have you done a iwfsck lately, could be backing store issues,

PayalGarg

Thanks Nipper. We are good on disk space. But next time it happens we'll see if iwstat provides any info.

Will run iwfsck as well.

Thanks
Payal

nipper

Thanks Nipper. We are good on disk space. But next time it happens we'll see if iwstat provides any info.

Will run iwfsck as well.

Thanks
Payal

I did not want a df -h for the disk space.

Run it anyway. I have seen cases where df -h will hang when it tried to get data about the IFS.

PayalGarg

Yeah, Even we have found those issues, specially when trying some direct file operation (like zip / tar) on the IFS.

PayalGarg

Running iwfsck on content store did not show any errors:

$ sudo /usr1/app/interwoven/TeamSite/private/bin/iwfsck -x -a -t -b /usr1/app/interwoven/iw-store/ -d -s -nofmtchk

Total points read: 7118156
Directories walked: 68056
Files walked: 1548960
Symlinks walked: 0
TeamXML files walked: 0

Zombies walked: 7534
Total disk reads: 6270922
Total Run time: 1289 secs -> 21 Mins 29 Secs
FINISHED 0 error(s) found

Adam Stoller

Running iwfsck on content store did not show any errors:

$ sudo /usr1/app/interwoven/TeamSite/private/bin/iwfsck -x -a -t -b /usr1/app/interwoven/iw-store/ -d -s -nofmtchk

Generally speaking, unless advised by Support, you should only use the '-d' (and possibly '-b') flags when running iwfsck. However, you seemed to have ruled out one potential source of the issue (possibly). Other possible issues:

PayalGarg

Thanks for your replies.

We are getting this issue again and still don't have any clue. Have shared the logs with Support as well.

It looks like, we are good on RAM and hard disk space:

1) RAM:
$ free -t -m
total used free shared buffers cached
Mem: 32187 32021 165 0 2074 12122
-/+ buffers/cache: 17824 14362
Swap: 16002 0 16002
Total: 48189 32021 16167

2) Iwstore - 242 G available:
/dev/mapper/datavg-iw--store
296G 42G 242G 15% /usr1/app/interwoven/iw-store

3) IWHOME - 46 G available
/dev/mapper/datavg-lv--usr1
148G 98G 46G 68% /data01

4) Good on tmp and /var/tmp:
/dev/sda6 3.8G 107M 3.5G 3% /tmp
/dev/sda5 4.8G 1.2G 3.4G 26% /var

5) Apart from TeamSite and OpenDeploy, we run some data jobs on the server. These are quite extensive jobs that read xmls from ESB and save to database and so on.

6) We do not run any anti virus software.

Following are the errors that we get:

[Tue Sep 11 15:14:59 2012] [error] [client 10.160.8.5] proxy: error reading status line from remote server localhost, referer: http:///iw-cc/command/iw.ui
[Tue Sep 11 15:14:59 2012] [error] [client 10.160.8.5] proxy: Error reading from remote server returned by /iw-cc/command/iw.group.delete, referer: http:///iw-cc/command/iw.ui
[Tue Sep 11 15:19:20 2012] [error] [client 10.160.8.5] proxy: error reading status line from remote server localhost, referer: http:///iw-cc/command/iw.ui
[Tue Sep 11 15:19:20 2012] [error] [client 10.160.8.5] proxy: Error reading from remote server returned by /iw-cc/command/iw.group.move, referer: http://iw-cc/command/iw.ui

When checked through firebug, it actually shows up a "502 bad gateway error". We also get the same error, when we try to use PLC workflow for images. It errors out on "setting metadata" step and displays "bad gateway".

Ran iwstat -c as well:

Store Status
iwadmin Running
elt Running
workflow Running
ID Thread User Duration Store Operation
0xe758d993 0xd05fcb90 vgargpa 0.000 - GetServerStatus
Cache Active Dirty To Purge Available
iwadmin 1605 0 0
elt 23695 0 0
workflow 1559 0 0
Total 26859 0 0 3141

No luck so far..

Apprciate your inputs.

Thanks
Payal

sanu

Hi All,

I am facing the same issue and was checking the forum for some help. Payal, How did you get this issue fixed? Appreciate your response.

lesbleus

we had same issue yesterday and found DB connectivity errors in content_center.log. Fixing DB connection resolved the problem.