Teamsite 6.7.1 iwteamsite starts and stops immediately

Hi, Yes, we are still using 6.7.1 Smiley Sad . This morning, all of a sudden Teamsite service (iwteamsite.exe) has stopped running. When I try to start the service, it starts and then immediately stops (I refresh immeidately after starting and then see the service is stopped).

I'm able to see the login page and when tried to login it says "Unable to establish connection with server"

All I see in iwserver.log is

[Tue Jun 23 14:37:44 2015] ERROR: iwserver did not come up in 600 seconds
[Tue Jun 23 14:37:44 2015] ERROR: Please look at \iw-home\local\logs\iwtrace.log for more information
[Tue Jun 23 14:37:44 2015] Cannot execute iwserver

And iwtrace.log does not provide much insight.

TeamSite I18N initialization successful!
[Tue Jun 23 14:37:41 2015]
*** IWServer Locale is --> English_UnitedStates.MS1252@Binary
[Tue Jun 23 14:37:41 2015] Watch Event Thread started
[Tue Jun 23 14:37:41 2015] TeamSite Server Starting.
[Tue Jun 23 14:37:41 2015] iwserver: 6.7.1 Build 65609 Interwoven 20070509
[Tue Jun 23 14:37:41 2015] Network monitor is started.
[Tue Jun 23 14:37:41 2015] IW Global Group = xxxxxxxxx\iwglobal
[Tue Jun 23 14:37:41 2015] Reading userops file \iw-home\private\etc\userops.xml ...
[Tue Jun 23 14:37:41 2015] Done reading userops file \iw-home\private\etc\userops.xml
[Tue Jun 23 14:37:41 2015] Reading userops file \iw-home\private\etc\userops_ui.xml ...
[Tue Jun 23 14:37:41 2015] Done reading userops file \iw-home\private\etc\userops_ui.xml
[Tue Jun 23 14:37:41 2015] Reading users and groups...
[Tue Jun 23 14:37:41 2015] Local system: "xxxxxxxxxxx"
[Tue Jun 23 14:37:41 2015] Loading disk cache from Tue Jun 23 13:06:05 2015, revision = 1.
[Tue Jun 23 14:37:41 2015] done. 38 users and 65 groups read.
[Tue Jun 23 14:37:41 2015] Log Permissions Disabled
[Tue Jun 23 14:37:41 2015] Warning: this system is configured with different values for the
[Tue Jun 23 14:37:41 2015] initial and maximum pagefile size. It's strongly recommended that
[Tue Jun 23 14:37:41 2015] you use the same value. Low memory condition is triggered based
[Tue Jun 23 14:37:41 2015] on the initial size.
[Tue Jun 23 14:37:41 2015] Cache size: 30000
[Tue Jun 23 14:37:41 2015] setting rollover threshold to 10485760
[Tue Jun 23 14:37:41 2015] setting num rollovers to 10
[Tue Jun 23 14:37:41 2015] Initialized store 0x1 at "D:\iw-store\workflow" (version=300008, next id=0x153400).
[Tue Jun 23 14:37:42 2015] Initialized store 0x64 at "D:\iw-store\default" (version=300008, next id=0x378800).

Can some one please help? please let me know if any other information is required.

Find more posts tagged with

Comments

Adam Stoller

Did anything change with regard to the network (domain controllers, active directory servers, etc.) or did anyone change anything in IWHOME/etc/iw.cfg recently?

Often, the error "iwserver did not come up in 600 seconds" relates to iwserver attempting to connect to all the domains / active directory servers on the network, or for which it has been configured in the iw.cfg file.

Quick_Learner

Hi Ghoti,

Nothing had changed in any of the configuration files. And as far as I know nothing was modified in the network (although I cannot say for sure). I will check on the network thing.

I have also made this observation. I see in error.log a notice that the Apache server was built on

Server built: Jun 23 2006 10:06:35

Is it any coincidence that the server simply stopped on the same date? And I received the first error report this morning at 11:30.

Do we have any expiry for the Apache server?

Also, I see the following Warn and Notice in error.log (And I just observed that these statements are logging up since the installation of this Teamsite). So, I dont see any net new errors that would trigger a problem

we rebooted the server at 5:00 PM (17:00)

Tue Jun 23 17:00:34 2015] [warn] RSA server certificate CommonName (CN) `teamsite.interwoven.com' does NOT match server name!?
[Tue Jun 23 17:00:34 2015] [warn] RSA server certificate CommonName (CN) `teamsite.interwoven.com' does NOT match server name!?
[Tue Jun 23 17:00:34 2015] [notice] Apache/2.0.58 (Win32) mod_ssl/2.0.58 OpenSSL/0.9.8 mod_fastcgi/2.4.2 configured -- resuming normal operations
[Tue Jun 23 17:00:34 2015] [notice] Server built: Jun 23 2006 10:06:35
[Tue Jun 23 17:00:34 2015] [notice] Parent: Created child process 2648
[Tue Jun 23 17:00:35 2015] [warn] RSA server certificate CommonName (CN) `teamsite.interwoven.com' does NOT match server name!?
[Tue Jun 23 17:00:35 2015] [warn] RSA server certificate CommonName (CN) `teamsite.interwoven.com' does NOT match server name!?
[Tue Jun 23 17:00:35 2015] [notice] Child 2648: Child process is running
[Tue Jun 23 17:00:35 2015] [notice] Child 2648: Acquired the start mutex.
[Tue Jun 23 17:00:35 2015] [notice] Child 2648: Starting 100 worker threads.
[Tue Jun 23 17:00:35 2015] [notice] FastCGI: process manager initialized
[Tue Jun 23 17:00:35 2015] [warn] FastCGI: server "D:/iw-home/cssdk/fastcgi/cssdkfcgi.exe" started (pid 3108)

Also, these were some of the logs that got into error.log this morning. However they are not logged after the reboot. Should the ip address of the client below match the ip address of the Teamsite server?:

[Tue Jun 23 13:12:17 2015] [error] [client xx.xx.xx.****] proxy: error reading status line from remote server localhost
[Tue Jun 23 13:12:17 2015] [error] [client xx.xx.xx.****] proxy: Error reading from remote server returned by /default/main/desktop.ini
[Tue Jun 23 13:12:17 2015] [error] [client xx.xx.xx.****]
[Tue Jun 23 13:12:18 2015] [error] [client xx.xx.xx.****] proxy: error reading status line from remote server localhost
[Tue Jun 23 13:12:18 2015] [error] [client xx.xx.xx.****] proxy: Error reading from remote server returned by /default/main
[Tue Jun 23 13:12:18 2015] [error] [client xx.xx.xx.****]
[Tue Jun 23 13:48:38 2015] [error] [client xx.xx.xx.****] proxy: error reading status line from remote server localhost
[Tue Jun 23 13:48:38 2015] [error] [client xx.xx.xx.****] proxy: Error reading from remote server returned by /default

Thanks,

Nagesh K

Quick_Learner

Hi Ghoti,

I just observed that the domain drop down is not showing up on the login page. So, this looks like teamsite server is not able to contact the Active Directory. Thanks for your input. I will check on this.

Thanks,

Nagesh K

Quick_Learner

Hi Ghoti,

I had always wondered about this:

As per the Teamsite documentation:

If your [iwcgi] section does not contain a domain_list, domains are automatically detected and displayed

Do you know how exactly Teamsite detects the domains? How does it even know what Active Directory to connect to? Is it based on the domain of the server where Teamsite is being installed?

Thanks,

Nagesh K

Quick_Learner

Hi Ghoti,

I see this in the System event log:

The Interwoven TeamSite service terminated with service-specific error 1067 (0x42B)

And when I search with this error in the forums, I saw a thread that says this error is related to the TS license being invalid (and the thread was for TS 6.5). I did verify that our license is valid and I also verified the tsisvalid.log. I dont see any license related issue.

Can you please help?

Thanks,

Nagesh K

nipper

How.old is the log telling you the license is valid? Did you run tsisvalid yourself?

If networking changed then you may need a new license.

Quick_Learner

Hi Andy,

The log is very old. And I did not run the tsivalid before. However, I just ran the command and looks like the license is valid. Please see the attached screenshot.

I'm hoping that this is something related to network. Network team is looking into it, but we did not hear anything yet.

Thanks,

Nagesh K

Quick_Learner

Hi Ghoti, Andy,

I did the following to see if the server is actually able to contact AD and it works:

1. Right click on my computer, and click manage

2. Local users and groups

3. Open Administrators and tried adding a user. Now, in the search I selected "Entire Directory". And I was able to search for users in multiple domains

I'm guessing that Teamsite also uses the native OS process/services for contacting AD for retrieving domains. Doesn't it? Or is there another way how Teamsite works?

Can you please help? Entire Business is on hold for an entire day.

Thanks,

Nagesh K

Adam Stoller

Do you have any domain_list settings in your iw.cfg file? Specifically within the [iwserver] section (the one in the [iwcgi] section controls what shows up on the login page, the one in the [iwserver] section controls what the server attempts to connect to for authentication purposes.

If not, try adding one that explicitly lists one domain (not the local server though).

Also, you could try adding the following entries to the [iwserver] section of the iw.cfg file too (temporarily):

debug_adsi=true

show_user_list=true

Then, restart the iwserver process and check the results in the iwtrace.log

Ultimately though, you should look into upgrading to a supported version of the product so that you can utilize HP Support for issues like this (you can try contacting them about it, but it is likely they're knee-jerk response will be to tell you that you need to upgrade rather than trying to assist you with the problem; but you never know....)

Quick_Learner

Hi Ghoti,

I did try adding domain_list under [iwcgi]. However that did not help. I will add it under [iwserver] too and see. Also, I will add the adsi debug and show user list.

Thanks a lot for the input.

Thanks,

Nagesh K

Quick_Learner

Hi Ghoti,

I tried adding domain_list but they do not show up on the login page at all. Also, debug_adsi=true did not help any.

Any other troubleshoot we can do?

Thanks,

nagesh K

Adam Stoller

Please post your iw.cfg file - feel free to obfuscate any domain or hostname information that you believe should be kept "confidential"

Quick_Learner

Hi Ghoti,

I do see that the service is listing most of the AD groups the Teamsite users belong to. So, I think it is able to connect to the AD. However, the trace stops after reading few users suddenly. And I dont see any error/warn/info messages in the iwtrace.log.

Should I be looking for anything in particular?

Thanks,

Nagesh K

Adam Stoller

Try restarting the service a couple of times and see if the iwtrace.log stops consistently after only a few (and especially after a particular) users

Quick_Learner

Hi Ghoti, attaching the iw.cfg file.

Adam Stoller

Thanks, but I was only interested in that when you indicated that adding the entries to the iw.cfg didn't do anything - you since indicated that there *was* output in the iwtrace.log which means that you made the changes that were suggested (and I verified that just now).

Quick_Learner

Hi Ghoti,

I just observed that after reading all users and groups, it goes till creating new inode table and then log stops. But then next time service is run, it stopped after reading a user. It did not even go till inode. Any ideas?

Thanks,

Nagesh K

Adam Stoller

Nope - but it gives you a bit more information - try removing those entries from iw.cfg and see if it still stops at the same place.

Quick_Learner

Ghoti,

AM I right in saying that when we start the iwteamsite.exe (interwoven teamsite service) it will inturn start iwserver.exe?

Or is it that the interwoven teamsite service should be pointing to iwserver.exe instead of iwteamsite.exe?

Thanks,

NageshK

Adam Stoller

Not sure - I usually just do: net stop iwserver -and- net start iwserver

Quick_Learner

Hi Ghoti, is it possible to re-install iwserver alone without impacting other parts?

Adam Stoller

Not without doing a lot of questionable work - essentially figuring out how to open up the installer and extract out just that piece. I wouldn't recommend that. I *think* you could re-run the installer and it will attempt to "refresh" the installation, but I don't know what customizations it might overwrite in such a case. I don't think the problem is iwserver.exe itself - I think there has to be something in the environment (server or network) that changed to cause the problem - so that's where it is best for you to focus. (planning a new-server fresh-install upgrade is the best plan though, even though it might take a bit of time to do it, you'd be left with a supported system)