nav[aria-label="Primary Navigation"] { padding: 0; & ul { list-style: none; width: 100%; display: flex; flex-direction: row; justify-content: start; align-items: start; gap: 30px; padding: 0; & li { margin: 0; } & ul li { list-style: none; } } }

Daemon unable to Interwoven Serlvet Engine

System

I could really use help here... this is a production server, and for what ever reason this started happening this afternoon around 16:40 EDT. there have been no patches, and to my knowledge, no "sysadmin" activity.

Upon accessing the url, I get this:

The Interwoven Web Daemon was unable to contact the Interwoven Servlet Engine. The Interwoven Servlet Engine may be down, or running on an unexpected host or port.

iwserver.log looks fine, however in the error.log I have:
# tail -f error.log
[Thu Nov 01 21:31:17 2007] [error] (128)Network is unreachable: proxy: HTTP: attempt to connect to [::1]:8080 (localhost) failed

Yes, the TS.lic is valid.

Here are the last 40 lines from the iwserver.log
# tail -40 iwserver.log
iwmount: Unmounting /iwmnt/iwadmin...
iwmount: Unmounting /.iwmnt/iwadmin...
iwetcboot: Running iwstop ...
iwstop: aborting the current queue... done
iwstop: waiting for the server to shutdown... done
iwauthdstop: Stopped iwauthdstart script ... DONE
iwauthdstop: Stopped iwauthend ... DONE
iwutildstop: Stopped iwutildstart script ... DONE
iwutildstop: Stopped iwutild ... DONE
iw.server: ---------- STOPPED: Thu Nov 1 21:54:43 CDT 2007
iw.server: ---------- STARTING: Thu Nov 1 21:54:54 CDT 2007
iwetcboot: Running iwstart /local/iw-store in /apps/iw-home
iwstart: The events log file is /var/iwov/iwevents.log
iwstart: WFS device driver path is /dev/iwovwfs
iwstart: resetting wfs queue state ... done
iwstart: Resetting aborted state of kernel queue
iwstart: checking for standard services
iwstart: nfsd running
iwstart: mountd running
iwstart: Starting iwutildstart script (User: root)...
iwstart: Starting iwauthdstart script...
iwstart: Spawning iwserver.sol (trace file: /var/iwov/iwtrace.log)...
iwstart: Waiting up to 60 seconds for iwserver to come up... READY
iwmount: Mounting /iwmnt/default...
iwmount: Mounted /iwmnt/default.
iwmount: Mounting /.iwmnt/default...
iwmount: Mounted /.iwmnt/default.
iwmount: Mounting /iwmnt/iwadmin...
iwmount: Mounted /iwmnt/iwadmin.
iwmount: Mounting /.iwmnt/iwadmin...
iwmount: Mounted /.iwmnt/iwadmin.
Interwoven Event Subsystem starting...
Interwoven Web Daemon starting ...
Interwoven Proxy Server starting ...
Interwoven Servlet Engine starting ...
Interwoven CSSDK SOAP Server starting ...
oxy Server STARTED: Thu Nov 1 21:55:09 CDT 2007
iwuiboot: ---------- Servlet Engine STARTED: Thu Nov 1 21:55:12 CDT 2007
iwuiboot: ----- Interwoven CSSDK SOAP Server STARTED:Thu Nov 1 21:55:12 CDT 2007
iw.server: ---------- STARTED: Thu Nov 1 21:55:12 CDT 2007

Here are the last 40 lines from iwtrace.log
# tail -40 iwtrace.log
[Thu Nov 1 21:54:55 2007] Reading userops file /apps/iw-home/private/etc/userops.xml ...
[Thu Nov 1 21:54:55 2007] Done reading userops file /apps/iw-home/private/etc/userops.xml
[Thu Nov 1 21:54:55 2007] Reading userops file /apps/iw-home/private/etc/userops_ui.xml ...
[Thu Nov 1 21:54:55 2007] Done reading userops file /apps/iw-home/private/etc/userops_ui.xml
[Thu Nov 1 21:54:55 2007] Reading users and groups...
[Thu Nov 1 21:54:55 2007] Error reading user_databases.xml. The content of the file is ignored.
[Thu Nov 1 21:54:55 2007] Loading disk cache from Thu Nov 1 21:45:11 2007, revision = 1.
[Thu Nov 1 21:54:55 2007] done. 6 users and 86 groups read.
[Thu Nov 1 21:54:55 2007] [iwserver] branch_default_perm=775
[Thu Nov 1 21:54:55 2007] [iwserver] directory_default_perm=775
[Thu Nov 1 21:54:55 2007] [iwserver] file_default_perm=664
[Thu Nov 1 21:54:55 2007] [iwserver] workarea_default_perm=775
[Thu Nov 1 21:54:55 2007] New webserver_uid web (1000645)
[Thu Nov 1 21:54:55 2007] Log Permissions Disabled
[Thu Nov 1 21:54:55 2007] Log Authentication Disabled
[Thu Nov 1 21:54:55 2007] User database enabled
[Thu Nov 1 21:54:55 2007] Login Daemon Enabled
[Thu Nov 1 21:54:55 2007] Ldap user data cache thread delay = 1440
[Thu Nov 1 21:54:55 2007] Cache size: 100000
[Thu Nov 1 21:54:55 2007] setting rollover threshold to 10485760
[Thu Nov 1 21:54:55 2007] setting num rollovers to 10
[Thu Nov 1 21:54:55 2007] TEnvironment: fd limit 65536 max 65536
[Thu Nov 1 21:54:55 2007] Initialized store 0x1 at "/local/iw-store/workflow" (version=300008, next id=0x200).
[Thu Nov 1 21:54:55 2007] TEnvironment: fd limit 65536 max 65536
[Thu Nov 1 21:54:55 2007] Initialized store 0x64 at "/local/iw-store/default" (version=300008, next id=0x600).
[Thu Nov 1 21:54:56 2007] TFSEnvironment: inode cache 100000
[Thu Nov 1 21:54:56 2007] TEnvironment: fd limit 65536 max 65536
[Thu Nov 1 21:54:56 2007] Initialized store 0x65 at "/local/iw-store/iwadmin" (version=300008, next id=0x400).
[Thu Nov 1 21:54:56 2007] TFSEnvironment: inode cache 100000
[Thu Nov 1 21:54:56 2007] Batch Jobs Thread started
[Thu Nov 1 21:54:56 2007] Batch Jobs Thread started
[Thu Nov 1 21:54:56 2007] Started 4 FS threads
[Thu Nov 1 21:54:56 2007] Started Hopi server with 256 threads
[Thu Nov 1 21:54:56 2007] Hopi http listener on port 8898
[Thu Nov 1 21:54:56 2007] Hopi https listener on port 8899
[Thu Nov 1 21:54:56 2007] Started License Monitor
[Thu Nov 1 21:54:56 2007] License Info --> Number of Stores: 1
[Thu Nov 1 21:54:56 2007] License Info --> Number of CPUs: 4
[Thu Nov 1 21:54:56 2007] License Info --> Server Type: Corporate
[Thu Nov 1 21:54:56 2007] License Info --> Expiration Date: 20080130

So, after the restart (iw.server start) I tail the error.log

everything appears normal...

this is what I see

[Thu Nov 01 21:55:09 2007] [warn] RSA server certificate CommonName (CN) `rnss0014.cards.citicorp.com' does NOT match server name!?
[Thu Nov 01 21:55:09 2007] [notice] Apache/2.0.59 (Unix) mod_ssl/2.0.59 OpenSSL/0.9.8e configured -- resuming normal operations

all fine... normal 'errors'... won't stop the app from loading the ui...

so then I load up a browser, and hit the cms site...

immediately this next line posts to the log:
[Thu Nov 01 21:58:53 2007] [error] (128)Network is unreachable: proxy: HTTP: attempt to connect to [::1]:8080 (localhost) failed

I don't get it.

I know TS pretty darn good too, this one's got me frustrated. Any help I would appreciate it.

One last thing, the servletd_out.log shows a nnnnasty error:

# tail -f servletd_out.log
at com.sun.net.ssl.internal.ssl.AppOutputStream.write(AppOutputStream.java:59)
at java.nio.channels.Channels$WritableByteChannelImpl.write(Channels.java:267)
at com.interwoven.hopi.HopiChannel.write(HopiChannel.java:333)
at com.interwoven.hopi.HopiContext.sendData_(HopiContext.java:792)
... 12 more
Caused by: java.io.EOFException: SSL peer shut down incorrectly
at com.sun.net.ssl.internal.ssl.InputRecord.read(InputRecord.java:333)
at com.sun.net.ssl.internal.ssl.SSLSocketImpl.readRecord(SSLSocketImpl.java:723)
... 18 more
2007-11-01 21:56:15,139 [JMSBridge] ERROR com.interwoven.serverutils100.IWConfigFileImpl () - SEVERE FAILURE: Server stopping because connection to utild cannot be opened.

Find more posts tagged with

Comments

rajathi

Hi,

Check whethere there was a change in the hostname or ipaddress. Also check for any other service running on the default teamsite ports

nbwoven

I'm sure you must've tried this already but have you ruled out any port/nw issues: #lsof -i :8080 & #netstat -lvt|grep 8080

bill_brama

Hi, I have the same problem.

Does TS record IP address of the host? if yes, where can I find it. My DEV TS machine has dynamic IP address

'rmonoson' - did you manage to fix it? If yes, can you tell me how?

ISCBorisB

Hi, I have the same problem.

Does TS record IP address of the host? if yes, where can I find it. My DEV TS machine has dynamic IP address

'rmonoson' - did you manage to fix it? If yes, can you tell me how?

What is your platform and TS Version? For 6.x see KB 48503 and KB 54841

Btw, Development or not you'd make your life much easier if you allocate fixed IP to your TS Server. Tell your Administrators that
the System requires static (rather than leased) IP with DHCP reservation.

bill_brama

The problem was in DNS Smiley Happy

Our administrator did removed the DNS entry for our dev sever - for some testing purposes, but he did forgot to put it back in Smiley Very Happy

. It's all working fine now.

And 'ISCBorisB' thanks for response - we have static IP addresses on our DEV servers.