OpenText server and Tomcat are installed on Windows server.
I updated Tomcat from 9.0.0.M20 to 9.0.71 version.
OpenText content server 16.2.0
Made a backup of Tomcat Config and WebApps folder, and after Tomcat upgrade i restored those folders and started Tomcat service.
http://localhost:8080 available, but when accessing http://localhost:8080/otdsws/login getting "The server encountered an unexpected condition that prevented it from fulfilling the request. The root cause and detailed information is available in the server logs."
otds.logs
2023-02-17 19:39:03.550 [ main] INFO com.opentext.otds.server.OtdsService - Stopping Enterprise Sync...
2023-02-17 19:39:03.551 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: persistent search restarted
2023-02-17 19:39:03.552 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: restarting persistent search due to error: null
2023-02-17 19:39:03.554 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: persistent search restarted
2023-02-17 19:39:03.554 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: restarting persistent search due to error: null
2023-02-17 19:39:03.556 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: persistent search restarted
2023-02-17 19:39:03.556 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: restarting persistent search due to error: null
2023-02-17 19:39:03.561 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: persistent search restarted
2023-02-17 19:39:03.561 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: restarting persistent search due to error: null
2023-02-17 19:39:03.564 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: persistent search restarted
2023-02-17 19:39:03.564 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: restarting persistent search due to error: null
2023-02-17 19:39:03.565 [ main] INFO com.opentext.otds.server.OtdsService - Stopping Enterprise Sync Processor...
2023-02-17 19:39:03.566 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: persistent search restarted
2023-02-17 19:39:03.566 [ main] INFO com.opentext.otds.server.OtdsService - Stopping SPS...
2023-02-17 19:39:03.566 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: restarting persistent search due to error: null
2023-02-17 19:39:03.568 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: persistent search restarted
2023-02-17 19:39:03.568 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: restarting persistent search due to error: null
2023-02-17 19:39:03.570 [ main] INFO com.opentext.otds.server.OtdsService - Stopping AuthenticationServiceAPI...
2023-02-17 19:39:03.570 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: persistent search restarted
2023-02-17 19:39:03.570 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: restarting persistent search due to error: null
2023-02-17 19:39:03.572 [ main] INFO com.opentext.otds.server.OtdsService - Stopping Session Cache...
2023-02-17 19:39:03.573 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: persistent search restarted
2023-02-17 19:39:03.573 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: restarting persistent search due to error: null
2023-02-17 19:39:03.573 [ main] INFO com.opentext.otds.as.OtdsSessionCache - OTDS Session Cache Cleaner started
2023-02-17 19:39:03.574 [n Cache Cleaner] INFO com.opentext.otds.as.OtdsSessionCache - OTDS Session Cache Cleaner exited
2023-02-17 19:39:03.574 [ main] INFO com.opentext.otds.server.OtdsService - Stopping Replay Cache...
2023-02-17 19:39:03.575 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: persistent search restarted
2023-02-17 19:39:03.575 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: restarting persistent search due to error: null
2023-02-17 19:39:03.577 [y Cache Cleaner] INFO com.opentext.otds.as.ReplayCache - OTDS Replay Cache Cleaner exited
2023-02-17 19:39:03.578 [ main] INFO com.opentext.otds.server.OtdsService - Stopping Repo...
2023-02-17 19:39:03.580 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: persistent search restarted
2023-02-17 19:39:03.580 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: restarting persistent search due to error: null
2023-02-17 19:39:04.009 [ main] INFO com.opentext.otds.server.OtdsService - Stopping OpenDJ...
2023-02-17 19:39:08.818 [ main] INFO com.opentext.otds.ds.directory.opends.EmbeddedDirectoryImpl - OpenDJ stopped.
2023-02-17 19:39:08.818 [ main] INFO com.opentext.otds.server.OtdsService - Stopping Push Layer...
2023-02-17 19:39:08.818 [ main] INFO com.opentext.otds.server.OtdsService - Stopping Push Connector...
2023-02-17 19:39:08.818 [ main] INFO com.opentext.otds.server.OtdsService - Stopping Message Broker...
2023-02-17 19:39:08.833 [ main] INFO com.opentext.otds.server.OtdsService - Stopping JMS Server...
2023-02-17 19:39:08.849 [ main] INFO com.opentext.otds.server.OtdsServlet - ===============================================================================================
2023-02-17 19:39:08.849 [ main] INFO com.opentext.otds.server.OtdsServlet - OTDS STOPPED
2023-02-17 19:39:08.849 [ main] INFO com.opentext.otds.server.OtdsServlet - ===============================================================================================
2023-02-17 19:44:33.717 [ main] INFO com.opentext.otds.server.OtdsServlet - ===============================================================================================
2023-02-17 19:44:33.717 [ main] INFO com.opentext.otds.server.OtdsServlet - OTDS Version: 16.4.3 (16.4.3.2211 2018-12-20 02:41:35).
2023-02-17 19:44:33.717 [ main] INFO com.opentext.otds.server.OtdsServlet - Operating System: Windows Server 2012 R2
2023-02-17 19:44:33.718 [ main] INFO com.opentext.otds.server.OtdsServlet - Java VM: Java HotSpot(TM) 64-Bit Server VM
2023-02-17 19:44:33.718 [ main] INFO com.opentext.otds.server.OtdsServlet - Java Version: 1.8.0_341
2023-02-17 19:44:33.718 [ main] INFO com.opentext.otds.server.OtdsServlet - Java Runtime Version: 1.8.0_341-b10
2023-02-17 19:44:33.718 [ main] INFO com.opentext.otds.server.OtdsServlet - Application Server: Apache Tomcat (9.0.71).
2023-02-17 19:44:33.718 [ main] INFO com.opentext.otds.server.OtdsServlet - User Name: LOCAL SERVICE
2023-02-17 19:44:33.718 [ main] INFO com.opentext.otds.server.OtdsServlet - Max JVM Heap Size: 954728448 bytes
2023-02-17 19:44:33.718 [ main] INFO com.opentext.otds.server.OtdsServlet - ===============================================================================================
2023-02-17 19:44:33.740 [ main] INFO com.opentext.otds.server.OtdsService - Starting JMS Server...
2023-02-17 19:44:35.439 [ main] INFO com.opentext.otds.server.OtdsService - Starting MessageBroker...
2023-02-17 19:44:35.518 [ main] INFO com.opentext.otds.server.OtdsService - Starting Push Connector...
2023-02-17 19:44:35.518 [ main] INFO com.opentext.otds.server.OtdsService - Starting Push Layer...
2023-02-17 19:44:35.571 [ main] INFO com.opentext.otds.server.OtdsService - Starting DSConfigurationFramework...
2023-02-17 19:44:35.571 [ main] INFO com.opentext.otds.server.OtdsService - Starting OpenDJ...
2023-02-17 19:44:38.370 [ main] INFO com.opentext.otds.ds.directory.opends.EmbeddedDirectoryImpl - Creating audit backend...
2023-02-17 19:44:38.733 [ main] ERROR com.opentext.otds.ds.directory.opends.EmbeddedDirectoryImpl - Failed to create audit backend: 1
2023-02-17 19:44:38.733 [ main] INFO com.opentext.otds.ds.directory.opends.EmbeddedDirectoryImpl - OpenDJ Started.
2023-02-17 19:44:38.733 [ main] INFO com.opentext.otds.server.OtdsService - Verifying Indexes...
2023-02-17 19:44:38.733 [ main] INFO com.opentext.otds.server.OtdsService - Starting Repo...
2023-02-17 19:44:38.885 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: restarting persistent search due to error: null
2023-02-17 19:44:38.900 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: persistent search restarted
2023-02-17 19:44:38.900 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: restarting persistent search due to error: null
2023-02-17 19:44:38.916 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: persistent search restarted
2023-02-17 19:44:38.916 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: restarting persistent search due to error: null
2023-02-17 19:44:38.916 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: persistent search restarted
2023-02-17 19:44:38.916 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: restarting persistent search due to error: null
2023-02-17 19:44:38.916 [ main] ERROR com.opentext.otds.ds.repositoryhandler.common.RepoFactory - Failed to start RepoFactory: null
java.lang.NullPointerException
at com.opentext.otds.ds.repositoryhandler.impl.RecycleBinImpl.init(RecycleBinImpl.java:54)
at com.opentext.otds.ds.repositoryhandler.impl.RepositoryHandlerImpl.initRecycleBin(RepositoryHandlerImpl.java:103)
at com.opentext.otds.ds.repositoryhandler.common.RepoFactory.initRepositoryHandler(RepoFactory.java:109)
at com.opentext.otds.ds.repositoryhandler.common.RepoFactory.start(RepoFactory.java:84)
at com.opentext.otds.server.OtdsService.initialize(OtdsService.java:141)
at com.opentext.otds.server.OtdsServlet.init(OtdsServlet.java:85)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1161)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1114)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1007)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4948)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5256)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:726)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:698)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:696)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:689)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1888)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:582)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:472)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1617)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:318)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)
at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:423)
at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:366)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:943)
at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:835)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1393)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1383)
at java.util.concurrent.FutureTask.run(Unknown Source)
at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:916)
at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:265)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.StandardService.startInternal(StandardService.java:430)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:930)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.startup.Catalina.start(Catalina.java:772)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:347)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:478)
2023-02-17 19:44:38.916 [ main] ERROR com.opentext.otds.server.OtdsServlet - OTDS STARTUP FAILED
com.opentext.otds.common.exceptions.DSRuntimeException: Failed to start RepoFactory: null
at com.opentext.otds.ds.repositoryhandler.common.RepoFactory.start(RepoFactory.java:93)
at com.opentext.otds.server.OtdsService.initialize(OtdsService.java:141)
at com.opentext.otds.server.OtdsServlet.init(OtdsServlet.java:85)
at org.apache.catalina.core.StandardWrapper.initServlet(StandardWrapper.java:1161)
at org.apache.catalina.core.StandardWrapper.loadServlet(StandardWrapper.java:1114)
at org.apache.catalina.core.StandardWrapper.load(StandardWrapper.java:1007)
at org.apache.catalina.core.StandardContext.loadOnStartup(StandardContext.java:4948)
at org.apache.catalina.core.StandardContext.startInternal(StandardContext.java:5256)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase.addChildInternal(ContainerBase.java:726)
at org.apache.catalina.core.ContainerBase.addChild(ContainerBase.java:698)
at org.apache.catalina.core.StandardHost.addChild(StandardHost.java:696)
at org.apache.catalina.startup.HostConfig.deployDescriptor(HostConfig.java:689)
at org.apache.catalina.startup.HostConfig$DeployDescriptor.run(HostConfig.java:1888)
at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
at java.util.concurrent.FutureTask.run(Unknown Source)
at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
at org.apache.catalina.startup.HostConfig.deployDescriptors(HostConfig.java:582)
at org.apache.catalina.startup.HostConfig.deployApps(HostConfig.java:472)
at org.apache.catalina.startup.HostConfig.start(HostConfig.java:1617)
at org.apache.catalina.startup.HostConfig.lifecycleEvent(HostConfig.java:318)
at org.apache.catalina.util.LifecycleBase.fireLifecycleEvent(LifecycleBase.java:123)
at org.apache.catalina.util.LifecycleBase.setStateInternal(LifecycleBase.java:423)
at org.apache.catalina.util.LifecycleBase.setState(LifecycleBase.java:366)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:943)
at org.apache.catalina.core.StandardHost.startInternal(StandardHost.java:835)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1393)
at org.apache.catalina.core.ContainerBase$StartChild.call(ContainerBase.java:1383)
at java.util.concurrent.FutureTask.run(Unknown Source)
at org.apache.tomcat.util.threads.InlineExecutorService.execute(InlineExecutorService.java:75)
at java.util.concurrent.AbstractExecutorService.submit(Unknown Source)
at org.apache.catalina.core.ContainerBase.startInternal(ContainerBase.java:916)
at org.apache.catalina.core.StandardEngine.startInternal(StandardEngine.java:265)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.StandardService.startInternal(StandardService.java:430)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.core.StandardServer.startInternal(StandardServer.java:930)
at org.apache.catalina.util.LifecycleBase.start(LifecycleBase.java:183)
at org.apache.catalina.startup.Catalina.start(Catalina.java:772)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(Unknown Source)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(Unknown Source)
at java.lang.reflect.Method.invoke(Unknown Source)
at org.apache.catalina.startup.Bootstrap.start(Bootstrap.java:347)
at org.apache.catalina.startup.Bootstrap.main(Bootstrap.java:478)
Caused by: java.lang.NullPointerException
at com.opentext.otds.ds.repositoryhandler.impl.RecycleBinImpl.init(RecycleBinImpl.java:54)
at com.opentext.otds.ds.repositoryhandler.impl.RepositoryHandlerImpl.initRecycleBin(RepositoryHandlerImpl.java:103)
at com.opentext.otds.ds.repositoryhandler.common.RepoFactory.initRepositoryHandler(RepoFactory.java:109)
at com.opentext.otds.ds.repositoryhandler.common.RepoFactory.start(RepoFactory.java:84)
... 46 more
2023-02-17 19:44:38.916 [o localhost:389] WARN com.opentext.otds.ds.repositoryhandler.repoconnector.impl.jndiconnector.JNDIConnector - ChangeNotifierResultHandler: persistent search restarted.
Catalina logs shows this error when Tomcat service is stopping:
17-Feb-2023 19:44:25.968 WARNING [Thread-34] org.apache.catalina.loader.WebappClassLoaderBase.clearReferencesThreads The web application [otdsws] appears to have started a thread named [Monitor Provider State Updater] but has failed to stop it. This is very likely to create a memory leak. Stack trace of thread:
sun.misc.Unsafe.park(Native Method)
java.util.concurrent.locks.LockSupport.park(Unknown Source)
java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.await(Unknown Source)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(Unknown Source)
java.util.concurrent.ThreadPoolExecutor.getTask(Unknown Source)
java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
java.lang.Thread.run(Unknown Source)
17-Feb-2023 19:44:25.983 SEVERE [Thread-34] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [otdsws] created a ThreadLocal with key of type [com.sun.xml.ws.api.server.ThreadLocalContainerResolver$1] (value [com.sun.xml.ws.api.server.ThreadLocalContainerResolver$1@12e1d80]) and a value of type [com.sun.xml.ws.api.server.Container.NoneContainer] (value [com.sun.xml.ws.api.server.Container$NoneContainer@964d4e3]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
17-Feb-2023 19:44:25.983 SEVERE [Thread-34] org.apache.catalina.loader.WebappClassLoaderBase.checkThreadLocalMapForLeaks The web application [otdsws] created a ThreadLocal with key of type [com.opentext.ecm.components.runtimecontext.impl.RuntimeContext$1] (value [com.opentext.ecm.components.runtimecontext.impl.RuntimeContext$1@78d70a52]) and a value of type [com.opentext.ecm.components.runtimecontext.impl.RuntimeContextThread] (value [com.opentext.ecm.components.runtimecontext.impl.RuntimeContextThread@a7433b5]) but failed to remove it when the web application was stopped. Threads are going to be renewed over time to try and avoid a probable memory leak.
Anyone knows what might be cause for this ?
