Containers not starting

Andres Aguilar

Installed the xECM helm chart in EKS, all the pods are Running fine except for:
otac-0, otcs-admin-0 and otcs-frontend-0

kubectl describe pod otac-0:
Warning Unhealthy 50s (x1177 over 173m) kubelet Readiness probe failed: Error: Not ready, file /opt/opentext/archive_center/ac_config/acInstalled does not exist, container automation not done

kubectl describe pod otcs-admin-0:
Warning Unhealthy 20s (x1333 over 3h15m) kubelet Readiness probe failed: Error: file /opt/opentext/cs/support/csconfigured does not exist, container automation not done

kubectl describe pod otcs-frontend-0:
Warning Unhealthy 90s (x1327 over 3h16m) kubelet Readiness probe failed: Error: file /opt/opentext/cs/support/csconfigured does not exist, container automation not done

When checking the logs for each:
kubectl logs otac-0:
2024-05-10 17:20:20,598 INFO supervisord started with pid 1
2024-05-10 17:20:21,601 INFO spawned: 'as_Master' with pid 9
2024-05-10 17:20:21,605 INFO spawned: 'as_logs' with pid 10
2024-05-10 17:20:21,609 INFO spawned: 'master_logs' with pid 11
2024-05-10 17:20:21,615 INFO success: as_Master entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024-05-10 17:20:21,616 INFO success: as_logs entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
2024-05-10 17:20:21,616 INFO success: master_logs entered RUNNING state, process has stayed up for > than 0 seconds (startsecs)
######## Started master script ########
/opt/opentext/archive_center/ac_config/ volume is empty copiying data
Updating parameters in Setup files
Running database setup
Found Postgres DB, Configuring postgres SQL
Error: Problem occurred during the configuring of DataBase ... Terminating
######## Started master script ########
Updating parameters in Setup files
Running database setup
Found Postgres DB, Configuring postgres SQL
2024-05-10 17:30:33,034 INFO exited: as_Master (exit status 1; not expected)
Error: Problem occurred during the configuring of DataBase ... Terminating

kubectl logs otcs-admin-0:
INFO :: Pre-flight environment setup done
INFO :: Performing pre-flight check
WARNING :: License file not found at /opt/opentext/container_files/custom_config/. License must be manually applied.
INFO :: Trying to retrieve url: http://localhost:8080/cs/cs
INFO :: Success retrieving url: http://localhost:8080/cs/cs
INFO :: Trying to retrieve url: http://otds:80/otdsws/rest/systemconfig/hello
INFO :: Success retrieving url: http://otds:80/otdsws/rest/systemconfig/hello
INFO :: Trying to retrieve url: http://otac-0:8080/archive?admInfo&pVersion=0046&resultAs=html
WARNING :: Request http://otac-0:8080/archive?admInfo&pVersion=0046&resultAs=html failed with return code: -1
Return value: HTTPConnectionPool(host='otac-0', port=8080): Max retries exceeded with url: /archive?admInfo&pVersion=0046&resultAs=html (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7fa6b989f4e0>: Failed to establish a new connection: [Errno 111] Connection refused',))
Silently retrying the request forever...

kubectl logs otcs-frontend-0:
INFO :: Pre-flight environment setup done
INFO :: Performing pre-flight check
WARNING :: License file not found at /opt/opentext/container_files/custom_config/. License must be manually applied.
INFO :: Trying to retrieve url: http://localhost:8080/cs/cs
INFO :: Success retrieving url: http://localhost:8080/cs/cs
INFO :: Trying to retrieve url: http://otds:80/otdsws/rest/systemconfig/hello
INFO :: Success retrieving url: http://otds:80/otdsws/rest/systemconfig/hello
INFO :: Trying to retrieve url: http://otac-0:8080/archive?admInfo&pVersion=0046&resultAs=html
WARNING :: Request http://otac-0:8080/archive?admInfo&pVersion=0046&resultAs=html failed with return code: -1
Return value: HTTPConnectionPool(host='otac-0', port=8080): Max retries exceeded with url: /archive?admInfo&pVersion=0046&resultAs=html (Caused by NewConnectionError('<urllib3.connection.HTTPConnection object at 0x7f797077d0f0>: Failed to establish a new connection: [Errno 111] Connection refused',))
Silently retrying the request forever...

It is clear to me that the otcs-admin-0 and otcs-frontend-0 pods are waiting on the otac-0 pod to be runnnig, but I'm currently blocked and without any ideas on why otac-0 is not starting up. What else can I do to troubleshoot what happened with otac-0?

appuq

Post your helm commandline I can compare it to mine…

Updating parameters in Setup files
Running database setup
Found Postgres DB, Configuring postgres SQL
Error: Problem occurred during the configuring of DataBase ... Terminating