Tomcat suddenly shuts down on its own

Solution 1:

As I remarked in the comments, my first idea would be that a shutdown command (SHUTDOWN) was sent on Tomcat's shutdown port (by default listening on port 8005 of the interface localhost). The usage of this method is recommended in Tomcat's documentation.

However Ubuntu (following Debian) disabled the shutdown port by default in version 8.5.32-2. Newer versions of Tomcat perform a clean shutdown, when the JVM receives a SIGTERM, so enabling the shutdown port is just a security risk.

So the main candidates for your unexpected shutdown are:

  • the Linux OutOfMemory killer, which logs to the kernel log (journalctl -k),
  • a crash of the JVM. While the virtual machine itself is pretty stable, I have seen libtcnative bring it down. Check Tomcat9's main log (/var/log/catalina.<date>.log), which under your Ubuntu is also nicely collected by systemd-journald (journalctl -u tomcat9) and sent to syslog.
  • Java's own OutOfMemoryError, which should also be in Tomcat9's main log.

Update: based on your syslog excerpts, someone got access to the user app. A lot of user sessions for user app where created which used up the system's resources and caused many services to be shut down.