Ubuntu - why would /var/log/dmesg stop updating after boot? does not show panic/cpu_hung errors which the console shows

Solution 1:

Whether you fix kernel logging past boot time or not will not help with kernel panic messages. When your kernel panics, it stops scheduling, so your logging daemon won't ever get to write down the kernel messages. If you want to grab those, you can look into kdump to get complete kernel core dumps and/or the netconsole kernel module to send the kernel messages over UDP to a remote syslog server.

As to getting kernel messages into /var/log/dmesg past boot time but outside of serious kernel crashes, try to have something like this in /etc/syslog.conf (or /etc/rsyslog.conf if using rsyslog):

kern.* /var/log/dmesg

For rsyslog, the file must also contain:

$ModLoad imklog.so

Let me know if you're using syslog-ng, it'd be a bit trickier to cover.

Solution 2:

The reason is that /var/log/dmesg is a static file, created at the end of the boot process.

I'm more familiar with Debian, in which the /etc/init.d/bootmisc.sh file creates /var/log/dmesg in a call to savelog. Ubuntu does this through /etc/init/dmesg.conf.

For system logging during normal operations, you'll want to refer to /var/log/syslog, as Pierre Carrier notes.

Since a kernel panic frequently (though not always) breaks user processes (such as syslog) and disk I/O, you'll have to use a kernel logging feature (kdump, netconsole) to capture kernel panics. Another option is to log via serial console (possibly serial-over-LAN with appropriate IPMI/ILOM support), or with an attached gdb session.