kernel warning in logs: set_rtc_mmss can't update from 0 to 58
I've been seeing a bunch of these log messages
Jan 3 00: 58: 57 foo kernel: set_rtc_mmss: can't update from 0 to 58
They occur on CentOS 6.4 VMs running on VMware. I understand it's something to do with the hardware clock not being set properly on the guest OS. I found this command which sets the hardware clock to the current system time:
sudo hwclock --systohc
Is this the correct setting for a virtual machine? Also, where can this be set so it's persistent? In the kernel boot parameters? I'd like for newly provisioned VMs to not have this problem.
UPDATE 1
As requested:
me@foo:~> ntpq -p
remote refid st t when poll reach delay offset jitter
==============================================================================
LOCAL(0) .LOCL. 10 l 43 64 377 0.000 0.000 0.000
+dtc-nist01.ntp. .ACTS. 1 u 174 1024 377 3.311 -8.554 0.497
*nist1-nj.ustimi .ACTS. 1 u 205 1024 377 6.737 3.775 0.433
+nist1-pa.ustimi .ACTS. 1 u 55 1024 377 8.610 4.688 0.337
I see vmwaretools is out of date on this VM. Perhaps the puppet module I have for managing the vmwaretools install did not install it properly. I'll take a look and get back to you.
UPDATE 2
Yes, vmware tools is installed and at the latest version.
me@foo:~> ps aux | grep vmtools
root 56021 0.0 0.1 59508 4156 ? S Jan09 3:29 /usr/sbin/vmtoolsd
UPDATE 3
I tried enabling the "Synchronize guest time with host" on the VM:
me@foo:~> vmware-toolbox-cmd timesync status
Disabled
me@foo:~> vmware-toolbox-cmd timesync enable
me@foo:~> vmware-toolbox-cmd timesync status
Enabled
but I'm still getting those messages. In fact, date and hwclock --show are several minutes apart now, whereas they used to be pretty tight.
In the past, older SLES 9 VMs benefited from settings in the VMware article Timekeeping best practices for Linux guests, but it states that CentOS/RHEL 6 guests do not need any additional kernel parameters set.
UPDATE 4
Upgrading to CentOS 6.5 did not help. The kernel is:
Linux foo 2.6.32-431.1.2.0.1.el6.x86_64 #1 SMP Fri Dec 13 13:06:13 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
VMWare does have some suggestions in there knowledge base on how to configure the ntp client in your guest os. First thing is to make sure the vmware-toolbox timesync is disabled, as you only want ntp to update the time.
From Timekeeping best practices for Linux guests:
This is their sample /etc/ntp.conf:
tinker panic 0
restrict 127.0.0.1
restrict default kod nomodify notrap
server 0.vmware.pool.ntp.org
server 1.vmware.pool.ntp.org
server 2.vmware.pool.ntp.org
driftfile /var/lib/ntp/drift
The first line (tinker panic 0) allows for big time jumps. (e.g. the system state was saved/restored)
An alternative would be to disable ntp in your guest and enable the vmware-tools time syncing.
Are you running the latest kernel version for CentOS 6.4? I'd try booting with clocksource=acpi_pm. Also, I'd do ntpdate -u tick.usno.navy.mil && hwclock --systohc and let things sit. If things continue, then add divizor=10 as well.
Honestly, I suspect the problem is partly to do with the code in your particular kernel for setting and reading the real time clock - that's what gets you the error message. The other part is due to virtual time interrupts not being delivered regularily enough or the timer interrupt handler not scaling the kernel time correctly - and this is what gets you the clock drift.
Forcing ntp to be more aggressive is just a crutch for the problem, unfortunately.
Is there a newer kernel available to you? That might be your safest bet. Good Luck.
Based on everything else I've read, it sounds as if these warnings are benign, especially if the date on your servers is fine. A little annoying though.
I'll let this tidbit from ntp.org put me at ease:
Per http://www.ntp.org/ntpfaq/NTP-s-trbl-spec.htm#Q-LINUX-SET-RTC-MMSS:
8.3.4.1.1. What does set_rtc_mmss: can't update from 54 to 5 mean?
The function set_rtc_mmss() updates minutes and seconds of the CMOS clock from system time. It does not update the hour or date to avoid problems with timezones.[1] The message shown was added to make users and implementers aware of the problem that not all time updates will succeed.
Imagine the system time is 17:56:23 while the CMOS clock is already at 18:03:45. Updating just minutes and seconds would set the hardware clock to 18:56:23, a wrong value. The solution for this problem is either to wait a few minutes, or to install a kernel patch that fixes the problem. Normally a wrong time in the hardware clock will not show up until after reboot, or maybe after APM slowed down your system.