Using NTP to sync a group of linux servers to a common time source
I have 20 or so linux servers and I want to sync all of their clocks to a single NTP server, which is in the same rack and switch as my servers. Nothing is virtualized.
Our admins are having trouble getting the clocks on the various machines synced closer than about 500 ms. I would have guessed, and this post implies that we should be able to get the linux boxes synced to within 2 ms of of the source and each other.
Are my expectations for NTP unreasonable? Any hints as to what the admins should be doing/checking?
Solution 1:
I own a hosting company and we do exactly this. Here is how we accomplish this.
To start with, you need a NTP master source. So one of your Linux servers will become the master. I would create a DNS A record called time.example.com (assuming example.com is the domain). This way, if your master moves you need not update the other 19 servers.
On the master server you need to have an appropriately configured ntp.conf file.
Here is what one of our master /etc/ntp.conf files looks like. Note, this is a data center with a private address space (RFC1918) using 172.17.x.x so you'll need to adjust accordingly. If you want more than one master, create more than one DNS A record each with different IP to get a bit of fault tolerance if so desired.
server 127.127.1.0 # local clock
fudge 127.127.1.0 stratum 10
server 0.north-america.pool.ntp.org
server 1.north-america.pool.ntp.org
server 2.north-america.pool.ntp.org
server 3.north-america.pool.ntp.org
# Logging & Stats
statistics loopstats
statsdir /var/log/ntp/
filegen peerstats file peers type day link enable
filegen loopstats file loops type day link enable
# Drift file. Put this in a directory which the daemon can write to.
# No symbolic links allowed, either, since the daemon updates the file
# by creating a temporary in the same directory and then rename()'ing
# it to the file.
#
driftfile /etc/ntp/drift
broadcastdelay 0.008
restrict default noquery nomodify
restrict 0.north-america.pool.ntp.org mask 255.255.255.255 nomodify notrap noquery
restrict 1.north-america.pool.ntp.org mask 255.255.255.255 nomodify notrap noquery
restrict 2.north-america.pool.ntp.org mask 255.255.255.255 nomodify notrap noquery
restrict 3.north-america.pool.ntp.org mask 255.255.255.255 nomodify notrap noquery
# Allow LAN to query us
restrict 172.17.0.0 mask 255.255.0.0 nomodify notrap
# Trust ourselves. :-)
restrict 127.0.0.1
Now on each client, we have an /etc/ntp.conf file that looks like this:
server 127.127.1.0 # local clock
fudge 127.127.1.0 stratum 10
server time.example.com
# Drift file. Put this in a directory which the daemon can write to.
# No symbolic links allowed, either, since the daemon updates the file
# by creating a temporary in the same directory and then rename()'ing
# it to the file.
driftfile /etc/ntp/drift
multicastclient # listen on default 224.0.1.1
broadcastdelay 0.008
# Don't serve time or stats to anyone else by default (more secure)
restrict default noquery nomodify
restrict time.example.com mask 255.255.255.255 nomodify notrap noquery
# Allow LAN to query us
restrict 172.17.0.0 mask 255.255.0.0 nomodify notrap
# Trust ourselves. :-)
restrict 127.0.0.1
Use the ntpq command to see the servers with which you are synchronized. It provided you with a list of configured time servers and the delay, offset and jitter that your server is experiencing with them. For correct synchronization, the delay and offset values should be non-zero and the jitter value should be under 100.
Also on our client nodes, we have a rc script (/etc/rc.d/rc.local) that synchronizes the clock before starting the NTPD daemon. Here are the important parts... They are order dependent.
Synchronize the client's clock with the master time source /usr/sbin/ntpdate -b time.example.com
Start the NTPD daemon allowing for large time adjustments during start-up. /usr/sbin/ntpd -g -x
Finally, depending on your set up, you'll need to punch a firewall rule to allow your time.example.com master to reach the Public Internet over UDP port. Here is a typical and appropriately placed IPTables rule
iptables -t nat -A POSTROUTING -o $PUB_IF -p udp --dport 123 -j MASQUERADE
Where PUB_IF is the public interface (eth0, eth1, whatever)
Solution 2:
Correctly configured NTP will achieve synchronisation within a few ms. I always ensure each NTP client communicates with at least three NTP servers.
Use ntpq -p to monitor status - it should give some indication why you are not getting better sync.