What is an acceptable secure time source in a datacentre environment?
GPS and NTP are two very typical time sources when accurate clocks are required. However, each has an important failing. NTP does not provide authentication of the time source, and so may be vulnerable to spoofing. GPS doesn't work very well inside the walls of a datacentre.
Accurate time is an operational concern for any number of reasons, as well as being a security concern due to things like session token lifetimes, key expiration, rate limiting, time-of-day restrictions, and behavioural pattern analysis in support of anti-fraud measures. Clock retrogression introduces instability in some network protocols which may create exploitable scenarios; IRC privilege escalation or impersonation ("split riding") is the typical example.
It is also extremely important that this time be consistent with time obtained from standard sources, to avoid drift during network disruption (or exploitation by time service disruption).
Accordingly, what technology or technique can be used to supply accurate and trusted time (within one second preferably) in sync with UTC, where GPS and other radio clocks are ineffective, while avoiding the need to manually true the time source frequently?
Solution 1:
NTP 4 provides authentication and some servers provide it.
Solution 2:
Make sure your systems are peering with each other. Authenticate the communication between them. This part is intended to ensure an attacker cannot cause your systems to drift relative to each other.
Put a cap on the change in drift. If all your systems were synchronizing with each other, but no external source, they would probably be drifting slowly, but the drift would not vary dramatically. The compensation that NTP would need to apply should be pretty much constant.
Alert if the drift gets close to the threshold.
Obviously thresholds will have to depend on the accuracy you need. Keeping your own machines within 1ms of each other and within 1s of the rest of the world is accurate enough for most needs.
Solution 3:
GPS doesn't work very well inside the walls of a datacentre.
You just need the GPS antenna to be near a window or outside. If this is a datacenter, and time accuracy is that critical for you, you should have the budget to for a contractor to poke a hole in the roof or side of the building to mount the antenna outside. You can run the antenna back inside via cable/conduit back inside where the GPS receiver / NTP server lives (in your rack or in a closet somewhere).
You can purchase a GPS NTP server device for a reasonable cost, or you could build your own out of a spare PC or 1U server, a USB or serial port GPS, and a Linux distro. You will also need an outdoor-rated GPS antenna.
We had a contractor mount an antenna on the grass outside on a simple post and run the wire in a buried conduit. It wasn't that expensive.
If you have multiple sites, setup a GPS NTP server at each location so they have an authoritative source, and also make sure the servers at each site are peered with each other over the secure VPN link between the two datacenters.
Bonus Paranoia: If you are concerned about a targeted attack with a rouge GPS signal to disrupt your time source, you could also add other hardware time sources to the mix (WWVB and CDMA).
Regardless if your specific implementation, you should also have max/min thresholds configured on all your input time sources so if something starts broadcasting a wildly inaccurate time stamp you know to ignore it. There was an incident in 2012 where a Navy NTP server rolls back to the year 2000 and caused havoc.