What is NTP dispersion and how do I control it?

I see some confusion going on in the answers here. For starters, ntpclient, at least in -s mode, isn't acting as a full NTP client, it's only sending and receiving one packet, so there's no "last 8 packets received". It isn't actually estimating its own dispersion at all.

Instead, the value it's printing is the value called "root dispersion" (rootdisp) in the packet returned by the server, which is an estimate of the total amount of error/variance between that server and the correct time. The way this is calculated is pretty simple: every NTP server either gets its time from an external clock (for example a radio or GPS receiver), or from another NTP server. If a server gets its time from an external clock, its root dispersion is the estimated maximum error of that clock. If it gets its time from another NTP server, its root dispersion is that server's root dispersion plus the dispersion added by the network link between them.

One point of confusion here is that while ntpq and chrony display dispersion and root dispersion in seconds, which is what people are used to looking to, ntpclient displays it in microseconds. Regardless, a value of 1217163 is still quite high. A good NTP server knows the time within a few milliseconds; a bad one within a few tens or hundreds of milliseconds. Yours is telling you that its time can only be trusted to within +/- 1.2 seconds.

You can actually get ntpclient to synchronize to this server anyway by passing the -x 0 or -t option (depending on version of ntpclient), which disables NTP sanity checks. If you only need roughly accurate time (to within a few seconds), that may be good enough. However, ntpclient is being pretty reasonable in refusing to synchronize to such a bad server. Your ntpq output on the ubuntu machine is showing a jitter of hundreds of milliseconds for all of its servers, even though they have low delay, which indicates either a very unreliable network, a conspiracy of all of the servers to provide erratic time, or a basic timekeeping problem on the server itself.

It also concerns me that the server 10.31.10.22 is advertising a refid of LOCL (undisciplined local clock) but has a stratum of 1. Usually the local clock is fudged to a stratum of 10 so that it's only used as a last-resort synchronization source to keep a herd from drifting apart. Either 10.31.10.22 is misconfigured and providing bad time to the rest of the network, or it's being disciplined to good time by some program outside of NTP's control, in which case the misconfiguration is simply that it's advertising the LOCL refid; it should be overridden to e.g. GPS or whatever is providing its time.

Just a partial answer for "What is dispersion?":

A typical NTP round trip:

client |        | server
    t1 |------->| t2
    t3 |<-------| t4

This yields two values, offset (the time difference between client and server), and the delay (essential the network travel time) with the following formulas:

offset= ((t4 - t3) + (t1 - t2)) / 2
delay = (t4 - t1) - (t3 - t2)

The client selects the current offset from the last 8 packets received, choosing the one with the smallest delay.

The same 8 packets are used to calculate the dispersion by doing a weighted average of the difference of these 8 offsets to the one selected in the last step, where the delay is used as the weighting factor, giving greater weight to smaller delays. It is a measure for the "spread" of the values and used to calculate the quality of a time server, especially if you have multiple to choose from.

Your dispersion and skew are enormous, there is a very large offset from the local clock to that peer. You should compare the offsets with the local date and set the clock manually.

Get ntpd running and show ntpq -p from a host using all of the peers. It will select the better ones.