A high-powered explanation for $\exp U(n)=2\iff n\mid24$?

In What's so special about the divisors of $24$? it is noted that the exponent of the group of units modulo $n$, that is the highest order of an element of $U(n):=(\Bbb Z/n\Bbb Z)^\times$, is precisely $2$ if and only if the integer $n$ divides $24$. An elementary argument is given (see also answers here for a one-line proof), as well as some analytic machinery, but I recall (perhaps not quite accurately) that the number $24$ shows up a lot in high-powered math related to number theory, like lattices, moonshine, modular forms, string theory etc: suspicious.

If I were a believer in the magical and skeptical of coincidences, I might want to know if there is a high-powered explanation of this fact from the cited theoretical areas (not including asymptotic or statistical heuristics from analytic number theory). Or is it merely a collision of small numbers?

I believe I've tracked down what Qiaochu was referencing in Gannon's Moonshine beyond the Monster, pg168-169 §2.5 (alas, five pages too late). I can't claim to understand any of it.

$\qquad$