A quote from Arnold

He means elliptic functions.

If, instead of using the harmonic oscillator equation for a pendulum, you use the expression involving the actual force, you get an elliptic integral for the time as a function of the angle (see for instance John Baez's example sheet http://math.ucr.edu/home/baez/classical/pendulum.pdf).

Taking the inverse to get the angle as a function of the time you get an elliptic function, by Jacobi's definition as inverse functions of elliptic integrals. Jacobi made that definition in an analogy to how you get sin cos and tan as inverse functions of certain integrals.

As for decompositions into squares, I recall that in Hardy and Wright's Introduction to the Theory of Numbers there are three proofs of the four square theorem. One is "elementary", one uses the integral quaternions/Hurwitz integers, and the third uses elliptic functions. This last one is (a newer version of?) Jacobi's proof.

OP's guess of "theta functions" isn't really that far off; it is well known (to those who know them) that the Jacobian elliptic function $\mathrm{sn}(u\mid m)$, which turns up in the solution of the DE for the pendulum, is expressible as a ratio of theta functions. (It should probably be emphasized that Jacobi studied both the theta functions and the elliptic functions now named after him, so he was certainly well-aware of this connection.)

As for the sum of four squares: as noted in this review article, Jacobi showed that the theta function $\vartheta_3(0,q)^d$ is the generating function for the number of ways to represent $k$ as $d$ squares $r_d(k)$; that is,

$$\vartheta_3(0,q)^d=\sum_{k=0}^\infty r_d(k) q^k$$

There you have it: two seemingly unrelated applications where the theta functions crop up.