Are closed geodesics the prime numbers of Riemannian manifolds?

I wonder to what extent one can support the analogy that primitive closed geodesics are the prime numbers of Riemannian manifolds? ("Primitive": traced once, as opposed to $m$-fold for $m \ge 2$.) In fact, primitive closed geodesics are also known as "prime geodesics," and there is a "prime geodesics theorem" whose count of "primitive conjugacy classes" is analogous to the prime number theorem.

But I am especially interested in whether the analogy can be supported by characteristics of geodesics and of prime numbers that could be understood by those who are neither experts in number theory nor in Riemannian geometry. I once rashly claimed to students in a class that "closed geodesics are the prime numbers of Riemannian manifolds," but in fact I couldn't flesh out the analogy in much detail.

There is work in this direction by Alexander Reznikov and in a different vein by Christopher Deninger, but perhaps the most accessible analogy is in terms of the Selberg trace formula. The role closed geodesics play in determining the spectrum of the Laplacian is parallel to the role that prime numbers play in determining the Riemann zeta function.

This is just a long comment (as I do not know much about number theory). First, I think you need some curvature assumptions for this analogy to work, since it breaks down in the case of the round spheres and product tori (you do not want continuum of "prime numbers", I suppose!). You probably should assume that the sectional curvature of the manifold is negative (maybe constant) and the manifold is complete and either compact or has finite volume. You should also stick to closed geodesics which are not multiples of other closed geodesics (since they would be analogous to composite numbers of the form $p^k$). On the other hand, I have no idea what an analogue of the product of two distinct prime numbers would be under even most favorable circumstances. One suggestion for this would be to restrict to compact hyperbolic surfaces and declare that "prime" geodesics are the same thing as simple geodesics. Given this, the first analogy (that anybody can understand) would be that there are infinitely prime numbers and for every compact hyperbolic surface there are (countably) infinitely many simple closed geodesics.