Does the group of Diffeomorphisms act transitively on the space of Riemannian metrics?

Let $M$ be a smooth manifold (maybe compact, if that helps). Denote by $\operatorname{Diff}(M)$ the group of diffeomorphisms $M\to M$ and by $R(M)$ the space of Riemannian metrics on $M$. We obtain a canonical group action $$ R(M) \times \operatorname{Diff}(M) \to R(M), (g,F) \mapsto F^*g, $$ where $F^*g$ denotes the pullback of $g$ along $F$. Is this action transitive? In other words, is it possible for any two Riemannian metrics $g,h$ on $M$ to find a diffeomorphism $F$ such that $F^*g=h$? Do you know any references for this type of questions?

This map will not be transitive in general. For example, if $g$ is a metric and $\phi \in Diff(M)$ then the curvature of $\phi^* g$ is going to be the pullback of the curvature of $g$. So there's no way for a metric with zero curvature to be diffeomorphic to a manifold with non-zero curvature. Or for example, if $g$ is einstein $(g = \lambda Ric)$ then so is $\phi^* g$. So there are many diffeomorphism invariants of a metric.

Indeed, this should make sense because you can think of a diffeomorphism as passive, i.e. as just a change of coordinates. Then all of the natural things about the Riemannian geometry of a manifold should be coordinate ($\Leftrightarrow$ diffeomorphism) invariant.