The very short version

For any isometry $\sigma$, the fixed point set $\text{Fix}(\sigma)$ is a union of submanifolds. When is the dimension well-defined?

The long version

In the following, let $M$ always be a Riemannian manifold. The quotes were slightly changed for consistency.

What I found in the literature

In Klingenberg [1] we can read

1.10.15 Theorem. Let $\sigma\colon M \to M$ be an isometry. Then every connected component of the fixed point set $\text{Fix}(\sigma) := \{p \in M : \sigma(p) = p\}$ is a totally geodesic submanifold.

and Kobayashi [2] writes

Theorem 5.1. Let $\mathfrak S$ be any set of isometries of $M$. Let $\text{Fix}(\mathfrak S)$ be the set of points of $M$ which are left fixed by all elements of $\mathfrak S$. Then each connected component of $\text{Fix}(\mathfrak S)$ is a closed totally geodesic submanifold of $M$.

The latter is at least as strong as the first one, of course. The only concrete example given is in Klingenberg and appears to be just a reflection of the $n$-sphere, where we easily see the components to be of identical dimension. Neither explicitly mentions the possibility of the components having different dimensions.

However, Donnelly and Patodi [3][4] write (emphasis mine)

Let $M$ be a compact Riemannian manifold of dimension $d$ and $\sigma\colon M \to M$ an isometry […]. $\text{Fix}(\sigma)$ is the disjoint union of closed connected submanifolds $N$ of dimension $n$.

This gave me hope that in the compact case all the connected components had the same dimension, but later on [5] they write

Let $\sigma$ be an isometry of the compact Riemannian manifold $M$. The fixed point set of $\sigma$ is the disjoint union of compact connected totally geodesic submanifolds $N$.

and as far as I can tell, they only use $n$ as soon as an $N$ is fixed, so I'm assuming I overinterpreted an implicit dependency of $n$ on $N$.

What I would like to know

My question is easily phrased as three:

• What are examples where the components of such fixed point sets have different dimensions?
• What conditions can be required of $\mathfrak S \subseteq \text{Aut}(M)$ / $\sigma \in \text{Aut}(M)$ such that $\text{Fix}(\mathfrak S)$ / $\text{Fix}(\sigma)$ is of uniform dimension?
• What conditions can be placed on $M$ such that $\text{Fix}(\mathfrak S)$ / $\text{Fix}(\sigma)$ has a well-defined dimension for all such $\mathfrak S$ / $\sigma$?

In the end, I would like to have as much of a sharp separation between the cases as is possible. The questions above all tackle this same problem from different directions.

What is trivial

The following repeatedly led to confusion, so I'm saying these right now:

• Yes, if $M$ is not connected, counterexamples are trivial to come by. I'm only interested in the case where $M$ is connected.
• Yes, if $\text{Fix}(\mathfrak S)$ is connected, too, the dimension is well-defined by the standard argument.
• No, the fixed points are not always connected. Think about a reflection of the 1-sphere or a rotation of the 2-sphere.

What we found out

I'm probably forgetting something but here is what else we know:

• All the standard examples we know and most stuff we came up with was highly symmetrical. This might explain why we failed to find counterexamples.
• Isometries fix geodesics between fixed points as long as the geodesics are unique for their length. This shows for example that on the sphere the only way to obtain a disconnected fixed point set is for it to consist of two antipodal points only.

Links

[1]: Wilhelm Klingenberg, Riemannian Geometry. Page 95 at Google Books.

[2]: Shoshichi Kobayashi, Transformation Groups in Differential Geometry. Referenced at Fixed Points Set of an Isometry.

[3]: Harold Donnelly, Spectrum and fixed point sets of isometries. I. Directly in the beginning.

[4]: Harold Donnelly and V. K. Patodi, Spectrum and fixed point sets of isometries—II. Directly in the beginning.

[5]: Same as [4]. At the start of §2.

For an example of variable dimension of the fixed-point set, take $M=RP^2$ (with a constant curvature metric) and $\sigma$ an isometric reflection. (The projection of a reflection on $S^2$.) Then $Fix(\sigma)$ is a disjoint union of a point and a projective line. You can get more examples like this in higher dimensions. I am not sure about the rest of your questions.

Edit: If $\sigma$ is an isometry of a 2-dimensional orientable connected Riemannian manifold then $Fix(\sigma)$ has constant dimension. The reason is that near an isolated fixed point, $\sigma$ would preserve orientation and near a non-isolated fixed point $\sigma$ would have to reverse orientation. But a homeomorphism of a connected oriented manifold cannot both preserve and reverse orientation.

By working a bit more one can show the following: For every $n$ there exists an $n$-dimensional connected compact Riemannian manifold $M$ and an isometry $\sigma: M\to M$ whose fixed point set contains components of all possible dimensions: $0, 1, 2,..., n-1$.

Here is a construction. First of all, let $M_0,...,M_{n-1}$ be compact connected $n$-dimensional manifolds equipped with involutions $\sigma_0,...,\sigma_{n-1}$ such that $dim(Fix(\sigma_i))=i$. Furthermore, for each $i$ pick points $x_i, y_i\in M_i- Fix(\sigma_i)$ and set $x_i':= \sigma_i(x_i), y_i':= \sigma_i(y_i)$. I am assuming that these choices are generic so that for each $i$ all four points $x_i, x_i', y_i, y_i'$ are distinct. Now, form a connected manifold $M$ by performing a generalized connected sum along small disjoint balls $B_i, B_i', C_i, C_i'$ centered at the points $x_i, x_i', y_i, y_i'$, $i=0,...,n-1$, so that $\sigma_i(B_i)=C_i, \sigma(B_i')=C_i'$.

Namely, remove all these balls from the disjoint union $$ \coprod_i M_i $$ ("puncturing each $M_i$") and perform gluing via diffeomorphisms $$ f_i: \partial C_i\to \partial B_{i+1}, \partial f_i': C_i'\to \partial B_{i+1}', $$ $i=0, 1,....$. Choose these diffeomorphisms so that $$ \sigma_{i+1}\circ f_i= f_i'\circ \sigma_i, i=0, 1,.... $$ The result of this gluing is a smooth connected manifold $M$ equipped with a diffeomorphic involution $\sigma$ (whose restriction to each punctured $M_i$ equals $\sigma_i$). Lastly, put a $\sigma$-invariant Riemannian metric on $M$.