Are there spaces that 'look the same' at every point, but are not homogeneous?

A metric space is homogeneous if for any two points there is a global isometry that maps one into the other. It is locally homogeneous if any two points have isometric neighborhoods, i.e. the space 'looks the same' near them. Take open flat disk, it is clearly locally homogeneous, but there is no global isometry that maps its center to any other point. However, the disk is incomplete near the boundary, and if we complete it boundary points will no longer 'look the same' as interior ones.

Can complete connected Riemannian manifold be locally homogeneous but not homogeneous? How about closed one? I suspect yes, but I can not think of any examples.

In cosmology locally homogeneous is usually just called homogeneous, but I wonder if this is in line with mathematical usage even for 'nice' spaces.

Any two Riemannian manifolds with constant sectional curvature $C$ are locally homogeneous (in normal coordinates, one has an explicit description of the metric and by composing two normal coordinate systems around two different points one obtains a local isometry). However, such spaces need not be homogeneous.

For example, consider a closed oriented surface $S$ of genus $g \geq 2$ with the Riemannian metric of constant curvature $-1$. You can choose a compatible almost complex structure $J$ which will be a honest complex structure because we are in the two dimensional case. The orientation preserving isometries are in particular conformal maps and thus are biholomorphisms of $S$ but a result of Hurwitz shows that that biholomrophism group of such a surface is finite and in particular, $S$ cannot be homogeneous.