How to appreciate Riemannian geometry

In the spirit of answering a small piece of your large question with a visual image (and noting the meta aspect of gradually covering the complicated manifold of Riemannian geometry with local patches of conceptual coordinates), here's the prototypical non-trivial example of parallel transport, illustrating holonomy (parallel transport around a closed loop is not the identity map on the tangent space) and curvature (the holonomy around a geodesic triangle in a surface is the integral of the curvature over the triangle's interior):

The first book you should look at is Vladimir Arnold's Mathematical Methods of Classical Mechanics where he has a nice introductory discussion of differential geometry and curvature.

The lemma you cited does not have far-reaching consequences and you shouldn't be focusing on it. One direction of research that is quite popular is the relation between curvature and topology. It became clear relatively recently (in the 1980s) that positive sectional curvature imposes extremely stringent conditions on the manifold; e.g., one gets a universal upper bound on the sum of all Betti numbers of the manifold by a result of Gromov. In negative curvature, on the contrary, there is a great wealth of examples, related also to the popular field of Cannon-Gromov-hyperbolic groups. In general, to get motivated I would suggest looking up work by Gromov. You may not follow all the details (if the details are there :-) but you are likely to be inspired.

Perhaps this will help, as it is quite intuitive: "Surface in 3D that realizes all pairs of principal curvatures": angel's curl surface: