Solution 1:

It is easy to see that inclusion $U\cap F \supset \exp_p(U^*\cap V)$: geodesics are uniquely specified by an initial point and an initial velocity. So if an isometry $\mathfrak{f}:M\to M$ fixes $\mathfrak{f}(p) = p$ and $d\mathfrak{f}(p)(v) = v$, the entire geodesic $\mathfrak{f}(\gamma_{p,v}) = \gamma_{p,v}$. (Because isometries send geodesics to geodesics, and the two geodesics have the same initial point and initial velocity.)

On the other hand, suppose there exists a point $q$ in $U\cap F$ such that unique minimizing geodesic $\gamma$ joining $p$ to $q$ is not in $F$ ($\gamma$ exists since $U$ is a normal neighbourhood). Then by definition $\mathfrak{f}(\gamma) \neq \gamma$ for some isometry, yet this isometry fixes the end-points $p,q$. This contradicts the assumption that we are in a normal neighbourhood (uniqueness of minimizing geodesics). Hence $U\cap F \subset \exp_p(U^*\cap V)$.