Geodesic convexity and the 2nd fundamental form

Let $(M,g)$ be a Riemannian manifold, $\Omega\subset M$ be a closed set with smooth boundary $\partial\Omega$ and $\nu$ be the unit normal of $\partial\Omega$ pointing into $\Omega$.

$\Omega$ is said to be geodesically convex iff $\forall x_0, x_1\in\Omega$ $\exists c:[0,1]\stackrel{\text{geodesic}}\to(M,g)$ s.t. $c(0)=x_0, c(1)=x_1$, $c([0,1])\subset\Omega$, $\mathrm{Length}[c]=d_g(x_0,x_1)$.

Suppose $\Omega$ is geodesically convex. Then...

[Q.1] Does it hold that the 2nd fundamental form of $\partial\Omega$ toward $\nu$ is nonnegative definite at each point on $\partial\Omega$?

[Q.2] Let $\psi_r(x):=\mathrm{exp}^g_x [r\nu(x)]\in N$ $(x\in\partial\Omega)$. Then for small $|r|$, $\psi_r$ is an embedding. Here, does it hold that the inner 2nd fundamental form of $\psi_r$ is nonnegative definite at each point on $\partial\Omega$ when $r>0$ and is sufficinetly small?

Thank you.

Solution 1:

I will address your first question. First define, for a fixed point $p\in\partial \Omega$, three conditions on $\partial \Omega$.

a) There is an open subset $U\subset M$ with $p\in U$, such that any two points in $U\cap \Omega$ can be joined by length minimising geodesic $c:[0,1]\rightarrow M$ with $c[0,1]\subset U\cap \Omega$.

b) Any geodesic $c:(-\epsilon,0]\rightarrow M$ with $c(-\epsilon,0)\subset \mathrm{int(\Omega)}$ and $c(0)= p$ hits the boundary transversally, i.e. $g(\dot c(0),\nu(p))\neq 0$.

c) The second fundamental form $l_\nu(\cdot,\cdot)=g(\nabla_\cdot \cdot, \nu)$ is non-negative at $p$.

We will prove that a) $\Rightarrow$ b) $\Rightarrow$ c), which answers your first question.

Step 1 Take geodesic normal coordinates $x^1,\dots,x^n$ , centred at $p$, with $\partial_n\vert_p=\nu(p)$. Using the implicit function theorem, one can show that there is a smooth function $f:\mathbb{R}^{n-1}\rightarrow \mathbb{R}$ such that $$ \Omega \cap V =\{x^n\ge f(x^1,\dots,x^{n-1})\}, $$ where $V$ denotes the coordinate patch. Next define smooth vector fields $X_1,\dots,X_{n-1}$ on $V$ by $$ X_j(q)=\partial_j\vert_q + \partial_jf(x^1(q),\dots,x^{n-1}(q))\partial_n\vert_q. $$ If $q\in V \cap \partial \Omega$, then $X_j(q) \in T_q\partial \Omega$ and hence for $1\le j\le n-1$ we have $$ \partial_jf(0) = X_j^k(p) g_{kl}(p) \nu^l(p)=g(X_j(p),\nu(p)) = 0 \tag{1}. $$

Step 2 We claim that $$ \text{b)} \quad \Leftrightarrow \quad f \text{ has a local minimum at 0.} \tag{2} $$ Note that in the coordinates fixed above, b) means that $\{x^n=0\}$ (the hypersurface spanned by geodesics through $p$ which are not transversal) does not intersect $int(\Omega)\cap V =\{x^n> f(x^1,\dots,x^{n-1})\}$ near $p$, or in other words that $f(x^1,\dots,x^{n-1})\ge 0$ in a neighbourhood of $p$. This proves (2).

Step 3 We're now in a position to prove a) $\Rightarrow$ b). Suppose b) is wrong, then $f$ does not have a local minimum at $0$. I.e. for any neighbourhood $U$ of $p$ there is a point $q\in \partial \Omega \cap U$ such that $$x^n(q)=f(x^1(q),\dots,x^{n-1}(q))<0. \tag{3}$$ Since we are in geodesic coordinates, the unique minimising geodesic joining $p$ and $q$ is given by $L =\{x^j=tx^j(q): 0\le t \le 1\}$. Assuming a), we must have $L\subset \Omega \cap U$, which implies $$ tx^n(q) \ge f(tx^1(q),\dots,tx^{n-1}(q)). $$ Divide by $t$ and take the limit $t\rightarrow 0$, then the right hand side will converge to $0$, since $Df(0)=0$ (see $(1)$). We obtain $x^n(q)\ge0$, which is a contradiction to $(3)$.

Step 4 We want to relate the second fundamental form to the Hessian of $f$. To this end note that, for $1\le j \le n-1$ and $1\le k \le n$ we have $$ \nabla_kX_j (p)= (\Gamma_{kj}^l \partial_l + \partial_k \partial_j f \partial_n + \Gamma_{kn}^r \partial_r)(p) = \partial_j\partial_kf(p) \cdot \nu(p) $$ and since $f$ is independent of $x^n$, we further obtain $$ \nabla_{X_k}X_j(p) = \partial_j\partial_kf(p) \cdot \nu(p), $$ which implies $l_{\nu}(X_k,X_j)\vert _p = \partial_j\partial_kf(p)$. Since the $X_j$ form a basis of $T_p\partial \Omega$ we have $$ \text{c)}\quad \Leftrightarrow \quad f \text{ has non-negative Hessian at $0$}. \tag{4} $$ From $(2)$ and $(4)$ it is evident that b) implies c).

Remark: With the same kind of argument one obtains an interesting result in the other direction. Assuming that the second fundamental form of $\partial \Omega$ is strictly positive, all geodesics coming from $\Omega$ hit $\partial \Omega$ transversally.