How to know whether Lagrange multipliers gives maximum or minimum?
My book tells me that of the solutions to the Lagrange system, the smallest is the minimum of the function given the constraint and the largest is the maximum given that one actually exists.
But what if we only have one point as a solution? How to know whether Lagrange multipliers gives maximum or minimum?
As Om(nom)$^3$ said in the comments, if you're working on a closed and bounded region then it's not possible to get only one critical point.
If you're not on a closed and bounded region then it's no longer guaranteed that you'll have more than one critical point. If you only have one critical point then you can use the Bordered Hessian technique. (Thanks to ziggurism for clearing that up.)
On a closed bounded region a continuous function achieves a maximum and minimum. If you use Lagrange multipliers on a sufficiently smooth function and find only one critical point, then your function is constant because the theory of Lagrange multipliers tells you that the largest value at a critical point is the max of your function, and the smallest value at a critical point is the min of your function. Thus max = min, i.e. the function is constant. Also note that "critical point" should probably be called something else, like "point of interest" because usually critical points are defined as points where the gradient is zero.
In fact the normal second derivative test doesn't apply to constrained extremum problems. You should instead use the Bordered Hessian method. In brief, instead of computing the positive-definiteness of the Hessian matrix of second partial derivatives of $f$, you instead compute the Hessian of $f-\lambda g$, including derivatives with respect to $\lambda$
In the Lagrange Multipliers Method the points obtained will be critical points (solutions of an equation which have the form $\nabla f(x)=\lambda\varphi(x)$) of an objective function $f$ (of class $C^1$) restrict to a region $M$ which have the form $M=\varphi^{-1}(c)$, where $\varphi$ is a function (of classe $C^1$) that comes from the constraint (which have the form $\varphi(x)=c$).
Usually, the existence of the maximum and the minimum comes from the continuity of $f$ and the compactness of $\overline{M}$. In this case, $f$ have at least two critical points on $\overline{M}$. However, there are cases in which the equation $\nabla f(x)=\lambda\varphi(x)$ give us only one solution $p\in M$ (this is because the other critical point is in $\overline{M}\setminus M$). Here is a possible approach that sometimes works for these cases:
Show that the maximum (or minimum) is not in $\overline{M}\setminus M$.
Conclude that $p$ is the maximum (or minimum) of $f$ on $M$.
You can see examples of this case here, here and here.