Why is $\text{maximize} \frac{1}{\lVert x \rVert}$ equivalent to $\text{minimize}\ \lVert x \rVert^2$?

Consider a similar situation:

Maximising $f(x)$ is equivalent to minimising $-f(x)$.

As Michael Hardy mentioned in the comment, it is easy to see that as $f(x)$ gets bigger, $-f(x)$ gets smaller. We can say the same thing about reciprocal:

Maximising $\displaystyle\frac 1{\Vert x \Vert}$ is equivalent to minimising $\Vert x \Vert$.

Moverover, $\Vert x\Vert^2$ is an increasing function of $\Vert x\Vert$; when $\Vert x\Vert$ increases, $\Vert x\Vert^2$ increases as well; therefore

Minimising $\Vert x \Vert$ is equivalent to minimising $\Vert x \Vert^2$.

Combining the results above, we arrive at the conclusion

Maximising $\displaystyle\frac 1{\Vert x \Vert}$ is equivalent to minimising $\Vert x \Vert^2$.

You are right about the reason we choose to minimise $\Vert x \Vert^2$: it behaves better around zero, namely, it is differentiable, and the first and second derivatives are important for optimisation problems.