What is the difference between $\arg\max$ and $\max$?

Yes, you are right. $\max f(x)$ is the maximum value (if it exists) of $f(x)$ as $x$ varies through some domain, while $\arg\max f(x)$ is the value of $x$ at which this maximum is attained.

However, there could be more than one $x$ value which gives rise to the maximum $f(x)$, in which case $\arg\max f(x)$ would be this set of values of $x$ instead.

Suppose $f(x) = 100 - (x-6)^2$.

Then $\max\limits_x f(x) = 100$ and $\operatorname*{argmax}\limits_x f(x) = 6$.

$\arg \max_x f(x) = f^{-1} \{ \max_x f(x) \}$.

For example, $\max_x \cos x = 1$, $\arg \max_x \cos x = \{ n 2 \pi\}_{n \in \mathbb{Z}}$.

Yes. $\arg\max f(x)$ is the function argument $x$ at which the maximum of $f$ occurs, and $\max f(x)$ is the maximum value of $f$. Hence $\max f(x)=f\left(\arg\max f(x)\right)$.