Number Theory: confusion with notation

Usually the notation $\mathbb{Z}_p$ or $\mathbb{Z}/p\mathbb{Z}$ mean the integers modulo $p$, that is $\{0,\ldots,p-1\}$ where you add and multiply as usually and then take the reminder modulo $p$.

The notation of $\mathbb{Z}_p^\times$ is for those numbers which have a multiplicative inverse modulo $p$, namely all $n$ such that exists $m$ such that $n\cdot m$ is $1$ modulo $p$. These are the numbers coprime to $p$, the greatest common divisor of them and $p$ is $1$.

A subgroup of $\mathbb{Z}_p^\times$ is a subset of these numbers which is closed under multiplication (but not necessarily addition), and every number in this subset also has its multiplicative inverse in there.

For example,

$\mathbb{Z}_5 = \{0,1,2,3,4\}$
$\mathbb{Z}_5^\times = \{1,2,3,4\}$
$\{1,4\}$ is a subgroup of $\mathbb{Z}_5^\times$. Can you see why?

$\mathbb{Z}_p$ is, in my opinion, problematic notion. In slightly less elementary number theory it refers to the p-adic numbers, which are very different from the integers $\bmod p$. An unambiguous, if slightly more cumbersome, notation would be $\mathbb{Z}/(p)$ or $\mathbb{Z}/p\mathbb{Z}$.