Is the cyclic group $\langle x\rangle$ always a subgroup of $G$ for any $x\in G$?

Yes.

In fact, neither the order of the element $x$ nor of the group $G$ need be finite. I will concentrate on the case when ${\rm ord}\, x$ is finite.

Use the one-step subgroup test.

Since $e=x^0\in\langle x\rangle$, we have $\langle x\rangle\neq \varnothing$.

Let $y\in\langle x\rangle$. Then $y=x^n$ for some $n\in\Bbb Z$. But all powers of $x$ are in $G$ by closure of $G$. Hence $y\in G$. Hence $\langle x\rangle\subseteq G$.

Let $a=x^r, b=x^s\in \langle x\rangle$. Then

$$\begin{align} ab^{-1}&=x^r(x^s)^{-1}\\ &=x^rx^{-s}\\ &=x^{r-s}, \end{align}$$

which is in $\langle x\rangle$ by taking the power modulo ${\rm ord}\, x$: we can do this by Euclid's algorithm, by finding $0\le m<{\rm ord}\, x$ and $q\in\Bbb Z$ such that $r-s=q{\rm ord}\, x+m$, since then

$$\begin{align} x^{r-s}&=x^{q{\rm ord}\, x+m}\\ &=(x^{{\rm ord }\, x})^qx^m\\ &=e^qx^m\\ &=x^m. \end{align}$$

Hence $ab^{-1}\in\langle x\rangle$.

Hence $\langle x\rangle\le G$.

We have disscussed several solutions, and as suggested from @user1729 I put in a correct solution with the comments. I use the way I started in the question. So we check the following three points:

1. $1_G\in \langle x\rangle$. This is clear since $x^0=1$.

2. Stable under inverses: Let $u\in \langle x\rangle$. Then $u=x^i$ for some $1\leq i \leq n$. I claim that $u^{-1}=(x^{n-1})^i$ is the inverse of $u$. Indeed $$x^i(x^{n-1})^i=x^i(x^{ni-i})=x^{ni}=(x^n)^i=1_G^i=1_G$$ And since $\langle x\rangle$ is abelian $u^{-1}$ is indeed the inverse of $u$. Furthermore it is clear that $u^{-1}\in \langle x\rangle$.

3. Stable under composition: let $u,v\in \langle x\rangle$, then $u=x^i, v=x^j$ for some $1\leq i,j\leq n$. Then $uv=x^ix^j=x^{i+j}$. Here it is not clear that $x^{i+j} \in \langle x\rangle$. But remark that since $0\leq i,j<n$ implies that $i+j<2n$ thus we can find $0\leq k<n$ such that $i+j=n+k$ and therefore $$x^{i+j}=x^{n+k}=x^nx^k=x^k\in \langle x \rangle$$

I hope this works now