Ring theory notation question

By definition, $R[a]$ is the smallest subring of $K$ that contains both $R$ and $a$.

As you note, if $p(x)\in R[x]$, then $p(a)\in R[a]$ necessarily. Therefore, $$\{p(a)\mid p(x)\in R[x]\} \subseteq R[a].$$ Conversely, note that $$\{p(a)\mid p(x)\in R[x]\}$$ contains $a$ (as $p(a)$ where $p(x)=x$), contains $R$ (the constant polynomials); and is a subring of $K$: every element like is $K$; it is nonempty; it is closed under differences (since $p(a)-q(a) = (p-q)(a)$); and it is closed under products (since $p(a)q(a) = (pq)(a)$). Thus, $R[a]\subseteq \{p(a)\mid p(x)\in R[x]\}$. Hence we have equality.

More generally, if $a$ is integral over $R$ (satisfies a monic polynomial with coefficients in $R$, then letting $n$ be the smallest degree of a monic polynomial that is satisfied by $a$), then $R[a] = \{r_0 + r_1+\cdots + r_{n-1}a^{n-1}\mid r_i\in R\}$.

To see this, note that clearly the right hand side is contained on the left hand side. To prove the converse inclusion, let $p(x)$ be a monic polynomial of smallest degree such that $p(a)=0$. By doing induction on the degree, we can prove in the usual way that every element $f(x)$ of $R[x]$ can be written as $f(x)=q(x)p(x) + r(x)$, where $r(x)=0$ or $\deg(r)\lt deg(p)$; the reason being that the leading coefficient of $p$ is $1$, so we can perform the long-division algorithm without problems. Thus, $f(a) = q(a)p(a)+r(a) = r(a)$, so every element of $R[a]$ can be expressed as in the right hand side.

In the case where $a=\sqrt{2}$, $R=\mathbb{Z}$, $K=\mathbb{R}$ (or $K=\mathbb{Q}(\sqrt{2})$), we have that $a$ satisfies $x^2-2$< so that is why we get $$\mathbb{Z}[\sqrt{2}] = \{ p(\sqrt{2})\mid p(x)\in\mathbb{Z}[x]\} = \{r_0+r_1\sqrt{2}\mid r_0,r_1\in\mathbb{Z}\}.$$