integral cohomology ring of real projective space

The ring structure of the integral cohomology can be determined by the ring structure of the mod 2 cohomology. Indeed, the reduction mod $2$ map $H^*(\mathbb{R}P^\infty;\mathbb{Z})\to H^*(\mathbb{R}P^\infty;\mathbb{Z}/2)$ is a ring homomorphism and is injective in all positive degrees (this is easy to see using cellular cohomology, or by computing both cohomology groups from integral cohomology via the universal coefficient theorem and then using the naturality of the maps in universal coefficient theorem with respect to the coefficient group). It follows that $H^*(\mathbb{R}P^\infty;\mathbb{Z})\cong\mathbb{Z}[y]/(2y)$, where $|y|=2$ and the mod $2$ reduction of $y$ is $x^2$ where $x$ is the polynomial generator of the mod $2$ cohomology.

For $H^n(\mathbb{R}P^n;\mathbb{Z})$, the story is similar, except that if $n$ is odd then mod $2$ reduction will also be non-injective in degree $n$ (but there are no nontrivial products involving the degree $n$ cohomology, so this is no obstruction to computing the ring structure). We thus get that if $n$ is odd, $H^*(\mathbb{R}P^n;\mathbb{Z})$ is just the quotient ring of $H^*(\mathbb{R}P^\infty;\mathbb{Z})$ by all classes of degree $>n$. When $n$ is odd, we have the same description except that there is an additional generator in $H^n$ (which of course has trivial products with all classes of positive degrees).

As pointed out by fixedp in the comments, Hatcher computes $H^*(\mathbb{RP}^{\infty}; \mathbb{Z})$ after the proof of Theorem $3.19$. The result is

$$H^*(\mathbb{RP}^{\infty}; \mathbb{Z}) \cong \mathbb{Z}[\alpha]/(2\alpha),\quad |\alpha| = 2.$$

He then says the same technique can be used to show

\begin{align*} H^*(\mathbb{RP}^{2k}; \mathbb{Z}) &\cong \mathbb{Z}[\alpha]/(2\alpha, \alpha^{k+1}), \quad |\alpha| = 2\\ H^*(\mathbb{RP}^{2k+1}; \mathbb{Z}) &\cong \mathbb{Z}[\alpha, \beta]/(2\alpha, \alpha^{k+1}, \beta^2, \alpha\beta),\quad |\alpha| = 2, |\beta| = 2k + 1. \end{align*}