Definition of $\mathbb{R}^\infty$

Question: Why is the topological space $\mathbb{R}^\infty$ defined to be the subset of $\prod_{i=1}^\infty \mathbb{R}_i$ consisting of sequences $(a_i)_{i=1} ^{\infty}$ such at most finitely many $a_i\neq 0$? Why does one insist on the condition that $a_i\neq0$ for at most finitely many $i$?

This condition makes $\mathbb{R}^\infty$ a CW-complex. This basically means it is a "good" topological space.

It also makes $\mathbb{R}^\infty$ the coproduct in the category of topological spaces (i.e. direct sum) as compared to the product (Cartesian product) $\prod_{n\in\mathbb{N}} \mathbb{R}^n$. Compare with the difference between the coproduct (direct sum) of infinitely many abelian groups, for example, and the product (direct product).

Another more elementary reason is the following theorem:

Let $f: A \rightarrow \prod_{\alpha \in J} X_{\alpha}$ be given coordinate-wise, i.e. $f(a) = (f_{\alpha}(a))_{\alpha \in J}$ where $f_{\alpha}:A \rightarrow X_{\alpha}$ with the product topology (i.e. the finite support condition you described) we have that $f$ is continuous if and only if $f_{\alpha}$ is.

This fails if we do not insist the finite support condition and the simplest counterexample $f: \mathbb{R} \rightarrow \prod_i \mathbb{R}_i$ given by $f(t) = (t, t, ..., )$ works