Nonexistence of a homeomorphism between a open set and the unit sphere

This is a collection of remarks that should answer your question:

• The most common tool to prove that open sets of different dimension cannot be homeomorphic is invariance of domain. See the section on consequences.

  Two open sets on manifolds of different dimensions cannot be homeomorphic either. The reason is that if you study the restriction of the homeomorphism to a small set, you get a homeomorphism between Euclidean open sets via coordinate charts. (This remark will only make sense if you are familiar with topological or smooth manifolds.)

  However, now your two sets have the same dimension, so this method doesn't work.

• In this case the sphere is compact and an open set isn't.
• Not every connected open set of the Euclidean space is homeomorphic to the whole space. For example, the punctured space $\mathbb R^n\setminus\{0\}$ isn't homeomorphic to $\mathbb R^n$.

Since the sphere is compact and open sets are not, there cannot be a homeomorphism between them, since homeomorphisms preserve compactness

The results you need is:

$\bullet$ Image of compact set over continuous function is compact.

$\bullet$ $\mathbb{S^n}$ is compact.

$\bullet$ With the standard topology, There's no (no empty) open set $U\subset \mathbb{R^n}$ that is also compact (compact in $\mathbb{R^n}$ is equivalent to be closed and bounded, and the only no empty open and closed set in $\mathbb{R^n}$ is $\mathbb{R^n}$ itself, which is not bounded).

By absurd, suppose now that there is a homeomorphism $f:\mathbb{S^n}\to U$.

Than if you define $ {f}^*:\mathbb{S^n}\to \mathbb{R^n}$ as $f^*(x)=f(x)$ for all $x\in\mathbb{S^n}$ (we just extended the codomain of $f$), we see that $f^*$ is a continuous function and $f^*(\mathbb{S^n})=U$. but this would imply that $U$ is compact, absurd since $U$ is open.