Why are even primes notable?

There are much-discussed theorems like Fermat's theorem on sums of two squares which make statements about odd primes only. This makes $2$ seem to be a "special" prime. In their book The book of numbers, Conway and Guy accordingly state that "Two is celebrated as the only even prime, which in some sense makes it the oddest prime of all."

On the other hand, the fact that $2$ is the only even prime is completely trivial, because the term "even" means the same thing as "divisible by $2$" and every prime number has the property that it is the only prime which is divisiable by itself.

So my question is: Is there really something special about even primes and if yes, what is it? Does aesthetics with regard to the theorems we are looking for play a role or is there a mathematical reason? Do we have theorems about primes which aren't divisible by $3, 5, ... $ or are there only results which don't apply to even primes?

Edit: As the user A.G has mentioned in a comment below, in many cases where we have a regular pattern, the fact that $2$ is too small for the pattern to kick in yet seems to be the decisive thing. So in these cases, the notable thing is not that $2$ is the only even prime but that it is the smallest prime.

The quip about "2 being the only even prime" is a bit silly, as you say, since 3 is the only divisible-by-3 prime, etc. For that quip, it's just that parity (odd-or-even) exists in the ambient language.

For $p$ prime, the $p$th roots of unity are in $\mathbb Q$ for $p=1$. Similarly, the $p$th roots of unity lie in all finite fields (of characteristic not $p$...) only for $p=2$.

Quadratic forms and bilinear forms behave differently in characteristic two.

Groups $SL(n,\mathbb F_q)$ do not assume their general pattern yet for small $n$ and $q=2$.

The index of alternating groups in symmetric groups is $2$.

Subgroups of index $2$ are normal.

The canonical anti-involution on a non-commutative ring, that reverse the order of multiplication, is of order $2$.

Here is a personal view on the 'oddness' of $2$:

Parity is important in a logically dichotomous universe; anything other than nothing in the universe is either $A$ or not-$A$ (for each categorization $A$ of things). As others have pointed out, this linguistic or logical 'ambience' brings $2$ to the forefront of our thinking about many things, even when its properties as a prime are not truly unique.

But $2$ is really unusual among the primes (for me) because it is the only prime (indeed the only positive integer $n>1$) for which $x^n+y^n=z^n$ has integer solutions. I consider this fact to be bafflingly improbable. Why are there solutions (and infinitely many of them) for only one integer exponent, and given that is the case, why is that exponent $2$, rather than another among the infinitude of possible primes?

Parity, either every number is one thing or another, is pretty important.

It's true that $3$ is the only prime divisible by $3$ but some of the primes not divisible by $3$ are $\equiv 1 \pmod 3$ and others are $\equiv -1\pmod 3$ whereas all primes other then $2$ are odd.

If $p<q$ are two different primes then $p+q$ is odd only if $p=2$ but $p+q$ could be any divisibility of $3$. ($3|p+q$ if $p\ne 3$ and $p\equiv -q\pmod 3$. $p+q\equiv 1$ if $p=3$ and $q\equiv 1$ or if $p\equiv q\equiv -1\pmod 3$ and $p+q\equiv -1$ if $p=3$ and $q\equiv 1$ or if $p\equiv q\equiv 1\pmod 3$).

And for $m\le n$ then $p^{m} + p^n = p^m(1+p^n)$ so $p^{m+1}\not \mid p^{m} + p^{n}$ should be a valid result. But... if $p=2$ and $m=n$ then....