What's so special about characteristic 2?

I've often read about things which do not work in a field with a characteristic $2$, mainly things which have to do with factoring, or similar things. I'm not exactly sure why, but the only example of such a field I could think of is $\mathbb{Z}/2\mathbb{Z}$, which itself is an interesting field because it contains only the identity elements for the two groups, and naturally, it is a cyclic field. Do these properties lead to the fact that many things don't work if the charateristic is $2$

Any examples of things which break in such a field are also welcome.

Two is the smallest (and as people sometimes say: the oddest) of all primes.

Just to take a contrived example, let's say you want to show

If the sum of two squares equals the square of the sum then one of the two is zero.

Well, that's easy, you just transform $a^2+b^2=(a+b)^2$ to obtain $0=2ab$; and as a product is $0$ only if one of the factors is zero, you conclude that $a=0$ or $b=0$. Done? No! We forgot the third factor. We should have said: $a=0$ or $b=0$ or $2=0$. And the latter is exactly what happens in characteristic $2$, i.e., in characteristic $2$ our claim does not (necessarily) hold.

To put it differently: In the attempt to arrive at $ab=0$ we had to divide by $2$, and as always when dividing we must make sure that we do not accidentally divide by zero. It happens ever so often that you have to divide by something. If you need to divide by $a-b$, say, you can circumvent the problem by adding a condition to your claim ("... provided $a\ne b$"). But sometimes you need to divide by an explicit constant such as $2$ in our example. In that case the condition to be added to the claim must be that the characteristic of the field is not a divisor of that constant.

The fact that it is often only characteristic $2$ that needs to be mentioned as exception might be called a consequence of the law of small numbers: It happens much more often that a factor $2$ pops up naturally than a factor $97$, say. That's why characteristic $2$ so often and characteristic $3$ sometimes plays a special role.

In a field of characteristic two, everything is its own additive inverse. so $x=-x$ for every element, and in a lot of proofs, we show something is $0$ by showing it is equal to its own additive inverse. This obviously fails in fields of characteristic 2.

Any finite field of characteristic 2 turns out to be the unique finite field of order $2^n$ for some $n$. Infinite fields exist as well, the simplest would be the field of rational polynomials over $\mathbb Z / \mathbb 2Z$