Is a Bijection From a Group to Itself Automatically an Isomorphism If It Maps the Identity to Itself?

The reason this occurs for the Klein group is that the three non-identity elements all behave identically: they all square to give the identity, and multiplying any two of them gives the third. So, they're indistinguishable from the group-theoretic perspective.

On the other hand, as Gerry Myerson and Tobias Kildetoft point out in the comments above, this is a special phenomenon. E.g., take the cyclic group of order $4$. What is its automorphism group? Try other groups: $S_3$, the cyclic group of order $n$ where $n$ is an arbitrary positive integer, $S_4$ (if you're up for it).

I'd just like to mention a wonderful theorem for those reading this answer. It's pretty deep but it really is marvellous and related to the question:

Theorem (Horosevskii)

If $G$ is a finite group, then the order of an automorphism $\sigma\in \text{Aut}(G)$ (where $\text{Aut}(G)$ is the automorphism group of $G$) is at most $\left|G\right|$.

Why is this deep? In general, $\text{Aut}(G)\subseteq S_{\left|G\right|!}$ so you can say that the order of any automorphism of $G$ in its automorphism group is at most $\left(\left|G\right|!\right)!$. It is remarkable that, indeed, you can tighten this bound to $\left|G\right|$ for all finite groups.

I hope this helps!

Here is a more detailed explanation of what is going on.

First, if the group $G$ acts on a set $X$, then the action is said to be transitive if for any two elements $x$ and $y$ in $X$, there is some element $g$ in $G$ such that $g.x = y$.

Now, the group action we are looking at here is that of the automorphism group on the group itself (so for $\varphi\in\rm{Aut}(G)$ and $g\in G$ the action is $\varphi.g = \varphi(g)$).

Obviously, this action cannot be transitive, as we have $\varphi(1) = 1$ for any $\varphi$. However, we can ask whether it is transitive on $G\setminus\{1\}$.

It turns out that those finite groups $G$ for which this is the case are precisely those of the form $(C_p)^n$ for some prime $p$ and some natural number $n$ (here $C_p$ denotes the cyclic group of order $p$).

The way to see this is to note that any automorphism will preserve the order of any element, so if the action is transitive on the non-identity elements, then these must all have precisely the same order. But then that order must be a prime, and hence the order of $G$ is a power of some prime $p$.

Now, a group of prime power order has a non-trivial center, and the image of a central element under an automorphism is again central, so since the action on the non-identity elements is transitive, we see that the center must in fact be the entire group, so $G$ is abelian. Now the claimed form of the group follows from the classification of finite abelian groups.

To see that the converse is also true, note that any group of this form is a vector space over the field of $p$ elements, and the claim is simply that given any two non-zero vectors, there is a linear map sending one to the other, which is an easy exercise in linear algebra.

So now we know that to even have transitive action on the non-identity elements (and if we can have all permutations, then the action will certainly be transitive), the group must be as explained above.

If $|G|$ is $1$, $2$, $3$, or $4$ (and $G$ has the above form), then we do indeed get all the permutations as automorphisms, as is easily checked directly.

However, in all other cases, this will not be the case, since in those, the number of non-identity elements will be large enough that if we have all permutations, then in fact the action would be 2-transitive, which means that given any four elements $x,y,z,w$ with $x\neq y$ and $z\neq w$ then there is some element sending $x$ to $y$ and $z$ to $w$.

But the automorphism group cannot be 2-transitive on the non-identity elements unless $p = 2$, since if $p\geq 3$ (and $n\geq 2$), we can pick the four element $x,x^2, z, w$ with $w\neq z$ and $w\neq z^2$, and if an automorphism sends $x$ to $z$ then it must send $x^2$ to $z^2$ (so it cannot send $z$ to $w$ as at least one of them would have to if the action was 2-transitive).

(If $p= 2$ then the automorphism group is in fact 2-transitive on the non-identity elements. This is because two distinct vectors over the field of $2$ elements will always be linearly independent).

So this leaves the question in case $p = 2$ and $n\geq 3$. But in this case, we would need a 3-transitive action (defined similarly to 2-transitive but with six elements such that none of the first three are equal and none of the last three are equal). And we would now have enough elements that we could take the elements $x,y,xy,z,w,v$ with $v\neq zw$ and the same argument as before shows that the action cannot be 3-transitive.

So we now conclude that the only groups for which all permutations of the non-identity elements give automorphisms are the ones of order $1$, $2$, $3$ and the one of order $4$ which is not cyclic.

Just to augment Tobias's excellent answer:

There is no infinite group $G$ such that every bijection of $G$ fixing the identity is an automorphism.

Suppose $G$ is an infinite group with this property. Then as above every nonidentity element must have the same order, and now this order could either be a prime number $p$ or infinite.

If every nonidentity element has order $2$, then $G$ is an infinite-dimensional vector space over $\mathbb{F}_2$, and again the action cannot be $3$-transitive, since some triples are linearly independent and some are linearly dependent. (Indeed, this argument works unless $G$ is trivial, has order $2$ or is isomorphic to $C_2 \times C_2$.)

If every nonidentity element has order greater than $2$, then for $\sigma \in \operatorname{Aut} G$ and $e \neq g \in G$, we have that if $\sigma(g) = x$, then $\sigma(g^{-1}) = x^{-1}$, but a bijection $\sigma$ of $G$ fixing the identity with $\sigma(g) = x$ need not map $g^{-1}$ to $x^{-1}$: there are $\# G - 3 > 0$ other choices. Thus $\operatorname{Aut} G$ is not $2$-transitive on $G \setminus \{e\}$. (Indeed, this argument works unless $G$ is trivial or has order $3$.)