Taking the automorphism group of a group is not functorial.
Solution 1:
Though the question has effectively been answered by the combined comments of Martin Brandenburg and Derek Holt, I thought I'd write up a complete answer for completeness' sake.
Let $N:=\Bbb{F}_{11}$ the finite field of $11$ elements and $H:=\Bbb{F}_{11}^{\times}$ its unit group. Let $G:=N\rtimes H$ the semidirect product of $N$ and $H$ given by the multiplication action of $H$ on $N$. Note that $G$ is isomorphic to the group of affine transformations of $\Bbb{F}_{11}$. Then the maps $$f:\ H\ \longrightarrow\ G:\ h\ \longmapsto\ (0,h)\qquad\text{ and }\qquad g:\ G\ \longrightarrow\ H:\ (n,h)\ \longmapsto\ h,$$ are group homomorphisms and satisfy $g\circ f=\operatorname{id}_H$. Now suppose there exists a covariant functor $$F:\ \mathbf{Grp}\ \longrightarrow\ \mathbf{Grp}:\ X\ \longmapsto\ \operatorname{Aut}(X).$$ Then we have group homomorphisms $$F(f):\ \operatorname{Aut}(H)\ \longrightarrow\ \operatorname{Aut}(G)\qquad\text{ and }\qquad F(g):\ \operatorname{Aut}(G)\ \longrightarrow\ \operatorname{Aut}(H),$$ satisfying $$F(g)\circ F(f)=F(g\circ f)=F(\operatorname{id}_H)=\operatorname{id}_{\operatorname{Aut}(H)},$$ so the identity on $\operatorname{Aut}(H)$ factors over $\operatorname{Aut}(G)$, i.e. $\operatorname{Aut}(G)$ has a subgroup isomorphic to $\operatorname{Aut}(H)$.
We have $\operatorname{Aut}(H)\cong\Bbb{Z}/4\Bbb{Z}$ because $H$ is abelian of order $10$. By this question we have $\operatorname{Aut}(G)\cong G$. But $|\operatorname{Aut}(G)|=|G|=11\times10=110$ is not divisible by $|\operatorname{Aut}(H)|=|\Bbb{Z}/4\Bbb{Z}|=4$, a contradiction. This shows that no such covariant functor exists. The contravariant case is entirely analogous.