Prove that $Z(\operatorname{Aut}(G))=\{e\}$, if $Z(G) =\{e\}$.
If $\phi$ is our central automorphism, then since it commutes with the conjugation automorphisms, we have for all $g,h\in G$, $$\phi(ghg^{-1})=g\phi(h)g^{-1}$$
Since $\phi$ is an automorphism, we can expand and rearrange to obtain, for all $g,h\in G$:$$\big(g^{-1}\phi(g)\big)\phi(h)\big(g^{-1}\phi(g)\big)^{-1}=\phi(h)$$
So for any fixed $g$, the element $g^{-1}\phi(g)$ commutes with every element in $\phi(G)=G$, so since $Z(G)={e}$, we have that $g^{-1}\phi(g)=e$ for all $g\in G$.
Thus, our central automorphism $\phi$ must be the identity.