Is there a reason why $Z(G)$ is named the "centre" of a group?

An element is called central if it commutes with everything else...i.e., it does not matter whether you multiply from the left or right, so you can think of such an element as being multiplied in the "center" of any product it is in. Starting from there, it is an easy step to start calling the subgroup of all such elements the center. And from there we call it $Z(G)$, the Z being an abbreviation for the German word for center if I remember right.

By virtue of left and right multiplications, a group $G$ "naturally lives" in $Sym(G)$ (the group of all the bijections of $G$ into itself) in the shape of a pair of subgroups of $Sym(G)$, say $\Theta$ and $\Gamma$, both of which it is isomorphic to. These subgroups commute, so $\Theta\Gamma$ is also a subgroup of $Sym(G)$. Finally, and this is mostly relevant for your question, $Z(G)$ turns out to be isomorphic to $\Theta \cap \Gamma$. Then, in $Sym(G)$ everything looks symmetric around the "center" $\Theta \cap \Gamma$:

For clarity, I'm not saying this is really the reason why the center was historically named that way. It's just a way I "pictorially" found for myself to accept that such a name actually makes sense.

Likewise, I've given here an interpretation of the wording "inner automorphism" (see Proposition 3 therein and the following comment).