Why should we expect the connection between complex arithmetic and geometry?

We can start way earlier to get a geometric interpretation, at the real numbers. Multiplication by a real number is a combination of scaling and mirroring. Multiplying by a a positive number is scaling the real line, multiplying by $-1$ is mirroring it at the origin. On an abstract level, a core feature of mirroring is that doing it twice returns the original picture. This gives rise to the interpretation that multiplication by $-1$ is a mirroring, since $(-1)^2=1$, so multiplying by $-1$ twice is the identity.

The complex numbers give rise to a similar interpretation. We can still view multiplication by $-1$ as mirroring the plane at the origin, but in a 2d context, we can also see it as a $180^\circ$ rotation. They are really the same. But we also get a new element, $\mathrm i$. Its basic feature is that $\mathrm i^2=-1$, that is, multiplying by $\mathrm i$ twice is rotation by $180^\circ$. But that's also a core feature of rotation by $90^\circ$: rotating by that amount twice is the same as rotating by $180^\circ$ once. So that's a good hint that complex multiplication can have something to do with rotations. We just need to find a fitting topology (a scalar product to describe angles, most importantly) which makes multiplication by $\mathrm i$ an actual $90^\circ$ rotation. And it turns out that the scalar product wrt which $1$ and $\mathrm i$ form an orthonormal basis does just that. So it's a good idea to choose those as a basis of $\mathbb C$ as a real vector space, making them span the coordinate axes. In this picture, multiplication by $\mathrm i$ will be guaranteed to be a $90^\circ$ rotation. And using some algebra, all other complex multiplications can then be shown to also be rotations and scalings.

From the point of view of group theory there is a deep reason : the group of similitudes ( ratio-of-lengths preserving maps) of a (Euclidean) plane is isomorphic to the group of affine (or anti-affine) transformation of a complex line $(z\to az+b$ or $z\to a \bar z+b$). This (exceptional) isomorphism enables us to do geometry by using complex numbers.

This is even more clear if we go to the projective line (the Riemann Sphere). The group of projective transformations of a projective line $PGL(2,C)$ is isomorphic to Möbius group of conformal maps of a sphere $PSO(3,1)$.