Singularities at infinity

Functions do not have residues, but rather differentials do. When you are calculating the residue of a function $f(z)$ at a point in the (finite) plane, you are really calculating the residue of the $1$-form $f(z)\,dz$.

If you try to calculate the "residue of a function" on a Riemann surface, you'll find that the result depends on the chart chosen, which makes no sense. However, the residue of a differential is invariant under a change of chart.

The differential $\frac{dz}{z}$, in the coordinate $w=1/z$, is $-\frac{dw}{w}$, which has residue $-1$ at $w=0$. Thus, even though $1/z$ is holomorphic at $\infty$ (and even has a simple zero there!), the differential $dz/z$ has a simple pole there. (In fact the differential $dz$ has a double pole at infinity!) Thus $dz/z$ has residues of equal magnitude and opposite signs at $0$ and $\infty$.

It is true that if $f(z)\,dz$ is holomorphic at $\infty$ then the integral of $f(z)\,dz$ around a small circle centred at $\infty$ is $0$. The point at infinity is not any different than any other point.