How to find $\int_{0}^{2\pi} \frac{\cos(n\theta)}{(\cos(\theta)+\alpha)^2}d\theta, \alpha>1$

Doing the calculation with residues is indeed the easiest way. Write first

$$J_n(a)=\int_{0}^{2\pi}\frac{\cos n\theta}{a+\cos\theta}d\theta=\text{Re}\int_{0}^{2\pi}\frac{e^{i n\theta}}{a+\cos\theta}d\theta=\text{Re}~~(-2i)\oint\frac{z^n}{z^2+2az+1}dz$$

to express it as an integral on the unit circle. Here you can use simple residue calculus to obtain

$$J_n(a)=(-1)^n\frac{2\pi}{\sqrt{a^2-1}}\left(a-\sqrt{a^2-1}\right)^n$$

However the integral you want is given by $-dJ_n(a)/da$. Take a derivative here to show

$$\int_{0}^{2\pi}\frac{\cos n\theta}{(a+\cos\theta)^2}d\theta=(-1)^n\frac{2\pi}{(a^2-1)^{3/2}}\left(a-\sqrt{a^2-1}\right)^n\left(a+n\sqrt{a^2-1}\right)$$

Interestingly, one can obtain an infinite series in various ways from this integral, which can be seen as a generating function for the following combinatorial coefficients:

$$\sum_{m=0}^{\infty}{2m+n\choose m}a^{2m+n+1}=(-1)^n\frac{J_n(1/a)}{2\pi}=\frac{(1-\sqrt{1-a^2})^n}{a^{n-1}\sqrt{1-a^2}}$$