How to evaluate $I=\int_0^\pi\frac{\cos(\theta+\alpha)\sin(\theta)d\theta}{\sqrt{r^2+a^2-2ra\cos(\theta)}}$

For $\cos(\theta+\alpha) = \cos\theta\cos\alpha-\sin\theta\sin\alpha$ $$ I = \cos\alpha \int_{0}^{\pi} \frac{\cos\theta\sin\theta}{\sqrt{r^2+a^2-2ra\cos\theta}} \,\mathrm{d}\theta - \sin\alpha \int_{0}^{\pi} \frac{\sin^2\theta}{\sqrt{r^2+a^2-2ra\cos\theta}} \,\mathrm{d}\theta $$ The first integral is trivial, here I focus on the second $$ \begin{aligned} \int_{0}^{\pi} \frac{\sin^2\theta}{\sqrt{r^2+a^2-2ra\cos\theta}} \,\mathrm{d}\theta &= \int_{0}^{\pi} \frac{\sin^2\theta}{\sqrt{(r+a)^2-4ra\cos^2\frac{\theta}{2}}} \,\mathrm{d}\theta \quad \text{let }k^2=\frac{4ra}{(r+a)^2} \text{ and } \frac{\theta}{2}\mapsto\theta \\ &= \frac2{r+a} \int_{0}^{\pi/2} \frac{\sin^22\theta}{\sqrt{1-k^2\cos^2\theta}} \,\mathrm{d}\theta \quad \theta\mapsto\frac{\pi}{2}-\theta \\ &= \frac2{r+a} \int_{0}^{\pi/2} \frac{\sin^22\theta}{\sqrt{1-k^2\sin^2\theta}} \,\mathrm{d}\theta \end{aligned} $$ Integrating by parts gives $$ \int_{0}^{\pi/2} \frac{\sin^22\theta}{\sqrt{1-k^2\sin^2\theta}} \,\mathrm{d}\theta = \frac4{k^2} \int_{0}^{\pi/2} \cos2\theta \sqrt{1-k^2\sin^2\theta} \,\mathrm{d}\theta $$ where denote $I=\frac4{k^2}J$.

From another aspect, setting $$ \sin^22\theta=A\cos2\theta(1-k^2\sin^2\theta) + B(1-k^2\sin^2\theta) + C $$ or $$ -4\sin^4\theta+4\sin^2\theta = 2k^2A\sin^4\theta - ((k^2+2)A+k^2B)\sin^2\theta + (A+B+C) $$ comparing the coefficients, we solve $$ A=-\frac2{k^2}, \quad B=-\frac{2(k^2-2)}{k^4}, \quad C=\frac{4(k^2-1)}{k^4} $$ which suggests $$ I=-\frac2{k^2}J + \frac{4(k^2-1)}{k^4}K(k^2) - \frac{2(k^2-2)}{k^4}E(k^2) $$ Item of $J$ can be cancelled with $I=\frac4{k^2}J$, hence $$ I = \int_{0}^{\pi/2} \frac{\sin^22\theta}{\sqrt{1-k^2\sin^2\theta}} \,\mathrm{d}\theta = \frac{8(k^2-1)}{3k^4}K(k^2) - \frac{4(k^2-2)}{3k^4}E(k^2) $$ or plug-in $k^2$ $$ \int_{0}^{\pi} \frac{\sin^2\theta}{\sqrt{r^2+a^2-2ra\cos\theta}} \,\mathrm{d}\theta = -\frac{r+a}{3r^2a^2}\left((r-a)^2K\left(\frac{4ra}{(r+a)^2}\right) - (r^2+a^2)E\left(\frac{4ra}{(r+a)^2}\right)\right) $$ the rest of calculation is obvious.


First, simplify the expression writing $${\sqrt{r^2+a^2-2ra\cos(\theta)}}=\sqrt{r^2+a^2} \sqrt{1-k \cos(\theta)}\quad \text{where} \quad k=\frac {2ra}{r^2+a^2}$$

There is even an antiderivative which contains a bunch of elliptic integrals of the first and second kind.

Using @Andrei suggestion, the numerator write $$\sin (\alpha ) \cos ^2(\theta )+\cos (\alpha ) \sin (\theta ) \cos (\theta )-\sin (\alpha )$$ and the substitution $x=\cos(\theta)$ leads to "simple" integrals, the most difficult being the one with $\cos ^2(\theta )$ (but perfectly doable).

If I did not make any mistake, for the definite integral, we should arrive at $$\color{blue}{\frac{2 a^3 \cos (\alpha )-(a+r) \sin (\alpha ) \left(\left(a^2+r^2\right) E\left(\frac{4 a r}{(a+r)^2}\right)-(a-r)^2 K\left(\frac{4 a r}{(a+r)^2}\right)\right)}{3 a^2 r^2}}$$ I just made a numerical check with $a=1$, $r=2$, $\alpha=\frac \pi 6$ which gives $$\frac{1}{24} \left(3 K\left(\frac{8}{9}\right)+2 \sqrt{3}-15 E\left(\frac{8}{9}\right)\right)=-0.235672$$ which is the value given by numerical integration.