Compute this following integral without Fourier series : $\int_0^{\pi/4}x\ln(\tan x)dx$

The Clausen Function $\operatorname{Cl}_2(z)$ can be defined as a Fourier-type Series. On the other hand it is afterall a very nice special function with a convenient relation to the Dilogarithm and capable of providing closed-form anti-derivatives for logarithmo-trigonometirc integrals such as the given one.

All we need are two identities involving the Clausen Function. To be precise

\begin{align*} \int_0^z\log(\tan t)\mathrm dt~&=~-\frac12\operatorname{Cl}_2(2z)-\frac12\operatorname{Cl}_2(\pi-2z)\tag1\\ \int_0^z\operatorname{Cl}_2(t)\mathrm dt~&=~\zeta(3)-\operatorname{Cl}_3(z)\tag2 \end{align*}

Using Integration By Parts and firstly $(1)$ and then $(2)$ we obtain

\begin{align*} \int_0^\frac\pi4x\log(\tan x)\mathrm dx&=\left[x\left(-\frac12\operatorname{Cl}_2(2x)-\frac12\operatorname{Cl}_2(\pi-2x)\right)\right]_0^\frac\pi4+\frac12\int_0^\frac\pi4\operatorname{Cl}_2(2x)+\operatorname{Cl}_2(\pi-2x)\mathrm dx\\ &=-\frac\pi4\operatorname{Cl}_2\left(\frac\pi2\right)+\frac14\int_0^\frac\pi2\operatorname{Cl}_2(x)\mathrm dx+\frac14\int_0^\frac\pi2\operatorname{Cl}_2(\pi-x)\mathrm dx\\ &=-\frac\pi4G+\frac14\int_0^\pi\operatorname{Cl}_2(x)\mathrm dx\\ &=-\frac\pi4G+\frac14\left[\zeta(3)-\operatorname{Cl}_3(\pi)\right]\\ &=-\frac\pi4G+\frac14\left[\zeta(3)+\eta(3)\right]\\ &=-\frac\pi4G+\frac7{16}\zeta(3) \end{align*}

$$\therefore~\int_0^\frac\pi4x\log(\tan x)\mathrm dx~=~-\frac\pi4G+\frac7{16}\zeta(3)$$

All here used identities can be rather easy deduced from the integral and series representation of the Clausen Function. In my opinion it gives an elegant way of solving such and similiar integrals.

The integral you obtained after the substitution $\tan x\mapsto x$ can be solved using the Inverse Tangent Integral $\operatorname{Ti}_2(z)$, another auxiliary function with roots within the theory of Polylogarithms.

Applying Integration By Parts twice yields to

\begin{align*} I=\int_0^1\frac{\arctan x}{1+x^2}\log x~\mathrm dx&=\underbrace{\left[\frac12\arctan^2x\log x\right]_0^1}_{\to0}-\frac12\int_0^1\arctan x\frac{\arctan x}x\mathrm dx\\ &=-\frac12\left[\operatorname{Ti}_2(x)\arctan x\right]_0^1+\frac12\int_0^1\frac{\operatorname{Ti}_2(x)}{1+x^2}\mathrm dx\\ &=-\frac\pi8G+\frac12\int_0^\frac\pi4\operatorname{Ti}_2(\tan x)\mathrm dx\\ &=-\frac\pi8G+\frac12\int_0^\frac\pi4\sum_{n\ge1}\frac{\sin[(4n-2)x]}{(2n-1)^2}+x\log(\tan x)\mathrm dx\tag{$\star$}\\ &=\frac12I-\frac\pi8G+\frac12\sum_{n\ge1}\frac1{(2n-1)^2}\int_0^\frac\pi4\sin[(4n-2)x]\mathrm dx\\ &=\frac12I-\frac\pi8G+\frac12\sum_{n\ge1}\frac1{(2n-1)^2}\left[\frac{\cos[(4n-2)x]}{4n-2}\right]_0^\frac\pi4\\ &=\frac12I-\frac\pi8G+\frac14\sum_{n\ge0}\frac1{(2n+1)^3}\\ &=\frac12I-\frac\pi8G+\frac14\lambda(3)\\ &=\frac12I-\frac\pi8G+\frac7{32}\zeta(3) \end{align*}

$$\therefore~I~=~\int_0^1\frac{\arctan x}{1+x^2}\log x~\mathrm dx~=~-\frac\pi4G+\frac7{16}\zeta(3)$$

Here $\lambda(s)$ is the Dirichlet Lambda Function. The result used at $(\star)$ is due to Ramanuajan and in his spirit I will omit a proof here. However, there are some steps within this second approach which are, indeed, in need of a more careful argumentation but the purpose of showing this approach is to illustrate the possibilities which may be used in order to evaluate this integral

\begin{align}\int_0^{\frac{\pi}{4}}x\ln(\tan x)\,dx=\int_0^{\frac{\pi}{4}}x\ln(\sin x)\,dx-\int_0^{\frac{\pi}{4}}x\ln(\cos x)\,dx\end{align}

and see: https://math.stackexchange.com/a/3200545/186817