Evaluating $\int_0^{\pi/4} \ln(\tan x)\ln(\cos x-\sin x)dx=\frac{G\ln 2}{2}$

Solution 1:

$\newcommand{\angles}[1]{\left\langle\,{#1}\,\right\rangle} \newcommand{\braces}[1]{\left\lbrace\,{#1}\,\right\rbrace} \newcommand{\bracks}[1]{\left\lbrack\,{#1}\,\right\rbrack} \newcommand{\dd}{\mathrm{d}} \newcommand{\ds}[1]{\displaystyle{#1}} \newcommand{\expo}[1]{\,\mathrm{e}^{#1}\,} \newcommand{\half}{{1 \over 2}} \newcommand{\ic}{\mathrm{i}} \newcommand{\iff}{\Longleftrightarrow} \newcommand{\imp}{\Longrightarrow} \newcommand{\Li}[1]{\,\mathrm{Li}_{#1}} \newcommand{\mc}[1]{\mathcal{#1}} \newcommand{\mrm}[1]{\mathrm{#1}} \newcommand{\ol}[1]{\overline{#1}} \newcommand{\pars}[1]{\left(\,{#1}\,\right)} \newcommand{\partiald}[3][]{\frac{\partial^{#1} #2}{\partial #3^{#1}}} \newcommand{\root}[2][]{\,\sqrt[#1]{\,{#2}\,}\,} \newcommand{\totald}[3][]{\frac{\mathrm{d}^{#1} #2}{\mathrm{d} #3^{#1}}} \newcommand{\ul}[1]{\underline{#1}} \newcommand{\verts}[1]{\left\vert\,{#1}\,\right\vert}$

$\ds{\int_{0}^{1}{\ln\pars{x}\ln\pars{1 - x} \over 1 + x^{2}}\,\dd x - \half\int_{0}^{1}{\ln\pars{x}\ln\pars{1 + x^{2}} \over 1 + x^{2}}\,\dd x = \half\,\ln\pars{2}\,\mrm{G}}$.

$\ds{\mrm{G}:\ \mbox{Catalan Constant.}}$

---

\begin{align} &\color{#f00}{\int_{0}^{1}{\ln\pars{x}\ln\pars{1 - x} \over 1 + x^{2}}\,\dd x - \half\int_{0}^{1}{\ln\pars{x}\ln\pars{1 + x^{2}} \over 1 + x^{2}}\,\dd x} \\[5mm] = &\ \int_{0}^{1}{\ln\pars{x}\ln\pars{1 - x} \over 1 + x^{2}}\,\dd x - \Re\int_{0}^{1}{\ln\pars{x}\ln\pars{1 + x\ic} \over 1 + x^{2}}\,\dd x \label{1}\tag{1} \end{align} $\ds{\ln}$-function branch-cut is chosen along the 'negative real axis'. Namely, in $\ds{\left.\vphantom{\large A}\ln\pars{z}\right\vert_{\ z\ \not=\ 0}}$ we have $\ds{-\pi < \mrm{arg}\pars{z} < \pi}$. For instance, when $\ds{x \in \pars{0,1}}$ we have: \begin{align} \ln\pars{1 + \ic x} & = \ln\pars{\root{1 + x^{2}}} + \arctan\pars{x}\ic = \ol{\bracks{\ln\pars{\root{1 + x^{2}}} - \arctan\pars{x}\ic}} \\[5mm] & = \ol{\ln\pars{1 - x\ic}} \\[5mm] \mbox{and}\ \ln\pars{1 + x^{2}} & = \ln\pars{1 + x\ic} + \ln\pars{1 - x\ic} = 2\,\Re\ln\pars{1 + x\ic}\quad \mbox{which we already used in \eqref{1}}. \end{align}

---

With the identity $\ds{ab = \half\,a^{2} + \half\,b^{2} - \half\,\pars{a - b}^{2}}$, the expression \eqref{1} can be rewritten in the form \begin{align} &\color{#f00}{\int_{0}^{1}{\ln\pars{x}\ln\pars{1 - x} \over 1 + x^{2}}\,\dd x - \half\int_{0}^{1}{\ln\pars{x}\ln\pars{1 + x^{2}} \over 1 + x^{2}}\,\dd x} \\[5mm] = &\ \overbrace{\half\int_{0}^{1}{\ln^{2}\pars{1 - x} \over 1 + x^{2}}\,\dd x} ^{\ds{\color{#f00}{\mc{J}_{1}}}}\ \overbrace{-\,\half\int_{0}^{1}{\ln^{2}\pars{x/\bracks{1 - x}} \over 1 + x^{2}} \,\dd x} ^{\ds{\color{#f00}{\mc{J}_{2}}}}\ \overbrace{- \half\,\Re\int_{0}^{1}{\ln^{2}\pars{1 + x\ic} \over 1 + x^{2}}\,\dd x} ^{\ds{\color{#f00}{\mc{J}_{3}}}} \\[5mm] + &\ \underbrace{% \half\,\Re\int_{0}^{1}{\ln^{2}\pars{x/\bracks{1 + x\ic}} \over 1 + x^{2}} \,\dd x}_{\ds{\color{#f00}{\mc{J}_{4}}}}\ =\ \color{#f00}{\mc{J}_{1}} + \color{#f00}{\mc{J}_{2}} +\color{#f00}{\mc{J}_{3}} +\color{#f00}{\mc{J}_{4}} \end{align}

It turns out that the above integrals can be reduced to the form $$ \left.\int{\ln^{2}\pars{x} \over a - x}\,\dd x\,\right\vert_{\ a\ \not=\ 0} \,\,\,\,\,\stackrel{x\ =\ at}{=}\,\,\,\,\, \int{\ln^{2}\pars{at} \over 1 - t}\,\dd t = -\int\ln^{2}\pars{at}\,\dd\bracks{\ln\pars{1 - t}} $$ which can be easily evaluated by successive integration by parts: \begin{equation} \int{\ln^{2}\pars{x} \over a - x}\,\dd x = \left\lbrace\begin{array}{lcl} \ds{-\ln^{2}\pars{x}\ln\pars{1 - {x \over a}} - 2\ln\pars{x}\Li{2}\pars{x \over a} + 2\Li{3}\pars{x \over a}} & \mbox{if} & \ds{a \not= 0} \\ \ds{-\,{1 \over 3}\,\ln^{3}\pars{x}} & \mbox{if} & \ds{a = 0} \end{array}\right. \end{equation} $$ \begin{array}{|c|}\hline\mbox{}\\ \quad\mbox{Hereafter, we'll use this result to evaluate}\ \ds{\braces{\vphantom{\large A}\color{#f00}{\mc{J}_{k}}\,,\ k = 1,2,3,4}} \quad \\ \mbox{}\\ \hline \end{array} $$

---

With $\ds{r \equiv 1 + \ic}$:

1. $\ds{\large\color{#f00}{\mc{J}_{1}}:\ ?}$. \begin{align} \color{#f00}{\mc{J}_{1}} & \equiv \half\int_{0}^{1}{\ln^{2}\pars{1 - x} \over 1 + x^{2}}\,\dd x\,\,\,\,\, \stackrel{x\ \mapsto\ \pars{1 - x}}{=}\,\,\,\,\, \half\int_{0}^{1}{\ln^{2}\pars{x} \over x^{2} - 2x + 2}\,\dd x \\[5mm] & = \half\int_{0}^{1}{\ln^{2}\pars{x} \over \pars{x - r}\pars{x - \ol{r}}}\,\dd x = -\,\half\,\Im\int_{0}^{1}{\ln^{2}\pars{x} \over r - x}\,\dd x = \color{#f00}{\Im\Li{3}\pars{\half\,r}}\label{J1}\tag{J1} \end{align}

   ---

2. $\ds{\large\color{#f00}{\mc{J}_{2}}:\ ?}$. \begin{align} \color{#f00}{\mc{J}_{2}} & \equiv -\,\half\int_{0}^{1}{\ln^{2}\pars{x/\bracks{1 - x}} \over 1 + x^{2}}\,\dd x \,\,\,\,\,\stackrel{x/\pars{1 - x}\ \mapsto\ x}{=}\,\,\,\,\, -\,\half\int_{0}^{\infty}{\ln^{2}\pars{x} \over 2x^{2} + 2x + 1}\,\dd x \\[5mm] & = -\,\half\int_{0}^{1}{\ln^{2}\pars{x} \over 2x^{2} + 2x + 1}\,\dd x - \half\int_{0}^{1}{\ln^{2}\pars{x} \over x^{2} + 2x + 2}\,\dd x \\[5mm] & = -\,{1 \over 4}\int_{0}^{1} {\ln^{2}\pars{x} \over \pars{x + r/2}\pars{x + \ol{r}/2}}\,\dd x - \half\int_{0}^{1}{\ln^{2}\pars{x} \over \pars{x + r}\pars{x + \ol{r}}}\,\dd x \\[5mm] & = -\,\half\,\Im\int_{0}^{1}{\ln^{2}\pars{x} \over -r/2 - x}\,\dd x - \half\,\Im\int_{0}^{1}{\ln^{2}\pars{x} \over -r - x}\,\dd x \end{align} However, $$ \left\lbrace\begin{array}{rcl} \ds{-\,\half\,\Im\int_{0}^{1}{\ln^{2}\pars{x} \over -r/2 - x}\,\dd x} & \ds{=} & \ds{-\,{5 \over 128}\,\pi^{3} - {1 \over 32}\,\ln^{2}\pars{2}\pi - \Im\Li{3}\pars{-\,{r \over 2}}} \\[3mm] \ds{-\,\half\,\Im\int_{0}^{1}{\ln^{2}\pars{x} \over -r - x}\,\dd x} & \ds{=} & \ds{-\Im\Li{3}\pars{-\,{\ol{r} \over 2}}} \end{array}\right. $$ Then, \begin{equation} \color{#f00}{\mc{J}_{2}} = \color{#f00}{-\,{5 \over 128}\,\pi^{3} - {1 \over 32}\,\ln^{2}\pars{2}\pi} \label{J2}\tag{J2} \end{equation}

   ---

3. $\ds{\large\color{#f00}{\mc{J}_{3}}:\ ?}$. \begin{align} \color{#f00}{\mc{J}_{3}} & \equiv -\,\half\,\Re\int_{0}^{1}{\ln^{2}\pars{1 + x\ic} \over 1 + x^{2}}\,\dd x \,\,\,\,\,\stackrel{\pars{1 + x\ic}\ \mapsto\ x}{=} -\,\half\,\Im\int_{1}^{r}{\ln^{2}\pars{x} \over \pars{2 - x}x}\,\dd x \\[5mm] & = -\,{1 \over 4}\,\Im\int_{1}^{r}{\ln^{2}\pars{x} \over 2 - x}\,\dd x - {1 \over 4}\,\Im\int_{1}^{r}{\ln^{2}\pars{x} \over x}\,\dd x \end{align} The remaining integrals are given by: $$ \left\lbrace\begin{array}{rcl} \ds{-\,{1 \over 4}\,\Im\int_{1}^{r}{\ln^{2}\pars{x} \over 2 - x}\,\dd x} & \ds{=} & \ds{{1 \over 96}\,\pi^{3} - {3 \over 32}\,\ln^{2}\pars{2}\pi + {1 \over 4}\,\ln\pars{2}\,\mrm{G} - \half\,\Im\Li{3}\pars{r \over 2}} \\[3mm] \ds{-\,{1 \over 4}\,\Im\int_{1}^{r}{\ln^{2}\pars{x} \over x}\,\dd x} & \ds{=} & \ds{{1 \over 768}\,\pi^{3} - {1 \over 64}\,\ln^{2}\pars{2}\,\pi} \end{array}\right. $$ Then, \begin{equation} \color{#f00}{\mc{J}_{3}} = \color{#f00}{{3 \over 256}\,\pi^{3} - {7 \over 64}\,\ln^{2}\pars{2}\pi + {1 \over 4}\,\ln\pars{2}\,\mrm{G} - \half\,\Im\Li{3}\pars{r \over 2}} \label{J3}\tag{J3} \end{equation}

   ---

4. $\ds{\large\color{#f00}{\mc{J}_{4}}:\ ?}$. \begin{align} \color{#f00}{\mc{J}_{4}} & \equiv \half\,\Re\int_{0}^{1}{\ln^{2}\pars{x/\bracks{1 + x\ic}} \over 1 + x^{2}}\,\dd x \,\,\,\,\,\stackrel{x/\pars{1 + x\ic}\ \mapsto\ x}{=}\,\,\,\,\, {1 \over 4}\,\Im\int_{0}^{\ol{r}/2}{\ln^{2}\pars{x} \over -\ic/2 - x}\,\dd x \\[5mm] & = \color{#f00}{{7 \over 256}\,\pi^{3} + {9 \over 64}\,\ln^{2}\pars{2}\pi + {1 \over 4}\,\ln\pars{2}\,\mrm{G} - \half\,\Im\Li{3}\pars{r \over 2}} \label{J4}\tag{J4} \end{align}

   ---

   Summarising $\ds{\pars{~\vphantom{\large A}\mbox{see}\ \eqref{J1}, \eqref{J2}, \eqref{J3}\ \mbox{and}\ \eqref{J4}~}}$: \begin{equation} \left\lbrace\begin{array}{rcccccccc} \ds{\color{#f00}{\mc{J}_{1}}} & \ds{=} &&&&&&&\ds{\Im\Li{3}\pars{r \over 2}} \\[3mm] \ds{\color{#f00}{\mc{J}_{2}}} & \ds{=} & \ds{-\,{5 \over 128}\,\pi^{3}} & \ds{-} & \ds{{1 \over 32}\,\ln^{2}\pars{2}\pi} &&&& \\[3mm] \ds{\color{#f00}{\mc{J}_{3}}} & \ds{=} & \ds{{3 \over 256}\,\pi^{3}} & \ds{-} & \ds{{7 \over 64}\,\ln^{2}\pars{2}\pi} & \ds{+} & \ds{{1 \over 4}\,\ln\pars{2}\,\mrm{G}} & \ds{-} & \ds{\half\,\Im\Li{3}\pars{r \over 2}} \\[3mm] \ds{\color{#f00}{\mc{J}_{4}}} & \ds{=} & \ds{{7 \over 256}\,\pi^{3}} & \ds{+} & \ds{{9 \over 64}\,\ln^{2}\pars{2}\pi} & \ds{+} & \ds{{1 \over 4}\,\ln\pars{2}\,\mrm{G}} & \ds{-} & \ds{\half\,\Im\Li{3}\pars{r \over 2}} \end{array}\right. \end{equation} The $\ds{\quad\ul{final\ result}\quad}$ is given by: \begin{align} &\color{#f00}{\int_{0}^{1}{\ln\pars{x}\ln\pars{1 - x} \over 1 + x^{2}}\,\dd x - \half\int_{0}^{1}{\ln\pars{x}\ln\pars{1 + x^{2}} \over 1 + x^{2}}\,\dd x} \\[5mm] = &\ \mc{J}_{1} + \mc{J}_{2} + \mc{J}_{3} + \mc{J}_{4} = \color{#f00}{\half\,\ln\pars{2}\,\mrm{G}}\,,\qquad \pars{~\mrm{G}:\ \mbox{Catalan Constant}~} \end{align}

Solution 2:

Following is the proof that,

\begin{equation*} \displaystyle \int_0^{\pi/4} \ln(\tan x)\ln(\cos x-\sin x)dx=\dfrac{G\ln 2}{2} \end{equation*}

Knowing that, \begin{equation*} \displaystyle \beta(3)=\sum_{n=1}^{\infty} \dfrac{(-1)^n}{(2n+1)^3} \end{equation*}

then, \begin{equation*} \displaystyle\int_0^1 \dfrac{\arctan x\ln x}{x}dx=-\beta(3) \end{equation*}

(series expansion of $\dfrac{\arctan x}{x}$)

Let, \begin{align*} \displaystyle A&=\int_0^1 \dfrac{x\arctan x\ln x}{1+x^2}dx\\ \displaystyle B&=\int_0^1 \dfrac{\ln x \ln(1+x^2)}{1+x^2}dx\\ \displaystyle C&=\int_0^1 \dfrac{\arctan x\ln x}{1+x}dx \end{align*}

Let $R$ the function defined on $[0;1]$ such that for all $x$ in $[0;1]$,

\begin{equation*} \displaystyle R(x)=\int_0^x \dfrac{\ln t}{1+t^2}dt=\int_0^1 \dfrac{x\ln(tx)}{1+t^2x^2}dt \end{equation*}

Let $\epsilon$ such that $0<\epsilon<1$. \begin{align*} \varphi(\epsilon)&=\int_0^{1-\epsilon}\dfrac{\ln x\ln(1-x)}{1+x^2}dx\\ &=\Big[\left(R(x)-R(1)\right)\ln(1-x)\Big]_0^{1-\epsilon}+\int_0^{1-\epsilon}\dfrac{\left(R(x)-R(1)\right)}{1-x}dx\\ &=\displaystyle \left(R(1-\epsilon)-R(1)\right)\ln(\epsilon)+R(1)\ln(\epsilon)+\int_0^{1-\epsilon}\dfrac{R(x)}{1-x}dx\\ &=\displaystyle \left(R(1-\epsilon)-R(1)\right)\ln(\epsilon)+R(1)\ln(\epsilon)+\int_0^{1-\epsilon} \left(\dfrac{x\ln(tx)}{(1+t^2x^2)(1-x)}dt\right)dx\\ &=\displaystyle \left(R(1-\epsilon)-R(1)\right)\ln(\epsilon)+R(1)\ln(\epsilon)+\int_0^{1-\epsilon} \left(\int_0^1 \dfrac{x\ln x}{(1+t^2x^2)(1-x)}dt\right)dx+\int_0^1 \left(\int_0^{1-\epsilon} \dfrac{x\ln t}{(1+t^2x^2)(1-x)}dx\right)dt\\ &=\displaystyle \left(R(1-\epsilon)-R(1)\right)\ln(\epsilon)+R(1)\ln(\epsilon)+\int_0^{1-\epsilon} \left[\dfrac{\ln x\arctan(tx)}{1-x}\right]_{t=0}^{t=1}dx+\\ &\int_0^1 \left[\dfrac{\ln t\ln(1+t^2x^2)}{2(1+t^2)}+\dfrac{t\ln t\arctan(tx)}{1+t^2}-\dfrac{\ln t\arctan(tx)}{t}-\dfrac{\ln t\ln(1-x)}{1+t^2}\right]_{x=0}^{x=1-\epsilon} dt\\ &=\displaystyle \left(R(1-\epsilon)-R(1)\right)\ln(\epsilon)+R(1)\ln(\epsilon)+\int_0^{1-\epsilon} \dfrac{\ln x\arctan(x)}{1-x}dx+\int_0^1 \dfrac{\ln t\ln\left(1+t^2(1-\epsilon)^2\right)}{2(1+t^2)}dt+\\ &\int_0^{1-\epsilon}\dfrac{t\ln t\arctan\left(t(1-\epsilon)\right)}{1+t^2}dt-\int_0^1\dfrac{\ln t\arctan\left(t(1-\epsilon)\right)}{t}dt-\int_0^1\dfrac{\ln t\ln \epsilon}{1+t^2}dt\\ &=\displaystyle \left(R(1-\epsilon)-R(1)\right)\ln(\epsilon)+\int_0^{1-\epsilon} \dfrac{\ln x\arctan(x)}{1-x}dx+\int_0^1 \dfrac{\ln t\ln\left(1+t^2(1-\epsilon)^2\right)}{2(1+t^2)}dt+\\ &\int_0^{1-\epsilon}\dfrac{t\ln t\arctan\left(t(1-\epsilon)\right)}{1+t^2}dt-\int_0^1\dfrac{\ln t\arctan\left(t(1-\epsilon)\right)}{t}dt \end{align*}

\begin{equation*} \displaystyle\lim_{\epsilon\rightarrow 0}\varphi(\epsilon)=\int_0^{1} \dfrac{\ln x\arctan(x)}{1-x}dx+\int_0^1 \dfrac{\ln t\ln\left(1+t^2\right)}{2(1+t^2)}dt+\int_0^{1}\dfrac{t\ln t\arctan t}{1+t^2}dt-\int_0^1\dfrac{\ln t\arctan t}{t}dt \end{equation*}

Thus,

\begin{equation} (1)\boxed{\displaystyle\int_0^{1}\dfrac{\ln x\ln(1-x)}{1+x^2}dx=A+\dfrac{1}{2}B+\beta(3)+\int_0^1\dfrac{\ln x\arctan x }{1-x}dx } \end{equation}

In the following integral apply the change of variable $y=\dfrac{1-x}{1+x}$,

\begin{align*} \displaystyle\int_0^1\dfrac{\ln x\arctan(x)}{1-x}dx&=\int_0^1 \dfrac{\left(\ln(1+x)-\ln(1-x)\right)\arctan\left(\dfrac{x-1}{x+1}\right)}{x}dx+\int_0^1 \dfrac{\left(\ln(1-x)-\ln(1+x)\right)\arctan\left(\dfrac{x-1}{x+1}\right)}{1+x}dx\\ &=\displaystyle \int_0^1 \dfrac{\ln(1+x)\arctan x}{x}dx-\int_0^1 \dfrac{\ln(1-x)\arctan x}{x}dx-\dfrac{\pi}{4}\int_0^1\dfrac{\ln\left(\tfrac{1+x}{1-x}\right)}{x}dx+\\ &\int_0^1\dfrac{\ln\left(\tfrac{1-x}{1+x}\right)\arctan\left(\tfrac{x-1}{x+1}\right)}{1+x}dx \end{align*}

and,

\begin{align*} \displaystyle \int_0^1 \dfrac{\ln(1+x)\arctan x}{x}dx&=\Big[\ln x\ln(1+x)\arctan x\Big]_0^1-\int_0^1\ln x\left(\dfrac{\ln(1+x)}{1+x^2}+\dfrac{\arctan x}{1+x}\right)dx\\ &=\displaystyle -\int_0^1 \dfrac{\ln x\ln(1+x) }{1+x^2}dx-C \end{align*}

\begin{align*} \displaystyle \int_0^1 \dfrac{\ln(1-x)\arctan x}{x}dx&=\Big[\ln x\ln(1-x)\arctan x\Big]_0^1-\int_0^1\ln x\left(\dfrac{\ln(1-x)}{1+x^2}-\dfrac{\arctan x}{1-x}\right)dx\\ &=\displaystyle \int_0^1 \dfrac{\ln x\arctan x }{1-x}dx-\int_0^1 \dfrac{\ln x\ln(1-x) }{1+x^2}dx \end{align*}

In the following integral apply the change of variable $y=\dfrac{1-x}{1+x}$,

\begin{align*} \displaystyle \int_0^1\dfrac{\ln\left(\tfrac{1+x}{1-x}\right)}{x}dx&=-2\int_0^1\dfrac{\ln x}{1-x^2}dx\\ &=\dfrac{\pi^2}{4} \end{align*}

In the following integral apply the change of variable $y=\dfrac{1-x}{1+x}$,

\begin{equation*} \displaystyle \int_0^1\dfrac{\ln\left(\tfrac{1-x}{1+x}\right)\arctan\left(\tfrac{x-1}{x+1}\right)}{1+x}dx=-C \end{equation*}

Thus,

\begin{equation*} (2)\boxed{\displaystyle \int_0^1\dfrac{\ln x\arctan(x)}{1-x}dx=\dfrac{1}{2}\int_0^1 \dfrac{\ln x\ln(1-x)}{1+x^2}dx-\dfrac{1}{2}\int_0^1 \dfrac{\ln x\ln(1+x)}{1+x^2}dx-C-\dfrac{\pi^3}{32}} \end{equation*}

Let $S$ the function defined on $[0;1]$ such that for all $x$ in $[0;1]$,,

\begin{align*} \displaystyle S(x)&=\int_0^x\dfrac{\ln x}{1+x^2}dt\\ &=\int_0^1\dfrac{x\ln(tx)}{1+t^2x^2}dt \end{align*}

Note that $S(1)=-G$, $G$ being the Catalan constant,

\begin{align*} \int_0^1\dfrac{\ln x\ln(1+x)}{1+x^2}dx&=\Big[S(x)\ln(1+x)\Big]_0^1-\int_0^1 \dfrac{S(x)}{1+x}dx\\ &=-G\ln 2-\int_0^1 \int_0^1 \dfrac{x\ln(tx)}{(1+t^2x^2)(1+x)}dtdx\\ &=-G\ln 2-\int_0^1 \int_0^1 \dfrac{x\ln x}{(1+t^2x^2)(1+x)}dtdx-\!\!\int_0^1 \int_0^1 \dfrac{x\ln t}{(1+t^2x^2)(1+x)}dtdx\\ &=-G\ln 2-\!\!\!\int_0^1\left[\dfrac{\ln x\arctan(tx)}{1+x}\right]_{t=0}^{t=1}dx-\\ &\int_0^1\!\!\left[\dfrac{\ln t\ln(1+x^2t^2)}{2(1+t^2)}\!-\!\!\dfrac{\ln t\ln(1+x)}{1+t^2}\!-\!\!\dfrac{t\ln t\arctan(tx)}{1+t^2}\!+\!\!\dfrac{\ln t\arctan(tx)}{t}\right]_{x=0}^{x=1}\!dt\\ &=-G\ln 2-\int_0^1\dfrac{\ln x\arctan x}{1+x}dx-\dfrac{1}{2}\int_0^1\dfrac{\ln t\ln(1+t^2)}{1+t^2}dt+\\ &\ln 2\int_0^1\dfrac{\ln t}{1+t^2}dt+\int_0^1\dfrac{t\ln t\arctan t}{1+t^2}dt-\int_0^1\dfrac{\ln t\arctan t}{t}dt\\ \end{align*}

Thus,

\begin{equation*} (3)\boxed{\displaystyle \int_0^1\dfrac{\ln x\ln(1+x)}{1+x^2}dx=A-\dfrac{1}{2}B-C-2G\ln 2+\beta(3)} \end{equation*}

Let $T$ the function defined on $[0;1]$ such that for all $x$ in $[0;1]$,

\begin{align*} \displaystyle T(y)&=\int_0^y \dfrac{t\ln(t)}{1+t^2}dt\\ &=\displaystyle\int_0^1 \dfrac{ty^2\ln(ty)}{1+t^2y^2}dt \end{align*}

Note that $T(1)=-\dfrac{\pi^2}{48}$

\begin{align*} \displaystyle A&=\Big[T(y)\arctan y\Big]_0^1-\int_0^1 \dfrac{S(y)}{1+y^2}dy\\ &=\displaystyle-\dfrac{\pi^3}{192}-\int_0^1\int_0^1\dfrac{ty^2\ln(ty)}{(1+t^2y^2)(1+y^2)}dtdy\\ \displaystyle &=-\dfrac{\pi^3}{192}-\int_0^1\int_0^1\dfrac{ty^2\ln y }{(1+t^2y^2)(1+y^2)}dtdy-\int_0^1\int_0^1\dfrac{ty^2\ln t }{(1+t^2y^2)(1+y^2)}dtdy\\ \displaystyle &=-\dfrac{\pi^3}{192}-\dfrac{1}{2}\int_0^1\left[\dfrac{\ln y \ln(1+t^2y^2)}{1+y^2}\right]_{t=0}^{t=1}dy-\\ &\dfrac{1}{2}\int_0^1 \left[\dfrac{\ln t \arctan y+\ln t\arctan(ty)}{1+t}-\dfrac{\ln t \arctan(ty)-\ln t\arctan y}{t-1}\right]_{y=0}^{y=1}dt \end{align*}

Thus, \begin{align*} (4)\boxed{\displaystyle A=-\dfrac{1}{64}\pi^3-\dfrac{1}{2}B-\dfrac{1}{2}C-\dfrac{1}{2}\int_0^1\dfrac{\ln x\arctan x}{1-x}dx} \end{align*}

Plug $\displaystyle \int_0^{1}\dfrac{\ln x\ln(1-x)}{1+x^2}dx$ from (1) and $\displaystyle \int_0^{1}\dfrac{\ln x\ln(1+x)}{1+x^2}dx$ from (4) into (2),

Thus,

\begin{align*} (5)\boxed{\displaystyle \int_0^1\dfrac{\ln x\arctan x}{1-x}dx=B-C+2G\ln 2-\dfrac{1}{16}\pi^3} \end{align*}

Plug $\displaystyle \int_0^1\dfrac{\ln x\arctan x}{1-x}dx$ from (5) into (4)

Thus,

\begin{align*} (6)\boxed{A=\dfrac{1}{64}\pi^3-B-G\ln 2} \end{align*}

In the following integral apply the change of variable $y=\arctan x$,

\begin{align*} (7) \boxed{\displaystyle \int_0^{\pi/4} \ln(\tan x)\ln(\cos x-\sin x)dx=\int_0^1\dfrac{\ln x\ln(1-x)}{1+x^2}dx-\dfrac{B}{2}} \end{align*}

and, from (1), it follows that,

\begin{align*} (8) \boxed{\displaystyle \int_0^{\pi/4} \ln(\tan x)\ln(\cos x-\sin x)dx=A+\beta(3)-\int_0^1\dfrac{\ln x\arctan x}{1-x}dx} \end{align*}

Plug $A$ from (6) and $\displaystyle \int_0^1\dfrac{\ln \arctan x}{1-x}dx$ from (5) into (7),

\begin{align*} (8) \boxed{\displaystyle \int_0^{\pi/4} \ln(\tan x)\ln(\cos x-\sin x)dx=-\dfrac{3}{64}\pi^3+G\ln 2+\beta(3)-C} \end{align*}

$C$ have been already evaluated (see Evaluating $\int_0^1 \frac{\arctan x \log x}{1+x}dx$ )

\begin{equation} \boxed{\displaystyle C=\dfrac{G\ln 2}{2}-\dfrac{\pi^3}{64}} \end{equation}

and, knowing that,

\begin{equation} \beta(3)=\dfrac{\pi^3}{32} \end{equation}

Therefore,

\begin{equation} \boxed{\displaystyle \int_0^{\pi/4} \ln(\tan x)\ln(\cos x-\sin x)dx=\dfrac{G\ln 2}{2}} \end{equation}

Solution 3:

\begin{align} I&=\int_0^1 \dfrac{\ln x\ln(1-x)}{1+x^2}dx-\dfrac{1}{2}\int_0^1 \dfrac{\ln x\ln(1+x^2)}{1+x^2}dx \end{align} For the first integral, We proved here:

$$\int_0^1\frac{\ln x\ln(1-x)}{1+x^2}\ dx=\frac{3\pi^3}{64}+\frac{\pi}{16}\ln^22-\Im\operatorname{Li}_3(1+i)$$

and for the second integral, we proved here:

$$\int_0^1\frac{\ln x\ln(1+x^2)}{1+x^2}\ dx=\frac3{32}\pi^3+\frac{\pi}8\ln^22-\ln2~G-2\Im\operatorname{Li_3}(1+i)$$

and by combining these two results, we get $\ \displaystyle I=\frac12G\ln2$

Solution 4:

Denote $$K= \int _0^1 \frac{\tan^{-1}x \ln x}{1-x^2}dx $$ Note $\begin{align} \int _0^1\frac{\ln x \ln \frac{1-x}{1+x}}{1+x^2}dx &= \int _0^1 \ln x \ln \frac{1-x}{1+x}\> d(\tan^{-1}x) \overset{IBP}= 2K - 2\int _0^1\overset{x\to \frac{1-x}{1+x}} {\frac{\tan^{-1}x \ln \frac{1-x}{1+x}}{x}}dx \\& =2K - 2\int _0^1\frac{(\frac\pi4-\tan^{-1}x ){\ln x}}{1-x^2}dx =4K +\frac{\pi^3}{16} \tag1 \end{align}$ $\begin{align} & \int _0^1\frac{\ln x \ln \frac{1-x}{1+x}}{1+x^2}dx= \int _0^1 \ln \frac{1-x}{1+x}\> d \left(\int_1^x \frac{\ln t}{1+t^2}dt\right)\\ &\hspace{-9mm}\overset{IBP}= \int _0^1 \frac{2}{1-x^2}\left( \int_0^x \right. \overset{t=xs }{\left. \frac{\ln t}{1+t^2}dt+G\right)}dx =\int_0^1 \frac2{1-x^2}\left( \int_0^1 \frac{x\ln x+x\ln s}{1+x^2s^2}ds +G \right)dx \\& \hspace{-9mm} =2\int_0^1 \left(\int_0^1 \frac{x\ln x \> ds }{(1-x^2)(1+x^2s^2)}+ \int_0^1 \frac{\ln s \>ds }{1+s^2} \left(\frac{xs^2}{1+x^2s^2} +\frac x{1-x^2} \right) +\frac G{1-x^2} \right)dx \\& \hspace{-9mm} =2 \int_0^1 \frac{\ln x }{1-x^2}\left(\int_0^1 \frac{x\> ds }{1+x^2s^2}\right)dx + 2 \int_0^1 \frac{\ln s }{1+s^2}\left(\int_0^1 \frac{xs^2\> dx }{1+x^2s^2}\right)ds \\ &+2 \int_0^1 \left(-\frac {Gx}{1-x^2}+ \frac G{1-x^2} \right)dx = \int _0^1\frac{\ln s\ln \left(1+s^2\right)}{1+s^2}ds +2K +2G\ln2 \tag2 \end{align}$

Combine (1) and (2) $$\int _0^1\frac{\ln s\ln \left(1+s^2\right)}{1+s^2}ds= 2 K -2G\ln 2 + \frac{\pi^3}{16}\tag3$$ Evaluate \begin{align} K=& \int _0^1 \tan^{-1}x\> d \left(\int_0^x \frac{\ln t}{1-t^2}dt\right)\overset{IBP}=-\frac{\pi^3}{32}- \int_0^1 \frac{dx }{1+x^2}\int_0^x \overset{t=xs} {\frac{\ln t}{1-t^2}}dt \\ =&-\frac{\pi^3}{32}- \int_0^1\int_0^1\frac{ x\ln s }{(1+x^2)(1-x^2s^2)}dsdx - \int_0^1\int_0^1\overset{x\leftrightarrow s} {\frac{x\ln x }{(1+x^2)(1-x^2s^2)}}dsdx\\ =&-\frac{\pi^3}{32} -\int_0^1 \frac{\ln s\>ds}{1+s^2} \int_0^1 \left(\frac x{1+x^2}+\frac{xs^2}{1-x^2s^2} \right) dx - \int_0^1 \int_0^1 { \frac{s\ln s \>dsdx }{(1+s^2)(1-x^2s^2)} }\\ = &-\frac{\pi^3}{32} -\int_0^1 \frac{\ln s}{1+s^2}ds \int_0^1 \frac {x}{1+x^2}dx -\int_0^1 \frac{\ln s}{1+s^2}\left( \int_0^1 \frac {s}{1-xs}dx\right)ds \\ =& -\frac{\pi^3}{32} +\frac12G\ln2 + \int_0^1 \frac{\ln s\ln(1-s)}{1+s^2}ds \end{align} Substitute above expression into (3) to obtain $$ \int_0^1 \frac{\ln s\ln \frac{(1-s)^2}{1+s^2}}{1+s^2}ds = G\ln 2$$ which, via $s=\tan t$, leads to $$\int_0^{\pi/4} \ln(\tan t)\ln(\cos t-\sin t)dt=\frac12G\ln 2$$