(Elegant) proof of an inequality: $h(x) \geq 1- (1-\frac{x}{1-x})^2$, where $h$ is the binary entropy function

I am looking for the most concise and elegant proof of the following inequality: $$ h(x) \geq 1- \left(1-\frac{x}{1-x}\right)^2, \qquad \forall x\in(0,1) $$ where $h(x) = x \log_2\frac{1}{x}+(1-x) \log_2\frac{1}{1-x}$ is the binary entropy function. Below is a graph of the two functions.

Of course, an option would be to differentiate $1,2,\dots,k$ times, and study the function this way — it may very well work, but is not only computationally cumbersome, it also feels utterly inelegant. (For my purposes, I could go this way, but I'd rather not.)

I am looking for a clever or neat argument involving concavity, Taylor expansion around $1/2$, or anything — an approach that would qualify as "proof from the Book."

I do not think that this is elegant enough.

Considering $$ h(x) = x \log_2\frac{1}{x}+(1-x) \log_2\frac{1}{1-x}$$ $$g(x)=1- \left(1-\frac{x}{1-x}\right)^2$$ $$f(x)=h(x)-g(x)$$ Expanding $f(x)$ as a Taylor series built at $x=\frac 12$, the result is $$f(x)= \left(16-\frac{2}{\log (2)}\right)\left(x-\frac{1}{2}\right)^2+64 \left(x-\frac{1}{2}\right)^3+ O\left(\left(x-\frac{1}{2}\right)^4\right)$$

Hopefully, this is right!

Note that from the weighted AM-GM inequality, We have that $$h(x)=\log_2{\frac{1}{x^x(1-x)^{1-x}}} \ge \log_2\frac{1}{x^2+(1-x)^2}$$ Thus we have to show $$\left(1-\frac{x}{1-x}\right)^2 \ge 1-\log_2\frac{1}{2x^2-2x+1}=\log_2{(4x^2-4x+2)}$$ Substitute $x=\frac{a+1}{a+2}$, and we have $$f(a)=a^2 -\log_2\left(\frac{a^2}{(a+2)^2}+1 \right) \ge 0$$ For $a \ge -1$. Differentiating gives $$f'(a)=2a\left(1-\frac{1}{(a+2) (a^2+2 a+2) \log(2)}\right)$$ and this $f'(a)>0$ for $a>0$ alternatively $f(a) \ge f(0)=0$ for $a \ge 0$.

Also, notice the local maxima lies between $-1$ and $0$. But since $f(-1)=f(0)=0$, we have that $f(a) \ge 0$ for $-1 \le a \le 0$.

Our proof is done.

As noted in my comment, it suffices to show the inequality for $x\in[0,\frac{1}{2}]$. Note that the inequality becomes an equality at the endpoints, $0$ and $\frac{1}{2}$. The inequality is tighter around $\frac{1}{2}$ than around $0$. In our proof, we will distinguish two (overlapping) cases, $x$ near $0$ or $x$ near $\frac{1}{2}$. When $x$ is near $\frac{1}{2}$, we Taylor-expand the logs at $\frac{1}{2}$. When $x$ is near $0$, we use cruder (constant, in fact) bounds on the logs.

We have to show that

$$ x\log(x)+(1-x)\log(1-x) \leq \log(2)\left(\frac{3x^2-2x}{(1-x)^2}\right) \tag{1} $$

As $\frac{\log(1-x)-\log(\frac{1}{2})}{\frac{1}{2}-x}\leq 2 \leq \frac{\log(\frac{1}{2})-\log(x)}{\frac{1}{2}-x}$, we have $\log(x) \leq (-1-\log(2))+2x$ and $\log(1-x) \leq (1-\log(2))-2x$, so (1) will be true whenever

$$ x\left[(-1-\log(2))+2x\right]+(1-x)\left[(1-\log(2))+2x\right] \leq \log(2)\left(\frac{3x^2-2x}{(1-x)^2}\right) \tag{2} $$

By construction, (2) is simplifiable by $(x-\frac{1}{2})^2$ and a little cleanup massaging shows that (2) is equivalent to $(1-x)^2 \leq \log(2)$. This shows (1) for $x\geq 1-\sqrt{\log(2)}$. Note that the number $1-\sqrt{\log(2)} \approx 0.167$ is strictly less than $0.2$.

Now, let us deal with the case when $x\leq 0.2$. Then $\log(x)\leq\log(0.2)$ and $\log(1-x)\leq 0$, so that it (1) is true whenever $$ x\log(0.2) \leq \log(2)\left(\frac{3x^2-2x}{(1-x)^2}\right) \tag{3} $$

Clearly, (3) is equivalent to $$ \frac{\log(0.2)}{\log(2)} \leq \frac{3x-2}{(1-x)^2} $$ Now, the RHS can be rewritten $-\frac{35}{16}+\frac{35(\frac{1}{5}-x)(\frac{3}{7}-x)}{16(1-x)^2}$ and we conclude the proof by noting that $\frac{\log(0.2)}{\log(2)} < -\frac{35}{16}$ because $\frac{\log(0.2)}{\log(2)} \approx -2.32$ and $-\frac{35}{16} \approx -2.18$.

$$h(x) = x \log_2\frac{1}{x}+(1-x) \log_2\frac{1}{1-x}\ge1- \left(1-\frac{x}{1-x}\right)^2$$

If we let $x=\frac{1+y}{2}$ and push through the algebra, the claim is equivalent to:

$$1-\frac{1}{2}\log_2(1-y^2)-\frac{y}{2}\log_2\left(\frac{1+y}{1-y}\right)\ge1-\frac{4y^2}{(1-y)^2}$$

which can be rearranged to:

$$\frac{8y^2}{(1-y)^2}\ge \log_2(1-y^2) + y\log_2\left(\frac{1+y}{1-y}\right)$$

Now, using the well-known $\ln u\le u-1$:

$$\ln v^2\le v^2-1\implies \ln v \le \frac{v^2-1}{2}$$

and taking $v=\frac{1+y}{1-y}$, this becomes:

$$\ln\left(\frac{1+y}{1-y}\right)\le \frac{2y}{(1-y)^2}$$

Noting that $\log_2(1-y^2)\le0$, we can see

$$\log_2(1-y^2) + y\log_2\left(\frac{1+y}{1-y}\right)\le y\frac{1}{\ln2}\frac{2y}{(1-y)^2}=\frac{2}{\ln2}\frac{y^2}{(1-y)^2}\le \frac{8y^2}{(1-y)^2}$$

and we are done for $y>0$.

For $y<0$ it's a little trickier, because the other $\log$ term does come into play - still working out a nice argument for how to bound things appropriately.