If $abc=1$ then $ \sum_{cyc} \frac{a^2\sqrt{3bc}}{\sqrt{2b^2-bc+2c^2}} \ge 3 $

Initial manipulation

Remove a $\sqrt 3$: It's enough to prove : $$ \sum_{cyc} \frac{a^2\sqrt{bc}}{\sqrt{2b^2+2c^2-bc}} \geq \sqrt 3 $$

after this, notice that $\sqrt{bc} = \frac 1{\sqrt a}$. So we use this fact above to get the equivalent inequality : $$ \sum_{cyc} \frac{a^2}{\sqrt{2b^2a+2c^2a-1}} \geq \sqrt 3 $$

Hölder

Well, CS/AM-GM may work or not , but I've seen, more often than not, the use of the generalized Hölder's inequality in this situation very often : where you have something like $\sum_{cyc} \frac{f(a,b,c)}{\sqrt{g(a,b,c)}}$ and you want to get rid of the square root and the denominator.

It fits really well for expressions of the form $\sum_{cyc} \frac{f(a,b,c)}{\sqrt{g(a,b,c)}}$. In our case, we use it to a higher degree, because the simplest use does not work on this occasion(but works for a lot of other problems : which is what makes it quite usable for simpler Hölder-inequality illustrations). We use a five-product Holder inequality where four of the terms are the same, and the fifth is used as a "cancellation" term. Let the term $\frac{a^2}{\sqrt{2b^2a+2c^2a-1}}$ appear four times. To "cancel" the denominator, we use the term $a^2(2b^2a+2c^2a-1)^2$. We have the cyclic counterparts with which the respective cyclic sums can form.

Applying the Hölder ​inequality gives : $$ \left[\sum_{cyc} \frac{a^2}{\sqrt{2b^2a+2c^2a-1}}\right]^4 \left[\sum_{cyc} a^2(2b^2a+2c^2a-1)^2\right] \geq (a^2+b^2+c^2)^5 $$

Therefore , it is sufficient to prove that $$ (a^2+b^2+c^2)^5 \geq 9 \left[\sum_{cyc} a^2(2b^2a+2c^2a-1)^2\right] \tag{1} $$

since combining this with the previous inequality would give : $$ \left[\sum_{cyc} \frac{a^2}{\sqrt{2b^2a+2c^2a-1}}\right]^4 \left[\sum_{cyc} a^2(2b^2a+2c^2a-1)^2\right] \geq 9 \left[\sum_{cyc} a^2(2b^2a+2c^2a-1)^2\right] $$

and you can cancel the second term and take the fourth root.

Introduction to Muirhead

The inequality above needs to be expanded first. This will involve a lot of symmetry, and the eventual simplification will involve the Muirhead inequality. We will stick to three variables : it can be generalized.

Define, for a three dimensional vector $[v_1,v_2,v_3]$ of positive real number entries, the symmetric sum $$ T[v_1,v_2,v_3] = \sum_{\sigma \in S_3} a^{v_{\sigma(1)}}b^{v_{\sigma(2)}}c^{v_{\sigma(3)}} $$

So for example,$$T[3,2,1] = a^1b^2c^3+a^1c^3b^2+a^2b^1c^3+a^2b^3c^1+a^3b^1c^2+a^3b^2c^1$$

And there can be repetitions e.g. $$T[3,0,0] = a^3b^0c^0+a^3b^0c^0+a^0b^3c^0+a^0b^3c^0+a^0b^0c^3+a^0b^0c^3 = 2(abc^3+ab^3c+a^3bc)$$

Muirhead's inequality states :

Suppose $[v_1,v_2,v_3]$ and $[w_1,w_2,w_3]$ are positive real number entries of two vectors, such that$$ v_1 \geq w_1,v_1+v_2 \geq w_1+w_2,v_1+v_2+v_3 = w_1+w_2+w_3$$ Then we have $T[v_1,v_2,v_3] \geq T[w_1,w_2,w_3]$.

We also observe , because $abc = 1$, that for all $p,q,r$ positive and $s \leq p,s \leq q,s \leq r$, we have $T[p,q,r] = T[p-s,q-s,r-s]$. This observation is paramount.

Final flourish

We now expand both sides of the inequality $(1)$ : both via the multinomial theorem. I can expand the working if necessary, but it's a little tedious (and gets better with practice). Following that, one then uses the fact that $abc=1$ at various points to reduce the indices of the vectors involved. In the end , after transposing the last term of the RHS to the LHS so that both sides consist of linear combinations of symmetric sums, you land up with the inequality : $$ \frac 12 T[10,0,0] + 5 T[8,2,0]+10T[6,4,0]+10T[4,0,0]+15T[2,2,0]+36T[3,2,0] \\ \geq 36 T[2,2,0] + 36T[0,0,0] + \frac 92 T[2,0,0] $$

(Note : $T[0,0,0] = 6$, I'm just writing it so I can use Muirhead on it). Now, the tightness of this inequality can actually be verified when you set $a=b=c=1$, so that each $T[x,y,z]$ equals $6$. Once the six cancels out on both sides, the coefficients on both sides add up to $76.5$, and that's how one verifies that the inequality is probably correct.

Once we do this, we now begin "Muirheading". Let's start with the "worst" Muirhead term (this is the term where the coefficients are as far from each other : that's the kind of term that's difficult to dominate). In this case, that'd be $[2,2,0]$.

We begin by cancelling the $T[2,2,0]$ terms out on both sides :$$ \frac 12 T[10,0,0] + 5 T[8,2,0]+10T[6,4,0]+10T[4,0,0]+36T[3,2,0] \\ \geq 21 T[2,2,0] + 36T[0,0,0] + \frac 92 T[2,0,0] $$

Then we notice that$$ T[3,2,0] \geq T\left[\frac{7}{3},\frac{7}{3},\frac 13\right] = T[2,2,0] $$

So using that gives : $$ \frac 12 T[10,0,0] + 5 T[8,2,0]+10T[6,4,0]+10T[4,0,0]+15T[3,2,0] \\ \geq 36T[0,0,0] + \frac 92 T[2,0,0] $$

We can now get rid of the $T[2,0,0]$ by using $$ T[3,2,0] \geq T[3,1,1] = T[2,0,0] $$

So that leaves us with : $$ \frac 12 T[10,0,0] + 5 T[8,2,0]+10T[6,4,0]+10T[4,0,0]+\frac{21}{2}T[3,2,0] \\ \geq 36T[0,0,0] $$

Now, it's enough to show that each $T$ term is bigger than $T[0,0,0]$, because the coefficients will do the rest of the job. That's quite easy to see (the first inequality is true for any of the three terms being on the LHS): $$ T[6,4,0],T[8,2,0],T[10,0,0] \geq T\left[\frac{10}{3},\frac{10}{3},\frac{10}{3}\right] = T[0,0,0] \\ T[4,0,0] \geq T\left[\frac{4}{3},\frac{4}{3},\frac{4}{3}\right] = T[0,0,0]\\ T[3,2,0] \geq T\left[\frac{5}{3},\frac{5}{3},\frac{5}{3}\right] = T[0,0,0] $$

which finishes the proof. I hope there's a better proof around.

Holder in another way:

Using Holder, we have $$\left(\sum_{\mathrm{cyc}} \frac{a^2\sqrt{3bc}}{\sqrt{2b^2 - bc + 2c^2}}\right)^2 \sum_{\mathrm{cyc}} 9a^3(2b^2 - bc + 2c^2) \ge (3a^2 + 3b^2 + 3c^2)^3.$$ It suffices to prove that $$(3a^2 + 3b^2 + 3c^2)^3 \ge 9 \sum_{\mathrm{cyc}} 9a^3(2b^2 - bc + 2c^2). \tag{1}$$

Hope to see a nice proof of (1). I have a complicated proof using the pqr method.

Let $p = a + b + c, q = ab + bc + ca, r = abc = 1$. We have $p\ge 3$ and $p^2 \ge 3q$.

(1) is written as $$27(p^2 - 2q)^3 \ge 81 p(2q^2 - 5pr)$$ or $$27(p^2 - 2q)^3 \ge 81 p(2q^2 - 5p).$$

Using Schur's inequality $a^2(a - b)(a - c) + b^2(b - c)(b - a) + c^2(c - a)(c - b) \ge 0$ which is written as $p^4 - 5p^2q + 6pr + 4q^2 \ge 0$, we have $p^4 - 5p^2q + 6p + 4q^2 \ge 0$ or $\frac{1}{16}(5p^2 - 8q)^2\ge \frac{9p^4 - 96p}{16}$ which results in (using $p\ge 3$ and $p^2 \ge 3q$) $$q \le \frac{5p^2 - \sqrt{9p^4 - 96p}}{8}.$$

Since $p^2 - 2\cdot \frac{5p^2 - \sqrt{9p^4 - 96p}}{8} = \frac{\sqrt{9p^4 - 96p} - p^2}{4} > 0$, it suffices to prove that $$27\left(p^2 - 2\cdot \frac{5p^2 - \sqrt{9p^4 - 96p}}{8}\right)^3 \ge 81 p\left(2\cdot \left(\frac{5p^2 - \sqrt{9p^4 - 96p}}{8}\right)^2 - 5p\right)$$ or $$(81p^4 + 405p^3 - 648p)\sqrt{9p^4 - 96p} \ge 189p^6 + 1377p^5 - 1944p^3 - 10368p^2.$$

Since $81p^4 + 405p^3 - 648p > 0$, it suffices to prove that $$(81p^4 + 405p^3 - 648p)^2(9p^4 - 96p) \ge (189p^6 + 1377p^5 - 1944p^3 - 10368p^2)^2$$ or $$23328p^3(p - 3)(p^8 + 6p^7 - 36p^5 - 183p^4 + 432p^2 + 1728p + 576) \ge 0$$ which is true (using $p \ge 3$).

We are done.