Comparing sums of surds without any aids
Solution 1:
This is answer is incomplete it only handles the case of six surds.
For six surds there is still way to do it in general using squaring. Suppose you want to check if the inequality : $$ \sqrt{a} + \sqrt{b} + \sqrt{c} < \sqrt{a'} + \sqrt{b'} + \sqrt{c'} $$ is true. Squaring both sides and letting $$ A = bc,\quad B=ac,\quad C=ab,\quad K = a^2+b^2+c^2,\quad \dots $$ we get an expression of the form: $$ K + 2\sqrt{A} + 2\sqrt{B} + 2\sqrt{C} < K' + 2\sqrt{A'} + 2\sqrt{B'} + 2\sqrt{C'}\quad\quad (*)$$ now square again, you get another expression of the same shape $$ K_1 + (4K+2a) \sqrt{A} + (4K+2b)\sqrt{B} + (4K+2c)\sqrt{C} <\\ K'_1 + (4K'+2a') \sqrt{A'} + (4K'+2b')\sqrt{B'} + (4K'+2c')\sqrt{C'} \quad\quad(**)$$
Now if $X,Y$ and $\theta$ are positive numbers and $X \ge \theta /2$, $Y \ge \theta /2$ then as it is easy to check $$ X^2-\theta X < Y^2 - \theta X \quad\quad \text{if and only if}\quad\quad X < Y $$ letting $X$ equal to the left hand side of (*), $Y$ equal to the right hand side and $\theta = 2K+a$, (assuming $a$ is the smallest of $a,b,c,a',b',c'$) as we have trivially $$ X = K + 2 \sqrt{A} + \dots > \frac{2K+a}{2} = \frac{\theta}2 $$ and the same for $Y$, we can subtract $2K+a$ times (*) from (**) and obtain an inequality equivalent to the original but with one surd less.
With some care I think you can manage to extend this to positive and negative coefficients. But I can't see how to extend it in general for larger number of surds. For example if you have four surds in one side then squaring twice you get at most seven surds $$ \sqrt{ab}, \sqrt{ac}, \sqrt{ad}, \sqrt{bc}, \sqrt{bd}, \sqrt{cd},\sqrt{abcd} $$ if you manage somehow to reduce the number of surds but leave more than three surds, then squaring again a couple of times recovers all the surds, so you need to reduce four surds in a single blow.
By the way in your example squaring the inequality we see that it is equivalent to $$ 20 + 2 \sqrt{10} + 2\sqrt{26}+2\sqrt{65} < 21 + 2 \sqrt{21}+2\sqrt{33}+2\sqrt{77} $$ in this case you have finished as every term in the left is smaller than the corresponding term in the right.
Solution 2:
Robert Israel's answer in my question About rationalizing expressions is the the brute-force, but always-working method to do it. Although it is of exponential complexity, it shows the possibility of a finite-time algorithm besides calculation.