Why do we still do symbolic math?

Try to solve $x^{1/3} = 0$ with a "symbolic calculation". Now try to solve $x^{1/3} = 0$ with Newton's Method. The point is numerical methods can fail and anyone using numerical methods should have an understanding of mathematics so that they can detect when a numerical method is not working and perhaps even fix the problem. Also as this case shows sometimes the symbolic calculation is easier.

All these computational methods need to be designed and implemented. It takes an understanding of the underlying mathematics to do this.

$x^2 = 2$ doesn't have a solution in the rationals. So what could you possibly mean by it having a numerical solution?

Do you mean that you can produce a rational numbers whose squares are arbitrarily close to $2$? Congratulations: your notion of "having a numerical solution" is simply reinventing the notion of "real number", just in a form that is much more cumbersome to think about and manipulate than usual.

If there was no symbolic math, we would still be living in caves and eating roots.

Formulas have meaning:

$$F=m\cdot a$$ is enough to explain a great deal of the universe.

Numbers are uninformative and unstructured: $$41.04=7.54\cdot5.443$$ tells you about nothing.

Real numbers have been invented because they model very well our intuition of a continuum, like points in space, and they give rise to billions of interesting topics.

Numerical values are essentially integers, desperately trying to mimic the former.

Symbolic solutions to a problem are compact and show patterns that can be handled by the human mind.

Numerical solutions are huge and as dumb as computers.

In addition to what others are saying: Many mathematical results, and even entire branches of math, can't be done numerically because they're about objects that are much more general than numbers. Group theory and topology spring to mind as examples.

This is a great question, and I like it especially since I'm a student of computational science and so spend my days around numerical software and approximations.

Symbolic math exists for the same reason that not all physics, chemistry, biology, economy etc. can be reduced to one large computer program. We still need to think and reason about problems. Even for pure math, this is no exception.

In a way, your queston answers itself. Numerical methods can be great for lots of problems, but none of them are perfect. There is continuing ongoing development for new ones, and attempts to develop efficient numerical algorithms for problems that can not yet be solved numerically. However, numerical software cannot develop new numerical software. And you cannot numerically prove that certain numerical algorithms are stable, or show how efficient they are, or why one is better than the other in certain instances. This entire research in numerical methods themselves has to be done symbolically.

(Oh, and of course, there is a lot of math which is not based around computation and cannot be done numerically in any way.)

Also: As another answer points out, some problems can be solved more easily symbolically than with a complicated numerical method. The mathematical approach can often yield more information, more insight, than just the answer.