Pure maths vs applied

Is there any point in an aspiring theoretical physicist doing pure math topics such as analysis? (Assuming that he would not be doing them out of pure interest.)

As a physicist who completed an undergraduate math major, I'd have to say yes. Of all the reasons I can give, the most important is that it makes you a more competent physicist. Simply put, the physics professors may not be able to explain why they do things, but the math professors can and do. (And, force you to understand it, too.) Also, the additional perspective on the different techniques gives you a better understanding of them.

I honestly don't recall real analysis being immediately useful in physics. But, it gives you a good base for the more advanced ideas, like calculus of variations, and the extra practice doesn't hurt. However, if the opportunity presents itself, take complex analysis. First of all, contour integration is extremely useful on its own. Second, all of the most advanced ideas rely on it. For instance, the poles of a Green's function determines particle energy and lifetime.

Abstract algebra, on the other hand, is extremely useful as it gives you an intimate understanding of the role of symmetry. For instance, if you have a quantum mechanical system with a known symmetry, you can partition your Hamiltonian according to the representations of that symmetry as they don't interact with each other. Perturbations on top of that will either couple different representations, or split previously degenerate levels (multi-dimensional representation). For instance, in a system with cubic symmetry, the a $p_x$, $p_y$, and $p_z$ orbitals are degenerate, but if it undergoes a rectangular distortion, i.e. the z-axes is lengthened, the $p_z$ orbital splits from the other two orbitals. This is immediately determined from the character tables of the two symmetry groups. And, don't forget that all constants of motion are due to symmetry.

Edit: Some additional things came to mind. Most (if not all) partial differential equations encountered in physics are of the Sturm-Liouville type, so their solutions form a complete basis over a vector (Hilbert) space. This ties in linear algebra rather intimately into most advanced physics. (Operator theory in particular comes to mind.) Since Hilbert spaces are infinite dimensional, the ideas of convergence, which aren't important in a finite vector space, become essential, and weren't discussed in my undergrad (or grad, for that matter) physics courses. But, definitely worth paying attention to in an analysis class.

I'm not sure what sort of theoretical physics you are considering, but the answer doesn't depend too much on the particular field.

Most often, at least in my immediate vicinity, the terms theoretical physics are used to refer mostly to quantum theory, scattering theory (and other particle physics), something relativistic in nature, or something of the like. In all of these, the tools of analysis are absolutely essential. What would quantum or particle interaction analysis be without partial differential equations or advanced techniques from analysis (like fourier analysis, harmonic analysis, complex analysis)?

One who has looked into quantum mechanics or quantum field theory (the areas of theoretical physics with which I am most familiar) has almost certainly run into large amounts of high math. Lie algebras (a very interesting part of abstract algebra, important in particular because of the structurally sound non-commutative sort, often branded quantum groups) play an important role in the prediction of different particles. The unification of different theories is often accompanied by a proposed algebraic structure. Doing a quick arxiv search, I present a paper with an abstract that appears to demonstrate the incredible amount of pure math in theoretical physics.

I will mention a few more topics that have big roles too. Complex analysis and, in particular, contour integration have played a big role. This was one of Feynman's big contributions to the field (one of many, of course). Lp theory and spaces and the relationships of different norms can have a big impact on many areas of physics. I first came across Plancheral's Theorem (or Equality, Relationship, or many of the other things it has been called) in an undergraduate quantum class, not a math class.

In short, I think that many people do not distinguish between high level theoretical physics and high level math. In some ways, I agree - the idea that pure math is required for physics, for example. The motivation is very different, I think, but then again I continue to learn more and keep up with quantum theory, even though I am aspiring to become an algebraist or number theorist (which really just means that most of what I do will not, unfortunately, be useful to theoretical physics).

That's my two cents.