Who is a Math Historian?

First of all, as far as I know, serious historians of mathematics are or were in their majority mathematicians: the reason is that mathematics is very difficult and you can't analyze it in depth without a very serious technical background.
There might be exceptions for very ancient mathematics, but even there I wouldn't trust a historian studying Diophantus who wouldn't have some knowledge of arithmetic/algebraic geometry and number theory.

What often happens is that aging mathematicians start writing about the history of the subject they have devoted their life to.
Prestigious examples are for example:
Weil on number theory,
Dieudonné and his wonderful histories of algebraic geometry and algebraic topology,
Marcel Berger on differential geometry,
Dickson and his monumental history of the theory of numbers.

Younger mathematicians may also be interested :
Bourbaki has very nice historical surveys at the end of some of his chapters, written at the time by necessarily young members (there was an age limit for participants)
Schappacher is an excellent research mathematician who already as a young researcher wrote about the history of number theory,
Krömer has written a great thesis on the genesis of category theory (including the incredible beginnings of sheaf theory in a prisoner of war camp ) ,
and to finish on a personal note, here is the fairly recent thesis on the birth of group cohomology by Nicolas Babois, whom I taught at the undergraduate level (but I had no rôle in his thesis).

In conclusion, my point of view is that a historian of mathematics is essentially a mathematician, and historical science in the usual sense is of secondary importance.
This is certainly controversial.
My convictions on this subject essentially derive from Dieudonné's and Houzel's points of view. (Houzel is an other example of a mathematician with high technical skills attracted very early by the history of mathematics)

It depends on the historian. Most historians of mathematics probably have deep associations with mathematics in some way, but being a mathematician it is not required any more than a sports historian should be a professional athelete. But surely the more mathematics one knows the more one tends to appreciate its history.