When to omit an article

We are discussing the sentence "We can obtain the result by analytic inversion." It's from a math context.

1. Is this an OK sentence, or does it have to be "an inversion"? If so, why? If not, why? Is this sentence particularly problematic because "inversion" can have both a non-countable (the process of inverting) and a countable (if you have different ways of inverting) interpretation?

2. If this is an OK sentence, why is the sentence so much different if you replace "inversion" by "algorithm"? Clearly, in this case, you do need "an".

You've more or less answered your own question:

"inversion" can have both a non-countable (the process of inverting) and a countable (if you have different ways of inverting) interpretation

With the first interpretation, "We can obtain the result by analytic inversion" is correct. With the second, "We can obtain the result by an analytic inversion" is correct.
The latter would only be used if you're particularly interested in which analytic inversion is being used. If the point is that analytic inversion is being performed, and you don't care whether it's Fourier inversion, Laplace inversion, etc., then the first version is normally used.

"Algorithm", on the other hand, does not have a non-countable interpretation, so "We can obtain the result by an analytic algorithm" is correct while "We can obtain the result by analytic algorithm" is not.

_{Source: I am a mathematician, specialising in analysis, and my research involves analytic inversion!}