What causes a converted Holy City to regain its original religion?
Solution 1:
A holy city can only be prevented from regaining a religion if the religion is destroyed. If it isn't, the city will be affected by internal pressure, usually 30. For the religion to be destroyed, there can not be a single city with that faith as its dominant religion. The holy city will continue to become holy again unless you do one of three things.
Option one: Station an inquisitor in the city. Problem solved.
Option two: Find every city with that faith as the dominant religion and change it.
Option three: Surround the holy city with enough cities of your faith so that the internal pressure is not enough to convert it back. This is probably 5, since internal pressure is usually 30.
PS: Don't ask me why internal pressure isn't always 30, I am not sure. It may regenerate over time after an inquisitor or missionary? I have seen it below 30 but never above.
Solution 2:
Religion can't be permanently destroyed, unless you Raze the Holy City (Not always possible, as this is often a civ's original capital).
Holy Cities have a special "Self-pressure" that is always 5x the pressure / city value (so 45 on quick, 30 on normal, 20 on epic, 10 on Marathon). This is why they regain their old religion, even when no other followers of that religion still exist. (Such as after a Great Prophet / Inquisitor spiel)
This means that you need to surround a holy city with at least 6 cities of your chosen religion, all within 10 squares (if you do only 5, the city might tie for a majority religion, which may 'unconvert' the city). In this case, the religion might come back, but they will never be able to achieve a majority in a city, rendering them, effectively, eliminated.
The exception, however, is if the holy city's original owner generates enough faith for a great prophet (Great Prophets are always the religion that the civ founded, even if they don't have any cities currently following that religion). Unless you conquer the holy city, all it takes is a great prophet to regain its original religion.
Solution 3:
The right answer to your question is: Conquer the holy city and annex it. That must be done. Then send in an inquisitor. This will destroy the holy city's pressure. It won't come back. How do I know this? I've done this many times. I always do this. If you leave it as a puppet, it won't work. I like to have almost all the cities at the end of the game following my religion. I take tithe, so it gives me a lot of gold and helps with city states.