"Insistently believing that this is a photo of Neil Armstrong in the face of everyone else's..." Can one really insistently believe something?
Solution 1:
Yes, saying that somebody 'insistently believes' something is indeed odd. What makes it odd is that insisting on something normally proceeds from a conscious choice, while believing, as usually understood, doesn't. (One may choose whether to consider evidence for something, but after one has finished examining and weighing the evidence, it either leads one to believe it, or it doesn't; there is no room for any insisting to take place.) Moreover, insisting is something that one normally does when interacting with others, while believing takes place, so to speak, in one's own head.
The people who might be tempted to say that somebody 'insistently believes' something, either want to say that he focuses a great deal of conscious attention on that belief, or that he insistently defends that belief in interaction with others.