What's the meaning of "in God's name"?

Without more context, it's probably figurative and used for emphasis. And yes, you could say that both "on earth" and "in God's name" can be used as intensifiers in this sentence, but I think "on earth" implies more incredulity while "in God's name" is more like regret.

on earth: What is the nature of this thing that we have done?

in God's name: Why did we do this (stupid, horrible, some other negative descriptor) thing?

It seems there is a slight difference, but I'm not sure if I've defined it well.

That is merely one kind of ejaculation:

a short sudden emotional utterance

It acts as an intensifier for the sentence.

You can see that

What the hell have we done?

and

What in God's name have we done?

are considerably more excited, if not stronger, than

What have we done?