"Back then" or "back there"

When referring to someone from a different geographical area and a different time, what is the proper phrase to refer to them? "Back then" or "back there"? Does it depend on if your intended meaning is to highlight the time difference, or if you are highlighting a geographical difference?

Example: referring to first century Roman attire

"Back then, men wore togas" vs "Back there, men wore togas"

Example: referring to 19th century British fashion

"Back then, men sported long side-whiskers" vs "Back there, men sported long side-whiskers"

It's true that back there is more appropriate if you wish to highlight a geographical distance, and back then is more appropriate if you wish to highlight temporal distance. However, I would say that in both your examples, back then is much more appropriate.

Generally, the phrase back there is for locations which are considered 'behind' the speaker's present location in some sense.

I'll meet you back there, in the store room.

This applies to being physically behind the speaker's current location or to locations that are temporally 'behind' the speaker. In other words, back there is used to describe conditions in a place where the speaker used to be.

I just arrived from Colorado; it was pretty cold back there.

Even though the second sentence is describing conditions in the past, the speaker is talking about very recent past—he/she just arrived—so back then would be inappropriate.

The only time these phrases can be used interchangeably is when describing something happened in the speaker's (relatively) distant past.

I had a happy childhood back there in Missoula
I had a happy childhood back then in Missoula

I suppose you could say, "back there, in ancient Rome…" but that just doesn't seem quite right for ordinary usage. It seems like more of a colloquialism or a literary technique intended to de-emphasize the temporal distance between the speaker and the topic and make it seem like ancient Rome wasn't that long ago.