How to make a teleport command that does not work in the Nether?

I have a vanilla server 1.8.7 that uses a book to tp players to spawn or to their home (an invisible armor stand with their uuid connected to it). Problem is, if they set home (place their armor stand) in the overworld and go to the nether and try to tp home (activated using a trigger), it sends them to the same co-ordinates in the nether, and most likely kills them.

I did some digging, and there seems to be a dimension parameter, but I can't quite figure out the syntax or how to get it to work.

I updated the command block to:

/tp @a[score_tpHome_min=1,Dimension=0] 68 68 68

I was hoping it would only tp the player when they were in the overworld (Dimension 0), but it seems to be ignoring that parameter in the nether and still teleporting to those co-ordinates in the nether.

I also tried:

/tp @a[score_tpHome_min=1,Dimension:0] 68 68 68

Which is the way other people were using the parameter in other ways, but that gave an error.

This is almost a version of @MrLemon's answer. Run the following command once:

/scoreboard objectives add Dimension dummy

Put the following commands on a fill/fast clock:

/scoreboard players set @a Dimension -2 {Dimension:-2}
/scoreboard players set @a Dimension -1 {Dimension:-1}
/scoreboard players set @a Dimension 0 {Dimension:0}    

Then do the tp as following:

/tp @a[score_tpHome_min=1,score_Dimension_min=0] 68 68 68

How this works is it gets the player dimension by setting the scoreboard objective Dimension to the dimension that they are in.

The Dimension parameter does not exist as a target selector argument, but is present in the entity data.

Sadly, /tp does not allow you to check for entity data, so you will have to create a workaround using a scoreboard objective.

/scoreboard objectives add inNether dummy

One the objective exists, create a fill/setblock clock and run the following two commands, in this order:

/scoreboard players set @a inNether 0
/scoreboard players set @a inNether 1 {Dimension:-1}

This will set the objective to 1 for everyone in the Nether and you can use this to limit your tp command to

/tp @a[score_tpHome_min=1,score_inNether=0] 68 68 68

That being said, this would allow the use of the TP in the End, which almost definitely results in death. I suggest turning the logic around by doing

/scoreboard players set @a inOverworld 0
/scoreboard players set @a inOverworld 1 {Dimension:0}

instead, and check for score_inOverworld_min=1.

Minecraft 1.13's addition of NBT to the selector argument syntax means the general technique described in the other answers can be done without scoreboards:

/tp @a[score_tpHome_min=1,nbt={Dimension:0}] 68 68 68

That said, /tp can now teleport players across dimensions, so it may be better to do that instead of disabling the teleport ability entirely in the nether.

Additionally, Minecraft 1.16 changed the NBT format so that the Dimension tag is stored as a string, meaning that you have to use:

 /tp @a[score_tpHome_min=1,nbt={Dimension:"minecraft:overworld"}] 68 68 68

instead if you are playing the most recent version.