Decrease lives when a player dies

Solution 1:

First, add the lives count objective (for display) and the death count objective (for behind the scenes):

/scoreboard objectives add lives dummy Lives
/scoreboard objectives add deaths Deathcount

Then wire these up to a redstone clock:

/scoreboard scoreboard players set * lives 5
/scoreboard players operation @a lives -= @a deaths
/gamemode 3 @a[score_deaths_min=5]
/scoreboard players set @a[playerGameType=3] deaths 0

Make sure the first two lines are executed in that order, otherwise players will always have what looks like 5 lives.
Note: I have not tested this, so there are probably some bugs in it.

Lives will then be the number of lives remaining, which you can display wherever you want to.