Is there any idiom which is exactly opposite of 'head start'?

I am searching for an idiom which means

to have a disadvantage that makes your chances of winning bleak

It should appropriately fit in this analogy:

Head Start - Win
___ - Lose

Solution 1:

"Handicap"

Noun: 3. Any disadvantage that makes success more difficult:

e.g. "The main handicap of our business is lack of capital."

Solution 2:

A late start would fit the description.

"John had a head start and so he did very well in the race. Derek had a late start and finished last."

Solution 3:

In actual races, it's usually the chest that has to clear the winning line rather than the head, but if we're sticking with "named body parts" relevant to one's chances of winning,...

a hamstrung competitor is unlikely to win the race.

hamstring noun - the large tendon at the back of the hock in the hind leg of a horse, etc.
hamstring verb - to cripple by cutting the hamstring of