Relative pronoun "whose"

Solution 1:

Just for clarity, I will say that in this text game means:

wild animals and birds that are hunted for food or sport (Cambridge)

The sentence may seem circumvoluted, but it is perfectly grammatical. I will write between brackets the omitted nouns, and the meaning will become clear.

If he start game on one man's lands, and pursue it [the game] to those [lands] of another, ...it [the game] is neither the property of the man on whose lands it [the game] started, nor of him on whose [lands] it [the game] is killed, but belongs to the killer...

So the sentence says that

The game is not the property of the man on whose lands the game started. The game is not the property of the man on whose lands the game is killed. It belongs to the killer of the game.

In order to avoid repetition, the second the man on whose lands was replaced by him on whose.

I don't see why you would want to complicate the sentence even more by replacing that whose with "of whom" or "what person's": it is not correct and it would make no sense:

*nor of him on of whom (?) it is killed

*nor of him on what person's(?) it is killed.

On whose lands could be replaced on the lands of whom but it is extremely rare. I found very few instances on GNgram with the noun shoulders (none with lands):