Usage of infinitive or gerund
Solution 1:
"For him to sail back" is a noun phrase, which can stand as the subject of "unthinkable".
"Sailing back" is also a noun phrase, which can also stand as the subject of "unthinkable"; but there is nowhere to attach "for him" to it. You could attach the subject in a different way: "his sailing back", or (less literary) "him sailing back", could stand as the subject.
As sumelic said in a comment, if "for him" is in a separate breath group (before a comma), it is not part of the syntactic structure, and works as an external modifier.
You could just about manage with "for him" afterwards: "Sailing back for him is unthinkable"; but I think most people would there treat it as a parenthetical, between commas: "Sailing back, for him, is unthinkable".
Solution 2:
[For him to sail back] is unthinkable.
[For him sailing back] is unthinkable.
Only 1. is grammatical.
When a to-infinitival contains a subject, it also contains the clause subordinator for which appears at the beginning of the clause, right before the subject. Thus, the subject of for him to sail back, is him.
But the subordinator for does not occur with gerund-participial clauses, and hence 2. is ungrammatical.
Your paraphrases are fine. The second is actually the extraposed version of your example 1.
Infinitivals containing a subject, like 1. are quite common:
[For you to accept blame] would be a serious mistake.
[For them to refuse you a visa] was quite outrageous.
All I want is [for us to be reunited].