My colleague spotted the solution in the tcpdump, that the only other difference between the good and bad connection attempts is the source port.

Then Google told me that I need the insecure option to the export, because after the forward the source port is above 1024 and in the normal case it is below 1024 and insecure tells to the server that it can accept clients with source port above 1024.