Client Port Exhaustion Despite Using Different Destination Ports
When reading about port exhaustion, I often see that it takes in account the quadruplet of the tcp connection (src ip, src port, dst ip, dst port). So it would seem that with a port range of 20000 (40000 to 60000) and targeting destination port 8088 and 8087 with two clients on the same machine, using the same src address and dst address, I should be able to open 40000 connections. However on my test this is not the case. Even in this configuration I am able to only open 20000 connections. I've dumped the output of ss -s and sort it by source port. Here is an extract :
CLOSE-WAIT 26378 0 172.24.131.110 40001 172.24.131.97 9088
CLOSE-WAIT 25029 0 172.24.131.110 40002 172.24.131.97 9087
CLOSE-WAIT 23840 0 172.24.131.110 40003 172.24.131.97 9087
CLOSE-WAIT 25207 0 172.24.131.110 40004 172.24.131.97 9087
CLOSE-WAIT 25572 0 172.24.131.110 40005 172.24.131.97 9088
CLOSE-WAIT 26334 0 172.24.131.110 40006 172.24.131.97 9087
CLOSE-WAIT 27089 0 172.24.131.110 40007 172.24.131.97 9087
CLOSE-WAIT 23860 0 172.24.131.110 40008 172.24.131.97 9088
CLOSE-WAIT 25463 0 172.24.131.110 40009 172.24.131.97 9087
CLOSE-WAIT 26603 0 172.24.131.110 40010 172.24.131.97 9088
CLOSE-WAIT 25436 0 172.24.131.110 40011 172.24.131.97 9087
ESTAB 0 0 172.24.131.110 40012 172.24.131.97 9087
CLOSE-WAIT 25042 0 172.24.131.110 40013 172.24.131.97 9087
CLOSE-WAIT 25738 0 172.24.131.110 40014 172.24.131.97 9087
CLOSE-WAIT 27363 0 172.24.131.110 40015 172.24.131.97 9088
CLOSE-WAIT 25860 0 172.24.131.110 40016 172.24.131.97 9088
It goes on until the value 60000 is reached. There is never a source port used twice, as if the pool of values is in fact common. Is this is the case or I am missing on a configuration flag somewhere ?
The kernel version is 4.15.0-144-generic. The port range is really in control here : if I change the port range to 40000-50000 I only have 10000 possible connections. The client and server for the tests are web polygraph tools.
Ok in fact I think I have the answer, the client I am using creates its connections with bind + connect, instead of just connect. Here is an strace dump :
bind(1490, {sa_family=AF_INET, sin_port=htons(0), sin_addr=inet_addr("172.24.131.110")}, 16) = 0
getsockname(1490, {sa_family=AF_INET, sin_port=htons(50929), sin_addr=inet_addr("172.24.131.110")}, [16]) = 0
connect(1490, {sa_family=AF_INET, sin_port=htons(9088), sin_addr=inet_addr("172.24.131.97")}, 16) = -1 EINPROGRESS (Operation now in progress)
From my understanding in this case you are in did going back to a unique pool for the different clients, as at the moment of the bind you don't have any information on the remote side.
A useful link on this subject : https://idea.popcount.org/2014-04-03-bind-before-connect/