NAT Gateway - Maximum connection limit
I know just enough networking to be dangerous. The nitty gritty low level details of NAT are not something I am particularly knowledgeable about.
I accidentally found myself in a discussion earlier today about placing a bunch of our nodes behind a NAT Gateway. (1 public IP address and X private LAN addresses). I called up the 16 bit limit to source and destination port fields in the TCP protocol, (http://www.ietf.org/rfc/rfc793.txt - page 15) and mentioned that it would limit us to some 65,000 connections (65536). -- I am not so confident about that answer anymore. Can you help me with some details?
I understand that an incoming port (server port) on our side can accept as many connections as there are sourceIP x SourcePort combinations. Let's discount those for the time being and focus on connections originating in the LAN, traveling through the NAT Gateway, and ending on a random host at a random port.
On a normal [Linux] system, outgoing connections I believe are limited to 1 per port per Source IP. If we pretend that we live in a simple world where each system only has 1 IP address, then a 'normal system' would be limited to an absolute maximum of 65536 connections.
1) In TCP is a single source IP limited to 65536 MAX theoretical outgoing connections?
2) Or is the limit actually 65536 connections for each Remote Host?
2) [Written another way]: Can the same source port be used for a different remoteHostIP:RemotePort combination?
For example: (Is the following OK?)
Source IP |Source Port |Remote IP|Remote Port
192.168.0.20:36500 --> 8.8.8.8:23
192.168.0.20:36500 --> 8.8.4.4:23
3) Are the answers to questions 1 and 2 different for a ...'not normal system' [Cisco router acting as a NAT Gateway]?
Ex: A specialized networking device that has one public facing IP and up to ~65,000 Lan IPs [or more] behind it? Is there magic at place or is the answer to question 2 just always: yes? (or no)
4) The above questions all assume a stateful TCP connection. Is the story any different with a stateless conection like UDP?
And Ultimately:
5) Will our LAN be limited to 65536 (or some other theoretical limit) concurrent connections to the outside world through a single public IP address?
Thank you! :)
For purposes of this question, we are behind very BEEFY AND BRAND NEW Cisco Nexus gear (7000 series I think). It may be better to ignore memory/etc limitations unless they can be specifically quantified.
Solution 1:
Correct me if I'm wrong but this is the way I understand it. The limits are per client / server / port. So in light of that.
1) In TCP is a single source IP limited to 65536 MAX theoretical outgoing connections?
No. I believe it's limited to 65536 theoretical max to the same destination IP.
Windows workstations (non server versions) have limits imposed which make this number much less. Linux has resource limits, but they generally aren't hit by the average user and you can easily tweak them.
You'll probably hit other resource limits as you start increasing the number anywhere near 64K.
Consumer grade routers might have much lower limits due to the limited resources.
2) Or is the limit actually 65536 connections for each Remote Host?
Yes
3) Are the answers to questions 1 and 2 different for a ...'not normal system' [Cisco router acting as a NAT Gateway]?
No
4) The above questions all assume a stateful TCP connection. Is the story any different with a stateless conection like UDP?
UDP is connectionless. So this isn't really relevant to UDP.
5) Will our LAN be limited to 65536 (or some other theoretical limit) concurrent connections to the outside world through a single public IP address?
No.
In the context of stateful firewalls that track connections and provide other tracking features, yes these modules themselves may have limits. The op has not said anything about which firewall/NAT router is being used so we can't even speculate as to what limitations it might impose at the moment.
Solution 2:
5) Will our LAN be limited to 65536 (or some other theoretical limit) concurrent connections to the outside world through a single public IP address?
No, because one port NAT IP can be used for multiple connections:
cat /proc/net/ip_conntrack | grep 51380
tcp 6 191 ESTABLISHED src=10.1.8.5 dst=17.133.254.23 sport=51380 dport=5223 src=17.133.254.23 dst=my.nat.pub.ip sport=5223 dport=51380 [ASSURED] mark=0 use=2
tcp 6 24 CLOSE_WAIT src=10.1.26.1 dst=80.68.255.71 sport=51380 dport=80 src=80.68.255.71 dst=my.nat.pub.ip sport=80 dport=51380 [ASSURED] mark=0 use=2
Solution 3:
The long-story-short is that it's heavily platform-dependent, configuration-dependent, and implementation-dependent.
But let me explain quickly:
Apparently, other answers state about the theory limit reaching >65535 (notice port 0 is often reserved), which migth be true to certain extent such as...:
- In big CGNAT systems or similar high-grade routers, whose main purpose is this one, including NAT-PAT.
- In certain Linux distro might be possible in theory when using a PC for software NAT/routing by CPU under certains circumstances such as ram > 1GB, kernel prepared, etc.
However in the real world, where hardware-accelerated routing takes place with limited resouce, the NAT table has a well-known limit, which is often a configuration parameter for protection.
-
Cisco mentions starting at IOS 12, the max NAT depends on DRAM resulting on around ~10K translations (source), which is less than those 65K in your question.
-
Take your old xDSL router, if you want to bring up a P2P at home with many connections, most of them have configured a global max-limit of 1024~4096. For instance my high-end FTTH router at home has NAT limit configured to 8K by the vendor.
Finally, to answer Q2-rewrite, I have seen products with dispair implementations of RFC3489 with the following NAT tables. Obviously the last one does limit considerably the NAT posiblities:
- iAddr:iPort - eAddr:ePort - dAddr:dPort (typical symmetric nat)
- iAddr:port - eAddr:port - dAddr (very low end product)
Tumbs up if you like this answer!