Lot of FIN_WAIT2, CLOSE_WAIT , LAST_ACK and TIME_WAIT in Haproxy

We are running haproxy in production for around 10k+ concurrent users. But we are seeing lot of FIN_WAIT2, CLOSE_WAIT , LAST_ACK and TIME_WAIT in the netstat output. This output is on a 8G ubuntu-12.04 node.

   8046 CLOSE_WAIT  
      1 CLOSING  
      1 established)  
  40869 ESTABLISHED  
   1212 FIN_WAIT1  
   7575 FIN_WAIT2  
      1 Foreign  
   2252 LAST_ACK  
      7 LISTEN  
    143 SYN_RECV  
   4920 TIME_WAIT  

Can someone please tell me what tweaking I need to do?
Please note that all these connections are persistent connections.

tcp_fin_timeout = 30  
tcp_keepalive_time = 1800

Right now, the application is working fine. But wondering will be there any issues as we add more users to this haproxy node.

Solution 1:

The connections you listed in the first paragraph are either ESTABLISHED or in process of being cleaned up after they have been used. Established means what the name implies. A connection is established between one of your users and the HAProxy. Usage as intended.

The other states you mention in the first paragraph all indicate that an previously established connection has finished transferring data. The best way I can explain that is with a diagram.

Briefly: If you have a lot of FIN_WAIT 1 and FIN_WAIT 2's then there is nothing wrong with the server. You are simply waiting for the clients to finish.

Since it is not a problem with the server, adding more users should not be a problem until you hit kernel network limits. You did not post what those are, so I can not comment on how close you are to them.