Sockets found by lsof but not by netstat
This can occur if you create a socket, but never connect() or bind() with it. Your best bet may be to strace (-fF) the application, and then cross-reference with the output of lsof to determine which sockets are causing the issue. As a bonus method of debugging: if you wrap your socket calls with debugging information and write them out to /dev/null, it'll appear in strace without giving you hilariously-large log files.
Using Python, I have encountered the same problem on SSL sockets:
- When I use socket.close(), the socket stays in CLOSE_WAIT state for an indefinite time
- when I use socket.shutdown(), lsof says "can't identify protocol"
The solution was to unwrap the SSL layer before closing:
- origsock = socket.unwrap()
- origsock.close()
This closes the sockets properly in my app.