How to debug Node + Socket.io CPU Issues
We are running a Node Socket.io server with Express 3. The server is monitored using Forever. The service is running well, but the CPU grows throughout the day, until it reaches 90%+ and then suddenly drops back down to ~20%, as shown in the graphs below. I believe that the drop is caused by Forever restarting the app.
What I would like to know is;
- What are the typical factors that could cause a Node.js app to behave like this?
- What tools / methods are available to debug memory leaks / cpu hogging in node apps?
I think it could be something to do with Socket.io not cleaning up resources once users have disconnected, although the docs say that Socket.io will manage this automatically.
Any help would be greatly appreciated, this issue is making managing our server very difficult. Please let me know if this question would be better suited on StackOverflow.
Update: After more research, it appears that the CPU does not directly correlate with the number of connections. Our critical mass seems to be around ~1500 concurrent connections split up like so:
- xhr-polling: 767
- websocket: 692
- jsonppolling: 80
Sometimes we could be at 100% CPU with only 500 connections, other times its 1500 connections. I'm aware that the rate of messages sent has a big affect, however the rate is fairly consistent.
I've got exactly the same problem, sad that noone has any suggestions for us...
here is my problem explained node.js, mongodb, redis, on ubuntu performance degradation in production, RAM is free, CPU 100%