SSH Connections freezing with "Write failed: Broken pipe"
Solution 1:
It looks like the CentOS box's SSHD config is not set to do the client KeepAlive.
Drop these two lines in your CentOS sshd config (/etc/ssh/sshd_config), restart it, and enjoy!
KeepAlive yes
ClientAliveInterval 60
While you're at it, I'd recommend using gnu screen
to keep your session alive on the CentOS side.
Solution 2:
The actual answer is almost always that you have a NAT device of some sort in the path, usually a firewall, whose state tables have a fairly aggressive timeout. Because you leave your ssh connection idle for some periods of time, the NAT device "forgets" the mapping between your inside address and source port number, and your ephemeral outside NATted address and port number.
When you later try to do something in that ssh window, a new ephemeral address/port pair is assigned to you, which the destination ssh server has no knowledge of, and doesn't respond to; later, some local timeout is reached, and the connection is dropped by your local machine.
The practical fix for this is to do exactly what yuriismaster suggests: enable KeepAlives (which ensure regular traffic to "tickle" that state table entry), and use screen
on the remote side (to preserve state in the event things do get dropped). I only post this answer because you asked what's happening, as well as what to do about it. Hopefully this clarifies why yuriismaster's suggestions are good ones.