Why does terraria get stuck at "connecting to...."?

I have a server running on my desktop pc for terraria that my brothers and myself try to play on. They don't seem to have a problem getting on, but I have difficulties even though its my server. I'll lose the connection and when trying to reconnect it gets stuck on the "connecting to " screen. I can only reconnect after restarting the server. I play on a laptop, so is this an issue with the IP I'm connecting from or just a network issue in general? I've done port forwarding for each computer's IP address and am allowing the game through the firewall.

I have run into this issue a number of times. When you first start up the server, it should ask you how many player spots to reserve, from 1-8. I'm assuming you selected 2 - one for yourself, and one for your brother. Try increasing this to 3 or higher (I set mine to 6). This seems to get rid of this problem, or at least reduce the occurrences by a significant amount.

The following is speculation, since I don't have the client/server source code in front of me, but I believe this error happens because whenever you disconnect, the server keeps temporary session in memory in case you reconnect. This "ghost session" uses up one of those reserve slots, and sometimes the server bugs out and doesn't end those sessions for a long time, which prevents you from logging back in. By increasing the number of available slots, you bypass this problem... at least for a while, until hopefully the server un-bugs itself. If there are enough disconnections to take up all the slots before the server fixes itself, you may still need to restart the server.

I believe one of the latest server updates was supposed to fix this error, but I have occasionally run into it again anyways. The above workaround still works for me just fine.