Can a port be bound when the process has terminated?
If the original process spawns child processes, it's possible for the child processes to keep the socket open past the parent process exit. If the parent process is the only process with the socket open, it should clean up upon termination. An existing process will close all open file descriptors, which includes all its sockets.
Once a close is initiated, the socket may still be in TIME_WAIT state. TCP attempts to guarantee delivery of all data being transmitted. When a socket is closed, it goes into TIME_WAIT to fulfil this guarantee. There are multiple variables that may require some time if they're TCP sockets such as to finish handshaking or whether the sockets were configured with SO_LINGER option.
Here's a link that explains TIME_WAIT from the Unix Socket FAQ: http://www.developerweb.net/forum/showthread.php?t=2941