What is the difference between DTR/DSR and RTS/CTS flow control?
- DTR - Data Terminal Ready
- DSR - Data Set Ready
- RTS - Request To Send
- CTS - Clear To Send
There are multiple ways of doing things because there were never any protocols built into the standards. You use whatever ad-hoc "standard" your equipment implements.
Just based on the names, RTS/CTS would seem to be a natural fit. However, it's backwards from the needs that developed over time. These signals were created at a time when a terminal would batch-send a screen full of data, but the receiver might not be ready, thus the need for flow control. Later the problem would be reversed, as the terminal couldn't keep up with data coming from the host, but the RTS/CTS signals go the wrong direction - the interface isn't orthogonal, and there's no corresponding signals going the other way. Equipment makers adapted as best they could, including using the DTR and DSR signals.
EDIT
To add a bit more detail, its a two level hierarchy so "officially" both must happen for communication to take place. The behavior is defined in the original CCITT (now ITU-T) standard V.28.
The DCE is a modem connecting between the terminal and telephone network. In the telephone network was another piece of equipment which split off to the data network, eg. X.25.
The modem has three states: Powered off, Ready (Data Set Ready is true), and connected (Data Carrier Detect)
The terminal can't do anything until the modem is connected.
When the terminal wants to send data, it raises RTS and the modem grants the request with CTS. The modem lowers CTS when its internal buffer is full.
So nostalgic!
The difference between them is that they use different pins. Seriously, that's it. The reason they both exist is that RTS/CTS wasn't supposed to ever be a flow control mechanism, originally; it was for half-duplex modems to coordinate who was sending and who was receiving. RTS and CTS got misused for flow control so often that it became standard.
An important difference is that some UARTs (16550 notably) will stop receiving characters immediately if their host instructs them to set DSR to be inactive. In contrast, characters will still be received if CTS is inactive. I believe that the intention here is that DSR indicates that the device is no longer listening and so sending any further characters is pointless, while CTS indicates that a buffer is getting full; the latter allows for a certain amount of 'skid' where the flow control line changed state between the DTE sampling it and the next character being transmitted. In (relatively) later devices that support a hardware FIFO it's possible that a number of characters could be transmitted after the DCE has set CTS to be inactive.