With regard to anti-spam practices, what exactly does a typical reverse DNS check look for?

My understanding is that a typical reverse DNS lookup will compare the SMTP hostname (provided in the EHLO/HELO command) of an incoming connection and verify that it matches the PTR record for the source IP the connection it is coming from. Is that correct?

I've recently heard that it's common for mail providers to require the hostname used in an incoming connection to match an MX entry for the sending domain. This doesn't make sense to me though since the sending organization may have completely different MTAs for outbound.

That's not what it does. It doesn't really care what the hostname is in the HELO/EHLO. Even if this matched a PTR look up, that proves nothing because it could be spoofed (and if you lied in the HELO then you're probably going to lie in your PTR as well, so that would be a doubly useless check.).

What it does do a PTR look up on the client address. It then does an A (or AAAA) look up for the name returned in the PTR record. If this matches then you know that the owner of the DNS zone is also the owner of the IP.

It's then up to the rest of the configuration to act on this status. It's almost never enough to get a blanket OK, but if there is no match it's a good indicator that mail can be refused.

You're right. It doesn't make sense for exactly the reason you stated. An MX record defines where email goes to, not where it comes from. Anyone using any kind of MX record check to validate incoming email is doing it wrong as far as I'm concerned.

An SPF record defines where email comes from.