How does the RSA private key passphrase work under the hood?

ssh-keygen uses OpenSSL to generate RSA keys and store it in PEM format. The encryption you are talking about is specific to PEM. If you look at your key file,

-----BEGIN RSA PRIVATE KEY-----
Proc-Type: 4,ENCRYPTED
DEK-Info: DES-EDE3-CBC,5B01E932988DC66B

EPESt4ZVIrxnQXxxWWVa7cCR+vgNZ/4vTu4mdq6pjaW7jMZoB8HV+mA745mQkQw7
i+YtdVs/JqOeyGiw/3McxYYKZTlhyh7MvfIr1n8ZdZmcjQz+oFqMxChFU3r8BGgA

"DEK-Info" header has all the information you need to decrypt the key as long as you know the passphrase. "DES-EDE3-CBC" means Triple DES (in EDE mode). CBC is the chaining mode. The hex number is the initial vector needed for CBC.

PEM is a very old format so it only supports DES/TripleDES. AES and Blowfish were added later on but not supported by all implementations. My ssh (OpenSSH 5.2) only supports DES and TripleDES.

The passphrase is just a key used to encrypt the file that contains the RSA key, using a symmetric cipher (usually DES or 3DES). In order to use the key for public-key encryption, you first need to decrypt its file using the decryption key. ssh does this automatically by asking your for the passphrase.

If somebody got a hold of the key's file, they wouldn't be able to use it unless they knew the passphrase used to encrypt the file.

Private keys stored on general-purpose file systems (as opposed to tamperproof, special-purpose hardware tokens) could be easily stolen if not protected. File system permissions might seem sufficient, but they can often be bypassed, especially if an attacker has physical access to the machine.

A strong symmetric cipher, keyed with a good password, helps prevent this. A good RSA private key is too long to remember (for me, anyway), but far smaller symmetric keys can provide the same level of security. A relatively short, symmetric key stored in one's brain is used to protect a large private key stored on disk.