How to quickly identify SSH private key file formats?

The file extension is often either random or not enough to identify the format.

Broad categories:

• PEM files with ASN.1 data, encoded with DER
• PEM files with data encoded in some other format
• Non-PEM formats

PEM files wrap Base64 between -----BEGIN----- and -----END----- "tags". They are also commonly used to contain both private key and SSL certificate (-chain). Use an online ASN.1 decoder to check the Base64 contents of a PEM file.

PEM Files

PKCS#1 / OpenSSL: id_rsa, *.pem, *.der, *.key, ...

-----BEGIN RSA PRIVATE KEY-----

PuTTY Key Generator calls this "OpenSSH SSH-2 private key (old PEM format)" (?). The "SSLeay" or "traditional" format, according to this answer. Base64 starts with MII.... ASN.1 content. More info.

PKCS#8: *.pem, *.der, *.key, ...

-----BEGIN PRIVATE KEY----- or -----BEGIN ENCRYPTED PRIVATE KEY-----

Base64 of the unencrypted variation starts with MII...IBADAN. ASN.1 content, basically PKCS#1 plus version info. More info.

OpenSSH: *.??? (don't know what a typical file extension would be)

-----BEGIN OPENSSH PRIVATE KEY-----

PEM on the outside, but non-ASN.1 content. Apparently a somewhat undocumented format.

Non-PEM Files

PuTTY Private Key: *.ppk

Content also contains human readable words identifying it as a putty private key.

PKCS#12 / PFX: *.p12, *.pfx

PFX is a Microsoft format, later released in cleaned-up form as PKCS#12. The content is binary, and can contain not only a private key, but also an SSL certificate (-chain).