How to decrypt a broken S/MIME message sent by Outlook?

Recently, I received an encrypted message sent by Outlook which I cannot decrypt. Thunderbird is printing following error message instead:

Thunderbird cannot decrypt this message

The sender encrypted this message to you using one of your digital certificates, however Thunderbird was not able to find this certificate and corresponding private key. Possible solutions:

• If you have a smartcard, please insert it now.
• If you are using a new machine, or if you are using a new Thunderbird profile, you will need to restore your certificate and private key from a backup. Certificate backups usually end in ".p12".

Also other mail clients (including a recent version of Outlook!) failed at decrypting the message. As the mail is very important and I don't really want to ask the sender to resend the mail, what can I do?

Solution 1:

The Issue

This is a known issue with Microsoft Outlook 2010, but a fix is provided -- you might want to notify the sender that he should apply it.

X.509 certificates have several attributes attached, some of them can be used to identify certificates. One such way is to use the serial number together with the certificate issuer (together, they have to form a unique identifier). For S/MIME encrypted messages, this is called issuerAndSerialNumber. An alternative is the also standardized subjectKeyIdentifier, which "should be" be derived from the public key in some form, but is not specifically defined.

Outlook 2010 (pre-SP1) uses the subjectKeyIdentifier and creates one if no such identifier is provided (from the knowledge base article linked above, highlighting added by me):

The Cryptographic Message Syntax (CMS) is documented in RFC 5652. That specification allows using either the subjectKeyIdentifier or issuerAndSerialNumber as the SignerIdentifier. The release (RTM) version of Outlook 2010 uses subjectKeyIdentifier as the SignerIdentifier, whereas earlier versions use issuerAndSerialNumber. If the subjectKeyIdentifier extension is not defined in the certificate, Outlook 2010 RTM generates one. Some email clients or third-party operating systems are unable to use the Outlook-generated subjectKeyIdentifier. This results in the recipient being unable to decrypt and read the message.

With other words, Microsoft Outlook 2010 pre-SP1 uses a certificate identifier very likely not understood by any other mail application. I actually failed in using a recent version of Outlook for decrypting such a message!

How to Decrypt the Message Anyway

This won't be easy, and involves dropping to the command line. This should work on pretty much all operating systems (Linux, Windows, macOS, any BSD), make sure to have OpenSSL installed. Using OpenSSL, we can enforce decryption using a specific key, ignoring the broken subjectKeyIdentifier.

1. Save the message to some folder (Thunderbird will save it as an .eml file). I named it mail.eml in all further steps.

2. Export the private key (open the Preferences, Advanced, Certificates, View Certificates, select the appropriate certificate, Backup, select the same folder as used for the message). Thunderbird will query for a passphrase. You should have another file now with .p12 extension. I named it certificate.p12.

3. Open a terminal. All further steps will be completed on the command line.

4. Navigate to the folder using the cd command.

5. For decrypting the message, we need the private key in the PEM format. To convert the key, run openssl pkcs12 -in certificate.p12 -out privatekey.pem -nodes`. You will be asked for the passphrase you entered in Thunderbird.

6. Now use the exported key to actually decrypt the message:

    openssl cms -decrypt -in mail.eml -inkey privatekey.pem -out decrypted.txt
   

   The decrypted message will be stored in the decrypted.txt file.

The message is likely to be encoded as quoted-printable. If you encounter weird character sequences like Gr=FC=DFe and there is a header Content-Transfer-Encoding: quoted-printable included, convert the message to plain text (you need Perl, probably restricted to version 5, and the MIME::QuotedPrint module):

perl -MMIME::QuotedPrint -pe '$_=MIME::QuotedPrint::decode($_);' <decrypted.txt >decoded.txt

The decoded.txt file will finally include the decrypted message. If the encoding of special characters still seems wrong, use the conversion tools of your choice or simply try opening the file in Firefox or another browsers -- usually, they do a great job at fixing messed up encoding.

Putting together a new, unencrypted .eml message requires stripping all Content-* headers and moving any Content-* headers from the decrypted message in this place. More details are out of scope for this tutorial, there are too many different encodings to provide reasonable assistance.