Decrypt SSL traffic with the openssl command line tool

I'm not entirely sure but I think the answer is no. The openssl command line client is a heterogeneous collection of tools. The X.509 commands can be useful to manipulate certificates, but the cryptography commands are rarely useful for anything other than testing OpenSSL itself.

If you need to do cryptographic calculations with common algorithms, I recommend the Python interactive command line with the Cryptodome library.

But to decrypt SSL connections, the easiest way is usually to use Wireshark. Tell Wireshark where to find the private key and it will decrypt a TLS connection that uses RSA encryption. For connections using ephemeral Diffie-Hellman, you can't decrypt the traffic with the key alone, you need additional information from either the client or the server.

Note that using the TLS_RSA_WITH_AES_256_CBC_SHA ciphersuite is a bad idea for several reasons:

• It doesn't have forward secrecy, so if the server's private key is ever compromised, all connections made with this key are also compromised. Ciphersuites that use a Diffie-Hellman key exchange (with EDH or ECDHE in their name) have forward secrecy.
• It uses RSA decryption, which involves padding, which is a classic source of implementation bugs and leakage through side channels. Ciphersuites with EDH or ECDHE in their name in addition to RSA, or with DSA or ECDSA, use signatures instead of decryption and are less likely to suffer from implementation defects.
• It uses CBC decryption, which involves padding, which is a classic source of implementation bugs and leakage through side channels. Ciphersuites without CBC in their name are less likely to suffer from implementation defects.

RSAES-PKCS1v1_5, which SSL/TLS-through-1.2 plain-RSA keyexchange uses, can be decrypted by OpenSSL commandline operations rsautl or pkeyutl (the latter since 1.1.0 in 2010). See their respective man pages, which should be available on your system if not Windows, or online.

Note this gives you the premaster secret, which by itself cannot decrypt (or authenticate) traffic. You must use the premaster plus nonces to derive the master secret, and then the master secret plus nonces to derive the working keys (plural). The derivation function 'PRF' differs between SSLv3 (no longer used), TLS 1.0 and 1.1 (RFCs 2246 and 4346), and TLS 1.2 (RFC 5246). (PRF will differ again in TLS 1.3 as drafted, but that will also eliminate plain-RSA keyexchange entirely.)

Commandline cannot access the SSL/TLS PRFs directly, but can do the HMACs from which they are built (except SSLv3); see the man page for dgst in the same places as above, and note -hmac $key can only handle byte sequences that can be passed from shell/etc, which is not all, so you may need -mac hmac -macopt hexkey:$hexkey instead.

That said, I concur with Gilles it's much easier to let wireshark do it. If your problem is you have the data in some form other than a capture file, wireshark distros come with several auxiliary commandline tools for manipulating files that can usually construct a fake capture, which in this case wireshark or tshark main can then decrypt.