Are private keys generated by OpenSSL when FIPS mode is disabled usable when FIPS mode is enabled?

genrsa writes OpenSSL's 'traditional' format -- the one with PEM label RSA PRIVATE KEY and added headers Proc-Type and DEK-Info. This uses OpenSSL's (really SSLeay's) nonstandard key derivation (EVP_BytesToKey) based on MD5, which is not FIPS approved. (And also isn't very secure, but that's a different question, and has already been asked or answered many times, probably more on security.SX than here.)

You can generate a key in 'new' (since ~2000!) PKCS8 format using genpkey (which is much more capable, but thus has more options) or you can convert the old format with either

openssl pkey -in tradfile -passin whatever -aes256 -out pkcs8file -passout whatever
openssl pkcs8 -topk8 -in tradfile -passin whatever -v2 aes-256-cbc -out pkcs8file -passout whatever

Either of these uses PBES2 and (thus) PBKDF2 with default HMAC-SHA1, which is okay for FIPS. (Technically pkey wasn't available before 1.0.0, but if there was a FIPS version of 0.9.x, which I don't recall, it must have expired. And not that it helps you today, but AIUI 3.0.0, which finally started beta about a month ago, will have FIPS in the normal build, not requiring all the extra steps.)

FYI default_md in [req] is for the signature on the CSR (or with -x509 selfsigned cert) and in [CA_default] similarly is for the signature on the certificate by ca -- e.g. you get RSAwithSHA256 versus RSAwithSHA1. It has nothing to do with any keyfile, and only req even can generate a key, ca cannot.