FIPS 140-3 changes for PKCS #12

With the planned release of Red Hat Enterprise Linux (RHEL) 10 in 2025, the PKCS #12 (Public-Key Cryptography Standards #12) files created in FIPS mode now use Federal Information Processing Standard (FIPS) cryptography by default. In other words, PKCS #12 files allow for backup or easy transfer of keying material between RHEL systems using FIPS approved algorithms.

What are PKCS #12 files?

PKCS #12 is currently defined by RFC 7292 and is a format for storing and transferring private keys, certificates, and miscellaneous secrets. Typically, PKCS #12 is used for transferring private RSA, EdDSA, or ECDSA keys between systems while maintaining the privacy and integrity of the transferred data.

The PKCS #12 standard is fairly old, originating from the Personal Information Exchange (PFX) standard published by Microsoft in 1996. It was revised by RSA Laboratories and then published as PKCS #12 version 1.0 in 1999. Its age means that a lot of the mechanisms and algorithms widely used in PKCS #12 today come from a different era of computing. Because the PKCS #12 standard is quite extensive, there are sometimes compatibility problems between different implementations.

OpenSSL FIPS provider

OpenSSL 3.0 introduced the mechanism of providers, which are loadable modules providing implementations of different cryptographic algorithms. Because the FIPS 140-2 and FIPS 140-3 standards define the Key Derivation Functions (KDF) in scope for FIPS certification, those KDFs  are included in the OpenSSL provider API. Applications can ask for specific KDF algorithms and get them from any available provider that offers them.

In RHEL, we ship an OpenSSL FIPS provider module that includes FIPS-approved algorithms, and only FIPS approved algorithms. When RHEL is operating in FIPS mode, the OpenSSL library is configured to use cryptographic algorithms exclusively from the FIPS provider (or other loaded providers that claim to use FIPS-certified algorithms) by default.

When we started testing FIPS mode in RHEL 9, we noticed that operations on PKCS #12 files stopped working. It turned out that OpenSSL was trying to use PKCS12KDF, the KDF specific to PKCS #12, but that algorithm isn't available in the FIPS provider.

For RHEL 9, we documented that the created PKCS #12 files are not FIPS-compliant, and provided a workaround in OpenSSL. We modified OpenSSL so that it can automatically use PKCS12KDF from the FIPS non-certified default provider. For future releases, we’ve started work on updating the PKCS #12 standard.

PBMAC1 in PKCS #12 files

At the time of writing, the FIPS 140-3 standard allows only one KDF that's intended for use with passwords: Password-Based Key Derivation Function 2 (PBKDF2). We needed to update the PKCS #12 standard to allow for its use for deriving a key used for the whole-file Message Authentication Code (MAC).

Fortunately, the RFC 8018 standard defines a Password-Based Message Authentication Code 1 (PBMAC1) construction that allows combining any KDF with any MAC operation. We've developed a method to use PBMAC1 in PKCS #12 files that's partially backwards-compatible with existing implementations, specifically old versions of OpenSSL, and published it as RFC 9579. This has the added benefit of removing the last dependency on legacy algorithms defined in the original PKCS #12 specification.

We've since implemented RFC 9579 in OpenSSL, GnuTLS, and NSS libraries.

Handling modern PKCS #12 files on old operating systems

When RHEL 10 is operating in FIPS mode, OpenSSL, GnuTLS and NSS generates PKCS #12 files that use PBMAC1 instead of a KDF specific to PKCS #12. Unfortunately, that means those files aren't directly readable by old versions of the same libraries, or other components or appliances that do not implement RFC 9579.

Older versions of OpenSSL are able to read the new files and convert them to an old format, to ensure compatibility with legacy systems.

If you try to read a new file format with an old version of OpenSSL, it returns errors. For example:

$ openssl pkcs12 -in modern.p12 -nodes -out plaintext.pem
MAC: PBMAC1, Iteration 1
MAC length: 32, salt length: 8
Mac verify error: invalid password?
140642650199872:error:2306B076:PKCS12 routines:PKCS12_gen_mac:unknown digest algorithm:crypto/pkcs12/p12_mutl.c:105:
140642650199872:error:2307E06D:PKCS12 routines:PKCS12_verify_mac:mac generation error:crypto/pkcs12/p12_mutl.c:162:
140642650199872:error:2306B076:PKCS12 routines:PKCS12_gen_mac:unknown digest algorithm:crypto/pkcs12/p12_mutl.c:105:
140642650199872:error:2307E06D:PKCS12 routines:PKCS12_verify_mac:mac generation error:crypto/pkcs12/p12_mutl.c:162:

The output on OpenSSL 3.0.2 on RHEL 9:

MAC: PBMAC1, Iteration 1
MAC length: 32, salt length: 8
Mac verify error: invalid password?
405C9269A07F0000:error:0308010C:digital envelope routines:inner_evp_generic_fetch:unsupported:crypto/evp/evp_fetch.c:373:Global default library context, Algorithm (PBMAC1 : 0), Properties (<null>)
405C9269A07F0000:error:11800076:PKCS12 routines:pkcs12_gen_mac:unknown digest algorithm:crypto/pkcs12/p12_mutl.c:122:
405C9269A07F0000:error:1180006D:PKCS12 routines:PKCS12_verify_mac:mac generation error:crypto/pkcs12/p12_mutl.c:191:
405C9269A07F0000:error:0308010C:digital envelope routines:inner_evp_generic_fetch:unsupported:crypto/evp/evp_fetch.c:373:Global default library context, Algorithm (PBMAC1 : 0), Properties (<null>)
405C9269A07F0000:error:11800076:PKCS12 routines:pkcs12_gen_mac:unknown digest algorithm:crypto/pkcs12/p12_mutl.c:122:
405C9269A07F0000:error:1180006D:PKCS12 routines:PKCS12_verify_mac:mac generation error:crypto/pkcs12/p12_mutl.c:191

To convert a PBMAC1-using PKCS #12 file into the old format you have to explicitly disable MAC verification. For this purpose you can use the following commands:

openssl pkcs12 -in modern.p12 -nomacver -nodes -out plaintext.pem
openssl pkcs12 -in plaintext.pem -inkey plaintext.pem -export -out legacy.p12

This creates a legacy.p12 PKCS #12 file encrypted with the default algorithms for the version of OpenSSL being used. You can use the -certpbe and -keypbe options to control the encryption algorithm used for a certificate and private key, respectively. For example:

$ openssl pkcs12 -in plaintext.pem -inkey plaintext.pem \
-export -out legacy.p12 -keypbe aes-128-\
cbc -certpbe aes-128-cbc

Creating modern PKCS#12 files when not in FIPS mode

When a RHEL 10 system is not running in FIPS mode, but there is a need to  create PKCS #12 files using the PBMAC1 algorithm, it is possible to override the default MAC algorithm using command line tools.

GnuTLS library

To create a PKCS #12 file with PBMAC1 using GnuTLS outside of FIPS mode, you must use the --pbmac1 option. In case the private key is in the private_key.pem file, and the certificate is in the cert.pem file, the PKCS #12 can be created with the following command:

$ certtool --to-p12 --pbmac1 \
--password RedHatEnterpriseLinux10.0 \
--p12-name localhost --load-certificate cert.pem \
--load-privkey private_key.pem --outfile combined.p12

This creates a file that uses SHA256 for both the pseudo-random function (PRF) used by PBKDF2 and the hash-based message authentication Code (HMAC).

GnuTLS doesn’t provide a way to change the hash used as PRF, but you can use a different hash for the HMAC with the --hash option. For example, to create a file with SHA-512 HMAC:

$ certtool --to-p12 --pbmac1 \
--password RedHatEnterpriseLinux10.0 --p12-name localhost \
--load-certificate cert.pem --load-privkey private_key.pem \
--hash sha512 --outfile combined.p12

NSS library

When an NSS database includes a key and certificate (for example,  with a nickname of localhost), it's possible to export it to a file with PBMAC1 by specifying the -M option:

$ pk12util -o /tmp/out.p12 -n localhost \
-d sql:./nssdb -M ‘HMAC SHA-256’

Other algorithms supported are HMAC SHA-384 and HMAC SHA-512.

NSS does not support selecting a different PRF for the PBKDF2 and file HMAC. Both algorithms use the specified algorithm.

OpenSSL library

To create a PKCS #12 file with PBMAC1, you must have the private key in one file (for example, private_key.pem) and the certificate in another (for example, cert.pem). It's then possible to create the file in the new format with the -pbmac1_pbkdf2 option.

$ openssl pkcs12 -export -in cert.pem -inkey private_key.pem \
-name localhost -pbmac1_pbkdf2 -out combined.p12 \
-passout pass:RedHatEnterpriseLinux10.0

This creates a file that uses SHA-256 for both the HMAC and the PRF. To use the SHA-384 hash as the PRF, use the -pbmac1_pbkdf2_md sha384 option. To change the whole-file HMAC to SHA-384, use the -macalg sha384 option.

Conclusions

Standard PKCS #12 files, as documented in RFC 7292 do not use FIPS 140-3 approved algorithms. We've proposed, and had RFC 9579 published, to rectify that situation. In RHEL 10, we intend to ship OpenSSL, NSS, and GnuTLS implementing that standard, and those will be used by default when the system is running in FIPS mode.