Migrate encrypted data files

Migrate encrypted data files to be V13-compatible

FIPS-140.2 security requires a revision of FairCom DB file encryption.

When the Advanced Encryption option is enabled (ADVANCED_ENCRYPTION YES), FairCom supports file encryption.

File encryption requires a Master Password, a secret known only to the server administrator, to access the encrypted data. This Master Password is the basis for file encryption security.

A Key Derivation Function (KDF) uses the Master Password to derive a fixed-width Master Key. Prior to V12.5, this KDF was based on MD5, which is not FIPS-140.2 compliant.

When an application requests that a file be encrypted, it specifies the type of encryption algorithm to use.

(create table mytable (f1 integer) STORAGE_ATTRIBUTES 'encr=aes32')

FIPS-140.2 allows only the AES encryption algorithm, while a number of non-compliant algorithms were available in previous versions.

A cryptographic random generator is used to generate a file-specific key (File Key) used to encrypt the data. The File Key is encrypted with a master cipher algorithm using the Master Key, and stored in the header of the file. Prior to V12.0, the master cipher was not FIPS-140.2 compliant.

Both the data and the File Key are encrypted using a cipher block chaining (CBC) mode.

For FIPS-140.2 the following changes were made:

The Master Cipher used for new files or when changing the Master Password now defaults to AES.
Support for non-AES algorithms has been removed. DCOD_ERR (606) is returned for any other encryption algorithms. A conversion utility or library is required to access such files.
A new KDF based on the SHA3 hash function is now the default. Files encrypted with the MD5-based KDF can still be read unless FIPS_ENCRYPTION YES is enabled.
The format of the Master Password verification file (ctsrvr.pvf) was changed.

FIPS Encryption Provider

The described changes allow the FairCom DB server to be run in FIPS mode by enabling the ctsrvr.cfg keyword: FIPS_ENCRYPTION YES.

When specified, FairCom DB uses the OpenSSL 3.0 FIPS module for encryption routines.

Compatibility

Compatibility with prior versions of encrypted files requires some type of conversion or custom FairCom DB libraries depending on the specific options the file was created with. The ctinfo utility has been enhanced to display file encryption details.

For an AES file created with V12.5 server defaults (FIPS compatible), ctinfo reports the following:

Data visibility: AES encrypted      << Required For FIPS-140.2
Encryption Key length: 256 bits
Encryption Attributes: 0x1d
    256 bit master key
    Master key using AES            << Required For FIPS-140.2
    OpenSSL created
    KDF uses SHA3                   << Required For FIPS-140.2

For an AES encrypted file created by V11.8, ctinfo* shows:

Data visibility: AES encrypted
Encryption Key length: 256 bits
Encryption Attributes: 0x1
    256 bit master key
    KDF uses MD5

Data Migration

When migrating a Pre-V12 file to a FIPS-140.2 compatible encoding, if ctinfo reports data visibility as "AES encrypted" then only the File Key in the header of the file needs to be re-encrypted. This can be accomplished using the standalone utility ctencrypt to change the Master Password (the new and old master password can be the same if desired), which re-encrypts the File Key.

If ctinfo reports data visibility as anything but "AES encrypted", then the entire file must be re-encrypted. This can be done using the standalone version of ctcmpcif* with -encrypt = {aes16,aes24,aes32} .

(*) This requires compilation with NO_ctFeatOpenSSL_ONLY to avoid DCOD_ERR(606) due to the Master Key encryption using a non-AES cipher.

Backwards compatibility

To use V12.5 servers to create file encryption options that remain compatible with pre-V12 servers requires a custom database built with #define NO_ctFeatOpenSSL_ONLY. This restores support for legacy encryption routines but disables explicit FIPS support (FIPS_ENCRYPTION YES will fail). New files will still default to FIPS compatible encryption options, and old files using the MD5 KDF and/or the legacy Master CIpher can be accessed normally.

To force new files to use keys derived using MD5 requires creating the Master Key verification file (ctsrvr.pvf) with the -kdfv1 option of the ctcpvf utility.

ctcpvf -kdfv1

To force new files to be encrypted with the legacy Master Cipher use ctsrvr.cfg option: COMPATIBILITY PREV_V12_MASTER_CIPHER.

Note

When this is enabled, the database is unable to access encrypted files using AES as the Master Cipher.

Migrate encrypted data files