Product Documentation

FairCom ISAM for C

Previous Topic

Next Topic

Millions of Open Files

In licensed enterprise editions of V12 and later, c-tree supports opening more than 32,767 files. This enhancement has been designed so no application changes should be required. Each individual database connection is still limited to a maximum of 32,767 files (preserving the original definition of data and index file numbers as two-byte signed values, using the COUNT data type). Internally, new data types were introduced that support up to 2 billion files. Note that we have a compile-time limit of 1,000,000 files to reduce the memory footprint of the c-tree database engine. If you need more than 1,000,000 open files, please contact FairCom.

If opening many files, consider the possibility of using multiple c-tree database engines to host the files, rather than having one c-tree database engine hosting all the files. This is recommended because the files will compete for resources such as data and index cache, file system cache, operating system kernel memory, and transaction logs. Note the c-tree database engine is very resource friendly and it’s possible to execute multiple occurrences of the engine on the same host computer (be sure to comply with the c-tree license which requires a license per installed instance).

Compatibility Notes:

Transaction log compatibility: The checkpoint log entry now contains a 4-byte number of open files rather than a 2-byte value. This means that the transaction logs created by a server without 4-byte file number support are incompatible with a server that uses 4-byte file number support, and vice-versa. When this incompatibility is detected, the database engine fails to start up with error LFRM_ERR (666), "incompatible log format."

The following message in CTSTATUS.FCS indicates a server with 4-byte file numbers found transaction logs that use 2-byte file numbers:

- User# 00001 Incompatible log file format [10: 45800400x 47a00490x 02200090x]

- User# 00001 L0000001.FCS

The following message in CTSTATUS.FCS indicates a server with 2-byte file numbers found transaction logs that use 4-byte file numbers (or some other new feature that this server doesn't support):

- User# 00001 Incompatible log file format [5: 44800400x 07a00490x 43200090x]

- User# 00001 L0000001.FCS

Be sure to review the server upgrade best practices in the FairCom knowledgebase. The following document lists the only recommended procedures for safely upgrading to a server that has a transaction log file format change: Steps to Upgrade a FairCom DB Server.


Standalone Mode Utilities: Remember that this transaction log incompatibility also affects standalone mode utilities such as ctrdmp and ctldmp. These utilities must be compiled with 4-byte file number support in order to read transaction logs that were created by a database engine that supports 4-byte file numbers.

New API function: ctGetOpenFilesXtd()

The function ctGetOpenFiles() uses the ctFILINF structure, which contains a two-byte field for the system file number. To support retrieving the full four-byte system file number, we introduced the function ctGetOpenFilesXtd() (ctGetOpenFilesXtd, ctGetOpenFilesXtd), which uses the new ctFILINFX structure, containing a 4-byte system file number field. An application that calls ctGetOpenFiles() uses the ctFILINF structure, so the output for each file contains a 2-byte system file number. When the file number is greater than 32,767, this function returns -1 for the system file number. To avoid this limitation, call ctGetOpenFilesXtd() instead. Note that the ctadmn utility has been updated to call ctGetOpenFilesXtd(). If that call returns an unsupported function number, ctadmn calls ctGetOpenFiles() instead.

State Variable: The c-tree connection-level state variable isam_fil is still a 16-bit value. A new state variable, isam_fil32, holds the full 32-bit file number in case of an error.

License

Support for handling more than 32,767 files is a licensed feature. The maximum number of files that can be specified by the FILES keyword is limited to 32,767 files when the feature is not enabled.

TOCIndex