Product Documentation

FairCom ISAM for C

Previous Topic

Next Topic

ctcmpcif - IFIL-based Compact Utility

This utility can be used to encrypt and decrypt c-tree data files.

Operational Model:

  • Client
  • Standalone
  • FairCom DB Standalone SQL Service

Standalone Usage

ctcmpcif <data file name> <user> <pass> <server> <-sectors> <options>


  • <-sectors> - The sector size to use. Enter a dash followed by the size of the sectors. The sector parameter is especially useful if you need to adjust a file’s PAGE_SIZE to match the FairCom Server. This option must go after <server>. The sector size is 256 is in the example below:

    ctcmpcif xx8.dat ADMIN ADMIN FAIRCOMSMS -256

<options> can be:

  • -callback - Enable notifications of compact progress.
  • -compress - Create the compacted data file with compression (defaults to zlib library if not specified in server configuration).
  • -encrypt=cipher - Create the compacted file using the specified cipher. See the encryption sections below.
    Accepted values:
    • AES: aes16, aes24, or aes32
    • Blowfish: blf8, blf9, ..., or blf56
    • DES: des8, des16, or des24
    • none
  • -flexrec - (Supported in V11.5 and later) Add Hot Alter Table support to the file.
  • -idxseg=<M>@<S> - If creating an index, use M automatic segments of size S. Size is specified in megabytes, or you can specify MB or GB as a suffix. (Example: -idxseg=10@1GB)
  • -nocompress - Create the compacted data file without compression.
  • -noflexrec - (Supported in V11.5 and later) Remove Hot Alter Table support from the file.
  • -oldsec - Use the utility's original security attribute assignment.
  • -online - (In V13 and later) Defragment the file in ctSHARED mode. Online compact is not allowed for changing encryption or compression settings.
  • -purge - Delete records that have illegal duplicate key values.
  • -redosrl - Reassign serial numbers to the records in the compacted file instead of copying the values from the original file.
  • -repairdata - Attempt repair if damaged data records are detected.
  • ‑sortmem=<N> - Set the size of sort memory to N KB in non-client builds. This option can be used to increase the rebuild speed for large files.
  • -temppath=<temporary_path> - Use the specified directory for temporary files.
  • -updifil - Update IFIL resources in the data file.
  • -x8 - Use extended create blocks read from data and index files.

Client-Side Usage

# ctcmpcif DataFileName [-purge] [-updifil] [<UserId>]
[<UserPassword> [<ServerName>]]


  • -purge - purge duplicate records.
  • -updifil - update IFIL resources in the data file.

UserID, UserPassword, and ServerName are only needed for client versions of this utility. FairCom recommends building this utility as a Single-user Standalone application.

ctcmpcif reads the IFIL structure from DataFileName and calls CompactIFileXtd() and RebuildIFileXtd() to compact and rebuild DataFileName and its associated indexes. If ctcmpcif cannot extract the IFIL from the target file, it will ask for the name of another copy of the file from which to extract the IFIL information.

Prior to FairCom DB V9.3, several option values were defined such as updateIFIL, purgeIFIL, and badpartIFIL, that can be specified. These values are specified by adding them together. For example: myifil.tfilno = updateIFIL + purgeIFIL. A new approach simplifies checking of these options. The following option values are now specified in ctport.h:

#define updateIFILoption 0x0002

#define purgeIFILoption 0x0004

#define badpartIFILoption 0x0008

#define redosrlIFILoption 0x0010

These values can now be OR-ed together and the negative of the result stored in the tfilno field of the IFIL structure passed to the compact or rebuild function. For example, to indicate that you want to assign new serial numbers, do the following:

myifil.tfilno = -redosrlIFILoption;


If you want to also use more than one option when compacting or rebuilding a file, OR them in to the value and negate the result. For example,

/* assign new serial numbers and update IFIL resource. */

myifil.tfilno = -(redosrlIFILoption | updateIFILoption);


ctcmpcif will attempt to open files of any page size definition. In addition, in standalone mode, it will create the new file based on the provided page size.

Note: This tool does not compact partitioned files. If you attempt to compact a partitioned file, the tool will return PCMP_ERR (954, compacting partitioned file not supported).

Changing Encryption Attributes

In V11 and later, the compact utility can optionally change the encryption attributes. To use this option, OR in the setencryptIFILoption bit into the tfilno field of the IFIL structure whose address you pass to the compact API function. When using this and other options, remember to negate the tfilno value after you OR in the options. For example:

myifil.tfilno = -(redosrlIFILoption | setencryptIFILoption);


Specifying the Encryption Cipher

In V11 and later, the ctcmpcif utility supports an option to specify the encryption cipher for the data and index files created by the compact operation. Usage:

-encrypt=cipher - Create the compacted file using the specified cipher:

  • for AES, use aes16, aes24, or aes32
  • for Blowfish, use blf8, blf9, ..., or blf56
  • for DES, use des8, des16, or des24
  • for Twofish, use twf16, twf24, or twf32
  • for no encryption, use none

Note: If an index file does not exist, the original data file's encryption attributes are used when creating that index file.

To change the encryption attributes of a file using this ctcmpcif compact utility or the compact API function from a client, you must add the option CHANGE_ENCRYPTION_ON_COMPACT YES to ctsrvr.cfg. Otherwise, the operation fails with error NSUP_ERR (454, not supported).

Environment Variable to Enable Advanced Encryption

In V11.5 and later, c-tree supports enabling advanced encryption at run time using an environment variable. Set the environment variable CTREE_ADVANCED_ENCRYPTION to YES to enable advanced encryption if it is supported. This environment variable can be used to allow c-tree utilities to enable advanced encryption even if they haven't been updated yet to automatically enable advanced encryption when needed. Examples include the rebuild and compact utilities, ctrbldif and ctcmpcif.

Note: If c-tree does not support advanced encryption and this environment variable is set, the c-tree initialization will fail.


  • Even without the -x8 option, some extended create block settings such as the huge file, extended header, and 6-byte transaction number options, are always applied to new files.
  • The -x8 option requires all associated index files that are referenced in the data file's IFIL structure to exist, as the utilities use OpenIFile() to open the data file and all associated index files to read the extended create block values.

When an application calls the standalone version of the Xtd8 file compact or rebuild functions (for example, CMPIFILX8() or RBLIFILX8()), index files created by the compact or rebuild no longer have the 6-byte transaction number attribute enabled if the specified extended create block's x8mode field has the ctNO6BTRAN bit set.

  • A rebuild or compact will fail with the new error code TFLN_ERR (943) when a client library that supports the new rebuild or compact option format attempts to use this feature with a FairCom Server that does not support this new format.
  • In V11 and later, the compact and rebuild API functions preserve the original files' encryption attributes by default. If the index files don't exist when compact/rebuild are called, they will be created with the data file's encryption attributes.
  • In V11 and later, the compact utility always deletes and recreates the index files after saving the resources from the original index file. This provides the expected behavior of (possibly) reducing the size of the index file.

See also

Previous Topic

Next Topic

-oldsec - Updates in handling of security attributes

In V10.3 and later, the ctrbldif utility assigns data file owner/group/permissions to index files and ctcmpcif assigns original data file owner/group/permissions to compacted data/index files.

The rebuild and compact utilities now read the permission mask, owner, and group settings from the original data file and after the rebuild or compact is completed, the utilities assign these same attributes to the new files.

A command-line option has been added to the utilities to restore the previous security attribute behavior. Use the -oldsec option to cause the rebuild and compact utilities to set the security attributes as they did before this revision. For example:

ctrbldif mark.dat -oldsec ADMIN ADMIN FAIRCOMS

The -oldsec option can be used if rebuild and compact are failing with error 455 (user does not belong to group) if you delete an index then run the rebuild or compact utility on the data file using a user account that does not belong to the group that is assigned to the data file.

Note: These changes only apply to the client and server versions of these utilities. Setting the security attributes is not supported in standalone mode, so the standalone rebuild and compact utilities behave as follows:

1) When the standalone mode rebuild utility creates new index files instead of reusing the new index files (for example if the original index files have been deleted before running the rebuild utility), the new index files are assigned a permission mask of zero (no restrictions on permissions), and the owner and group are unassigned (empty).

2) The standalone mode compact utility always preserves the security attributes of the data file. If the index files do not exist, the newly-created index files are assigned the same values as the standalone rebuild does when it creates new index files.

Previous Topic

Next Topic

Preventing Possible Data Loss with Compact & Rebuild Operations

In most cases, compact and/or rebuild operations perform as expected: scanning your data file for potential invalid record headers, fixing those it finds invalid, and finally writing out a new compressed file and rebuilding indexes. Recently, a case was investigated where our default behavior should not be used due to potential data file corruption.

In a few rare and select cases, FairCom identified situations where compacting a variable-length and (typically) non-transaction enabled (non-TRNLOG) file, using the ctcmpcif compact utility or any of the CompactIFile functions, can fail, potentially, and unexpectedly, also altering your original file being compacted such that further attempts result in data loss.

Typically, it is only expected to see this situation occur with non-TRNLOG enabled files. However, in case #2 discussed below, a valid TRNLOG enabled file data file was found to exhibit this very unusual behavior. This raises an important point – if you encounter situations where you believe you must rebuild TRNLOG enabled files, please contact FairCom, as this is a highly unexpected event and we want to fully understand the context and nature of your data integrity situation. The recoverable nature of TRNLOG files should always maintain full data integrity through nearly any circumstance.

We have modified prior behavior to take a much more conservative approach protecting your original data state. Compact and rebuild (which can produce similar results) facilities listed below have been altered such that, when any data file integrity issue is detected, default behavior is now to stop further operations. In addition, no changes are made to existing original data records.

Note: This is a compatibility Change.

Full List of Functions Changed

The following utilities and function calls now default to stopping if an error is encountered:

ctcmpcif – Compact utility
ctrbldif – Rebuild utility
GUI tool - Dr. ctree has compact and rebuild functionality

Long function name

Short function name













Default Behavior Changes the Original File

Prior to FairCom DB V11.5, the default behavior for compact and rebuild operations over variable-length record data files was to scan for invalid record headers before compacting a file. If the logic found invalid headers, it attempted to fix the headers in the original file. If the operation failed at this point, the original file could become corrupted. Two significant classes of situations are known to arise:

  1. In the vast majority of cases, this can only occur if the original data file is already corrupted due to invalid and irrecoverable record headers. Because the original data file is corrupted, restoring a backup is recommended.
  2. In some rare cases, it is possible the operation could exhibit vulnerability caused by mistaking old marks or binary record data as valid record header marks. In a very specific case, compact operations can treat space management nodes with inuse length 0, while logically a correct value, as invalid. This case could lead to data loss in a valid file, making it necessary to restore from a backup.

To ensure the second case does not cause data loss, observe our recommendations below and always back up all concerned data files before compacting or rebuilding.

Compact and Rebuild Best Practice

To avoid potential data loss, FairCom strongly recommends only using our latest FairCom DB V11.5 compact and rebuild logic that default to stopping if a corrupt data record header is encountered, and returns error DCPT_ERR (1107) rather than attempting to repair the data link in the record header. This allows an administrator to know with greater certainty that serious damage has occurred and they may prefer to recover the file from a backup.

If an administrator prefers allowing FairCom DB to attempt to recover a file, as done prior to V11.5, the following options are available to revert to prior behavior:

  • -repairdata command-line switch - Use this switch with the ctcmpcif and ctcmpcif compact utilities, e.g., ctcmpcif -rdata
  • -repairCorruptIFILoption option - The repairCorruptIFILoption option should be OR-ed into the tfilno of the rebuild (RebuildIFIL()) and compact (CompactIFile()) functions using the setIFILoptions() macro.

    For example:
    myIFIL.tfilno = setIFILoptions(repairCorruptIFILoption | updateIFILoption);

Best Practices:
1.) It is always important to back up concerned data files before you attempt to compact or rebuild them.
2.) Ensure sufficient disk space is available before starting a compact or rebuild. You will need up to at least 3X the size of the data file plus index files (original files + new files + copy) for a successful compact or rebuild.
3.) Consider keeping a copy of the data file for sufficient time and ensure all expected data is available before removing the backup.