FairCom ISAM for C

CompactIFile

Compresses a data file, then rebuilds to regenerate the indexes.

Short Name

CMPIFIL()

Type

ISAM function

Declaration

COUNT CompactIFile(pIFIL ifilptr)

Description

CompactIFile() removes any deleted records from the data file pointed to by ifilptr. Up to twice the data file size is needed to perform this operation. This function calls RebuildIFile to rebuild the associated indexes. CompactIFile() produces a compressed data file containing only active records and new optimized indexes. By calling this function, remote client processes can compact and rebuild files in the client/server model.

InitISAM() must be called prior to calling CompactIFile(). The file(s) pointed to by ifilptr must be closed at the time of the call to CompactIFile().

CompactIFile() also supports two features that the file rebuild function RebuildIFile() supports:

Purging of records having illegal duplicate keys.
Updating the IFIL resource in the data file with the IFIL and IIDX and ISEG structure definitions passed to CompactIFile().

These features are used in the same way they are used when calling RebuildIFile():

To instruct the compact operation to purge records containing illegal duplicate keys, set the purgeIFIL mode in the tfilno member of the IFIL structure passed to CompactIFile().
To instruct the compact operation to update the IFIL resource in the data file with the structure definitions passed to CompactIFile(), set the updateIFIL mode in the tfilno member of the IFIL structure passed to CompactIFile().
To specify both options, add both modes to the tfilno member of the IFIL structure. For example:

myifil.tfilno = updateIFIL + purgeIFIL;

To enable or disable compression when calling CompactIFile(), set the chgcompressIFILoption bit in the tfilno member of the IFIL structure whose address you pass to CMPIFIL(). When this bit is set, CMPIFIL() switches the compression state of the file: compressed files become un-compressed, un-compressed files become compressed.

Serial Number Reassignment

When a serial number (SRLSEG or SCHSRL) field reaches the maximum value, FairCom DB simply increments the serial number such that it wraps around and continues with 0, 1, 2, and so on. This can cause a duplicate key error when an attempt is made to add a record. However, a record having that serial number already exists. To provide an option to recover from this situation, FairCom DB now permits the ISAM file compact functions CMPIFIL(), CMPIFILX(), and CMPIFILX8() to assign new serial numbers to the records in the compacted version of a file. Prior to this modification, the compact always copied the records with their original serial numbers to the new compacted file, so the new file used the same serial numbers as the old file.

The caller of the file compact function indicates that the serial numbers are to be reassigned by specifying a special value in the tfilno field of the IFIL structure that it passes to the compact function.

Prior to FairCom DB V9.5, several option values were defined such as updateIFIL, purgeIFIL, and badpartIFIL, that can be specified. These values are specified by adding them together. For example: myifil.tfilno = updateIFIL + purgeIFIL. A new approach simplifies checking of these options. The following option values are now specified in ctport.h:

#define updateIFILoption 0x0002

#define purgeIFILoption 0x0004

#define badpartIFILoption 0x0008

#define redosrlIFILoption 0x0010

#define chgcompressIFILoption 0x0020 - set compression on/off for new file

#define setencryptIFILoption 0x0040 - use current ctSETENCRYPT state for new file

These values can now be OR-ed together and, using the setIFILoptions() macro, assigned to the tfilno field of the IFIL structure passed to the compact or rebuild function. For example, to indicate that you want to assign new serial numbers, do the following:

myifil.tfilno = setIFILoptions(redosrlIFILoption);

CMPIFIL(&myifil);

If you want to also use more than one option when compacting or rebuilding a file, OR them in to the value. For example:

/* assign new serial numbers and update IFIL resource. */

myifil.tfilno = setIFILoptions(redosrlIFILoption | updateIFILoption);

CMPIFIL(&myifil);

While it is possible to rebuild a mirrored file in c-tree, it is not possible to compact such a file.

Updates for V11

Beginning in FairCom DB V11, several improvements were made to the API function and the compact utility. The file compact utility (ctcmpcif) and the CompactIFile() API function now better handle file encryption The following improvements were made:

The compact API function and utility can optionally change encryption attributes. To use this option when calling the compact API function, use the setIFILoptions() macro to set the setencryptIFILoption bit in the tfilno field of the IFIL structure whose address you pass to the compact API function, and call ctSETENCRYPT() with the desired encryption attributes before calling the compact API function. For example:

NINT rc;

myifil.tfilno = setIFILoptions(setencryptIFILoption);

rc = ctSETENCRYPT(ctAES32, NULL, 32);

rc = CMPIFIL(&myifil);
The ctcmpcif utility now supports an option to specify the encryption cipher for the data and index files created by the compact operation. Usage:
-encrypt=cipher - Create the compacted file using the specified cipher:
- for AES, use aes16, aes24, or aes32
- for Blowfish, use blf8, blf9, ..., or blf56
- for DES, use des8, des16, or des24
- for Twofish, use twf16, twf24, or twf32
- for no encryption, use none
- for the deprecated CAMO option, use camo:MY_ENCRYPTION_KEY Note: CAMO or "Camouflage" is an older, legacy method of hiding data, which is not a standards-conforming encryption scheme, such as AES. It is not intended as a replacement for Advanced Encryption or other security systems.
Note: If an index file does not exist, the original data file's encryption attributes are used when creating that index file.

To change the encryption attributes of a file using the compact API function from a client, you must add the option CHANGE_ENCRYPTION_ON_COMPACT YES to ctsrvr.cfg. Otherwise, the operation fails with error NSUP_ERR (454, not supported).
Compact now always deletes and recreates index files after saving resources from the original index file. This provides the expected behavior of (possibly) reducing the size of the index file.

Updates for V13

Files can now be compacted while the file is open in ctSHARED mode. To enable this request, call CMPIFIL() (CompactIFile) using the new onlineIFILoption bit:

myifil.tfilno = setIFILoptions(onlineIFILoption);

This carries out a more limited form of file compact, referred to as defragmentation, while the file remains available to other users. Defragmentation attempts to reorganize the records within the file to minimize file size. When defragmentation is completed, the size of the file is reduced, if possible.

If the file is under ctTRNLOG control, a ctSHARED mode index rebuild will automatically follow a successful ctSHARED mode compact.

LIMITATIONS: Server Only. The calling user should have the target file initially closed with no active transactions. Other CMPIFIL options specified using setIFILoptions()are not available in combination with onlineIFILoption. Defragmentation is not supported for memory files, super files, partition host files, segmented files, or index files.

NOTES: Defragmentation works similar to a regular user normally updating records within the file. Due to the type and extent of changes during defragmentation, it may highlight artifacts of multiuser interference not frequently observed by applications. Applications using files without transaction control, or using physical order scans, or using more relaxed locking protocols may see increased rates of these artifacts when defragmentation is in progress. This may appear as unexpected missing or even duplicate records (which a future scan doesn't encounter).

Return

A zero (0) return indicates successful operation. A non-zero return indicates an error, check isam_err. The error returns are similar OpenCtFile() and RebuildIFile(). See FairCom DB Error Codes for a complete listing of valid FairCom DB error values.

Example

extern IFIL customer;

COUNT retval;

main() {

if (retval = InitISAM(10,10,16))

printf("\nInitISAM error = %d",retval);

if (!(CompactIFile(&customer)))

printf("\nSuccessful compact");

else

printf("\nCompactIFile isam_err = %d",isam_err);

CloseISAM();

}

Limitations

CompactIFile() does not support FairCom DB Superfiles, see Superfiles.