Product Documentation

FairCom ISAM for C

Previous Topic

Next Topic

ctSetCompress

Sets the data compression type and attribute to be used when creating files with data compression.

Note: To enable data compression, you need to either use the COMPRESS_FILE keyword or programmatically add ctCompressRec in the XCREblk.splval (for ISAM and Low-Level API).

Declaration

ctSETCOMPRESS( NINT comptype, NINT compvrsn, pTEXT dllname, pTEXT attrstr, VRLEN attrlen )

Description

  • comptype - either ctCMPRECzlib (default), ctCMPRECsRLE, ctCMPRECuser (user supplied shared library)
  • compvrsn - alternative versions for a given type
  • dllname - required when comptype==ctCMPRECuser

    attrstr - NULL if no fine-tuning parameters are specified, or the following optional parameter format is:

    "parm1=value; ... parmn=value;"

    zlib specific parameter names are:

    • level
    • strategy
    • windowbits
    • memlevel
    • method

    sRLE specific parameter names are:

    • level [ 0 - 7 ]
    • strategy [ 0 ]
  • attrlen - combined length of the attrstrf strings including the null-termination bytes

Similar in operation to ctSETENCRYPT(), ctSETCOMPRESS() determines the compression specifics that will be used for all compressed data files created after the call to ctSETCOMPRESS(). To use different compression routines for different files, the application creating the files must call ctSETCOMPRESS() before each of the creates. Calling ctSETCOMPRESS() does not affect the compression used by existing files. When existing files are opened the compression resource is used to specify how the records are compressed. Such an open will fail if the resource contains a DLL name that is not available.

If no call is made to ctSETCOMPRESS(), then default, built-in zlib compression routines are used, as long as no server configuration is specified otherwise.

If no DLL is required, dllname is NULL. If no fine-tuning parameters are specified, then attrstrf is NULL and attrlen is zero. (See User Defined Compression Dynamic Shared Library for additional details.)

Notes

  • ctCompressRec is only allowed for data files. An error 944 (AREC_MOD) will be generated if it is used in the XCREblk corresponding to an index file.
  • If a data file to be compressed is fixed-length, it will be changed to a variable-length file at create time. This means there will be an additional 10 bytes of information for each record in the file for the variable-length record management (as discussed in Variable-Length Records). Therefore, be sure compression makes sense on a fixed-length file before turning it on.
  • This feature is available only for ISAM files fully maintained by ISAM updates. Low-Level FairCom DB functions will return an error when used on files with the compression attribute enabled.
  • Some files compress better than others (text data vs. binary data for instance). Initial testing revealed up to an 80:1 compression ratio in some cases.
  • Additional compression routines may become available in the future.

Server Configuration

Default FairCom DB compression can be specified with the following configuration keywords:

CMPREC_TYPE < "ZLIB" | "USER" | "RLE" >

CMPREC_VERSION <a number >= 1>

CMPREC_DLL <name of DLL>

These keywords should be entered in the configuration file in the order shown, and a DLL name is required for CMPREC_TYPE of USER.

The COMPRESS_FILE keyword can be used to enable compression in files whose names match specified file names (including wildcards).

Run Length Encoding (RLE) compression option is available in V10.3 and later. Configure the RLE compression algorithm in ctsrvr.cfg, similar to using the keywords ZLIB or USER.

In v10.3.0 and later, c-tree on Unix/Linux platforms dynamically loads zlib instead of statically linking with it.

Compressed Files in c-treeDB - In V10 and later, the c-treeDB file create mode, CTCREATE_COMPRESS, enables compressed record support in c-treeDB. When CTCREATE_COMPRESS mode is used, c-treeDB automatically creates the file as variable-length.

Return

AREC_BUF

936

Could not allocate a buffer for an augmented variable-length record. (for example, ctCMPREC).

AREC_SUP

937

Augmented variable-length records not supported for SUPERFILE host, fixed length files or index files.

AREC_DCM

938

Could not decompress the data record. sysiocod set to Z_xyz_ERROR.

AREC_ISM

939

Compressed record-length files require ISAM update; not Low-Level update.

AREC_NOP

940

Requested operation not supported for augmented records.

AREC_MOD

944

Bad augmented record file mode at create. Two conditions return this error:

  • Setting ctCompressRec in the XCREblk corresponding to an index file. It is only allowed for data files.
  • Setting ctCompressRec on a data file that is not variable-length (ctVLENGTH).

See also

User Defined Compression Dynamic Shared Library

Previous Topic

Next Topic

User Defined Compression Dynamic Shared Library

The FairCom DB data record compression feature supports user definable compression modules. To include a custom compression module, the following information must be made available from within the shared library.

Function Arguments

pCMPRECFNC pfnc; input pointer to function pointer/DLL structure defined in ctstrc.h.

ppVOID pattrbuf; on input, a pointer to optional, fine-tuning parameter buffer. *pattrbuf==NULL implies no fine-tuning parameters specified. On output, *pattrbuf points to a parameter buffer. *pattrbuf is not guaranteed to be NULL on output if it is NULL on input, and vice versa.

pVRLEN pattrlen; on input, pointer to length of fine-tuning parameter buffer. If *pattrbuf is NULL on input, *pattrlen must be zero. On output *pattrlen contains the length of the parameter buffer.

pCMPRECRES pres; input pointer to compression resource structure defined in ctstrc.h.

pVOID context; input pointer to optional, compression/decompression support structure. If the compression (CmpActn) and/or decompression (ExpActn) routines do not use a support structure, then context will be NULL or ((pVOID) 1).

pVOID source; input pointer to byte stream to be compressed or decompressed.

VRLEN slen; input length of source buffer.

pVOID destination; output pointer to buffer that will contain the result of compression/decompression.

pVRLEN pdlen; on input, *pdlen is length of destination buffer. On output, *pdlen is the length of the byte stream result from the compression/decompression.

Function Prototypes

The module ctzlib.c has implementations of these functions for the built-in zlib support.

FncInit

NINT FncInit(pCMPRECFNC pfnc,pTEXT attrstr,VRLEN attrlen)

This routine performs a one-time setup on each physical open of a file after the DLL has been loaded. attrstr and attrlen are the same as those in ctSETCOMPRESS(). In addition to any DLL specific requirements, FncInit() should verify that the compression type and version (pfnc->comptype and pfnc->compvrsn) are compatible with the DLL, and that the fine-tuning parameters, if any, are valid. If no fine-tuning parameters are passed in, then if the routines require a parameter set, FncInit() should allocate and initialize such a parameter buffer. If it does so, then it should set pfnc->allocbuf to the address of the allocated buffer.

FncInit() returns NO_ERROR or CMPR_ERR (). In the case of CMPR_ERR FncInit() should set the sysiocod (using the routine usys(code)) as listed in cterrc.h for CMPR_ERR. FncInit() does not actually set uerr_cod.

FncExit

NINT FncExit(pCMPRECFNC pfnc )

FncExit() is called just before a file is closed and its associated DLL is unloaded. In addition to DLL specific requirements, FncExit() should free memory allocated for a parameter buffer. FncExit() can test pfnc->allocbuf to find the allocated buffer address, if any.

FncExit() should return NO_ERROR.

CmpInit

pVOID CmpInit(pCMPRECFNC pfnc)

CmpInit() is called each time a user opens a compressed data file. CmpInit() should allocate a context buffer if the DLL requires one, and initialize the context as needed. The context buffer would typically be used to initialize and maintain the compression state. If no such context buffer is needed, then return ((pVOID) 1).

CmpInit() returns the address of the context buffer, or (pVOID) 1 if no context is used, or NULL on error. In case of error call usys(35) to indicate the CmpInit() failure.

CmpActn

NINT CmpActn(pVOID context, pCMPRECFNC pfnc, pVOID source, VRLEN slen, pVOID destination, pVRLEN pdlen)

CmpActn() performs the actual compression of the source into the destination. *pdlen holds the length of the destination buffer on input, and is set to the length of the compressed output on return. CmpActn should be aware that an input value of ((pVOID) 1) means that no context structure has been provided.

CmpActn() returns CMPREC_OK () on success, CMPREC_BUF () if the destination buffer is too small, or CMPR_ERR for an unexpected error. In case of an unexpected error, usys(1000 + err) should be called to set sysiocod where err is the internal error code used by the compression routines. (See zlib1_CmpActn() for an example.)

CmpExit

NINT CmpExit(pVOID context, pCMPRECFNC pfnc)

CmpExit() is called each time a user closes a compressed data file. CmpExit() should de-initialize a context buffer, if one has been allocated in CmpInit(), and free the associated memory.

CmpExit() typically returns NO_ERROR unless the DLL specific code to de-initialize the context buffer returns an error code in which case return this error code.

The following three functions behave just as their compression counterparts except that they apply to decompression. ExpActn() has the same return values as CmpExp() except that attempting to decompress corrupted data results in a return value of CMPREC_DATA and usys(2000 + err) is called where err is the DLL specific internal code signifying the corruption.

pVOID ExpInit(pCMPRECFNC pfnc)

NINT ExpActn(pVOID context, pCMPRECFNC pfnc, pVOID source, VRLEN slen, pVOID destination, pVRLEN pdlen)

NINT ExpExit(pVOID context, pCMPRECFNC pfnc)

TOCIndex