FairCom ISAM for C

Blocking Record Reads

The FairCom Server’s blocking record read feature permits an application to attempt a record read with an optional blocking condition, and if no records satisfy the read request, to wait until such a record exists or the block times out. This feature provides a convenient way for an application to process specified records from a file as they become available.

Using Blocking Record Reads

A blocking record read is performed by calling the BlockingISAMRead() c-tree API function. The BlockingISAMRead() function operates by performing the requested ISAM operation (FIRST(), NEXT(), etc.). If successful, it then checks if the optional blocking condition is satisfied. If no record was found or if the record does not satisfy the blocking condition, and a non-zero time-out is specified, then the read blocks (sleeps) for the specified time-out period. The sleep is interrupted if the target file is updated, and the process repeats.

Either a record is found that satisfies the condition, or the block times out. A time out condition is indicated by returning NTIM_ERR (156).

Some Parameters of BlockingISAMRead()

opcode specifies one of the following ISAM read operations:

ISAM Read Operation	Explanation
ctBLKIREC_FIRST	Read first record in physical or key order
ctBLKIREC_NEXT	Read next record in physical or key order
ctBLKIREC_PREV	Read previous record in physical or key order
ctBLKIREC_LAST	Read last record in physical or key order
ctBLKIREC_GT	Read record with key value greater than target key
ctBLKIREC_GTE	Read record with key value greater than or equal to target key
ctBLKIREC_EQL	Read record with key value equal to target key
ctBLKIREC_LTE	Read record with key value less than or equal to target key
ctBLKIREC_LT	Read record with key value less than target key

timeoutsec specifies the number of seconds the blocking read will wait before returning if no record satisfies the optional blockcond argument or no record is found. Set timeoutsec to zero to return immediately.
blockcond is an optional conditional expression which may be a logical expression of the same form as used for conditional index support or a function callback as in a SetDataFilter() call. Set blockcond to an empty string ("") to specify no condition.
target should be non-NULL only if filno specifies an index and the opcode requires a target value (for example, ctBLKIREC_GT).
The record is read into the buffer pointed to by recptr.
plen should be NULL for a fixed-length read, and should point to the length of the output buffer (recptr) for a variable-length read. *plen is updated to the actual record length upon successful return.

Data Filters

Developers consistently search for fast, efficient ways to retrieve data. While it is relatively easy to acquire huge masses of data, the more data available, the more critical it becomes to quickly and efficiently retrieve specific data.

FairCom data filters are a powerful method to define temporary specific criteria restricting return of only those records meeting these criteria with dynamic conditional expressions. Generally, indexes are also used to reduce unnecessary record retrievals; however, a filter allows "as needed" criteria on demand.

Unlike conditional index expressions, which become a permanent index definition (except for temporary indexes), a specific data filter lasts only until another filter is specified for the same data file or the data file is closed. Further, a specific filter applies only to the calling user: it does not affect retrieval of records by other users. Filters are similar to, though less powerful than, queries.

Data filters in client/server environments can substantially reduce network traffic as only records satisfying a given filter condition are returned from the server. ISAM-level record requests, including sets and batches, skip records failing filter criteria transparently to the user.

For example, given a data file of 100,000 records, if a filter excludes 90,000 of those records, a FirstRecord() - NextRecord() loop results in only 10,000 round-trip calls to the FairCom Server rather than traversing all 100,000 records individually. That's a 90% reduction with measurable performance gains!

Data filters are implemented with the SetDataFilter() API function. When a filter is established, record returns, such as from FirstRecord() and GetRecord(), can be limited to a specific range. Conditional expressions, in conjunction with the schema map and symbolic field names, describe the range of values returned. For example, the expression “(ZipCode >= 65000) && (ZipCode < 66000)” would limit returns to records with 65xxx ZipCodes. While there is a limit of one expression, expressions can be as complex as necessary to accomplish your goals.

For partitioned files, call SetDataFilter() for the host data file and the filter automatically applies to each partition member file.

Conditional Indexes

A powerful way to enhance data filtering is directly with index-level control. Conditional indexes are sophisticated index-level filters. To illustrate, consider the following example:

An application requires an ability to dynamically view all new customers within the past 6 months who have purchased at least $50,000 worth of product.

This type of search can be accomplished with existing c-tree batch operations, set functions, or a compound index. In fact, with low-level c-tree functions, this was a common approach, that is, only adding index keys when necessary, as developers maintained complete control over all data files and indexes.

Each of these methods can be quite efficient, however an index will likely contain information unrelated to a specific request, requiring additional scanning and searching. Using low-level calls further precludes advanced ISAM features and desired functionality.

With conditional index support, an index can be uniquely defined containing only information satisfying exact search criteria, thereby providing a fast and elegant solution.

Conditional index expressions are defined by UpdateConditionalIndex() and GetConditionalIndex(). UpdateConditionalIndex() allows conditional expression associated with an index to be added, removed or changed. GetConditionalIndex() retrieves the current conditional expression for a given index.

With these functions, indexes can be limited to a specific range. The conditional expressions, in conjunction with the schema map and symbolic field names, describe the range of values included. For example, the expression “(ZipCode >= 65000) && (ZipCode < 66000)” would limit the index to entries with 65xxx ZipCodes. While there is a limit of one expression per index, the expressions can be as complex as necessary to accomplish your goals.

A potential disadvantage with this feature is when a data file has a large quantity of indexes. Each additional index requires extra processing when adding, updating or deleting key values, especially in batch processing situations. To minimize overhead caused by numerous indexes, see Faster ISAM Access (Disable Key Buffer Support for Faster ISAM Access, /doc/ctreeplus/36768.htm).

Note: Conditional index support requires ISAM support, RESOURCES, and a DODA in the data file to provide the schema map and symbolic field names.