FairCom ISAM for C

Delayed Durability Behavior

When the configuration option DELAYED_DURABILITY is specified in ctsrvr.cfg with a value greater than 0, FairCom Server does not sync its log buffer simply because a transaction commits. Instead, it syncs the log buffer if it becomes full, or if a write to the data cache requires the log buffer to be flushed to disk, or if the maximum defer time in seconds specified by the keyword is exhausted.

When this feature is in effect, a transaction commit writes to the in-memory transaction log buffer and data files in the following order:

1. Write transaction log entries to the in-memory transaction log buffer. The log entries are copied to the in-memory transaction log buffer and remain in the buffer until the log buffer becomes full or until a write to a transaction-controlled data or index file requires the log buffer to be written to disk. When the transaction commit call returns to the caller, there is no guarantee that the transaction log entries are on disk, and so the transaction is not guaranteed to be recoverable.
2. Write data file updates to c‑tree’s data cache and/or to disk. Depending on the options that are in effect for the file or for the database engine, this step will either write the data file updates to c‑tree’s data cache or write the changes to file system cache or to disk. It is not required that the changes to the data files are on disk when the transaction commits, because the transaction log entries can be used to redo the transaction if c‑tree terminates abnormally before the updates to the data files are known to be on disk (which can be sometime after the transaction commits).
3. Mark the transaction as completed. This step causes other connections to view the key level locks as committed rather than pending commit.
4. Unlock records. This step allows other connections to lock and update the records that the transaction touched, if desired.

FairCom DB tracks the highest log position assigned to the contents of a data cache page for those items in preimage space that correspond to a data record. When a data-cache page is about to be written to the file system, c‑tree checks if the last log byte synced to disk includes the highest required log position for the data cache page. If not, c‑tree flushes the log buffer to the file system cache and syncs the file system cache to the transaction log file on disk.

Configuration Entries for Delayed Durability

DELAYED_DURABILITY <N> (default 0) controls whether or not to use a modified log syncing strategy.

With delayed durability enabled transaction logs are no longer sync'd to persisted storage on each commit (or other internal log buffer flush events) and instead, transaction log data is allowed to be written to the filesystem cache, and a background thread then periodically and consistently syncs transaction log contents to disk.

By allowing committed transaction entries to be written to filesystem cache and deferring the file flush can result in a very large performance gain in many cases. However, there is a trade off as a window of potential data loss vulnerability is then introduced. The period of time that transaction log contents are present in volatile filesystem cache before the flush could mean transactions already reported to the application as committed, in rare cases, might not make it to persisted storage.

With many modern storage devices there is a limited presumption that available capacitance on the system and storage device hardware that data is usually persisted even in a power outage situation, though this is not guaranteed. Thus alternate recovery strategies should be considered.

One strategy to coordinate the known state of committed database transactions with a known application state can be achieved with restore points. Restore points can be triggered by an application to create a known point in time where the application and the database are in sync. A restore point creates a special database transaction log checkpoint that can be later referenced by the application as a known good start point.

With delayed durability enabled, it is recommended to consider the use of restore points for a robust recoverable solution.

• When DELAYED_DURABILITY set to 0 disables delayed durability/
• When DELAYED_DURABILITY is set to a positive value, <N>, the new strategy is in use and the log sync is guaranteed to occur within <N> seconds. A setting of 1 second is recommended because it results in a good performance gain (higher values offer very little additional benefit). The following configuration options are set as shown below:
  

  SUPPRESS_LOG_FLUSH	YES	(no idle flush of transaction files)
  SUPPRESS_LOG_SYNC	YES
  IDLE_TRANFLUSH	-1
  COMMIT_DELAY	-1	(no commit delay)
  FORCE_LOGIDX	ON	(all transaction indices use ctLOGIDX)
  COMPATIBILITY LOG_WRITETHRU	Disabled

Note: If the configuration file has one or more of these configuration entries set inconsistently after the DELAYED_DURABILITY entry, the server logs a message to CTSTATUS.FCS and continues to start, disabling any incompatible options after processing the configuration file.

Warning

When DELAYED_DURABILITY is enabled, recently committed transactions could be lost if FairCom Server terminates abnormally. For automatic recovery to succeed after FairCom Server terminates abnormally, either of the following should be considered.

1. The application must write a restore point to the log (using the ctflush utility or calling ctQUIET() with mode of ctQTlog_restorepoint) such that a restore point exists prior to the time the server terminated abnormally. In this case, automatic recovery recovers to that restore point.

   or

2. ctsrvr.cfg must contain the option RECOVER_TO_RESTORE_POINT NO, indicating that no restore point is needed. In this case, automatic recovery recovers to the last data that was written to the log on disk. This is the default configuration.

See Also

• Transaction Processing Keywords in the FairCom Server Administrator's Guide