C API to customize replication

C API to extend the FairCom replication agent to detect conflicts, transform replicated data, filter replicated data, and deliver data to other systems

Abstract

This section describes using the replication extension library with the original Replication Agent. The extension library can be also used with the newer Replication Manager. For procedures, contact us.

FairCom replication supports loading a user-defined external library that can perform customized actions when processing replication operations. The library contains functions that are called when certain predefined events occur.

This feature allows a software developer to customize the Replication Agent's behavior to support capabilities such as:

Conflict detection and resolution
At their core, replication solutions need to copy data from a source server to a target server to provide a failover server, maintain backup copies, or create an environment for testing or reporting. A foremost consideration is detecting and resolving conflicts between the data on the source and target servers. The Replication Agent is able to provide basic default handling of conflicts; however, many situations require adherence to individual corporate procedures based on proprietary business logic. The extension library allows developers to provide specialized logic to handle these situations according to your corporate policies.
Data transformation
Closely related to conflict detection, some replication environments must accommodate replication between non-identical data stores — for example, it may sometimes be necessary to replicate between databases with different schemas. The extension library can be used to provide functions that transform the source data to comply with the schema of the target data.
Example 1. Sever offline during a schema upgrade
Suppose a replicated server is taken offline and a schema upgrade is performed on it. Because it now has a newer schema than the other replicated servers, the extension library could transform the data structure from the original server to the upgraded server.
Note
Be sure to also consider the use of FairCom’s new Hot Alter Table logic for helping to solve this scenario.
Filter or redirect data updates
In the Example 1, “Sever offline during a schema upgrade”, an alternative strategy could be to selectively replicate to the updated server. In this case, the extension library could be used to filter out the data that should not be replicated.
Replicate to a third-party database
When replicating to a third-party database, the extension library could be used to translate replication operations into inserts and updates to be applied to the third-party database.

Replication Agent extension library SDK

The Replication Agent extensions are provided by an application developer within an SDK framework in an external library loaded by the Replication Agent at runtime. When a user-defined extension library is used, the Replication Agent calls a function in the external library for each operation that it reads from the source server. The function can modify the operation values, or take custom actions, and it can indicate to the agent what action to take for that operation. To use this feature, the developer needs access to FairCom DB Professional and to a development environment that supports writing and compiling C code into a DLL or shared library.

Callback function input parameters

The structures described in this section hold input parameters that are passed to the callback functions

Abstract

The structures described in this section hold input parameters that are passed to the callback functions

The structures described in this section hold input parameters that are passed to the callback functions.

`rxEVENT`

Example 2. Replication Agent event structure (rxEVENT)

typedef struct _rxevent {

    LONG    versn;         /* [IN] version of this structure         */

    LONG    errcod;        /* [IN] Error code                        */

    pctCNXH psrccnxhnd;    /* [IN] Source server connection handle   */

    pctCNXH ptgtcnxhnd;    /* [IN] Target server connection handle   */

    pVOID   pusrctx;       /* [OUT] User context pointer             */

    RXACT   action;        /* [OUT] Set to action to take            */

    union {

        rxVEROP verop;     /* [IN] Parameters for version check      */

        rxFILOP filop;     /* [IN] Parameters for file events        */

        rxTRNOP trnop;     /* [IN] Parameters for transaction events */

        rxDATOP datop;     /* [IN] Parameters for data events        */

        rxEXCOP excop;     /* [IN] Parameters for exceptions         */

    } ev;

Table 1. Inputs and outputs

Input/output	Description
Input	Structure version
Input	Error code for the current event
Input	Source server connection handle (can be used to call Replication API functions such as `ctReplOpenFile()` on the source server)
Input	Target server connection handle (an be used to call Replication API functions such as `ctReplOpenFile()` on the target server)
Input	Replication Agent unique ID
Input	Source server name
Input	Target server name
Input	Source server node ID
Input	Target server node ID
Input	One of the following structures depending on the type of event: `rxVEROP` For the start agent callback `rxFILOP` For file operation callbacks `rxTRNOP` For transaction operation callbacks `rxDATOP` For data operation callbacks `rxEXCOP` For exception callbacks
Output	Optional user-defined context pointer (can be used to store a pointer to memory allocated by the external library)
Output	Action to take for this event Supported actions are: `RXA_DEFAULT` Perform the default action for this event. `RXA_SKIP` Skips the default action for this event. For some events this action has no effect; that is, it is treated as RXA_DEFAULT. `RXA_SHUTDOWN` Shut down the Replication Agent.

`rxVEROP`

Example 3. The Replication Agent version structure

The Replication Agent version structure (rxVEROP) is only used for the start agent event. The start agent callback function must set the fields of this structure to the structure versions in use by the external library so that the agent can verify that the external library is compatible. See the sample start agent callback function for an example of setting these values.

typedef struct _rxverop {

    LONG    verop_ver;  /* [OUT] Set to version of rxVEROP    */

    LONG    event_ver;  /* [OUT] Set to version of rxEVENT    */

    LONG    filop_ver;  /* [OUT] Set to version of rxFILOP    */

    LONG    trnop_ver;  /* [OUT] Set to version of rxTRNOP    */

    LONG    datop_ver;  /* [OUT] Set to version of rxDATOP    */

    LONG    excop_ver;  /* [OUT] Set to version of rxEXCOP    */

    LONG    filh_ver;   /* [OUT] Set to version of ctFILH     */

} rxVEROP, *prxVEROP;

`rxFILOP`

Example 4. The file operation structure (rxFILOP)

The file operation structure (rxFILOP) is passed to file operation callbacks.

typedef struct _rxfilop {

    pctCHGB pchgbuf;    /* [IN] File operation details    */

    pctFILH pfilhnd;    /* [IN] File information          */

} rxFILOP, *prxFILOP;

Table 2. Inputs and outputs

Input/output	Description
Input	File operation details for this event (includes log position
Input	File information for this event (includes data file ID, data file name, and data file number if open)

`rxTRNOP`

Example 5. The transaction operation structure (rxTRNOP)

The transaction operation structure (rxTRNOP) is passed to transaction operation callbacks.

typedef struct _rxtrnop {

    pctCHGB pchgbuf;    /* [IN] Transaction operation details     */

} rxTRNOP, *prxTRNOP;

Table 3. Inputs and outputs

Input/output	Description
Input	Transaction operation details for this event (includes log position and transaction number)

`rxDATOP`

Example 6. The data operation structure (rxDATOP)

The data operation structure (rxDATOP) is passed to data operation callbacks.

typedef struct _rxdatop {

    pctCHGB pchgbuf;    /* [IN] Data operation details    */

    pctFILH pfilhnd;    /* [IN] File information          */

} rxDATOP, *prxDATOP;

Table 4. Inputs and outputs

Input/output	Description
Input	Data operation details for this event (includes log position, file ID, unique key, record image (the callback can change the key value and record image)

`rxEXCOP`

Example 7. The exception operation structure (rxEXCOP)

The exception operation structure (rxEXCOP) is passed to exception callbacks.

typedef struct _rxexcop {

    LONG    errcod;     /* [IN] Error code              */

    pctCHGB pchgbuf;    /* [IN] Operation that failed   */

    pctFILH pfilhnd;    /* [IN] File information        */

} rxEXCOP, *prxEXCOP;

Table 5. Inputs and outputs

Input/output	Description
Input	Error code for this exception.
Input	Details for the operation that failed (includes information relevant to the type of operation that failed — for example, data operation includes log position, file ID, unique key, and record image)
Input	File information for this exception (includes data file ID, data file name, and data file number if open)

Replication events

List of replication events and their descriptions

Abstract

List of replication events and their descriptions

Types of replication events

The Replication Agent calls the corresponding callback function in the external library when the events listed in this section occur.

Table 6. Replication Agent state change events

Event	Description
Start agent	The Replication Agent is starting.
Connected to source	The Replication Agent successfully connected to the source server.
Connected to target	The Replication Agent successfully connected to the target server.
Lost connection to source	The Replication Agent has lost its connection to the source server.
Lost connection to target	The Replication Agent has lost its connection to the target server.
Disconnecting from source	The Replication Agent is disconnecting from the source server.
Disconnecting from target	The Replication Agent is disconnecting from the target server.
Terminating agent	The Replication Agent is terminating due to an unhandled error.
Pause agent	The Replication Agent has been requested to pause its operation.
Resume agent	The Replication Agent has been requested to resume its operation.
Stop agent	The Replication Agent is shutting down.

Table 7. File events

Event	Description
Open file	The Replication Agent is opening a replicated file on the target server.
After file open	The Replication Agent has successfully opened a replicated file on target server.
Close file	The Replication Agent is closing a replicated file on the target server.
Alter schema	The Replication Agent performs an alter schema operation on the target server's replicated file.

Table 8. Transaction events

Event	Description
Start transaction	The Replication Agent is starting a transaction on the target server.
Commit transaction	The Replication Agent is committing a transaction on the target server.
Abort transaction	The Replication Agent is aborting a transaction on the target server.
User-defined log entry	The Replication Agent is processing a user-defined log entry operation.
Checkpoint	The Replication Agent is processing a checkpoint log entry.

Table 9. Data events

Event	Description
Add record	The Replication Agent is adding a record to a replicated file on the target server.
Update record	The Replication Agent is updating a record in a replicated file on the target server.
Delete record	The Replication Agent is deleting a record from a replicated file on target server.
Deferred key	The Replication agent is processing a deferred key log entry.

Table 10. Exception events

Event	Description
Exception	An action performed by the Replication agent has failed.

Replication Agent state change event details

Start agent

The Replication Agent is starting. The external library must provide a function named rxOnStartAgent() for the start agent event to handle any extension library initialization that may be needed.

VOID rxOnStartAgent(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxVEROP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent continues startup.
- RXA_SKIP
  Returning this action for this event has no effect on the Replication Agent (same as RXA_DEFAULT).
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Connected to source

The Replication Agent successfully connected to the source server.

VOID rxOnSourceConnected(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent continues normal operation.
- RXA_SKIP
  Returning this action for this event has no effect on the Replication Agent (same as RXA_DEFAULT).
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Connected to target

The Replication Agent successfully connected to the target server.

VOID rxOnTargetConnected(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent continues normal operation.
- RXA_SKIP
  Returning this action for this event has no effect on the Replication Agent (same as RXA_DEFAULT).
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Lost connection to source

The Replication Agent has lost its connection to the source server.

VOID rxOnSourceLostConnection(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent continues normal operation.
- RXA_SKIP
  Returning this action for this event has no effect on the Replication Agent (same as RXA_DEFAULT).
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Lost connection to target

The Replication Agent has lost its connection to the target server.

VOID rxOnTargetLostConnection(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent continues normal operation.
- RXA_SKIP
  Returning this action for this event has no effect on the Replication Agent (same as RXA_DEFAULT).
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Disconnecting from source

The Replication Agent is disconnecting from the source server.

VOID rxOnSourceDisconnected(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT (see Callback function input parameters)
Output parameters: rxEVENT action field:
- RXA_DEFAULT
  The Replication Agent continues normal operation.
- RXA_SKIP
  Returning this action for this event has no effect on the Replication Agent (same as RXA_DEFAULT).
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Disconnecting from target

The Replication Agent is disconnecting from the target server.

VOID rxOnTargetDisconnected(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent continues normal operation.
- RXA_SKIP
  Returning this action for this event has no effect on the Replication Agent (same as RXA_DEFAULT).
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Terminating agent

The Replication Agent is terminating due to an unhandled error.

VOID rxOnTerminateAgent(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent continues normal operation.
- RXA_SKIP
  Returning this action for this event has no effect on the Replication Agent (same as RXA_DEFAULT).
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Pause agent

The Replication Agent has been requested to pause its operation.

VOID rxOnPauseAgent(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent pauses its operation.
- RXA_SKIP
  Returning this action for this event has no effect on the Replication Agent (same as RXA_DEFAULT).
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Resume agent

The Replication Agent has been requested to resume its operation.

VOID rxOnResumeAgent(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent pauses its operation.
- RXA_SKIP
  Returning this action for this event has no effect on the Replication Agent (same as RXA_DEFAULT).
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Stop agent

The Replication Agent is shutting down.

VOID rxOnStopAgent(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent continues normal operation.
- RXA_SKIP
  Returning this action for this event has no effect on the Replication Agent (same as RXA_DEFAULT).
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Synchronous server shutdown

The Replication Agent is shutting down from a synchronous replication state.

VOID rxOnSyncShutdown (prxEVENT prxevent);

Parameters:

Input parameters: rxEVENT (see Callback function input parameters)
Output parameters: rxEVENT action field
The replication state as of the shutdown is passed to this function in the prxevent->ev.shutdownInfo structure; prxevent->unqid holds the agent unique ID, and state info is also in the JSON file recovery_<agentid>.json.

File event details

Open file

The Replication Agent is opening a replicated file on the target server.

VOID rxOnOpenFile(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxFILOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent opens the file on the target server.
- RXA_SKIP
  The Replication Agent does not open this file on the target server.
- RXA_SHUTDOWN
  The Replication Agent shuts down.

After open file

The Replication Agent has successfully opened a replicated file on the target server.

VOID rxAfterOpenFile(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxFILOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent continues normal operation.
- RXA_SKIP
  Returning this action for this event has no effect on the Replication Agent (same as RXA_DEFAULT).
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Close file

The Replication Agent is closing a replicated file on the target server.

VOID rxOnCloseFile(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxFILOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent closes the file on the target server.
- RXA_SKIP
  The Replication Agent skips closing the file.
- RXA_SHUTDOWN
  The Replication Agent shuts down.

After create IFIL

The Replication Agent has successfully replicated a create file using an IFIL on the target server.

VOID rxAfterCREIFIL(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxFILOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent continues normal operation.
- RXA_SKIP
  Returning this action for this event has no effect on the Replication Agent (same as RXA_DEFAULT).
- RXA_SHUTDOWN
  The Replication Agent shuts down.

After schema

The Replication Agent performs an alter schema operation on the target server's replicated file.

VOID rxOnAlterSchema(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxFILOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent performs the alter schema operation on the target data file.
- RXA_SKIP
  The Replication Agent skips the alter schema operation.
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Resync file

The Replication Agent is resyncing a replicated file with the target server.

VOID rxOnResyncFile(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxFILOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent performs the resync file operation on the target data file.
- RXA_SKIP
  The Replication Agent skips the resync file operation.
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Transaction event details

Start transaction

The Replication Agent is starting a transaction on the target server.

VOID rxOnStartTransaction(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxTRNOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent starts a transaction on the target server.
- RXA_SKIP
  The Replication Agent does not start a transaction on the target server.
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Commit transaction

The Replication Agent is committing a transaction on the target server.

VOID rxOnCommitTransaction(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxTRNOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent commits the transaction on the target server.
- RXA_SKIP
  The Replication Agent does not commit the transaction on the target server.
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Abort transaction

The Replication Agent is aborting a transaction on the target server.

VOID rxOnAbortTransaction(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxTRNOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent aborts the transaction on the target server.
- RXA_SKIP
  The Replication Agent does not abort the transaction on the target server.
- RXA_SHUTDOWN
  The Replication Agent shuts down.

User-defined log entry

The Replication Agent is processing a user-defined log entry operation.

VOID rxOnUserTransaction(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxTRNOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent processes the user-defined log entry.
- RXA_SKIP
  The Replication Agent ignores the user-defined log entry.
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Checkpoint

The Replication Agent is processing a checkpoint log entry.

VOID rxOnCheckpoint(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxTRNOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent processes the checkpoint entry.
- RXA_SKIP
  The Replication Agent ignores the checkpoint entry.
  Note
  Be aware that if you skip the Replication Agent's processing of checkpoint entries, you must manually inform the source server of the Replication Agent's current minimum transaction log requirement.
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Data event details

Add record

The Replication Agent is adding a record to a replicated file on the target server.

VOID rxOnAddRecord(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxDATOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent adds the specified record to the target file.
- RXA_SKIP
  The Replication Agent ignores the add record entry.
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Update record

The Replication Agent is updating a record in a replicated file on the target server.

VOID rxOnUpdateRecord(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxDATOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent updates the specified record in the target file.
- RXA_SKIP
  The Replication Agent ignores the update record entry.
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Delete record

The Replication Agent is deleting a record from a replicated file on the target server.

VOID rxOnDeleteRecord(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxDATOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent deletes the specified record from the target file.
- RXA_SKIP
  The Replication Agent ignores the delete record entry.
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Deferred key

The Replication Agent is processing a deferred key log entry.

VOID rxOnDeferredKey(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxDATOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent processes the deferred key entry.
- RXA_SKIP
  The Replication Agent ignores the deferred key entry.
- RXA_SHUTDOWN
  The Replication Agent shuts down.

Exception event details

Exception

An action performed by the Replication Agent has failed.

VOID rxOnException(prxEVENT prxagentev);

Parameters:

Input parameters: rxEVENT with rxEXCOP (see Callback function input parameters)
Output parameters: rxEVENT action field
- RXA_DEFAULT
  The Replication Agent handles the specified exception using its normal exception handling procedure.
- RXA_SKIP
  The Replication Agent ignores the exception.
- RXA_SHUTDOWN
  The Replication Agent shuts down.

C API to customize replication

C API to extend the FairCom replication agent to detect conflicts, transform replicated data, filter replicated data, and deliver data to other systems

Note

Replication Agent extension library SDK

Callback function input parameters

The structures described in this section hold input parameters that are passed to the callback functions

rxEVENT

rxVEROP

rxFILOP

rxTRNOP

rxDATOP

rxEXCOP

Replication events

List of replication events and their descriptions

Types of replication events

Replication Agent state change event details

Start agent

Connected to source

Connected to target

Lost connection to source

Lost connection to target

Disconnecting from source

Disconnecting from target

Terminating agent

Pause agent

Resume agent

Stop agent

Synchronous server shutdown

File event details

Open file

After open file

Close file

After create IFIL

After schema

Resync file

Transaction event details

Start transaction

Commit transaction

Abort transaction

User-defined log entry

Checkpoint

Note

Data event details

Add record

Update record

Delete record

Deferred key

Exception event details

Exception

`rxEVENT`

`rxVEROP`

`rxFILOP`

`rxTRNOP`

`rxDATOP`

`rxEXCOP`