Skip to main content

Configure the replication agent

Use ctreplagent.cfg to configure FairCom's replication agent

Use these options to configure your replication agent

  • If you have changed your ADMIN passwords, the target_auth.set and source_auth.set files will need to be regenerated.

  • By default, the replicated files will be placed on the target in a directory structure that matches the directories on the source. The redirect option can be used if the path on the target machine does not match the source.

  • The unique_id keyword is required in the multi-server environments. It is a good practice to set this keyword in any environment to set the stage for more advanced topologies.

  • Be sure that log_change_details is commented out. This key word is only used for diagnostic testing because it can increase the size of the log and degrade performance.

  • To see if replication is working, use the repadm utility with the getstats command-line option.

The replication agent operations are directed by a single text-based configuration file ctreplagent.cfg. An example is shown below:

; FairCom c-treeACE Server Replication Agent Configuration File - v12.0.0.89 Build(201020)

;The unique ID of the Replication Agent for multi server replications
unique_id REPLAGENT1

; Target server connection info
target_authfile ../config/replication/target_auth.set
target_server FAIRCOMS@localhost

; Source server connection info
source_authfile ../config/replication/source_auth.set
source_server FAIRCOMS@SourceIP

;syncagent yes
; Read 8 KB batches from source server
batch_size 8192

; Use a 5-second timeout when reading from source server
read_timeout_ms 5000

; Log file name
log_file_name  ../config/replication/replagent1.log

; Write change details to log file
;log_change_details changes.log

; The exception_mode option determines how the replication agent handles errors
; that occur when applying operations to the target server.
;
; exception_mode transaction
; When an operation fails, abort the transaction. This option preserves
; atomicity of transactions: either all or none of the operations for the
; transaction are applied to the target c-tree Server.
;
; exception_mode operation
; When an operation fails, continue the transaction, skipping only the
; operation that failed. This option does not preserve atomicity of
; transactions: only the operations that the replication agent successfully
; applies to the target c-tree Server are committed.

exception_mode transaction

; The source server will remember the replication agent's log position

; even when the replication agent disconnects.
;remember_log_pos yes

; file filter specified in the external XML rules
file_filter <../config/replication/filter1.xml

; enables replicating create operations and other schema definition operations
replicate_data_definitions yes

; End of File
  • debug_minlog

    Enables the replication agent to log messages to ctreplagent.log when it changes its minimum log number.

  • diagnostics

    Enables diagnostic logging to the replication agent log file.

  • exception_logging

    Configures exception logging.

  • function_timing

    Enables the replication agent to collect function timings.

  • log_change_details

    Logs the change details to a file.

  • log_file_name

    Defines the filename and location of the replication log.

  • notify_events

    Triggers the execution of an external program, named replmon, when certain events occur.

  • system_event_log

    Specifies for the replication agent to write its messages to the Windows event log in addition to the replication agent log file.

  • batch_size

    Sets the batch size to read from the source server's transaction logs.

  • convert_subst

    Causes the source server to return file names of replicated files with substituted drives converted to the actual path.

  • full_filenames

    Determines whether the replication agent will return a full or relative file path.

  • read_timeout_ms

    Sets the polling timeout value to use when reading from the source server.

  • remember_log_pos

    Requests the source server to persist required logs in lieu of using the server-side KEEP_LOGS option.

  • start_position

    Specifies the starting position on the source server in the replication agent configuration file.

  • socket_timeout

    Specifies a timeout option on the replication agent's send and receive calls.

  • source_authfile

    Sets the source server user connection information.

  • source_nodeid

    Forces the required node ID for the source server.

  • source_server

    Sets the source server connection information.

  • source_tls_cert_file

    Specifies the name of the certificate file to use when connecting to the source server using TLS.

  • source_use_tls

    Causes the connection to the source server to use TLS.

  • target_authfile

    Sets the target server user connection information.

  • target_nodeid

    Forces required node ID for the target server.

  • target_server

    Sets the target server connection information.

  • target_tls_cert_file

    Specifies the name of the certificate file to use when connecting to the target server using TLS.

  • target_use_tls

    Causes the connection to the target server to use TLS.

  • file_filter

    Specifies an XML file filter from an external file.

  • resync_file_list

    Specifies the name of the resync file list that contains the names of all the data files to deploy.

  • unique_id

    Sets the unique ID of the replication agent for multi-server replication.

  • check_update

    Checks record updates for adds, deletes, and rewrites.

  • delete_to_recycle_bin

    Enables the replication agent to use the recycle bin when it deletes a file on the target server.

  • exception_mode

    Specifies how the replication agent handles errors that occur when applying operations to the target server.

  • extension_library

    Registers an external library to be used by the replication agent.

  • lock_retry_count

    Specifies how many times to attempt to lock a record.

  • lock_retry_sleep

    Specify how much time to sleep between record lock attempts.

  • redirect

    Allows files to be redirected to an alternative location on the target server.

  • replicate_data_definitions

    Enables the replication of data definition operations.

  • use_target_alignment

    Specifies for the replication agent to use the alignment setting of the file on the target server.

Set the batch size to read from the source server's transaction logs. When using serial (not parallel) replication a larger batch size can improve throughput. Default is 8192. Recommended for large numbers of replicated tables 1M (1048576).

batch_size <batchBufferSizeInBytes>

Default: 8192

Enable transaction dependency checking when using parallel replication with more than one apply thread.

check_dependencies <yes|no>

Default: yes

If the replication agent is configured to use parallel replication and check_dependencies yes is specified, the replication agent checks the dependencies between transactions it reads from the source server when it schedules the transactions to be applied by the replication agent apply threads.

(V11.5 and later) Configure basic conflict detection for operations that modify data records.

check_update <yes | no | log_only>

Default: yes

When check_update yes is specified: For record updates (add/deletes/rewrites), if the current record image on the target server differs from the record image recorded in the source server's transaction log entry, the Replication Agent logs a replication conflict error REPCNF_ERR(1105) in the exception log and fails the operation.

When check_update log_only is specified: For record updates (add/deletes/rewrites), if the current record image on the target server differs from the record image recorded in the source server's transaction log entry, the Replication Agent logs a replication conflict error REPCNF_ERR(1105) in the exception log and attempts to apply the operation.

The entry that is logged to the exception log in case of error 1105 includes both the information read from the source server (old and new keys and record images) and the current record image read from the target server.

For Windows systems only: When reading replicated file names from the source server's transaction logs, convert substituted drive letters to the actual path.

convert_subst <yes | no>

Default: no

Enable diagnostic logging of the replication agent's minimum transaction log requirement.

debug_minlog <yes | no>

Default: no

When debug_minlog yes is specified, the replication agent writes a message to ctreplagent.log each time its minimum required transaction log number changes.

When replication file deletes, make the deletes recoverable.

delete_to_recycle_bin <yes | no>

Default: yes

Enable the Replication Agent to use the recycle bin when it deletes a file on the target server. Otherwise, the file is deleted from disk immediately.

Enables diagnostic logging to the replication agent log file.

diagnostics <diagnosticsOption>

Default: not set

Supported options:

  • diagnostics analyzer

    enable logging of dependency analysis

  • diagnostics dependency_graph

    enable logging of dependency graph

  • diagnostics file_open

    enable logging of file open/close operations by apply threads

  • diagnostics file_open_err

    enable logging of file open errors

  • diagnostics latency

    enable logging of replication latency

  • diagnostics logread

    enable logging of log read thread

  • diagnostics logship

    enable logging of log ship thread

  • diagnostics tran_state

    enable logging of transaction state processing by log read thread

(Supported in V11.5 and later) Configure exception logging.

exception_logging <on | off | errors_only | target_server>

Default: errors_only

exception_logging on enables logging of every operation in a transaction where at least one operation failed.

exception_logging off turns off logging of failed operations.

exception_logging errors_only enables logging of only those operations that failed to be applied.

exception_logging target_server causes the replication agent to write exception log messages to a table on the target server instead of the server in which the replication agent is running.

The options exception_logging on, exception _logging off, and exception_logging errors_only are mutually exclusive. If more than one of these options is specified in the replication agent configuration file, the last option that is specified takes effect.

Configure how the Replication Agent handles errors that occur when applying operations to the target server.

exception_mode <transaction | operation>

Default: transaction

Modes:

  • transaction When an operation fails, abort the transaction. This option preserves atomicity of transactions: either all or none of the operations for the transaction are applied to the target FairCom DB Server.

  • operation When an operation fails, continue the transaction, skipping only the operation that failed. This option does not preserve atomicity of transactions: only the operations that the Replication Agent successfully applies to the target FairCom DB Server are committed.

Register an external library to be used by the Replication Agent.

extension_library <libraryName>

Default: not set

See our Replication Extension documentation on this feature.

Example:

extension_library ctrepluser.dll

Specify an XML file filter for the Replication Agent to use to determine which files it replicates.

file_filter <xmlFilterFileName>

Default: not set

Two forms of usage are supported.

  1. The XML file exists on the system where the source server is running.

    file_filter filter1.xml
  2. The XML file exists on the system where the replication agent is running.

    file_filter <./filter2.xml

See Also

Configure the use of parallel replication within a file group.

file_group_parallel_apply <yes | no>

Default: no

When using replication file groups, file_group_parallel_apply yes enables parallel replication of transactions within a file group. Otherwise, replication of transactions within a file group is performed serially.

Convert file names that the replication agent reads from the source server's transaction logs to the full path.

full_filenames <yes | no>

Default: no

When the Replication Agent reads a 'file open' change entry from the source server, the entry contains a file number, and the Replication Agent makes a call to the server to get the file name. The source server returns the filename that was read from the transaction log, which is the filename that was specified by the client that opened the file. This name might specify either a full or relative path.

The configuration option full_filenames yes causes the Replication Agent to return the filename with the full path.

(V11.5 and later) Enable the Replication Agent to collect function timings. This can be helpful to analyze replication performance.

function_timing <on | off>

Default: off

Specify how many times to attempt to lock a record. This option is used when the Replication Agent attempts to update a record on the target FairCom DB server.

lock_retry_count <count>

Default: 0 (meaning no retries)

See Also

Record Lock Error Retry and Diagnostics

Specify how much time to sleep between record lock attempts. This option is used when the Replication Agent attempts to update a record on the target FairCom DB server.

lock_retry_sleep <sleep_ms>

Default: 0 (meaning no sleep)

See Also

  • Record lock error retry and diagnostics

Log change details to the specified text file on the system where the Replication Agent is running.

log_change_details <changeDetailLogFileName>

Default: not set

Note

This configuration should only be used during diagnostic testing due to the extensive performance impact from the numerous log entries.

Defines the filename and location of the replication log.

log_file_name <filename>

By default, replication outputs a log file named ctreplagent.log in its working directory. This path and filename can be configured for locating this file in a central area and to specifically identify which replication environment it is associated with.

Trigger execution of an external program, named replmon, by the Replication Agent when certain events occur.

notify_events <yes | no>

Default: no

In V11 and later, the Replication Agent supports a configuration option, notify_events yes, that triggers execution of an external program. This external execution can be a Windows batch file, Unix shell script, or executable and is expected to be named replmon. When an event in the list of event codes below occurs, replmon is called.

Two string values are passed to replmon upon event notification: an error code and one of the numeric event codes listed below. replmon can then process this exception notification as appropriate for the environment.

  1. Replication Agent is starting

  2. Replication Agent is shutting down

  3. Replication Agent is entering paused state

  4. Replication Agent is resuming operation

  5. Replication Agent connected to source server

  6. Replication Agent connected to target server

  7. Replication Agent lost connection to source server

  8. Replication Agent lost connection to target server

  9. Replication Agent disconnected from source server

  10. Replication Agent disconnected from target server

  11. Replication Agent terminating due to exception

Example

In the example below, replmon.bat logs the date, time, event description, and error code to the file replevent.log:

@echo off

if "%1" == "1" (
 set evdesc=Replication agent is starting. 
) else if "%1" == "2" (
 set evdesc=Replication agent is shutting down. Error code %2. 
) else if "%1" == "3" (
 set evdesc=Replication agent has been paused. 
) else if "%1" == "4" (
 set evdesc=Replication agent is resuming normal operation. 
) else if "%1" == "5" (
 set evdesc=Replication agent connected to source server. 
) else if "%1" == "6" (
 set evdesc=Replication agent connected to target server. 
) else if "%1" == "7" (
 set evdesc=Replication agent lost connection to source server. Error code %2. 
) else if "%1" == "8" (
 set evdesc=Replication agent lost connection to target server. Error code %2. 
) else if "%1" == "9" (
 set evdesc=Replication agent disconnected from source server. 
) else if "%1" == "10" (
 set evdesc=Replication agent disconnected from target server. 
) else if "%1" == "11" (
 set evdesc=Replication agent terminating due to exception. Exception code %2. 
) else (
 set evdesc=Replication agent event code %1. Error code %2. 
) 

for /f "usebackq tokens=*" %%j in (`date /t`) do set dt=%%j 
for /f "usebackq tokens=*" %%j in (`time /t`) do set tm=%%j 
echo %dt%%tm% %evdesc% >> replevent.log 

On Windows, The Replication Agent installs an exception handler so it can notify the external process if an unhandled exception occurs.

Set the number of analyzer threads used with parallel replication. Analyzer threads coordinate the internal sequence of operations for correct ordering of transactions to the target server when replication is parallelized.

num_analyzer_threads <n>

Default: 1

  • <n> - number of threads for the analyzer engine to use.

See Also

Set the number of threads to use when applying transactions using parallel replication.

num_apply_threads <n>

Default: 1

  • <n> - number of threads

See Also

Enable parallel replication.

parallel_apply <yes | no>

Default: no

See Also

Set the polling timeout value to use when reading the source server's transaction logs.

read_timeout_ms <timeoutInMilliseconds> 

Default: 5000

By default, the replication agent uses the file path from the source server's transaction logs when accessing the replicated files on the target server. The redirect option allows file names to be redirected to an alternative location on the target server. A simple string match and substitute method is used to modify the new path location.

redirect </path/on/source> </path/on/target>

Default: not set

See Also

Request the source server to persist required logs in lieu of using the server-side KEEP_LOGS option.

For example, logs are required to be retained when the target server or network connection to the target goes offline for a period of time. When the target server is again available, replication will then commence from it's last current position through all the retained logs thereby bringing the system back to a synchronized state.

remember_log_pos <yes | no>

Default: yes

Note

Replication agent threads store their current minimum required log in the REPLSTATEDT.FCS table on the target server. This file should be removed when the replication state is reset between a pair of source and target servers. For example, after resyncing files, or starting a fresh environment.

See Also

Replicate data definition operations.

replicate_data_definitions <yes | no>

Default: no

The data definition operations that are replicated when this option is set to 'yes' include:

  • Create a file at ISAM level

  • Change field definitions (DODA)

  • Create an index file

  • Add a resource

  • Update a resource

  • Delete a resource

  • Delete a file at ISAM level

  • Drop an index

  • Rename a file

  • Add an identity field

  • Delete an identity field

  • Reset an identity field

  • Set extended attributes for an identity field

  • Create a data file at low level

  • Delete a data file at low level

  • Change the serial number for a file

  • Update IFIL resource

  • Update header

  • Add change id field

  • Delete change id field

  • Set null key value

resync_file_list <filename>

Default: not set

Specifies the name of the resync file list that contains the names of all the data files to deploy. Used when using Replication Agent Manager to deploy file groups to a Replication Agent.

A loss of network connectivity can cause the Replication Agent to hang in a socket send or receive call to a source or target server. To allow the Replication Agent to detect and recover from this situation, V11 and later now supports a configuration option to set a timeout on its send and receive calls. The following option can be specified in the Replication Agent configuration file, ctreplagent.cfg:

socket_timeout <timeout_in_seconds>

Default: 5

A setting of zero disables the timeout.

Set the source server user connection information. Use the ctcmdset utility to generate a .set file from the .cfg file. The .cfg file is defined as follows:

USERID ADMIN

PASSWD ADMIN

source_authfile <sourceAuthFileName>

Default: not set

Forces required node ID for the source server. This is provided as an arbitrary dotted notation such as 1.1.1.1.

Note: This nodeid has no relationship whatsoever with any IP address or other network information regarding the source and target servers.

source_nodeid <dotted_notation>

Default: not set

See Also

Set the source server connection information. The format is <servername>@<hostname.domainname>. Any qualified hostname, domain name or IP address that can be resolved by the server is allowed. The server name can also be specified as a direct IP address such as #5597@localhost.localdomain.

source_server <serverName>

Default: not set

source_tls_cert_file <certificateFileName>

Default: not set

Specifies the name of the certificate file to use when connecting to the source server using TLS. Requires source_use_tls yes to be enabled.

source_use_tls <yes | no>

Default: no

source_use_tls yes causes the connection to the source server to use TLS.

start_position <startPositionOptions>

Default: not set (see note below for default behavior)

The Replication Agent supports setting its starting position on the source server in the Replication Agent configuration file, ctreplagent.cfg, with the start_position configuration option. The option accepts three possible formats:

start_position #current

start_position <log number> <log position> <minimum log>

start_position <log number> <log position> <last commit log> <last commit position>

Examples

Use the source server's current transaction log position and minimum log:

start_position #current

Start with transaction log 10 at offset 62697158, and set minimum required log to 7:

start_position 10 62697158 7

Start with transaction log 12 at offset 7782963, and set last commit position to log 12 offset 7287745:

start_position 12 7782963 12 7287745

The Replication API function, ctReplAgentSetConfiOption(), can be used to set the position. For example:

ctReplAgentSetConfigOption(ctRAOPT_start_position, "10 62697158 7", &repstt, lineno, configfile);

Default start position behavior

If start_position is not specified in the replication agent configuration file, the Replication Agent's start position is set as follows:

  1. If a file named ctreplagent_<unique_id>.ini exists, the Replication Agent uses the start position that is specified in that .ini file.

  2. Otherwise, if the Replication Agent has previously saved its start position on the target server, the Replication Agent uses that start position.

  3. Otherwise, the Replication Agent uses the source server's current transaction log position as its start position.

Enable immediate flushing of transaction log writes.

sync_log_writes <yes | no>

Default: yes

sync_log_writes yes causes each write to the local transaction log to be flushed to disk.

system_event_log <machineName>

Default: not set

This option is only supported on Windows systems

When this option is specified, the replication agent writes its messages to the Windows event log in addition to the replication agent log file.

<machineName> is the name of the machine where you want to write event log messages. Specify localhost to write to the local system's event log.

Set the target server user connection information. Use the ctcmdset utility to generate a .set file from the .cfg file. The .cfg file is defined as follows:

USERID ADMIN

PASSWD ADMIN

target_authfile <targetAuthFileName>

Default: not set

Forces a required node ID for the target server. This is provided as an arbitrary dotted notation such as 1.1.1.1.

Note: This nodeid has no relationship whatsoever with any IP address or other network information regarding the source and target servers.

target_nodeid <dotted_notation>

Default: not set

See Also

Set the target server connection information. The format is <servername>@<hostname.domainname>. Any qualified hostname, domain name or IP address that can be resolved by the server is allowed. The server name can also be specified as a direct IP address such as #5597@localhost.localdomain.

target_server <targetServerName>

Default: not set

target_tls_cert_file <certificateFileName>

Default: not set

Specifies the name of the certificate file to use when connecting to the target server using TLS. Requires target_use_tls yes to be enabled.

target_use_tls <yes | no>

Default: no

target_use_tls yes causes the connection to the target server to use TLS.

unique_id <replagentUniqueId>

Default: REPLAGENT

Set the unique ID of the Replication Agent for multi-server replication. The unique ID is case-insensitive, and is limited to 32 bytes. This ID is also referred to as the agent ID.

See Also

use_target_alignment <yes | no>

Default: yes

When use_target_alignment yes is specified, the Replication Agent uses the alignment setting of the file on the target server when applying a record image that it has read from the source server to the target server. Otherwise, the client library's alignment is used.