Configure the replication agent
Use ctreplagent.cfg to configure FairCom's replication agent
Use these options to configure your replication agent
If you have changed your ADMIN passwords, the
target_auth.set
andsource_auth.set
files will need to be regenerated.By default, the replicated files will be placed on the target in a directory structure that matches the directories on the source. The redirect option can be used if the path on the target machine does not match the source.
The
unique_id
keyword is required in the multi-server environments. It is a good practice to set this keyword in any environment to set the stage for more advanced topologies.Be sure that
log_change_details
is commented out. This key word is only used for diagnostic testing because it can increase the size of the log and degrade performance.To see if replication is working, use the
repadm
utility with thegetstats
command-line option.
The replication agent operations are directed by a single text-based configuration file ctreplagent.cfg. An example is shown below:
; FairCom c-treeACE Server Replication Agent Configuration File - v12.0.0.89 Build(201020) ;The unique ID of the Replication Agent for multi server replications unique_id REPLAGENT1 ; Target server connection info target_authfile ../config/replication/target_auth.set target_server FAIRCOMS@localhost ; Source server connection info source_authfile ../config/replication/source_auth.set source_server FAIRCOMS@SourceIP ;syncagent yes ; Read 8 KB batches from source server batch_size 8192 ; Use a 5-second timeout when reading from source server read_timeout_ms 5000 ; Log file name log_file_name ../config/replication/replagent1.log ; Write change details to log file ;log_change_details changes.log ; The exception_mode option determines how the replication agent handles errors ; that occur when applying operations to the target server. ; ; exception_mode transaction ; When an operation fails, abort the transaction. This option preserves ; atomicity of transactions: either all or none of the operations for the ; transaction are applied to the target c-tree Server. ; ; exception_mode operation ; When an operation fails, continue the transaction, skipping only the ; operation that failed. This option does not preserve atomicity of ; transactions: only the operations that the replication agent successfully ; applies to the target c-tree Server are committed. exception_mode transaction ; The source server will remember the replication agent's log position ; even when the replication agent disconnects. ;remember_log_pos yes ; file filter specified in the external XML rules file_filter <../config/replication/filter1.xml ; enables replicating create operations and other schema definition operations replicate_data_definitions yes ; End of File
Enables the replication agent to log messages to ctreplagent.log when it changes its minimum log number.
Enables diagnostic logging to the replication agent log file.
Configures exception logging.
Enables the replication agent to collect function timings.
Logs the change details to a file.
log_file_name
Defines the filename and location of the replication log.
Triggers the execution of an external program, named replmon, when certain events occur.
Specifies for the replication agent to write its messages to the Windows event log in addition to the replication agent log file.
Sets the batch size to read from the source server's transaction logs.
Causes the source server to return file names of replicated files with substituted drives converted to the actual path.
Determines whether the replication agent will return a full or relative file path.
Sets the polling timeout value to use when reading from the source server.
Requests the source server to persist required logs in lieu of using the server-side KEEP_LOGS option.
Specifies the starting position on the source server in the replication agent configuration file.
Checks the dependencies between transactions it reads from the source server.
Specifies whether a file group is replicated in parallel.
Specifies the number of analyzer threads to coordinate the internal sequence of operations.
Specifies the number of threads to use for parallel replication streams.
Enables parallel replication.
Enables immediate flushing of transaction log writes.
Specifies a timeout option on the replication agent's send and receive calls.
Sets the source server user connection information.
Forces the required node ID for the source server.
Sets the source server connection information.
Specifies the name of the certificate file to use when connecting to the source server using TLS.
Causes the connection to the source server to use TLS.
Sets the target server user connection information.
Forces required node ID for the target server.
Sets the target server connection information.
Specifies the name of the certificate file to use when connecting to the target server using TLS.
Causes the connection to the target server to use TLS.
Specifies an XML file filter from an external file.
Specifies the name of the resync file list that contains the names of all the data files to deploy.
Sets the unique ID of the replication agent for multi-server replication.
Checks record updates for adds, deletes, and rewrites.
Enables the replication agent to use the recycle bin when it deletes a file on the target server.
Specifies how the replication agent handles errors that occur when applying operations to the target server.
Registers an external library to be used by the replication agent.
Specifies how many times to attempt to lock a record.
lock_retry_sleep
Specify how much time to sleep between record lock attempts.
Allows files to be redirected to an alternative location on the target server.
Enables the replication of data definition operations.
Specifies for the replication agent to use the alignment setting of the file on the target server.
Set the batch size to read from the source server's transaction logs. When using serial (not parallel) replication a larger batch size can improve throughput. Default is 8192. Recommended for large numbers of replicated tables 1M (1048576).
batch_size <batchBufferSizeInBytes>
Default: 8192
Enable transaction dependency checking when using parallel replication with more than one apply thread.
check_dependencies <yes|no>
Default: yes
If the replication agent is configured to use parallel replication and check_dependencies yes
is specified, the replication agent checks the dependencies between transactions it reads from the source server when it schedules the transactions to be applied by the replication agent apply threads.
(V11.5 and later) Configure basic conflict detection for operations that modify data records.
check_update <yes | no | log_only>
Default: yes
When check_update yes
is specified: For record updates (add/deletes/rewrites), if the current record image on the target server differs from the record image recorded in the source server's transaction log entry, the Replication Agent logs a replication conflict error REPCNF_ERR(1105) in the exception log and fails the operation.
When check_update log_only
is specified: For record updates (add/deletes/rewrites), if the current record image on the target server differs from the record image recorded in the source server's transaction log entry, the Replication Agent logs a replication conflict error REPCNF_ERR(1105) in the exception log and attempts to apply the operation.
The entry that is logged to the exception log in case of error 1105 includes both the information read from the source server (old and new keys and record images) and the current record image read from the target server.
For Windows systems only: When reading replicated file names from the source server's transaction logs, convert substituted drive letters to the actual path.
convert_subst <yes | no>
Default: no
Enable diagnostic logging of the replication agent's minimum transaction log requirement.
debug_minlog <yes | no>
Default: no
When debug_minlog yes
is specified, the replication agent writes a message to ctreplagent.log
each time its minimum required transaction log number changes.
When replication file deletes, make the deletes recoverable.
delete_to_recycle_bin <yes | no>
Default: yes
Enable the Replication Agent to use the recycle bin when it deletes a file on the target server. Otherwise, the file is deleted from disk immediately.
Enables diagnostic logging to the replication agent log file.
diagnostics <diagnosticsOption>
Default: not set
Supported options:
diagnostics analyzer
enable logging of dependency analysis
diagnostics dependency_graph
enable logging of dependency graph
diagnostics file_open
enable logging of file open/close operations by apply threads
diagnostics file_open_err
enable logging of file open errors
diagnostics latency
enable logging of replication latency
diagnostics logread
enable logging of log read thread
diagnostics logship
enable logging of log ship thread
diagnostics tran_state
enable logging of transaction state processing by log read thread
(Supported in V11.5 and later) Configure exception logging.
exception_logging <on | off | errors_only | target_server>
Default: errors_only
exception_logging on
enables logging of every operation in a transaction where at least one operation failed.
exception_logging off
turns off logging of failed operations.
exception_logging errors_only
enables logging of only those operations that failed to be applied.
exception_logging target_server
causes the replication agent to write exception log messages to a table on the target server instead of the server in which the replication agent is running.
The options exception_logging on
, exception _logging off
, and exception_logging errors_only
are mutually exclusive. If more than one of these options is specified in the replication agent configuration file, the last option that is specified takes effect.
Configure how the Replication Agent handles errors that occur when applying operations to the target server.
exception_mode <transaction | operation>
Default: transaction
Modes:
transaction
When an operation fails, abort the transaction. This option preserves atomicity of transactions: either all or none of the operations for the transaction are applied to the target FairCom DB Server.operation
When an operation fails, continue the transaction, skipping only the operation that failed. This option does not preserve atomicity of transactions: only the operations that the Replication Agent successfully applies to the target FairCom DB Server are committed.
Register an external library to be used by the Replication Agent.
extension_library <libraryName>
Default: not set
See our Replication Extension documentation on this feature.
Example:
extension_library ctrepluser.dll
Specify an XML file filter for the Replication Agent to use to determine which files it replicates.
file_filter <xmlFilterFileName>
Default: not set
Two forms of usage are supported.
The XML file exists on the system where the source server is running.
file_filter filter1.xml
The XML file exists on the system where the replication agent is running.
file_filter <./filter2.xml
See Also
Configure the use of parallel replication within a file group.
file_group_parallel_apply <yes | no>
Default: no
When using replication file groups, file_group_parallel_apply yes
enables parallel replication of transactions within a file group. Otherwise, replication of transactions within a file group is performed serially.
Convert file names that the replication agent reads from the source server's transaction logs to the full path.
full_filenames <yes | no>
Default: no
When the Replication Agent reads a 'file open' change entry from the source server, the entry contains a file number, and the Replication Agent makes a call to the server to get the file name. The source server returns the filename that was read from the transaction log, which is the filename that was specified by the client that opened the file. This name might specify either a full or relative path.
The configuration option full_filenames yes
causes the Replication Agent to return the filename with the full path.
(V11.5 and later) Enable the Replication Agent to collect function timings. This can be helpful to analyze replication performance.
function_timing <on | off>
Default: off
Specify how many times to attempt to lock a record. This option is used when the Replication Agent attempts to update a record on the target FairCom DB server.
lock_retry_count <count>
Default: 0 (meaning no retries)
See Also
Specify how much time to sleep between record lock attempts. This option is used when the Replication Agent attempts to update a record on the target FairCom DB server.
lock_retry_sleep <sleep_ms>
Default: 0 (meaning no sleep)
See Also
Record lock error retry and diagnostics
Log change details to the specified text file on the system where the Replication Agent is running.
log_change_details <changeDetailLogFileName>
Default: not set
Note
This configuration should only be used during diagnostic testing due to the extensive performance impact from the numerous log entries.
Defines the filename and location of the replication log.
log_file_name <filename>
By default, replication outputs a log file named ctreplagent.log
in its working directory. This path and filename can be configured for locating this file in a central area and to specifically identify which replication environment it is associated with.
Trigger execution of an external program, named replmon, by the Replication Agent when certain events occur.
notify_events <yes | no>
Default: no
In V11 and later, the Replication Agent supports a configuration option, notify_events yes
, that triggers execution of an external program. This external execution can be a Windows batch file, Unix shell script, or executable and is expected to be named replmon
. When an event in the list of event codes below occurs, replmon
is called.
Two string values are passed to replmon
upon event notification: an error code and one of the numeric event codes listed below. replmon
can then process this exception notification as appropriate for the environment.
Replication Agent is starting
Replication Agent is shutting down
Replication Agent is entering paused state
Replication Agent is resuming operation
Replication Agent connected to source server
Replication Agent connected to target server
Replication Agent lost connection to source server
Replication Agent lost connection to target server
Replication Agent disconnected from source server
Replication Agent disconnected from target server
Replication Agent terminating due to exception
Example
In the example below, replmon.bat
logs the date, time, event description, and error code to the file replevent.log
:
@echo off if "%1" == "1" ( set evdesc=Replication agent is starting. ) else if "%1" == "2" ( set evdesc=Replication agent is shutting down. Error code %2. ) else if "%1" == "3" ( set evdesc=Replication agent has been paused. ) else if "%1" == "4" ( set evdesc=Replication agent is resuming normal operation. ) else if "%1" == "5" ( set evdesc=Replication agent connected to source server. ) else if "%1" == "6" ( set evdesc=Replication agent connected to target server. ) else if "%1" == "7" ( set evdesc=Replication agent lost connection to source server. Error code %2. ) else if "%1" == "8" ( set evdesc=Replication agent lost connection to target server. Error code %2. ) else if "%1" == "9" ( set evdesc=Replication agent disconnected from source server. ) else if "%1" == "10" ( set evdesc=Replication agent disconnected from target server. ) else if "%1" == "11" ( set evdesc=Replication agent terminating due to exception. Exception code %2. ) else ( set evdesc=Replication agent event code %1. Error code %2. ) for /f "usebackq tokens=*" %%j in (`date /t`) do set dt=%%j for /f "usebackq tokens=*" %%j in (`time /t`) do set tm=%%j echo %dt%%tm% %evdesc% >> replevent.log
On Windows, The Replication Agent installs an exception handler so it can notify the external process if an unhandled exception occurs.
Set the number of analyzer threads used with parallel replication. Analyzer threads coordinate the internal sequence of operations for correct ordering of transactions to the target server when replication is parallelized.
num_analyzer_threads <n>
Default: 1
<n> - number of threads for the analyzer engine to use.
See Also
Set the number of threads to use when applying transactions using parallel replication.
num_apply_threads <n>
Default: 1
<n> - number of threads
See Also
Enable parallel replication.
parallel_apply <yes | no>
Default: no
See Also
Set the polling timeout value to use when reading the source server's transaction logs.
read_timeout_ms <timeoutInMilliseconds>
Default: 5000
By default, the replication agent uses the file path from the source server's transaction logs when accessing the replicated files on the target server. The redirect
option allows file names to be redirected to an alternative location on the target server. A simple string match and substitute method is used to modify the new path location.
redirect </path/on/source> </path/on/target>
Default: not set
See Also
Request the source server to persist required logs in lieu of using the server-side KEEP_LOGS
option.
For example, logs are required to be retained when the target server or network connection to the target goes offline for a period of time. When the target server is again available, replication will then commence from it's last current position through all the retained logs thereby bringing the system back to a synchronized state.
remember_log_pos <yes | no>
Default: yes
Note
Replication agent threads store their current minimum required log in the REPLSTATEDT.FCS table on the target server. This file should be removed when the replication state is reset between a pair of source and target servers. For example, after resyncing files, or starting a fresh environment.
See Also
Replicate data definition operations.
replicate_data_definitions <yes | no>
Default: no
The data definition operations that are replicated when this option is set to 'yes' include:
Create a file at ISAM level
Change field definitions (DODA)
Create an index file
Add a resource
Update a resource
Delete a resource
Delete a file at ISAM level
Drop an index
Rename a file
Add an identity field
Delete an identity field
Reset an identity field
Set extended attributes for an identity field
Create a data file at low level
Delete a data file at low level
Change the serial number for a file
Update IFIL resource
Update header
Add change id field
Delete change id field
Set null key value
resync_file_list <filename>
Default: not set
Specifies the name of the resync file list that contains the names of all the data files to deploy. Used when using Replication Agent Manager to deploy file groups to a Replication Agent.
A loss of network connectivity can cause the Replication Agent to hang in a socket send or receive call to a source or target server. To allow the Replication Agent to detect and recover from this situation, V11 and later now supports a configuration option to set a timeout on its send and receive calls. The following option can be specified in the Replication Agent configuration file, ctreplagent.cfg
:
socket_timeout <timeout_in_seconds>
Default: 5
A setting of zero disables the timeout.
Set the source server user connection information. Use the ctcmdset utility to generate a .set file from the .cfg file. The .cfg file is defined as follows:
USERID ADMIN
PASSWD ADMIN
source_authfile <sourceAuthFileName>
Default: not set
Forces required node ID for the source server. This is provided as an arbitrary dotted notation such as 1.1.1.1.
Note: This nodeid has no relationship whatsoever with any IP address or other network information regarding the source and target servers.
source_nodeid <dotted_notation>
Default: not set
See Also
Specify Required Node IDs for Source and Targeting
Set the source server connection information. The format is <servername>@<hostname.domainname>. Any qualified hostname, domain name or IP address that can be resolved by the server is allowed. The server name can also be specified as a direct IP address such as #5597@localhost.localdomain.
source_server <serverName>
Default: not set
source_tls_cert_file <certificateFileName>
Default: not set
Specifies the name of the certificate file to use when connecting to the source server using TLS. Requires source_use_tls yes
to be enabled.
source_use_tls <yes | no>
Default: no
source_use_tls yes
causes the connection to the source server to use TLS.
start_position <startPositionOptions>
Default: not set (see note below for default behavior)
The Replication Agent supports setting its starting position on the source server in the Replication Agent configuration file, ctreplagent.cfg
, with the start_position
configuration option. The option accepts three possible formats:
start_position #current start_position <log number> <log position> <minimum log> start_position <log number> <log position> <last commit log> <last commit position>
Examples
Use the source server's current transaction log position and minimum log:
start_position #current
Start with transaction log 10 at offset 62697158, and set minimum required log to 7:
start_position 10 62697158 7
Start with transaction log 12 at offset 7782963, and set last commit position to log 12 offset 7287745:
start_position 12 7782963 12 7287745
The Replication API function, ctReplAgentSetConfiOption()
, can be used to set the position. For example:
ctReplAgentSetConfigOption(ctRAOPT_start_position, "10 62697158 7", &repstt, lineno, configfile);
Default start position behavior
If start_position
is not specified in the replication agent configuration file, the Replication Agent's start position is set as follows:
If a file named
ctreplagent_<unique_id>.ini
exists, the Replication Agent uses the start position that is specified in that.ini
file.Otherwise, if the Replication Agent has previously saved its start position on the target server, the Replication Agent uses that start position.
Otherwise, the Replication Agent uses the source server's current transaction log position as its start position.
Enable immediate flushing of transaction log writes.
sync_log_writes <yes | no>
Default: yes
sync_log_writes yes
causes each write to the local transaction log to be flushed to disk.
system_event_log <machineName>
Default: not set
This option is only supported on Windows systems
When this option is specified, the replication agent writes its messages to the Windows event log in addition to the replication agent log file.
<machineName> is the name of the machine where you want to write event log messages. Specify localhost to write to the local system's event log.
Set the target server user connection information. Use the ctcmdset utility to generate a .set file from the .cfg file. The .cfg file is defined as follows:
USERID ADMIN
PASSWD ADMIN
target_authfile <targetAuthFileName>
Default: not set
Forces a required node ID for the target server. This is provided as an arbitrary dotted notation such as 1.1.1.1.
Note: This nodeid has no relationship whatsoever with any IP address or other network information regarding the source and target servers.
target_nodeid <dotted_notation>
Default: not set
See Also
Specify Required Node IDs for Source and Target
Set the target server connection information. The format is <servername>@<hostname.domainname>. Any qualified hostname, domain name or IP address that can be resolved by the server is allowed. The server name can also be specified as a direct IP address such as #5597@localhost.localdomain.
target_server <targetServerName>
Default: not set
target_tls_cert_file <certificateFileName>
Default: not set
Specifies the name of the certificate file to use when connecting to the target server using TLS. Requires target_use_tls yes
to be enabled.
target_use_tls <yes | no>
Default: no
target_use_tls yes
causes the connection to the target server to use TLS.
unique_id <replagentUniqueId>
Default: REPLAGENT
Set the unique ID of the Replication Agent for multi-server replication. The unique ID is case-insensitive, and is limited to 32 bytes. This ID is also referred to as the agent ID.
See Also
use_target_alignment <yes | no>
Default: yes
When use_target_alignment yes
is specified, the Replication Agent uses the alignment setting of the file on the target server when applying a record image that it has read from the source server to the target server. Otherwise, the client library's alignment is used.