Manual replication

Manual data replication using Replication Agent

Abstract

This section explains how to configure manual replication using Replication Agent.

This section explains how to configure manual replication using Replication Agent. Manual replication is intended for customers transitioning from existing FairCom replication using the first generation of the Replication Agent. It allows the latest FairCom replication to use existing configuration files already in place under previous FairCom releases.

Synchronizing data

To commence replication, desired replicated files must be first synchronized between systems. There are multiple methods to achieve this. Simple file system copies can be used as well as Hot Backups and restores which directly coordinate starting positions with replication. This section will describe the best options for your specific scenario.

Initial data synchronization

Note

When adding a server to a replicated system it is important to pause log purging on the source server before copying the files. This prevents activity on the source server while the files are being copied, which would cause any transaction logs to be deleted.

You will need to start the Replication Agent reading the source server's transaction log at the position that corresponds to the point in time where you copied the files. To accomplish this, you must ensure the source server keeps the transaction logs from the time you copied the files until you have started the new Replication Agent. Step 1 through Step 3 in the Preferred procedures show exactly how this is done. The Quick guide provides the basic steps.

Quick guide:

Stop the source server and target servers.
Copy the files.
Block all traffic to the source server at the system or network level.
Note
Any source activity at this point may break the point-in-time relationship between the copied files and the transaction logs, leading to inconsistent data between the source and target. Preferred procedures prevent this vulnerability.
Generate a ctreplagent_<agentid>.ini with entry #current.
Start the source server, the target server, and the Replication Agent.
Release traffic to the source server.

Preferred procedures:

The steps below explain the general approach to adding a server to a replicated system and starting it at the correct point in time.

When you are ready to make a copy of the files from the source server, first run ctstat ‑var on the source server.

Example 1. Run ctstat ‑var

# ctstat -var -h 1 -u ADMIN -p ADMIN -s source

name                               lowlog  curlog       curpos
source(SERVER)                          6       9      9237667

Observe and take note of the server's current log number (curlog) and log position (curpos).
In the Example 1, “Run ctstat ‑var” log 9 is the current log used by the server. It is recommended to set the minimum log to the previous log (in this case log 8).
Use the ctrepd utility with the ‑setlog and ‑unqid options to instruct the source server to keep the current log and later logs based on the log number determined in Step 2.
Example 2. ctrepd utility
If your new Replication Agent will use the ID MYREPLAGENTID and you want to set the minimum transaction log to 8, and the source server is named source, use this command.
This will cause the source server to keep log 8 and later logs until your new Replication Agent has been started and has processed that log's operations.
```
# ctrepd -unqid:MYREPLAGENTID -setlog:8 -s source

name                               lowlog  curlog       curpos
source(SERVER)                          6       9      9237667
-MYREPLAGENTID                          8       0            0 
```
Create a point-in-time copy of the files from the source server using either of the following options:
- Dynamic Dump
  Use dynamic dump to make a copy of the files from the source server, and then restore them on the target server using the ctrdmp utility. The utility creates the file ctreplagent_<agentid>.ini containing the log number and log position for your new Replication Agent to use. This approach has the advantage that the source server is paused only for a brief time while it reaches a quiet transaction state. The server can then resume normal operations while the files are copied to the dynamic dump backup file.
- Quiesce & Copy
  Use the ctQUIET() function (for example through the ctadmn utility) to pause the source server and close the files. Run ctstat ‑var on the source server and note the server's current log number and log position after you have paused the source server. Make a copy of the files using a system copy command, then resume server operation. In the Replication Agent's working directory, create ctreplagent_<agentid>.ini as a text file containing the following two lines:
```
log_number log_offset
log_number log_offset 
```
  Example 3. Two lines in the ctreplagent_<agentid>.ini file
  If your source server is currently on log 35 at offset 12345678, create ctreplagent_<agentid>.ini containing the following two lines.
  35 12345678 35 12345678
Copy the files created in Step 4 from the source server to the target server.
Note
If you used Dynamic Dump in Step 4, copy the ctreplagent_<agentid>.ini file from the dump restore area to the Replication Agent's working directory.
If you used Quiesce & Copy in step Step 4 , create ctreplagent_<agentid>.ini in the Replication Agent's working directory.
Start the Replication Agent.
Note
The Replication Agent looks for the ctreplagent_<agentid>.ini file in its working directory when it starts up. If it exists, it sets its position to the position specified in the file. It then deletes the ctreplagent_<agentid>.ini file so it does not use that position again the next time it starts. If you used Dynamic Dump in Step 4, the Replication Agent will find the correct starting point in the ctreplagent_<agentid>.ini created by the dynamic dump restore that you copied to the Replication Agent’s working directory.
Using ctstat ‑var on the source server, observe that the Replication Agent is now connected and is progressing in processing the transaction logs.

Manually copy files

Copy files manually when appropriate. As long as the source and target servers are not operational and data and index files are in a clean state they can be copied with any OS utility such as rsync, or compressed into an archived, transferred, and uncompressed on the remote target. When copying in this manner it is assumed that replication will be commenced from a clean starting point. It is important to remove all existing REPLSTATEDT.FCS files from the target.

Alternatively, creating a ctreplagent_<agentid>.ini file with a single #current entry will force replication to commence with the first entry added to existing source transaction logs.

Manually synchronize with Hot Backups

The FairCom DB Dynamic Dump restore process automatically generates a ctreplagent_<agentid>.ini starting file after a successful restore. The position in ctreplagent_<agentid>.ini is in sync with the position at which time the restored files were originally backed up.

Create a Dynamic Dump script to back up files for replication.
Stop the secondary server and the Replication Agent if already running.
Execute the Dynamic Dump at the desired point in time on the primary server.
Tip
Use the !DATE and !TIME options to automatically schedule your dump.
Using the same dynamic dump script used to back up the files, restore the files on the secondary server with the ctrdmp utility.
Tip
Consider the !REDIRECT option for positioning files if different directory structures are different between the servers.
Copy the ctreplagent_<agentid>.ini file into the Replication Agent's working directory.
Restart the secondary server with the data now in sync with the primary server.
Restart the Replication Agent.
Replication then commences from the log position on the primary server when the files were initially backed up, replicating changes since the backup and quickly synchronizing the secondary server with the primary.

Keep the required transaction logs

Once the Replication Agent starts, it will register its log requirements with the source server (if using the default agent configuration, remember_log_pos) and from that point on the source server should maintain enough logs for that Replication Agent to continue from its position when it restarts — for example, if a replica went offline for a day, all the logs from the time it stopped will be kept by the source server.

However, before a particular Replication Agent is first connected to a source server (only the very first time) you need to ensure that the transaction logs from the time of your dynamic dump are maintained until you get the Replication Agents started. You can do this by starting and then stopping each Replication Agent prior to making the dynamic dump. This will establish a minimum log requirement associated with that Replication Agent (reader) unique ID so the source server does not discard any required transaction logs until the Replication Agent has consumed them.

If you have already had these agents running against the source server and are simply changing target servers, then you can ignore this since they will already have established a minimum log requirement.

Synchronize with `repadm resync`

Ideally, the target file in a replication environment would never be out-of-sync with the source. In some situations, the target server or the Replication Agent goes down or loses its connection. When all parts of the system are up again and the connections are reestablished, the Replication Agent is able to catch up with all the updates that must be applied from the moment that it stopped replicating. Although it may take some time to apply the missing updates from the backlog, it should not miss any. Unfortunately, some problems could happen in this process, making it possible that the replica file might get out of sync.

Problems that may occur causing the replica file to be out of sync:

A lost transaction log file from the source side was not applied to the target, making it impossible to bring the target copy back in sync.
A conflict while applying the modification to the target (perhaps an inadvertent update to the target while replication was inactive).

The replication resync feature provides a way to stop replication on one or more files in a replicated environment and resynchronize them. It copies current files from the source server to the target server and then restarts the replication of these files.

The administrator can use repadm -c resync and repadm -c resyncclean to manage the resync operation. The new modes allow the user to indicate the name of the source file to be synchronized. A text file with a list of source file names can be provided in case of multiple files to be synchronized. Based on the source file name, the Replication Agent identifies the target file based on the existing redirect rules.

Enable manual replication mode from `ctagent.json`

Note

FairCom replication support is backward compatible with replication deployments prior to V12. Legacy replication configurations can continue to be used independently of Replication Manager, which may be desirable in scenarios such as an application using a predefined replication setup.

`ctagent` plugin

The 2020 release of the FairCom Database Engine introduces plugins, which run applications inside the database process. The ctagent plugin provides the built-in replication feature of each FairCom server. It replaces previous replication components. All original Replication Agent process functionality is now encapsulated into this advanced multi-threaded server-side module.

Replication models

Three models of replication are supported:

Note

All modes are available in V12 FairCom servers and newer as well as the Replication Manager for extended flexibility of your replication needs.

Managed
FairCom Replication Manager is the default FairCom replication support. This model takes advantage of a centrally managed replication database for persisting replication plans containing publications and subscriptions. Details for this method are available in the Replication Manager guide.
Manual Replication Manager Remote Mode
In this model, Replication Manager can be deployed and alternatively used exactly as the prior Replication Agent model for existing environments as they migrate to full Replication Manager support. That is, it can be located on any node in your replication network topology. Using Replication Manager in this mode is a smart transition model to consider. This model does not persist replication plans, publications, and subscriptions in the central database. Replication is configured exactly as before in the external ctreplagent.cfg files. However, you gain the benefit of having new replication support deployed and ready to tackle new replication challenges.
Manual Local Mode
This model enables local direct server replication based on the previous Replication Agent ctreplagent.cfg configuration. This mode is much like prior FairCom replication support, however, without a separate Replication Agent executable required. Replication functionality is provided within servers themselves from the ctagent plugin simplifying deployment and configuration.

Configure `ctagent` plugin for manual mode

Each FairCom database plugin, including ctagent, is configured with its own configuration file in JSON format for ease of use.

Manual replication mode refers to using existing ctreplagent.cfg configurations, that is, not managed via <FC_PROD_REPLCIATION_MGR>. A simple JSON format configuration in ctagent.json points to existing configuration files for immediate transitioning to this model.

Note

In manual mode, all replication-related operations (definition reads and persistence) are skipped. From a configuration standpoint, in general, all paths are relative to the configured ctagent server’s executing process directory.

Only two configuration elements are required to be defined:

"managed": < true | false >
The default is true, which is the default Replication Manager usage. false enables manual mode. When false, the ctagent plugin doesn't connect to a central replication database and executes within the process in which it is configured.
Note
In case of false, or omitted, even if the replication database is not available it keeps trying to reconnect automatically from time to time.
"configurationFileList": [ <cfg file1>, <cfg file>, and so forth ]
The default is an empty array. Each configuration filename element can be a full path or relative to the server working directory. Each element creates one extra instance of a Replication Agent thread to be started from the server ctagent plugin. No central replication database connection is necessary for starting these instances, however, if ctagent is configured within the Replication Manager in this manner, it is also able to start these separated ctreplagent.cfg instances coexisting with managed instances managed from replication definitions for total flexibility.

Example

Given all classic legacy files are located in a replication folder alongside config and server folders, consider the following manual ctagent configuration.

{
	"managed": false,
	"configurationFileList":[
		"../replication/ctreplagent1.cfg",
		"../replication/ctreplagent2.cfg"
	]
}

As with prior replication, authentication files are required in manual mode. In ctreplagent.cfg, source_authfile and target_authfile files should include a path relative to the main server process.

; Source server connection info
source_authfile ../replication/source_auth.set
source_server FAIRCOMS@localhost
; Target server connection info
target_authfile ../replication/target_auth.set
target_server FAIRCOMT@localhost

On success, you should find the following messages logged to CTSTATUS.FCS.

Wed Oct 28 15:49:20 2020
 - User# 00026	ctagentd - Starting Agent
Wed Oct 28 15:49:20 2020
 - User# 00026	ctagentd - Agent is detached from Memphis...
Wed Oct 28 15:49:20 2020
 - User# 00026	ctagentd - Agent plugin is ready

The replication status log (ctreplagent.log) is located in the main server folder unless specifically configured by the log_file_name option in ctreplagent.cfg. This is essential when more than one replication agent is configured such that each replication log is separately maintained.

Fri Nov 13 09:15:28 2020: INF: ** Starting replication agent logging for process 9616
Fri Nov 13 09:15:28 2020: INF: Using replication change buffer version 2
Fri Nov 13 09:15:30 2020: INF: Connected to data target 'FAIRCOMT@localhost'.
Fri Nov 13 09:15:30 2020: INF: Connected to data source 'FAIRCOMS@localhost'.
Fri Nov 13 09:15:30 2020: INF: Successfully set file filter using XML file '<..\config\replication\filefilter.xml'
Fri Nov 13 09:15:30 2020: INF: Started replication agent.
Fri Nov 13 09:15:30 2020: INF: Starting scan with log 1, position 65536

Replicated file definitions

Replication support previously required specific server configuration entries in ctsrvr.cfg defining exactly what files are eligible for replication and allowed wildcard support for flexibility. However, this required server downtime making it an administrative exception task.

Replicated file definitions can now be added directly to your ctreplagent.cfg file, avoiding downtime on production servers. This allows for easier configuration and ad hoc modifications. An XML-based definition format is used. A file filter definition is used to define replicated files. This can be embedded directly in ctreplagent.cfg, or referenced from an external XML file.

File filter definition

A replication file filter contains a set of <file> tags. Each <file> tag indicates if the specified filename is included by or excluded from replication.

In the following example, the replication agent using the replication file filter behaves as follows:

a) Reads and applies replicated operations for all files named *.dat in the prod directory and its subdirectories,

b) Ignores replicated operations for all files named *.dat in the prod/temp directory and its subdirectories,

c) Reads and applies replicated operations for all files named *.data, *.dat, and *.db in the ctreeSQL.dbs directory and its subdirectories.

<?xml version="1.0" encoding="us-ascii"?>
<replfilefilter version="1" recurse="y" persistent="y" agent="REPLAGENT">
	<file status="include">prod/*.dat</file>
	<file status="exclude">prod/temp/*.dat</file>	
        <file status="include" regexPath=".\ctreeSQL.dbs">.*((data|dat|db))</file>
	<purpose>create_file</purpose>
	<purpose>read_log</purpose>
</replfilefilter>

Each <file> entry takes the place of a REPLICATE ctsrvr.cfg entry.

Note

The use of expression filters for the file descriptions. Regular expressions allow for detailed and complex file specifications in a single entry.

This XML entry is referenced in your ctreplagent.cfg file with the file_filter configuration option.

 file_filter <../config/replication/filter1.xml

Note

The use of a single < character preceding the filename is intended. Do not include a closing > or you will receive an error.

The <purpose> element defines operations available for this replication usage, including:

create_file
read_log
sync_commit

Manual Replication Agent example

This section outlines procedures to configure replication between two FairCom servers running on the same machine. The following procedure includes basic steps for setting up Classic Replication Agent from the FairCom DB V12 package (these same procedures apply for both FairCom Edge and FairCom RTG packages).

Extract the FairCom DB V12 package (FairCom-DB.<platform>.v12.<version>.<tar.gz or zip>) on the target machine.
Check the following directory tree:
Rename config/ctagent.json to config/ctagent.json.old to cause the original file to be ignored by the server.
Note
This can easily be reverted.
Copy config/replication/ctagent.json to config/ctagent.json.
Edit config/ctagent.json to leave only one ctreplagent.cfg in the configurationFileList.
```
{
    "managed": false,
    "configurationFileList": [
    "../config/replication/ctreplagent1.cfg"
    ]
}
```
Note
The ctreplagent.cfg path is relative to the server’s working directory.
Edit config/replication/ctreplagent1.cfg and set both source_server and target_server to reflect the SERVER_NAME from ctsrvr.cfg and the IP address of the machine that the FairCom server is running on.
```
; Target server connection info
target_authfile ../config/replication/target_auth.set
target_server FAIRCOM2@127.0.0.1
; Source server connection info
source_authfile ../config/replication/source_auth.set
source_server FAIRCOM1@127.0.0.1
```
Note
- If using SERVER_PORT instead, the format is <port number>@<network address>. The # indicates that the next part will be a number, instead of a string.
- If authentication information has changed, you will need to generate new .set files (see ctcmdset - Configuration File Encoding Utility).
Edit the config/replication/filter1.xml file, which is referenced in the following keyword in ctreplagent1.cfg: file_filter <../config/replication/filter1.xml.
If using synchronous replication, add the option syncagent yes to the ctreplagent.cfg replication agent configuration file.

Update the replication filter rule for replicating all the data files in the default ctreeSQL database path.

<?xml version="1.0" encoding="us-ascii"?>
<replfilefilter version="1" persistent="y" agent="REPLAGENT">
    <file status="include">.\ctreeSQL.dbs\*.dat</file>
    <purpose>create_file</purpose>
    <purpose>read_log</purpose>
</replfilefilter>

Note

The "include" path is relative to the "data" directory on the source server.

Edit the config/ctsrvr.cfg to enable the ctagent plugin by removing the ; in front of the following keyword.
```
PLUGIN ctagent;./agent/ctagent.dll
```
Start the target server.
Just after initializing the regular FairCom DB server, the ctagent plugin will be initialized and the embedded Replication Agent configured in the config/ctagent.json file will be started.

Check the following messages in the data/CTSTATUS.FCS log file.

Wed Nov 11 17:53:12 2020
 - User# 00025	ctagentd - Starting Agent
Wed Nov 11 17:53:12 2020
 - User# 00025	ctagentd - Agent is not managed by Replication Manager...
Wed Nov 11 17:53:12 2020
 - User# 00025	ctagentd - Agent plugin is ready

Check the FairCom DB log, which is referenced in the following keyword in ctreplagent.cfg: log_file_name ../config/replication/replagent1.log.

It should present messages saying that it was able to connect to both source and target servers.

Wed Nov 11 17:53:12 2020: INF: ** Starting replication agent logging for process 21356
Wed Nov 11 17:53:12 2020: INF: Using replication change buffer version 2
Wed Nov 11 17:53:12 2020: INF: Connected to data target 'FAIRCOMS2@localhost'.
Wed Nov 11 17:53:12 2020: INF: Connected to data source 'FAIRCOMS@localhost'.
Wed Nov 11 17:53:12 2020: INF: Successfully set file filter using XML file '<../config/replication/filter1.xml'
Wed Nov 11 17:53:12 2020: INF: Started replication agent.
Wed Nov 11 17:53:12 2020: INF: Starting scan with log 1, position 65536

Using SQL Explorer (or any other means), create a new table in the source server’s default ctreeSQL database.
Note
- Replication will cause the same file to be created in the target server’s ./data/ctreeSQL.dbs/ directory.
- In this test, the database operations of adding the new physical files into the database dictionaries are not replicated. The files will be physically created, but they must be manually added to the dictionary.

`replfilefilter`

File definitions can be specified by an external XML definition. This allows dynamically enabling replication without stopping and starting servers for configuration changes. This file can be set in ctreplagent.cfg and changed at runtime using repadm -c changefilter and -c removefilter with -x option indicating the external filter file name. repadm requests the replication agent to perform the specified filter operation and the replication agent applies the filter to the source server if the purposes indicate that it applies to the source server.

Example 4. XML file filters definition

<?xml version="1.0" encoding="us-ascii"?>
<replfilefilter version="1" persistent="y" agent="REPLAGENT">
	<file status="include">mark.dat</file>
	<file status="exclude">donotreplicate*.dat</file>
	<file status="include" regexPath=".\ctreeSQL.dbs">.*((data|dat|db))</file>
	<purpose>create_file</purpose>
	<purpose>open_file</purpose>
	<purpose>read_log</purpose>
	<purpose>sync_commit</purpose>
</replfilefilter>

Attributes

Table 1. Attributes

Attribute

Description

Type

Supported values

Defaults

agent="<agentid>"

specifies the unique ID of the replication agent to which the replication file filter applies. Using this option makes it possible for a server to have more than one replication file filter in effect when more than one replication agent is using its own replication file filter.

string

0 to 31 bytes

""

(empty string)

persistent="y"

specifies the persistent mode. If set to "y", the filter definition on the source persists such that it is automatically loaded on startup.

string

"y"

"n"

recurse="y"

specifies recursive directory matching for wildcard filenames or not.

The supported values are "y" (the default), and "n". A value of "y" causes all wildcard filenames specified in the replication file filter to match subdirectory names. For example, the name C:\data\*.dat matches the file name C:\data\subdir1\mark.dat.

string

"y"

"n"

"y"

update="0"

specifies the file filter update process. If set to "0", this file filter replaces the existing file filter if it exists. If set to "1", the current replication file filter is updated with this file filter's definitions.

string

"0"

"1"

"0"

version="1.0"

specifies the XML format version defining the current specification.

String

A value containing a version number in the format "major.minor"

Required - No default value

Elements

Table 2. Elements

Element	Description
`<file>`	specifies options for listing files to include or exclude from replication supporting regular expressions Note The `regexPath` tag specifies an optional path as a regular expression and is applied to only the file name without a path. The following tags are supported: regexPath: (optional) If used, regexPath causes the filename to be split into its path and its filename without the path. The path is compared to the specified directory name and the filename without its path is compared to the regular expression.If regexPath is not specified, the filename can be either a literal filename or a filename that includes wildcard characters supported by FairComDB (? matching a single character or * matching zero or more characters). status: (required) either status="include" to treat the specified file name as replicated or status="exclude" to exclude the specified file name from replication.
`<purpose>`	indicates the purposes of the filter.
`create_file`	specifies automatic replication on files that the source server creates. It is similar in its effect to specifying the `REPLICATE <filename>` option in the source server configuration file, but it is more flexible because the file filter can be changed at runtime, without restarting the source server. When the filter `<purpose>` includes the `create_file` option, the replication agent sends the file filter to the source server. Each call to create a file on the source server uses the file filter to check if the file is included as a replicated file (in addition to any REPLICATE options specified in the source server's configuration file). If the name of the file being created is included in the file filter and the file does not have replication enabled, the source server enables replication on the file.
`open_file`	specifies automatic replication on files that the source server opens. It is similar in its effect to specifying the `REPLICATE <filename>` option in the source server configuration file, but it is more flexible because the file filter can be changed at runtime, without restarting the source server. When the filter `<purpose>` includes the `open_file` option, the replication agent sends the file filter to the source server. Each call to open a file on the source server uses the file filter to check if the file is included as a replicated file (in addition to any REPLICATE options specified in the source server's configuration file). If the name of the file being opened is included in the file filter and the file does not have replication enabled, the source server enables replication on the file.
`read_log`	specifies that the file filter is to be used when reading transaction log entries from the source server. When the file filter includes this purpose, the replication agent reads and applies only the replicated operations for the files whose names match an include rule in the file filter and do not match an exclude rule in the file filter. When the file filter does not include this purpose, the replication agent reads and applies the replicated operations for all files in the source server's transaction logs. When the replication agent uses serial replication, the log read filtering occurs on the source server, and so the replication agent sends the file filter to the source server. When the replication agent uses parallel replication, the log read filtering occurs on the server in which the replication agent is running, because parallel replication reads replicated operations from a copy of the source server's transaction logs.
`sync_commit`	specifies that this replication agent is synchronously replicating the files that are specified in the file filter. When the filter contains the `sync_commit` purpose, the replication agent sends the filter to the source server so that it can ensure proper synchronous commit behavior for all transactions that change a replicated file whose name is specified in the file filter. Note When using this option, add `"syncagent yes"` to your `ctreplagent.cfg` replication agent configuration file.

Manual replication

Manual data replication using Replication Agent

Synchronizing data

Initial data synchronization

Note

Note

Note

Note

Manually copy files

Manually synchronize with Hot Backups

Tip

Tip

Keep the required transaction logs

Synchronize with repadm resync

Enable manual replication mode from ctagent.json

Note

ctagent plugin

Replication models

Note

Configure ctagent plugin for manual mode

Note

Note

Example

Replicated file definitions

File filter definition

Note

Note

Manual Replication Agent example

Note

Note

Note

Note

Note

replfilefilter

Attributes

Elements

Note

Note

Synchronize with `repadm resync`

Enable manual replication mode from `ctagent.json`

`ctagent` plugin

Configure `ctagent` plugin for manual mode

`replfilefilter`