Skip to main content

Install replication manager

Follow along with the following guides to download, configure, and run the replication manager.

The FairCom Replication Manager is distributed in a .zip or .tar package — for example, FairCom DB.<platform>.ReplicationManager.<build>.

  1. Extract the package onto your server file system.

    The package contains centralized replication components (the Replication Manager controller, database, and browser‑based graphical tools).

    Note

    It is possible to install and run Replication Manager on the same system where a FairCom DB, FairCom Edge, or FairCom RTG server is running, but you will need to change the https port in config/cthttpd.json (see Configuration).

  2. Edit the config/ctagent.json configuration files on the FairCom DB, FairCom Edge, and/or FairCom RTG server nodes to point to the Replication Manager.

  3. Ensure replication is enabled for each database server node by updating the licenses on the server nodes.

    Note

    FairCom DB, FairCom Edge, and/or FairCom RTG servers will need to be updated to the latest build.

  4. Once the Replication Manager package is downloaded, unzip the package.

  5. Observe the directory:

    • ReplicationManager

      This is the folder containing Replication Manager. The application to launch (memphis, or Memphis.exe on Windows) is contained in this directory.

    • ReplicationManager\web

      This is the browser-based versions of the graphical tools, including Replication Manager as well as FairCom SQL Explorer, FairCom ISAM Explorer, and FairCom Monitor.

Several features of replication depend on specific server configurations.

Important

If the server has both SERVER_NAME and SERVER_PORT configured, it will always use the SERVER_PORT, so even if it is a local machine connection, it will use TCP-IP instead of shared memory.

  1. Enable replication by inserting the following lines into the ctsrvr.cfg file:

    ; Plugins
    PLUGIN cthttpd;./web/cthttpd.dll
    PLUGIN ctagent;./agent/ctAgent.dll

    These lines enable the plugin that handles web communication (services.json) and the agent that performs the replication (ctagent).

  2. Use the ctagent.json and services.json files to configure the replication plugin:

    ctagent.json
    {
    	"embedded": true,
    	"log_file": "agentLog.txt",
    	"memphis_server_name": "MEMPHIS",
    	"memphis_sql_port": 7000,
    	"memphis_host": "localhost",
    	"memphis_database": "MEMPHIS",
    	"ctree_check_mask": "*.dat;*.idx;*.fdd;*.fsd",
    	"inactive_timeout": 600
    }
    services.json
    {
    	"listening_http_port": 8080,
    	"listening_https_port": 8443,
    	"ssl_certificate": "./web/fccert.pem",
    	"document_root": "./web/apps",
    	
    	"applications": [
    		"ctree;ctREST.dll",
    		"AceMonitor;ctMonitor.dll",
    		"SQLExplorer;ctSQLExplorer.dll",
    		"ISAMExplorer;ctISAMExplorer.dll",
    		"ReplicationManager;ctReplicationManager.dll"
    	]
    }
  3. Optionally enable diagnostic logging by adding the following lines to ctreplagent.cfg:

    diagnostics logship
    diagnostics logread

    The replication agent supports optional diagnostic logging for its log ship and log read threads. When diagnostic logging is enabled, these threads write messages to ctreplagent.log to indicate the offset that the log ship thread has written to the copy of the log files, and the log read thread is reading. The offset=[a,b,c] values in the log read message are (a) the requested log read offset, (b) the last byte actually read, and (c) the last requested byte to read. (b) and (c) are equal if the full amount was read (that is, bytesread is equal to size). We expect these log read offset values to always be less than the log ship's current write log position. These example messages show an expected situation: the last byte read by the log read thread is 0x789ab, which is equal to the last byte written by the log ship thread as shown in the "write to log copy position" message.

Use the repadm utility to turn log diagnostic options on and off

Turn on diagnostic logging for log ship and log read threads:

repadm -c setconfig:"diagnostics logship" -s replagent
repadm -c setconfig:"diagnostics logread" -s replagent

Turn off diagnostic logging for log ship and log read threads:

repadm -c setconfig:"diagnostics ~logship" -s replagent
repadm -c setconfig:"diagnostics ~logread" -s replagent

Example diagnostic messages:

Fri Jul 26 16:40:56 2024: AGENT1: Logship: DEBUG: LOGSHIPDIAG: write to log copy position 8 0x78179 (491897) through 0x789ab (493995)
Fri Jul 26 16:40:56 2024: AGENT1: Logship: DEBUG: LOGSHIPDIAG: set read log position to 8 0x789ab (493995)
Fri Jul 26 16:40:56 2024: AGENT1: Logship: DEBUG: LOGSHIPDIAG: set write log position to 8 0x789ac (493996)
  • A replication plugin runs in the FairCom server and executes all data replication.

  • If you are running Replication Manager on the same machine as the FairCom DB engine, you will need to adjust the ports so they do not conflict. Select unused values for "listening_http_port" and "listening_https_port". The ports available on your machine will depend on your environment. A good starting point is to simply add 1 to each of the port numbers listed in the configuration file.

  • If you want to experiment with replication in your lab, you can set up Replication Manager and two FairCom DB servers on the same machine. In that case, all three servers must use unique ports.

The FairCom server assigns a node ID to FairCom replication. The local/master replication scheme also assigns a node ID to connections that are established from the target server to the source server.

A local FairCom server can set its replication node ID by using the REPL_NODEID option in ctsrvr.cfg.

Example 1. A source server and two target servers
; source server configuration
SERVER_NAME MASTER
REPL_NODEID 10.0.0.1

; target server 1 configuration
SERVER_NAME LOCAL01
REPL_NODEID 10.0.0.2

; target server 2 configuration
SERVER_NAME LOCAL02
REPL_NODEID 10.0.0.3

ID values are arbitrary and do not need to match the IP address of the system on which the FairCom server is running. They only need to be unique and not change during replication. In this example, IP v4 addresses are demonstrated.

FairCom replication reads the node ID of the source and target servers. If the node ID is not set for a source or target server, FairCom replication uses the IP address of the system on which that FairCom DB is running.



Dynamically set replication node ID

A replication node ID value (REPL_NODEID) associates a replication agent to a specific source or target server. Having this required association prevents replication agents from arbitrarily connecting to inappropriate servers. Setting a REPL_NODEID required modifying the server configuration file, ctsrvr.cfg, and restarting a server, which is challenging in round-the-clock production environments.

The FairCom server now supports setting a REPL_NODEID value dynamically at runtime if it has not yet been set. This keyword sets the node ID of a local FairCom server for use in replication.

To set the value, use the ctadmn utility's option to change a server configuration option value using option 10 or call the ctSETCFG() function.

Example 2. ctSETCFG() function
NINT rc;

rc = ctSETCFG(setcfgCONFIG_OPTION, value);

Where value is the REPL_NODEID value to set, specified as a NULL-terminated string in IPv4 format (n.n.n.n.) — for example, "10.0.0.1" (ID values are arbitrary and do not need to match the IP address of the server).

Note

The value is only changed in memory and not persisted to disk. Be sure to update the configuration file such that the new value is persisted and enabled on the next server startup.



The FairCom server has an option allowing the replication log reader thread to read transaction log data into a buffer for enhanced performance. When a reader requests data that is in the buffer, a noticeable performance gain can usually be observed. This feature is on by default. The size of the replication log read buffer can be set using the configuration keyword.

Example 3. Configuration keyword
REPL_READ_BUFFER_SIZE 8192


Without this keyword, the default buffer size is 8 KB. A minimum buffer size of 512 bytes is enforced. A value of zero will disable the replication log read buffer entirely.

To aid in troubleshooting when first using this feature, an additional diagnostic keyword can be specified to check that the log data is correctly read.

Example 4. Diagnostic keyword
DIAGNOSTICS REPL_READ_BUFFER


Serial segment values

To support the replication of a file that contains an auto-incrementing SRLSEG index, FairCom server configuration options are available to control the effect of this value. These values are set in the respective FairCom server's configuration file.

Serial segment values:
  • REPL_SRLSEG_ALLOW_UNQKEY YES causes the FairCom DB server to allow an index that contains a SRLSEG segment to be the replication unique key.

    Note

    The SRLSEG mode of indexes provides for auto-incrementing values to become part of a data record. By default, this index precludes the ability to be the replication unique key index. However, the option is available.

  • REPL_SRLSEG_USE_SOURCE YES causes a FairCom server replication writer thread to fill in the serial number value from the source server when adding a record to a replicated file standard replication model.) This preserves existing serial numbering.

    Note

    The actual value of a replicated serial segment value is an application-dependent attribute of replication. To control if this number should be re-assigned or not during replication, this option is provided. By default, the serial segment value is re-assigned when applied to the target server.

  • REPL_SRLSEG_USE_MASTER YES causes a local FairCom server to fill in the serial number value from the master FairCom server when adding a record to a local replica (this keyword applies to the synchronous master/local replication model).

    Note

    The actual value of a replicated serial segment value is an application-dependent attribute of replication. To control if this number should be re-assigned or not during replication, this option is provided. By default, the serial segment value is re-assigned when applied to the target server.

IDENTITY values

SQL support frequently requires an additional IDENTITY field beyond the FairCom SRLSEG key mode, already reserved for ROWID values. Replicating IDENTITY values employs additional options.

IDENTITY values:
  • REPL_IDENTITY_USE_SOURCE YES causes the target FairCom server to use the identity field value from the source FairCom server when adding a record to a replicated file.

  • REPL_IDENTITY_USE_MASTER YES causes the local FairCom server to use the serial number value from the master FairCom server when adding a record to a local replica.

The error IMIS_ERR (958, data record is missing identity field value) is returned by a record add operation when the supplied record image does not contain the identity field value. This is an unusual case, however, could occur if the record image does not contain all the fields for the record.

Note

These options also do not address issues of bidirectional replication involving files containing SRLSEG and/or IDENTITY.

Transaction log requirements for Replication Agent

Note

This is a compatibility change for the 2017 release of the FairCom engine and later.

The FairCom replication thread for TRNLOG files reads transaction logs and informs the server of their minimum transaction log requirements.

The FairCom server now limits by default the number of active transaction logs that are kept for FairCom replication to 100. This default is to avoid potentially running out of critical drive space.

The MAX_REPL_LOGS <max_logs> configuration option controls this limit by controlling the maximum number of logs to be held specifically for replication.

FairCom server administrator guide:

These sections provide more information about the MAX_REPL_LOGS <max_logs> setting (and the related setting for deferred indexing logs).

KEEP_LOGS setting to keep inactive logs

To ensure the database engine always keeps enough logs to allow the log reader to process them before they are deleted, use the KEEP_LOGS configuration keyword.

Example 5. KEEP_LOGS configuration keyword
;Keep all transaction logs

KEEP_LOGS -1

Specify -1 to keep all logs (in which case old logs must be manually deleted when done), or specify a positive number that indicates the number of logs to keep. This number should be the largest number of logs required if the log reader is stopped or falls behind in processing logs.



A server configuration option has been added that allows you to change the log offset returned by the replication log scan for an operation that consists of multiple log entries, such as REPL_ADDREC. By default, the offset of the first log entry that comprises that operation is returned.

This configuration option makes it possible to change the behavior so that the offset of the last log entry that comprises the operation is returned.

REPL_USE_LAST_LOGPOS <YES | NO>
  • The default is NO.

  • This option can be changed at runtime with the server administrator utility ctadmn.

  • This option should be used when applications expect replication operations to be returned in ascending log offset order.

  • In addition to this option, the option COMPATIBILITY NO_REPL_DEFER_TRAN must be used to avoid reordering of transaction begin operations that are caused by deferring the return of the begin transaction operation until an operation of interest to the replication reader has been read from the log.

After unzipping Replication Manager into the desired folder and checking the configuration, it is ready to run.

  1. In the ReplicationManager folder, launch Replication Manager by executing memphis (Linux) or Memphis.exe (Windows).

    Note

    When a server that uses Replication Manager starts up, if it already has a UUID, it searches the list of servers that are registered with Replication Manager for an entry that matches its UUID. If the server does not have a UUID, or no entry in the Replication Manager list matches its UUID, it searches the list for an entry that matches its host name and local directory.

  2. Open your other c-tree servers (faircom.exectreesql.exectreeEdge.exe, or ctsrvr.exe).

    Important

    It is important that the c-tree server engines are started after the replication application is running, so the automatic registration is completed. You will know this step did not complete correctly if your c-tree server does not appear in the Replication Manager.

  3. In a web browser, enter a URL using this format https://hostname-of-your-server:8440/ReplicationManager/ — for example, https://localhost:8440/ReplicationManager/.

    Tip

    If the localhost domain name does not work, use the IP Address 127.0.0.1.

    If you get a security or certificate error, check the details and click the link that allows you to continue on to the web page.

  4. Observe the login page for Replication Manager with the username and password fields automatically filled in — for example, User: "admin", Password: "ADMIN".

  5. Once logged in, observe the interface described in this section.