Skip to main content

C++ Replication API

Concepts for using the C++ Replication API

Abstract

Concepts for using the C++ Replication API

Table 1. Sections

Section

Description

JSON Replication API

This section summarizes the JSON Replication API.

C++ interface

This section lists the formations of the C++ interface.

File system and database classes

This section lists classes that depict the file system and its hierarchy.

Replication classes

The replication plan is the link between two database engines: the source and the target.

Class relationships

This section shows a figure that summarizes classes and their relationships.

Configuration

The replication API requires a running Replication Manager server. You will find this server in the Replication Manager package. Its executable name is called Memphis as a metaphor for a central management hub.

Tutorial for the C++ API

Two tutorials are provided to assist you in learning to use the Replication API.

REST and C/C++ error codes

Error codes contained in fcRepl.h.



Table 2. The Replication Manager API defines a variety of classes (see Class relationships for the hierarchy).

Section

Description

Action

An action can be pushed from the central point of control to any node in the site and executed by that node.

Agent

Each machine in the Site must have at least one Replication Agent running to be included in the site.

Connect/disconnect pool

This class provides connection management.

Database

Each DBEngine can host one or more databases.

DBEngine

Each OpSystem (operating system) may contain one or more database servers.

File

Volume may contain one or more files. Files and folders are both considered to belong to the File class.

Hardware

Each Site may contain one or more physical devices, including computers, tablets, phones, and so forth that are referred to as hardware.

Log

This class is used to retrieve a log or a Trace.

OpSystem

Each piece of physical Hardware may contain one or more operating systems — for example, a single computer could have a virtual machine with Windows installed and another running Linux.

Publication

A publication is a list of Files to be replicated.

replication plan

A replication plan defines which files are being replicated.

Site

The site class represents all the Hardware that is eligible for replication.

Subscription

A sSubscription contains one Publication and indicates the target where the published files are to be replicated.

Trace

This class allows you to manage debug and tracing.

Volume

Each OPSystem (operating system) can host one or more volumes (a volume may contain one or more Files).



The FairCom replication solution uses distributed architecture based on centralized metadata:
  • The system is distributed, with a FairCom DB database and Replication Agent on each node.

  • The metadata about the system is centralized in the Memphis database.

This means that any point on the system that can connect to the centralized metadata can access and control any part of the system.

Figure 1. Replication Manager
Replication Manager


The C++ API allows you to access and control the whole system by connecting to it through the Replication Manager. The replication API is a programmatic interface to the centralized replication metadata in the Memphis database.

The Replication Manager uses the API to manage the Replication Agent on each node. Using the API, developers can integrate the same functionality provided by the Replication Manager into their own custom applications.

The FairCom DB development environment includes the JSON Replication API as well as the C++ interface based on this API.

The architecture of the JSON RPC API defines a hierarchy of objects that represent the various components of a replication environment, such as physical hardware, operating systems, volumes, folders, files, as well as database servers, databases, and so forth.

The C++ interface is formed by:

Note

Both are documented in the header files in the \ctreeApps\FcFatDB\source\fcRepl directory.

  • Class definitions

  • High-level function headers

Figure 2. File system and databases classes
File system and databases classes


The following classes depict the file system and its hierarchy, which provides a way to uniquely represent each file:
  • Site

    At the highest level, the site class represents all the systems eligible for replication. A system must have a Replication Agent installed to be eligible for replication. The site includes every replication-eligible system that is connected via local networks, remote networks, the cloud, and so forth.

  • Hardware

    Each site may contain one or more physical devices, such as computers, tablets, phones, and so forth. These devices are referred to as hardware.

  • OpSystem

    Each piece of physical hardware may contain one or more operating systems, referred to as OpSystems. For example, a single computer could have a virtual machine with Windows installed and another VM running Linux.

  • DBEngine

    Each OpSystem may contain one or more database servers. In this release, only FairCom DB database servers are recognized.

  • Database

    Each DBEngine can host one or more databases. The FairCom replication APIs recognize both c-treeDB and FairCom DB SQL databases.

  • Volume

    Each operating system can host one or more volumes. A volume is a logical drive with a single file system. It may refer to a physical drive (fixed or removable) or a mount point within another volume.

  • File

    A volume may contain zero or more files or folders. The terms file and folder are used interchangeably in this API as files and folders are both considered to belong to the file class.

Figure 3. replication plan
replication plan


The replication plan is the link between two database engines: the source and the target. The FairCom replication APIs use a publish and subscribe metaphor for defining which files will be replicated from the source to the target.

Note

Two-way replication is supported, in which case the data is replicated in both directions (from the source to the target and from the target to the source).

Defined classes:
  • Publication

    A publication is a list of files to be replicated. The files are defined using the hierarchy of file system classes defined above. These files will be the source for replication.

  • Subscription

    A subscription contains one publication and indicates the target where the published files are to be replicated.

  • replication plan

    A replication plan defines which files are being replicated. It can contain one or more subscriptions and, therefore, defines the replication target (the subscription) and the source (the publication used by the subscription).

Figure 4. Class summary
Class summary

This figure summarizes the classes and their relationships.



In addition to the classes shown above, several secondary classes are defined. These classes are used primarily internally. A complete list of classes and functions is contained in all the header files in the \repl\include\fcRepl directory.

The Replication API requires a running Replication Manager server that is in the Replication Manager package. Its executable name is called Memphis as a metaphor for a central management hub. Similarly, all methods in the API are prefixed with ctMemphis. Memphis is a synonym for Replication Manager.

Servers on a network must register with Memphis to be replicated.

Connection management

The Replication Manager normally communicates to registered servers by using their hostname or list of IP addresses retrieved from the local network adapters. There are cases, depending on the network topology, that both Replication Manager and the other servers need to use some specific IP address to connect. The DBENGINE_IP_ADDRESS keyword found in the ctagent.json file facilitates this feature. 

Replication Manager has an automated process for connecting with registered servers.

  • All IP addresses of servers that can be identified in a local network are automatically detected and listed in a DBEngine table.

  • The list is used to connect servers with the Replication Manager server. When a connection is made, the Replication Manager checks to see if the agent server is registered.

  • If a registered server has specified a dbengine_ip_address in its ctagent.json file, that IP address replaces the original in the DBEngine table.

JSON Replication API

This feature impacts ctMemphisGetDBEnginesctMemphisPersistDBEnginectMemphisGetReplPlansDetailsctMemphisGetArchivedReplPlansDetails, and ctMemphisGetDBEnginesByGroup web APIs.

C++ API

This feature also impacts the FCREPLDBEngine class with the extra Getters and Setters for this information.

Two tutorials are provided to assist you in learning to use the Replication API. Tutorials are provided for both the JSON Replication and C++ APIs.

These two tutorials are in the following directories:
  • drivers\cpp.replication\

    This folder contains the tutorial for use with C or C++.

  • drivers\java.rest.replication

    This folder contains the tutorial for the Replication JSON Replication API.

These tutorials provide examples of how to use these APIs. They include sample projects that you can compile and run. After compiling the projects, you will start the servers and define the source and target folders that will be used for replication. You can run the projects in debug mode and add breakpoints so you can see how the various calls are used. Additionally, you can review the source code to study the use of the APIs. Procedures for these tutorials are provided in this section.

Requirements:
  1. A tutorial is included in the cpp.replication or java.rest.replication folder found in the <faircom>\drivers folder. Compile and run this tutorial to create a simple replication environment.

  2. Ensure you have already installed the Replication Manager package for your desired platform.

  3. Navigate to the Replication Manager directory and start a Replication server to create an empty replication database.

  4. Start two FairCom servers and create source and target directories for replication.

  5. Locate the source provided in the tutorial directory.

Compile and run the C++ high-level API example:
  1. Compile the tutorial using BuildTutorial.bat (BuildTutorial shell file or makefile on Linux platforms).

    Note

    • BuildTutorial.bat needs to be run from the Visual Studio x64 Native Tools command prompt unless vcvars64.bat is run prior to running BuildTutorials.bat.

    • Use the x64 platform.

    • The only DLL linked for the API is fcRepl.dll. The apiTestCPP.exe has dependencies on several DLLs (fcRepl.dllRCESDPCtree.dllRCESBasic.dllctReplAgent.dllctExtFileFilter.dll).

  2. Start the replication server (memphis.exe on Windows and memphis on Linux) from the Replication package to create an empty replication database.

  3. Prepare two FairCom servers to serve as the source server and the target server.

  4. Ensure the following two lines (after "; Plugins") are uncommented in ctsrvr.cfg.

    ; Plugins
    
    PLUGIN cthttpd;./web/cthttpd.dll
    
    PLUGIN ctagent;./agent/ctagent.dll
  5. Start servers 1 and 2 to populate both DBEngines into Replication Agent.

  6. Create a source data directory on the source machine — for example, c:\temp.

  7. Create a target data directory on the target machine — for example, c:\temp2.

  8. Place some FairCom files that qualify for replication in the c:\temp source directory.

  9. Execute (or run in debug) apiTestCPP.exe.

    Note

    This step uses the masterAuthFile.set authentication file with both user and password encrypted to connect to the replication server.

    1. Select one server from the provided list as the source DBEngine.

    2. Enter the source data path as c:\temp.

      Note

      The path must match the directory name made in Step 6.

    3. Ensure that the files placed in the source data directory are correctly listed.

    4. Select another server from the provided list as the target DBEngine.

    5. Enter the target data path as c:\temp2.

      Note

      The path must match the directory name made in Step 7.

    6. Ensure that the files placed in the target data directory are correctly listed.

  10. Open the graphical interface (https://localhost:8440/ReplicationManager/) in a web browser to view what has been executed.

Table 3. Error codes contained in fcRepl.h

Symbolic

Code

Description

FCREPL_SUCCESS

0

Success

FCREPL_EOF

1

Cursor is EOF (end of file)

FCREPL_DUPKEY

2

Key value already exists

FCREPL_NOTFOUND

100

Record not found

FCREPL_DISCONNECTED

129

Disconnected server

FCREPL_SERVERNOTFOUND

133

Unable to connect to server

FCREPL_SHUTDOWN_ERR

150

Shutdown error

FCREPL_TIMEOUT_ERR

156

Timeout error

FCREPL_BATFINISHED

428

No more info (batch canceled)

FCREPL_NOTSUP_ERR

454

Feature not supported

FCREPL_WARNING

9000

Warning

FCREPL_CONFIG_ERR

9001

Could not initialize the configuration object

FCREPL_LOGGER_ERR

9002

Could not initialize the logger object

FCREPL_NOTIF_ERR

9003

Could not initialize the notification object

FCREPL_FILESYSTEM_ERR

9004

Could not initialize the file system object

FCREPL_PROVIDER_ERR

9005

Could not initialize the data provider object

FCREPL_INVARG_ERR

9006

Invalid argument count

FCREPL_INVARGVAL_ERR

9007

Invalid argument value

FCREPL_MISARG_ERR

9008

Missing required argument

FCREPL_HASHFULL_ERR

9009

Hash table exceeded the max size

FCREPL_MEM_ERR

9010

Memory allocation error

FCREPL_QUEUEFULL_ERR

9011

Queue exceeded max size

FCREPL_INVACTION_ERR

9012

Invalid notification action

FCREPL_QUEUE_ERR

9013

Queue is not initialized

FCREPL_MUTEX_ERR

9014

Mutex is not initialized

FCREPL_PROCTHRD_ERR

9015

Mutex has not started the process thread

FCREPL_ACTION_ERR

9016

Invalid action

FCREPL_DB_TYPE_ERR

9017

Mismatch provider DB type

FCREPL_LOCALCT_ERR

9018

Could not load local c-tree libirary

FCREPL_SESSION_ERR

9019

Could not allocate c-tree session

FCREPL_INVFLD_ERR

9020

Invalid field number

FCREPL_SCHEMATHRD_ERR

9021

Could not start schema update thread

FCREPL_INTERN_ERR

9022

Internal error

FCREPL_REPL_ERR

9023

Could not initialize the replication object

FCREPL_INVREPLOBJ_ERR

9024

Invalid replication object

FCREPL_STARTED_ERR

9025

Already started

FCREPL_CONSUMER_ERR

9026

Consumer not found

FCREPL_ALREADYEX_ERR

9027

Already exists

FCREPL_INVFILTYP_ERR

9028

Invalid file type

FCREPL_INVFILVOL_ERR

9029

Invalid file volume

FCREPL_REPLNDEP_ERR

9030

Replication plan is not deployed

FCREPL_MISAUTH_ERR

9031

Missing authentication information

FCREPL_FTRNSFR_ERR

9032

File transfer not initialized

FCREPL_REPLACT_ERR

9033

Replication plan is not stopped

FCREPL_REPLNFOUND_ERR

9034

Replication plan is not found

FCREPL_INUSE_ERR

9035

Entity is in use

FCREPL_MISREPL_ERR

9036

Missing replication plan error

FCREPL_MISDB_ERR

9037

Missing database in subscription

FCREPL_MISVOL_ERR

9038

Missing volume information

FCREPL_FNOTFOUND_ERR

9039

File not found error

FCREPL_INVSUBS_ERR

9040

Invalid subscription

FCREPL_INVPATH_ERR

9041

Invalid file path

FCREPL_INVFUNC_ERR

9042

Invalid function

FCREPL_INVOS_ERR

9043

Invalid opSystem

FCREPL_NOTRUN_ERR

9044

Thread is not running

FCREPL_NACTIVE_ERR

9045

replication plan is not started

FCREPL_MISLPOS_ERR

9046

replication plan is missing log position

FCREPL_NCONN_ERR

9047

Agent is not connected

FCREPL_INVTOK_ERR

9048

Invalid token (wrong or expired)

FCREPL_DBINUSE_ERR

9049

DBEngine is in use

FCREPL_REPLAGENT_ERR

9050

Replication agent error

FCREPL_MASTERNOTRDY_ERR

9051

Master server is not ready yet

FCREPL_MISSUBS_ERR

9052

Replication does not have any subscription

FCREPL_TARFIL_ERR

9053

Target file already exists

FCREPL_REPLDISC_ERR

9054

Replication Agent is disconnected

FCREPL_RESYNC_ERR

9055

Replication resync failed

FCREPL_INRESYNC_ERR

9056

Replication is in a resync operation now

FCREPL_EREPLFILE_ERR

9057

Replication does not have any file

FCREPL_CANCELLED_ERR

9058

Operation has been cancelled

FCREPL_RESTORE_ERR

9059

File dump restore error

FCREPL_SQLPORT_ERR

9060

SQL port mismatch

FCREPL_REPLEXIST_ERR

9061

replication plan information already exists

FCREPL_NINACTIVE_ERR

9062

replication plan is not inactive



Table 4. FCREPLDP

Symbolic

Code

Description

FCREPLDP_SUCCESS

0

RCESSDP API returned OK

FCREPLDP_MEM_ERR

9501

Could no allocate memory

FCREPLDP_INVHDL_ERR

9502

Invalid handle

FCREPLDP_INVSTR_ERR

9503

Invalid connection string

FCREPLDP_PREPARE_ERR

9504

Error during the SQL command prepare

FCREPLDP_MORE_RESULT

9505

There are more records to be retrieved

FCREPLDP_INVFILETYPE_ERR

9506

Invalid file type

FCREPLDP_INVARG_ERR

9507

Invalid argument

FCREPLDP_MISSYM_ERR

9508

Missing symbol

FCREPLDP_CIRFIL_ERR

9509

Circular file replication

FCREPLDP_NOFILE_ERR

9510

Empty publication