C++ Replication API

Concepts for using the C++ Replication API

Abstract

Concepts for using the C++ Replication API

Table 1. Sections

Section	Description
JSON Replication API	This section summarizes the JSON Replication API.
C++ interface	This section lists the formations of the C++ interface.
File system and database classes	This section lists classes that depict the file system and its hierarchy.
Replication classes	The replication plan is the link between two database engines: the source and the target.
Class relationships	This section shows a figure that summarizes classes and their relationships.
Configuration	The replication API requires a running Replication Manager server. You will find this server in the Replication Manager package. Its executable name is called Memphis as a metaphor for a central management hub.
Tutorial for the C++ API	Two tutorials are provided to assist you in learning to use the Replication API.
REST and C/C++ error codes	Error codes contained in `fcRepl.h`.

Table 2. The Replication Manager API defines a variety of classes (see Class relationships for the hierarchy).

Section	Description
Action	An action can be pushed from the central point of control to any node in the site and executed by that node.
Agent	Each machine in the Site must have at least one Replication Agent running to be included in the site.
Connect/disconnect pool	This class provides connection management.
Database	Each DBEngine can host one or more databases.
DBEngine	Each OpSystem (operating system) may contain one or more database servers.
File	A Volume may contain one or more files. Files and folders are both considered to belong to the File class.
Hardware	Each Site may contain one or more physical devices, including computers, tablets, phones, and so forth that are referred to as hardware.
Log	This class is used to retrieve a log or a Trace.
OpSystem	Each piece of physical Hardware may contain one or more operating systems — for example, a single computer could have a virtual machine with Windows installed and another running Linux.
Publication	A publication is a list of Files to be replicated.
replication plan	A replication plan defines which files are being replicated.
Site	The site class represents all the Hardware that is eligible for replication.
Subscription	A sSubscription contains one Publication and indicates the target where the published files are to be replicated.
Trace	This class allows you to manage debug and tracing.
Volume	Each OPSystem (operating system) can host one or more volumes (a volume may contain one or more Files).

The FairCom replication solution uses distributed architecture based on centralized metadata:

The system is distributed, with a FairCom DB database and Replication Agent on each node.
The metadata about the system is centralized in the Memphis database.

This means that any point on the system that can connect to the centralized metadata can access and control any part of the system.

Figure 1. Replication Manager

The C++ API allows you to access and control the whole system by connecting to it through the Replication Manager. The replication API is a programmatic interface to the centralized replication metadata in the Memphis database.

The Replication Manager uses the API to manage the Replication Agent on each node. Using the API, developers can integrate the same functionality provided by the Replication Manager into their own custom applications.

The FairCom DB development environment includes the JSON Replication API as well as the C++ interface based on this API.

JSON Replication API

The architecture of the JSON RPC API defines a hierarchy of objects that represent the various components of a replication environment, such as physical hardware, operating systems, volumes, folders, files, as well as database servers, databases, and so forth.

C++ interface

The C++ interface is formed by:

Note

Both are documented in the header files in the \ctreeApps\FcFatDB\source\fcRepl directory.

Class definitions
High-level function headers

File system and database classes

Figure 2. File system and databases classes

The following classes depict the file system and its hierarchy, which provides a way to uniquely represent each file:

Site
At the highest level, the site class represents all the systems eligible for replication. A system must have a Replication Agent installed to be eligible for replication. The site includes every replication-eligible system that is connected via local networks, remote networks, the cloud, and so forth.
Hardware
Each site may contain one or more physical devices, such as computers, tablets, phones, and so forth. These devices are referred to as hardware.
OpSystem
Each piece of physical hardware may contain one or more operating systems, referred to as OpSystems. For example, a single computer could have a virtual machine with Windows installed and another VM running Linux.
DBEngine
Each OpSystem may contain one or more database servers. In this release, only FairCom DB database servers are recognized.
Database
Each DBEngine can host one or more databases. The FairCom replication APIs recognize both c-treeDB and FairCom DB SQL databases.
Volume
Each operating system can host one or more volumes. A volume is a logical drive with a single file system. It may refer to a physical drive (fixed or removable) or a mount point within another volume.
File
A volume may contain zero or more files or folders. The terms file and folder are used interchangeably in this API as files and folders are both considered to belong to the file class.

Replication classes

Figure 3. replication plan

The replication plan is the link between two database engines: the source and the target. The FairCom replication APIs use a publish and subscribe metaphor for defining which files will be replicated from the source to the target.

Note

Two-way replication is supported, in which case the data is replicated in both directions (from the source to the target and from the target to the source).

Defined classes:

Publication
A publication is a list of files to be replicated. The files are defined using the hierarchy of file system classes defined above. These files will be the source for replication.
Subscription
A subscription contains one publication and indicates the target where the published files are to be replicated.
replication plan
A replication plan defines which files are being replicated. It can contain one or more subscriptions and, therefore, defines the replication target (the subscription) and the source (the publication used by the subscription).

Class relationships

Figure 4. Class summary

This figure summarizes the classes and their relationships.

In addition to the classes shown above, several secondary classes are defined. These classes are used primarily internally. A complete list of classes and functions is contained in all the header files in the \repl\include\fcRepl directory.

Configuration

The Replication API requires a running Replication Manager server that is in the Replication Manager package. Its executable name is called Memphis as a metaphor for a central management hub. Similarly, all methods in the API are prefixed with ctMemphis. Memphis is a synonym for Replication Manager.

Servers on a network must register with Memphis to be replicated.

Connection management

The Replication Manager normally communicates to registered servers by using their hostname or list of IP addresses retrieved from the local network adapters. There are cases, depending on the network topology, that both Replication Manager and the other servers need to use some specific IP address to connect. The DBENGINE_IP_ADDRESS keyword found in the ctagent.json file facilitates this feature.

Replication Manager has an automated process for connecting with registered servers.

All IP addresses of servers that can be identified in a local network are automatically detected and listed in a DBEngine table.
The list is used to connect servers with the Replication Manager server. When a connection is made, the Replication Manager checks to see if the agent server is registered.
If a registered server has specified a dbengine_ip_address in its ctagent.json file, that IP address replaces the original in the DBEngine table.

JSON Replication API

This feature impacts ctMemphisGetDBEngines, ctMemphisPersistDBEngine, ctMemphisGetReplPlansDetails, ctMemphisGetArchivedReplPlansDetails, and ctMemphisGetDBEnginesByGroup web APIs.

C++ API

This feature also impacts the FCREPLDBEngine class with the extra Getters and Setters for this information.

Tutorial for the C++ API

Two tutorials are provided to assist you in learning to use the Replication API. Tutorials are provided for both the JSON Replication and C++ APIs.

These two tutorials are in the following directories:

drivers\cpp.replication\
This folder contains the tutorial for use with C or C++.
drivers\java.rest.replication
This folder contains the tutorial for the Replication JSON Replication API.

These tutorials provide examples of how to use these APIs. They include sample projects that you can compile and run. After compiling the projects, you will start the servers and define the source and target folders that will be used for replication. You can run the projects in debug mode and add breakpoints so you can see how the various calls are used. Additionally, you can review the source code to study the use of the APIs. Procedures for these tutorials are provided in this section.

Requirements:

A tutorial is included in the cpp.replication or java.rest.replication folder found in the <faircom>\drivers folder. Compile and run this tutorial to create a simple replication environment.
Ensure you have already installed the Replication Manager package for your desired platform.
Navigate to the Replication Manager directory and start a Replication server to create an empty replication database.
Start two FairCom servers and create source and target directories for replication.
Locate the source provided in the tutorial directory.

Compile and run the C++ high-level API example:

Compile the tutorial using BuildTutorial.bat (BuildTutorial shell file or makefile on Linux platforms).
Note
- BuildTutorial.bat needs to be run from the Visual Studio x64 Native Tools command prompt unless vcvars64.bat is run prior to running BuildTutorials.bat.
- Use the x64 platform.
- The only DLL linked for the API is fcRepl.dll. The apiTestCPP.exe has dependencies on several DLLs (fcRepl.dll, RCESDPCtree.dll, RCESBasic.dll, ctReplAgent.dll, ctExtFileFilter.dll).
Start the replication server (memphis.exe on Windows and memphis on Linux) from the Replication package to create an empty replication database.
Prepare two FairCom servers to serve as the source server and the target server.

Ensure the following two lines (after "; Plugins") are uncommented in ctsrvr.cfg.

; Plugins

PLUGIN cthttpd;./web/cthttpd.dll

PLUGIN ctagent;./agent/ctagent.dll

Start servers 1 and 2 to populate both DBEngines into Replication Agent.
Create a source data directory on the source machine — for example, c:\temp.
Create a target data directory on the target machine — for example, c:\temp2.
Place some FairCom files that qualify for replication in the c:\temp source directory.
Execute (or run in debug) apiTestCPP.exe.
Note
This step uses the masterAuthFile.set authentication file with both user and password encrypted to connect to the replication server.
1. Select one server from the provided list as the source DBEngine.
2. Enter the source data path as c:\temp.
  Note
  The path must match the directory name made in Step 6.
3. Ensure that the files placed in the source data directory are correctly listed.
4. Select another server from the provided list as the target DBEngine.
5. Enter the target data path as c:\temp2.
  Note
  The path must match the directory name made in Step 7.
6. Ensure that the files placed in the target data directory are correctly listed.
Open the graphical interface (https://localhost:8440/ReplicationManager/) in a web browser to view what has been executed.

REST and C/C error codes

Table 3. Error codes contained in fcRepl.h

Symbolic	Code	Description
FCREPL_SUCCESS	0	Success
FCREPL_EOF	1	Cursor is EOF (end of file)
FCREPL_DUPKEY	2	Key value already exists
FCREPL_NOTFOUND	100	Record not found
FCREPL_DISCONNECTED	129	Disconnected server
FCREPL_SERVERNOTFOUND	133	Unable to connect to server
FCREPL_SHUTDOWN_ERR	150	Shutdown error
FCREPL_TIMEOUT_ERR	156	Timeout error
FCREPL_BATFINISHED	428	No more info (batch canceled)
FCREPL_NOTSUP_ERR	454	Feature not supported
FCREPL_WARNING	9000	Warning
FCREPL_CONFIG_ERR	9001	Could not initialize the configuration object
FCREPL_LOGGER_ERR	9002	Could not initialize the logger object
FCREPL_NOTIF_ERR	9003	Could not initialize the notification object
FCREPL_FILESYSTEM_ERR	9004	Could not initialize the file system object
FCREPL_PROVIDER_ERR	9005	Could not initialize the data provider object
FCREPL_INVARG_ERR	9006	Invalid argument count
FCREPL_INVARGVAL_ERR	9007	Invalid argument value
FCREPL_MISARG_ERR	9008	Missing required argument
FCREPL_HASHFULL_ERR	9009	Hash table exceeded the max size
FCREPL_MEM_ERR	9010	Memory allocation error
FCREPL_QUEUEFULL_ERR	9011	Queue exceeded max size
FCREPL_INVACTION_ERR	9012	Invalid notification action
FCREPL_QUEUE_ERR	9013	Queue is not initialized
FCREPL_MUTEX_ERR	9014	Mutex is not initialized
FCREPL_PROCTHRD_ERR	9015	Mutex has not started the process thread
FCREPL_ACTION_ERR	9016	Invalid action
FCREPL_DB_TYPE_ERR	9017	Mismatch provider DB type
FCREPL_LOCALCT_ERR	9018	Could not load local c-tree libirary
FCREPL_SESSION_ERR	9019	Could not allocate c-tree session
FCREPL_INVFLD_ERR	9020	Invalid field number
FCREPL_SCHEMATHRD_ERR	9021	Could not start schema update thread
FCREPL_INTERN_ERR	9022	Internal error
FCREPL_REPL_ERR	9023	Could not initialize the replication object
FCREPL_INVREPLOBJ_ERR	9024	Invalid replication object
FCREPL_STARTED_ERR	9025	Already started
FCREPL_CONSUMER_ERR	9026	Consumer not found
FCREPL_ALREADYEX_ERR	9027	Already exists
FCREPL_INVFILTYP_ERR	9028	Invalid file type
FCREPL_INVFILVOL_ERR	9029	Invalid file volume
FCREPL_REPLNDEP_ERR	9030	Replication plan is not deployed
FCREPL_MISAUTH_ERR	9031	Missing authentication information
FCREPL_FTRNSFR_ERR	9032	File transfer not initialized
FCREPL_REPLACT_ERR	9033	Replication plan is not stopped
FCREPL_REPLNFOUND_ERR	9034	Replication plan is not found
FCREPL_INUSE_ERR	9035	Entity is in use
FCREPL_MISREPL_ERR	9036	Missing replication plan error
FCREPL_MISDB_ERR	9037	Missing database in subscription
FCREPL_MISVOL_ERR	9038	Missing volume information
FCREPL_FNOTFOUND_ERR	9039	File not found error
FCREPL_INVSUBS_ERR	9040	Invalid subscription
FCREPL_INVPATH_ERR	9041	Invalid file path
FCREPL_INVFUNC_ERR	9042	Invalid function
FCREPL_INVOS_ERR	9043	Invalid opSystem
FCREPL_NOTRUN_ERR	9044	Thread is not running
FCREPL_NACTIVE_ERR	9045	replication plan is not started
FCREPL_MISLPOS_ERR	9046	replication plan is missing log position
FCREPL_NCONN_ERR	9047	Agent is not connected
FCREPL_INVTOK_ERR	9048	Invalid token (wrong or expired)
FCREPL_DBINUSE_ERR	9049	DBEngine is in use
FCREPL_REPLAGENT_ERR	9050	Replication agent error
FCREPL_MASTERNOTRDY_ERR	9051	Master server is not ready yet
FCREPL_MISSUBS_ERR	9052	Replication does not have any subscription
FCREPL_TARFIL_ERR	9053	Target file already exists
FCREPL_REPLDISC_ERR	9054	Replication Agent is disconnected
FCREPL_RESYNC_ERR	9055	Replication resync failed
FCREPL_INRESYNC_ERR	9056	Replication is in a resync operation now
FCREPL_EREPLFILE_ERR	9057	Replication does not have any file
FCREPL_CANCELLED_ERR	9058	Operation has been cancelled
FCREPL_RESTORE_ERR	9059	File dump restore error
FCREPL_SQLPORT_ERR	9060	SQL port mismatch
FCREPL_REPLEXIST_ERR	9061	replication plan information already exists
FCREPL_NINACTIVE_ERR	9062	replication plan is not inactive

Table 4. FCREPLDP

Symbolic	Code	Description
FCREPLDP_SUCCESS	0	RCESSDP API returned OK
FCREPLDP_MEM_ERR	9501	Could no allocate memory
FCREPLDP_INVHDL_ERR	9502	Invalid handle
FCREPLDP_INVSTR_ERR	9503	Invalid connection string
FCREPLDP_PREPARE_ERR	9504	Error during the SQL command prepare
FCREPLDP_MORE_RESULT	9505	There are more records to be retrieved
FCREPLDP_INVFILETYPE_ERR	9506	Invalid file type
FCREPLDP_INVARG_ERR	9507	Invalid argument
FCREPLDP_MISSYM_ERR	9508	Missing symbol
FCREPLDP_CIRFIL_ERR	9509	Circular file replication
FCREPLDP_NOFILE_ERR	9510	Empty publication