In the IoT it is very common to stream data to the cloud for storage and other processing, such as running reports. FairCom EDGE allows you to store and process data on the edge device, saving the time it takes to move data to the cloud. It also has the benefit replicating data to the cloud.
Replicating from the edge to the cloud consists of three pieces:
Replication differs from streaming in that, if a connectivity outage occurs, replication will pick up where it left off when the outage is over and no data will be skipped. With streaming, data generated during the outage most often is lost. Replication also moves data in the same transaction chunks that were originally added to the source server, preserving atomicity on the target server.
You can configure your replication using a graphical tool, the Replication Manager. The configuration is stored in the Memphis database and propagated throughout the system. To simplify setup, FairCom EDGE and the files it creates are replication-aware.
For full documentation see the Replication Manager Guide in the FairCom online documentation.
Install a FairCom EDGE database on the source machine. See the FairCom EDGE Administration Guide for detailed steps on installing and configuring a server. To enable replication, you must do either of the following:
If you wish to replicate more than one table, you can add multiple lines or use wildcards such as "*.dat". The REPLICATE keyword identifies tables that will be replicated for the source server. The path is relative to the server's data folder. Tables created via MQTT persistence will be located in data/ctreeSQL.dbs/ by default.
Add REPL_SRLSEG_ALLOW_UNQKEY YES to the ctsrvr.cfg file. All tables to be replicated must have a unique index. This keyword allows simple tables to be replicated by using our built in identity index.
Connect the source server to the Replication Manager server by editing the /server/config/ctagent.json file. Change the "memphis_host": "localhost", line by replacing "localhost" with the IP of the target server.
Start the source server.
Install a FairCom DB Server on the target machine. If this server is on the same machine as the source server, you will have to change its name. Look for the SERVER_NAME keyword in the ctsrvr.cfg file in the server folder. Make sure this name is unique (we sum up the ASCII equivalent of the name, so SRV1 and 1SRV will be identical, which will cause problems).
Open a browser window and point it to this URL:
https://<target server IP>:8443/ReplicationManager/
You should see the Replication Manager app load. Verify that both servers appear.
Drag a line between the FairCom EDGE source server and the FairCom DB target server. You will be warned that the source server must push the changes, this is normal for replication from a FairCom EDGE server.
Create a new publication with the table from the source server.
If you want all data to stay on the target server, make sure Ignore Partition is checked under Advanced Options.
Subscribe to the publication just created.
Right-click on the publication line and select Deploy.
You should see the table be created on the target server and the data moved over.
If there is a VPN or firewall between any of the machines you will have to provide network access from the machine the Replication Agent is running on to both the target and source server. The port you will need to open is the ISAM (NoSQL) port each server is using. The port the server uses is based on the server name. For the default name of FAIRCOMS, the port will be 5597. For other server names, check the ctstatus.fcs file in the servers "data" directory for the port in use by searching for a line similar to:
"Server using TCP/IP Socket Port Number: 5597"
See the Replication Manager Guide for more about installing, configuring, and using replication.