|
|
|
The Automatic Data Aggregation family of features simplify the collection and aggregation of sensor data. Many real-time sensors generate large streams of data, which can quickly fill available storage on smaller devices. The Auto-Purge feature can be used to purge the oldest data making room for new data. Before purging, the data can be sent to the cloud for storage — depending on your environment, either the entire set of raw data can be streamed to the cloud or, if bandwidth is a consideration, a smaller set of aggregated values can be sent.
c-tree leverages the advanced partitioning capabilities of the FairCom Database Engine to accomplish this. Partitioning allows you to divide a database into smaller partitioned files. As records are inserted into the database, they are sorted into the partitions based on defined criteria, such as time or date. For example, for sensors producing a large number of records, you can set a partition to accommodate an hour's worth of records: Each hour, a new partition is created to hold that hour's records.
The Auto-Purge feature allows automatic purging of data so the storage available on your device does not fill. Using this feature, you might purge aged data so that, say, only 24 hours of current data are kept on the device. In the example of setting each partition to hold data for 1 hour, 24 individual file partitions are created on the device. As soon as the 25th partition is begun, the 1st partition is purged. When the 26th partition is begun, the 2nd partition is purged. And so on. The data collection demands of your application determine how much data is stored in a partition and how many partitions to keep.
In the case of "time series" data, where a new record is constantly inserted into the table (such as a stream of sensor data), new partitions will be created constantly and Auto-Purge will be constantly removing old partitions. Notice that, if you stop inserting records, the old partitions will never be removed.
API Support
Support for partitions with Auto-Purge is included in the FairCom DB APIs. Typically, you only need to add a REST call parameter.
If you are developing with the other APIs, consider the topics in this section:
