Automatic data aggregation

Automatic data aggregation simplifies the task of performing simple processing on timestamped data — for example, a continuous series of readings from a temperature sensor can be stored in a table. The automatic system time function is used to add data timestamps if necessary. Automatic data aggregation is used to calculate average temperature readings based on this data.

Some applications, especially those in the Internet of Things (IoT), can generate a huge amount of data due to the nature of sensors, many of which produce a constant stream of readings. This may result in more data than can be stored locally on a small device. It may be impractical to send all of the raw data to the cloud for storage or processing due to bandwidth considerations. Developers may be interested in storing data that has been aggregated over a given amount of time — for example, a temperature sensor may produce readings many times in a minute. However, the moving window average of those readings may be more meaningful than individual data points and requires far less storage.

Automatic data aggregation provides developers and DBAs with an easy solution for aggregating and storing data. Automatic data aggregation monitors automatic system time fields, aggregates data record values within a given amount of time (user-specified sample window), and stores defined aggregated values in an aggregation table.

{
    "sourceTableList": [
        {
            "database": "ctreeSQL",
            "sourceTableName": "sensor",
            "aggregatedTableList": [
                {
                    "aggregatedTableName": "aggregation1",
                    "timestampField": "ts",
                    "unitOfTime": "second",
                    "unitOfTimeValue": 20,
                    "aggregatedFieldList": [
                        {
                            "aggregatedFieldName": "TempAverage",
                            "sourceFieldName": "temperature",
                            "aggregateFunction": "average"
                        },
                        {
                            "aggregatedFieldName": "PressMax",
                            "sourceFieldName": "pressure",
                            "aggregateFunction": "maximum"
                        }
                    ]
                } 
            ]
        }
    ]
}