Documentation

Automatic Data Aggregation

Previous Topic

Next Topic

Using Automatic Data Aggregation

Automatic Data Aggregation simplifies the task of performing simple processing on timestamped data. For example, a continuous series of readings from a temperature sensor can be stored in a table. The Automatic System Time function is used to add data timestamps if necessary. Automatic Data Aggregation is used to calculate average temperature readings based on this data.

Some applications, especially those in the Internet of Things, can generate a huge amount of data due to the nature of sensors, many of which produce a constant stream of readings. This may result in more data than can be stored locally on a small device. It may be impractical to send all of the raw data to the cloud for storage or processing due to bandwidth considerations. Developers may be interested in storing data that has been aggregated over a given amount of time. For example, a temperature sensor may produce readings many times in a minute, however the moving window average of those readings may be more meaningful than individual data points and requires far less storage.

Automatic Data Aggregation provides developers and DBAs an easy solution for aggregating and storing data.

Automatic Data Aggregation monitors Automatic System Time fields, aggregates data record values within a given amount of time (user specified sample window), and stores defined aggregated values in an aggregation table.

How to Use the Automatic Data Aggregation Feature

This plug-in feature is enabled by adding the following server configuration in ctsrvr.cfg:

PLUGIN cttimestamp;<cttimestamp dynamic library>

where:

  • <cttimestamp dynamic library> is the path and filename of the plug-in, e.g., ./cttimestamp/libcttimestamp.so

This plug-in is loaded at server startup and begins aggregating tables listed in the cttimestamp.json configuration file described below.

Requirements:

  • The table to aggregate must have at least one auto-timestamp field of type CT_TIMESTAMP.
  • The auto-timestamp field must be the first segment of any index on that table.

Supported Field Types - The fields to aggregate must have one of the following field types:

  • Signed integer types: CT_TINYINT, CT_SMALLINT, CT_INTEGER, CT_BIGINT
  • Unsigned integer types: CT_UTINYINT, CT_USMALLINT, CT_UINTEGER, CT_UBIGINT
  • Float types: CT_SFLOAT, CT_EFLOAT, CT_DFLOAT

In This Chapter

Configuring Aggregation

TOCIndex