"createInput"
(jsonAction)
JSON hub "createInput"
action creates a new FairCom Edge input
The "createInput"
action creates a new input. It configures a service to collect tags from a data source and insert those tags into an integration table.
Note
A tag is a piece of named data with an associated value. For example, the tag temperature 70
includes both the named data temperature
and the value 70
. In JSON, a tag is a JSON property, such as "temperature": 70
. In a SQL table, a tag is a data field, such as a field named temperature
with a value of 70
.
An input configures a connector to collect data tags periodically from a device and store them in an integration table. For example, an input can configure the OPC UA connector to collect and store data from an OPC UA device.
An input is a mapping from a service to an integration table.
The action
"createInput"
is the only way to create this mapping. If the mapping already exists, this action returns an error. To modify an existing input, use the"alterInput"
action.Each input typically configures a connector to store data in its own dedicated integration table.
You may create multiple inputs that configure a variety of connectors to collect data from different devices for storage in the same integration table. This is useful when different devices collect the same type of data, such as temperature, and you want all temperature data to be stored in one table.
When you want the same collected data to exist in multiple integration tables, to publish the data to multiple MQTT topics for example, you can create a
"jsonToDifferentTableFields"
transform to replicate data from one integration table to another.If the specified integration table does not exist when an input is created, FairCom Edge creates it.
A connector inserts records into an integration table, which is a historical log of data collection events. If your account has permissions, you may also update or delete collected records.
A connector typically converts the data tags it collects into JSON and writes them to the
"source_payload"
field of a newly inserted record.Transforms can take data from the
"source_payload"
field (or from other fields in the table), transform it, and place the results in other fields, such as user-defined fields.
Request examples
The following examples are for the Modbus connector. For examples of other connectors, visit the Allen-Bradley, OPC UA, or Siemens S7 API references.
Minimal request
{ "api": "hub", "action": "createInput", "params": { "inputName": "modbusTCP", "serviceName": "modbus", "settings": { "modbusProtocol": "TCP", "modbusServer": "127.0.0.1", "modbusServerPort": 1505, "propertyMapList": [ { "propertyPath": "temperature", "modbusDataAccess": "holdingregister", "modbusDataAddress": 1199, "modbusUnitId": 5, "modbusDataLen": 1 } ] }, "tableName": "modbusTableTCP" }, "authToken": "replaceWithAuthTokenFromCreateSession" }
{ "api": "hub", "action": "createInput", "params": { "inputName": "modbusRTU", "serviceName": "modbus", "dataPersistenceStrategy": "onChange", "dataCollectionIntervalMilliseconds": 5000, "immediatelyCollectDataOnStart": true, "dataCollectionBufferCount": 2, "settings": { "modbusProtocol": "RTU", "modbusSerialPort": "COM2", "modbusBaudRate": 19200, "modbusParity": "None", "modbusDataBits": 8, "modbusStopBits": 2, "propertyMapList": [ { "propertyPath": "temperature", "modbusDataAccess": "holdingregister", "modbusDataAddress": 1199, "modbusUnitId": 5, "modbusDataLen": 1 } ] }, "tableName": "modbusTableRTU" }, "authToken": "replaceWithAuthTokenFromCreateSession" }
{ "api": "hub", "action": "createInput", "params": { "inputName": "modbusTCP", "serviceName": "modbus", "databaseName": "ctreeSQL", "ownerName": "admin", "tableName": "modbusTableTCP", "sourcePayloadFormat": "binary", "settings": { "modbusProtocol": "TCP", "modbusServer": "127.0.0.1", "modbusServerPort": 1505, "modbusDataCollectionIntervalMilliseconds": 15000, "propertyMapList": [ { "propertyPath": "temperature", "modbusDataAddress": 1199, "modbusDataAccess": "holdingregister", "modbusUnitId": 5, "modbusDataLen": 1 }, { "propertyPath": "volume", "modbusDataAddress": 1299, "modbusDataAccess": "holdingregister", "modbusUnitId": 5, "modbusByteOrder": "ABCD", "modbusDataLen": 2 }, { "propertyPath": "status", "modbusDataAddress": 1199, "modbusDataAccess": "coil", "modbusUnitId": 5, "modbusDataLen": 1 } ] }, "retentionPolicy": "autoPurge", "retentionPeriod": 4, "retentionUnit": "week", "metadata": {} }, "requestId": "00000007", "authToken": "replaceWithAuthTokenFromCreateSession" }
A response to a request is "0" when successful. A non-zero value response indicates an error occurred.
{ "result": {}, "requestId": "00000007", "errorCode": 0, "errorMessage": "" }
{ "result": {}, "requestId": "00000011", "debugInfo": { "request": { "api": "hub", "action": "createInput", "params": { "inputName": "modbusTCP", "serviceName": "modbus", "settings": { "modbusProtocol": "TCP", "modbusServer": "127.0.0.1", "modbusServerPort": 1505, "propertyMapList": [ { "propertyPath": "temperature", "modbusDataAddress": "holdingregister", "modbusDataAddress": 1199, "modbusUnitId": 5, "modbusDataLen": 1 } ] }, "tableName": "modbusTableTCP" }, "requestId": "00000011", "debug": "max", "authToken": "replaceWithAuthTokenFromCreateSession" } }, "errorCode": 12012, "errorMessage": "There is already an integration with name of [modbusTCP].", "authToken": "replaceWithAuthTokenFromCreateSession" }
Use the createInput JSON API action to create a new FairCom Edge input
The "params"
property is an object that contains an action's parameters. Each action defines its own required and optional properties. See System limits for a comprehensive look at property requirements and limitations.
"params"
property summariesProperty | Description | Default | Type | Limits (inclusive) | |||||||
---|---|---|---|---|---|---|---|---|---|---|---|
databaseName | (optional) contains the name of a database | Defaults to the | string | 1 to 64 bytes | |||||||
(optional) specifies the number of milliseconds to wait until reading PLC data again. For example, |
| integer |
| ||||||||
(optional) specifies the number of times the collector retrieves and caches data from the device before writing the data to the integration table |
| integer |
| ||||||||
(optional) specifies when the connector writes collected data to the specified table |
| string |
| ||||||||
(optional) configures the input to collect data immediately when the server or connector starts. This occurs even if the data persistence strategy is to collect data changes |
| Boolean |
| ||||||||
inputName | specifies the unique name of an input. | Required - No default value | string | 1 to 64 bytes | |||||||
(optional) exists primarily for the user interface to find integration information |
| object | May contain any number and type of user-defined properties | ||||||||
(optional) specifies the unique name of a schema in a database | Defaults to the default Owner Name set during | string | 1 to 64 bytes | ||||||||
(optional) specifies the number of retention units, which controls how long data is retained – see |
| integer |
| ||||||||
(optional) specifies how messages persist |
| string |
| ||||||||
(optional) purges expired messages each time this unit cycles – see |
| string |
| ||||||||
specifies the name of a service that collects data from an external source and inserts it into an integration table | Required - No default value | string | 1 to 64 bytes | ||||||||
(optional) contains properties needed to configure the data source of the specified plugin |
| object |
| ||||||||
(optional) specifies the variant type format of the |
| string |
| ||||||||
(optional) specifies the name of a table |
| string | 1 to 64 bytes | ||||||||
(optional) specifies the name of a transform process you have created. See the "createTransform" and "alterTransform" actions |
| string | 1 to 64 bytes |
The "dataCollectionBufferCount"
property specifies the number of times the collector retrieves and caches data from the device before writing the data to the integration table.
This option combines multiple data collection events and inserts them into the integration table as one MQTT message.
If this value is more than 1
, the connector converts each set of collected data into a JSON object and adds the object to an array inside a JSON document. When the count is reached, the connector writes the JSON document to the source_payload
field of a record it inserts into the integration table.
The "dataCollectionIntervalMilliseconds"
property schedules the connector to collect data periodically using the specified number of milliseconds. It defaults to 10000
, which is ten seconds. It must be an integer number >= 1
.
The "dataPersistenceStrategy"
property controls when a connector writes collected data to a record in the input table specified by "tableName"
. It is an optional string enum property with two values: "onSchedule"
and "onChange"
. The default value when the property is omitted or set to null
is "onSchedule"
.
The
"onSchedule"
setting causes the connector to persist each collected value, which occurs on the schedule set by the"dataCollectionIntervalMilliseconds"
property. This setting is useful for continuously collecting device data for trend analysis and machine learning.The
"onChange"
setting causes the connector to collect data on the schedule set by the"dataCollectionIntervalMilliseconds"
property, but it only persists data when it changes. This allows the connector to detect and store data changes as events, such as alerts and status updates. This setting greatly reduces data storage for infrequently changing telemetry, such as changes to a temperature sensor.Note
The connector misses events when its data collection schedule is not frequent enough to see data changes. Thus, the
"onChange"
setting makes a polling protocol, such as Modbus, behave like a push protocol, but it is not a push protocol like MQTT.
The "immediatelyCollectDataOnStart"
property controls how a connector first collects data after being started.
When
true
, it immediately collects and stores data and thereafter collects data according to the strategy specified in"dataPersistenceStrategy"
. In other words, it ignores the"dataPersistenceStrategy"
for the initial data collection event and unconditionally stores the first set of data it collects. This setting allows subscribers to receive an initial data collection message each time the server or connector starts, which is useful for systems that display live status information.When
false
, which is the default, it collects data according to the strategy specified in"dataPersistenceStrategy"
. For example, when the"dataPersistenceStrategy"
is"onChange"
, the connector collects data on a schedule and persists it only when the data changes - even after a restart.
The "metadata"
property is an optional JSON object. It exists primarily for the user interface to find integration information. By default, it is an empty JSON object.
It contains a flexible set of properties.
It typically contains tags and description properties.
Example
{ "description": "", "tags": [""], "yourOwnProperties": "usage, purpose, notes, location, etc.", }
The "ownerName"
property is an optional string from 1 to 64 bytes that specifies the account that owns an object.
The "retentionPeriod"
property specifies how many units of data to retain. It must be an integer value from 1
to 100
. It refers to the unit of time specified by the "retentionUnit"
property — for example, if "retentionPeriod"
is 14
and "retentionUnit"
is "day"
, then message data is retained for 14 days. This property is optional.
Periodically, the system increments the age of each table partition.
"Minute"
units are incremented at the zero second of each minute, whether the table partition was created 1 second before or 59 seconds before."Day"
units are incremented at midnight GMT, not midnight of the local UTC time zone. A table partition becomes one day old at midnight GMT, whether it was created one second before or 23 hours and 59 seconds before."Week"
units are incremented at midnight GMT on the last day of each week, even if the table partition was created one second before. The last day of the week is Saturday GMT."Month"
units are incremented at midnight GMT on the last day of each month."Year"
units are incremented at midnight GMT on Dec 31.
"retentionPeriod"
implicitly calculates an upper bound on how many partitions are able to accumulate on your hard drive. However, partitions are not deleted just because this calculated number of partitions is reached. The system also does not restrict the deletion of all partitions.If the FairCom database is shut down for a month, when the database is started up again, all partitions that are retained for less than one month are immediately deleted.
If someone purposely or accidentally sets the date to a future date, all partitions immediately become older, and auto-deletion ensues accordingly.
When partitions are auto-purged, some data are maintained "forever"
in a global index. Auto-purge does not prevent these files from growing without bounds and filling up your hard drive.
If not specified, the default found in the services.json file is used. Initially, it is 4 (weeks).
Automatically purging data is important to ensure that retained data does not consume all storage and shut down the computer. The default value of 4 weeks allows FairCom's servers to store 1 TB of messages when 200 topics send one 2K message per second.
Note
If the value is not an integer from
1
to100
, FairCom's servers set it to the default value.Smaller numbers improve SQL performance.
Each time the
"retentionPeriod"
cycles, FairCom's servers automatically and efficiently delete expired data.The
"retentionUnit"
and"retentionPeriod"
properties are used only when the"retentionPolicy"
is set to"autoPurge"
.FairCom's servers only use the
"retentionPeriod"
property when the"retentionPolicy"
is"autoPurge"
.The
"retentionPeriod"
can be changed to retain fewer or more messages. Changing it does not necessarily destroy existing data, but data may expire more quickly or be retained longer.The
"retentionPeriod"
and"retentionUnit"
properties control data granularity as well as the retention time. In other words,"retentionPeriod"
defines how many sets of data are stored, and"retentionUnit"
defines how often data is purged.For example, if
"rententionPeriod"
is set to14
, the server stores 14 sets of data. At the beginning of the 15th cycle, the server automatically purges the oldest set of data. If"retentionUnit"
is set today
, then data will be purged daily. If set to"week"
, then data will be purged weekly.The current calendar date affects purging.
FairCom's servers automatically purge all retained data that has expired. This is noticeable when FairCom's servers come online after having been offline for a long time. When a server comes back online, it automatically purges all expired data.
For example, if a FairCom server is offline for four weeks when it comes online, it will completely purge all retained data that has a retention time of less than 4 weeks.
The "retentionPolicy"
property controls how messages are persisted. This property is optional.
If not specified, the default found in the services.json
file is used. Initially, it is "autoPurge"
.
retentionPolicy
values:"autoPurge"
This is the default. It is automatically applied when a new topic is created. It is preferred because it allows FairCom's servers to automatically remove messages that are older than the retention time. This helps ensure message data does not consume all storage space. It also minimizes storage costs and speeds up data access. The server partitions a table into multiple files so it can efficiently delete expired files.
"neverPurge"
This stores messages on disk and never removes them. This is useful when you need the entire history of the message stream. If message velocity is high, this can consume all storage space and cause an outage. The server creates a non-partitioned table, which is slightly faster than a partitioned table because it stores all records in one file.
Each time this unit cycles, FairCom purges expired messages. For example, if you want a week's worth of messages to be purged once a week, set "retentionUnit"
to "week"
. This property is optional.
If not specified, the default found in the services.json
file is used. Initially, it is "week"
.
This property is used in concert with
"retentionPeriod"
to determine retention time."retentionUnit"
values:"minute"
"hour"
"day"
"week"
"month"
"year"
"forever"
Note
For best performance, set the
"retentionUnit"
to a value that keeps"retentionPeriod"
between5
and30
.When you set
"retentionUnit"
property to"forever"
the server will not purge messages. This setting is the same as setting"retentionPolicy"
to"neverPurge"
.The
"retentionUnit"
and"retentionPeriod"
properties are used only when the"retentionPolicy"
is set to"autoPurge"
.
The "serviceName"
property contains the name of a FairCom input or output service. This property is required.
See the "params" topic of each specific service for the requirements of this property.
"MODBUS"
"SIEMENSUDT2JSON"
"OPCUA"
Note
The SQL, JSON RPC, and REST services can automatically query any integration table in FairCom's servers without requiring configuration.
Note
MQTT always represents both input and output services. This is because once a topic is created and assigned to an integration table, any MQTT client can publish messages to it and subscribe to those messages.
The "settings"
property contains properties that are specific for each connector type. Settings for Modbus are different than settings for OPC UA, and so forth. See the API reference "params" property of each connector for details of the "settings"
property for that connector.
Connector-specific "settings"
The "sourcePayloadFormat"
property is an optional string that specifies the expected data type of the MQTT payload. When omitted or set to null
, it defaults to "binary"
. When the server receives an MQTT message, it always takes the binary value from the MQTT payload and stores it in the source_payload
field. The source_payload
field is a variant, which means it stores the type of data it contains. If there are no errors in validating the MQTT payload to match the type specified in "sourcePayloadFormat"
, the server assigns the value of "sourcePayloadFormat"
as the variant field's type.
Depending on the values of the "sourcePayloadFormat"
and "validateMqttPayload"
properties, the server may validate the MQTT payload to ensure it matches the type. The default value for the "validateMqttPayload"
property is true
. The server may also convert the MQTT payload into another form before storing it. If the MQTT payload does not match the type or cannot be converted, the server stores the payload in the source_payload
field, sets its variant type to "binary"
, and sets the error field to 1
to indicate there is an error. It also adds an error entry to the JSON array in the log field.
FairCom's MQTT server does the following when a topic receives an MQTT message and "sourcePayloadFormat"
is set to one of the following values:
"sourcePayloadFormat" Value | Description |
---|---|
| The server stores the MQTT payload in the |
| The server stores the MQTT payload in the
|
| The server stores the MQTT payload in the
|
| The server validates the payload as a variant object.
The server ignores the value of |
| The server stores the MQTT payload in the |
| The server stores the MQTT payload in the |
| The server stores the MQTT payload in the |
This property is a hint to the server about the format and type of the MQTT message payload.
This property does not cause server to validate the MQTT payload to see if it matches the type you set. The server stores the payload as is without validation. For example, setting the type to
"json"
, it does not stop the server from receiving and storing a non-JSON value or invalid JSON document in the source payload.The FairCom Edge Explorer application may use the value of this property to determine the default way to display a topic's payload.
The transform engine may use the value of this property to help it transform the source payload.
The "tableName" property contains the name of the table in the database where the event occurred. It is a non-zero-length string.
See table name
in System limits for table naming requirements and limitations.
A table in DBnotify is defined by "databaseName", "ownerName" and "tableName" or by "dataFilePath".
The "transformName"
property is an optional string that contains the unique name of a transform process, which consists of one or more transform steps.
The value of "transformName"
must be the name of a transform process you created. It cannot be the name of a FairCom-provided transform method.
You must create a transform before you can assign it to an integration table. See the "createTransform"
and "alterTransform"
actions.
A transform is a process that works like a pipeline where the output of one transformation can become the input for another transformation.
The following actions use the
"transformName"
property to assign a transform to an integration table:"configureTopic"
"createInput"
"alterInput"
"createIntegrationTable"
"alterIntegrationTable"