Skip to main content

"alterIntegrationTable" (JSON Action)

JSON hub "alterIntegrationTable" action modifies settings for existing tables that are safe to modify

The "alterIntegrationTable" action alters table settings that are safe to modify.

  • The settings that are safe to modify include:

    • Renaming a table

    • Adding fields

    • Increasing field size

    • Changing table retention policy

    • Changing table metadata

    • Changing a table's transformation pipeline

  • As you refine your integration processes, you may want to rename an integration table to better label the data it holds.

    You can use the "alterIntegrationTable" action with the "newTableName" property to rename an integration table.

    Tip

    You can also use the "tableName" property of the "configureTopic" action to rename an integration table that is automatically created by an MQTT message. This is easy because you can rename the integration table using its MQTT topic.

  • This action cannot shrink the size of fields because this destroys data.

  • This action cannot rename fields because it breaks compatibility with bridges between services (integrations / configurations).

Request examples

Prerequisites: You must first create the "test1" integration table using the "createIntegrationTable" action.

{
  "api": "hub",
  "action": "alterIntegrationTable",
  "params": {
    "databaseName": "faircom",
    "tableName": "test1",
    "rebuildTable": true,
    
    "retentionPolicy": "autoPurge",
    "retentionPeriod": 7,
    "retentionUnit": "day"
  },
  "authToken": "replaceWithAuthTokenFromCreateSession"
}

Prerequisites: You must first create the "test1" integration table using the "createIntegrationTable" action.

{
  "api": "hub",
  "action": "alterIntegrationTable",
  "params": {
    "databaseName": "faircom",
    "tableName": "test1",
    "addFields": [
      {
        "name": "t1",
        "type": "json"
      },
      {
        "name": "t2",
        "type": "varchar",
        "length": 128,
        "nullable": true
      },
      {
        "name": "temperature",
        "type": "double"
      }
    ],
    "rebuildTable": true
  },
  "authToken": "replaceWithAuthTokenFromCreateSession"
}

Prerequisites: You must first run the previous "alterIntegrationTable" example to add fields to the "test1" integration table.

{
  "api": "hub",
  "action": "alterIntegrationTable",
  "params": {
    "databaseName": "faircom",
    "tableName": "test1",
    "alterFields": [
      {
        "name": "t2",
        "newName": "x1",
        "type": "varchar",
        "length": 150
      }
    ],
    "rebuildTable": true
  },
  "authToken": "replaceWithAuthTokenFromCreateSession"
}

Prerequisites: You must first modify the "test1" integration table using the previous "alterIntegrationTable" example.

{
  "api": "hub",
  "action": "alterIntegrationTable",
  "params": {
    "databaseName": "faircom",
    "tableName": "test1",
    "deleteFields": [
      "x1"
    ],
    "rebuildTable": true
  },
  "authToken": "replaceWithAuthTokenFromCreateSession"
}

This example changes the transform steps assigned to the "test1" integration table.

{
  "api": "hub",
  "action": "alterIntegrationTable",
  "params": {
    "tableName": "test1",
    "logTransformOverwrites": false,
    "transformSteps": [
      {
        "transformStepMethod": "jsonToTableFields",
        "mapOfPropertiesToFields": [
          {
            "recordPath": "source_payload.temperature",
            "fieldName": "temperature"
          }
        ]
      },
      {
        "transformStepMethod": "tableFieldsToJson",
        "mapOfPropertiesToFields": [
          {
            "fieldName": "temperature",
            "recordPath": "t1.temperature"
          }
        ]
      }
    ]
  },
  "authToken": "replaceWithAuthTokenFromCreateSession"
}

This example changes the metadata assigned to the "test1" integration table.

{
  "api": "hub",
  "action": "alterIntegrationTable",
  "params": {
    "databaseName": "faircom",
    "tableName": "test1",
    
    "metadata": {
      "description": "This can be any description you want.",
      "tags": [
        "minimal",
        "example",
        "alterIntegrationTable"
      ],
      "yourOwnProperties": "Minimal example, showing basic settings, metadata is very flexible, You can put anything here"
    }
  },
  "authToken": "replaceWithAuthTokenFromCreateSession"
}

The example below shows all possible properties for each type of transform step.

{
  "api": "hub",
  "action": "testIntegrationTableTransformSteps",
  "params": {
    "databaseName": "faircom",
    "ownerName": "admin",
    "tableName": "myTable1",

    "logTransformOverwrites": true,
    "disableTransformSteps": false,
    
    "transformSteps": [
      {
        "transformStepName": "step1",
        "transformStepMethod": "javascript",
        "transformStepService": "v8TransformService",
        "codeName": "convertFahrenheitToCelsius",
        "ownerName": "admin",
        "databaseName": "faircom"
      },


      {
        "transformStepName": "step2",
        "transformStepMethod": "tableFieldsToJson",
        "mapOfPropertiesToFields": [
          {
            "fieldName": "my_source_field",
            "recordPath": "my_output_field.myProperty",
            "binaryFormat": "byteArray",
            "numberFormat": "string",
            "variantFormat": "json",
            "dateFormat": "mm.dd.ccyy",
            "timeFormat": "hh.mm.am/pm"
          }
        ]
      },


      {
        "transformStepName": "step3",
        "transformStepMethod": "jsonToTableFields",
        "mapOfPropertiesToFields": [
          {
            "recordPath": "my_source_field.myProperty",
            "fieldName": "my_target_field",
            "binaryFormat": "base64"
          }
        ]
      },




      {
        "transformStepName": "step4",
        "transformStepMethod": "jsonToDifferentTableFields",
        "targetTableName": "myTargetIntegrationTable",
        "targetOwnerName": "admin",
        "targetDatabaseName": "faircom",
        "mapOfPropertiesToFields": [
          {
            "recordPath": "my_source_field.myProperty",
            "fieldName": "my_target_field",
            "binaryFormat": "hex"
          }
        ]
      }


    ]


  },
  "authToken": "replaceWithAuthTokenFromCreateSession"
}
{
    "result": {},
    "requestId": "1",
    "errorCode": 0,
    "errorMessage": "",
    "authToken": "replaceWithAuthTokenFromCreateSession"
}
{
    "result": {},
    "requestId": "5",
    "errorCode": 100,
    "errorMessage": "Not able to find integration table by name [test3/admin/faircom].",
    "authToken": "replaceWithAuthTokenFromCreateSession"
}
{
    "result": {},
    "requestId": "5",
    "errorCode": 100,
    "errorMessage": "Transform pipeline [firstCreateMe] was not found.",
    "authToken": "replaceWithAuthTokenFromCreateSession"
}
{
  "authToken": "replaceWithAuthTokenFromCreateSession",
  "result": {},
  "requestId": "00000001",
  "debugInfo": {
    "request": {
      "authToken": "replaceWithAuthTokenFromCreateSession",
      "api": "hub",
      "action": "alterIntegrationTable",
      "params": {
        "databaseName": "faircom",
        "tableName": "test1",
        "rebuildTable": false,
        "retentionPolicy": "doNotPersist",
        "retentionPeriod": 1,
        "retentionUnit": "day"
      },
      "requestId": "00000001",
      "debug": "max"
    }
  },
  "errorCode": 12006,
  "errorMessage": "Before you can change the data retention policy, period, or unit, you must use "alterIntegrationTable" and add "rebuildTable": true to the request. WARNING: rebuilding a table disrupts data collection while it rewrites all records."
}

Use the alterIntegrationTable JSON API action to alter settings for existing tables that are safe to modify

API actionsJSON hub APIJSON Actionintegration tables actionsalter integration tablealterIntegrationTable

The "params" property is an object that contains an action's request parameters as defined by a set of properties. Each action defines its own required and optional properties. See System limits for a comprehensive overview of property requirements and limitations.

Table 1. alterIntegrationTable "params" property summaries

Property

Description

Default

Type

Limits (inclusive)

addFields

(optional) adds new fields to the table and specifies their properties.

[]

array of objects

0 or more objects

alterFields

(optional) defines fields to alter in a table.

[{"name":""}]

array of objects

0 or more objects

databaseName

(optional) specifies the name of a database.

Defaults to the "defaultDatabaseName" value that is set during "createSession". If no default is set during "createSession", then "faircom" is used.

string

1 to 64 bytes

deleteFields

(optional) contains the names of fields to remove from a table.

[]

array

1 to 64 bytes

disableTransformSteps

(optional) prevents the server from running an integration table's transform steps on newly inserted records.

false

Boolelan

false
true

logTransformOverwrites

(optional) logs when a transform step overwrites a field that already contains a value.

false

Boolean

false
true

metadata

(optional) exists primarily for the user interface to find integration information.

{}

object

May contain any number and type of user-defined properties

ownerName

(optional) specifies the account that owns an object.

Defaults to the account that created the JSON Action session.

string

1 to 64 bytes

retentionPeriod

(optional) specifies the number of retention units, which controls how long data is retained – see "retentionUnit".

4

integer

1 to 100

retentionPolicy

(optional) specifies how messages persist.

"autoPurge"

string

"autoPurge"
"neverPurge"

retentionUnit

(optional) purges expired messages each time this unit cycles – see "retentionPeriod".

"week"

string

"minute"
"hour"
"day"
"week"
"month"
"year"
"forever"

tableName

(optional) specifies the name of a table.

""

string

1 to 64 bytes

transformSteps

specifies an array of transform objects.

Required - No default value

array of objects

0 or more objects

transformSteps
.codeName

specifies the name of a code package that runs when a record is inserted into an integration table.

Required - No default value

string

1 to 64 bytes

transformSteps
.databaseName

(optional) specifies the database that contains an object, such as a table or code package.

""

string

1 to 64 bytes

transformSteps
.ownerName

(optional) specifies the account that owns an object, sucha s a table or code package.

""

string

1 to 64 bytes

transformSteps
.mapOfPropertiesToFields

(optional) specifies field in the table to be mapped to a field containing JSON properties.

[]

array

"binaryFormat"
"dateFormat"
"fieldName"
"numberFormat"
"recordPath"
"timeFormat"
"variantFormat"
transformSteps
mapOfPropertiesToFields
.binaryFormat

(optional) specifies how binary values are returned.

"base64"

string

"base64"
"byteArray"
"hex"
transformSteps
mapOfPropertiesToFields
.dateFormat

(optional) specifies the format of a date or a datetime embedded in a JSON string.

"iso8601"

string enum

"ccyy.mm.dd"
"mm.dd.ccyy"
"mm.dd.yy"
"dd.mm.ccyy"
"dd.mm.yy"
"ccyymmdd"
"yymmdd"
"iso8601"
"utc"
transformSteps
mapOfPropertiesToFields
.fieldName

specifies the name of a field in a table.

Required - No default value

string

1 to 64 bytes

transformSteps
mapOfPropertiesToFields
.numberFormat

(optional) specifies how numbers are formatted in the JSON response message.

"number"

string

"number"
"string"
transformSteps
mapOfPropertiesToFields
.recordPath

specifies the location in a record where the server reads or writes a JSON value.

Required - No default value

string

0 to 256 bytes

transformSteps
mapOfPropertiesToFields
.timeFormat

(optional) specifies how to interpret JSON strings and numbers as time fields.

"hh.mm.am/pm"

string

"hh.mm.am/pm"
"hh.mm.ss.am/pm"
"hh.mm.ss"
20:15:30
20.15&30
20H15M30S
"hh.mm"
"hhmm"
transformSteps
mapOfPropertiesToFields
.variantFormat

(optional) specifies how the server will represent the values of variant fields in its response to your request.

"json"

string

"binary"
"json"
"string"
"variantObject"
transformSteps
.targetDatabaseName

(optional) specifies the name of the database that contains the target table.

""

string

1 to 64 bytes

transformSteps
.targetOwnerName

(optional) specifies the name of the account that owns the target table.

""

string

1 to 64 bytes

transformSteps
.targetTableName

(optional) specifies the name of the target table.

""

string

1 to 64 bytes

transformSteps
.transformStepMethod

specifies the transform method the step will use.

Required - No default value

string enum

"javascript"
"jsonToDifferentTableFields"
"jsonToTableFields"
"tableFieldsToJson"
transformSteps
.transformStepName

(optional) specifies the name that will be assigned to each transform step.

""

string

1 to 64 bytes

transformSteps
.transformStepService

(optional) specifies the name of a transform service, which is the user-defined name for a library (.dll, .so, .dylib) that implements the transform step method.

""

string

1 to 64 bytes



The "addFields" property is optional. It is an array of objects. It allows you to add new fields to a table and specify their properties. Each object in the array defines the properties of a field being added.

Example

"addFields": [
  {
    "autoValue": "none",
    "name": "field1",
    "type": "varchar",
    "length": 50,
    "scale": null,
    "defaultValue": null,
    "nullable": false,
    "primaryKey":1
  }
]

The "alterFields" property is optional. It is an array of objects. It allows you to modify properties of existing fields in a table. Each object in the array defines the modifications to the named field.

In this example, the field named "field1" is being modified.

Example

"alterFields": [
  {
    "autoValue": "none",
    "name": "field1",
    "type": "varchar",
    "newName": "address",
    "newPosition": 5,
    "length": 50,
    "scale": null,
    "defaultValue": null,
    "nullable": false
  }
]

The "databaseName" property is an optional string from 1 to 64 bytes that specifies the database that contains an object, such as a table or code package. If it is set to null or is omitted, it defaults to the default database of the JSON Action session, see "createSession" and the "defaultDatabaseName" property. 

You specify this property when you want to use a different database instead of the default. Your session's account must have the appropriate privileges to access the code package.

This property is useful because objects, such as tables and code packages, can have the same name in multiple databases. This feature allows you to create multiple environments in the same server and reuse the same JSON actions in each environment. For example, you can create "dev", "test", "stage", and "prod" databases on the same server and use the "defaultDatabaseName" or "databaseName" properties to specify the desired environment.

It is an error to set "databaseName" to the empty string "".

If no default database is specified during "createSession", the server sets the "defaultDatabaseName" to the "defaultDatabaseName" value specified in the services.json file.

The "deleteFields" property is an optional array of strings. Each string is the name of a field to remove from a table.

  • Each string is 1 to 64 bytes.

  • Removing a field destroys the data in that field.

  • A zero-length "fieldName" is invalid.

The optional "disableTransformSteps" property has a true or false value. When true, it prevents the server from running an integration table's transform steps on newly inserted records. It is useful to quickly stop a broken transform process. It disables transform steps without removing them from the integration table.

The optional "logTransformOverwrites" property has a true or false value and defaults to true. When true and a record is inserted into the integration table, the server adds log entries to the log field when a transform step overwrites a field that already contains a value. When true, it protects fields, such as source_payload, from being overwritten accidentally. 

You typically set "logTransformOverwrites" to true when testing transform steps. Once they are working as expected, you can set it to false.

If multiple "recordPath" properties write to the same JSON properties in a "tableFieldsToJson" transform, the server will return an error because "recordPath" cannot overwrite. Multiple occurrences of the "recordPath" property may reference the same property as long as the property is in different fields/tables.

Additionally, non-JavaScript transform steps cannot overwrite protected fields or the source_payload field. The only exception is that a single "tableFieldsToJson" transform step may write to the source_payload field.

You may create transform steps to take the value of one JSON property and store it in multiple fields as long as no previous transform steps have already put values in these fields. Conversely, you may take the value of one field and store it in multiple JSON properties.

Tip

Ensure JavaScript transform steps do not overwrite fields updated by other transform steps.

Do not use the "tableFieldsToJson" transform method to overwrite the source_payload field; instead, use "tableFieldsToJson" to write to a different field and use the "configureTopic" action with the "outputPayloadField" property to configure the MQTT topic to deliver that field's value to subscribers.

Ensure SQL, JSON DB, and other APIs set the value of the source_payload field and do not set field values that transform steps update.

The "metadata" property is an optional JSON object. It exists primarily for the user interface to find integration information. By default, it is an empty JSON object.

  • It contains a flexible set of properties.

  • It typically contains tags and description properties.

Example

{
  "description": "",
  "tags": [""],
  "yourOwnProperties": "usage, purpose, notes, location, etc.",
}

The "ownerName" property is an optional string from 1 to 64 bytes that specifies the account that owns an object, such as a table or code package. If it is set to null or is omitted, it defaults to the account that created the JSON Action session, see "createSession" and the "defaultOwnerName" property. 

You specify this property when you want to use a different account instead of the default. Your session's account must have the appropriate privileges to access the code package. 

This property is useful because objects, such as tables and code packages, can have the same name in the same database as long as different accounts own each object. This feature allows you to create duplicate objects for different users on the same server and reuse the same JSON actions on those objects. For example, an administrator can copy objects from a production environment to her account so she can troubleshoot an issue using the same JSON actions, JavaScript, and SQL code.

It is an error to set "ownerName" to the empty string "".

If no default owner is specified during "createSession", the server sets the "defaultOwnerName" to the "defaultOwnerName" value specified in the services.json file.

The "retentionPeriod" property specifies how many units of data to retain. It must be an integer value from 1 to 100. It refers to the unit of time specified by the "retentionUnit" property — for example, if "retentionPeriod" is 14 and "retentionUnit" is "day", then message data is retained for 14 days. This property is optional.

  • Periodically, the system increments the age of each table partition.

    • "Minute" units are incremented at the zero second of each minute, whether the table partition was created 1 second before or 59 seconds before.

    • "Day" units are incremented at midnight GMT, not midnight of the local UTC time zone. A table partition becomes one day old at midnight GMT, whether it was created one second before or 23 hours and 59 seconds before.

    • "Week" units are incremented at midnight GMT on the last day of each week, even if the table partition was created one second before. The last day of the week is Saturday GMT.

    • "Month" units are incremented at midnight GMT on the last day of each month.

    • "Year" units are incremented at midnight GMT on Dec 31.

  • "retentionPeriod" implicitly calculates an upper bound on how many partitions are able to accumulate on your hard drive. However, partitions are not deleted just because this calculated number of partitions is reached. The system also does not restrict the deletion of all partitions.

  • If the FairCom database is shut down for a month, when the database is started up again, all partitions that are retained for less than one month are immediately deleted.

  • If someone purposely or accidentally sets the date to a future date, all partitions immediately become older, and auto-deletion ensues accordingly.

When partitions are auto-purged, some data are maintained "forever" in a global index. Auto-purge does not prevent these files from growing without bounds and filling up your hard drive.

If not specified, the default found in the services.json file is used. Initially, it is 4 (weeks).

Automatically purging data is important to ensure that retained data does not consume all storage and shut down the computer. The default value of 4 weeks allows FairCom's servers to store 1 TB of messages when 200 topics send one 2K message per second.

Note

  • If the value is not an integer from 1 to 100, FairCom's servers set it to the default value.

  • Smaller numbers improve SQL performance.

  • Each time the "retentionPeriod" cycles, FairCom's servers automatically and efficiently delete expired data.

  • The "retentionUnit" and "retentionPeriod" properties are used only when the "retentionPolicy" is set to "autoPurge".

  • FairCom's servers only use the "retentionPeriod" property when the "retentionPolicy" is "autoPurge".

  • The "retentionPeriod" can be changed to retain fewer or more messages. Changing it does not necessarily destroy existing data, but data may expire more quickly or be retained longer.

  • The "retentionPeriod" and "retentionUnit" properties control data granularity as well as the retention time.  In other words, "retentionPeriod" defines how many sets of data are stored, and "retentionUnit" defines how often data is purged.

    For example, if "rententionPeriod" is set to 14 , the server stores 14 sets of data. At the beginning of the 15th cycle, the server automatically purges the oldest set of data. If "retentionUnit" is set to day, then data will be purged daily. If set to "week", then data will be purged weekly.

  • The current calendar date affects purging.

    FairCom's servers automatically purge all retained data that has expired. This is noticeable when FairCom's servers come online after having been offline for a long time. When a server comes back online, it automatically purges all expired data.

    For example, if a FairCom server is offline for four weeks when it comes online, it will completely purge all retained data that has a retention time of less than 4 weeks.

The "retentionPolicy" property controls how messages are persisted. This property is optional.

If not specified, the default found in the services.json file is used. Initially, it is "autoPurge".

retentionPolicy values:
  • "autoPurge"

    This is the default. It is automatically applied when a new topic is created. It is preferred because it allows FairCom's servers to automatically remove messages that are older than the retention time. This helps ensure message data does not consume all storage space. It also minimizes storage costs and speeds up data access. The server partitions a table into multiple files so it can efficiently delete expired files.

  • "neverPurge"

    This stores messages on disk and never removes them. This is useful when you need the entire history of the message stream. If message velocity is high, this can consume all storage space and cause an outage. The server creates a non-partitioned table, which is slightly faster than a partitioned table because it stores all records in one file.

Each time this unit cycles, FairCom purges expired messages. For example, if you want a week's worth of messages to be purged once a week, set "retentionUnit" to "week" . This property is optional.

If not specified, the default found in the services.json file is used. Initially, it is "week"

  • This property is used in concert with "retentionPeriod" to determine retention time.

  • "retentionUnit" values:

    • "minute"

    • "hour"

    • "day"

    • "week"

    • "month"

    • "year"

    • "forever"

Note

  • For best performance, set the "retentionUnit" to a value that keeps "retentionPeriod" between 5 and 30

  • When you set "retentionUnit" property to "forever" the server will not purge messages. This setting is the same as setting "retentionPolicy" to "neverPurge".

  • The "retentionUnit" and "retentionPeriod" properties are used only when the "retentionPolicy" is set to "autoPurge".

The "tableName" property contains the name of the table in the database where the event occurred. It is a non-zero-length string.

See table name in System limits for table naming requirements and limitations.

A table in DBnotify is defined by "databaseName", "ownerName" and "tableName" or by "dataFilePath".

The "transformSteps" property is an array of transform step objects. 

Different actions interpret "transformSteps": [] differently.

  • In the "alterIntegrationTable" action, "transformSteps" is optional. Setting it to [] removes transform steps from the integration table.

  • In the "createIntegrationTable" action, "transformSteps" is optional. Setting it to [] is the same as setting it to null or omitting it.

  • In the "testIntegrationTableTransformSteps" actions, "transformSteps" is required; thus, omitting it or setting it to [] or null is an error.

Each transform step object contains the required "transformStepMethod" property. It also supports the following optional properties:

  • "transformStepName"

  • "transformStepService"

Each transform step method also supports unique properties that are defined below.

"transformStepMethod": "javascript"

  • Required properties

    • "codeName"

  • Optional properties

    • "ownerName"

    • "databaseName"

"transformStepMethod": "tableFieldsToJson"

  • Required properties

    • "mapOfPropertiesToFields"

"transformStepMethod": "jsonToTableFields"

  • Required properties

    • "mapOfPropertiesToFields"

"transformStepMethod": "jsonToDifferentTableFields"

  • Required properties

    • "mapOfPropertiesToFields"

    • "targetTableName"

  • Optional properties

    • "targetOwnerName"

    • "targetDatabaseName"

The "codeName" property is a required string from 1 to 64 bytes. It is the name of a code package that runs when a record is inserted into an integration table. See "codeName" for more information.

It is an error to set "codeName" to the empty string "".

The "databaseName" property is an optional string from 1 to 64 bytes that specifies the database that contains an object, such as a table or code package. If it is set to null or is omitted, it defaults to the default database of the JSON Action session, see "createSession" and the "defaultDatabaseName" property. 

You specify this property when you want to use a different database instead of the default. Your session's account must have the appropriate privileges to access the code package.

This property is useful because objects, such as tables and code packages, can have the same name in multiple databases. This feature allows you to create multiple environments in the same server and reuse the same JSON actions in each environment. For example, you can create "dev", "test", "stage", and "prod" databases on the same server and use the "defaultDatabaseName" or "databaseName" properties to specify the desired environment.

It is an error to set "databaseName" to the empty string "".

If no default database is specified during "createSession", the server sets the "defaultDatabaseName" to the "defaultDatabaseName" value specified in the services.json file.

The "ownerName" property is an optional string from 1 to 64 bytes that specifies the account that owns an object, such as a table or code package. If it is set to null or is omitted, it defaults to the account that created the JSON Action session, see "createSession" and the "defaultOwnerName" property. 

You specify this property when you want to use a different account instead of the default. Your session's account must have the appropriate privileges to access the code package. 

This property is useful because objects, such as tables and code packages, can have the same name in the same database as long as different accounts own each object. This feature allows you to create duplicate objects for different users on the same server and reuse the same JSON actions on those objects. For example, an administrator can copy objects from a production environment to her account so she can troubleshoot an issue using the same JSON actions, JavaScript, and SQL code.

It is an error to set "ownerName" to the empty string "".

If no default owner is specified during "createSession", the server sets the "defaultOwnerName" to the "defaultOwnerName" value specified in the services.json file.

The "mapOfPropertiesToFields" property takes fields in the table and maps them to a field containing JSON properties. It is an array of property map objects. Each object maps a field in a table to a JSON property in another field.

  • Required properties

    • "recordPath"

    • "fieldName"

  • Optional properties

    • "binaryFormat"

    • "numberFormat"

    • "variantFormat"

    • "dateFormat"

    • "timeFormat"

The optional "dateFormat" property specifies the format of a date or a datetime embedded in a JSON string. It applies to user-provided JSON and to server-generated JSON. The server uses it when it needs to transform or convert a date or datetime embedded in a JSON string into a FairCom date or timestamp. It also uses it when it needs to convert a date or datetime field value into a date embedded in a JSON string.

It is an enumerated string with one of the following values: "ccyy.mm.dd", "mm.dd.ccyy", "mm.dd.yy", "dd.mm.ccyy", "dd.mm.yy", "ccyymmdd", "yymmdd", "iso8601", and "utc".

The default value for "dateFormat" is the "defaultDateFormat" property defined in the "createSession" or "alterSession" actions. If it is omitted there, it defaults to the value of the "defaultDateFormat" property in the <faircom>/config/services.json file. If it is not there, the FairCom server defaults it to "iso8601" because the ISO8601 date is the defacto standard in JSON.

The enumerated string defines how the server parses a date in a JSON string and how it writes a date into a JSON string. The following key explains the parts of each enumerated value:

  • ccyy is a four-digit year (0000-9999).

  • yy is a two-digit year (00-99).

  • mm is a two-digit month (01-12).

  • dd is a two-digit day of the month (01-31).

  • hh is a two-digit hour (00-23).

  • . represents one character that can be one of the following values: /, ., or -.

  • "utc", "iso8601", and "ccyy.mm.dd" are the same.

The required "fieldName" property specifies a field in a record. Depending on the context, the server reads or writes the field's value. 

For example, in the "tableFieldsToJson" transform method, "fieldName" is the location where the transform step reads a value that it transforms.  In the "jsonToTableFields" and "jsonToDifferentTableFields" transform methods, "fieldName" is the location where the transform step writes a transformed value.

The "numberFormat" property is an optional, case-insensitive string enum. It defines the format of JSON numbers. The default value for "numberFormat" is the "defaultNumberFormat" defined in the "createSession" or "alterSession" actions. If it is omitted there, it defaults to the value of the "defaultNumberFormat" property in the <faircom>/config/services.json file.

When "numberFormat" occurs in "mapOfPropertiesToFields", it tells the server how to encode or decode a number assigned to a JSON property.

For example, including "numberFormat" in a "tableFieldsToJson" transform step controls if the server encodes a number in a JSON property as a number or a number embedded in a string.

Possible values:
  • "number"

    This causes the server to return numeric values as JSON numbers, such as -18446744073709551616.000144722494 .

    This is most efficient.

    JSON represents numbers are base-ten numbers that may have any number of digits.

    Large numbers, such as 18446744073709551616.000144722494 are known to cause problems with JSON parsers and some programming languages, such as JavaScript, which are limited to the smaller range and binary rounding errors of IEEE floating point numbers.

  • "string"

    This returns the server to embed numeric values in JSON strings, such as "18446744073709551616.000144722494" .

    This is slightly less efficient because it includes two extra double quote characters

    Returning numbers embedded in strings ensures JSON parsers and programming languages will not automatically convert the number to a numeric representation that loses precision, introduces rounding errors, truncates values, or generates errors. This allows your application to control how numbers are processed.

  • Omitted or set to null

    This defaults to "number".

Example request

{
  "action": "someAction",
  "responseOptions":
  {
    "numberFormat": "string"
  },
  "authToken": "replaceWithAuthTokenFromCreateSession"
}

The "recordPath" property specifies the location in a record where the server reads or writes a JSON value. It specifies a field name followed by an optional JSONPath. Depending on the context, it refers to a value the server reads or writes. 

For example, in the "tableFieldsToJson" transform method, "recordPath" is the location where the transform step writes a transformed value.  In the "jsonToTableFields" and "jsonToDifferentTableFields" transform methods, "recordPath" is the location where the transform step reads the value that it transforms.

  • The "recordPath" property must always include a field name.

    • The server returns an error when the field name is missing or does not exist.

  • The "recordPath" property may include a JSONPath after the field name.

    • If no JSONPath follows the field, the "recordPath" property refers to the entire field, and the field is not required to contain a JSON document.

    • If a JSONPath follows the field, the "recordPath" property refers to a part of the JSON document inside the field.

Several FairCom APIs, such as the DB and Transform APIs, treat each record in each table as a JSON document. The top-level fields in the record are treated as top-level properties in the JSON document. In other words, you can think of each record as a JSON document, and you can use the "recordPath" property to refer to any field in the record and any JSON property or array item within a JSON field.

When the server writes a value to a "recordPath", it looks for the specified field and JSONPath and does the following.

  • If the specified field does not exist, the server returns an error.

  • If the JSONPath is not specified after the fieldname, the server writes the value directly to the field.

  • If the field's value does not contain a JSON document, the server returns an error.

  • If the JSONPath specifies a property or array item that does not exist in the field's JSON document, the server adds it.

  • If the JSONPath specifies a property or array item that already exists in the field's JSON document, the server replaces the value.

When the server reads a value from a "recordPath", it looks for the specified field and JSONPath and does the following.

  • If the specified field does not exist, the server returns an error.

  • If the JSONPath is not specified after the fieldname, the server reads the entire field value.

  • If the field's value does not contain a JSON document, the server returns an error.

  • If the JSONPath specifies a property or array item that does not exist in the field's JSON document, the server returns null because FairCom's data model treats a missing JSON property the same as one assigned to null.

  • If the JSONPath specifies a property or array item that already exists in the field's JSON document, the server replaces the value.

Note

Unlike "propertyPath", a "recordPath" includes the name of the field that contains the JSON document.

Simple "recordPath" example

For example, a recordPath of "temperature" refers to a field named "temperature".

Example record in an integration table

The following record contains three fields: id, source_payload, and create_ts.

id

source_payload

create_ts

1

{
  "humidity": 
  [
    {
      "temperature": 20.1,
      "pressure": 1003
    }
  ]
}

"2025-07-07T14:14:02.012"

JSON representation of the example record

The following JSON shows how the JSON DB and Transform APIs represent the previous record. The top-level properties are fields in the integration table.

{
  "id": 1,
  "source_payload": 
  {
    "humidity": 
    [
      {
        "temperature": 20.1,
        "pressure": 1003
      }
    ]
  },
  "create_ts": "2025-07-07T14:14:02.012"
}

"recordPath" properties for each value in the example record

"recordPath"

Field Value

"recordPath": "id"

1

"recordPath": "source_payload"

{
  "humidity": 
  [
    {
      "temperature": 20.1,
      "pressure": 1003
    }
  ]
}

"recordPath": "source_payload.humidity"

[
  {
    "temperature": 20.1,
    "pressure": 1003
  }
]

"recordPath": "source_payload.humidity[0]"

{
  "temperature": 20.1,
  "pressure": 1003
}

"recordPath": "source_payload.humidity[0].temperature"

20.1

"recordPath": "source_payload.humidity[0].pressure"

1003

"recordPath": "create_ts"

"2025-07-07T14:14:02.012"

The required "recordPath" property specifies the location of a JSON property in a record. It starts with the name of a field in the table followed by an optional JsonPath to a JSON property within that field.

The optional "timeFormat" property specifies the format of a time or a datetime embedded in a JSON string. The server uses it when it needs to transform or convert a time or datetime embedded in a JSON string into a FairCom time or timestamp. It also uses it when it needs to convert a time or datetime field value into a time embedded in a JSON string.

It is an enumerated string with one of the following values: "hh.mm.ss.ttt", "hh.mm.am/pm", "hh.mm.ss.am/pm", "hh.mm.ss", "hh.mm", "hhmm", "iso8601", and "utc".

The default value for "timeFormat" is the "defaultTimeFormat" property defined in the "createSession" or "alterSession" actions. If it is omitted there, it defaults to the value of the "defaultTimeFormat" property in the <faircom>/config/services.json file. If it is not there, the FairCom server defaults it to "iso8601" because the ISO8601 date is the defacto standard in JSON.

The enumerated string defines how the server parses a time in a JSON string. The following key explains the parts of each enumerated value:

  • hh is a one- or two-digit hour (0-24).

  • mm is a two-digit minute (00-59).

  • ss is a two-digit second (00-59).

  • ttt is a fraction of a second with up to 3 digits of precision (.0-.999).

  • . represents one character that can be one of the following values: :, ., -, or /.

  • "utc", "iso8601", and "hh.mm.ss.ttt" are the same.

The "variantFormat" property tells the server how to format the values of variant fields in its response to your request.

The "variantFormat" property has one of the following values: "binary", "json", "string", and "variantObject". It tells the server how to store and return values stored in variant fields. The default value is "json".

  • The server applies the "variantFormat" property to all variant fields affected by a JSON action.

  • If you want to control the variant format of each field, set the "variantFormat" property to "variantObject" and use variant objects to set the variant type and encoding of each variant field.

The following sections describe each value of the "variantFormat" property.

"variantFormat": "binary"
  • Use the "variantFormat": "binary" setting to encode a variant field's value as a JSON string.

  • The "binaryFormat" property specifies how to encode the binary value.

    • The "binaryFormat": "hex" setting uses hexadecimal to encode a binary value in a JSON string.

    • The "binaryFormat": "base64" setting uses Base64 to encode a binary value in a JSON string.

    • The "binaryFormat": "byteArray" setting uses an array of integer numbers to encode each byte in the binary value.

  • When "variantFormat" is "binary", the server sets the variant's type to "binary" and stores raw bytes in the variant field.

  • The "binary" variant type ensures maximum compatibility and performance across all FairCom APIs, including JSON DB, SQL, CTDB, and ISAM.

  • The server does not validate the binary value, which can be any sequence of bytes.

"variantFormat": "json"
  • Use the "variantFormat": "json" setting to encode a variant field's value as a JSON value.

    • A JSON value can be an object, array, string, number, true, false, or null.

    • A binary field value is returned in a JSON string.

    • The "binaryFormat" property specifies the format of a binary variant value embedded in a JSON string.

    • The "numberFormat" property causes the server to return numbers as JSON numbers or JSON strings.

  • When "variantFormat" is "json", the server sets the variant's type to "json" and stores a JSON value in the variant field.

    • The server validates JSON before it assigns it to a variant field.

    • The server converts a variant field to valid JSON before it returns the value of a variant field.

  • When "variantFormat" is set to "json" in "responseOptions" and the "type" of variant object is "binary", the JSON API returns the binary variant value embedded in a string or as a byte array depending on the "binaryFormat" property in "responseOptions". In other words, when "variantFormat" is set to "json" in "responseOptions", the API returns a JSON value. All variant values can be returned as native JSON values except for binary values because JSON cannot represent them. Thus, when a variant has a binary value and the response option is JSON, the API embeds the binary value in a JSON string according to the value of "binaryFormat".

  • The server converts variant values to and from JSON as follows:

    • JSON null: The server does not store a JSON null; instead it marks the variant field as NULL. If the field is not nullable, the API returns an error.

    • JSON string: The server stores a JSON string in the variant field as is. It includes the double quotes before and after the string's value and stores escaped characters without decoding them. For example, the server stores the JSON string, "my // string",  as "my // string". The server returns a JSON string exactly as it stores it.

    • JSON number: The server stores a JSON number in the variant field as is. A number consists of the following ASCII characters 1234567890+-.eE. For example, the server stores the JSON number -123.45  as the ASCII characters -123.45. The server returns a JSON string exactly as it stores it.

    • JSON true & false: The server stores a JSON true and false in the variant field as is. It stores and returns the ASCII characters true or false.

    • JSON object & array: The server stores a JSON object and array as is. When it validates the JSON, it minimizes whitespace while preserving the UTF-8 characters representing the actual object or array. For example, the server stores the array [1,2,3]  as the UTF-8 characters [1,2,3].

"variantFormat": "variantObject"
  • Use the "variantFormat": "variantObject" setting to encode a variant field's value as a variant object.

  • When the "insertRecords" and "updateRecords" actions have "variantFormat": "variantObject" in "params", you must assign a variant object to each variant field.

  • A JSON action returns a variant object for a variant field value when you put "variantFormat": "variantObject" in "responseOptions".

"variantFormat": "string"
  • Use the "variantFormat": "string" setting to encode a variant field's value as a JSON string.

  • This setting in the "responseOptions" property causes all "getRecords..." actions to return string values from variant fields.

    • Binary values are returned as a JSON string containing an embedded binary value. The "binaryFormat" property specifies how to encode the binary value.

    • Number values are returned as a JSON number embedded in a string, such as "123.45". The server ignores the "numberFormat" property in the "responseOptions" object.

    • Boolean values are returned as a "true" or "false" string.

    • String values are returned as a JSON string, which may contain JSON escaped characters.

    • JSON values are returned embedded in a string.

      • A JSON object is escaped, encoded, and embedded in a string, such as "{\"key\": \"value\"}".

      • A JSON array is escaped, encoded, and embedded in a string, such as "[1, \"my string\", true ]".

      • A JSON string is returned as a JSON string.

      • A JSON number is embedded in a JSON string, such as "-123.456".

      • A JSON true is returned as "true".

      • A JSON false is returned as "false".

      • A JSON null is returned as "null".

  • This setting in the "params" property for "insertRecords" and "updateRecords" actions requires a string value to be assigned to each variant field.

    • FairCom does not recommend this setting because it can only store strings in variant fields. Consider using the other "variantFormat" options to store many more types of data, such as binary, JSON objects, JSON arrays, numbers, etc.

The "targetDatabaseName" property is the optional name of the database that contains the target table. See

"databaseName" for more information.

The "targetOwnerName" property is the optional name of the account that owns the target table. See "ownerName" for more information.

The required "targetTableName" property specifies the name of the target table that has its transform steps replaced by this action.

Note

The fully qualified name of a table includes the database, owner, and table names.

The "transformStepMethod" property is a required string that specifies the type of transform, such as the "javascript" transform method that runs JavaScript to change the table's data or the "jsonToTableFields" transform method that extracts values from properties in a JSON field and stores them in other fields.

Note

The "transformStepMethod" property replaces the deprecated "transformActionName" property.

The value of the "transformStepMethod" affects the value of the "transformService" property. The following table defines the possible values of the "transformService" property when combined with the "transformStepMethod". Notice that some transform step methods are built into the server and do not require you to specify the "transformService".

"transformStepMethod" value

"transformService" value

"jsonToTableFields"

null or omitted

"tableFieldsToJson"

null or omitted

"jsonToDifferentTableFields"

null or omitted

"javascript"

"v8TransformService"

"siemensUdtPayloadToJson"

"siemensUdtPayloadToJson"

Note

If the "transformStepMethod" property is set to "javascript", the "transformService" property must be set to "v8TransformService".

The "transformStepName" property is an optional string that assigns a name to each transform step.

The "transformStepService" property is an optional string that specifies the name of a transform service, which is the user-defined name for a library (.dll, .so, .dylib) that implements the transform step method.

This property allows you to register your own transform libraries or use an older version of a FairCom library for backward compatibility.

Transform services are defined in the services.json file under the "transformServices" section. You can add and remove transform libraries to this list, such as your own transform libraries or specific versions of FairCom's transform libraries. Use the "serviceName" property to give each transform library a unique transform service name and use the "serviceLibrary" property to specify the filename of the library that implements the transform step method. On Windows the library filename ends with .dll. On Linux it ends with .so. On MacOS it ends with .dylib. Lastly, you must enable the transform service by setting "enabled" to true.

Example "transformServices" section in the services.json file.

"transformServices": [
  {
    "serviceName": "v8TransformService",
    "serviceLibrary": "v8transformservice.dll",
    "enabled": true
  }
],

Note

If the "transformStepMethod" property is set to "javascript", the "transformStepService" property must be set to "v8TransformService" or to a user-defined transform service name associated with the "v8transformservice.dll" service library.