Skip to main content

"getRecordsUsingSQL"

JSON DB "getRecordsUsingSQL" action runs a SQL query and returns the resulting records

The "getRecordsUsingSQL" action runs a SQL query and returns the resulting records. It is particularly useful when joining data from multiple tables, for aggregating data, for running ad hoc queries, and for sorting results by unindexed fields.

It is the most efficient way to return all records from a SQL query. When query results contain too many records to retrieve all at once, use a cursor, which makes it easy, fast, and efficient to paginate results. See Best ways to return data. Returning a cursor and returning records directly are mutually exclusive operations.

Only the SQL SELECT statement can be used. Any other SQL statements fail and return an error. The only unusual requirement is that JSON requires the double-quote character to be escaped by the backslash character. The semicolon that SQL requires at the end of a SELECT statement is optional.

Notice

JSON DB API transactions do not support SQL stored procedures, stored functions, and triggers. It also does not support "revertTransactionToSavepoint" when the transaction contains a call to "getRecordsUsingSQL" or "runSqlStatements". Unsupported actions return an error.

  • This and all FairCom JSON actions that run SQL statements use the FairCom "Direct SQL" API and as such access the data at full speed.

  • The "sqlParams" property is an optional array of objects (containing zero or more SQL parameters) embedded in the SQL statement. It specifies the value of each parameter. SQL parameters are used instead of embedded values in the SQL statements as it makes it easier to reuse SQL code. It also prevents SQL injection attacks.

  • Each object in the array represents one SQL parameter.

  • Each object contains the parameter’s name and its value.

  • For SQL and "sqlParams" objects, put a semicolon in front of the parameter name.

  • JSON DB API does not support specifying parameters by ordinal position because changing the parameter position in the SQL statement introduces bugs and makes the code harder to read.

  • If the fields returned by the SQL statement include JSON, you optionally can use the "includePaths" property to filter which JSON properties are returned.

  • If the results can be processed with one call to the operation, use "maxRecords" with a value of -1 to return all the results.

  • If the results are large, set "returnCursor" to true and use "getRecordsFromCursor" to process the results.

Performance considerations:
  • Results return less quickly when:

    • The SQL query processes many records.

    • There are many records in the results.

  • Do not call this action repeatedly to return paginated results; it reruns the SQL query. Instead, you must return a cursor.

  • By default, the "totalRecordCount" property in the response message is -1.  To get an exact count, set the "forceRecordCount" to true in the request message. It will count all the records returned by the query. Forcing a record count may be a resource-intensive and time consuming process when the query walks many filtered records.

Request examples

Minimal request

{
  "api": "db",
  "authToken": "replaceWithAuthtokenFromCreateSession",
  "action": "getRecordsUsingSQL",
  "params": {
    "sql": "SELECT * from athlete"
  }
}
{
  "api": "db",
  "authToken": "replaceWithAuthtokenFromCreateSession",
  "action": "getRecordsUsingSQL",
  "params": {
    "sql": "SELECT * from athlete",
    "returnCursor": true
  },
  "requestId": "2"
}
{
  "api": "db",
  "apiVersion": "1.0",
  "requestId": "3",
  "authToken": "replaceWithAuthtokenFromCreateSession",
  "action": "getRecordsUsingSQL",
  "params": {
    "databaseName": "ctreeSQL",
    "sql": "SELECT * from athlete WHERE earnings > ?",
    "sqlParams": [
      1000000000
    ],
    "returnCursor": false,
    "forceRecordCount": true,
    "sqlForwardOnly": true
  },
  "responseOptions": {
    "binaryFormat": "hex",
    "dataFormat": "objects",
    "numberFormat": "string",
    "includeFields": [],
    "excludeFields": []
  },
  "debug": "max"
}
{
  "authToken": "authToken",
  "result": {
    "dataFormat": "objects",
    "numberFormat": "number",
    "binaryFormat": "hex",
    "fields": [
      {
        "name": "id",
        "type": "bigint"
      },
      {
        "name": "changeid",
        "type": "bigint"
      },
      {
        "name": "name",
        "type": "varchar",
        "length": 30
      },
      {
        "name": "ranking",
        "type": "smallint"
      },
      {
        "name": "birthdate",
        "type": "date"
      },
      {
        "name": "playernumber",
        "type": "numeric",
        "length": 32,
        "scale": 6
      },
      {
        "name": "livedpast2000",
        "type": "bit"
      },
      {
        "name": "earnings",
        "type": "money",
        "length": 32,
        "scale": 4
      },
      {
        "name": "favoritesaying",
        "type": "varchar",
        "length": 500
      }
    ],
    "data": [
      {
        "birthdate": "1963-02-17",
        "changeid": 1291366,
        "earnings": 1700000000,
        "favoritesaying": "There is no 'i' in team but there is in win.",
        "id": 1,
        "livedpast2000": true,
        "name": "Michael Jordan",
        "playernumber": 23,
        "ranking": 1
      },
      {
        "birthdate": "1942-01-17",
        "changeid": 1291366,
        "earnings": 60000000,
        "favoritesaying": "Float like a butterfly, sting like a bee.",
        "id": 3,
        "livedpast2000": true,
        "name": "Muhammad Ali",
        "playernumber": 1,
        "ranking": 3
      },
      {
        "birthdate": "1895-02-06",
        "changeid": 1291366,
        "earnings": 800000,
        "favoritesaying": "Every strike brings me closer to the next home run.",
        "id": 2,
        "livedpast2000": false,
        "name": "Babe Ruth",
        "playernumber": 3,
        "ranking": 2
      },
      {
        "birthdate": "1940-10-23",
        "changeid": 1291366,
        "earnings": 115000000,
        "favoritesaying": "Everything is practice.",
        "id": 4,
        "livedpast2000": true,
        "name": "Pele",
        "playernumber": 10,
        "ranking": 4
      },
      {
        "birthdate": "1961-01-26",
        "changeid": 1291366,
        "earnings": 1720000,
        "favoritesaying": "You miss 100 percent of the shots you never take.",
        "id": 5,
        "livedpast2000": true,
        "name": "Wayne Gretzky",
        "playernumber": 99,
        "ranking": 5
      },
      {
        "birthdate": "1969-01-03",
        "changeid": 1291366,
        "earnings": 990000000,
        "favoritesaying": "Once something is a passion, the motivation is there.",
        "id": 6,
        "livedpast2000": true,
        "name": "Michael Schumacher",
        "playernumber": 1,
        "ranking": 6
      }
    ],
    "moreRecords": false,
    "requestedRecordCount": 6,
    "returnedRecordCount": 6,
    "totalRecordCount": 6
  },
  "debugInfo": {
    "request": {
      "authToken": "authToken",
      "api": "db",
      "action": "getRecordsUsingSQL",
      "params": {
        "sql": "SELECT * from athlete"
      },
      "debug": "max"
    },
    "serverSuppliedValues": {
      "databaseName": "ctreeSQL",
      "ownerName": null
    },
    "errorData": {
      "errorData": null
    },
    "warnings": []
  },
  "errorCode": 0,
  "errorMessage": ""
}
{
  "authToken": "authToken",
  "result": {
    "cursorId": "cursorId",
    "totalRecordCount": 6
  },
  "requestId": "2",
  "debugInfo": {
    "request": {
      "authToken": "authToken",
      "api": "db",
      "action": "getRecordsUsingSQL",
      "params": {
        "sql": "SELECT * from athlete",
        "returnCursor": true
      },
      "requestId": "2",
      "debug": "max"
    },
    "serverSuppliedValues": {
      "databaseName": "ctreeSQL",
      "ownerName": null
    },
    "errorData": {
      "errorData": null
    },
    "warnings": []
  },
  "errorCode": 0,
  "errorMessage": ""
}
{
  "authToken": "authToken",
  "result": {
    "dataFormat": "objects",
    "numberFormat": "string",
    "binaryFormat": "hex",
    "fields": [
      {
        "name": "id",
        "type": "bigint"
      },
      {
        "name": "changeid",
        "type": "bigint"
      },
      {
        "name": "name",
        "type": "varchar",
        "length": 30
      },
      {
        "name": "ranking",
        "type": "smallint"
      },
      {
        "name": "birthdate",
        "type": "date"
      },
      {
        "name": "playernumber",
        "type": "numeric",
        "length": 32,
        "scale": 6
      },
      {
        "name": "livedpast2000",
        "type": "bit"
      },
      {
        "name": "earnings",
        "type": "money",
        "length": 32,
        "scale": 4
      },
      {
        "name": "favoritesaying",
        "type": "varchar",
        "length": 500
      }
    ],
    "data": [
      {
        "birthdate": "1963-02-17",
        "changeid": "1291366",
        "earnings": "1700000000",
        "favoritesaying": "There is no 'i' in team but there is in win.",
        "id": "1",
        "livedpast2000": true,
        "name": "Michael Jordan",
        "playernumber": "23",
        "ranking": "1"
      }
    ],
    "moreRecords": false,
    "requestedRecordCount": 1,
    "returnedRecordCount": 1,
    "totalRecordCount": 1
  },
  "requestId": "3",
  "debugInfo": {
    "request": {
      "authToken": "authToken",
      "api": "db",
      "action": "getRecordsUsingSQL",
      "params": {
        "databaseName": "ctreeSQL",
        "sql": "SELECT * from athlete WHERE earnings > ?",
        "sqlParams": [
          1000000000
        ],
        "returnCursor": false,
        "forceRecordCount": true,
        "sqlForwardOnly": true
      },
      "apiVersion": "1.0",
      "requestId": "3",
      "responseOptions": {
        "binaryFormat": "hex",
        "dataFormat": "objects",
        "numberFormat": "string",
        "includeFields": [],
        "excludeFields": []
      },
      "debug": "max"
    },
    "serverSuppliedValues": {
      "databaseName": "ctreeSQL",
      "ownerName": null
    },
    "errorData": {
      "errorData": null
    },
    "warnings": []
  },
  "errorCode": 0,
  "errorMessage": ""
}

Use the getRecordsUsingSQL JSON API action to run a SQL query and return the resulting records

API actionsJSON DB APIjsonActionquery actionsget records using sqlgetRecordsUsingSQL

The "params" property is an object that contains an action's parameters. Each action defines its own required and optional properties.

Properties summary

Table 1. "params" properties summary

Property

Description

Default

Type

Limits (inclusive)

databaseName

specifies the name of a database

Defaults to the "defaultDatabaseName" value that is set during "createSession". If no default is set during "createSession", then "faircom" is used.

string

1 to 64 bytes

forceRecordCount

forces a query to return a record count when true

false

Boolean

true
false

ownerName

specifies the unique name of a schema in a database

""

string

1 to 64 bytes

returnCursor

configures the server to return a customized response

false

Boolean

true
false

sql

specifies an SQL statement

Required - No default value

string

1 to 32,000,000 bytes

sqlForwardOnly

returns records in forward-only order

false

Boolean

true
false

sqlParams

specifies optional values where each value becomes a parameter in the SQL statement

[]

array

transformCodeName

specifies the name of a "getRecordsTransform" code package to transform the output of the "getRecords..." action

Optional name of a code package

string

The string must contain the name of an existing code package with a "codeType" of "getRecordsTransform"



The "databaseName" property is an optional string that specifies the database that contains the tables. It defaults to the database name supplied at login.

Note

In the API Explorer, "defaultDatabaseName" is set to "ctreeSQL" in the "createSession" action that happens at login.

  • A zero-length "databaseName" is invalid.

  • Its limits are from 0 to 64 bytes.

  • If the "databaseName" property is omitted or set to null, the server will use the default database name specified at login.

  • If no default database is specified during "createSession", "defaultDatabaseName" will be set to the "defaultDatabaseName" value that is specified in the services.json file.

The "ownerName" property is an optional string from 1 to 64 bytes that specifies the account that owns an object.

The "returnCursor" property is an optional Boolean. When true, an action returns a cursor instead of directly returning records. It defaults to "false".

  • To retrieve records, call the "getRecordsFromCursor" action and pass the "cursorId" value into it.

  • When "returnCursor" is "true" and one of the "skipRecords", "maxRecords", and "reverseOrder" properties is specified, an error is returned.

    Note

    "returnCursor" is mutually exclusive with the "skipRecords", "maxRecords", and "reverseOrder" properties.

The "sql" property is a required string from 1 to 32,000,000 bytes. It is a SQL statement. A zero-length SQL statement is invalid.

The "sqlParams" is an optional array of values. Each value becomes a parameter in the SQL statement.

  • Each value may be a string, number, integer, true, false, or null.

  • The same value may be repeated within the array.

  • The order of the values must match the order of the SQL parameters.

  • The type of each value must correspond to the field type of the corresponding SQL parameter.

  • It is an error if the number of SQL parameters do not match the number of items in this array.

  • It is a best practice to use SQL parameters rather than embedding values in a SQL query.

  • It prevents SQL injection attacks.

  • Because the same value may be applied to multiple SQL parameters, items in the "sqlParams" array can have the same value.

The "transformCodeName" property is an optional string that specifies the name of an existing code package with a "codeType" of "getRecordsTransform".

When this property is present in a "getRecords..." transform, the server passes the query results to the code package as an array of objects. The code transforms the records and returns a valid JSON value. The server places the value in the "data" property of the response to the "getRecords..." action.

Properties summary

Table 2. "result" properties summary

Property

Description

Type

Limits (inclusive)

cursorId

specifies a unique identifier returned by the server

data

specifies an array or object that the server returns, such as records returned by a query. It is an empty array when there are no results available

array

Its contents are determined by the action

dataFormat

specifies the format of the data in the "data" property

string

"autoDetect"
"arrays"
"objects"

fields

specifies an array of objects set by the server, where each object is the definition of a field in a table defining the details of each field returned by a query

array

fields
.autoValue

specifies when and how the server automatically sets the field value

string

"none"
"incrementOnInsert"
"timestampOnInsert"
"timestampOnUpdate"
"timestampOnUpdateAndInsert"
"changeid"
fields
.defaultValue

specifies the default value of a field

string

0 to 65,500 bytes

fields
.length

specifies the length of a field's value in a record

integer

1 to 65500 

fields
.name

specifies the new name of the field

string

0 to 64 bytes

fields
.nullable

allows a field to contain a NULL value when true

Boolean

true
false
fields
.primarykey

adds a field to the specified ordinal position of the table's primary key when > 0

integer

0 to 32

fields
.scale

specifies the number of places to the right of the decimal point

integer

0 to 32

fields
.type

specifies the type of field

string

"bit"
"tinyint"
"smallint"
"integer"
"bigint"
"real"
"double"
"number"
"money"
"date"
"time"
"timestamp"
"char"
"varchar"
"lvarchar"
"binary"
"varbinary"
"lvarbinary"
"json"

requestedRecordCount

details how many records were requested in the "getRecordsFromCursor" action to determine if fewer records were returned than were requested

integer

0 to 2147483647

returnedRecordCount

details how many records were returned from the "getRecordsFromCursor" action to determine if fewer records were returned than were requested

integer

0 to 2147483647

totalRecordCount

specifies an integer, the server sets its value to the number of records returned by a query

integer

-1 to 99999999999999999999999999999999



This property controls when and how the server automatically sets the field value.

Specify only one of these values per field.

  • "none" indicates the server does not automatically set the field's value.

  • "incrementOnInsert" indicates the server automatically increments a field’s value each time the server inserts a new record. It applies to fields that are of the type of decimal or one of the integer types, such as "bigint". Only one field per table can have this attribute. The server returns an error when assigning this attribute to multiple fields. The JSON DB API automatically creates the "id" field as an "incrementOnInsert" field. If you apply this attribute to another field, it becomes the only automatically incremented field in the table. If you want that field to be the primary key, assign  "primaryKey": 1 to it.

  • "timestampOnInsert" indicates the server automatically sets a field’s value to the current date and time of an insert. It applies only to fields with a type of "timestamp".

  • "timestampOnUpdate" indicates the server automatically sets a field’s value to the current date and time of an update. It applies only to timestamp fields.

  • "timestampOnUpdateAndInsert" indicates the server automatically sets a field’s value to the current date and time of an insert and an update. It applies only to fields with a type of "timestamp".

  • "changeId" indicates the server uses the field for optimistic locking. The server automatically sets the field's value to the internal transaction number used during the last update of the record. This value changes each time the server updates the record. A table may only have one change tracking field. The field type must be "bigint".

    The JSON DB API automatically creates a "changeid" field with change-tracking functionality.

    Change tracking is optional in the CTDB and ISAM APIs. The application must create a 64-bit integer field and assign change-tracking functionality to it.

The "cursorId" property is a required string from 0 to 255 bytes. It is a unique identifier returned by the server.

  • The "getRecordsFromCursor" action uses it to quickly and efficiently retrieve paginated records.

  • Setting a zero-length "cursorId" in the request is invalid.

  • It is not returned when "returnCursor" is false.

Important

Do not assume the "cursorId" is a number embedded in a string.

The "dataFormat" property is an optional, case-insensitive string enum that defines the format of the response in the "data" property. The default format is an array of arrays. The alternative is an array of objects. The default for "dataFormat" can be changed during a "createSession" action by assigning a different value to the "dataFormat" property in "defaultResponseOptions".

There are three different (but similar) versions of the "dataFormat" property:

Two of those versions occur in a request and another occurs in a response. They all indicate how data is formatted.

  • "dataFormat" in the request inside "responseOptions" determines how the "data" property in the response is formatted.

    Possible values include:

    • "arrays"

      This is the default and causes the server to return results as an array of arrays, which is the most efficient.

    • "objects"

      This returns results as an array of objects. This is less efficient but is simpler to generate, read, and troubleshoot.

  • "dataFormat" in the request in the "params" object notifies the server how the "sourceData" property is formatted in the request. This version is rarely used because of the default "autoDetect" behavior.

    Possible values include:

    • "arrays"

      This causes the server to return results as an array of arrays, which is the most efficient.

    • "objects"

      This returns results as an array of objects. This is less efficient but is simpler to generate, read, and troubleshoot.

    • "autoDetect"

      This is the default and causes the server to automatically detect the format of the data in the "sourceData" property.

  • "dataFormat" in the response shows the client how the server formatted the "data" property.

    Possible values include:

    • "arrays"

      This is the default and causes the server to return results as an array of arrays, which is the most efficient.

    • "objects"

      This returns results as an array of objects. This is less efficient but is simpler to generate, read, and troubleshoot.

The "fields" property is a required array of field-type objects. There is one object for each top-level field in the array. Each object identifies the actual data type of the field as stored in the database.

A field-type object is used when creating a table. It contains one object for each field definition returned in the data.

Full request example

"fields": 
[
    {
      "name": "name",
      "type": "varchar",
      "length": 50,
      "scale": null,
      "defaultValue": null,
      "nullable": false,
      "primaryKey":1
   }
]

Full response example

"fields":
[
  {
      "name": "id",
      "type": "bigint",
      "length": null,
      "scale": null,
      "autoValue": "none",
      "defaultValue": null,
      "nullable": false,
      "primaryKey": 1
  }
]

The "requestedRecordCount" property is a signed, 32-bit integer set by the server in response to the "getRecordsFromCursor" method.

  • It makes it easy to know how many records were requested in the last call to "getRecordsFromCursor".

  • An application can use "requestedRecordCount" in conjunction with "returnedRecordCount" to determine if fewer records were returned than requested, which occurs when the cursor reaches the end of the recordset.

The "returnedRecordCount" is a 32-bit integer set by the server in response to the "getRecordsFromCursor" method.

  • It makes it easy to know how many records were returned from the last call to "getRecordsFromCursor".

  • An application can use "returnedRecordCount" in conjunction with "requestedRecordCount" to determine if fewer records were returned than requested, which occurs when the cursor reaches the end of the recordset.

The "totalRecordCount" property contains the total available number of records that can be returned from a query.

  • The "totalRecordCount" is set to -1, when the server does not know the total record count.

  • A very fast way to get the total number of records in a table is to call the "getRecordsByTable" method without applying a "tableFilter". This immediately returns the count without reading and counting records.

  • For most methods, the server does not calculate "totalRecordCount" because calculating it requires walking all records in the query, which may take a significant amount of time.

  • When the result is returned as a cursor, "totalRecordCount" is the total number of records that the cursor can traverse.

    Note

    This does not apply to cursor responses.

  • When the result returns records directly, "totalRecordCount" is the total number of records that can be retrieved – not necessarily the number of records returned.