Event streaming concepts
Event streaming combines messaging with an event log.
Messaging uses the pub/sub model where applications publish messages to topics on a message broker. They also subscribe to topics on the broker to receive each message published on those topics. Messaging provides real-time push notifications, fast delivery, and guaranteed delivery. The problem with messaging is that once a message broker delivers a message, there is no record of the message.
Event streaming uses an event log to persist each message in the order the broker receives it. Applications read the event log to replay messages. They can replay messages from the beginning or end. They can paginate through messages efficiently in batches and filter messages to process only those messages that interest them.
FairCom DB and MQ seamlessly converge FairCom's database engine with its MQTT broker. The messaging engine persists each message to a special integration table that has built-in features designed for messaging. The engine creates an integration table for each topic. Each message is a record in the integration table, and each table can transform incoming messages using the built-in V8 JavaScript engine. It also has the unique ability to deliver each message to various outputs and APIs including MQTT, REST, SQL, JSON DB API, etc. The integration table persists delivery state to guarantee delivery to offline subscribers when they come online.
Applications can read the records in each integration table to retrieve messages from the beginning, end, or at any point along the way. The engine can track where applications left off reading and start where they left off. Applications can use SQL and JSON DB to filter messages to process messages that interest them.
It makes it easier to use MQTT in any application.
MQTT drivers provide many ways to use MQTT incorrectly or inefficiently. The JSON MQ Message API provides a single standard way to use MQTT that is efficient and easy to use correctly.
It provides intuitive names and simplifies using advanced MQTT 5 features.
You can use any REST tool, such as POSTman, to publish and subscribe to MQTT.
It makes it easy to centrally manage the settings of all MQTT clients.
MQTT expects each client to configure its session settings, which is difficult and expensive.
The JSON MQ Message API provides the
"alterMqttSession"action to alter the MQTT session settings of any MQTT client.
It guarantees you retrieve messages in published order.
MQTT has corner cases that deliver messages out of order.
It efficiently retrieves messages in batches.
MQTT pushes messages one at a time.
It delivers messages when you are ready to process them.
MQTT pushes messages as soon as they are published.
It simplifies and standardizes security.
The MQTT 3.1 and 5 specifications leave security largely undefined, making security prone to misconfiguration and creating security incompatibilities across brokers.
The JSON MQ Message API uses HTTPS and a simple, standard, and proven security model for authentication that supports client certificates or usernames and passwords.
It works through firewalls.
Some organization's security policies reject MQTT from passing through their firewalls.
It eliminates the need for browser-based applications to use WSS to send and receive MQTT messages.
Browser-based applications cannot directly use MQTT, but they can upgrade HTTPS to WSS and tunnel MQTT over WSS to a broker that supports this capability.
Client certificates do not work over WSS.
When a request succeeds, the server sets the "errorCode" property to 0.
When a request fails, the server sets the "errorCode" property to a FairCom Error Number.
The server sets the "errorMessage" property to a string describing the particular failure and suggesting how to correct it.
MQTT requires a client to create an authorized connection before publishing and subscribing to topics. This is not the case for the JSON MQ Message API, which can publish and retrieve messages with or without connecting to MQTT sessions.
The JSON MQ Message API uses the "createSession" JSON Action to authenticate an account. This account must have appropriate permissions before the JSON MQ Message API allows it to perform actions, such as publishing to a topic, retrieving messages from a topic or subscription, and modifying a connection and subscription.
An MQTT session contains information about a client's subscriptions and settings. An MQTT broker stores this information and retrieves it when a client connects.
Each MQTT client has a unique identifier called the "clientName" that associates it with precisely one MQTT session.
The MQTT protocol is stateful and exclusive. If an authorized client uses the same client identifier as another connected client, the MQTT broker connects the new client and disconnects the existing client.
The JSON MQ Message API works differently because it is stateless. It allows multiple processes to simultaneously publish and retrieve messages using the same client identifier, which is specified using the "clientName" property. Unlike the MQTT protocol, the JSON MQ Message API does not disconnect an existing MQTT session that uses the same client name. This allows a broker administrator to publish messages on behalf of a device without disconnecting the device.
Warning
Be sure to take extra care when using the stateful "getMessagesFromMqttSession" action to ensure that no other process is using the same MQTT client identifier; otherwise, each process that retrieves messages denies other processes from retrieving them. If your usage of MQTT allows multiple processes to use the same client identifier, use the "getMessagesFromMqttTopic" to retrieve messages safely.
When you omit "clientName", the "connectToMqtt" action generates and returns a unique client name. You should persist and reuse generated client names to avoid polluting the MQTT broker with abandoned sessions.