For specifically configured topics, the database can store MQTT messages and forward them to subscribers. These topics are called "Store & Forward Topics." This feature enhances MQTT to guarantee delivery for specific topics even when a subscriber is unavailable to receive them.
When an existing subscriber to a Store & Forward Topic becomes unavailable and later becomes available, the MQTT Broker forwards messages to the subscriber starting after the last successfully delivered message.
This feature enhances MQTT to guarantee delivery for specific topics even when a subscriber is unavailable to receive them.
This requires the MQTT Broker to do the following:
Unless the Store & Forward Topic is configured with "retentionUnit": "forever", the MQTT Broker automatically removes expired messages.
Expired messages can no longer be forwarded. Thus, when a subscriber is unavailable longer than the retention period and later becomes available, the MQTT Broker can only send it unexpired messages.
For configured topics, the MQTT Broker collects and stores messages in anticipation of future subscribers.
The MQTT Broker delivers messages in the order they were originally received.
An MQTT message for a Store & Forward Topic may contain any binary content up to 2GB in length per message.
This feature is available only for topics that are configured specifically for this purpose.
Because storing messages impacts performance and storage, the MQTT Broker only stores and forwards messages for Store & Forward Topics.
The broker can be configured to do the following with Store & Forward Topics:
Creating a Store & Forward Topic
A Store & Forward Topic is created by setting the storeAndForward property to True in the CreatePersistenceTopic operation. The following optional configuration properties can be set:
The default rate per second for delivering stored messages is 100.
"maxDeliveryRatePerSecond": 100 - This is the maximum delivery rate for messages being delivered to a subscriber when it becomes available or when it is a new subscriber and asks for all stored messages to be delivered to it. The actual rate will be bounded by the network, local resources, and server performance. This keyword provides an artificial “throttling” of the delivery rate to avoid saturating the network and other resources.
"newSubscriberDeliveryMode": "NewMessages"
Values for "newSubscriberDeliveryMode":
"NewMessages" - A new subscriber only receives new messages published to the topic.
"StoredMessages" - A new subscriber receives all stored messages published to the topic.
Depending on the quantity of stored messages, it may take a significant amount of time before current messages are delivered.
A new subscriber may not be sent all messages ever received by the MQTT Broker.
"newSubscriberMaxStoredMessages": 0 - This applies only to new subscribers and it limits the number of previously stored messages that the MQTT Broker will send. Thus, it applies only when the Store & Forward Topic is configured as "newSubscriberDeliveryMode": "StoredMessages"
"newSubscriberMaxStoredMessages": "all" is used to set no limits on how many messages will be sent to a new subscriber.
"newSubscriberMaxStoredMessages": 0 is used when no previous messages should be sent to a new subscriber.
Example 1:
Configuration message (JSON) to create a Store & Forward Topic:
{
"operation": "CreatePersistenceTopic",
"persistenceTopic": "Example1",
"tableName": "Example1",
"storeAndForward": "True",
"tableRetentionPeriod": 3,
"tableRetentionUnit": "month",
"maxDeliveryRatePerSecond": 100,
"newSubscriberDeliveryMode": "StoredMessages",
"newSubscriberMaxStoredMessages": 1000
}
The Example 1 topic will store 3 months' worth of messages in the table named Example1. If an outage occurs, the messages missed by existing subscribers will be delivered at the throttled rate of 100 messages per second, to keep from saturating the system. New subscribers will receive the last 1000 stored messages.
Example 2:
Configuration Message to Remove a Store & Forward Topic
{
"operation": "RemoveStoreForwardTopic",
"storeForwardTopic": "Example1"
}
In Example 2, “myTopic” will remove the Topic, which will remove all currently stored messages and will also remove all files on disk used for storing the messages.