Faster, better with Sparkplug B
Because of its flexibility, MQTT has been adopted for many IoT solutions. It imposes few naming conventions and accommodates any payload of an appropriate size with equal efficiency. Developers can customize MQTT’s basic structure (topic paths) and behaviors (retained messages, persistent sessions, and so on) for their MQTT client as best suits the application, with no extraneous design elements to account for. However, this same flexibility can also be a limitation, particularly for the kind of large, diverse networks characteristic of industrial installations.
With no uniform naming standard, no common data format, and no contextual information on published topics, the details of each publisher’s data—which topic paths to subscribe to, how to decode or interpret the payload, the appropriate QoS level, and how to recognize a publisher’s LWT—must be known ahead of time in order to be discovered and used by subscribers. Since each device is free to use different conventions, configuring a large network can require significant effort akin to mapping conventional tag data between applications. These factors slow the pace of development and may even inhibit interoperability, adding to the cost of integration and undermining the goals of IIoT.
Likewise, while MQTT includes features to monitor client connection status and data quality, there’s no guarantee that a vendor will utilize them in a field device or software client.
These weaknesses, and other observations about MQTT’s adoption in the marketplace, inspired the development of the open-source Sparkplug MQTT Topic Namespace and Payload Definition, under the leadership of MQTT co-inventor Arlen Nipper and Cirrus Link Solutions. The current version, Sparkplug B (SpB), expands on the basic MQTT architecture to address common industrial use cases and adds a handful of important implementation details to MQTT clients that conform to the specification.
Components of an MQTT/Sparkplug B network
Sparkplug adds to and clarifies the roles of basic MQTT clients, enabling new features and more explicit messaging that are the basis for other enhancements in the specification.
The specification distinguishes between two types of MQTT clients:
- MQTT/Sparkplug B Edge of Network (EoN) Nodes: These clients provide physical and/or logical gateway functions to enable MQTT/Sparkplug B communications for legacy devices and sensors. EoN nodes also include smart devices and sensors capable of publishing their own Sparkplug B data, process variables, or metrics directly to an MQTT broker.
- MQTT/Sparkplug B Application Nodes: These are software clients, optionally including one primary application responsible for sending commands and receiving historical data (more information below). An MQTT/SpB application node may also be a gateway to legacy software systems.
Sparkplug nodes use predefined message types (see next section) to distinguish between internal data and data originating from connected devices. Nodes are also responsible for reporting on the state of their connected devices, if present.
Any MQTT 3.1.1-compliant broker will support Sparkplug B payloads, and one or more can be used as needed for redundancy, high availability, or scalability.
Topic namespace
Sparkplug defines a standard format for MQTT topic paths, creating a unified namespace for all SpB clients on the network:
spBv1.0/<Group ID>/<MESSAGE TYPE>/<Edge Node ID>/<Device ID>
Payload definition
Sparkplug B defines a standard, structured, data-rich but efficient payload format.
The core payload contains a structured series of key-value pairs defining an array of metrics and associated metadata. The specification defines a variety of optional fields for each metric, such as name, description, datatype, checksum, historical data flag, and many more.
Data type indicators accommodate complex types common in industrial applications, like matrices and user-defined types (UDTs). The payload specification also permits an array of custom properties for each metric, such as engineering units or scaling limits, that are published along with the primary process variable or data object.
The full payload is then timestamped, sequenced, and encoded using Google protocol buffers (protobufs), an efficient, interoperable, binary representation of the structured data, which maintains a small on-the-wire footprint without sacrificing complexity. When decoded, the payload is typically represented in JavaScript object notation (JSON) but is compatible with many formats. Here is an example of a simple SpB payload decoded into JSON:
{
"timestamp": 1486144502122,
"metrics": [{
"name": "My Metric",
"alias": 1,
"timestamp": 1479123452194,
"dataType": "String",
"value": "Test"
}],
"seq": 2
}
State management
Sparkplug introduces the concept of birth and death certificate messages to define and ensure the use of appropriate state monitoring mechanisms.
The death certificate simply formalizes the use of MQTT’s last will and testament message. Triggered when a client’s keep-alive timer has expired, the death certificate indicates an unexpected disconnection. The topic path follows the standard Sparkplug format, using the message types NDEATH and DDEATH, indicating that the certificate pertains to a node or to an attached device, respectively.
The birth certificate, on the other hand, is a new addition of the Sparkplug spec. It is a special message, utilizing message types NBIRTH or DBIRTH, that a client is required to publish for itself and each of its attached devices when first coming online. These messages contain all the published topics for that client or device and inform all subscribers that the client or device is online.
Primary application
While an MQTT network may contain any number of application clients, an industrial setting often has a single critical application used for operations management or process control, like an IIoT or SCADA host. This application is typically the only one permitted to send commands into the network. The Sparkplug specification adapts the MQTT framework to this reality by allowing one application node to be designated as the primary application.
An SpB-compliant primary application—like Cirrus Link’s MQTT Engine module for Ignition SCADA or Canary Labs’s MQTT Data Collector—publishes special birth and death certificates using the topic STATE/<client ID>. Unlike regular death certificates, these messages are published with QoS 1 and are retained by the broker to ensure that any MQTT client can identify the primary application’s state at any time. This mechanism enables two additional functions that are critical to building fault-tolerant industrial networks.
- Enhanced failover. If an MQTT client loses its current broker connection and other connections are available, it can switch automatically. In addition, a Sparkplug B-compatible client can also switch connections based on the value of the primary application’s STATE topic. When the primary application’s connection is interrupted, its death certificate is published by the broker, indicating that STATE has changed to “OFFLINE.” SpB clients will see this and begin searching through their alternate connections for a broker that reflects an “ONLINE” status for the application.
- Store-and-forward historization. In combination with the flags built into the Sparkplug B payload, a client can indicate that a message contains historical, rather than real-time, data. If capable, a client can begin storing records any time it or the primary application disconnects, queuing them up until it is assured of safe delivery, then publishing them as historical records. Since these messages are distinguishable from real-time data, store-and-forward provides better continuity for time-series data and time-critical operations than MQTT’s standard quality of service levels.
The primary application mechanism enables smart failover and historization.
Advantages for industry
Interoperability
The Sparkplug B specification addresses the potential for inconsistency in MQTT implementations by defining standard client roles and data interfaces designed around industrial applications. By standardizing on Sparkplug B, MQTT clients from different vendors can identify, interpret, and make use of published data without needing to know the details of the originating client. Even brownfield sites with legacy automation networks and devices can leverage Sparkplug because of its explicit support for gateway-attached devices.
Unified namespace
This level of interoperability satisfies a critical requirement for digital transformation by making possible an enterprise-spanning unified namespace, a common data source defining all business data. Instead of hunting for operations data spread throughout a hierarchy of applications (PLC > SCADA > MES > ERP, for example), each with its own schema and data structures, MQTT devices and applications all contribute to a single data endpoint: the MQTT broker.
Enterprise clients can consume data from the field via the broker, using Sparkplug to provide a common exchange format and uniform context. Their results can be published back into the network to be detected, interpreted, and acted on by other clients. In this way, data can be shared seamlessly throughout the organization, eliminating data silos, reducing the potential for discrepancies, and allowing a connected system to scale up smoothly.
Enhanced data integrity
Sparkplug also improves data integrity and consistency with its combination of reliability features. By enforcing the last will and testament mechanism, all subscribers are notified when pertinent data becomes stale, and the birth certificate lets them know when data is fresh again. And for applications like those in the regulated industries that require historical data for auditing or root cause analysis, store-and-forward historization provides protection against critical data loss in the face of network or client instability.
Low administration
The combination of Sparkplug’s enhanced state management and interoperability unlocks an additional contributor to network scalability by reducing the administrative overhead required to integrate each node and device.
As mentioned, integrating traditional industrial data involves time-consuming tag/variable mapping between applications. But with Sparkplug’s birth certificate mechanism, this process is sped up by orders of magnitude. Subscribers, the primary application in particular, can use the information published in the birth certificate to map out available topics in a matter of moments. Since a birth certificate is required from each client whenever it connects to the network or updates its topic structure, the tag hierarchy can be discovered automatically. And thanks to Sparkplug B’s payload structure, it comes complete with all metadata and UDT definitions.