The message queuing telemetry transport (MQTT) protocolis a key contender for the most favored method of datatransference. The main reason why is MQTT’s open-sourcedesign and lightweight stature make it well suited to connectdisparate devices to supervisory control and data acquisition(SCADA) systems as well as other industrial networks.
As Omer Qadri, product marketing manager for edge andHMI products at Aveva, explains, MQTT uses a publish/subscribearchitecture that reduces bandwidth utilization by 95%compared to traditional polled communications and client/server communications using the hypertext transfer protocol(HTTP). “An HTTP header is typically around 8,000 bytes,”he says, “but the MQTT protocol uses only two bytes and afew lines of code.” This is key in an era where millions of IIoT(Industrial Internet of Things) devices have been deployed,many with low internal memory and processing power.
Besides having a much smaller footprint on the network,MQTT’s publish/subscribe architecture is also flatter than thearchitecture used by traditional industrial automation protocols,such as Modbus, EtherNet/IP, and Profinet. “This [MQTT]architecture replaces the traditional automation pyramid,”observes Garrett Schmidt, senior product manager for communicationinterfaces at Phoenix Contact USA.
Whereas clients in a client/server architecture communicatedirectly with an endpoint or server, publishers and subscribers—message senders and recipients, respectively—nevertalk directly to each other in a publish/subscribe architecture.Rather, they communicate with an intermediator called abroker; the publisher supplies the broker with data and thesubscribers consume the data.
“The broker can reside anywhere—on the cloud, on a privateserver, or just running on a PC somewhere,” says Schmidt. “Itfilters the incoming messages and distributes them to theappropriate subscribers.”
He adds that this decoupling of publishers and subscribersenhances flexibility in IIoT applications in at least three ways:“First, publishers and subscribers only need to know how tocontact the broker, not each other. Second, a broker can storemessages for clients that are not online and deliver them whenthe resource is available. And third, operations do not have tobe interrupted when waiting to receive or publish a message tocoincide with the asynchronous nature of most client libraries.”
MQTT also has the advantage of being an open-sourceprotocol built upon TCP/IP (transmission control protocol andinternet protocol). In essence, MQTT permits users to sendTCP/IP messages back and forth, according to Arlen Nipper,a co-creator of MQTT and president and chief technologyofficer at Cirrus Link Solutions.
Like HTTP, MQTT defines only a transport protocol. Itdoesn’t provide for security; it relies on TCP/IP for that.Like HTTP, MQTT also doesn’t define a payload specification.Although being payload agnostic offers the flexibility to transferany payload, including those from legacy systems, it cancomplicate the connection of some devices. In these cases, aprogrammer would be required to translate the data.
To eliminate this translation work and streamline implementation, the open-source Sparkplug payloadspecification was released in 2016. “Itmarked the first attempt to standardize on aninteroperable format for MQTT in industrialapplications,” says Josh Eastburn, director oftechnical marketing at Opto 22.
In 2018, the Eclipse Foundation sponsoredthe Tahu Project, which collected referenceimplementations of Sparkplug. The result hasbeen the emergence of plug-and-play IIoTdevices using MQTT.
Nipper says Sparkplug does for IIoT whatthe hypertext markup language (HTML) didfor the Internet of People. Consequently, heis expecting IIoT applications to explode, asthe Internet of People did once both HTTPand HTML were defined.
Explosive growth expected
MQTT is already making significant inroadsin industrial automation, as well as enjoyingwidespread use in other applications. Facebook,for example, adopted it as the transportlayer for its Messenger app back in 2011.
“Literally overnight, 800 million people wereusing MQTT,” notes Andy Stanford-Clark,MQTT’s other co-creator and distinguishedengineer and master inventor at IBM UK.
Since then, other Big Tech companies havefollowed suit. Amazon’s AWS, Microsoft’s Azure, IBM’s Watson, and Google IoT platforms,for example, all are using MQTT. Withsuch broad uptake, MQTT overtook HTTP in2018 as the transport protocol of choice forthe Internet of Things, reports Stanford-Clark.
Many automation suppliers expect MQTTto eventually dominate the industrial networkingspace. “We believe MQTT willbecome the de facto industrial standard inthe next 10 years,” predicts Qadri. “It willenjoy widespread adoption as industryreplaces legacy Modbus, OPC, and othertelemetry protocols that are still predominantin SCADA applications.”
Key milestonesAndy Stanford-Clark, MQTT co-creator, master inventor at IBM UK
The success of MQTT in the consumer spacehas obscured some fundamental facts aboutits origins. Namely that the protocol has beenaround for 23 years now and was originallydeveloped for industrial automation, specificallyfor Phillips 66.
The development of MQTT occurred afterAT&T had been broken up and a number ofvendors began offering their own SCADAsystems to deliver data in real time by satellite.“Every one of those companies had aproprietary transport layer,” recalls Nipper,who, at the time, was with Arcom ControlSystems Inc., a company he had co-foundedand which is now part of Eurotech.
The one exception was AT&T, whichdesigned its new SCADA offering to runnatively on TCP/IP. Phillips 66 had installedone of these systems and asked Nipper forhelp with increasing the efficiency of realtimedata flows between field devices andmultiple data consumers. “Polling over aVSAT [very small aperture terminal] is slow,”explains Nipper. “And it was very expensive ifyou had hundreds of sites, like we did at Phillips66.” Other constraints included the use of devices reliant on 8-bit embedded microprocessorsand 300-baud communications.
Because the SCADA manager at Phillips66 wanted to replicate the success the ITdepartment had been having with messageorientedmiddleware (MOM) from IBM, heintroduced Nipper to IBM’s Stanford-Clark.In 1999, the pair developed MQTT for MOMbasedSCADA.
Despite being an efficient, open-sourceprotocol, MQTT would not gain muchmomentum for nearly a decade. “It wasn’tuntil the protocol became available in a royalty-free license that it began to catch onoutside of IBM,” explains Eastburn. “In 2010,Mosquitto, the first open-source MQTT brokerwas released, proving that MQTT had alife outside of IBM and marking a turningpoint in its adoption.”
Two other milestones in industry’s adoptionof the protocol occurred in 2011. Firstwas the Eclipse Foundation initiating thePaho Project, which collected MQTT clientsimplemented in various languages. “In2011, IBM and Eurotech donated MQTT clientimplementations in C and Java to thefoundation, allowing for a complete MQTTsystem to be built from open-source components,”says Eastburn.
That same year, IBM also began the standardizationprocess of MQTT with the Organizationfor the Advancement of StructuredInformation Standards (OASIS) ultimatelyadopting version 3.1.1 as a standard in 2014.Then, in 2016, the International Organizationfor Standardization (ISO) and theGeneva-based International ElectrotechnicalCommission (IEC) also approved it asISO/IEC 20922:2016.
To keep up with advances in related technologies,OASIS released version 5 of MQTTin March 2019. This version allows users todo new things with MQTT via the cloud,large distributed infrastructures, and clustersof multiple brokers. “We were carefulnot to let too many things creep into it, aswe have to stick to the founding principles ofkeeping the protocol easy to understand andnot very chatty on the wire,” says Stanford-Clark. ISO is currently considering adoptionof version 5 as well.
Arlen Nipper, MQTT co-creator, president and CTO at Cirrus Link SolutionsPotential application concerns
Despite the success MQTT and its publish/subscribe architecture have had, it’s not optimalfor every application, according to KennethTran, founder and CEO of Koidra Inc., asupplier of artificial intelligence-driven IoTtechnologies. “We find the pub/sub model isoften not the best solution for higher-levelapplications, in part because they must beconfigured to consider asynchronous dataavailability,” he says. “In a factory, it’s typicalto have many sensors connected to acontroller, server, or sensor hub in the field.”
In the IoT systems that Koidra offers, anon-premises IoT hub aggregates data pulledfrom a factory’s sensors via smaller, localsensor hubs. “These IoT hubs perform lightweightdata cleansing, processing, and compression—and then push the resulting informationto the cloud,” explains Tran. In thiscase, “because there is only one consumer,the central cloud, the pub/sub frameworkwould be overkill.”
Another potential pitfall is getting lockedinto a particular vendor’s proprietary IoT platform.This can happen with data sent to thevendor’s cloud services, which can happendespite MQTT’s open-source origins. In theseinstances, users buy their edge device andsoftware and connect it using MQTT.
“But you have no access to the data if itall remains within the vendor’s cloud environment,”explains Travis Cox, co-director ofsales engineering at Inductive Automation.
Consequently, Cox urges users to makesure the configuration of these cloud-basedsystems allows them access to their data.“You can send the data to their cloud,” hesays, “but ultimately you should be able tosend that data to your systems too.”
A second way to get locked into proprietarytechnology, despite the use of MQTT, isthrough the payload format. This can occurbecause MQTT can transfer payloads in anyformat, including a vendor’s proprietarybinary format.
“If you don’t understand what’s being sent,then it’s going to be very hard for you to takeadvantage of it,” Cox points out. To avoid thispitfall, insist on either having a definition thattells you what the data look like or use theopen-source Sparkplug payload specification.
Cox also recommends building a resilientarchitecture. “If you were to lose a connectionor access to your central broker, thenyour applications would be blind,” he says.One way that he suggests for building resiliencyagainst such interrupted connectionswould be to store data in a local cache soit can be forwarded when the connectionis re-established. Another way to enhanceresiliency is to have two brokers, so that onecan continue working if the other should fail.
