MQTT Protocol – How it Works


MQTT is one of the most common used in IoT projects. It stands for Message Queuing Telemetry Transport.

It’s a lightweight messaging protocol that uses publish/subscribe operations to exchange data between clients and the server very fast and with low internet packets.

 

And here in our second blog in the IoT Blog Series, we are going to explain the MQTT protocol in details with examples and visual demonstration and real experiment by the end of the blog.

First of all, if you didn’t read the first blog of this series, I would recommend you go through it first from here.

 

MQTT is the best protocol for any IoT-based projects

That’s because MQTT has unique features you can hardly find in other protocols, like:

  • It’s a lightweight protocol. Therefore it makes it easy to implement in coding and fast in data transmission.
  • It’s based on a messaging technique. Of course, you know how fast your messenger/WhatsApp message is traveling over the internet to the target person and so the MQTT protocol. It uses a similar technique to make the data transmission faster.
    Well secured protocol.
  • Minimized data packets. Hence, low network usage.
  • Use less power. So, it saves the connected device’s battery. And that’s really a great feature for
  • battery power devices.
  • It’s real time! That’s is specifically what makes it perfect for IoT applications.

How MQTT works

Like any other internet protocol, MQTT is based on clients and a server. The server is the guy who is responsible for handling the client’s requests to:

  • receives their data
  • Send them data they required

But in contrast to other protocols, MQTT differs in the way it’s sending and receiving data from the clients of course.

MQTT server is called the broker and the clients are simply the connected devices.

When a device (a client) wants to send data to the broker, we call this operation a “publish”.

When a device (a client) wants to receive data from the broker, we call this operation a “subscribe”.

 

But these clients aren’t really sending and receiving data to/from the server “broker”, but what actually happens is that they are publishing and subscribing to topics. And the broker here is the one that handles the publishing/subscribing actions to the target topics.

Let’s take an example:

A device has a temperature sensor that wants to send his readings to the broker. Therefore, it should do 2 things:

  • Define the topic it wants to publish on, ex: “home/floor2/bedroom/temp”.
  • Then making a publishing operation with the value of the temperature as a message.

Hence the broker handles this publish to that topic the sensor defined.

 

But why should the broker care?

Because there are other devices “clients” in the network. These clients may ask for the data of the temperature sensor, and the one responsible for getting their question is the broker.

So, we have another device “client” which is a phone or a PC that’s connected to the same server “broker”.

The phone wants to know the current temperature, so it subscribes to the same topic as the temperature reading has been published to. That topic is “home/floor2/bedroom/temp”. Finally, the phone gets a“message” which is simply the value of the current temperature.


That takes us to the MQTT components, which are 5 as follows:

  • Broker, which is the server that handles the communication of clients by deciding who is subscribing to what topic and send him the data present on this topic or who is publishing to what topic and put the data on this topic.
  • Topics, which looks similarly the place a device want to put or retrieve a message to/from.
  • The message which is the data that a device receives “when subscribing” from a topic or send “when publishing” to a topic.
  • Publish, is the process a device do to share its data “just a message” over the IoT network.
  • Subscribe, which certainly is the opposite process. Hence, a device does subscribe to retrieve a data “just a message” from the IoT network.


Once you get these 5 concepts understood, you will know intuitively why MQTT is known as a pub/sub
“publish/subscribe” protocol!


How many devices you can connect to a broker

The number of connected devices “clients” to the broker “server” depends on the broker service provider and its limited number of clients in your subscription plan.

Certainly, it can reach a massive number of clients those are publishing and subscribing all the time. And surely all these clients can share their data with each other using the broker.

 

As a result, doesn’t this make MQTT like a mess!

Yes, and here comes the beautiful part of the MQTT, which is the availability of connecting a huge number of devices to the broker and huge amount of data that can be collected and shared between all devices.

The amazing part of this not only the huge number of these connected devices but also any device can get any other device data at any time.

That means any device can simply subscribe to anything “just a topic” he wants to know like:

  • GPS location data of car to publish to the AC to open once the car gets closer to the house.
  • Or the number of cars in a parking place to publish for all cars outside the parking building that this parking building is busy.
  • Or even the availability of milk in the fridge to publish a message to the market for purchasing new milk.

And more of IoT applications that MQTT made them easy to be achieved.


Why not HTTP

HTTP is slower, more overhead and power consuming protocol than MQTT. So, let’s get into each one separately:

  • Slower: because it used much bigger data packets for requesting data from the server.
  • Overhead: because of HTTP requests opens and closes the connection at each request, while MQTT stays online to make the channel always open between the broker “server” and clients.
  • Power consuming: since it takes a longer time and more data packets, therefore it uses much power.

Where the broker lies in the IoT system component

In the previous blog, we discussed the IoT system in details and its 4 components. If you need a memory refresh or this is your first blog reading in our IoT blog series, then you can check the previous blog from here.

One of these components was the IoT platform which is responsible mainly for connecting the devices together.

The IoT platform is just a cloud-based software that implements a broker server and some software with GUI to allow user manage and monitor his connected devices.

Therefore, MQTT broker is included implicitly inside the IoT platform.

But we won’t use an IoT platform here or in our upcoming IoT blogs and tutorials since we will only connect a few devices for practice and personal small projects. So IoT platform isn’t needed here.

 

Which broker to use

There are many brokers that implement the MQTT protocol. One of the most popular and commonly used is the mosquito broker.

Mosquitto is an open source message broker that implements the MQTT protocol. It’s lightweight and suitable for use on all devices from a low power single board like Arduino, ESP8266 to full computers and servers.

 

But rather than using the Mosquitto on a local PC, we need to use a cloud-based server that implements the Mosquitto broker to make our IoT projects controllable over the internet.

Cloud-based Mosquitto brokers are many, like:

ThingMQ, ThingStudio, MQTT.io, Heroku, and CloudMqtt.

And we will focus on MQTT in this blog.

 

CloudMqtt is one of the best and easiest cloud-based Mosquitto broker that has a free plan and well designed GUI to monitor the pub/sub processes and topics through an easy to use WebSocket UI.

Furthermore, you to set up your own CloudMqtt broker instance that will run on their hardware servers. Hence you can have an online broker that’s ready to use in your IoT project.

 

How to use CloudMqtt

Let’s give Cloud MQTT a try to figure out how it works since we are going to use it in our upcoming IoT tutorials.

Steps:

1- Go to the cloudmqtt and register an account.

2- Then press on the “+ create new instance” green button to get an instance of the MQTT based mosquito cloud broker that cloudMqtt offer.

3- Just write the broker instance name, like: “My_MQTT_Broker_Instance” and press “create new instance”

4- I have created a broker instance before and named it “try”, but on your account, you will find only the one you have created. Press on its name to see the details.

5- Here you can find the authentication details that any device will need to connect to your broker. We are interested only in the first 4 parameters.

6- cloudMqtt offers a pretty feature which is the WebSocket UI that allows you to monitor and publish and subscribe happens within this broker.


Now we have a free broker and we can connect any device to it.

 

Next, we will use our smartphone as a device since it’s already connected to the internet via wifi or data.

 

Example

Finally, the most pretty step has come, the IoT smartphone App that we will use!

 

There are many Apps there for connecting the smartphone to a broker and using it to publish and subscribe for topics. I have chosen a beautiful, simple and easy to use one, it’s called “IoT MQTT Panel Pro”.

You can download it from here for the Android phones.

It’s amazing and has many UI components to use as a visual indicator for the subscribed and published topics.


Let’s get started with it.

Steps:

1- Open the App and make sure your phone is connected to the internet.

2- Then ress on the “+” button to enter the broker authentication data.

3- Enter the broker data as follows “these data will differ from yours of course”.

You have to enter choose the password as it is, capital or small letters aren’t the same.

4- Then on the “Device list” press the “+” button and add a device that will connect to this broker. Let’s call it “Home”. Then press the “create” button.

5- You will see the broker that you have just made is ready, press it.

6- Let’s add some a button to use it to visualize publishing and a bar to visualize subscribing. Press “ADD PANEL” button.

7- You will see a list of many buttons and figures. Let’s select the second element, the “switch” to use as the event that’s responsible for publishing.

8- Then fill in the fields with the Panel name, the Topic which is “switch”, the message to be sent when the switch is on “called here Payload on” and the message to be sent when the switch is off “called here Payload off”

8- Press on “Use icon switch” then choose a color for the “on” icon to make it distinguishable.

9- Once finished, press “create”.

10- Press the “+” button to add the subscriber. Let’s choose the “Gauge”.

11- Then fill in the Panel name, the Topic that this Gauge will visualize its messages and the min, max values of the gauge.

12- Press the “create” button.

13- Then press on the top right arrows icon to connect this device to the broker.

14- As a result of the succeeded connection, the icon will be colored with orange to indicate that the device is now connected to the broker.

Run it


Now, let’s back to the WebSocket UI in the browser and monitor what’s going on.

When you press the switch to make it on, the device will publish a message which is “on” to a topic which is “switcher”. You will see that here in the WebSocket UI.

And when you publish a message “20” or any other number from the WebSocket UI on the topic “sensor”, the device subscriber “gauge” will get this message and updates the value to that number.

 

Furthermore, you have to draw your IoT structure like this image. That’s to make it easy for you to get a clear idea about your project elements and how each element will communicate with the others.


And check this video for a demo:


Congratulations! Now you have completed your first IoT project using MQTT protocol with a connected device!

 

What’s next!

Now, we are ready to move to another tutorial to know how we can implement what we have just done here into a real project.

Yes, a real project means using a hardware development board like the popular Arduino Uno board.

We will see how to program the Arduino to connect to the internet and connect to the broker to publish and subscribe for topics.


Of course, Arduino alone can’t connect to the internet, so we will use a shield that can connect the Arduino to both the internet and the broker. That shield is 1Sheeld+ which will facilitate the coding for our upcoming IoT tutorial.

 

Furthermore, I would recommend you check this getting started tutorial for a quick impression on 1Sheeld+ board and how it can be used.

 

If you liked this blog, please leave an encouraging comment below.

Also, don’t hesitate to comment with any question you want 🙂

 

References

readitquik.com, mosquitto.org, istsos.orgspindance.com




Related Blogs

Arduino Temperature Humidity System

IoT Arduino Temperature and Humidity System

An Arduino Temperature Humidity System is one of the most popular projects that most of us have tried or even saw before. It’s because of the ease of use of this DHT sensors those which give pretty accurate temperature and the humidity readings with a bit code. But as another tutorial of this continuous IoT …

Arduino Door Lock

IoT Arduino Door Lock using 1Sheeld and Smartphone

  Here we are again with another IoT tutorial using Arduino with 1Sheeld and just a smartphone. In previous tutorials, I have gone through controlling a lamp and fan using a phone and MQTT protocol. Today, I am going to get through another but more interesting IoT real-world project, controlling the Arduino Door Lock from …

IoT Fan Control

Arduino IoT Fan Control using 1Sheeld and Smartphone

Today, I have come with a new and quick IoT tutorial, IoT Fan Control! In a previous tutorial, I have gone through controlling a lamp using a phone and MQTT protocol. That was the first IoT tutorial using 1Sheel. Now,  I am going to get through my second IoT real-world project. So, in this tutorial, …

Arduino Light Control

IoT Arduino Light Control using 1Sheeld and Smartphone

Controlling things around us wirelessly is a common thing that Arduino geeks do all the time like Arduino Light Control projects using IR remote, Bluetooth, wifi or even RF modules. Certainly, this kind of projects is categorized under the home automation projects. But what if we have taken this to a whole different level where …

IoT

IoT: Understanding the Concepts

Connected (IoT) Machines have conquered the world in a way humanity has never undertaken before. That ’s not a phrase from the Terminator movies, it’s just the reality of nowadays. Statistics show that the number of connected devices in 2018 is 23+ billions. This is almost 3 times the number of humans which is 7.6 …