MQTT Concepts for Iot project

Introduction to MQTT

I recently picked up an ESP32-S3 with a built-in touchscreen — a sleek little board that instantly sparked an idea: what if every member of our APEX team at UniLaSalle Beauvais had their own personal, pocket-sized device to send and receive quick updates? Something tactile, fun, and a bit futuristic. Think messages like “Lunch in 10 minutes” or “Meet in the APEX room” — sent with a tap and confirmed by a subtle vibration.

I didn’t want to rely on rigid enterprise tools like Teams or platforms buried behind logins and firewalls. I wanted a fast, local, flexible network that we could fully control. A way to communicate instantly and privately — something built for us.

That’s where MQTT came in. This unassuming little protocol turned out to be the perfect backbone: lightweight, real-time, and rock solid. It made the system feel less like engineering and more like crafting a tool with personality.

This guide walks through how we built it — from setting up the broker to sending your first message. Whether you’re building something similar or just exploring how to get embedded devices talking simply and efficiently, you’ll find practical tips and working examples to get started.

Core Terminology

  • Broker: The centralized server that handles all message routing between clients.

  • Client: Any device or application that publishes or subscribes to messages.

  • Topic: A UTF-8 string used to route messages logically. Hierarchically structured (e.g., region/device/status).

  • Payload: The message content, often JSON or binary.

  • QoS (Quality of Service): Defines message delivery guarantees:

    • 0: At most once (fire and forget)
    • 1: At least once
    • 2: Exactly once

  • Retained Message: A last-known-good message that persists on a topic for late subscribers.

Installing and Configuring the MQTT Broker

MQTT brokers like Mosquitto are available on all major platforms. Here’s how to get started depending on your OS:

Installing Mosquitto on Linux

1
2
3
4
sudo apt update
sudo apt install mosquitto mosquitto-clients
sudo systemctl enable mosquitto
sudo systemctl start mosquitto

Testing the Broker

1
2
3
4
# Subscriber terminal
tmux new-session -d -s mqtttest 'mosquitto_sub -h localhost -t diagnostics/test'
# Publisher terminal
mosquitto_pub -h localhost -t diagnostics/test -m "Test message"

Installing on macOS

Using Homebrew:

1
2
brew install mosquitto
brew services start mosquitto

You may need to allow Mosquitto through your firewall in System Preferences > Security & Privacy.

Installing on Windows

  1. Download Mosquitto from the official site.
  2. Run the installer and ensure to install the service and CLI tools.
  3. Start the broker via PowerShell:
1
Start-Service mosquitto

Or run it manually:

1
"C:\Program Files\mosquitto\mosquitto.exe" -v

Using a Public Broker (for Development)

1
2
mosquitto_sub -h test.mosquitto.org -t testing/session
mosquitto_pub -h test.mosquitto.org -t testing/session -m "Public test message"

⚠️ Use public brokers only for prototyping or testing. They are not secured and offer no guarantees of message integrity or availability.

Connecting ESP32 Using Arduino Framework

Required Libraries

  • PubSubClient (via Arduino Library Manager)

Minimal Implementation

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
#include <WiFi.h>
#include <PubSubClient.h>

WiFiClient espClient;
PubSubClient client(espClient);

void callback(char* topic, byte* payload, unsigned int length) {
  Serial.printf("Message received on topic %s: ", topic);
  for (int i = 0; i < length; i++) Serial.print((char)payload[i]);
  Serial.println();
}

void connectMQTT() {
  while (!client.connected()) {
    client.connect("esp32_unit1");
    client.subscribe("iot/devices/esp32_unit1/commands");
  }
}

void setup() {
  WiFi.begin("SSID", "PASSWORD");
  while (WiFi.status() != WL_CONNECTED) delay(500);
  client.setServer("192.168.1.100", 1883);
  client.setCallback(callback);
  connectMQTT();
}

void loop() {
  if (!client.connected()) connectMQTT();
  client.loop();
}

Publishing a Status Update

1
client.publish("iot/devices/esp32_unit1/status", "Device online");

🛠 Consider adding retry logic, watchdog timers, and exponential backoff for production deployments.

Topic Structure and Naming Conventions

A robust topic naming scheme is essential for scalability and maintainability. Topics should be predictable, structured, and semantically meaningful.

Best Practices

  • Use slash-separated paths: building/room1/temp
  • Avoid dynamic or user-generated topic names
  • Stick to lowercase and avoid special characters

Examples

  • system/monitor/battery
  • alerts/security/fire
  • devices/device42/uptime

Wildcards

  • devices/+/status matches all device status messages
  • alerts/# captures all nested alert messages

Debugging and Tooling

GUI: MQTT Explorer

CLI Tools

1
2
mosquitto_sub -h localhost -t '#'
mosquitto_pub -h localhost -t system/health -m "OK"

Example Use Cases and Testing Scenarios

Device-to-Device Messaging

1
client.publish("devices/esp32a/to_esp32b", "Ping");
1
client.subscribe("devices/esp32a/to_esp32b");

Environmental Sensing

1
client.publish("sensors/lab1/temp", "22.8C");

Emergency Notification

1
client.publish("alerts/fire/lab1", "Evacuate now");

Local Fleet Communication with ESP32

Architecture Diagram (ASCII)

Here’s a simplified overview of how our MQTT-based communication system is structured:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
+---------------------------+
|     PC (Dev Machine)     |
|---------------------------|
| CLI Tools (pub/sub)      |
| MQTT Broker (Mosquitto)  |
+-----------+---------------+
            |
            | Wi-Fi (LAN)
            v
+-----------+-------------------+       +---------------------+
|      ESP32 Device #1         |<----->|   ESP32 Device #2    |
|------------------------------|       |----------------------|
| Subscribed to: apex/device1 |       | Subscribed to: apex/device2 |
| and: apex/broadcast          |       | and: apex/broadcast       |
+------------------------------+       +---------------------+
            |
            | Wi-Fi (LAN)
            v
     +----------------------------+
     |       ESP32 Device #N      |
     | Subscribed to: apex/deviceN|
     | and: apex/broadcast        |
     +----------------------------+

This setup ensures all devices can receive targeted messages as well as group announcements. Each ESP32 connects to the same broker and listens to both its own channel and a common one for broadcasts.

In our APEX project setup, we’re not connecting to Microsoft Teams or any external cloud platform. Instead, we’re using a local MQTT broker on a PC or Raspberry Pi, and a fleet of ESP32s as both message publishers and subscribers. Messages are exchanged over the local network without relying on the internet — fast, private, and under full control.

During development and testing, we interact with the system directly from a PC using CLI tools. This allows us to simulate any ESP32 device and verify communication without flashing firmware repeatedly.

Example: Send a Message from the PC

1
mosquitto_pub -h localhost -t apex/chloe -m "Meeting in APEX room"

Example: Listen to Incoming Messages

1
mosquitto_sub -h localhost -t apex/#

This design enables one device (or a user on a PC) to message any subset of ESP32 devices through named topics like apex/david, apex/broadcast, or alerts/urgent. Each ESP32 subscribes to its own topic and a shared one for global announcements.

Usual Boring Security Considerations

  • Use authentication (username/password) and TLS encryption
  • Isolate environments (development, staging, production)
  • Avoid hardcoded secrets in firmware
  • Use access control lists (ACLs) to limit topic access
  • Monitor broker uptime and message throughput

Conclusion and Future Work

MQTT offers an elegant, low-overhead messaging model ideal for real-time, event-driven embedded systems. With its publish/subscribe architecture, extensibility, and lightweight footprint, it is particularly well suited for ESP32-based networks.

In future articles, we may explore advanced capabilities such as retained messages, last will/testament, bridging across MQTT brokers, MQTT over WebSockets, and integration with cellular IoT backends.

Feedback, contributions, and topic suggestions are always welcome.