RabbitMQ for IoT

For Internet of Things (IoT) projects that are a lot of different ways that the sensors, devices and client interfaces can be connected together.

RabbitMQ offers the unique ability of being able to use some of the common IoT protocols. RabbitMQ also supports a large variety of client applications and different RabbitMQ servers can mirror, cluster or gateway themselves together.

rabbitmq_overview

In this blog I will look at setting up a RabbitMQ server, and then I create an example of an MQTT value from an Arduino module that can be presented as an AMQP item.

Comparing MQTT and AMQP

Before we get started it’s useful to comment about some of the differences between MQTT (Message Queue Telemetry Transport) and AMQP (Advanced Message Queueing Protocol) .

MQTT is a light weight publish-subscribe-based messaging protocol that works well with lower end hardware and limited bandwidth. For Arduino type applications where you only need to pass some sensor data MQTT is an excellent fit.

AMQP has more overhead than MQTT, because it is a more advanced protocol that includes message orientation, queuing, routing, reliability and security. Presently there are no mainstream AMQP Arduino libraries, but numerous programming options for Raspberry Pi, Linux, Windows and OSX systems exist.  An AMQP IoT example would be to send sensor failures and alarms to dedicated maintenance and alarm queues.

Installing RabbitMQ

For RabbitMQ installation instructions see: https://www.rabbitmq.com/download.html . To install and run RabbitMQ on a Ubuntu system enter:

sudo apt-get update
sudo apt-get install rabbitmq-server
sudo service rabbitmq-server start

The next step is to add some plug-ins. For my project I loaded the MQTT and Web Administration plug-ins.

sudo rabbitmq-plugins enable rabbitmq_mqtt
sudo rabbitmq-plugins enable rabbitmq-management

The rabbitmqctl command line tool allows you to configure and review the RabbitMQ server. To add a user admin1,  with password admin1, that has config, write and read access for management and administrator rights, enter:

sudo rabbitmqctl add_user admin1 admin1
sudo rabbitmqctl set_permissions -p / admin1 ".*" ".*" ".*"
sudo rabbitmqctl set_user_tags admin1 management administrator

After you’ve defined an administrative user the RabbitMQ web management plug-in can be accessed by: http://ip_address:15672 .

Rabbit_webadmin

The RabbitMQ Web Management tool offers an overview of the present system load, connections, exchanges and queues. 

The RabbitMQ Web Management tool is excellent for small manual changes, if however you are looking a doing mass changes there is a management command line tool, rabbitmqadmin, that can be used. The cli tool is already on the system it just needs to be installed.

# Get the cli and make it available to use.
wget http://127.0.0.1:15672/cli/rabbitmqadmin
sudo chmod +x rabbitmqadmin

 

MQTT and AMQP Queues

One of the biggest differences is that MQTT queues are designed to show you the last available message, where as AMQP will store multiple messages in a queue.

The rabbitmqadmin tool can be used to create an AMQP queue, as well as publish and get values.

$./rabbitmqadmin declare queue name=myqueue1 durable=true 
queue declared 

$ ./rabbitmqadmin publish routing_key=myqueue1 payload='this is a test to myqueue1' 
Message published 

$ ./rabbitmqadmin get queue=myqueue1 
+-------------+----------+---------------+----------------------------+---------------+------------------+------------+--------- ----+ 
| routing_key | exchange | message_count | payload | payload_bytes | payload_encoding | properties | redelive red | 
+-------------+----------+---------------+----------------------------+---------------+------------------+------------+--------- ----+ 
| myqueue1    |          | 0             | this is a test to myqueue1 | 26 | string | | False | 
+-------------+----------+---------------+----------------------------+---------------+------------------+------------+--------- ----+

The Web plug-in can also be used to declare, publish and get messages. For the example below it’s important to note that there are 3 more messages waiting in the queue.

get_messages

RabbitMQ Messaging

There are a variety of ways that AMQP messages can be published and subscribed to. The simplest way is to create a queue and then messages can be published and subscribed to from that queue.

rabbitmq_layout

AMQP supports a number of different exchange types (direct, fanout, headers and topic) that can used to manage the distribution and filtering of messages. Messages in an exchange use bindings based on a routing key to link them to a queue.

After the MQTT plug-in is installed RabbitMQ can act like a standalone MQTT broker. MQTT data can also be made available through an AMQP subscription by binding the MQTT exchange to a queue.

Connecting MQTT

For an MQTT project any ESP8266 supported Arduino hardware can be used. There are a number of MQTT Arduino libraries that are available. For this project I used the PubSubClient that can be installed using the Arduino Library Manager.

As a test project I used at low cost MQ-2 Combustible Gas Sensor ($3) that measures a combination of LPG, Alcohol, Propane, Hydrogen, CO and even methane. To fully use this sensor some calibration is required. On the MQ-2 sensor the analog signal is connected to Arduino pin A0 and the analogRead(thePin) function is used to get the value.

MPq2_Setup.png

Below is some example Arduino code required to read the MQ2 gas sensor and publish it to the RabbitMQ MQTT broker with a topic name of : mq2_mqtt.

/*
 Basic ESP8266 MQTT publish client example
*/
#include <ESP8266WiFi.h>
#include <PubSubClient.h>

// Update these with values suitable for your network.
const char* ssid = "your_ssid";
const char* password = "your_password";
const char* mqtt_server = "192.168.0.121"; 
const char* mqtt_user = "admin1";
const char* mqtt_pass= "admin1";

const int mq2pin = A0;

WiFiClient espClient;
PubSubClient client(espClient);

void setup_wifi() {
  // Connecting to a WiFi network
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void reconnect() {
  // Loop until we're reconnected
  Serial.println("In reconnect...");
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("Arduino_Gas", mqtt_user, mqtt_pass)) {
      Serial.println("connected");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      delay(5000);
    }
  }
}

void setup() {
  Serial.begin(9600);
  setup_wifi();
  client.setServer(mqtt_server, 1883);
}

void loop() {
  char msg[8];
  if (!client.connected()) {
    reconnect();
  }

  sprintf(msg,"%i",analogRead(mq2pin));
  client.publish("mq2_mqtt", msg);
  Serial.print("MQ2 gas value:");
  Serial.println(msg);

  delay(5000);
}

Once the MQTT value is published any MQTT client can subscribe to it.  Below is a Python MQTT subscribe example.

# mqtt_sub.py - Python MQTT subscribe example 
#
import paho.mqtt.client as mqtt
 
def on_connect(client, userdata, flags, rc):
    print("Connected to broker")
 
def on_message(client, userdata, message):
    print ("Message received: "  + message.payload)

client = mqtt.Client()
client.username_pw_set("admin1", password='admin1')
client.connect("192.168.0.121", 1883) 

client.on_connect = on_connect       #attach function to callback
client.on_message = on_message       #attach function to callback

client.subscribe("mq2_mqtt") 
client.loop_forever()                 #start the loop

Read MQTT Messages Using AMQP

MQTT clients can subscribe to MQTT messages directly or it’s possible to configure RabbitMQ to have the MQTT data accessible as AMQP. To configure RabbitMQ to forward MQTT you’ll need to do the following steps:

  1. Create a new AMQP Queue – For an IoT project this would typically be a 1-to-1 mapping of the MQTT topic to an AMQP queue
  2. Create a binding between the MQTT Exchange and the Queue – by default all the MQTT topic go into the amq.topic exchange. For MQTT items in this exchange a binding, typically the MQTT topic name, is used as the routing key to the AMQP queue

The binding definitions can be done manually, in the RabbitMQ config file, using rabbitmqadmin or via a program.

For this example my MQTT topic is called mq2_mqtt  and we will create an AMQP queue called mq2_amqp. To manually create a queue, go to the Queues tab in the Web Management tool and define a new queue name.

create_amqp_q

The default MQTT setup puts all the MQTT topics into the amq.topic exchange. To create our binding we need to go to the Exchanges tab and select our MQTT item. Next we select the queue to bind to (mq2_amqp) and set the routing key to the MQTT topic, (mq2_mqtt).

bind_mqtt

The same operation with the command line tool would be:

./rabbitmqadmin declare queue name=mq2_amqp durable=true
./rabbitmqadmin declare binding source=amq.topic destination_type=queue destination=mq2_amqp routing_key=mq2_mqtt

The CLI tool can also be used to see if there are any values in the queue:

cli_get_queue

Now we can use any of the AMQP client libraries to read the Arduino MQ-2 gas value. Below is an example where I used Node-Red to create a web dashboard that showed the native MQTT gas value, the AMQP routed gas value, and a Raspberry Pi Python generated AMQP value.

nr_rabbitmq

Final Comments

RabbitMQ has a lot of features and functions that may or may not be applicable for an IoT project, but the key thing is that it is pretty easy to merge MQTT and AMQP messaging into a common platform.

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google+ photo

You are commenting using your Google+ account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s