Mosquitto MQTT Server

There are a number of excellent open source MQTT (Message Queue Telemetry Transport) brokers or servers that are available. Eclipse Mosquitto is a very popular light weight server that can be run on lower end hardware such as Raspberry Pi modules.

In this blog I’ll look at setting up a Mosquitto broker and I’ll setup some connections to it using testing tools and then use JavaScript with Web Sockets to look at Mosquitto diagnostics.

Getting Started

An MQTT project will have the following components:

  • MQTT Broker. The Mosquitto Server that manages and distributes messages
  • MQTT Publisher. A source for the data, such as a Arduino or Raspberry Pi
  • MQTT Subscriber. An application that wants the data

There is a good selection of MQTT libraries and test utilities for MQTT publishers and subscribers.

To install the Mosquitto broker and test utilities on a Raspberry Pi or Debian system enter:

sudo apt-get update
sudo apt-get install mosquitto mosquitto-clients

For Windows and other systems see the Mosquitto download documentation.

The features of the mosquitto broker are defined by the mosquitto configuration file (/etc/mosquitto/mosquitto.conf). For simple testing you can leave the configuration as is, however if you need to any added features like bridging of MQTT brokers, adding diagnostics or setting up MQTT for web access then you’ll need to edit mosquitto.conf.

For my project I wanted to add diagnostic (with a 10 second update), and I wanted MQTT to also run on Websockets. To do this I edited the config file by:

sudo nano /etc/mosquitto/mosquitto.conf

then I added the lines:

listener 8080
protocol websockets
sys_interval 10

The websockets can be set to any port number (9001, 8000,8080 etc.). After the file has been modified you’ll need to restart the Mosquitto service:

sudo service mosquitto restart

Test Clients

To test that your Mosquitto MQTT broker is running you can open 2 terminal sessions and run the mosquitto_sub and mosquitto_pub utilities.

In the first session we start mosquitto_sub with it subscribing to a topic called mytopic1:

~$ mosquitto_sub -t mytopic1
value1
1234

In the second session we use mosquitto_pub and we can publish some values to the topic mytopic1:

~$ mosquitto_pub -t mytopic1 -m "value1"
~$ mosquitto_pub -t mytopic1 -m 1234

If everything is working correctly then the published values should appear in the subscribing session.

JavaScript, Websockets and  Mosquitto Diagnostics

We can create a simple web page to test out the Javascript, Websockets and Mosquitto Diagnostics.

The Mosquitto Diagnostics have a topic of : $SYS/broker/… Some of these diagnostics include:

$SYS/broker/bytes/received
$SYS/broker/bytes/sent
$SYS/broker/clients/connected
$SYS/broker/clients/maximum
$SYS/broker/clients/total
$SYS/broker/heap/current size
$SYS/broker/heap/maximum size
$SYS/broker/load/connections/+
$SYS/broker/load/bytes/received/+
$SYS/broker/load/bytes/sent/+
$SYS/broker/load/messages/received/+
$SYS/broker/load/messages/sent/+
$SYS/broker/load/publish/dropped/+
$SYS/broker/load/publish/received/+
$SYS/broker/load/publish/sent/+
$SYS/broker/load/sockets/+
$SYS/broker/messages/received
$SYS/broker/messages/sent
$SYS/broker/publish/messages/received
$SYS/broker/publish/messages/sent
$SYS/broker/retained messages/count
$SYS/broker/store/messages/count
$SYS/broker/store/messages/bytes
$SYS/broker/subscriptions/count
$SYS/broker/version

where  + = 1min, 5min or 15min averages

For a full description of the Mosquitto diagnostics see: https://mosquitto.org/man/mosquitto-8.html .

Next we can create a web page where we can subscribe to some of these diagnostic topics. An MQTT javascript library can be remotely called from:

https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.1/mqttws31.js

This library is very similiar to Python Paho MQTT library, and it supports both subscribe and publish functions.

<html>
<head>
<title> MQTT Subscribe Message</title>
</head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.1/mqttws31.js" type="text/javascript"></script>


<script>
// Define some MQTT variables


function sub_mqtt_msg() {
// Send an MQTT message
  var mqtt_server = document.getElementById("mqtt_server").value;
  var mqtt_port = Number(document.getElementById("mqtt_port").value);


  client = new Paho.MQTT.Client(mqtt_server, mqtt_port,"");
  client.onMessageArrived = onMessageArrived;
  client.onMessageArrived = onMessageArrived;
  client.connect({onSuccess:onConnect});
  document.getElementById("submsg").innerHTML = "Trying to connect...<br>";

}
function onConnect() {
  document.getElementById("submsg").innerHTML = "New connection made...<br>";
  var mqtt_destname = document.getElementById("mqtt_destname").value;  
  client.subscribe(mqtt_destname);
  document.getElementById("submsg").innerHTML = "Subscribing to topic: " + mqtt_destname + " ...<br>";
}
function onMessageArrived(message) {
  var result = message.destinationName + " : " + message.payloadString + "<br>";
  document.getElementById("submsg").innerHTML += result;
}

</script>
<body>
<h1>MQTT Subscribe Test Page</h1>

Server Name: <input type="text" id="mqtt_server" value="192.168.0.121"><br><br>
Websocket: <input type="text" id="mqtt_port" value="8080"><br><br>
DestinationName: <input type="text" id="mqtt_destname" value="$SYS/broker/#"><br><br>

<button onclick="sub_mqtt_msg()">Subscript to MQTT</button>
<hr>
<h2>Subscribed Messages:</h2>
<div id=submsg></div>
</body>

</html>

By subscribing to $SYS/broker/# we can get see all the different diagnostic values.

java_ws_diag2

Summary

Mosquitto is an excellent MQTT broker that can be run on both high end servers and low end hardware like Raspberry Pi’s.

It is important to note that with most MQTT libraries, (i.e. Python Paho, JavaScript and Node Red) the default setting is to not retain only values. This can be a problem if a new MQTT subscribers starts because it will not have any data until new values are published. To ensure that old values are available set the RETAIN=YES option on the MQTT publish call.

In this blog I wanted to examine, the diagnostic features of the Mosquitto MQTT broker, however there are many more features and functions of MQTT communications that need to be examined. There are some excellent MQTT guides on the Internet. I tried to document some of my notes on MQTT with Javascript and Charts, and MQTT with Arduino and Node Red.

 

 

 

 

 

 

RabbitMQ for IoT

For Internet of Things (IoT) projects that are a lot of different ways that the sensors, devices and client interfaces can be connected together. For many projects using simple MQTT (Message Queue Telemetry Transport) is all that you need. However if you’re trying to merge and build IoT projects that use both MQTT and AMQP (Advanced Message Queue Protocol) or require a REST API then you should take a look at RabbitMQ.

RabbitMQ is an open source middleware solution that natively uses AMQP communications but it has a good selection of plug-ins to support features like: MQTT, MQTT Web Sockets, HTTP REST API and server-to-server communications.

rabbitmq_overview

In this blog we will setting up a RabbitMQ server, and we will look at some of the differences between MQTT and AMQP messaging. Finally an example of an Anduino MQTT message will be presented as both an MQTT and an AMQP item in a Node-Red dashboard.

Getting Started

RabbitMQ can be installed on Window, Linux, MacOS systems and there are also some cloud based offerings. For small systems lower end hardware like a Raspberry Pi can be used.  For complete RabbitMQ installation instructions see: https://www.rabbitmq.com/download.html . To install and run RabbitMQ on a Ubuntu system enter:

sudo apt-get update
sudo apt-get install rabbitmq-server
sudo service rabbitmq-server start

The next step is to add some plug-ins. For my project I loaded the MQTT and Web Administration plug-ins:

sudo rabbitmq-plugins enable rabbitmq_mqtt
sudo rabbitmq-plugins enable rabbitmq-management

The rabbitmqctl command line tool allows you to configure and review the RabbitMQ server. To add a user admin1,  with password admin1, that has config, write and read rights for management and administrator access, enter:

sudo rabbitmqctl add_user admin1 admin1
sudo rabbitmqctl set_permissions -p / admin1 ".*" ".*" ".*"
sudo rabbitmqctl set_user_tags admin1 management administrator

After you’ve defined an administrative user the RabbitMQ web management plug-in can be accessed by: http://ip_address:15672 .

Rabbit_webadmin

The RabbitMQ Web Management tool offers an overview of the present system load, connections, exchanges and queues. 

The RabbitMQ Web Management tool is excellent for small manual changes, if however you are looking a doing a large number of additions or changes then, rabbitmqadmin, the command line management tool can be used. This tool is  installed by:

# Get the cli and make it available to use.
wget http://127.0.0.1:15672/cli/rabbitmqadmin
sudo chmod +x rabbitmqadmin

Comparing MQTT and AMQP

It’s useful to comment about some of the differences between MQTT (Message Queue Telemetry Transport) and AMQP (Advanced Message Queueing Protocol) .

MQTT is a light weight publish-subscribe-based messaging protocol that works well with lower end hardware and limited bandwidth. For Arduino type applications where you only need to pass some sensor data MQTT is an excellent fit.

AMQP has more overhead than MQTT, because it is a more advanced protocol that includes message orientation, queuing, routing, reliability and security. Presently there are no mainstream AMQP Arduino libraries, but numerous programming options for Raspberry Pi, Linux, Windows and MacOS systems exist.  An AMQP IoT example would be to send sensor failures and alarms to dedicated maintenance and alarm queues.

MQTT and AMQP Queues

One of the biggest differences in queues is that MQTT queues are designed to show you the last available message, where as AMQP will store multiple messages in a queue.

A published MQTT message contains a message body, a retain flag and a quality of service (QoS) value.

An AMQP message can be published with added properties such as: time stamp, type of message and expiration information. AMQP messages also support the addition of custom header values. Below is a Python publish example that defines the message type to be “Pi Sensor”, and custom headers are include for status and alarm state.

#!/usr/bin/env python
import pika

node = "192.168.0.121"
user = "pete"
pwd = "pete"

# Connect to a remote AMQP server with a username/password
credentials = pika.PlainCredentials(user, pwd)
connection = pika.BlockingConnection(pika.ConnectionParameters(node,
        5672, '/', credentials))                                    
channel = connection.channel()

# Create a queue if it doesn't already exist
channel.queue_declare(queue='Rasp_1',durable=True)

# Define the properties and publish a message
props = pika.BasicProperties(
    headers= {'status': 'Good Quality',"alarm":"HI"},
    type ="Pi Sensor")
channel.basic_publish(exchange='',
    routing_key='Rasp_1',body='99.5', properties = props)

connection.close()

For this example the Rasp_1 queue can be examined using the Queue->Get Message option in the Web Management Interface. 

q_prop

RabbitMQ Messaging

There are a variety of ways that AMQP messages can be published and subscribed to. The simplest way is to create a queue and then messages can be published and subscribed to from that queue.

rabbitmq_layoutTo help with the distribution and filtering of messages AMQP supports a number of different exchange types. Messages in an exchange use bindings based on a routing key to link them to a queue. 

The main types of exchanges are: direct, fanout, headers and topic. An IoT example of a direct exchange would be if a group of Raspberry Pi sensor values are going into a “Rasp Pi Sensor” exchange. When the Rasp Pi publishes a sensor result to the exchange the message also includes a routing key to link the message to the correct queue.

direct

An IoT example of a fanout exchange would be with critical sensor failures. The sensor failure message is sent to Bill’s and Sam’s work or task queue and the All Maintenance point queue at the same time.

fanout

Connecting MQTT

After the MQTT plug-in is installed RabbitMQ can act like a standalone MQTT broker. MQTT data can also be made available through an AMQP subscription by binding the MQTT exchange to a RabbitMQ queue.

For an MQTT project any ESP8266 supported Arduino hardware can be used. There are a number of MQTT Arduino libraries that are available. For this project I used the PubSubClient that can be installed using the Arduino Library Manager.

As a test project I used at low cost MQ-2 Combustible Gas Sensor ($3) that measures a combination of LPG, Alcohol, Propane, Hydrogen, CO and even methane. Note to fully use this sensor some calibration is required. On the MQ-2 sensor the analog signal is connected to Arduino pin A0 and the analogRead(thePin) function is used to read the sensor value.

MPq2_Setup.png

Below is some example Arduino code required to read the MQ2 gas sensor and publish it to the RabbitMQ MQTT broker with a topic name of : mq2_mqtt.

/*
 Basic ESP8266 MQTT publish client example
*/
#include <ESP8266WiFi.h>
#include <PubSubClient.h>

// Update these with values suitable for your network.
const char* ssid = "your_ssid";
const char* password = "your_password";
const char* mqtt_server = "192.168.0.121"; 
const char* mqtt_user = "admin1";
const char* mqtt_pass= "admin1";

const int mq2pin = A0; //the MQ2 analog input pin

WiFiClient espClient;
PubSubClient client(espClient);

void setup_wifi() {
  // Connecting to a WiFi network
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void reconnect() {
  // Loop until we're reconnected
  Serial.println("In reconnect...");
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("Arduino_Gas", mqtt_user, mqtt_pass)) {
      Serial.println("connected");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      delay(5000);
    }
  }
}

void setup() {
  Serial.begin(9600);
  setup_wifi();
  client.setServer(mqtt_server, 1883);
}

void loop() {
  char msg[8];
  if (!client.connected()) {
    reconnect();
  }

  sprintf(msg,"%i",analogRead(mq2pin));
  client.publish("mq2_mqtt", msg);
  Serial.print("MQ2 gas value:");
  Serial.println(msg);

  delay(5000);
}

Once the MQTT value is published any MQTT client can subscribe to it.  Below is a Python MQTT subscribe example.

# mqtt_sub.py - Python MQTT subscribe example 
#
import paho.mqtt.client as mqtt
 
def on_connect(client, userdata, flags, rc):
    print("Connected to broker")
 
def on_message(client, userdata, message):
    print ("Message received: "  + message.payload)

client = mqtt.Client()
client.username_pw_set("admin1", password='admin1')
client.connect("192.168.0.121", 1883) 

client.on_connect = on_connect       #attach function to callback
client.on_message = on_message       #attach function to callback

client.subscribe("mq2_mqtt") 
client.loop_forever()                 #start the loop

Read MQTT Messages Using AMQP

MQTT clients can subscribe to MQTT messages directly or it’s possible to configure RabbitMQ to have the MQTT data accessible as AMQP. The routing of MQTT messages to AMQP queues is done using the direct exchange method.

mqtt_2_amqp

To configure RabbitMQ to forward MQTT the following steps are done:

  1. Create a new RabbitMQ Queue – For an IoT project this would typically be a 1-to-1 mapping of the MQTT topic to a queue name.
  2. Create a binding between the MQTT Exchange and the Queue – by default all the MQTT topic go into the amq.topic exchange. For MQTT items in this exchange a binding, typically the MQTT topic name, is used as the routing key to the AMQP queue.

These steps can be done in a number of ways : manually, in the RabbitMQ config file, using the rabbitmqadmin command line tool or via a program. Because I was doing this for multiple signals I used rabbitmqadmin tool, and the syntax is:

./rabbitmqadmin declare queue name=mq2_amqp durable=true
./rabbitmqadmin declare binding source=amq.topic destination_type=queue destination=mq2_amqp routing_key=mq2_mqtt

The RabbitMQ Web Admin can be used to verify the exchange to queue binding.

wp_mqtt_2_amqp

 

The CLI tool can also be used to see if there are any values in the queue:

cli_get_queue

Node-Red Dashboard

Node-Red is a visual programming environment that allows users to create applications by dragging and dropping nodes on the screen. Logic flows are then created by connecting the different nodes together.

Node-Red has been preinstalled on Raspbian Jesse since the November 2015. Node-Red can also be installed on Windows, Linux and OSX.  To install and run Node-Red on your specific system see https://nodered.org/docs/getting-started/installation.

To install the AMQP components, select the Manage palette option from the right side of the menu bar. Then search for “AMQP” and install node-red-contrib-amqp. If your installation of Node-Red does not have dashboards installed, search for: node-red-dashboard.

nr_install_amqp

For our Node-Red MQTT and AMQP example we will use a mqtt and a amqp node from the input palette group, and two gauge nodes from the dashboard group.  The complete Node-Red logic for this is done in only 4 nodes!

nr_logic

Nodes are added by dragging and dropped them into the center Flow sheet. Logic is created by making connection wires between inputs and output of a node. After the logic is laid out, double click on each of the nodes to configure their specific properties. You will need to specify the MQTT and AMQP definitions of your RabbitMQ IP address,user rights, MQTT topic and AMQP queue name. You will also need to double click on the gauge nodes to configure the look-and-feel of the web dashboard.

After the logic is complete, hit the Deploy button on the right side of the menu bar to run the logic.  The Node-Red dashboard user interface is accessed by: http://ipaddress:1880/ui. 

For my project I used a number of different MQ sensors and inputs. Below is a picture of Node-Red web dashboard that we created with the same MQTT value being shown natively and as a AMQP queued value.

project

Final Comments

I found that RabbitMQ was easy to install and the Web Administration plug-in along with the rabbitmqadmin command line tool made it very easy to maintain. 

If you’re just looking to show sensor values then a basic MQTT broker might be all you need. However if you’re looking at some future applications like alarm, maintenance or task lists then AMQP exchanges and queue make RabbitMQ an interesting option.

MQTT and Javascript

MQTT (Message Queuing Telemetry Transport) is a  publish-subscribe-based messaging protocol that is used on many Internet of Things (IoT) projects. It works on top of the TCP/IP protocol and it is designed for connections with remote locations where a “small code footprint” is required or the network bandwidth is limited. The publish-subscribe messaging pattern requires a message broker.

MQTT_js_overview

There is support for MQTT on a large variety of programming languages and platforms. An Arduino or Raspberry Pi module can sent (or publish) I/O to a MQTT broker, and they can also receive (or subscribe) to data.

There are a number of MQTT brokers that can be used. One of the most popular ones is the Mosquitto MQTT broker, and it can be loaded on Windows, OSX and Linux systems. For this blog we will be using the Mosquitto test MQTT server. This Internet based server should not be used for real systems, but it is excellent for small short terms tests.

MQTT Web Sockets

The MQTT server has configurable MQTT and Web Socket ports. For a typical Raspberry Pi or Arduino connection, the default MQTT port 1883 would be used. In many Internet applications only certain ports are open, so for this reason a different MQTT Web Socket is used. This is configurable but ports like 80 or 8080 are typically used.MQTT_web_layout

Javascript Application

There are a number of MQTT javascript libraries that are available. One of the more popular ones is the Paho library that is available at:

https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.1/mqttws31.js

As a first example we will look at creating 2 pages. The first page will  publish a value, and the second page will subscribe to the data.

MQTT_js_pubsub

The publish code is:

<html>
<head>
<title> MQTT Publish Message</title>
</head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.1/mqttws31.js" type="text/javascript"></script>

<script>
// Define some MQTT variables
var mqtt_server = "";
var mqtt_port = "";
var mqtt_destname = "";

function send_mqtt_msg() {
// Send an MQTT message
  mqtt_server = document.getElementById("mqtt_server").value;
  mqtt_port = Number(document.getElementById("mqtt_port").value);
 

  client = new Paho.MQTT.Client(mqtt_server, mqtt_port,"");
  client.connect({onSuccess:onConnect});
  document.getElementById("pubmsg").innerHTML = "Trying to connect...
";
}
function onConnect() {
  document.getElementById("pubmsg").innerHTML = "New connection made...
";
  var mqtt_destname = document.getElementById("mqtt_destname").value;
  var mqtt_msg = document.getElementById("mqtt_msg").value;   
  message = new Paho.MQTT.Message(mqtt_msg);
  message.destinationName = mqtt_destname;
  client.send(message);
  document.getElementById("pubmsg").innerHTML = "topic:" + mqtt_destname + " = " + mqtt_msg + " ...sent";
}  
// called when a message arrives
</script>
<body>
<h1>MQTT Publish Test Page</h1>

Server Name: <input type="text" id="mqtt_server" value="test.mosquitto.org"><br><br>
Websocket: <input type="text" id="mqtt_port" value="8080"><br><br>
DestinationName: <input type="text" id="mqtt_destname" value="my_IoT_value1"><br><br>
Message: <input type="text" id="mqtt_msg" value="test message"><br><br>

<button onclick="send_mqtt_msg()">Publish MQTT Message</button>
</body>
<hr>
<div id=pubmsg></div>
</html>

The subscribe code is:

<html>
<head>
<title> MQTT Subscribe Message</title>
</head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.1/mqttws31.js" type="text/javascript"></script>
<script>

function sub_mqtt_msg() {
// Send an MQTT message
  var mqtt_server = document.getElementById("mqtt_server").value;
  var mqtt_port = Number(document.getElementById("mqtt_port").value);

  client = new Paho.MQTT.Client(mqtt_server, mqtt_port,"");
  client.onMessageArrived = onMessageArrived;
  client.onMessageArrived = onMessageArrived;
  client.connect({onSuccess:onConnect});
  document.getElementById("submsg").innerHTML = "Trying to connect...
";

}
function onConnect() {
  document.getElementById("submsg").innerHTML = "New connection made...
";
  var mqtt_destname = document.getElementById("mqtt_destname").value;  
  client.subscribe(mqtt_destname);
  document.getElementById("submsg").innerHTML = "Subscribing to topic: " + mqtt_destname + " ...
";
}
function onMessageArrived(message) {
  var result = message.destinationName + " : " + message.payloadString + "
";
  document.getElementById("submsg").innerHTML = result;
}

</script>
<body>
<h1>MQTT Subscribe Test Page</h1>

Server Name: <input type="text" id="mqtt_server" value="test.mosquitto.org"><br><br>
Websocket: <input type="text" id="mqtt_port" value="8080"><br><br>
DestinationName: <input type="text" id="mqtt_destname" value="my_IoT_value1"><br><br>

<button onclick="sub_mqtt_msg()">Subscript to MQTT</button>
<hr>
<h2>Subscribed Messages:</h2>
<div id=submsg></div>
</body>

</html>

Once you’ve got the basics down it’s possible to make some more advanced web interface pages.

Chart and Gauges

For your IoT projects there are a lot of Dash board options that are available. One of my favorites is Node-Red because it is totally free and standalone.

If you would like to create your own web interface there are a number of good javascript charting and gauge libraries available. For my examples I used Google charts with the Gauge chart . To simulate the inputs I used three of our MQTT Publish pages.

MQTT_js_sim

The code for the gauges page is :

<html>
<head>
<title>IoT - MQTT to JS</title>
</head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.1/mqttws31.js" type="text/javascript"></script>
<script src="https://www.gstatic.com/charts/loader.js" type="text/javascript"></script>

<script>
// MQTT variables
var MQTTnames = ["my_IoT_value1","my_IoT_value2","my_IoT_value3"];
var MQTTvalues = [0,0,0];


// Define the Google gauge chart
      google.charts.load('current', {'packages':['gauge']});
      google.charts.setOnLoadCallback(drawChart);

      function drawChart() {

        var data = google.visualization.arrayToDataTable([
          ['Label', 'Value'], 
      [MQTTnames[0], MQTTvalues[0]],
      [MQTTnames[1], MQTTvalues[1]],
      [MQTTnames[2], MQTTvalues[2]],      
        ]);

        var options = {
          width: 800, height: 1000,
          redFrom: 90, redTo: 100,
          yellowFrom:75, yellowTo: 90,
          minorTicks: 5
        };

        var chart = new google.visualization.Gauge(document.getElementById('chart_div'));

        chart.draw(data, options);

        setInterval(function() {
      for (var i=0; i < MQTTnames.length; i++) {
      data.setValue(i, 1, MQTTvalues[i]);
      }
          chart.draw(data, options);

      }, 1000);
}

// Create a client instance
client = new Paho.MQTT.Client("test.mosquitto.org", 8080,"");
client.onMessageArrived = onMessageArrived;

// connect the client
client.connect({onSuccess:onConnect});

// called when the client connects
function onMessageArrived(message) {
  
  for (var i=0; i < MQTTnames.length; i++) {
  if (message.destinationName == MQTTnames[i]) {
    MQTTvalues[i] = Number(message.payloadString);
  }
  }
}
function onConnect() {
  // Once a connection has been made, make a subscription and send a message.
  for (var i=0; i < MQTTnames.length; i++) {
  client.subscribe(MQTTnames[i]);
  }
}
// called when a message arrives
</script>
<body>
<h1  style = 'font-size: xx-large'>IoT - MQTT to JavaScript</h1>
<div id="chart_div" style="width: 100%; height: 100%;"></div>
</body>
</html>

There are many other charting options that could be used. Below is an example using the Google Chart library with bars instead of gauges.

MQTT_js_bars.png

Final Comment

Using Javascript to interact with your IoT projects open up a lot of potential for adding functionality. I found that the Python version of the Paho MQTT library to have better documentation and perhaps some more functions, but at the end of the day I was able to do all the I wanted.

Arduino talking MQTT to Node-Red

There are some great Arduino modules with integrated ESP-8266 wireless chips, some of the more popular modules are:

  • Adafruit HUZZAH
  • NodeMCU
  • WeMos

These modules allow you to do some interesting IoT (Internet of Things) projects. To connect the Arduino modules to PCs, Raspberry Pi’s or Linux nodes that are a number of communication choices. MQTT (Message Queue Telemetry Transport) is becoming one of the standards for this and it is pre-installed with Node-Red.

Plant Moisture Monitoring MQTT Example

Ard_mqtt_overview2

For our example we wanted to do a simple plant moisture example that used a solar charger and an Arduino Wemos module. We then setup an MQTT server on our Node-Red Raspberry Pi with a web dashboard.

Our goal was to get the MQTT technologies working, with some moisture inputs (and not a final plant monitoring system).

Moisture Sensors

Moisture sensors are very low cost and they start at about $2. The basic moisture sensor has 3 inputs; VCC, GND, and AO. Some sensors also include a digital output with a potentiometer to adjust the digital 0-1 moisture limit.

Our Arduino plant moisture setup is good for testing but not a good long term solution. When voltage is applied long term to moisture sensors ionization in the soil will cause a combination of false reading and deterioration of the sensor plates. We plan to do a future project where we will use relays to turn the sensors on/off and we will include solenoid values in a watering system.

MQTT on Arduino

There are a number of excellent MQTT libraries for Arduino, for this example we used the PubSubClient library. This library can be installed from the Arduino IDE by selecting the menu items:

Sketch -> Add Library -> Manage Libraries

To get MQTT setup you’ll need to:

  • define the SSID and password for your WAN
  • define the IP address for the MQTT server (the Node Red/Raspberry Pi node)
  • define some topic for the data

The nice thing about MQTT is that you can define topics for each of your data points. For this example we define the topic humidity to show the moisture sensor value, and msgtext to show the message (‘Needs Water’ or ‘).

Below is our sample Arduino code for passing the moisture data to our MQTT server.

/*
 Basic ESP8266 MQTT publish client example for a moisture sensor
*/
#include <ESP8266WiFi.h>
#include <PubSubClient.h>

// Update these with values suitable for your network.
const char* ssid = "YOUR_SSID_NAME";
const char* password = "YOUR_PASSWORD";
const char* mqtt_server = "YOUR_NODE_RED_IP";

WiFiClient espClient;
PubSubClient client(espClient);

void setup_wifi() {
  // Connecting to a WiFi network
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void reconnect() {
  // Loop until we're reconnected
  Serial.println("In reconnect...");
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("Arduino_Moisture")) {
      Serial.println("connected");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      delay(5000);
    }
  }
}

void setup() {
  Serial.begin(9600);
  setup_wifi();
  client.setServer(mqtt_server, 1883);
}

void loop() {
  char msg[10];
  char msgtext[25];
  String themsg;
  if (!client.connected()) {
    reconnect();
  }
  
  int soil_moisture=analogRead(A0);  // read from analog pin A0
  Serial.print("analog value: ");
  Serial.println(soil_moisture);
  
  if((soil_moisture>300)&&(soil_moisture<700)) {
    Serial.println("Humid soil");
    sprintf(msgtext,"Humid soil",soil_moisture);
  } 
  else if ((soil_moisture>700)&&(soil_moisture<950)){
    Serial.println("Moist Soil");
    sprintf(msgtext,"Moist Soil",soil_moisture);
  }
  else if (soil_moisture <300) ){
    Serial.println("Needs water");    
    sprintf(msgtext,"Needs water",soil_moisture);
  }
  else
  {
      sprintf(msgtext,"Sensor Problem",soil_moisture);
  }

  sprintf(msg,"%i",soil_moisture);
  client.publish("humidity", msg);
  client.publish("soil", msgtext);
  delay(5000);
}

Node-Red

Node-Red is an excellent visual programming environment that is part of the Raspberry Pi base install. Node-Red is a simple tool to create your own Internet of Things applications. The base Node-Red installation includes MQTT interfacing components but it does not include an MQTT server.

If you don’t have a Raspberry Pi you can install Node-Red on Window, Mac OS or Linux systems. I’ve had good results running Node-Red on a very old low end laptop running Xubuntu, (see steps 1-3 in this linked guide).

MQTT Server on Node-Red

There are a number of free internet MQTT servers (iot.eclipse.org) that can be used or an MQTT server can be loaded directly on a local server (i.e. Mosquito).

Mosca is a standalone MQTT server that can be installed directly into Node-Red. The Mosca Node-Red component can be either installed at the command line by:

cd $HOME/.node-red

npm install node-red-contrib-mqtt-broker

Or the component can be install within the Node-Red web interface by selecting the “manage palette” option, and then search for mosca.

mosca_install

After Mosca is installed, all that is required is that a “mosca” node needs to be dragged and dropped into the Node-Red project.

mosca_mqtt_nodes

To connect the Arduino module to Node-Red mqtt inputs are added to the project.  The Arduino topics are defined in Node-Red by double-clicking on the mqtt node and then define the topic to match the Arduino topic.

mqtt_topic

After the MQTT connections are configured Web dashboards can present the final data. The Web dashboards offer a number of different components that could be used for this example I used a gauge and a text node.

To compile and view the Node-Red application, click on the Deploy button on the right side of the menu bar. To access the web dashboard enter: http://your-ip:1880/ui . Below is a example of what you should see.

NodeRed_MQTT_GUI

Final Thoughts

The Mosca MQTT server component allows for a simple standalone architecture to connect wireless Arduino modules into a Node-Red IoT solution.

We can also load Node-Red on Raspberry Pi data collection nodes, and have them publish data to a central Node-Red server.

 

 

 

House Music

Our goal was to have the same music playing throughout the house, and then use a phone to control the music station and volume.

screenshot

For this project we used a technology called MQTT (Message Queue Telemetric Transport). MQTT is based on the following standard concepts:
– MQTT Broker manages and distributes messages
– MQTT Publisher a source for the data
– MQTT Subscriber an application that wants the data

layout2

Pi Setup

On the Raspberry Pi’s we needed to load a Python MQTT library and a music player (mpd – music server, mpc – music client):

sudo pip install paho-mqtt
sudo apt-get install mpd mpc

The key commands to manage playing of Internet radio stations are:
mpc add radio station url– this will add a radio station to a play list
mpc play play list number – play a selected play list
mpc volume number – change the volume from 0-100%
– mpc clear – clear the play lists

Internet radio stations can be found by going to https://www.internet-radio.com. To find a radio station URL right click on any of the radio station links and select “Copy link address”

On each of the Pi’s the Python program subscribes to an MQTT topic called Radio. New Internet Radio Stations are passed as strings, and new volume settings are passed as numbers between 0 and 100. For this application the MQTT payload could be either an integer or a string so a decode function (msg.payload.decode(“utf-8”)) is used to convert from a byte array to a generic string.

When a new radio station is received:
– the current play list is cleared (mpc clear),
– the new radio station is added to the play list (mpc add),
– the volume is reset to it’s original (mpc volume), and
– the radio station is played (mpc play).

If the payload is numeric only the volume is changed. The full Pi Python program is shown below:

# mqtt_2_mpc.py - have MQTT change Internet Radio Stations
#
import paho.mqtt.client as mqtt
import os

thevolume = 75  # save the volume

# Subscribe to topic: Radio

def on_connect(client, userdata, flags, rc):
  print("Connected with result code "+str(rc))
  client.subscribe("Radio")

def on_message(client, userdata, msg):
  global thevolume
  print( msg.payload)
  themsg = msg.payload.decode("utf-8")
  # if the message is a number it's the volume
  if themsg.isnumeric():
    thevolume = themsg
    os.system("mpc volume " + thevolume)
  # if the message is a string it's the station URL
  else:
    os.system("mpc clear")
    os.system("mpc add '" + themsg + "'")
    os.system("mpc volume " + thevolume)
    os.system("mpc play")

client = mqtt.Client()
client.connect("iot.eclipse.org",1883,60)

client.on_connect = on_connect
client.on_message = on_message

print ("Waiting for messages .....")
client.loop_forever()

Web Page Setup

On the Web page the Paho JavaScript client is used to connect to an MQTT broker. It is important to note that the JavaScript interface uses Web Sockets to communicate with the MQTT broker, (typically on port 80) rather than the native MQTT port of 1883. If you install your own MQTT broker, you will need check the document for Web Socket setup and support.

layout3

The final Web page that we used is listed below.

websock1