RabbitMQ for IoT

For Internet of Things (IoT) projects that are a lot of different ways that the sensors, devices and client interfaces can be connected together. For many projects using simple MQTT (Message Queue Telemetry Transport) is all that you need. However if you’re trying to merge and build IoT projects that use both MQTT and AMQP (Advanced Message Queue Protocol) or require a REST API then you should take a look at RabbitMQ.

RabbitMQ is an open source middleware solution that natively uses AMQP communications but it has a good selection of plug-ins to support features like: MQTT, MQTT Web Sockets, HTTP REST API and server-to-server communications.

rabbitmq_overview

In this blog we will setting up a RabbitMQ server, and we will look at some of the differences between MQTT and AMQP messaging. Finally an example of an Anduino MQTT message will be presented as both an MQTT and an AMQP item in a Node-Red dashboard.

Getting Started

RabbitMQ can be installed on Window, Linux, MacOS systems and there are also some cloud based offerings. For small systems lower end hardware like a Raspberry Pi can be used.  For complete RabbitMQ installation instructions see: https://www.rabbitmq.com/download.html . To install and run RabbitMQ on a Ubuntu system enter:

sudo apt-get update
sudo apt-get install rabbitmq-server
sudo service rabbitmq-server start

The next step is to add some plug-ins. For my project I loaded the MQTT and Web Administration plug-ins:

sudo rabbitmq-plugins enable rabbitmq_mqtt
sudo rabbitmq-plugins enable rabbitmq-management

The rabbitmqctl command line tool allows you to configure and review the RabbitMQ server. To add a user admin1,  with password admin1, that has config, write and read rights for management and administrator access, enter:

sudo rabbitmqctl add_user admin1 admin1
sudo rabbitmqctl set_permissions -p / admin1 ".*" ".*" ".*"
sudo rabbitmqctl set_user_tags admin1 management administrator

After you’ve defined an administrative user the RabbitMQ web management plug-in can be accessed by: http://ip_address:15672 .

Rabbit_webadmin

The RabbitMQ Web Management tool offers an overview of the present system load, connections, exchanges and queues. 

The RabbitMQ Web Management tool is excellent for small manual changes, if however you are looking a doing a large number of additions or changes then, rabbitmqadmin, the command line management tool can be used. This tool is  installed by:

# Get the cli and make it available to use.
wget http://127.0.0.1:15672/cli/rabbitmqadmin
sudo chmod +x rabbitmqadmin

Comparing MQTT and AMQP

It’s useful to comment about some of the differences between MQTT (Message Queue Telemetry Transport) and AMQP (Advanced Message Queueing Protocol) .

MQTT is a light weight publish-subscribe-based messaging protocol that works well with lower end hardware and limited bandwidth. For Arduino type applications where you only need to pass some sensor data MQTT is an excellent fit.

AMQP has more overhead than MQTT, because it is a more advanced protocol that includes message orientation, queuing, routing, reliability and security. Presently there are no mainstream AMQP Arduino libraries, but numerous programming options for Raspberry Pi, Linux, Windows and MacOS systems exist.  An AMQP IoT example would be to send sensor failures and alarms to dedicated maintenance and alarm queues.

MQTT and AMQP Queues

One of the biggest differences in queues is that MQTT queues are designed to show you the last available message, where as AMQP will store multiple messages in a queue.

A published MQTT message contains a message body, a retain flag and a quality of service (QoS) value.

An AMQP message can be published with added properties such as: time stamp, type of message and expiration information. AMQP messages also support the addition of custom header values. Below is a Python publish example that defines the message type to be “Pi Sensor”, and custom headers are include for status and alarm state.

#!/usr/bin/env python
import pika

node = "192.168.0.121"
user = "pete"
pwd = "pete"

# Connect to a remote AMQP server with a username/password
credentials = pika.PlainCredentials(user, pwd)
connection = pika.BlockingConnection(pika.ConnectionParameters(node,
        5672, '/', credentials))                                    
channel = connection.channel()

# Create a queue if it doesn't already exist
channel.queue_declare(queue='Rasp_1',durable=True)

# Define the properties and publish a message
props = pika.BasicProperties(
    headers= {'status': 'Good Quality',"alarm":"HI"},
    type ="Pi Sensor")
channel.basic_publish(exchange='',
    routing_key='Rasp_1',body='99.5', properties = props)

connection.close()

For this example the Rasp_1 queue can be examined using the Queue->Get Message option in the Web Management Interface. 

q_prop

RabbitMQ Messaging

There are a variety of ways that AMQP messages can be published and subscribed to. The simplest way is to create a queue and then messages can be published and subscribed to from that queue.

rabbitmq_layoutTo help with the distribution and filtering of messages AMQP supports a number of different exchange types. Messages in an exchange use bindings based on a routing key to link them to a queue. 

The main types of exchanges are: direct, fanout, headers and topic. An IoT example of a direct exchange would be if a group of Raspberry Pi sensor values are going into a “Rasp Pi Sensor” exchange. When the Rasp Pi publishes a sensor result to the exchange the message also includes a routing key to link the message to the correct queue.

direct

An IoT example of a fanout exchange would be with critical sensor failures. The sensor failure message is sent to Bill’s and Sam’s work or task queue and the All Maintenance point queue at the same time.

fanout

Connecting MQTT

After the MQTT plug-in is installed RabbitMQ can act like a standalone MQTT broker. MQTT data can also be made available through an AMQP subscription by binding the MQTT exchange to a RabbitMQ queue.

For an MQTT project any ESP8266 supported Arduino hardware can be used. There are a number of MQTT Arduino libraries that are available. For this project I used the PubSubClient that can be installed using the Arduino Library Manager.

As a test project I used at low cost MQ-2 Combustible Gas Sensor ($3) that measures a combination of LPG, Alcohol, Propane, Hydrogen, CO and even methane. Note to fully use this sensor some calibration is required. On the MQ-2 sensor the analog signal is connected to Arduino pin A0 and the analogRead(thePin) function is used to read the sensor value.

MPq2_Setup.png

Below is some example Arduino code required to read the MQ2 gas sensor and publish it to the RabbitMQ MQTT broker with a topic name of : mq2_mqtt.

/*
 Basic ESP8266 MQTT publish client example
*/
#include <ESP8266WiFi.h>
#include <PubSubClient.h>

// Update these with values suitable for your network.
const char* ssid = "your_ssid";
const char* password = "your_password";
const char* mqtt_server = "192.168.0.121"; 
const char* mqtt_user = "admin1";
const char* mqtt_pass= "admin1";

const int mq2pin = A0; //the MQ2 analog input pin

WiFiClient espClient;
PubSubClient client(espClient);

void setup_wifi() {
  // Connecting to a WiFi network
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void reconnect() {
  // Loop until we're reconnected
  Serial.println("In reconnect...");
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("Arduino_Gas", mqtt_user, mqtt_pass)) {
      Serial.println("connected");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      delay(5000);
    }
  }
}

void setup() {
  Serial.begin(9600);
  setup_wifi();
  client.setServer(mqtt_server, 1883);
}

void loop() {
  char msg[8];
  if (!client.connected()) {
    reconnect();
  }

  sprintf(msg,"%i",analogRead(mq2pin));
  client.publish("mq2_mqtt", msg);
  Serial.print("MQ2 gas value:");
  Serial.println(msg);

  delay(5000);
}

Once the MQTT value is published any MQTT client can subscribe to it.  Below is a Python MQTT subscribe example.

# mqtt_sub.py - Python MQTT subscribe example 
#
import paho.mqtt.client as mqtt
 
def on_connect(client, userdata, flags, rc):
    print("Connected to broker")
 
def on_message(client, userdata, message):
    print ("Message received: "  + message.payload)

client = mqtt.Client()
client.username_pw_set("admin1", password='admin1')
client.connect("192.168.0.121", 1883) 

client.on_connect = on_connect       #attach function to callback
client.on_message = on_message       #attach function to callback

client.subscribe("mq2_mqtt") 
client.loop_forever()                 #start the loop

Read MQTT Messages Using AMQP

MQTT clients can subscribe to MQTT messages directly or it’s possible to configure RabbitMQ to have the MQTT data accessible as AMQP. The routing of MQTT messages to AMQP queues is done using the direct exchange method.

mqtt_2_amqp

To configure RabbitMQ to forward MQTT the following steps are done:

  1. Create a new RabbitMQ Queue – For an IoT project this would typically be a 1-to-1 mapping of the MQTT topic to a queue name.
  2. Create a binding between the MQTT Exchange and the Queue – by default all the MQTT topic go into the amq.topic exchange. For MQTT items in this exchange a binding, typically the MQTT topic name, is used as the routing key to the AMQP queue.

These steps can be done in a number of ways : manually, in the RabbitMQ config file, using the rabbitmqadmin command line tool or via a program. Because I was doing this for multiple signals I used rabbitmqadmin tool, and the syntax is:

./rabbitmqadmin declare queue name=mq2_amqp durable=true
./rabbitmqadmin declare binding source=amq.topic destination_type=queue destination=mq2_amqp routing_key=mq2_mqtt

The RabbitMQ Web Admin can be used to verify the exchange to queue binding.

wp_mqtt_2_amqp

 

The CLI tool can also be used to see if there are any values in the queue:

cli_get_queue

Node-Red Dashboard

Node-Red is a visual programming environment that allows users to create applications by dragging and dropping nodes on the screen. Logic flows are then created by connecting the different nodes together.

Node-Red has been preinstalled on Raspbian Jesse since the November 2015. Node-Red can also be installed on Windows, Linux and OSX.  To install and run Node-Red on your specific system see https://nodered.org/docs/getting-started/installation.

To install the AMQP components, select the Manage palette option from the right side of the menu bar. Then search for “AMQP” and install node-red-contrib-amqp. If your installation of Node-Red does not have dashboards installed, search for: node-red-dashboard.

nr_install_amqp

For our Node-Red MQTT and AMQP example we will use a mqtt and a amqp node from the input palette group, and two gauge nodes from the dashboard group.  The complete Node-Red logic for this is done in only 4 nodes!

nr_logic

Nodes are added by dragging and dropped them into the center Flow sheet. Logic is created by making connection wires between inputs and output of a node. After the logic is laid out, double click on each of the nodes to configure their specific properties. You will need to specify the MQTT and AMQP definitions of your RabbitMQ IP address,user rights, MQTT topic and AMQP queue name. You will also need to double click on the gauge nodes to configure the look-and-feel of the web dashboard.

After the logic is complete, hit the Deploy button on the right side of the menu bar to run the logic.  The Node-Red dashboard user interface is accessed by: http://ipaddress:1880/ui. 

For my project I used a number of different MQ sensors and inputs. Below is a picture of Node-Red web dashboard that we created with the same MQTT value being shown natively and as a AMQP queued value.

project

Final Comments

I found that RabbitMQ was easy to install and the Web Administration plug-in along with the rabbitmqadmin command line tool made it very easy to maintain. 

If you’re just looking to show sensor values then a basic MQTT broker might be all you need. However if you’re looking at some future applications like alarm, maintenance or task lists then AMQP exchanges and queue make RabbitMQ an interesting option.

IoT with Google Firebase

For many Internet of Things (IoT) projects a message queuing system like MQTT (Message Queue Telemetry Transport) is all that is needed to connect sensors, devices and graphic interfaces together. If however you require a database, with sorting, queuing and multi-media support then there are some great cloud storage platforms that are available. One that is definitely worth taking a look at is Google Firebase.

Like any IoT solution, Google Firebase can have inputs and sensors sending data directly into it, and a variety of client applications to view the data (Figure 1), but Google Firebase also offers other features such as: file storage, machine learning, messaging, and server side functions. In this article I will look at:

In this article I will look at:

  1. Setting up a sample Firebase IoT project
  2. Use Python to simulate I/O data
  3. Create a Web dashboard with Node-Red to view and write data
  4. Create an Android app using AppInventor to view and write data
  5. Look at a more complex data monitoring example in Python

Getting Started

Google Firebase [1] may not have the huge variety of options that Amazon Web Services (AWS) has, but I found as an IoT engineer Google Firebase had all the features that I needed to get up and running quickly, and I didn’t need a credit card for my free activation.

To begin log in with your Google account at https://firebase.google.com/, and select “Get Started”. I created a sample project called My-IoT. After the project is created Firebase will give it a unique identifier name, such as: my-iot-4ded7.

Firebase data can be stored in either a Realtime Database, or a Cloud Firestore. The Realtime Database is an unstructured NoSQL format that has been around for about 4 years, so there are lots of 3rd party components. The Cloud Firestore stores the data in structure collections and it’s fairly new so the 3rd party options are a little more limited.

For our test IoT project we will use the  real-time database, and it can be created from the Database -> Data menu option.

rtdb

The database structure can be built directly from the Firebase web console or imported from a JSON file. For this database 3 grouping are defined, one for Android, Node-Red and Raspberry Pi I/O values. The Firebase console allows for setting, changing and viewing the database values.

db_struct

The security is configured in the Database -> Rules menu option. To keep things simple for testing read and write security is set to true for public access,  (remember to change this for a production system).

security

The project’s remote access settings are shown in the Authentication section with the “Web setup” button.

webconfig

Python and Firebase

There are a number of Python libraries to access Firebase. The pyrebase library has some added functionality such as: queries, sorting, file downloads and streaming support. The pyrebase library only support Python 3, and it is installed by:

sudo pip3 install pyrebase
sudo pip3 install --upgrade google-auth-oauthlib

A Python 3 test program to set two values (pi/pi_value1 and pi/pi_value2) and read a single point and a group of point is shown below. Remember to change the configuration settings to match your project settings.

import pyrebase, random

# define the Firebase as per your settings
config = {
  "apiKey": "AIzaSyCq_WytLdmOy1AIzaSyCq_WytLdmOy1",
  "authDomain": "my-iot-4ded7.firebaseapp.com",
  "databaseURL": "https://my-iot-4ded7.firebaseio.com",
  "storageBucket": "my-iot-4ded7.appspot.com"
}

firebase = pyrebase.initialize_app(config)
db = firebase.database()

# set 2 values with random numbers
db.child("pi").child("pi_value1").set(random.randint(0,100))
db.child("pi").child("pi_value2").set(random.randint(0,100))

# readback a single value
thevalue = db.child("pi").child("pi_value1").get().val()
print ("Pi Value 1: ", thevalue)

# get all android values
all_users = db.child("android").get()
for user in all_users.each():
    print(user.key(), user.val())

The Firebase web console can be used to check the results from the test program.

Node-Red and Firebase

Node-Red is a visual programming environment that allows users to create applications by dragging and dropping nodes on the screen. Logic flows are then created by connecting the different nodes together.

Node-Red has been preinstalled on Raspbian Jesse since the November 2015. Node-Red can also be installed on Windows, Linux and OSX.  To install and run Node-Red on your specific system see https://nodered.org/docs/getting-started/installation.

To install the Firebase components, select the Manage palette option from the right side of the menu bar. Then search for “firebase” and install node-red-contrib-firebase.

nr_fb_install

For our Node-Red example we will use web gauges to show the values of our two Python simulated test points. We will also send a text comment back to our Firebase database. The complete Node-Red logic for this is done in only 6 nodes!

nr_logic

To read the Pi values two firebase.on() nodes are used. The output from these nodes is connected to two dashboard gauge nodes. Double-clicking on firebase_on() node to configure Firebase database and the item to read from. Double-clicking on the gauge node allows you to edit gauge properties.

nr_value1 To send a text string to Firebase a text input node is wired to a firebase modify node. Edit the firebase modify node with the correct database address and value to set.

nr_modify

After the logic is complete, hit the Deploy button on the right side of the menu bar to run the logic.  The Node-Red dashboard user interface is accessed by: http://ipaddress:1880/ui, so for example 192.168.1.108:1880/ui.

nr_screen

AppInventor

AppInventor is a Web based Android app creation tool (http://appinventor.mit.edu), that uses a graphical programming environment.

AppInventor has two main screens. The Designer screen is used for the layout or presentation of the Android app and the Blocks screen is used to build the logic. On the right side of the top menu bar, the Designer and Blocks buttons allow you to toggle between these two screens.

On the Designer screen, an app layout is created by dragging a component from the Palette window onto the Viewer window.

We will use AppInventor to create an Android app that will read Pi values from our Firebase IoT database, and write a value back.

For the visuals on this application a Button, Label (2),  ListView, Textbox and FirebaseDB component will be used. After a component is added to the application, the Components window is used to rename or delete that component. The Properties window is used to edit the features on a component. For the FirebaseDB component configure the Token and URL to match with your project settings.

fb_app_setup

Once the layout design is complete, logic can be added by clicking on the Blocks button on the top menu bar.

Logic is built by selecting an object in the Blocks window, and then click on the specific block that you would like to use.

AppInventor is pretty amazing when it comes to doing some very quick prototyping. Our entire app only uses two main blocks.

The when FirebaseDB1.DataChanged block is executed whenever new data arrives into the Firebase database. The DataChanged block returns a tag and a value variable. The tag variable is the top level item name, (“pi” or “nodered” for our example). The value will be a string of the items and their values, for example: “{pi_value2 = 34, pi_value1=77}”. We use a replace all text block to remove the “{” and “}” characters, then we pass the string to the ListView component. Note this same code will pass 2 or 200 tags in the pi tag section.

The when BT_PUT.click block will pass the text that we enter on the screen into the android/and_value1 item in our database.

fb_app_code

After the screen layout and logic is complete, the menu item Build will compile the app. The app can be made available as an APK downloadable file or as a QR code link.

Below is picture of the final Android app synced with the Firebase data.

Firebase2

Python Data Monitoring Example

Our starting IoT Firebase realtime database was quite simple. Data monitoring or SCADA (Supervisory Control and Data Aquisition) projects usually require more information than a just a value. A more realistic sensor database would includes fields such as tag name, description, status, time and units.

fb_db2

By adding some indexing in the Firebase Security Rules it is possible to create some queries and sorts of the data. Some typical queries would be: “Points in alarm” or “Point values between 2:00 and 2:15”.

fb_rules2

In the code listing below the syntax statement of .order_by_child(“status”).equal_to(“ALARM”).get() is used to show only records that have a status = ALARM. Some other filter options include: .start_at(time1).end_at(time2).get, .limit_to_first(n), and .order_by_value().

A good next step would be to pass important filtered information to Google Firebase’s messaging feature.

import pyrebase, random

# define the Firebase as per your settings
config = {
  "apiKey": "AIzaSyCq_AIzaSyCq_Wyt",
  "authDomain": "my-iot-7ded7.firebaseapp.com",
  "databaseURL": "https://my-iot-7ded7.firebaseio.com",
  "storageBucket": "my-iot-7ded7.appspot.com"
}

firebase = pyrebase.initialize_app(config)
db = firebase.database()

tag_sum = db.child("pi").order_by_child("status").equal_to("ALARM").get()

# print alarm points
for tag in tag_sum.each():
    info = tag.val()
    print (tag.key()," - ", info["description"],":", info["value"],info["units"], info["status"])

For more Python examples see the Pyrebase documentation.

Summary

Without a lot of configuration it is possible to configure a Google Firebase project and have Python, Node-Red and Android app read and write values. The level of programming complexity is on par with an MQTT implementation, however Google Firebase can offer a lot more future functionality that you wouldn’t have with MQTT, such as file storage, machine learning, messaging, and server side functions.

At the time of writing this article I found the Arduino Firebase library to be unreliable but Google offers some options like an MQTT bridge.

 

Control with Texts (Node-Red on Android)

In places where Internet connections are not possible or too expensive,  SMS text messaging can be a simple approach for monitoring and controlling your remote systems. Many of the mobile providers offer IoT packages for low data throughput, where you’d be looking at spending $1 to $5  per month for 1-5 MB of data.  From the hardware standpoint there are many options such digital modem modules that come in either a Raspberry Pi top or a USB form factor.

If you are looking at doing some prototyping, using an Arduino phone and Node-Red is a great way to get jump started.

Node-Red on Android

Node-Red is a graphical programming system that is used for Internet of Things (IoT) projects. Node-Red is installed in the base Raspberry Pi images, but it can also be loaded on Linux, Windows, MacOS and Android systems.

To load Node-Red on Arduino, you will first need to load Termux,  an Android Terminal Emulator app, that is available at Google Play. After Termux is loaded, enter the following commands to install  and run Node-Red:

apt update
apt upgrade
apt install coreutils nano nodejs
npm i -g --unsafe-perm node-red
node-red

Node-Red will start a Web graphical interface that is accessed by:  http://the_ip_address:1880.

nodered_pc

Extra features can be added and removed from Node-Red by selecting the “Manage Palette” menu option. For this project I needed terminux-api for the texting support, and bigtimer to scanning the text message buffer.

manage_palette

A Test Setup

For a basic setup I used:

  • 1 Android phone with Termux and Node-Red
  • 1 Raspberry Pi with Node-Red
  • 1 Powerswitch Tail II connected to a light
  • 1 Android phone for texting in

sms_pi_setup

Scanning for Text Messages

To create a simple text message application on the Android Note-Red system, the following components are used:

  • A Big Timer node is used to define how often the SMS inbox is scanned. Without doing any configuration, the second output will offer a 1/minute cycle time.
  • An SMS Inbox node will read in a defined number of text messages. To get the last message select the buffer limit to be 1.
  • A function node is used with some JavaScript to check for valid text messages
  • A TCP out node will send a message to another Node-Red system.

A basic logic setup will be as follows:

sms_in_logic

The function node needs some logic to interpret the latest text message and send out the required logic to the Raspberry Pi GPIO pins. The example logic uses a text message of  “LIGHTS ON” or “LIGHTS OFF” to control the output on GPIO pin 12.


// look for new action texts
smslast = context.get('smslast')|| 0;

// Do an action if there is a new SMS message
if ( msg.payload[0].received != smslast) {
    context.set('smslast', msg.payload[0].received )
    smsbody = msg.payload[0].body;
    if (smsbody.toUpperCase().trim() == "LIGHTS ON") {
        msg.payload = 1;  // this is sent to the Pi via TCP
        return msg;
    }
    if (smsbody.toUpperCase().trim() == "LIGHTS OFF") {
        msg.payload = 0;  // this is sent to the Pi via TCP
        return msg;
    }
}

The first time that the code runs you will be prompted on the phone to “Allow Termux:API to send and view SMS messages?”. After you acknowledge this message your Node-Red logic will run cleanly.

Reading TCP Inputs

Android phones can not be directly connected I/O devices, so TCP connections can used to pass commands to Raspberry Pi’s which have the I/O support. Node-Red is pre-installed on most Raspberry Pi images so no added installation is required.

On the Raspberry Pi Node-Red side only a couple of nodes are required:

  • A TCP in node is used to read the incoming communications. For the configuration of this node it is important to have the port (8888 in this case) match the TCP out node’s port. Also the output should be set as: single, and string.
  • An RPI GPIO out node is used set Raspberry Pi General Purpose (GPIO) pins.

sms_tcpin

Texting Out from Android

If your text scheme is simple, for example one light that is turned on and off, then you could manually just output an “ON” or “OFF” message. However if you are controlling multiple lights, heaters and other devices then manually typing the text messages gets awkward.

To manage multiple commands, I wrote a simple Android app in MIT’s App Inventor. App Inventor is a free online Android development environment, that only requires a Google login for access. In about 10 minutes I was able to get a prototype going with multiple inputs. The first step is to drag and drop some buttons from the User Interface Palette onto the viewer pane. Then drag and drop a Texting component from the Social Palette onto the view. Note, the Texting component will appear as a non-visible item.

text2red_gui

After you have a basic layout, select the Blocks button on the menu bar, and this will open a logic page. Logic is created by clicking on the objects in the Block panel, and then dragging and dropping the block onto the Viewer panel.

To have a button sent a text message the following pieces are required:

  • “when Button.Click”  will be called when the users touches the button object
  • “set Texting1.PhoneNumber”  defines the remote phone number
  • “set Texting1.Message” defines the text message
  • “call Texting.SendMessage”  will send the message

text2red_blocks

To build the application use the Build menu item on the menu bar.

app2qr

For my test project I configured four devices and eight commands.text2sms_app

Final Comments

I found that using Node-Red on Android to be a lot faster than I expected, however I noticed that some of the added features (like Bluetooth support) only worked on the Raspberry Pi/Linux version of the Node-Red.

For a final solution I would definitely move to dedicated SMS hardware, but I found it nice to be able to do proof of concept testing with just some basic Android phones. Also don’t forget to setup Node-Red for automatic startup on a power up.

Arduino talking MQTT to Node-Red

There are some great Arduino modules with integrated ESP-8266 wireless chips, some of the more popular modules are:

  • Adafruit HUZZAH
  • NodeMCU
  • WeMos

These modules allow you to do some interesting IoT (Internet of Things) projects. To connect the Arduino modules to PCs, Raspberry Pi’s or Linux nodes that are a number of communication choices. MQTT (Message Queue Telemetry Transport) is becoming one of the standards for this and it is pre-installed with Node-Red.

Plant Moisture Monitoring MQTT Example

Ard_mqtt_overview2

For our example we wanted to do a simple plant moisture example that used a solar charger and an Arduino Wemos module. We then setup an MQTT server on our Node-Red Raspberry Pi with a web dashboard.

Our goal was to get the MQTT technologies working, with some moisture inputs (and not a final plant monitoring system).

Moisture Sensors

Moisture sensors are very low cost and they start at about $2. The basic moisture sensor has 3 inputs; VCC, GND, and AO. Some sensors also include a digital output with a potentiometer to adjust the digital 0-1 moisture limit.

Our Arduino plant moisture setup is good for testing but not a good long term solution. When voltage is applied long term to moisture sensors ionization in the soil will cause a combination of false reading and deterioration of the sensor plates. We plan to do a future project where we will use relays to turn the sensors on/off and we will include solenoid values in a watering system.

MQTT on Arduino

There are a number of excellent MQTT libraries for Arduino, for this example we used the PubSubClient library. This library can be installed from the Arduino IDE by selecting the menu items:

Sketch -> Add Library -> Manage Libraries

To get MQTT setup you’ll need to:

  • define the SSID and password for your WAN
  • define the IP address for the MQTT server (the Node Red/Raspberry Pi node)
  • define some topic for the data

The nice thing about MQTT is that you can define topics for each of your data points. For this example we define the topic humidity to show the moisture sensor value, and msgtext to show the message (‘Needs Water’ or ‘).

Below is our sample Arduino code for passing the moisture data to our MQTT server.

/*
 Basic ESP8266 MQTT publish client example for a moisture sensor
*/
#include <ESP8266WiFi.h>
#include <PubSubClient.h>

// Update these with values suitable for your network.
const char* ssid = "YOUR_SSID_NAME";
const char* password = "YOUR_PASSWORD";
const char* mqtt_server = "YOUR_NODE_RED_IP";

WiFiClient espClient;
PubSubClient client(espClient);

void setup_wifi() {
  // Connecting to a WiFi network
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void reconnect() {
  // Loop until we're reconnected
  Serial.println("In reconnect...");
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("Arduino_Moisture")) {
      Serial.println("connected");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      delay(5000);
    }
  }
}

void setup() {
  Serial.begin(9600);
  setup_wifi();
  client.setServer(mqtt_server, 1883);
}

void loop() {
  char msg[10];
  char msgtext[25];
  String themsg;
  if (!client.connected()) {
    reconnect();
  }
  
  int soil_moisture=analogRead(A0);  // read from analog pin A0
  Serial.print("analog value: ");
  Serial.println(soil_moisture);
  
  if((soil_moisture>300)&&(soil_moisture<700)) {
    Serial.println("Humid soil");
    sprintf(msgtext,"Humid soil",soil_moisture);
  } 
  else if ((soil_moisture>700)&&(soil_moisture<950)){
    Serial.println("Moist Soil");
    sprintf(msgtext,"Moist Soil",soil_moisture);
  }
  else if (soil_moisture <300) ){
    Serial.println("Needs water");    
    sprintf(msgtext,"Needs water",soil_moisture);
  }
  else
  {
      sprintf(msgtext,"Sensor Problem",soil_moisture);
  }

  sprintf(msg,"%i",soil_moisture);
  client.publish("humidity", msg);
  client.publish("soil", msgtext);
  delay(5000);
}

Node-Red

Node-Red is an excellent visual programming environment that is part of the Raspberry Pi base install. Node-Red is a simple tool to create your own Internet of Things applications. The base Node-Red installation includes MQTT interfacing components but it does not include an MQTT server.

If you don’t have a Raspberry Pi you can install Node-Red on Window, Mac OS or Linux systems. I’ve had good results running Node-Red on a very old low end laptop running Xubuntu, (see steps 1-3 in this linked guide).

MQTT Server on Node-Red

There are a number of free internet MQTT servers (iot.eclipse.org) that can be used or an MQTT server can be loaded directly on a local server (i.e. Mosquito).

Mosca is a standalone MQTT server that can be installed directly into Node-Red. The Mosca Node-Red component can be either installed at the command line by:

cd $HOME/.node-red

npm install node-red-contrib-mqtt-broker

Or the component can be install within the Node-Red web interface by selecting the “manage palette” option, and then search for mosca.

mosca_install

After Mosca is installed, all that is required is that a “mosca” node needs to be dragged and dropped into the Node-Red project.

mosca_mqtt_nodes

To connect the Arduino module to Node-Red mqtt inputs are added to the project.  The Arduino topics are defined in Node-Red by double-clicking on the mqtt node and then define the topic to match the Arduino topic.

mqtt_topic

After the MQTT connections are configured Web dashboards can present the final data. The Web dashboards offer a number of different components that could be used for this example I used a gauge and a text node.

To compile and view the Node-Red application, click on the Deploy button on the right side of the menu bar. To access the web dashboard enter: http://your-ip:1880/ui . Below is a example of what you should see.

NodeRed_MQTT_GUI

Final Thoughts

The Mosca MQTT server component allows for a simple standalone architecture to connect wireless Arduino modules into a Node-Red IoT solution.

We can also load Node-Red on Raspberry Pi data collection nodes, and have them publish data to a central Node-Red server.

 

 

 

Arduino talking TCP to Node-Red and Python

There are some great Arduino modules with integrated ESP-8266 wireless chips, some of the most popular of these modules are:

  • Adafruit HUZZAH
  • NodeMCU
  • WeMos

Along with these modules comes some excellent libraries.

For Arduino to PC or Raspberry Pi communications that are a few options to choose from. A TCP client/server is simple and straightforward and it is excellent for sending single point information. For sending multiple data points take a look at MQTT (Message Queuing Telemetry Transport), it’s a common standard for IoT applications and it’s built into Node-Red.

Arduino TCP Client

The Arduino module can be a simple TCP client that can talk to either a Python or a Node-Red TCP server. Below is an example that sends a random integer to a TCP server every 5 seconds.

/*
Test TCP client to send a random number
 */
#include <ESP8266WiFi.h>
#include <ESP8266WiFiMulti.h>

ESP8266WiFiMulti WiFiMulti;

void setup() {
    Serial.begin(9600);

    // We start by connecting to a WiFi network
    const char * ssid = "your_ssid";       // your WLAN ssid
    const char * password = "your_password"; // your WLAN password
    WiFiMulti.addAP(ssid, password);

    Serial.println();
    Serial.print("Wait for WiFi... ");

    while(WiFiMulti.run() != WL_CONNECTED) {
        Serial.print(".");
        delay(500);
    }
    Serial.println("WiFi connected");
    Serial.println("IP address: ");
    Serial.println(WiFi.localIP());
    delay(500);
}

void loop() {
    const uint16_t port = 8888;          // port to use
    const char * host = "192.168.0.123"; // address of server
    String msg;

    // Use WiFiClient class to create TCP connections
    WiFiClient client;

    if (!client.connect(host, port)) {
        Serial.println("connection failed");
        Serial.println("wait 5 sec...");
        delay(5000);
        return;
    }
   
    // Send a random number to the TCP server
    msg = String(random(0,100));
    client.print(msg);
    Serial.print("Sent : ");
    Serial.println(msg);
    client.stop();    
    delay(5000);
}

Node-Red TCP Server

Node-Red is an excellent visual programming environment that is part of the Raspberry Pi base install. Node-Red is a simple tool to create your own Internet of Things applications. The base Node-Red installation includes a TCP server and client.

To install the web dashboards enter:

sudo apt-get install npm
cd ~/.node-red
npm i node-red-dashboard

To start Node-Red either use the on-screen menus or from the command line enter:

node-red-start &

Once Node-Red is running the programming is done via the web interface at: //localhost:1880 or //your_Pi_ip_address:1880 .

To configure the TCP server, go to the Input nodes section and drag and drop the TCP in node. After the node is inserted double-click on it and edit the port and message settings.

nodered_tcp

To create a gauge Web dashboard, go to the dashboard nodes section and drag and drop the gauge node. After the node is inserted double-click on it and edit the dashboard group, labels and ranges.

nodered_tcp_gauge

For debugging and testing an output debug node is useful.

To access the Web dashboard enter: //localhost:1880/ui or //your_Pi_ip_address:1880/ui

TCP Python Server

The python TCP server will see the incoming Arduino message as a Unicode (UTF-8) text, so convert message to an integer use: thevalue = int(data.decode(“utf-8”)). Below is the full code.

import socket
import sys

HOST = '' # Symbolic name, meaning all available interfaces
PORT = 8888 # Arbitrary non-privileged port

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print ('Socket created')

#Bind socket to local host and port
try:
s.bind((HOST, PORT))
except socket.error as msg:
print ('Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1])
sys.exit()

print ('Socket bind complete')

#Start listening on socket
s.listen(10)
print ('Socket now listening')

#now keep talking with the client
while True:
#wait to accept a connection - blocking call
conn, addr = s.accept()
data = conn.recv(1024)
print ('Connected with ' + addr[0] + ':' + str(addr[1]) + " " )
thevalue = int(data.decode("utf-8"))
print ("Value: ", thevalue)

s.close(

Summary

In our final application we mounted the Arduino module outside and we powered it with a small solar charger. We also include a humidity value with the temperature, and we used a QR code that linked to our web page.

Wake Up !!! Wake Up !!!

The goal of the “Wake me up” programs is to set a wake time that will turn on the lights and say something like: “Wake Up Wake Up”.  The logic was done in Node-Red and it was quite straightforward. The hardware used was:

The PowerSwitch Tail II ($26) is a power cord that is enabled/disabled with I/O pins. The PowerSwitch pins connect to GPIO18 and GND on the Pi. A desk light is plugged into the PowerSwitch Tail and speakers are connected to audio jack on the Pi.

pi2switch

With the PowerSwitch Tail II there are a lot of devices that could be controlled. For us, we simply connected it to a desk lamp. The setup shown below was later moved into a bedroom and place on a bedside table.

wakeup_hwd

For this project two Node-Red libraries were used; a scheduler library and a text to speech library. They are installed by:

 sudo apt-get install festival
cd $HOME/.node-red
npm install node-red-contrib-say
npm install node-red-contrib-simple-weekly-scheduler 

After the packages are loaded restart the Pi, and then start up Node-Red. On the Node-Red configuration Web page, drop a scheduler, and wire a Pi GPIO, and a Say node to the scheduler.

3nodes

Double-click the scheduler node and set the wake up times. The start/end payloads are numeric and 1/0. Below is the scheduler configuration.

schedule

For the Pi GPIO node, set the GPIO to Pin 12/GPIO18.

powerinfo

Next for the Say node, enter the text you want spoken .

wakeup

For the final circuit, inject nodes can be included for testing. An “on” inject sends a 1, and an “off” inject sends 0. At the wake up time the schedule node will send a payload that will trigger the speaking of the wake up text and the desk light will be turned on for 15 minutes.

final_wakeup

Pi/Node-Red Car

The goal of the Pi/Node-Red car project was to create a small vehicle that can be controlled from a smart phone . For the project we used:

  • 1 Car chassis for Arduino ($15)
  • 1 Pimoroni Explorer HAT Pro  ($23)
  • 1 Portable microUSB charger
  • 1 USB WiFi Adapter
  • 4 short alligator clips and 4 connectors
  • Duct tape

The Arduino car chassis may require a small amount of assembly. Rather than soldering connections we like to use short alligator clips. It is not recommended to wire DC motors directly to a Raspberry Pi so the Pimoroni Explorer HAT Pro is used to connect the 2 DC motors.

The Raspberry Pi and the portable microUSB charger are secured to the top of the car chassis with duct tape. The left motor is wired to the motor 1 connectors on the Explorer Hat, and the right motor is wired to motor 2 connectors. Note you may have to do a little trial and error on the Explorer HAT “+” and “-” motor connections to get both wheels spinning in a forward direction.

The Explorer HAT Node-Red library is installed by:

 cd $HOME/.node-red
npm install node-red-dashboard 

The Web dashboard presentation is configured in the “dashboard” tab. For this example we create 2 groups: a control group to drive the vehicle, and a light group to turn on the Explorer Pro lights. Use the “+group” button to add a group, and the “edit” to change an existing group.
dash_conf

To control a motor, an “Explorer HAT” node and a dashboard button node are dropped and connected together. All the configuration is done in the button node . The button configure options are:

  • the group the button will appear in (Controls)
  • the size of the button (3×1 = 50% of width and narrow)
  • Topic, motor.one or motor.twois used for motor control
  • Payload, -100 = reverse, 0=stop, 100 = forward

Control_conf

The Explorer HAT has 4 colored LEDs. To toggle the LEDS, the topic is light.color with 1=ON, and 0=OFF . We thought that it would be fun to also add some Web dashboard button to control the colored lights.

light_conf

The Node-Red dashboard user interface is accessed by: ipaddress:1880/UI, so for example 192.168.1.102:1880/ui. Below is a picture that shows the final vehicle logic and the Web dashboard.

 

final_logic2