RabbitMQ for IoT

For Internet of Things (IoT) projects that are a lot of different ways that the sensors, devices and client interfaces can be connected together.

RabbitMQ offers the unique ability of being able to use some of the common IoT protocols. RabbitMQ also supports a large variety of client applications and different RabbitMQ servers can mirror, cluster or gateway themselves together.

rabbitmq_overview

In this blog I will look at setting up a RabbitMQ server, and then I create an example of an MQTT value from an Arduino module that can be presented as an AMQP item.

Comparing MQTT and AMQP

Before we get started it’s useful to comment about some of the differences between MQTT (Message Queue Telemetry Transport) and AMQP (Advanced Message Queueing Protocol) .

MQTT is a light weight publish-subscribe-based messaging protocol that works well with lower end hardware and limited bandwidth. For Arduino type applications where you only need to pass some sensor data MQTT is an excellent fit.

AMQP has more overhead than MQTT, because it is a more advanced protocol that includes message orientation, queuing, routing, reliability and security. Presently there are no mainstream AMQP Arduino libraries, but numerous programming options for Raspberry Pi, Linux, Windows and OSX systems exist.  An AMQP IoT example would be to send sensor failures and alarms to dedicated maintenance and alarm queues.

Installing RabbitMQ

For RabbitMQ installation instructions see: https://www.rabbitmq.com/download.html . To install and run RabbitMQ on a Ubuntu system enter:

sudo apt-get update
sudo apt-get install rabbitmq-server
sudo service rabbitmq-server start

The next step is to add some plug-ins. For my project I loaded the MQTT and Web Administration plug-ins.

sudo rabbitmq-plugins enable rabbitmq_mqtt
sudo rabbitmq-plugins enable rabbitmq-management

The rabbitmqctl command line tool allows you to configure and review the RabbitMQ server. To add a user admin1,  with password admin1, that has config, write and read access for management and administrator rights, enter:

sudo rabbitmqctl add_user admin1 admin1
sudo rabbitmqctl set_permissions -p / admin1 ".*" ".*" ".*"
sudo rabbitmqctl set_user_tags admin1 management administrator

After you’ve defined an administrative user the RabbitMQ web management plug-in can be accessed by: http://ip_address:15672 .

Rabbit_webadmin

The RabbitMQ Web Management tool offers an overview of the present system load, connections, exchanges and queues. 

The RabbitMQ Web Management tool is excellent for small manual changes, if however you are looking a doing mass changes there is a management command line tool, rabbitmqadmin, that can be used. The cli tool is already on the system it just needs to be installed.

# Get the cli and make it available to use.
wget http://127.0.0.1:15672/cli/rabbitmqadmin
sudo chmod +x rabbitmqadmin

 

MQTT and AMQP Queues

One of the biggest differences is that MQTT queues are designed to show you the last available message, where as AMQP will store multiple messages in a queue.

The rabbitmqadmin tool can be used to create an AMQP queue, as well as publish and get values.

$./rabbitmqadmin declare queue name=myqueue1 durable=true 
queue declared 

$ ./rabbitmqadmin publish routing_key=myqueue1 payload='this is a test to myqueue1' 
Message published 

$ ./rabbitmqadmin get queue=myqueue1 
+-------------+----------+---------------+----------------------------+---------------+------------------+------------+--------- ----+ 
| routing_key | exchange | message_count | payload | payload_bytes | payload_encoding | properties | redelive red | 
+-------------+----------+---------------+----------------------------+---------------+------------------+------------+--------- ----+ 
| myqueue1    |          | 0             | this is a test to myqueue1 | 26 | string | | False | 
+-------------+----------+---------------+----------------------------+---------------+------------------+------------+--------- ----+

The Web plug-in can also be used to declare, publish and get messages. For the example below it’s important to note that there are 3 more messages waiting in the queue.

get_messages

RabbitMQ Messaging

There are a variety of ways that AMQP messages can be published and subscribed to. The simplest way is to create a queue and then messages can be published and subscribed to from that queue.

rabbitmq_layout

AMQP supports a number of different exchange types (direct, fanout, headers and topic) that can used to manage the distribution and filtering of messages. Messages in an exchange use bindings based on a routing key to link them to a queue.

After the MQTT plug-in is installed RabbitMQ can act like a standalone MQTT broker. MQTT data can also be made available through an AMQP subscription by binding the MQTT exchange to a queue.

Connecting MQTT

For an MQTT project any ESP8266 supported Arduino hardware can be used. There are a number of MQTT Arduino libraries that are available. For this project I used the PubSubClient that can be installed using the Arduino Library Manager.

As a test project I used at low cost MQ-2 Combustible Gas Sensor ($3) that measures a combination of LPG, Alcohol, Propane, Hydrogen, CO and even methane. To fully use this sensor some calibration is required. On the MQ-2 sensor the analog signal is connected to Arduino pin A0 and the analogRead(thePin) function is used to get the value.

MPq2_Setup.png

Below is some example Arduino code required to read the MQ2 gas sensor and publish it to the RabbitMQ MQTT broker with a topic name of : mq2_mqtt.

/*
 Basic ESP8266 MQTT publish client example
*/
#include <ESP8266WiFi.h>
#include <PubSubClient.h>

// Update these with values suitable for your network.
const char* ssid = "your_ssid";
const char* password = "your_password";
const char* mqtt_server = "192.168.0.121"; 
const char* mqtt_user = "admin1";
const char* mqtt_pass= "admin1";

const int mq2pin = A0;

WiFiClient espClient;
PubSubClient client(espClient);

void setup_wifi() {
  // Connecting to a WiFi network
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void reconnect() {
  // Loop until we're reconnected
  Serial.println("In reconnect...");
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("Arduino_Gas", mqtt_user, mqtt_pass)) {
      Serial.println("connected");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      delay(5000);
    }
  }
}

void setup() {
  Serial.begin(9600);
  setup_wifi();
  client.setServer(mqtt_server, 1883);
}

void loop() {
  char msg[8];
  if (!client.connected()) {
    reconnect();
  }

  sprintf(msg,"%i",analogRead(mq2pin));
  client.publish("mq2_mqtt", msg);
  Serial.print("MQ2 gas value:");
  Serial.println(msg);

  delay(5000);
}

Once the MQTT value is published any MQTT client can subscribe to it.  Below is a Python MQTT subscribe example.

# mqtt_sub.py - Python MQTT subscribe example 
#
import paho.mqtt.client as mqtt
 
def on_connect(client, userdata, flags, rc):
    print("Connected to broker")
 
def on_message(client, userdata, message):
    print ("Message received: "  + message.payload)

client = mqtt.Client()
client.username_pw_set("admin1", password='admin1')
client.connect("192.168.0.121", 1883) 

client.on_connect = on_connect       #attach function to callback
client.on_message = on_message       #attach function to callback

client.subscribe("mq2_mqtt") 
client.loop_forever()                 #start the loop

Read MQTT Messages Using AMQP

MQTT clients can subscribe to MQTT messages directly or it’s possible to configure RabbitMQ to have the MQTT data accessible as AMQP. To configure RabbitMQ to forward MQTT you’ll need to do the following steps:

  1. Create a new AMQP Queue – For an IoT project this would typically be a 1-to-1 mapping of the MQTT topic to an AMQP queue
  2. Create a binding between the MQTT Exchange and the Queue – by default all the MQTT topic go into the amq.topic exchange. For MQTT items in this exchange a binding, typically the MQTT topic name, is used as the routing key to the AMQP queue

The binding definitions can be done manually, in the RabbitMQ config file, using rabbitmqadmin or via a program.

For this example my MQTT topic is called mq2_mqtt  and we will create an AMQP queue called mq2_amqp. To manually create a queue, go to the Queues tab in the Web Management tool and define a new queue name.

create_amqp_q

The default MQTT setup puts all the MQTT topics into the amq.topic exchange. To create our binding we need to go to the Exchanges tab and select our MQTT item. Next we select the queue to bind to (mq2_amqp) and set the routing key to the MQTT topic, (mq2_mqtt).

bind_mqtt

The same operation with the command line tool would be:

./rabbitmqadmin declare queue name=mq2_amqp durable=true
./rabbitmqadmin declare binding source=amq.topic destination_type=queue destination=mq2_amqp routing_key=mq2_mqtt

The CLI tool can also be used to see if there are any values in the queue:

cli_get_queue

Now we can use any of the AMQP client libraries to read the Arduino MQ-2 gas value. Below is an example where I used Node-Red to create a web dashboard that showed the native MQTT gas value, the AMQP routed gas value, and a Raspberry Pi Python generated AMQP value.

nr_rabbitmq

Final Comments

RabbitMQ has a lot of features and functions that may or may not be applicable for an IoT project, but the key thing is that it is pretty easy to merge MQTT and AMQP messaging into a common platform.

IoT with Google Firebase

For many Internet of Things (IoT) projects a message queuing system like MQTT (Message Queue Telemetry Transport) is all that is needed to connect sensors, devices and graphic interfaces together. If however you require a database, with sorting, queuing and multi-media support then there are some great cloud storage platforms that are available. One that is definitely worth taking a look at is Google Firebase. Firebase is well documented and it has a free option (that does not require a credit card for activation) so it’s easy to start prototyping and testing.

In this article we will look at:

  1. Setting up a sample Firebase IoT project
  2. Use Python to simulate I/O data
  3. Create a Web dashboard with Node-Red to view and write data
  4. Create an Android app using AppInventor to view and write data
  5. Look at a more complex data monitoring example in Python

Getting Started

To begin log in with your Google account at https://firebase.google.com/ , and select “Get Started”. For this example we will create a sample project called My-IoT.  After the project is created Firebase will give it a unique identifier name, such as: my-iot-4ded7 .

Firebase data can be stored in either a Realtime Database, or a Cloud Firestore. The Realtime Database is an unstructured NoSQL format that has been around for about 4 years, so there are lots of 3rd party components. The Cloud Firestore stores the data in structure collections and it’s in beta so the 3rd party options are a little more limited.

For our test IoT project we will use the  real-time database, and it can be created from the Database -> Data menu option.

rtdb

The database structure can be built directly from the Firebase web console or imported from a JSON file. For this database 3 grouping are defined, one for Android, Node-Red and Raspberry Pi I/O values. The Firebase console allows for setting, changing and viewing the database values.

db_struct

The security is configured in the Database -> Rules menu option. To keep things simple for testing read and write security is set to true for public access,  (remember to change this for a production system).

security

The project’s remote access settings are shown in the Authentication section with the “Web setup” button.

webconfig

Python and Firebase

There are a number of Python libraries to access Firebase. The pyrebase library has some added functionality such as: queries, sorting, file downloads and streaming support. The pyrebase library only support Python 3, and it is installed by:

sudo pip3 install pyrebase
sudo pip3 install --upgrade google-auth-oauthlib

A Python 3 test program to set two values (pi/pi_value1 and pi/pi_value2) and read a single point and a group of point is shown below. Remember to change the configuration settings to match your project settings.

import pyrebase, random

# define the Firebase as per your settings
config = {
  "apiKey": "AIzaSyCq_WytLdmOy1AIzaSyCq_WytLdmOy1",
  "authDomain": "my-iot-4ded7.firebaseapp.com",
  "databaseURL": "https://my-iot-4ded7.firebaseio.com",
  "storageBucket": "my-iot-4ded7.appspot.com"
}

firebase = pyrebase.initialize_app(config)
db = firebase.database()

# set 2 values with random numbers
db.child("pi").child("pi_value1").set(random.randint(0,100))
db.child("pi").child("pi_value2").set(random.randint(0,100))

# readback a single value
thevalue = db.child("pi").child("pi_value1").get().val()
print ("Pi Value 1: ", thevalue)

# get all android values
all_users = db.child("android").get()
for user in all_users.each():
    print(user.key(), user.val())

The Firebase web console can be used to check the results from the test program.

Node-Red and Firebase

Node-Red is a visual programming environment that allows users to create applications by dragging and dropping nodes on the screen. Logic flows are then created by connecting the different nodes together.

Node-Red has been preinstalled on Raspbian Jesse since the November 2015. Node-Red can also be installed on Windows, Linux and OSX.  To install and run Node-Red on your specific system see https://nodered.org/docs/getting-started/installation.

To install the Firebase components, select the Manage palette option from the right side of the menu bar. Then search for “firebase” and install node-red-contrib-firebase.

nr_fb_install

For our Node-Red example we will use web gauges to show the values of our two Python simulated test points. We will also send a text comment back to our Firebase database. The complete Node-Red logic for this is done in only 6 nodes!

nr_logic

To read the Pi values two firebase.on() nodes are used. The output from these nodes is connected to two dashboard gauge nodes. Double-clicking on firebase_on() node to configure Firebase database and the item to read from. Double-clicking on the gauge node allows you to edit gauge properties.

nr_value1 To send a text string to Firebase a text input node is wired to a firebase modify node. Edit the firebase modify node with the correct database address and value to set.

nr_modify

After the logic is complete, hit the Deploy button on the right side of the menu bar to run the logic.  The Node-Red dashboard user interface is accessed by: http://ipaddress:1880/ui, so for example 192.168.1.108:1880/ui.

nr_screen

AppInventor

AppInventor is a Web based Android app creation tool (http://appinventor.mit.edu), that uses a graphical programming environment.

AppInventor has two main screens. The Designer screen is used for the layout or presentation of the Android app and the Blocks screen is used to build the logic. On the right side of the top menu bar, the Designer and Blocks buttons allow you to toggle between these two screens.

On the Designer screen, an app layout is created by dragging a component from the Palette window onto the Viewer window.

We will use AppInventor to create an Android app that will read Pi values from our Firebase IoT database, and write a value back.

For the visuals on this application a Button, Label (2),  ListView, Textbox and FirebaseDB component will be used. After a component is added to the application, the Components window is used to rename or delete that component. The Properties window is used to edit the features on a component. For the FirebaseDB component configure the Token and URL to match with your project settings.

fb_app_setup

Once the layout design is complete, logic can be added by clicking on the Blocks button on the top menu bar.

Logic is built by selecting an object in the Blocks window, and then click on the specific block that you would like to use.

AppInventor is pretty amazing when it comes to doing some very quick prototyping. Our entire app only uses two main blocks.

The when FirebaseDB1.DataChanged block is executed whenever new data arrives into the Firebase database. The DataChanged block returns a tag and a value variable. The tag variable is the top level item name, (“pi” or “nodered” for our example). The value will be a string of the items and their values, for example: “{pi_value2 = 34, pi_value1=77}”. We use a replace all text block to remove the “{” and “}” characters, then we pass the string to the ListView component. Note this same code will pass 2 or 200 tags in the pi tag section.

The when BT_PUT.click block will pass the text that we enter on the screen into the android/and_value1 item in our database.

fb_app_code

After the screen layout and logic is complete, the menu item Build will compile the app. The app can be made available as an APK downloadable file or as a QR code link.

Below is picture of the final Android app synced with the Firebase data.

Firebase2

Python Data Monitoring Example

Our starting IoT Firebase realtime database was quite simple. Data monitoring projects usually require more information than a just a value. Below is a more realistic sensor database with fields like tag name, description, status, time and unit.

fb_db2

By adding an .indexOn rule to the tags in the pi section of the database it is possible to create queries and sorts.

fb_rules2

A Python example to query the database and find all point in alarm would be:

import pyrebase, random

# define the Firebase as per your settings
config = {
  "apiKey": "AIzaSyCq_AIzaSyCq_Wyt",
  "authDomain": "my-iot-7ded7.firebaseapp.com",
  "databaseURL": "https://my-iot-7ded7.firebaseio.com",
  "storageBucket": "my-iot-7ded7.appspot.com"
}

firebase = pyrebase.initialize_app(config)
db = firebase.database()

tag_sum = db.child("pi").order_by_child("status").equal_to("ALARM").get()

# print alarm points
for tag in tag_sum.each():
    info = tag.val()
    print (tag.key()," - ", info["description"],":", info["value"],info["units"], info["status"])

 

For more Python example see the Pyrebase documentation.

Summary

Without a lot of configuration it is possible to configure a Google Firebase project and have both Python , Node-Red and Android app read and write values.

 

Control with Texts (Node-Red on Android)

In places where Internet connections are not possible or too expensive,  SMS text messaging can be a simple approach for monitoring and controlling your remote systems. Many of the mobile providers offer IoT packages for low data throughput, where you’d be looking at spending $1 to $5  per month for 1-5 MB of data.  From the hardware standpoint there are many options such digital modem modules that come in either a Raspberry Pi top or a USB form factor.

If you are looking at doing some prototyping, using an Arduino phone and Node-Red is a great way to get jump started.

Node-Red on Android

Node-Red is a graphical programming system that is used for Internet of Things (IoT) projects. Node-Red is installed in the base Raspberry Pi images, but it can also be loaded on Linux, Windows, MacOS and Android systems.

To load Node-Red on Arduino, you will first need to load Termux,  an Android Terminal Emulator app, that is available at Google Play. After Termux is loaded, enter the following commands to install  and run Node-Red:

apt update
apt upgrade
apt install coreutils nano nodejs
npm i -g --unsafe-perm node-red
node-red

Node-Red will start a Web graphical interface that is accessed by:  http://the_ip_address:1880.

nodered_pc

Extra features can be added and removed from Node-Red by selecting the “Manage Palette” menu option. For this project I needed terminux-api for the texting support, and bigtimer to scanning the text message buffer.

manage_palette

A Test Setup

For a basic setup I used:

  • 1 Android phone with Termux and Node-Red
  • 1 Raspberry Pi with Node-Red
  • 1 Powerswitch Tail II connected to a light
  • 1 Android phone for texting in

sms_pi_setup

Scanning for Text Messages

To create a simple text message application on the Android Note-Red system, the following components are used:

  • A Big Timer node is used to define how often the SMS inbox is scanned. Without doing any configuration, the second output will offer a 1/minute cycle time.
  • An SMS Inbox node will read in a defined number of text messages. To get the last message select the buffer limit to be 1.
  • A function node is used with some JavaScript to check for valid text messages
  • A TCP out node will send a message to another Node-Red system.

A basic logic setup will be as follows:

sms_in_logic

The function node needs some logic to interpret the latest text message and send out the required logic to the Raspberry Pi GPIO pins. The example logic uses a text message of  “LIGHTS ON” or “LIGHTS OFF” to control the output on GPIO pin 12.


// look for new action texts
smslast = context.get('smslast')|| 0;

// Do an action if there is a new SMS message
if ( msg.payload[0].received != smslast) {
    context.set('smslast', msg.payload[0].received )
    smsbody = msg.payload[0].body;
    if (smsbody.toUpperCase().trim() == "LIGHTS ON") {
        msg.payload = 1;  // this is sent to the Pi via TCP
        return msg;
    }
    if (smsbody.toUpperCase().trim() == "LIGHTS OFF") {
        msg.payload = 0;  // this is sent to the Pi via TCP
        return msg;
    }
}

The first time that the code runs you will be prompted on the phone to “Allow Termux:API to send and view SMS messages?”. After you acknowledge this message your Node-Red logic will run cleanly.

Reading TCP Inputs

Android phones can not be directly connected I/O devices, so TCP connections can used to pass commands to Raspberry Pi’s which have the I/O support. Node-Red is pre-installed on most Raspberry Pi images so no added installation is required.

On the Raspberry Pi Node-Red side only a couple of nodes are required:

  • A TCP in node is used to read the incoming communications. For the configuration of this node it is important to have the port (8888 in this case) match the TCP out node’s port. Also the output should be set as: single, and string.
  • An RPI GPIO out node is used set Raspberry Pi General Purpose (GPIO) pins.

sms_tcpin

Texting Out from Android

If your text scheme is simple, for example one light that is turned on and off, then you could manually just output an “ON” or “OFF” message. However if you are controlling multiple lights, heaters and other devices then manually typing the text messages gets awkward.

To manage multiple commands, I wrote a simple Android app in MIT’s App Inventor. App Inventor is a free online Android development environment, that only requires a Google login for access. In about 10 minutes I was able to get a prototype going with multiple inputs. The first step is to drag and drop some buttons from the User Interface Palette onto the viewer pane. Then drag and drop a Texting component from the Social Palette onto the view. Note, the Texting component will appear as a non-visible item.

text2red_gui

After you have a basic layout, select the Blocks button on the menu bar, and this will open a logic page. Logic is created by clicking on the objects in the Block panel, and then dragging and dropping the block onto the Viewer panel.

To have a button sent a text message the following pieces are required:

  • “when Button.Click”  will be called when the users touches the button object
  • “set Texting1.PhoneNumber”  defines the remote phone number
  • “set Texting1.Message” defines the text message
  • “call Texting.SendMessage”  will send the message

text2red_blocks

To build the application use the Build menu item on the menu bar.

app2qr

For my test project I configured four devices and eight commands.text2sms_app

Final Comments

I found that using Node-Red on Android to be a lot faster than I expected, however I noticed that some of the added features (like Bluetooth support) only worked on the Raspberry Pi/Linux version of the Node-Red.

For a final solution I would definitely move to dedicated SMS hardware, but I found it nice to be able to do proof of concept testing with just some basic Android phones. Also don’t forget to setup Node-Red for automatic startup on a power up.

Arduino talking MQTT to Node-Red

There are some great Arduino modules with integrated ESP-8266 wireless chips, some of the more popular modules are:

  • Adafruit HUZZAH
  • NodeMCU
  • WeMos

These modules allow you to do some interesting IoT (Internet of Things) projects. To connect the Arduino modules to PCs, Raspberry Pi’s or Linux nodes that are a number of communication choices. MQTT (Message Queue Telemetry Transport) is becoming one of the standards for this and it is pre-installed with Node-Red.

Plant Moisture Monitoring MQTT Example

Ard_mqtt_overview2

For our example we wanted to do a simple plant moisture example that used a solar charger and an Arduino Wemos module. We then setup an MQTT server on our Node-Red Raspberry Pi with a web dashboard.

Our goal was to get the MQTT technologies working, with some moisture inputs (and not a final plant monitoring system).

Moisture Sensors

Moisture sensors are very low cost and they start at about $2. The basic moisture sensor has 3 inputs; VCC, GND, and AO. Some sensors also include a digital output with a potentiometer to adjust the digital 0-1 moisture limit.

Our Arduino plant moisture setup is good for testing but not a good long term solution. When voltage is applied long term to moisture sensors ionization in the soil will cause a combination of false reading and deterioration of the sensor plates. We plan to do a future project where we will use relays to turn the sensors on/off and we will include solenoid values in a watering system.

MQTT on Arduino

There are a number of excellent MQTT libraries for Arduino, for this example we used the PubSubClient library. This library can be installed from the Arduino IDE by selecting the menu items:

Sketch -> Add Library -> Manage Libraries

To get MQTT setup you’ll need to:

  • define the SSID and password for your WAN
  • define the IP address for the MQTT server (the Node Red/Raspberry Pi node)
  • define some topic for the data

The nice thing about MQTT is that you can define topics for each of your data points. For this example we define the topic humidity to show the moisture sensor value, and msgtext to show the message (‘Needs Water’ or ‘).

Below is our sample Arduino code for passing the moisture data to our MQTT server.

/*
 Basic ESP8266 MQTT publish client example for a moisture sensor
*/
#include <ESP8266WiFi.h>
#include <PubSubClient.h>

// Update these with values suitable for your network.
const char* ssid = "YOUR_SSID_NAME";
const char* password = "YOUR_PASSWORD";
const char* mqtt_server = "YOUR_NODE_RED_IP";

WiFiClient espClient;
PubSubClient client(espClient);

void setup_wifi() {
  // Connecting to a WiFi network
  WiFi.begin(ssid, password);
  while (WiFi.status() != WL_CONNECTED) {
    delay(500);
    Serial.print(".");
  }
  Serial.println("WiFi connected");
  Serial.println("IP address: ");
  Serial.println(WiFi.localIP());
}

void reconnect() {
  // Loop until we're reconnected
  Serial.println("In reconnect...");
  while (!client.connected()) {
    Serial.print("Attempting MQTT connection...");
    // Attempt to connect
    if (client.connect("Arduino_Moisture")) {
      Serial.println("connected");
    } else {
      Serial.print("failed, rc=");
      Serial.print(client.state());
      Serial.println(" try again in 5 seconds");
      delay(5000);
    }
  }
}

void setup() {
  Serial.begin(9600);
  setup_wifi();
  client.setServer(mqtt_server, 1883);
}

void loop() {
  char msg[10];
  char msgtext[25];
  String themsg;
  if (!client.connected()) {
    reconnect();
  }
  
  int soil_moisture=analogRead(A0);  // read from analog pin A0
  Serial.print("analog value: ");
  Serial.println(soil_moisture);
  
  if((soil_moisture>300)&&(soil_moisture<700)) {
    Serial.println("Humid soil");
    sprintf(msgtext,"Humid soil",soil_moisture);
  } 
  else if ((soil_moisture>700)&&(soil_moisture<950)){
    Serial.println("Moist Soil");
    sprintf(msgtext,"Moist Soil",soil_moisture);
  }
  else if (soil_moisture <300) ){
    Serial.println("Needs water");    
    sprintf(msgtext,"Needs water",soil_moisture);
  }
  else
  {
      sprintf(msgtext,"Sensor Problem",soil_moisture);
  }

  sprintf(msg,"%i",soil_moisture);
  client.publish("humidity", msg);
  client.publish("soil", msgtext);
  delay(5000);
}

Node-Red

Node-Red is an excellent visual programming environment that is part of the Raspberry Pi base install. Node-Red is a simple tool to create your own Internet of Things applications. The base Node-Red installation includes MQTT interfacing components but it does not include an MQTT server.

If you don’t have a Raspberry Pi you can install Node-Red on Window, Mac OS or Linux systems. I’ve had good results running Node-Red on a very old low end laptop running Xubuntu, (see steps 1-3 in this linked guide).

MQTT Server on Node-Red

There are a number of free internet MQTT servers (iot.eclipse.org) that can be used or an MQTT server can be loaded directly on a local server (i.e. Mosquito).

Mosca is a standalone MQTT server that can be installed directly into Node-Red. The Mosca Node-Red component can be either installed at the command line by:

cd $HOME/.node-red

npm install node-red-contrib-mqtt-broker

Or the component can be install within the Node-Red web interface by selecting the “manage palette” option, and then search for mosca.

mosca_install

After Mosca is installed, all that is required is that a “mosca” node needs to be dragged and dropped into the Node-Red project.

mosca_mqtt_nodes

To connect the Arduino module to Node-Red mqtt inputs are added to the project.  The Arduino topics are defined in Node-Red by double-clicking on the mqtt node and then define the topic to match the Arduino topic.

mqtt_topic

After the MQTT connections are configured Web dashboards can present the final data. The Web dashboards offer a number of different components that could be used for this example I used a gauge and a text node.

To compile and view the Node-Red application, click on the Deploy button on the right side of the menu bar. To access the web dashboard enter: http://your-ip:1880/ui . Below is a example of what you should see.

NodeRed_MQTT_GUI

Final Thoughts

The Mosca MQTT server component allows for a simple standalone architecture to connect wireless Arduino modules into a Node-Red IoT solution.

We can also load Node-Red on Raspberry Pi data collection nodes, and have them publish data to a central Node-Red server.

 

 

 

Arduino talking TCP to Node-Red and Python

There are some great Arduino modules with integrated ESP-8266 wireless chips, some of the most popular of these modules are:

  • Adafruit HUZZAH
  • NodeMCU
  • WeMos

Along with these modules comes some excellent libraries.

For Arduino to PC or Raspberry Pi communications that are a few options to choose from. A TCP client/server is simple and straightforward and it is excellent for sending single point information. For sending multiple data points take a look at MQTT (Message Queuing Telemetry Transport), it’s a common standard for IoT applications and it’s built into Node-Red.

Arduino TCP Client

The Arduino module can be a simple TCP client that can talk to either a Python or a Node-Red TCP server. Below is an example that sends a random integer to a TCP server every 5 seconds.

/*
Test TCP client to send a random number
 */
#include <ESP8266WiFi.h>
#include <ESP8266WiFiMulti.h>

ESP8266WiFiMulti WiFiMulti;

void setup() {
    Serial.begin(9600);

    // We start by connecting to a WiFi network
    const char * ssid = "your_ssid";       // your WLAN ssid
    const char * password = "your_password"; // your WLAN password
    WiFiMulti.addAP(ssid, password);

    Serial.println();
    Serial.print("Wait for WiFi... ");

    while(WiFiMulti.run() != WL_CONNECTED) {
        Serial.print(".");
        delay(500);
    }
    Serial.println("WiFi connected");
    Serial.println("IP address: ");
    Serial.println(WiFi.localIP());
    delay(500);
}

void loop() {
    const uint16_t port = 8888;          // port to use
    const char * host = "192.168.0.123"; // address of server
    String msg;

    // Use WiFiClient class to create TCP connections
    WiFiClient client;

    if (!client.connect(host, port)) {
        Serial.println("connection failed");
        Serial.println("wait 5 sec...");
        delay(5000);
        return;
    }
   
    // Send a random number to the TCP server
    msg = String(random(0,100));
    client.print(msg);
    Serial.print("Sent : ");
    Serial.println(msg);
    client.stop();    
    delay(5000);
}

Node-Red TCP Server

Node-Red is an excellent visual programming environment that is part of the Raspberry Pi base install. Node-Red is a simple tool to create your own Internet of Things applications. The base Node-Red installation includes a TCP server and client.

To install the web dashboards enter:

sudo apt-get install npm
cd ~/.node-red
npm i node-red-dashboard

To start Node-Red either use the on-screen menus or from the command line enter:

node-red-start &

Once Node-Red is running the programming is done via the web interface at: //localhost:1880 or //your_Pi_ip_address:1880 .

To configure the TCP server, go to the Input nodes section and drag and drop the TCP in node. After the node is inserted double-click on it and edit the port and message settings.

nodered_tcp

To create a gauge Web dashboard, go to the dashboard nodes section and drag and drop the gauge node. After the node is inserted double-click on it and edit the dashboard group, labels and ranges.

nodered_tcp_gauge

For debugging and testing an output debug node is useful.

To access the Web dashboard enter: //localhost:1880/ui or //your_Pi_ip_address:1880/ui

TCP Python Server

The python TCP server will see the incoming Arduino message as a Unicode (UTF-8) text, so convert message to an integer use: thevalue = int(data.decode(“utf-8”)). Below is the full code.

import socket
import sys

HOST = '' # Symbolic name, meaning all available interfaces
PORT = 8888 # Arbitrary non-privileged port

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
print ('Socket created')

#Bind socket to local host and port
try:
s.bind((HOST, PORT))
except socket.error as msg:
print ('Bind failed. Error Code : ' + str(msg[0]) + ' Message ' + msg[1])
sys.exit()

print ('Socket bind complete')

#Start listening on socket
s.listen(10)
print ('Socket now listening')

#now keep talking with the client
while True:
#wait to accept a connection - blocking call
conn, addr = s.accept()
data = conn.recv(1024)
print ('Connected with ' + addr[0] + ':' + str(addr[1]) + " " )
thevalue = int(data.decode("utf-8"))
print ("Value: ", thevalue)

s.close(

Summary

In our final application we mounted the Arduino module outside and we powered it with a small solar charger. We also include a humidity value with the temperature, and we used a QR code that linked to our web page.

Wake Up !!! Wake Up !!!

The goal of the “Wake me up” programs is to set a wake time that will turn on the lights and say something like: “Wake Up Wake Up”.  The logic was done in Node-Red and it was quite straightforward. The hardware used was:

The PowerSwitch Tail II ($26) is a power cord that is enabled/disabled with I/O pins. The PowerSwitch pins connect to GPIO18 and GND on the Pi. A desk light is plugged into the PowerSwitch Tail and speakers are connected to audio jack on the Pi.

pi2switch

With the PowerSwitch Tail II there are a lot of devices that could be controlled. For us, we simply connected it to a desk lamp. The setup shown below was later moved into a bedroom and place on a bedside table.

wakeup_hwd

For this project two Node-Red libraries were used; a scheduler library and a text to speech library. They are installed by:

 sudo apt-get install festival
cd $HOME/.node-red
npm install node-red-contrib-say
npm install node-red-contrib-simple-weekly-scheduler 

After the packages are loaded restart the Pi, and then start up Node-Red. On the Node-Red configuration Web page, drop a scheduler, and wire a Pi GPIO, and a Say node to the scheduler.

3nodes

Double-click the scheduler node and set the wake up times. The start/end payloads are numeric and 1/0. Below is the scheduler configuration.

schedule

For the Pi GPIO node, set the GPIO to Pin 12/GPIO18.

powerinfo

Next for the Say node, enter the text you want spoken .

wakeup

For the final circuit, inject nodes can be included for testing. An “on” inject sends a 1, and an “off” inject sends 0. At the wake up time the schedule node will send a payload that will trigger the speaking of the wake up text and the desk light will be turned on for 15 minutes.

final_wakeup

Pi/Node-Red Car

The goal of the Pi/Node-Red car project was to create a small vehicle that can be controlled from a smart phone . For the project we used:

  • 1 Car chassis for Arduino ($15)
  • 1 Pimoroni Explorer HAT Pro  ($23)
  • 1 Portable microUSB charger
  • 1 USB WiFi Adapter
  • 4 short alligator clips and 4 connectors
  • Duct tape

The Arduino car chassis may require a small amount of assembly. Rather than soldering connections we like to use short alligator clips. It is not recommended to wire DC motors directly to a Raspberry Pi so the Pimoroni Explorer HAT Pro is used to connect the 2 DC motors.

The Raspberry Pi and the portable microUSB charger are secured to the top of the car chassis with duct tape. The left motor is wired to the motor 1 connectors on the Explorer Hat, and the right motor is wired to motor 2 connectors. Note you may have to do a little trial and error on the Explorer HAT “+” and “-” motor connections to get both wheels spinning in a forward direction.

The Explorer HAT Node-Red library is installed by:

 cd $HOME/.node-red
npm install node-red-dashboard 

The Web dashboard presentation is configured in the “dashboard” tab. For this example we create 2 groups: a control group to drive the vehicle, and a light group to turn on the Explorer Pro lights. Use the “+group” button to add a group, and the “edit” to change an existing group.
dash_conf

To control a motor, an “Explorer HAT” node and a dashboard button node are dropped and connected together. All the configuration is done in the button node . The button configure options are:

  • the group the button will appear in (Controls)
  • the size of the button (3×1 = 50% of width and narrow)
  • Topic, motor.one or motor.twois used for motor control
  • Payload, -100 = reverse, 0=stop, 100 = forward

Control_conf

The Explorer HAT has 4 colored LEDs. To toggle the LEDS, the topic is light.color with 1=ON, and 0=OFF . We thought that it would be fun to also add some Web dashboard button to control the colored lights.

light_conf

The Node-Red dashboard user interface is accessed by: ipaddress:1880/UI, so for example 192.168.1.102:1880/ui. Below is a picture that shows the final vehicle logic and the Web dashboard.

 

final_logic2