MQTT and Javascript

MQTT (Message Queuing Telemetry Transport) is a  publish-subscribe-based messaging protocol that is used on many Internet of Things (IoT) projects. It works on top of the TCP/IP protocol and it is designed for connections with remote locations where a “small code footprint” is required or the network bandwidth is limited. The publish-subscribe messaging pattern requires a message broker.

MQTT_js_overview

There is support for MQTT on a large variety of programming languages and platforms. An Arduino or Raspberry Pi module can sent (or publish) I/O to a MQTT broker, and they can also receive (or subscribe) to data.

There are a number of MQTT brokers that can be used. One of the most popular ones is the Mosquitto MQTT broker, and it can be loaded on Windows, OSX and Linux systems. For this blog we will be using the Mosquitto test MQTT server. This Internet based server should not be used for real systems, but it is excellent for small short terms tests.

MQTT Web Sockets

The MQTT server has configurable MQTT and Web Socket ports. For a typical Raspberry Pi or Arduino connection, the default MQTT port 1883 would be used. In many Internet applications only certain ports are open, so for this reason a different MQTT Web Socket is used. This is configurable but ports like 80 or 8080 are typically used.MQTT_web_layout

Javascript Application

There are a number of MQTT javascript libraries that are available. One of the more popular ones is the Paho library that is available at:

https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.1/mqttws31.js

As a first example we will look at creating 2 pages. The first page will  publish a value, and the second page will subscribe to the data.

MQTT_js_pubsub

The publish code is:

<html>
<head>
<title> MQTT Publish Message</title>
</head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.1/mqttws31.js" type="text/javascript"></script>

<script>
// Define some MQTT variables
var mqtt_server = "";
var mqtt_port = "";
var mqtt_destname = "";

function send_mqtt_msg() {
// Send an MQTT message
  mqtt_server = document.getElementById("mqtt_server").value;
  mqtt_port = Number(document.getElementById("mqtt_port").value);
 

  client = new Paho.MQTT.Client(mqtt_server, mqtt_port,"");
  client.connect({onSuccess:onConnect});
  document.getElementById("pubmsg").innerHTML = "Trying to connect...
";
}
function onConnect() {
  document.getElementById("pubmsg").innerHTML = "New connection made...
";
  var mqtt_destname = document.getElementById("mqtt_destname").value;
  var mqtt_msg = document.getElementById("mqtt_msg").value;   
  message = new Paho.MQTT.Message(mqtt_msg);
  message.destinationName = mqtt_destname;
  client.send(message);
  document.getElementById("pubmsg").innerHTML = "topic:" + mqtt_destname + " = " + mqtt_msg + " ...sent";
}  
// called when a message arrives
</script>
<body>
<h1>MQTT Publish Test Page</h1>

Server Name: <input type="text" id="mqtt_server" value="test.mosquitto.org"><br><br>
Websocket: <input type="text" id="mqtt_port" value="8080"><br><br>
DestinationName: <input type="text" id="mqtt_destname" value="my_IoT_value1"><br><br>
Message: <input type="text" id="mqtt_msg" value="test message"><br><br>

<button onclick="send_mqtt_msg()">Publish MQTT Message</button>
</body>
<hr>
<div id=pubmsg></div>
</html>

The subscribe code is:

<html>
<head>
<title> MQTT Subscribe Message</title>
</head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.1/mqttws31.js" type="text/javascript"></script>
<script>

function sub_mqtt_msg() {
// Send an MQTT message
  var mqtt_server = document.getElementById("mqtt_server").value;
  var mqtt_port = Number(document.getElementById("mqtt_port").value);

  client = new Paho.MQTT.Client(mqtt_server, mqtt_port,"");
  client.onMessageArrived = onMessageArrived;
  client.onMessageArrived = onMessageArrived;
  client.connect({onSuccess:onConnect});
  document.getElementById("submsg").innerHTML = "Trying to connect...
";

}
function onConnect() {
  document.getElementById("submsg").innerHTML = "New connection made...
";
  var mqtt_destname = document.getElementById("mqtt_destname").value;  
  client.subscribe(mqtt_destname);
  document.getElementById("submsg").innerHTML = "Subscribing to topic: " + mqtt_destname + " ...
";
}
function onMessageArrived(message) {
  var result = message.destinationName + " : " + message.payloadString + "
";
  document.getElementById("submsg").innerHTML = result;
}

</script>
<body>
<h1>MQTT Subscribe Test Page</h1>

Server Name: <input type="text" id="mqtt_server" value="test.mosquitto.org"><br><br>
Websocket: <input type="text" id="mqtt_port" value="8080"><br><br>
DestinationName: <input type="text" id="mqtt_destname" value="my_IoT_value1"><br><br>

<button onclick="sub_mqtt_msg()">Subscript to MQTT</button>
<hr>
<h2>Subscribed Messages:</h2>
<div id=submsg></div>
</body>

</html>

Once you’ve got the basics down it’s possible to make some more advanced web interface pages.

Chart and Gauges

For your IoT projects there are a lot of Dash board options that are available. One of my favorites is Node-Red because it is totally free and standalone.

If you would like to create your own web interface there are a number of good javascript charting and gauge libraries available. For my examples I used Google charts with the Gauge chart . To simulate the inputs I used three of our MQTT Publish pages.

MQTT_js_sim

The code for the gauges page is :

<html>
<head>
<title>IoT - MQTT to JS</title>
</head>
<script src="https://cdnjs.cloudflare.com/ajax/libs/paho-mqtt/1.0.1/mqttws31.js" type="text/javascript"></script>
<script src="https://www.gstatic.com/charts/loader.js" type="text/javascript"></script>

<script>
// MQTT variables
var MQTTnames = ["my_IoT_value1","my_IoT_value2","my_IoT_value3"];
var MQTTvalues = [0,0,0];


// Define the Google gauge chart
      google.charts.load('current', {'packages':['gauge']});
      google.charts.setOnLoadCallback(drawChart);

      function drawChart() {

        var data = google.visualization.arrayToDataTable([
          ['Label', 'Value'], 
      [MQTTnames[0], MQTTvalues[0]],
      [MQTTnames[1], MQTTvalues[1]],
      [MQTTnames[2], MQTTvalues[2]],      
        ]);

        var options = {
          width: 800, height: 1000,
          redFrom: 90, redTo: 100,
          yellowFrom:75, yellowTo: 90,
          minorTicks: 5
        };

        var chart = new google.visualization.Gauge(document.getElementById('chart_div'));

        chart.draw(data, options);

        setInterval(function() {
      for (var i=0; i < MQTTnames.length; i++) {
      data.setValue(i, 1, MQTTvalues[i]);
      }
          chart.draw(data, options);

      }, 1000);
}

// Create a client instance
client = new Paho.MQTT.Client("test.mosquitto.org", 8080,"");
client.onMessageArrived = onMessageArrived;

// connect the client
client.connect({onSuccess:onConnect});

// called when the client connects
function onMessageArrived(message) {
  
  for (var i=0; i < MQTTnames.length; i++) {
  if (message.destinationName == MQTTnames[i]) {
    MQTTvalues[i] = Number(message.payloadString);
  }
  }
}
function onConnect() {
  // Once a connection has been made, make a subscription and send a message.
  for (var i=0; i < MQTTnames.length; i++) {
  client.subscribe(MQTTnames[i]);
  }
}
// called when a message arrives
</script>
<body>
<h1  style = 'font-size: xx-large'>IoT - MQTT to JavaScript</h1>
<div id="chart_div" style="width: 100%; height: 100%;"></div>
</body>
</html>

There are many other charting options that could be used. Below is an example using the Google Chart library with bars instead of gauges.

MQTT_js_bars.png

Final Comment

Using Javascript to interact with your IoT projects open up a lot of potential for adding functionality. I found that the Python version of the Paho MQTT library to have better documentation and perhaps some more functions, but at the end of the day I was able to do all the I wanted.

IoT with Google Firebase

For many Internet of Things (IoT) projects a message queuing system like MQTT (Message Queue Telemetry Transport) is all that is needed to connect sensors, devices and graphic interfaces together. If however you require a database, with sorting, queuing and multi-media support then there are some great cloud storage platforms that are available. One that is definitely worth taking a look at is Google Firebase. Firebase is well documented and it has a free option (that does not require a credit card for activation) so it’s easy to start prototyping and testing.

In this article we will look at:

  1. Setting up a sample Firebase IoT project
  2. Use Python to simulate I/O data
  3. Create a Web dashboard with Node-Red to view and write data
  4. Create an Android app using AppInventor to view and write data
  5. Look at a more complex data monitoring example in Python

Getting Started

To begin log in with your Google account at https://firebase.google.com/ , and select “Get Started”. For this example we will create a sample project called My-IoT.  After the project is created Firebase will give it a unique identifier name, such as: my-iot-4ded7 .

Firebase data can be stored in either a Realtime Database, or a Cloud Firestore. The Realtime Database is an unstructured NoSQL format that has been around for about 4 years, so there are lots of 3rd party components. The Cloud Firestore stores the data in structure collections and it’s in beta so the 3rd party options are a little more limited.

For our test IoT project we will use the  real-time database, and it can be created from the Database -> Data menu option.

rtdb

The database structure can be built directly from the Firebase web console or imported from a JSON file. For this database 3 grouping are defined, one for Android, Node-Red and Raspberry Pi I/O values. The Firebase console allows for setting, changing and viewing the database values.

db_struct

The security is configured in the Database -> Rules menu option. To keep things simple for testing read and write security is set to true for public access,  (remember to change this for a production system).

security

The project’s remote access settings are shown in the Authentication section with the “Web setup” button.

webconfig

Python and Firebase

There are a number of Python libraries to access Firebase. The pyrebase library has some added functionality such as: queries, sorting, file downloads and streaming support. The pyrebase library only support Python 3, and it is installed by:

sudo pip3 install pyrebase
sudo pip3 install --upgrade google-auth-oauthlib

A Python 3 test program to set two values (pi/pi_value1 and pi/pi_value2) and read a single point and a group of point is shown below. Remember to change the configuration settings to match your project settings.

import pyrebase, random

# define the Firebase as per your settings
config = {
  "apiKey": "AIzaSyCq_WytLdmOy1AIzaSyCq_WytLdmOy1",
  "authDomain": "my-iot-4ded7.firebaseapp.com",
  "databaseURL": "https://my-iot-4ded7.firebaseio.com",
  "storageBucket": "my-iot-4ded7.appspot.com"
}

firebase = pyrebase.initialize_app(config)
db = firebase.database()

# set 2 values with random numbers
db.child("pi").child("pi_value1").set(random.randint(0,100))
db.child("pi").child("pi_value2").set(random.randint(0,100))

# readback a single value
thevalue = db.child("pi").child("pi_value1").get().val()
print ("Pi Value 1: ", thevalue)

# get all android values
all_users = db.child("android").get()
for user in all_users.each():
    print(user.key(), user.val())

The Firebase web console can be used to check the results from the test program.

Node-Red and Firebase

Node-Red is a visual programming environment that allows users to create applications by dragging and dropping nodes on the screen. Logic flows are then created by connecting the different nodes together.

Node-Red has been preinstalled on Raspbian Jesse since the November 2015. Node-Red can also be installed on Windows, Linux and OSX.  To install and run Node-Red on your specific system see https://nodered.org/docs/getting-started/installation.

To install the Firebase components, select the Manage palette option from the right side of the menu bar. Then search for “firebase” and install node-red-contrib-firebase.

nr_fb_install

For our Node-Red example we will use web gauges to show the values of our two Python simulated test points. We will also send a text comment back to our Firebase database. The complete Node-Red logic for this is done in only 6 nodes!

nr_logic

To read the Pi values two firebase.on() nodes are used. The output from these nodes is connected to two dashboard gauge nodes. Double-clicking on firebase_on() node to configure Firebase database and the item to read from. Double-clicking on the gauge node allows you to edit gauge properties.

nr_value1 To send a text string to Firebase a text input node is wired to a firebase modify node. Edit the firebase modify node with the correct database address and value to set.

nr_modify

After the logic is complete, hit the Deploy button on the right side of the menu bar to run the logic.  The Node-Red dashboard user interface is accessed by: http://ipaddress:1880/ui, so for example 192.168.1.108:1880/ui.

nr_screen

AppInventor

AppInventor is a Web based Android app creation tool (http://appinventor.mit.edu), that uses a graphical programming environment.

AppInventor has two main screens. The Designer screen is used for the layout or presentation of the Android app and the Blocks screen is used to build the logic. On the right side of the top menu bar, the Designer and Blocks buttons allow you to toggle between these two screens.

On the Designer screen, an app layout is created by dragging a component from the Palette window onto the Viewer window.

We will use AppInventor to create an Android app that will read Pi values from our Firebase IoT database, and write a value back.

For the visuals on this application a Button, Label (2),  ListView, Textbox and FirebaseDB component will be used. After a component is added to the application, the Components window is used to rename or delete that component. The Properties window is used to edit the features on a component. For the FirebaseDB component configure the Token and URL to match with your project settings.

fb_app_setup

Once the layout design is complete, logic can be added by clicking on the Blocks button on the top menu bar.

Logic is built by selecting an object in the Blocks window, and then click on the specific block that you would like to use.

AppInventor is pretty amazing when it comes to doing some very quick prototyping. Our entire app only uses two main blocks.

The when FirebaseDB1.DataChanged block is executed whenever new data arrives into the Firebase database. The DataChanged block returns a tag and a value variable. The tag variable is the top level item name, (“pi” or “nodered” for our example). The value will be a string of the items and their values, for example: “{pi_value2 = 34, pi_value1=77}”. We use a replace all text block to remove the “{” and “}” characters, then we pass the string to the ListView component. Note this same code will pass 2 or 200 tags in the pi tag section.

The when BT_PUT.click block will pass the text that we enter on the screen into the android/and_value1 item in our database.

fb_app_code

After the screen layout and logic is complete, the menu item Build will compile the app. The app can be made available as an APK downloadable file or as a QR code link.

Below is picture of the final Android app synced with the Firebase data.

Firebase2

Python Data Monitoring Example

Our starting IoT Firebase realtime database was quite simple. Data monitoring projects usually require more information than a just a value. Below is a more realistic sensor database with fields like tag name, description, status, time and unit.

fb_db2

By adding an .indexOn rule to the tags in the pi section of the database it is possible to create queries and sorts.

fb_rules2

A Python example to query the database and find all point in alarm would be:

import pyrebase, random

# define the Firebase as per your settings
config = {
  "apiKey": "AIzaSyCq_AIzaSyCq_Wyt",
  "authDomain": "my-iot-7ded7.firebaseapp.com",
  "databaseURL": "https://my-iot-7ded7.firebaseio.com",
  "storageBucket": "my-iot-7ded7.appspot.com"
}

firebase = pyrebase.initialize_app(config)
db = firebase.database()

tag_sum = db.child("pi").order_by_child("status").equal_to("ALARM").get()

# print alarm points
for tag in tag_sum.each():
    info = tag.val()
    print (tag.key()," - ", info["description"],":", info["value"],info["units"], info["status"])

 

For more Python example see the Pyrebase documentation.

Summary

Without a lot of configuration it is possible to configure a Google Firebase project and have both Python , Node-Red and Android app read and write values.

 

Pi Network Monitoring

There are some great full featured networking packages like Nagios and MRTG that can be loaded on Raspberry Pi’s. If, however, you are looking for something smaller scale that you can play with then Node-Red might be your answer. Node-Red is a visual programming environment that allows users to create applications by dragging and dropping blocks (nodes) on the screen. Logic flows are then created by connecting wires between the different blocks (nodes). Node-Red also comes with Web Dashboards, so you can view data or do control from your smart phone.

In this blog we’ll look at:

  • running some SNMP commands
  • setup NodeRed for SNMP
  • making read/write SNMP values on the Pi

 

Getting Started with SNMP

Simple Network Management Protocol (SNMP) is the standard for communicating and monitoring of network devices. Common device information is grouped into MIBs or Management Information Bases. Data items are called OIDs or Object Identifiers. OIDs are referenced by either their MIB name or by their OID numeric name. So for example the SNMP device name object could be queried by either its MIB name of: SNMPv2-MIB::sysName.0 or the object identifier number of: .1.3.6.1.2.1.1.5.0.

To install both the SNMP monitor and server on your Pi enter:

sudo apt-get update
sudo apt-get install snmp snmpd snmp-mibs-downloader

To show meaningful MIB names, you will need to modify the SNMP config file by:

sudo nano /etc/snmp/snmp.conf

The first line should be commented out, it should just read: #mibs .

There are many configuration options in the SNMP server agent that need to be considered. For a real/product system you will need to consider your user security but for a test system we can open up the Pi by:

sudo nano /etc/snmp/snmpd.conf

Then uncomment the agentAddress line so that all interfaces are open, and in the Access Control section comment out all the existing user access and add a new line with public access set to read/write (definitely not recommended in a real system):

# Listen for connections on all interfaces (both IPv4 *and* IPv6)
agentAddress udp:161,udp6:[::1]:161

# ACCESS CONTROL
#
# Set read/write access to public anywhere
#
rwcommunity public

After saving the changes to snmpd.conf, the service needs to be restarted:

sudo service snmpd restart

There are a number of useful SNMP command line programs, such as:

  • snmpget – gets a SNMP message for a specific OID
  • snmpset – sets a SNMP OID (OID needs to be writeable)
  • snmpwalk – gets multiple OID values in a MIB tree

The basic syntax for these commands is:

command -c community -v version node OID

To test that SNMP is working, enter the following:

pi@raspberrypi:~ $ snmpwalk -c public -v 1 localhost .1.3 

SNMPv2-MIB::sysDescr.0 = STRING: Linux raspberrypi 4.4.21-v7+ ...
SNMPv2-MIB::sysObjectID.0 = OID: NET-SNMP-MIB::netSnmpAgentOIDs.10
DISMAN-EVENT-MIB::sysUpTimeInstance = Timeticks: (319234) 0:53:12.34
SNMPv2-MIB::sysContact.0 = STRING: Me <me@example.org>
SNMPv2-MIB::sysName.0 = STRING: raspberrypi
...

If SNMP is working correctly a very long list of SNMP objects will be shown.

Get a Specific SNMP Result

There are thousands of SNMP results that can be read. Some of the more common and useful SNMP are:

1 minute CPU Load: .1.3.6.1.4.1.2021.10.1.3.1
5 minute CPU Load: .1.3.6.1.4.1.2021.10.1.3.2
15 minute CPU Load: .1.3.6.1.4.1.2021.10.1.3.3

Idle CPU time (%): .1.3.6.1.4.1.2021.11.11.0

Total RAM in machine: .1.3.6.1.4.1.2021.4.5.0
Total RAM used: .1.3.6.1.4.1.2021.4.6.0
Total RAM Free: .1.3.6.1.4.1.2021.4.11.0

Total disk/partion size(kBytes): .1.3.6.1.4.1.2021.9.1.6.1
Available space on the disk: .1.3.6.1.4.1.2021.9.1.7.1
Used space on the disk: .1.3.6.1.4.1.2021.9.1.8.1

An example to get the Idle CPU time with the SNMP command line tool would be:

$ snmpget -c public -v 1 localhost .1.3.6.1.4.1.2021.11.11.0

UCD-SNMP-MIB::ssCpuIdle.0 = INTEGER: 97

If the syntax is correct, a result is returned with the OID identifier, result type and the  result value.

Getting Started with NodeRed

NodeRed is pre-installed with the Raspbian images but it will need to have some SNMP and some support options loaded. At a terminal window enter the commands:

sudo apt-get update
sudo apt-get install npm
cd $HOME/.node-red
npm install node-red-node-snmp
npm install node-red-dashboard
npm install node-red-contrib-bigtimer

node-red-start &

Once Node-RED starts, you use a web browser to build applications. If you are working directly on your Pi, enter 127.0.0.1:1880 in the URL address box of your browser. You drop palettes from the left pane into the large flow window in the middle and wire them together in the correct order.

NodeRed Ping Monitor

A good starting program is to make a Web Dashboard that shows ping (node-to-node) delay times. The dashboards are defined in the right panel of Node-Red. A dashboard items are put into groups, and groups are put into tabs. Each tab will be shown as a separate page on your smart phone.

pinglogic

By double-clicking on the Ping node you can enter the different IP address.

pingnode

Similarly by double-clicking on the Chart node, you can define the label and look and feel of the chart.

chartnode

After the configuration is finished, click the Deploy button,  (top right on menu  bar). The Node-RED dashboard user interface is accessed by entering <IPaddress>:1880/ui (e.g., 192.168.1.102:1880/ui). Chart data values are shown by clicking on the chart line.

pingSS

NodeRed with SNMP Nodes

The ping node is quite simple and it returns just the ping value. The snmp node is more complex and it returns multiple pieces of information. To use snmp nodes in a Node-Red program you need some support nodes to parse/pass the payload messages. To send SNMP data to a chart dashboard, the following nodes are used:

  • Big Timer – to trigger the polling of data
  • snmp – gets SNMP/OID information
  • split – split the message into addressable variables
  • change – put the OID value into the message playload
  • chart – show the payload

snmplogic

The SNMP node configuration can get multiple values, a simple example to get the CPU free time is below (Note: the leading “.” is NOT included) :

snmpnode

The split node is used move the SNMP array data in a payload string. The change node is then used to move just the value into the payload.

changenode

The final step is to configure the charts to show the correct label information.

snmpSS

An SNMP Readable Pi Value

The Net-SNMP agent (snmpd) supports the creation of custom read/write objects (OIDs). The “Pass-through” MIB extension command in snmpd.conf allows for script files to be called.

Pass-through script files need to follow a few rules:

  • snmpget request passes a “-g” parameter (get)
  • the snmpget response needs to be 3 lines: OID, data type, and value
  • an snmpset request passes a “-s” parameter (set)

A simple example would be to define the Rasperry Pi board temperature as an SNMP object. The board temp is available by:

cat /sys/class/thermal/thermal_zone0/temp
46160

This returns the value in 1/1000s of a degree C. So to get just the Celsius temperature we can use:

echo $(($(cat /sys/class/thermal/thermal_zone0/temp) / 1000))
46

A SNMP bash script (/home/pi/pitemp), with our Pi CPU temp as OID  .1.3.6.1.4.1.8072.2.1 would be:

#!/bin/bash
if [ "$1" = "-g" ]
then
echo .1.3.6.1.4.1.8072.2.1
echo integer
echo $(($(cat /sys/class/thermal/thermal_zone0/temp) / 1000))
fi
exit 0

After the file is saved remember to make it executable (chmod +x pitemp).

In the “Pass-through” section of  /etc/snmp/snmpd.conf  a line is added to reference the new OID, shell and command:

#
# "Pass-through" MIB extension command
#
pass .1.3.6.1.2.1.8072.2.1 /bin/bash /home/pi/pitemp

After snmpd.conf is updated the snmpd service needs to restarted (sudo service snmpd restart), then our Pi temp OID can be accessed by:

snmpget -c public -v 1 localhost .1.3.6.1.2.1.8072.2.1
NET-SNMP-EXAMPLES-MIB::netSnmpExampleScalars = INTEGER: 47

Pi Writable SNMP GPIO Value

My goal was to use SNMP to turn on and off powered devices, so for this I used a PowerSwitch Tail II, however simple low cost relays could also be used.

The PowerSwitch Tail II ($26) is a power cord that is enabled/disabled with I/O pins. The PowerSwitch pins connect to the Pi pins 6 and 12.

pi_setup

The gpio tool is used to read and write to GPIO pins.  GPIO 1 is made writable by:

gpio mode 1 out

The SNMP script (/home/pi/powerswitch) to read and write to GPIO pin 1 (physical pin 12) is:

#!/bin/bash
if [ "$1" = "-g" ]
then
echo .1.3.6.1.2.1.8072.2.2
echo integer
gpio read 1
fi

if [ "$1" = "-s" ]
then
gpio write 1 $4
fi

exit 0

This new script file needs to made executable by: chmod +x powerswitch. The powerswitch script file is referenced in the SNMP server configuration file (/etc/snmp/snmpd.conf ):

#
# "Pass-through" MIB extension command
#
pass .1.3.6.1.2.1.8072.2.1 /bin/bash /home/pi/pitemp
pass .1.3.6.1.2.1.8072.2.2 /bin/bash /home/pi/powerswitch

Once again the smnpd needs to be restarted. Our read/write actions can be tested by:

$ snmpget -c public -v 1 localhost .1.3.6.1.2.1.8072.2.2
SNMPv2-SMI::mib-2.8072.2.2 = INTEGER: 0
$ snmpset -c public -v 1 localhost .1.3.6.1.2.1.8072.2.2 i 1
SNMPv2-SMI::mib-2.8072.2.2 = INTEGER: 1
$ snmpget -c public -v 1 localhost .1.3.6.1.2.1.8072.2.2
SNMPv2-SMI::mib-2.8072.2.2 = INTEGER: 1

NodeRed Setting SNMP Values

The exec node can be used to call the snmpset command.  A example with an ON and OFF button is:

gpio_logic

The configuration for the exec node is:

exec_node

And the web dashboard is:

gpioSS

Summary

Network monitoring and SNMP is huge topic, hopefully this will give you a good start.

Pi Sailboat

My daughters and I have built a number of boat projects with an assortment of Arduino, ESP-8266, Bluetooth and RFI components. I believe that this version using a Raspberry Pi and NodeRed offers one of the simplest solutions. This sailboat used a basic catamaran design with a Raspberry Pi mounting inside a waterproof container. Using NodeRed dashboards you can control the sailboat’s rudder from a smart phone. The complete NodeRed logic consisted of only 6 nodes.

Building the Sailboat

There are a lot of different building materials that you could choose from. K’Nex construction pieces are lighter than either Lego or Meccano and they allow you to create reasonably large structures with a minimal number of pieces. If you do not have access to K’Nex pieces then popsicle sticks and some card board would offer a good low cost solution.

To build the sailboat we used:
• K’Nex building pieces
• 4 plastic bottles
• 1 small plastic container with a lid
• String
• Duct tape
• Garbage bag
• Low torque servo
• Raspberry Pi Zero W or Pi 3
• Small USB phone charger

The base of the sailboat was a rectangular structure with 16 down facing K’Nex pieces that allowed plastic bottles to be duct taped in place.

boat_bottom

A few K’Nex pieces were used to create a compartment for the servo, and wire was used to secure the servo in place. A rudder was built by screwing a small piece of wood into the servo arm.

servobox

A garbage bag was cut to the required size and taped to the mast. The boom had a swivel connection to the mast and guide ropes were connected to both the boom and mast.

sailboat_details

Servo and Rudder Setup

Only very low torque servos can connected directly to Rasberry Pi GPIO pins.

Pi_servo_wiring

An example of a low torque servo would be the TowerPro SG90 ($4) that has a torque of 25.00 oz-in (1.80 kg-cm). If you have larger torque servos you will need to either use a custom Raspberry Pi servo hat (there are some good ones on the market), or you will need to use a separate power and ground circuit for the servo.

The wiringPi tool gpio can be used to control the servo. This package is pre-install in the Raspbian image, or it can be manually installed by:

sudo apt-get install -y wiringpi

Servos typically want a pulse frequency of 50 Hz, and the Raspberry Pi PWM (Pulse Width Modulation) pins have a frequency of 19200 Hz, so some range definitions and scaling is required:

gpio -g mode 18 pwm #define pin 18 as the PWM pin
gpio pwm-ms #use 'mark space' mode 
gpio pwmc 192 # set freq as 19200
gpio pwmr 2000 # use a range of 2000

The gpio pwm commands are not persistent after a reboot. A simple solution for this is to put these commands in the Pi user login file of: $HOME/.bash_login.

After the pwm setup commands are run you need to do some manual testing to define your different rudder (servo) positions (Figure 6), such as “Hard Left”, “Hard Right”, “Easy Left”, “Easy Right” and “Straight”. The pwr timing numbers will vary based on your requirements and servo arm positioning, for our sailboat we used:

gpio -g pwm 18 200 #straight
gpio -g pwm 18 260 #hard left
gpio -g pwm 18 140 #hard right
gpio -g pwm 18 230 #easy left
gpio -g pwm 18 170 #easy right

servo_settings

NodeRed Logic and Dashboards

NodeRed is pre-installed on the Raspbian image, but it will need to be set to autostart on a Pi reboot:  sudo systemctl enable nodered.service

NodeRed has a web configuration interface that is accessed by: http://localhost:1880 or http://pi_ip_address:1880.

On the options button (far right), by selecting: View -> Dashboard , you can define and change the web dashboard layouts.

dashboard

To create logic, nodes are selected from the left node panel and dragged and dropped on to the center flow panel. Logic flow are then created by clicking and joining together different inputs and outputs on the nodes. If a dashboard node is dropped on the flow panel it will be added to the default web dashboard. The gpio -g pwm commands can be called using the exec node. The button dashboard node will pass the defined payload value, for example a “Hard Left” 260 is passed when the button is pushed. The button’s payload value will be appended to the exec command to make a complete gpio -g pwm servo position command.

nodered

Once you’ve completed your logic setup press the Deploy button on the top right to make your configuration live and ready to test.

The final step is to enable a smart phone or tablet to connect to the Raspberry Pi, this can be done by either making the Raspberry Pi a WiFi access point or by tethering the Pi to a cell phone. There are some great guides on how to setup a Raspberry Pi as an access point. For this project the simple tethering method was used. Once the Pi is tethered to a phone, the PI’s IP address can be obtained from the hotspot users list.

pi_address

The NodeRed dashboard is accessed on your phone by: http://pi_ip_address:1880/ui .

nodered_ui

Assuming that everything is connected correctly you should be able to control the sailboard with your phone.

Summary

Once you’ve mastered the basic NodeRed and sailboat construction other projects such as motor boats, iceboats, airboats are possible.

airboat

 

 

 

Streaming Video Server

There are some reasonably priced video surveillance products available on the market. If however you have a USB Web Cam and a Raspberry Pi (or equivalent or an old PC) you can make your own.

If you are working in the Unix world I would recommend motion , it’s super easy to setup and it’s got lots of added features if you want to turn them on. To install:

sudo apt-get install motion

Once you have motion installed you’ll need to tweek some of it’s parameters by:

sudo nano /etc/motion/motion.conf

The /etc/motion/motion.conf file contains a lot of cool parameters to tweek, some of the more important ones are:

# Image width (pixels). Valid range: Camera dependent, default: 352
width 800

# Image height (pixels). Valid range: Camera dependent, default: 288
height 600

# Maximum number of frames to be captured per second.
framerate 1

# Maximum framerate for stream streams (default: 1)
stream_maxrate 1

# Restrict stream connections to localhost only (default: on)
stream_localhost off

The speed of your hardware and network will determine how many frames per second you can use.

To run the video server enter:

sudo motion

The motion package has a built in web server that is accessed by: http://your_ip:8081

livevideo

Hardware Setup

There are lots of options for the mounting of your video server. I like to use Lego Pi cases about ($8) and then use some Lego to secure the rest of the components. Below is an example where I used a PIR (Pyroelectric Infrared) module ($3) to manually turn the motion software on/off, and I added a USB card reader to enable/disable the system.

OLYMPUS DIGITAL CAMERA
Homemade Security System

Performance and Tuning

Ideally it would be nice to run at 24 frames per second (or better) to get smooth video action. Unfortunately I found that the Raspberry Pi 3 would often freeze up at this refresh rate with a USB camera. The linux command line tool vmstat can be useful to show your CPU and I/O status. Below is some sample vmstat output:

$ vmstat # Pi 3 video server at 24 fps
procs -----------memory---------- ---swap-- -----io---- -system-- ------cpu-----
 r b swpd free buff cache si so bi bo in cs us sy id wa st
 1 0 0 593552 17808 269856 0 0 41 57 803 180 7 1 74 18 0

The id is the idle time (74%), which isn’t bad. The problem however id wa, waiting for I/O.

You will have to do some trial and error to get a refresh rate that works best for you.

Port Forwarding

With the basic setup you will be able to access the video server from your local LAN/WAN. If however you want to access your video server from the Internet then you will need to allow a connection through your router to your video server, this is called port forwarding.

Unfortunately the configuration of port forwarding will vary from manufacturer to manufacture, so please see the documentation for your specific devices.

 

Control with Texts (Node-Red on Android)

In places where Internet connections are not possible or too expensive,  SMS text messaging can be a simple approach for monitoring and controlling your remote systems. Many of the mobile providers offer IoT packages for low data throughput, where you’d be looking at spending $1 to $5  per month for 1-5 MB of data.  From the hardware standpoint there are many options such digital modem modules that come in either a Raspberry Pi top or a USB form factor.

If you are looking at doing some prototyping, using an Arduino phone and Node-Red is a great way to get jump started.

Node-Red on Android

Node-Red is a graphical programming system that is used for Internet of Things (IoT) projects. Node-Red is installed in the base Raspberry Pi images, but it can also be loaded on Linux, Windows, MacOS and Android systems.

To load Node-Red on Arduino, you will first need to load Termux,  an Android Terminal Emulator app, that is available at Google Play. After Termux is loaded, enter the following commands to install  and run Node-Red:

apt update
apt upgrade
apt install coreutils nano nodejs
npm i -g --unsafe-perm node-red
node-red

Node-Red will start a Web graphical interface that is accessed by:  http://the_ip_address:1880.

nodered_pc

Extra features can be added and removed from Node-Red by selecting the “Manage Palette” menu option. For this project I needed terminux-api for the texting support, and bigtimer to scanning the text message buffer.

manage_palette

A Test Setup

For a basic setup I used:

  • 1 Android phone with Termux and Node-Red
  • 1 Raspberry Pi with Node-Red
  • 1 Powerswitch Tail II connected to a light
  • 1 Android phone for texting in

sms_pi_setup

Scanning for Text Messages

To create a simple text message application on the Android Note-Red system, the following components are used:

  • A Big Timer node is used to define how often the SMS inbox is scanned. Without doing any configuration, the second output will offer a 1/minute cycle time.
  • An SMS Inbox node will read in a defined number of text messages. To get the last message select the buffer limit to be 1.
  • A function node is used with some JavaScript to check for valid text messages
  • A TCP out node will send a message to another Node-Red system.

A basic logic setup will be as follows:

sms_in_logic

The function node needs some logic to interpret the latest text message and send out the required logic to the Raspberry Pi GPIO pins. The example logic uses a text message of  “LIGHTS ON” or “LIGHTS OFF” to control the output on GPIO pin 12.


// look for new action texts
smslast = context.get('smslast')|| 0;

// Do an action if there is a new SMS message
if ( msg.payload[0].received != smslast) {
    context.set('smslast', msg.payload[0].received )
    smsbody = msg.payload[0].body;
    if (smsbody.toUpperCase().trim() == "LIGHTS ON") {
        msg.payload = 1;  // this is sent to the Pi via TCP
        return msg;
    }
    if (smsbody.toUpperCase().trim() == "LIGHTS OFF") {
        msg.payload = 0;  // this is sent to the Pi via TCP
        return msg;
    }
}

The first time that the code runs you will be prompted on the phone to “Allow Termux:API to send and view SMS messages?”. After you acknowledge this message your Node-Red logic will run cleanly.

Reading TCP Inputs

Android phones can not be directly connected I/O devices, so TCP connections can used to pass commands to Raspberry Pi’s which have the I/O support. Node-Red is pre-installed on most Raspberry Pi images so no added installation is required.

On the Raspberry Pi Node-Red side only a couple of nodes are required:

  • A TCP in node is used to read the incoming communications. For the configuration of this node it is important to have the port (8888 in this case) match the TCP out node’s port. Also the output should be set as: single, and string.
  • An RPI GPIO out node is used set Raspberry Pi General Purpose (GPIO) pins.

sms_tcpin

Texting Out from Android

If your text scheme is simple, for example one light that is turned on and off, then you could manually just output an “ON” or “OFF” message. However if you are controlling multiple lights, heaters and other devices then manually typing the text messages gets awkward.

To manage multiple commands, I wrote a simple Android app in MIT’s App Inventor. App Inventor is a free online Android development environment, that only requires a Google login for access. In about 10 minutes I was able to get a prototype going with multiple inputs. The first step is to drag and drop some buttons from the User Interface Palette onto the viewer pane. Then drag and drop a Texting component from the Social Palette onto the view. Note, the Texting component will appear as a non-visible item.

text2red_gui

After you have a basic layout, select the Blocks button on the menu bar, and this will open a logic page. Logic is created by clicking on the objects in the Block panel, and then dragging and dropping the block onto the Viewer panel.

To have a button sent a text message the following pieces are required:

  • “when Button.Click”  will be called when the users touches the button object
  • “set Texting1.PhoneNumber”  defines the remote phone number
  • “set Texting1.Message” defines the text message
  • “call Texting.SendMessage”  will send the message

text2red_blocks

To build the application use the Build menu item on the menu bar.

app2qr

For my test project I configured four devices and eight commands.text2sms_app

Final Comments

I found that using Node-Red on Android to be a lot faster than I expected, however I noticed that some of the added features (like Bluetooth support) only worked on the Raspberry Pi/Linux version of the Node-Red.

For a final solution I would definitely move to dedicated SMS hardware, but I found it nice to be able to do proof of concept testing with just some basic Android phones. Also don’t forget to setup Node-Red for automatic startup on a power up.

Control Raspberry Pi GPIO with Java

There is some excellent Java examples at http://pi4j.com/ , but when I tried to pull together some prototypes I had some issues with:

  • Installing the Java GPIO library on my PI 3
  • Graphic interface examples for PI
  • Because of super user requirements the apps only worked locally

This write-up will hopefully address these issues.

Getting Started

Java is pre-installed on the Raspian image, so you will only need to install the pi4j java library and this is done by:

curl -s get.pi4j.com | sudo bash

Depending on your Pi setup you might be all good to go. However if you have problems see the section Workarounds at the end of this blog.

Java offers a lot of graphic library options. I used the swing library because it is built in and simple.

Hardware Setup

For the hardware setup I used a Pimoroni Explorer Hat because it offers 5V safe inputs and outputs, however this is not required. I wired a button switch into GPIO_22, and an LED into GPIO_04.

Java_setup

Simple Digital Output Example

A simple digital output example would be:


import com.pi4j.io.gpio.*;

public class gpio_do {

 public static void main(String[] args) throws InterruptedException {

System.out.println("GPIO Digital Output Example ... started.");

// get a handle to the GPIO controller
 final GpioController gpio = GpioFactory.getInstance();

 // creating the pin object
 final GpioPinDigitalOutput pin = gpio.provisionDigitalOutputPin(RaspiPin.GPIO_22);

 // turn pin on and wait 2 seconds
 pin.high();
 System.out.println("Pin High");
 Thread.sleep(2000);

 // turn off and wait 2 seconds
 pin.low();
 System.out.println("Pin Low");
 Thread.sleep(2000);

// turn pin on for 2 second and then off
 System.out.println("Pin High for 2 seconds");
 pin.pulse(2000, true);

 // release the GPIO controller resources
 gpio.shutdown();
 }
}

To compile and run this example, you will need to reference the location of the pi4j class library, and this done by:

javac -classpath .:classes:/opt/pi4j/lib/'*' gpio_do.java
sudo java -classpath .:classes:/opt/pi4j/lib/'*'  gpio_do

It is important to note the supervisor rights are required to control the GPIO pins, so sudo is used when you run the java app.

Graphical Digital Output

java_gpio_out

The swing library has some basic labels and buttons that can be used to view and set a GPIO pin.


import java.awt.*;
import java.awt.event.*;
import javax.swing.*;

import com.pi4j.io.gpio.*;
import com.pi4j.io.gpio.event.*;

public class gpio_di_swg {

JFrame jtfMainFrame;
JLabel label1;
static JLabel labelin;
JPanel jplPanel;

public gpio_di_swg() {
jtfMainFrame = new JFrame("GPIO Input Demo");
label1 = new JLabel("Pin Status: ");
label1.setFont(new Font("Serif", Font.PLAIN, 24));
labelin = new JLabel("INPUT STUTUS ???? ");
labelin.setFont(new Font("Serif", Font.BOLD, 36));
jplPanel = new JPanel();

jplPanel.setLayout(new BorderLayout());
jplPanel.add(label1,BorderLayout.PAGE_START);
jplPanel.add(labelin,BorderLayout.LINE_START);

jtfMainFrame.getContentPane().add(jplPanel) ;
jtfMainFrame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
//jtfMainFrame.setSize(150, 50);
jtfMainFrame.pack();
jtfMainFrame.setVisible(true);

}

public static void main(String args[]) throws InterruptedException {
System.out.println("GPIO Digital Listen Example ... started.");

// create gpio controller and input button objects
final GpioController gpio = GpioFactory.getInstance();
final GpioPinDigitalInput myButton = gpio.provisionDigitalInputPin(RaspiPin.GPIO_04);
myButton.setShutdownOptions(true);

// create and register gpio pin listener
myButton.addListener(new GpioPinListenerDigital() {
@Override
public void handleGpioPinDigitalStateChangeEvent(GpioPinDigitalStateChangeEvent event) {
// display pin state on console
System.out.println(" --> GPIO PIN STATE CHANGE: " + event.getPin() + " = " + event.getState());
if (event.getState().toString() == "HIGH") {
labelin.setText("INPUT HIGH");
labelin.setForeground(Color.RED);
} else {
labelin.setText("INPUT LOW");
labelin.setForeground(Color.BLACK);
}
}

});

System.out.println(" ... complete the GPIO #04 circuit and see the listener feedback here in the console.");

// Set the look and feel to Java Swing Look
try {
UIManager.setLookAndFeel(UIManager
.getCrossPlatformLookAndFeelClassName());
} catch (Exception e) {
}
gpio_di_swg application = new gpio_di_swg();
}
}

Digital Inputs

java_gpio_input

 

The PI4J library has an event handler to catch digital state changes. An graphical example would be:


import java.awt.*;
import java.awt.event.*;
import javax.swing.*;

import com.pi4j.io.gpio.*;
import com.pi4j.io.gpio.event.*;

public class gpio_di_swg {

JFrame jtfMainFrame;
JLabel label1;
static JLabel labelin;
JPanel jplPanel;

public gpio_di_swg() {
jtfMainFrame = new JFrame("GPIO Input Demo");
label1 = new JLabel("Pin Status: ");
label1.setFont(new Font("Serif", Font.PLAIN, 24));
labelin = new JLabel("INPUT STUTUS ???? ");
labelin.setFont(new Font("Serif", Font.BOLD, 36));
jplPanel = new JPanel();

jplPanel.setLayout(new BorderLayout());
jplPanel.add(label1,BorderLayout.PAGE_START);
jplPanel.add(labelin,BorderLayout.LINE_START);

jtfMainFrame.getContentPane().add(jplPanel) ;
jtfMainFrame.setDefaultCloseOperation(JFrame.EXIT_ON_CLOSE);
//jtfMainFrame.setSize(150, 50);
jtfMainFrame.pack();
jtfMainFrame.setVisible(true);

}

public static void main(String args[]) throws InterruptedException {
System.out.println("GPIO Digital Listen Example ... started.");

// create gpio controller and input button objects
final GpioController gpio = GpioFactory.getInstance();
final GpioPinDigitalInput myButton = gpio.provisionDigitalInputPin(RaspiPin.GPIO_04);
myButton.setShutdownOptions(true);

// create and register gpio pin listener
myButton.addListener(new GpioPinListenerDigital() {
@Override
public void handleGpioPinDigitalStateChangeEvent(GpioPinDigitalStateChangeEvent event) {
// display pin state on console
System.out.println(" --> GPIO PIN STATE CHANGE: " + event.getPin() + " = " + event.getState());
if (event.getState().toString() == "HIGH") {
labelin.setText("INPUT HIGH");
labelin.setForeground(Color.RED);
} else {
labelin.setText("INPUT LOW");
labelin.setForeground(Color.BLACK);
}
}

});

System.out.println(" ... complete the GPIO #04 circuit and see the listener feedback here in the console.");

// Set the look and feel to Java Swing Look
try {
UIManager.setLookAndFeel(UIManager
.getCrossPlatformLookAndFeelClassName());
} catch (Exception e) {
}
gpio_di_swg application = new gpio_di_swg();
}
}

 

Working Remotely

If you are working remotely from your Raspberry Pi with SSH, you will have a problem running the Java graphical applications. This problem is due to the fact that by default X windows applications requiring super user rights are not allowed to run remote over SSH.

By adding the sudo option XAUTHORITY=$HOME/.Xauthority  java X application can be run remotely. So an example would be:

sudo XAUTHORITY=$HOME/.Xauthority java -classpath .:classes:/opt/pi4j/lib/'*' myapp

Final Thoughts

The Pi4J library has a lot of examples to cover a good variety of Raspberry Pi I/O examples and TOPs, such as MCP3004 chips, LCD and PiFace Tops.

Workarounds

When I ran one of the examples I saw the message:

Unable to determine hardware version. I see: Hardware : BCM2835
,
 - expecting BCM2708 or BCM2709.
If this is a genuine Raspberry Pi then please report this
to projects@drogon.net. If this is not a Raspberry Pi then you
are on your own as wiringPi is designed to support the
Raspberry Pi ONLY.
user@system ~> gpio -v
gpio version: 2.38

There were a lot of comments on how to fix the problem. I found that rolling back the kernel worked for me, and this is done by:

sudo rpi-update 52241088c1da59a359110d39c1875cda56496764