STOMP Protocol with RabbitMQ, Node-Red and Python

The STOMP (Simple Text Oriented Messaging Protocol) is a messaging system that is similar to MQTT (Message Queue Telemetric Transport) and AMQP (Advanced Message Queue Protocol).

STOMP uses a send and subscribe mechanism to pass messages and like other messaging systems, the STOMP server is called a broker. Open Source STOMP brokers are available with ActiveMQ and RabbitMQ.

In this blog I wanted to document my setup that used the STOMP plug-in to RabbitMQ and show examples of getting Python and Node-Red subscribing and sending data.

STOMP Specification logo

Why use STOMP?

MQTT and AMQP are very commonly used messaging protocol, so why use STOMP?

AMQP has a lot of great features but for many simple IoT (Internet of Things) applications AMQP may be overkill. Also AMQP is not supported with any mainstream Arduino libraries whereas Arduino Libraries exist for STOMP.

Both STOMP and MQTT are really light-weight, however STOMP has the advantage of being able to add headers and properties along with the message (a feature missing in MQTT but available in AMQP). So you could send a STOMP sensor message and include things like: quality, status, area, and comments along with the value.

STOMP messages can also be used on client side JavaScript web pages. This isn’t supported on AMQP but it possible using Websockets with MQTT.

 

RabbitMQ Setup with STOMP

RabbitMQ is a very popular open source messaging middle-ware. By default RabbitMQ loads AMQP (Advanced Message Queue Protocol) but it can also load MQTT and STOMP protocols. RabbitMQ offers Web Administration and an environment that allows different messaging protocols to intercommunicate.

RabbitMQ can be installed on Window, Linux, MacOS systems and there are also some cloud based offerings. For small systems lower end hardware like a Raspberry Pi can be used.  For complete RabbitMQ installation instructions see: https://www.rabbitmq.com/download.html . To install and run RabbitMQ on a Ubuntu system enter:

sudo apt-get update
sudo apt-get install rabbitmq-server
sudo service rabbitmq-server start

The next step is to add some plug-ins. For this project I loaded the STOMP plug-in and the Web Administration plug-in:

sudo rabbitmq-plugins enable rabbitmq_stomp
sudo rabbitmq-plugins enable rabbitmq_management

The rabbitmqctl command line tool allows you to configure and review the RabbitMQ server. To add a user admin1,  with password admin1, that has config, write and read rights for management and administrator access, enter:

sudo rabbitmqctl add_user admin1 admin1
sudo rabbitmqctl set_permissions -p / admin1 ".*" ".*" ".*"
sudo rabbitmqctl set_user_tags admin1 management administrator

At this point a super-user (admin1) has been defined and that user has been given rights to talk to all features of RabbitMQ including STOMP. See the STOMP documentation for STOMP specific user and system configuration.

After you’ve defined an administrative user the RabbitMQ web management plug-in can be accessed by: http://ip_address:15672 . The Overview tab should show STOMP running on the default port of 61613.

rabbitMQ_ov

For clarity it is good to define a RabbitMQ exchange that is to be used for STOMP. There are a lot of options but the easiest thing is to make a DIRECT exchange that is DURABLE (available after a restart).

rabbitMQ_ex

Python and STOMP

Python has a library component : stompy.py that is both a standalone STOMP testing client and a library component. To install the component:

 pip install stomp.py

Once loaded stomp.py can be used to connect to a server and test subscribing and send data. Below is an example connecting to host 192.168.0.105 with user: pete and password pete.

pete@lubuntu:~$ stomp -H 192.168.0.105 -U pete -W pete


> subscribe /exchange/stomp1/tag1
Subscribing to '/exchange/stomp1/tag1' with acknowledge set to 'auto', id set to '1'
> send /exchange/stomp1/tag1 hi from python
> 
message-id: T_1@@session-8xhCjj0giLAiOFg8E6OGPQ@@1
subscription: 1

hi from python
> 

For this example the exchange stomp1 was defined. The topic tag1 was dynamically defined and a message was sent and subscribe to. The topic (tag1) doesn’t need to be defined in RabbitMQ.

A Python STOMP test program would be:

 #  
 # stomp_test.py - test STOMP  
 #  
 import time  
 import sys  
   
 import stomp  
 import random  
   
 class MyListener(stomp.ConnectionListener):  
   def on_error(self, headers, message):  
     print('received an error "%s"' % message)  
   def on_message(self, headers, message):  
     print('received a message "%s"' % message)  
   
 # Define a STOMP connection and port  
 conn = stomp.Connection([("192.168.0.105", 61613)])  
 conn.set_listener('', MyListener())  
 conn.connect('pete', 'pete', wait=True) # define the username/password  
   
 # Setup a subscription  
 conn.subscribe(destination='/exchange/stomp1/tag1', id=1, ack='auto')  
   
 while True:  
   time.sleep(15) # send a random message every 15 seconds  
   conn.send(body=str(random.randint(1,11)), destination='/exchange/stomp1/tag1')  
   
 #conn.disconnect()  

 

Node-Red and STOMP

The Node-Red STOMP node can be installed using the menu option “Manage Palette”  or manually:

cd $HOME
cd ~/.node-red 
npm install node-red-node-stomp

A simple Node-Red test circuit would include:

  • an injector node – manually send in a test string
  • a STOMP output node – to send messages to a broker
  • a STOMP input node – subscribe to messages
  • a debug node – to show messages in the debug tab

nr_sample

The logic requires the STOMP node to be defined, and the STOMP nodes need to have a valid exchange and topic:

Node-Red can be used to configure Web dashboards that can show trending and charting of STOMP values.

 

Testing STOMP with RabbitMQ

Once I had both Python and Node-Red working with STOMP I was able to go back to my RabbitMQ and view the connections.

rabbitMQ_conn

It is also possible to use RabbitMQ to publish STOMP messages. In this example I included in the header: status, quality and units.

rabbitMQ_ex_pub

I then was able to check the message in Node-Red.

nr_msg_debug

 

Final Comments

Compared to AMQP and MQTT there aren’t nearly as many examples so I needed to do a little bit of experimentation. As an example I found that if I send and subscribe to a queue (use: /queue/my_queue)  this was only a 1-to-1 connection, if I had 2 subscribers each subscriber would only get 1/2 the messages. So I’m recommend using RabbitMQ exchanges for STOMP messaging.

This was a quick introduction to STOMP with Node-Red and Python. I still need to look at Arduino and Javascript connections.

 

 

 

Julia programming on a Raspberry Pi

Julia is a free and open-source general purpose programming language made specifically for scientific computing.

In this blog I wanted to document my Pi Julia interface testing.

Getting Started

Julia is supported on Windows, MacOS and Linux. See the download documentation for your specific system at : https://julialang.org/downloads/. To install Julia on a Raspberry Pi enter:

sudo apt install julia

To check your install, you can run Julia at the command line:

pi@raspberrypi:~ $ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.0.3
 _/ |\__'_|_|_|\__'_|  |  Raspbian ⛬  julia/1.0.3+dfsg-4+rpi1
|__/                   |

julia> 

Julia packages can be installed at the command line by:

julia> using Pkg
julia> Pkg.add("some package")

Julia with Raspberry Pi GPIO

A Raspberry Pi package “PiGPIO” is installed by:

julia> using Pkg
julia> Pkg.add("PiGPIO")

Once the package is installed a daemon ( pigpiod) is created that the Julia script connects to. The pigpiod daemon is started by:

sudo pigpiod

Below is an example script that cycles a GPIO pin. For this example the GPIO pin is passed as a keyboard input. The readline() method returns a string, so parse() is used get an integer.

# pi1.jl - GPIO test 
using PiGPIO 

print("Enter the GPIO pin: ")

n = readline()
led_pin = parse(Int64,n) # convert the string input to an Integer

println("Pin used: $led_pin ")


#led_pin = 4 #use BCM pin 4 (physical pin 7) 
p=Pi() #connect to pigpiod daemon on localhost 
set_mode(p, led_pin, PiGPIO.OUTPUT) 
println("Julia is cycling an LED 5 times") 

try 
    for i in 1:5 # cycle 5 times 
        PiGPIO.write(p, led_pin, PiGPIO.HIGH) 
        sleep(0.5) 
        PiGPIO.write(p, led_pin, PiGPIO.LOW) 
        sleep(0.5) 
    end 
finally 
    println("Cleaning up!") 
    # if you wish to leave the pin as an input 
    #set_mode(p, led_pin, PiGPIO.INPUT) 
end

To run the code enter:

pi@raspberrypi:~ $ julia pi1.jl
Enter the GPIO pin: 4
Pin used: 4
[ Info: Successfully connected!
Julia is cycling an LED 5 times
Cleaning up!

Raspberry Pi Inputs

A simple GPIO read input programs is:

# pi2.jl - GPIO input test 

using PiGPIO

p = Pi() #connect to pigpiod daemon on localhost 

in_pin = 23
set_mode(p, in_pin, PiGPIO.INPUT)

println("\nInput status on Pin 23...")
println("hit Cntl-C to break out")

println(PiGPIO.read(p, in_pin))

while true
    println(PiGPIO.read(p, in_pin))
    sleep(1)
end

 

The output will look something like:

pi@raspberrypi: $ julia pi2.jl
[ Info: Successfully connected!

Input status on Pin 23...
hit Cntl-C to break out
1
1
0
0

Final Comments

As a Python user l found using Julia to be  frustrating, the documentation is weak and there aren’t the extensive libraries that I’m used to using in Python.

Neopixel Hats

There are some fun wearable projects that you can do with small Arduino modules like the Gemma ($9) and some neopixels. ($7).

 

Conductive thread is used to both secure the modules to the material and to do the wiring.

For this project we wanted to have a sequence of colour patterns, so we had:

  • a rainbow moving around in a circle
  • a red smiley face, that had a left/right smirk and then a wink
  • a rainbow moving around in a circle
  • a red smiley face, that had a left/right smirk and then a wink

However there a tons of possible patterns that could be used.

For the battery mounting there are a few options such as lipo batteries, coin batteries and small battery packs.

Below is some example code for the moving rainbow that we used.

Have Fun

#include <Adafruit_NeoPixel.h>

#define PIN 1
int theLED = 0;


Adafruit_NeoPixel strip = Adafruit_NeoPixel(12, PIN, NEO_GRB + NEO_KHZ800);

void setup() {
  strip.begin();
  strip.setBrightness(20);
  strip.show(); // Initialize all pixels to 'off'
}

void loop() {
  // Create a rainbow of colors that moves in a circle 

  int col;
  
     for (int i=0; i < strip.numPixels(); i++) {
        strip.setPixelColor(i, 0, 0, 255);
     }
    theLED = theLED + 1;
    if (theLED >= strip.numPixels()) {
      theLED = 0; 
    }
    strip.setPixelColor(theLED, 209, 31, 141);
    strip.setPixelColor(theLED - 1, 11, 214, 180);
    strip.setPixelColor(theLED - 2, 240, 210, 127);
    strip.setPixelColor(theLED - 3, 191, 127, 240);
    strip.show();
    delay (500);
}

Control Rasp Pi’s with Simple Lua GUIs

I was struggling to find a simple Lua graphic library. Love2D appears to be well regarded, but I wanted to find something that I could get up and running fast.

An old 1980’s graphic technology called curses has been available for years in most languages and I was familiar with it from C and Python.

In this blog I wanted to shared an example of using the Lua curses library to read and write Raspberry Pi general purpose I/O (GPIO).

Installing Lua

To install Lua on a Raspberry Pi:

sudo apt-get update
sudo apt-get install lua5.1
sudo apt-get install liblua5.1-0-dev -- development files, need by LuaRocks
sudo apt-get install lua-socket
sudo apt-get install luarocks -- package manager for Lua modules

sudo luarocks install luasocket

Lua has a package manager called luarocks, (this is similar to pip on Python), where you can install custom libraries or packages on the Pi.

There are a number of choices on how Lua can access Pi GPIO pin. I found that the lua-periphery library to be a reliable option. The Lua version of curses is not 100% compatible to the C version but it’s close.

To install these libraries enter:

sudo luarocks install lua-periphery
sudo luarocks install curses

Raspberry Pi Hardware

I used a Pimoroni Explorer Hat because it has some built in colored LEDs, but you could easily use some LEDs and resistors and wire your own equivalent setup.

 

For some details on how to use the Lua Raspberry Pi GPIO library see: https://funprojects.blog/2019/04/20/lua-and-raspberry-pi/

The Lua Curses App

My goal was to create a simple GUI with a title and a footer with the key commands, then show the values on the screen.

lua_curses_screen

To use colored text there are a few steps that are required:

  • enable color (curses.start_color())
  • define some color pairs (curses.init_pair)
  • create an attribute variable that is defined by a color pair(a_red = curses.color_pair(4))

Then use the attribute “ON” function to set the color  (stdscr:attron(a_red)).

The mvaddstr function is used to write text to position on the screen  object. (stdscr:mvaddstr(2, 5,”SET RASPBERRY PI LEDS” )).

Below is my code to setup 4 LED outputs, and use the keys 1,2,3 and 4 to write to these outputs. The “q” key is used to exit the code.

 -- A Lua curses example with some Raspberry Pi Data  
 -- Define Rasp Pi variables  
 local GPIO = require('periphery').GPIO  
 local gpio_in = GPIO(10, "in")  
 local led1 = GPIO(4,"out")  
 local led2 = GPIO(17,"out")  
 local led3 = GPIO(27,"out")  
 local led4 = GPIO(5,"out")  
 led1:write(1)  
 led2:write(1)  
 led3:write(1)  
 led4:write(1)  
 -- Define curses  
 local curses = require 'curses'  
 curses.initscr()  
 curses.echo(false) -- not noecho !  
 local stdscr = curses.stdscr() -- the screen object  
 -- setup color pairs and attribute variables  
 curses.start_color()  
 curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)  
 curses.init_pair(2, curses.COLOR_WHITE, curses.COLOR_BLACK)  
 curses.init_pair(3, curses.COLOR_BLUE, curses.COLOR_BLACK)  
 curses.init_pair(4, curses.COLOR_YELLOW, curses.COLOR_BLACK)  
 curses.init_pair(5, curses.COLOR_RED, curses.COLOR_BLACK)  
 curses.init_pair(6, curses.COLOR_GREEN, curses.COLOR_BLACK)  
 a_rw = curses.color_pair(1)  
 a_white = curses.color_pair(2)  
 a_blue = curses.color_pair(3)  
 a_yellow = curses.color_pair(4)  
 a_red = curses.color_pair(5)  
 a_green = curses.color_pair(6)  
 stdscr:clear()  
 -- Create a background  
 ncols = curses.cols()  
 nrows = curses.lines()  
  
 -- Create a top and bottom color strip  
 stdscr:attron(a_rw) -- set the fore/background colors  
 for i=0, (ncols - 1), 1 do -- write a top and bottom strip  
      stdscr:mvaddstr(0,i, " ")  
      stdscr:mvaddstr(nrows -1,i, " ")  
 end  
 stdscr:mvaddstr(0,0, " Curses Lua Dynamic Text Example")  
 stdscr:mvaddstr((nrows -1), 0, " Key Commands: q - to quit, 1,2,3,4 - to toggle LED")  
 -- Add the main screen static text  
 stdscr:attron(a_white) -- set the fore/background colors  
 stdscr:mvaddstr(2, 5,"SET RASPBERRY PI LEDS" )  
 for i=1,4,1 do   
      stdscr:mvaddstr(4+ i, 5, "LED " .. tostring(i) .. " : " )  
 end  
 stdscr:refresh()  
 local c = stdscr:getch ()  
 while c ~= 113 do -- 113 = q ,quit  
      if c == 49 then led1:write(not led1:read()) end  
      if c == 50 then led2:write(not led2:read()) end  
      if c == 51 then led3:write(not led3:read()) end  
      if c == 52 then led4:write(not led4:read()) end  
      -- show the inputs  
      stdscr:attron(a_blue)  
      stdscr:mvaddstr(5, 15, tostring(led1:read() ) .. " " )  
      stdscr:attron(a_yellow)  
      stdscr:mvaddstr(6, 15, tostring(led2:read() ) .. " " )  
      stdscr:attron(a_red)  
      stdscr:mvaddstr(7, 15, tostring(led3:read() ) .. " " )  
      stdscr:attron(a_green)  
      stdscr:mvaddstr(8, 15, tostring(led4:read() ) .. " " )  
      c = stdscr:getch ()  
 end  
 curses.endwin()  

Some Final Comments

Unfortunately I found the Lua curses documentation to be quite weak and there were very few examples.

My only major stumbling block was to find a stdscr.nodelay() function that allows the code to continue without waiting for a key stroke. This feature exists in the Python and C libraries.

Simple Terminal Interfaces

Typically our interfaces for projects use colorful web pages or custom GUIs. However there are many cases where a simple text interface is all that is required. This is especially true for SSH or remote connections from a Window’s client into a Raspberry Pi or Linux server.

In this blog I’d like to review a 1980’s technology called curses, with three examples. The first example will be simulated Rasp Pi scanning app in “C” and Python. The second and third examples will be in Python and they will show large text presentation and dynamic bars.

Python Curses

Python curses are standard in Python, and they include features such as:

  • support ASCII draw characters
  • basic color support
  • window and pad objects which can be written to and cleared independently

As a first example I wanted to have a colored background, a header and footer and some dynamic text.

curses_text

The first step is to define a curses main screen object (stdscr). The next step is to enable color and to create some color pairs. Using color pairs and the screen size (height, width = stdscr.getmaxyx()) it is possible to add a header and footer strip using the srtdscr.addstr command.

The stdscr.nodelay command allow the program to cycle until the stdscr.getch() call returns a key.

# curses_text.py - create a curses app with 10 dynamic values
#
import curses , time, random

# create a curses object
stdscr = curses.initscr()
height, width = stdscr.getmaxyx() # get the window size

# define two color pairs, 1- header/footer , 2 - dynamic text, 3 - background
curses.start_color()
curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)
curses.init_pair(3, curses.COLOR_WHITE, curses.COLOR_BLUE)

# Write a header and footer, first write colored strip, then write text
stdscr.bkgd(curses.color_pair(3))
stdscr.addstr(0, 0, " " * width,curses.color_pair(1) )
stdscr.addstr(height-1, 0, " " * (width - 1),curses.color_pair(1) )
stdscr.addstr(0, 0, " Curses Dynamic Text Example" ,curses.color_pair(1) )
stdscr.addstr(height-1, 0, " Key Commands : q - to quit " ,curses.color_pair(1) )
stdscr.addstr(3, 5, "RASPBERRY PI SIMULATED SENSOR VALUES" ,curses.A_BOLD )
stdscr.refresh()

# Cycle to update text. Enter a 'q' to quit
k = 0
stdscr.nodelay(1)
while (k != ord('q')):
# write 10 lines text with a label and then some random numbers
for i in range(1,11):
    stdscr.addstr(4+ i, 5, "Sensor " + str(i) + " : " ,curses.A_BOLD )
    stdscr.addstr(4+ i, 20, str(random.randint(10,99)) ,curses.color_pair(2) )
    time.sleep(2)
    k = stdscr.getch()

curses.endwin()

The simulated Pi values will refresh every  10 seconds until the “q” key is pushed and then the terminal setting are returned to normal (curses.endwin()) and the program exits.

“C” Curses Example

For this “C” example I used a Raspberry Pi. The curses library needs to be installed by:

 sudo apt-get install libncurses5-dev

The curses syntax is similar between “C” and Python but not 100%. For example in Python the addstr command includes a color pair reference, but in “C” this is not supported so an attribute on/off (attron/attroff) command is used to reference the color pair. Below is the “C” code:

/* c1.c - Basic Curses Example */

#include <curses.h>
#include <stdlib.h>
#include <unistd.h>

int main(void)
{
    int row, col, k;    
// Create a curses object and define color pairs
    initscr();
    getmaxyx(stdscr,row,col);
    start_color();
    init_pair(1,COLOR_RED,COLOR_WHITE);
    init_pair(2,COLOR_GREEN,COLOR_BLACK);
    init_pair(3,COLOR_WHITE,COLOR_BLUE);
    curs_set(0);
    noecho();
    //keypad(stdscr,TRUE);
    nodelay(stdscr, TRUE);
// Write a header and footer, first write colored strip, then write text
    bkgd(COLOR_PAIR(3));
    attron(COLOR_PAIR(1));
// Create a top and bottom color strip
    for (int i = 0; i < col; i++) {
        mvaddstr(0, i,  " ");
        mvaddstr(row-1, i,  " ");
    }
    mvaddstr(0, 0,  " Curses C Dynamic Text Example");
    mvaddstr(row-1, 0,  " Key Commands: q - to quit");
    attroff(COLOR_PAIR(1));   
    mvaddstr(2, 5,"RASPBERRY PI SIMULATED SENSOR VALUES" );
    refresh();
// Cycle with new values every 2 seconds until a q key (133) is entered    
    while (k != 113)
    {
        attroff(COLOR_PAIR(2));
        for (int i = 0; i < 10; i++) {
            mvprintw((4+i), 5,  " Sensor %d : ",i);
        }
        attron(COLOR_PAIR(2));
        for (int i = 0; i < 10; i++) {
            mvprintw((4+i), 20,  "%d",rand() %100);
        }
        k = getch();
        sleep(2);
    }
    endwin();
    exit(0);
}

To compile and run the program (c1.c) enter:

gcc -o c1 c1.c -lncurses
./c1

The “C” example should look very similar to the earlier Python example.

Figlet for Large Custom Text

Large Custom Text can be generated using the Python Figlet library.  Figlet has an extensive selection of text presentations and it uses standard ASCII character to generate the large text presentations. The Figlet library is installed by:

pip install pyfiglet

An example from the Python shell:

pyshell_figlet

For a Figlet example, I wanted to create a large heading and a large dynamic value.

curses_di

The Figlet library can be used to generate a string with user defined texted presented a large text-like format. A little bit of testing is required because the Figlet generated text can be 3,4,5 or more characters tall and the string needs to be added to very left end of the window.

# curses_di.py - show a large heading and large dynamic value
#
import curses, time
import pyfiglet, random

def get_io():
    global value1
    testvalue = str(random.randint(100,1000)/10) + " C"
    value1 = pyfiglet.figlet_format(testvalue, font = "starwars" )

# Create a string of text based on the Figlet font object
title = pyfiglet.figlet_format("Raspi Data", font = "small" ) 

stdscr = curses.initscr() # create a curses object
# Create a couple of color definitions
curses.start_color()
curses.init_pair(1, curses.COLOR_YELLOW, curses.COLOR_BLACK)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)

# Write the BIG TITLE text string
stdscr.addstr(1,0, title,curses.color_pair(1) )
stdscr.addstr(8,0, "Sensor 1: GPIO 7 Temperature Reading" ,curses.A_BOLD)

# Cycle getting new data, enter a 'q' to quit
stdscr.nodelay(1)
k = 0
while (k != ord('q')):
    get_io() # get the data values
    stdscr.addstr(10,0, value1,curses.color_pair(2) )
    stdscr.refresh()
    time.sleep(2)

    k = stdscr.getch()

curses.endwin()

I found that the the small and doom fonts worked well in my testing. To check out and test Figlet fonts online see:

http://patorjk.com/software/taag/#p=display&f=Slant&t=Dude%20what%20are%20you%20doing%20%3F

Curses Windows

By defining a curses window it is possible to clear and write to a window that it is independent from the background. The syntax to create a curses window object is:

mynewwindow = curses.newwin(height, width, begin_y, begin_x)

Windows are ideal for applications where multiple items such as Figlet objects are used. Below is an example with two large Figlet values.

Figlet2win


# Create a static 2 large values example
#
import curses, time
import pyfiglet, random
# Create a string of text based on the Figlet font object
title = pyfiglet.figlet_format("Weather Station 2", font = "small" )

stdscr = curses.initscr() # create a curses object
# Create a couple of color definitions
curses.start_color()
curses.init_pair(1, curses.COLOR_YELLOW, curses.COLOR_BLACK)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)

# Write the BIG TITLE text string
stdscr.addstr(1,0, title,curses.color_pair(1) )
stdscr.refresh()

win1 = curses.newwin(9, 44, 6, 4)
win1.addstr(8,0, "Sensor 1: Temperature Reading" ,curses.A_BOLD)

win2 = curses.newwin(9, 44, 6, 50)
win2.addstr(8,0, "Sensor 2: Humidity Reading" ,curses.A_BOLD)
value1 = pyfiglet.figlet_format("23 C", font = "doom" )
win1.addstr(0,0,value1,curses.color_pair(2) )
win1.refresh()
value2 = pyfiglet.figlet_format("35 %", font = "doom" )
win2.addstr(0,0, value2 ,curses.color_pair(2) )
win2.refresh()

# Hit any key to exit
stdscr.getch()
curses.endwin()

Dynamic Bars Example

For the Dynamic bars example I created a get_io function to simulate two real time data  values.

As a first step I created some background information such as headings, a header and a footer. By using the call: height, width = stdscr.getmaxyx() , I am able to position banners at the top and bottom of the terminal window. All of the background info is written to the stdscr object.

Two windows objects (win1 and win2) are used for the real time dynamic bars. Old bar data is removed using the win1.clear() and win2.clear() calls. Like the static example the dynamic bars are created by writing a fill character multiplied by the actual real time value (win1.addstr(1, 1, bar * value1) ). A window.refresh() command is used to show the changes.

The stdscr.getch() method is used to catch keyboard input, and the terminal program is exited when a quit character, “q” is entered.

The complete two dynamic bar program is shown below:


# Simple bar value interface
#
import curses
import time

# get_io is using random values, but a real I/O handler would be here
def get_io():
    import random
    global value1, value2
    value1 = random.randint(1,30)
    value2 = random.randint(1,30)

bar = '█' # an extended ASCII 'fill' character
stdscr = curses.initscr()
height, width = stdscr.getmaxyx() # get the window size
curses.start_color()
curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)
curses.init_pair(3, curses.COLOR_YELLOW, curses.COLOR_BLACK)

# layout the header and footer
stdscr.addstr(1,1, " " * (width -2),curses.color_pair(1) )
stdscr.addstr(1,15, "Raspberry Pi I/O",curses.color_pair(1) )
stdscr.hline(2,1,"_",width)
stdscr.addstr(height -1,1, " " * (width -2),curses.color_pair(1) )
stdscr.addstr(height -1,5, "Hit q to quit",curses.color_pair(1) )

# add some labels
stdscr.addstr(4,1, "Pi Sensor 1 :")
stdscr.addstr(8,1, "Pi Sensor 2 :")

# Define windows to be used for bar charts
win1 = curses.newwin(3, 32, 3, 15) # curses.newwin(height, width, begin_y, begin_x)
win2 = curses.newwin(3, 32, 7, 15) # curses.newwin(height, width, begin_y, begin_x)

# Use the 'q' key to quit
k = 0
while (k != ord('q')):
    get_io() # get the data values
    win1.clear()
    win1.border(0)
    win2.clear()
    win2.border(0)
# create bars bases on the returned values
    win1.addstr(1, 1, bar * value1, curses.color_pair(2))
    win1.refresh()
    win2.addstr(1, 1, bar * value2 , curses.color_pair(3))
    win2.refresh()
# add numeric values beside the bars
    stdscr.addstr(4,50, str(value1) + " Deg ",curses.A_BOLD )
    stdscr.addstr(8,50, str(value2) + " Deg ",curses.A_BOLD )
    stdscr.refresh()
    time.sleep(2)
    stdscr.nodelay(1)
    k = stdscr.getch() # look for a keyboard input, but don't wait

curses.endwin() # restore the terminal settings back to the original

cbars

For testing I used a random simulator for the data but the get_io function could be easily configured to connect to a Raspberry Pi or Arduino module.

The outline boxes in the window object could look strange if you are using a Window’s based SSH client like Putty. To create the problem in Putty’s settings, select: Window ->  Translations and use VSCII as the remote character set.

putty

Final Comments

Curses is definitely an ‘old school’ technology but it offers a simple solution for SSH and terminal based connections.

Curve Fits for Kids

I was helping my daughters with some school projects in graphing, and I thought that it would be good  to document and share our discussion. This blog will look at solving the equations for plots using Desmos, Google Docs and Excel.

Desmos

Desmos (https://www.desmos.com/calculator) is great online free graphing package that is used in many school systems. Desmos has a lot of options to simulate, tweek and view data.

To add a table of data, use the “+” button and select table.

demos1

Once the data table has been entered an f(x) expression can be linked to the data. In the example below a linear expression was used. It’s important to note, two things:

    • The subscript must match. So if the table uses: y2  x2 then the formula needs to use the same.
    • A ~ (tilda) is used instead of an = (equals) sign.

 

demos2

If the formula matches up with the table a plot will be drawn and the slope, intercept and r-squared (goodnes of fit) values will be shown.

We found that for simple first order (straight line curves) Desmos worked really well, however for polynomials and other more advanced equations it was a little awkward.

Google Docs

Our school board uses Google Classroom which is an online suite of tools that enables kids to do Microsoft Excel, Word and Powerpoint in a free web based environment.

The Sheets options (https://docs.google.com/spreadsheets) of Google Docs works very much like Excel. To do some charting, enter a table of data and use the “chart” button to present the desired plot.

In the “Customize” options of the chart, under Series, a trend line can be added with an equation and a R-squared (fitness of curve) text.

googledoc1

It is also possible to use the SLOPE, INTERCEPT and RSQ formulas show the results in a cell.

The regression type supports functions such as log, exponential, and polynomial. For the example below we got the equation for a 2nd order polynomial.

googledoc2

Note: Sheets can import and export to and from Excel.

Excel

Different versions of Excel (and LibreOffice) will have a slightly different way to get the stats on a curve. For our work we used Excel from Office 2016.

In the Chart Elements, under Trend Line, the more Options items will offer features that are almost identical to Google Sheets.

excel1

A type of trend line can be selected and the solved equation and R-squared value can be shown on the plot.

excel2

Advanced equations like higher order polynomials can also be used.

excel3

Excel (and LibreOffice) support function calls for : SLOPE, INTERCEPT and RSQ.

 

Summary

Desmos is an excellent package for kids to play with and understand the different plot types, but it probably isn’t the best tool for doing curve fits of test data for projects.

Excel and Google Sheets are excellent for most charting and plot statistical projects.

 

Control micro:bits from your PC

BBC Micro Bit, (micro:bit) is an open source hardware ARM-based embedded system designed by the BBC for use in computer education in the UK. The device is half the size of a credit card and has an ARM Cortex-M0 processor, accelerometer and magnetometer sensors, Bluetooth and USB connectivity, a display consisting of 25 LEDs, and two programmable button.

Depending on where you purchase it the price ranges between $15-$20. So this is a very attractive module for the beginning programmer.

The micro:bit module has 2 buttons to interface to it and a small 5×5 LED screen. This is good for small tests but its a little limiting if you want to see or do more complex tasks.

In this blog I wanted to discuss how to connect a micro:bit to a PC so that commands can be sent to the micro:bit and text responses can be viewed on the PC.

First Impressions

My first impression was that the micro:bit was super easy to hook up and get started on. All you need is a Web browser (https://makecode.microbit.org/) and a USB cable to get started on some programming.

Some of my other comments are:

  • It’s great how you can toggle between block programming and Javascript
  • Python support is not as complete as block or Javascript (i.e. no out of the box serial support)
  • MIT Scratch is supported but limited. You can not write out to pins, only read

 

PC to micro:bit Serial Connections

My goal was to create a menu example where I could read/control logic from a PC to a micro:bit. For my testing I used:

  • T – read the temperature of the micro:bit
  • L – return the light sensor value
  • 1 – set an output (LED on pin 8 is turned on)
  • 0 – reset an output (LED on pin 8 is turned off)

Micro:bit Logic

The micro:bit block logic is added by selecting an item from the left code block palette. To start catching serial communications a “Serial on data received” block is used.

mb_serial_logic

After this block is added other blocks are inserted into it. For this example I’m saving the “Serial read line” data into a variable called cmd. Because a new line character is sent from the PC after the command, a substring block is used to caption only the first character sent. (So cmd = “T” and not “T\n” for a temperature request).

The next step is to use an if block to determine which command is sent and do the requested action. The serial write line block is used to send back text to the PC.

The full logic for this example is shown below:

serial_blocks

This same logic could be done in JavaScript.

javascript

PC side communications

For the PC side communications I used Python. Depending if you are using Windows, MacOS or Linux you will need to determine which serial port you are using. If you are using Window you can use Device Manager to see which port is connected to the micro:bit.

device_manager

For the Python interface I used the Tkinter graphic library and I used 4 buttons to send commands and a label to show the feedback.

Some important points in the Python program:

  • The serial communication is by default at 115200 baud
  • The micro:bit is expecting a new line character at the end of a command, so a “\n” is needs to be appended to the end of a command.
  • The serial library needs the strings to be sent out encoded

 

# Python serial interface to a micro:bit
#
import serial
import tkinter as tk

# Send a command to the micro:bit and show the response
def myfunc(action):
   print ("Requested action: ",action)
   out = action + "\n"
   out2 = out.encode('utf_8')
   ser.write(out2)   
   lstatus.config(text = ser.readline())

# configure the serial connections (this will differ on your setup)
ser = serial.Serial(
    port='COM7',
    baudrate=115200
)

root = tk.Tk()
root.title("Micro:bit")

# Button for temperature request
bt_T = tk.Button(root, width=10,text= "Get Temp",bg='silver' ,
  command = lambda: myfunc("T"),relief="raised")
bt_T.grid(row = 1,column = 1)

# Button for light sensor data request
bt_L = tk.Button(root, width=10,text= "Get Light",bg='silver' ,
  command = lambda: myfunc("L"),relief="raised")
bt_L.grid(row = 1,column = 2)

# Button to set LED ON
bt_1 = tk.Button(root, width=10,text= "LED ON",bg='silver' ,
  command = lambda: myfunc("1"),relief="raised")
bt_1.grid(row = 2,column = 1)

# Button to set LED OFF
bt_0 = tk.Button(root, width=10,text= "LED OFF",bg='silver' ,
  command = lambda: myfunc("0"),relief="raised")
bt_0.grid(row = 2,column = 2)

# Label to show results
lstatus = tk.Label(root, width= 25, text= "                 ", relief="raised")
lstatus.grid(row = 3,column = 1,  columnspan = 2)

root.mainloop()

Microbit_menu

Summary

For this example I used a simple Python menu to send some commands down to the micro:bit and get a response back. Other projects could include:

  • Have the micro:bit periodically scan I/O (like moisture sensors) and sent the data to a PC for historical trending or alarms
  • control physical devices like servos and motors

 

 

 

 

 

 

 

ODROID – A Raspberry Pi Alternative

The ODROID is a series of single-board computers manufactured by Hardkernel in South Korea. The ODROID-C1+ ($35) and the ODROID-C2 ($46) have a form factor similar to the Raspberry Pi 3. The higher end ODROID-XU($59) which is around 5 times faster than the Pi 3 has a signifigently different board layout.

For my testing I looked at the ODROID-C2 ($46), it is a little more expensive than a Pi 3 but the literature states that it’s 2-3 times faster.

My goal was to see if I could use the ODROID-C2 for some typical Raspberry Pi applications. In this blog I will be looking at doing C, Python and NodeRed programming from a Pi user perspective.

OD_PI

I’ve been happy with the functionality and openness of the Raspberry Pi platform, however I find its desktop performance to be sluggish. For only a few dollars more than a Pi 3 the ODROID-C2 CPU, RAM and GPU specs are impressive.

ODROID-C2 / Raspberry Pi 3 Hardware Comparison
Odroid C2 Raspberry Pi 3
CPU Amlogic S905 SoC
4 x ARM Cortex-A53 1.5GHz
64bit ARMv8 Architecture @28nm
Broadcom BCM2837
4 x ARM Cortex-A53 1.2Ghz
64bit ARMv7 Architecture @40nm
RAM 2GB 32bit DDR3 912MHz 1GB 32bit LPDDR2 450MHz
GPU 3 x ARM Mali-450 MP 700MHz 1 x VideoCore IV 250MHz
USB 4 Ports 4 Ports
Ethernet / LAN 10 / 100 / 1000 Mbit/s 10 / 100 Mbit/s
Built in WiFi No Yes
Built in Bluetooth No Yes
IR Receiver Built in Needs add-on
I/O Expansion 40 + 7 pin port
GPIO / UART / I2C / I2S / ADC
40 pin port
GPIO / UART / SPI / I2S
Camera Input USB 720p MIPI CSI 1080p
List Price (US) $46 $35

First Impressions

The ODROID-C2 is almost the same footprint as the Raspberry Pi 3 but not exactly. I found that because the microSD mounting is different some, but not all, of my Pi cases could be used .

lego_case

When you are designing your projects it is important to note that the ODROID-C2 does not have a built in Wifi or Bluetooth adapters, so you’ll need wired connections or USB adapters. Like some of the Orange Pi modules the ODROID-C2 has a built-in IR connection.

ODROID-C2 can be loaded with Ubuntu, Arch Linux and Android images. For my testing I used the Armbian 5.40 Ubuntu desktop and the performance was signifigently faster than my Raspberry Pi 3 Raspian desktop. I could definitely see ODROID-C2 being used as a low cost Web browser station.

The ODROID-C2 images are quite lean, so you will need to go the ODROID Wiki, https://wiki.odroid.com, for instructions on loading additional software components.

The ODROID-C2 has a 40 pin General Purpose Input/Output (GPIO) arrangement like the Pi 3, so it is possible to use Pi prototyping hats on the ODROID-C2 .

pi_hats

There are some noticeable differences in the pin definitions between the two modules, so for this reason I didn’t risk using any of my intelligent Pi hats on the ODROID-C2. The gpio command line tool can be used to view the pin definitions:

Od_readall

The Raspberry Pi GPIO names are in the range of 2 to 27, whereas the ODROID-C2 GPIO ranges are in the 200’s, because of this don’t expect to be able to run all your Raspberry Pi code “as is” on the ODROID-C2.

Unlike the Arduino the Raspberry Pi platform has no built in support for analog inputs. I got pretty excited when I noticed that the ODROID-C2 had two built in Analog-to-Digital Converter (ADC) pins (AIN.1 on pin 37 and AIN.0 on pin 40). However after some investigation I found that these pins had virtually no example code and they only support 1.8 volts. Most of my analog input sensors require 3.3V or 5V so I’m not sure how often these ADC pins will be used.

Python Applications

The ODROID-C2 Wiki references the RPi.GPIO and wiringpi Python libraries. I tested both of these libraries and I found that standard reads and writes worked, but neither of these libraries supported the callback functions like the Raspberry Pi versions.

For existing Pi projects where you are using callback functions for rising and/or falling digital signals (like intrusion alarms) you will need to do some re-coding with a polling method. It’s also important to note that the ODROID RPi.GPIO library is a little confusing because it uses the Pi pin names and not the ODROID pin names, so for example ODROID-C2 physical pin 7 is referenced as GPIO.04 (as on a PI) and not GPIO.249 (the ODROID-C2 name). Below is simple Python example that polls for a button press and then toggles an LED output when a button press is caught.

import RPi.GPIO as GPIO

GPIO.setmode(GPIO.BCM)

button = 4 # physical pin 7, PI GPIO.04, ODROID-C2 GPIO.249
led = 17 # physical pin 11, PI GPIO.17, ODROID-C2 GPIO.247
GPIO.setup(led, GPIO.OUT, initial=GPIO.LOW)
GPIO.setup(button, GPIO.IN, pull_up_down = GPIO.PUD_UP)

print ('Wait for button...')
while True:
    if GPIO.input(button) == 1:
        GPIO.output(led,0)
    else:
    GPIO.output(led,1)
    print "button pressed"

There are some excellent Python libraries that are designed to work with the Raspberry Pi. However it will require some trial and error to determine which libraries will and won’t work with the ODROID-C2.

I tried testing the DHT11 temperature and humidity sensor with the ODROID-C2, and unfortunately the library had a “Segmentation Error” when I tried running an example.

Node-Red

NodeRed can be installed on ODROID-C2 by using the manual install instructions for Raspbian at nodered.org. This install procedure will add a start item to the desktop Application menu, but due to hardware differences the Raspberry Pi GPIO input/output components will not load.

To read and write to Raspberry Pi GPIO a simple workaround is to use exec nodes to call the gpio utility.

Odroid_NodeRed

The command line syntax for writing a gpio output is: gpio write pin state, and for reading it is: gpio read pin. One of the limitations of this workaround is that you will need to add a polling mechanism, luckily there are some good scheduling nodes such as Big Timer that can be used.

C Applications

Programming in C is fairly well documented and an ODROID “C Tinkering Kit” is sold separately. The wiringPi library is used in C applications.

Below is a C version of the Python example above. It is important to note that these 2 examples talk to the same physical pins but the C wiringPi library uses the ODROID-C2 wPi numbers and the Python RPi.GPIO library uses the Pi BCM numbers.

// led.c - Read/Write "C" example for an Odroid-C2
//
#include  <wiringPi.h>

int main(void)
{
    wiringPiSetup();
    int led = 0;
    int button = 7;
    pinMode(led, OUTPUT);
    pinMode(button, INPUT);
 
    for (;;)
    {
        if (digitalRead(button) == 1) {
           digitalWrite(led, LOW); 
        }
	else
	{
           digitalWrite(led, HIGH); 
        }
    }
    return 0;
}

To compile and run this program:

$ gcc -o led led.c -lwiringPi -lpthread
$ sudo ./led

Summary

I liked that I could reuse some of my Pi cases and prototyping hats with the ODROID-C2.

As a PI user I found that coding in C, Python and NodeRed on the ODROID-C2 was fairly easy, but there were many limitation compared to the Pi platform. The ODROID Wiki had the key product documentation, but it was no where near the incredibly rich documentation that exists with Raspberry Pi modules.

There are a some excellent Python libraries and PI hardware add-ons that support a variety of sensors and I/O applications, these may or not work with the ODROID hardware.

During the development cycle of a project it is nice to have a faster interface, but typically my final projects do not need any high end processing and they can often run on low end Raspberry Pi 1 modules. So for now I would stick to a Raspberry Pi for GPIO/hardware type projects.

I enjoying playing with the ODROID-C2 and for projects requiring higher performance such as video servers, graphic or Web applications then the ODROID-C2 module is definitely worth considering.

Get Your Duolingo Stats

Unfortunately the interface to https://www.duolingo.com has changed so the examples in the blog are no longer working. I left the page up for reference and I’ll try and get more information on an updated Duolingo API.

Doulingo is one of the most popular applications for learning a new language. Duolingo has a good selection of languages and it’s free.

If you wanted to create your apps to look at your progress or do some comparisons with other Doulingo users it’s a little challenging because there isn’t a documented API (Application Programming Interface).

In this blog I wanted to look at two programming choices that are available. The first is a Web page (REST API) query and the second is a Python library.

It is important to note that last year the Doulingo interface changed so a lot of the documentation on the Internet is not 100% valid.

The Web Page (REST API) Query

If you enter : https://www.duolingo.com/users/a_user_name

You will see a very large amount of JSON (JavaScript Object Notation) text. This undocumented text is quite ugly.

web_restapi

It is possible to use some Python code to call the Web page, and try to make the JSON a little more readable. The code below uses the requests library to get the Web data, and the json library to format the web output (JSON) data.

import requests
import json

r = requests.get('https://www.duolingo.com/users/some_users')

jtext = json.loads(r.text)

with open('user_data.txt', 'w', encoding='utf8') as outfile:
     json.dump(jtext, outfile, sort_keys = False, indent = 4,
               ensure_ascii = False)

This example created an outfile that is much easier (but far from simple) to analyze. Below is a section of the output file where we can see the languages section broken out.

    "tts_base_url_http": "http://d7mj4aqfscim2.cloudfront.net/",
    "id": 141898265,
    "dict_base_url": "http://d2.duolingo.com/",
    "cohort": null,
    "daily_goal": 10,
    "delete-permissions": false,
    "ads_enabled": true,
    "languages": [         {
            "streak": 4,
            "language_string": "Greek",
            "points": 0,
            "learning": false,
            "language": "el",
            "level": 1,
            "current_learning": false,
            "sentences_translated": 0,
            "to_next_level": 60
        },
        {
            "streak": 4,
            "language_string": "Esperanto",
            "points": 0,
            "learning": false,
            "language": "eo",
            "level": 1,
            "current_learning": false,
            "sentences_translated": 0,
            "to_next_level": 60
        },

Show the Languages and Work History

By looking at the outfile the languages studied and the work history can be determined:

import requests
import json
from datetime import datetime

r = requests.get('https://www.duolingo.com/users/pete_xxxxx')

j = json.loads(r.text)

print ("Languages Studied\n")
for thelang in j["languages"]:
    if thelang["points"] > 0 :
        print (thelang["language_string"], ",Level = ", thelang["level"], ",Points=", thelang["points"])

print ("\n\nWork History\n")

for ical in j["calendar"]:
    ddate =  datetime.fromtimestamp(ical["datetime"] / 1e3)
    sdate = ddate.strftime("%m/%d/%Y")
    print(sdate, ", Improvement = ", ical['improvement'])

For my example this gave the following output:


Languages Studied

Indonesian ,Level = 6 ,Points= 570
Spanish ,Level = 13 ,Points= 5102
Chinese ,Level = 2 ,Points= 70
French ,Level = 9 ,Points= 1710
German ,Level = 6 ,Points= 616
Japanese ,Level = 10 ,Points= 2297

Work History

02/22/2019 , Improvement = 10
02/22/2019 , Improvement = 10
02/23/2019 , Improvement = 10
02/23/2019 , Improvement = 10
02/24/2019 , Improvement = 10

Duolingo Python Library

There is an unofficial Duolingo Python Library, the author did an excellent job of putting together some useful calls. Unfortunately since the Duolingo interface changed last year not all the functions are now available or working. However the key ones are still usable.

To install the library go to a terminal window or command prompt and enter:

pip install duolingo-api

A Python example would be:

import duolingo
from datetime import datetime

lingo  = duolingo.Duolingo('pete_xxxxx')

# get_user_info : works but outputs too much info
#print (lingo.get_user_info())

print ("Languages I am studying:")
print (lingo.get_languages(abbreviations=False))

print("\n\nLanguage Progress\n")
for ilang in lingo.get_languages(abbreviations=False):
       jlang = lingo.get_language_details(ilang)
       print( jlang["language_string"]," ,Points: ",jlang["points"], "Level = ", jlang["level"])

print("\n\nWork History\n")
for ical in lingo.get_calendar():
    ddate =  datetime.fromtimestamp(ical["datetime"] / 1e3)
    sdate = ddate.strftime("%m/%d/%Y")
    print(sdate, ", Improvement = ", ical['improvement'])

For my example the output is:


Languages I am studying:
['Indonesian', 'Spanish', 'Chinese', 'French', 'German', 'Japanese']

Language Progress

Indonesian ,Points: 570 Level = 6
Spanish ,Points: 5102 Level = 13
Chinese ,Points: 70 Level = 2
French ,Points: 1710 Level = 9
German ,Points: 616 Level = 6
Japanese ,Points: 2297 Level = 10

Work History

02/22/2019 , Improvement = 10
02/22/2019 , Improvement = 10
02/23/2019 , Improvement = 10
02/23/2019 , Improvement = 10
02/24/2019 , Improvement = 10

Summary

Now that we have a basic interface to Duolingo it’s possible to start doing some trends and charts. Below a is some simple chart code for language levels.

import duolingo
import numpy as np
import matplotlib.pyplot as plt

lingo  = duolingo.Duolingo('pete_xxxx')

thelang = lingo.get_languages(abbreviations=False)

thelevel = []
for ilang in lingo.get_languages(abbreviations=False):
       jlang = lingo.get_language_details(ilang)
       thelevel.append(jlang["level"])


plt.bar(thelang,thelevel, color='r')

plt.legend()
plt.xlabel('Language')
plt.ylabel('Level')

plt.title('My Duolingo Progress')

plt.show()

Duo_progress