Home Assistant (REST) API

There are a few methods to communicate with Home Assistant. In this blog I wanted to document my notes on using the REST API.

Getting Started

I found that loading the File Editor Add-on made configuration changes quite easy. To load the File Editor, select the Supervisor item, then Add-on Store:

With the File Editor option you be able to modify your /config/configuration.yaml file. For this you’ll need to add an api: statement. I also added a command line sensor that shows the Raspberry Pi CPU idle time. I did this so that I could see a dynamic analog value:

After I made these changes I restarted my HA application, by the “Configuration” -> “Server Controls”.

Next I needed to create a user token. Select your user and then click on “Create Token” in the Long-Lived Access Tokens section. This token is very long and it only is shown once, so copy and paste it somewhere save.

Access the REST API with CURL

Curl is a command line utility that exists on Linux, Mac OS and Windows. I work in Linux mostly and it’s pre-installed with Ubuntu. If you’re working in Windows you’ll need to install CURL.

Getting started with curl isn’t required and you got straight to programming in your favourite language, however I found that it was usefully testing things out in curl before I did any programming.

The REST API is essentially an HTTP URL with some headers and parameters passed to it. For a full definition see the HA API document. The key items in REST API are:

  • Request type – GET or POST (note: there are other types)
  • Authorization – this is where the user token is passed
  • Data – is used for setting and defining tags
  • URL – the Home Assistant URL and the end point (option to view or set)

To check that the HA API is running a curl GET command can used with the endpoint of /api/.

$ curl -X GET -H "Authorization: Bearer eyJ0eXAiO....zLjc"   http://192.168.0.103:8123/api/

{"message": "API running."}

The user token is super long so your can use the \ character to break up your command. For example:

curl -X GET \
   -H "Authorization: Bearer eyJ0eXAiOiJKV......zLjc" \
   http://192.168.0.106:8123/api/states

Read a Specific HA Item

To get a specific HA item you’ll need to know its entity id. This can found by looking at the “Configuration” -> “Entities” page:

For my example I created a sensor called Idle Time, its entity id is: sensor.idle_time.

A curl GET command with the endpoint of /states/sensor.idle_time will return information on this sensor. The full curl command and the results would look like:

$ curl -X GET   -H "Authorization: Bearer eyJ0eXAiOiJKV1Q....zLjc"  \   http://192.168.0.103:8123/api/states/sensor.idle_time

{"entity_id": "sensor.idle_time", "state": "98.45", "attributes": {"unit_of_measurement": "%", "friendly_name": "Idle Time"}, "last_changed": "2020-12-12T17:34:10.472304+00:00", "last_updated": "2020-12-12T17:34:10.472304+00:00", "context": {"id": "351548f602f5a3887ff09f26903712bc", "parent_id": null, "user_id": null}}

Write to a New HA Item

A new or dynamic items can be created and written to remotely using a POST command with the definitions included in the data section. An example to create an entity called myput1 with a value of 88.6 would be:

curl -X POST \
   -H "Authorization: Bearer eyJ0eXAiOi....zLjc" \
   -H "Content-Type: application/json" \
   -d '{"state":"88.6", "attributes": {"unit_of_measurement": "%", "friendly_name": "Remote Input 1"}}' \
   http://192.168.0.103:8123/api/states/sensor.myinput1

This new entity is now available to HA and shown on the dashboard.

Write to a Switch

If you have a writeable device such as a switch you can use the REST to remotely control it.

For myself I have a Wemo switch with an entity name of : switch.switch1.

To control the switch the entity id is passed in the data section and the endpoint uses either a turn_on or turn_off parameter.

curl -X POST \
   -H "Authorization: Bearer eyJ0eXAiOiJ....zLjc" \
   -H "Content-Type: application/json" \
   -d '{"entity_id": "switch.switch1"}' \
   http://192.168.0.103:8123/api/services/switch/turn_on

Python and the HA API

Python can parse the JSON responses from the reading a sensor value:

from requests import get
import json

url = "http://192.168.0.103:8123/api/states/sensor.idle_time"
token = "eyJ0eXAiOiJK...zLjc"

headers = {
    "Authorization": "Bearer " + token,
    "content-type": "application/json",
}

response = get(url, headers=headers)

print("Rest API Response\n")
print(response.text)

# Create a json variable
jdata = json.loads(response.text)

print("\nJSON values\n")
for i in jdata:
    print(i, jdata[i])

The output will be something like:

Rest API Response

 {"entity_id": "sensor.idle_time", "state": "98.46", "attributes": {"unit_of_measurement": "%", "friendly_name": "Idle Time"}, "last_changed": "2020-12-12T19:29:10.655530+00:00", "last_updated": "2020-12-12T19:29:10.655530+00:00", "context": {"id": "2509c01cadb9e5b0681fa22d914e7b10", "parent_id": null, "user_id": null}}

 JSON values

 entity_id sensor.idle_time
 state 98.46
 attributes {'unit_of_measurement': '%', 'friendly_name': 'Idle Time'}
 last_changed 2020-12-12T19:29:10.655530+00:00
 last_updated 2020-12-12T19:29:10.655530+00:00
 context {'id': '2509c01cadb9e5b0681fa22d914e7b10', 'parent_id': None, 'user_id': None}

To write a value to myinput1 in Home Assistant:

from requests import post
import json

url = "http://192.168.0.103:8123/api/states/sensor.myinput1"
token = "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJkMDI2YjAxY2VkZWU0M2E1OWY1NmI1OTM2OGU1NmI0OSIsImlhdCI6MTYwNzc5Mzc0NCwiZXhwIjoxOTIzMTUzNzQ0fQ.qEKVKdadxNWp249H3s_nmKyzQMIu5WDQkS9hiT-zLjc"

mydata = '{"state":"99.3", "attributes": {"unit_of_measurement": "%", "friendly_name": "Remote Input 1"}}'

headers = {
    "Authorization": "Bearer " + token,
    "content-type": "application/json",
}

response = post(url, headers=headers,data =mydata)

print("Rest API Response\n")
print(response.text)

# Create a json variable
jdata = json.loads(response.text)


print("\nJSON values\n")
for i in jdata:
    print(i, jdata[i])

The output will look something like:

Rest API Response

 {"entity_id": "sensor.myinput1", "state": "99.3", "attributes": {"unit_of_measurement": "%", "friendly_name": "Remote Input 1"}, "last_changed": "2020-12-12T20:40:05.797256+00:00", "last_updated": "2020-12-12T20:40:05.797256+00:00", "context": {"id": "31b422d02db41cde94470ebae7fac48c", "parent_id": null, "user_id": "1392a10c7bbb4cf0891a7f8a351740c7"}}

 JSON values

 entity_id sensor.myinput1
 state 99.3
 attributes {'unit_of_measurement': '%', 'friendly_name': 'Remote Input 1'}
 last_changed 2020-12-12T20:40:05.797256+00:00
 last_updated 2020-12-12T20:40:05.797256+00:00
 context {'id': '31b422d02db41cde94470ebae7fac48c', 'parent_id': None, 'user_id': '1392a10c7bbb4cf0891a7f8a351740c7'}

Final Comments

A REST API interface allows foreign devices such as PCs, Raspberry Pi and Arduino module to be used as remote I/O devices.

There are REST client HTTP libraries that are available for the Arduino, however it might be cleaner to implement an MQTT interface instead.

Make your Python Apps Run Faster

Python apps on lower end hardware like a Raspberry Pi can be a bit slow but luckily there are some options that you can do to improve things.

There are a number of interesting packages that allow you to compile, interpret or repackage your Python apps. In this blog I’d like to highlight two packages that I have had good success with:

  • Pypy – a replacement to the native Python interpreter. Your code can run more that 4 times faster !
  • Nuitka – a native Python utility to compile Python apps to C code !

Pypy – a faster Python

The Pypy site has some performance results, and they state that on average Pypy will run 4.2 times faster than native Python, for my test run Pypy ran 9 times faster than native Python.

The improved performance of Pypy is due to its just-in-time compiler, as opposed to the native Python’s line-by-line interpreter.

Guido van Rossum, creator of Python, has been even been quoted saying: “If you want your code to run faster, you should probably just use PyPy.”

Pypy has a Python 2.7 and 3.6 version that is available for Linux, MacOS and Microsoft Windows. To install the Pypy 3 version in Ubuntu :

sudo add-apt-repository ppa:pypy/ppa
sudo apt update
sudo apt install pypy3

Pypy3 can also be installed using snap, (this might be the easiest way on a Raspberry Pi):

#if you need to install snap 
sudo apt install snapd 
# reboot after snap is installed 
sudo snap install pypy3 --classic

The nice thing about Pypy is that you can use your base Python code as is, and you can do basic testing with Pypy in command line mode.

To run your Python application from the command line substitute python with pypy (or pypy3).

$ pypy3 myapp.py

If you running your Python app as an executable script (i.e. with the chmod +x myapp) then change the first line of your script from #!/usr/bin/python to : #!/usr/bin/pypy3

Pypy Libraries and Limitations

Pypy has an excellent selection of supported libraries , but it is important to note that not all Python libraries are supported. It’s important to check to see if your required library in supported. Unfortunately TKinter is not in the supported list.

To load a Python library into Pypy3, the Python package installer (pip) module needs to be installed:

$ wget https://bootstrap.pypa.io/get-pip.py
$ pypy3 get-pip.py

Once Pypy3 has pip installed, Python packages can be loaded:

$ pypy3 -m pip install some-pymodule

I found that I was able to load some of the “lighter” modules such as: bottle, requests, beautifulsoup4, and pika etc. without any issues. However some of the “heavier” modules such as Numpy and MatPlotLib would not load directly. If you’re planning on using some of the “heavier” Python modules it is recommended that Pypy be run in virtualenv.

Nuitka – a Python compiler written in Python

Nuitka is the Python compiler. It is written in Python. It is a seamless replacement or extension to the Python interpreter and compiles every version of Python (2.6, 2.7, 3.3, 3.4, 3.5, 3.6, 3.7, and 3.8).

Nuitka has two requirements: 1) a “C” compiler, and 2) Python must be installed on the target machine.

Nuitka works on Linux, MacOS X and Window. For Linux the gcc compiler is the default (5.1 or later). The clang compiler is used on macOS X, and for Windows MinGW64 or Visual Studio 2019 compilers can be used.

To install Nuitka:

python -m pip install nuitka

To compile a test project (test1.py) simply enter:

python-m nuitka test1.py

The Nuitka compiled program will be test1.exe in Windows and test1.bin in Linux.

Performance Testing

Performance testing is very subjective and the results can vary greatly based on so many factors. To have some kind of standard I tried using Python’s pystone benchmark testing utility. I used a Raspberry Pi 3 with a number of cases.

The testing showed some interesting results.

  • The jython (Java VMS Python) interpreter was over 2 times slower than native Python
  • iPython (used in Jupyter notebooks) was almost the same speed at native Python
  • PyInstall (a cool Python packager) was 33% slower than native Python
  • A Nuitka executable was 30% faster than native Python
  • Pypy is 9 times faster than native Python and 6.5 times faster than Nuitka

Test Results with pystone.py

TestSpeedFactor
Jython 50000 passes = 5.406222.13
Python3 50000 passes = 2.53779s1
iPython 50000 passes = 2.547941
PyInstall package50000 passes = 3.38577s1.33
Nuitka Executable 50000 passes = 1.78449s0.70
Pypy3 50000 passes = 0.272579s0.11

Summary

There are a number of other choices that can be used to make Python code run faster, (such as Pyston and Cython) but I found Pypy and Nuitka to be the best supported.

Pypy is incredibly fast, and if you’re using “lighter” weight Python modules its a fantastic fit. I think that Pypy would be great for custom microWeb server and SQLite apps. However for data mining and AI projects you might have some issues getting Pypy working with these “heavier” Python modules.

I was super impressed with Nuitka it compiled a variety of my projects without any issues. It had a nice speed improvement. Nuitka also had no problem with GUI libraries like Tkinter, PySimpleGUI and tk_tools or data management libraries like Numpy or Pandas. However Nuitka isn’t bulletproof and I had some problems with video libraries (cw2).

Using AWK in Bash Scripts

In the past Python has been my go to language for quick scripts, however lately I’ve done a lot of projects where I’ve needed to use small Bash scripts.

I found that by adding a little bit of AWK to my Bash scripts I’ve been able to do something in one line of Bash/AWK that would of taken me multiple lines of Python.

AWK is named after it’s authors: Alfred Aho, Peter Weinberger, and Brian Kernighan, and it is an old school (1994) text extraction and reporting tool.

The nice thing about AWK is that you only need to learn a couple of commands to make it usefully.

A Bash/AWK Example

I was setting up a home automation system using a Raspberry Pi with Home Assistant. I was loading a lot of add-on components and I was worried that I might be overloading the Pi so I wanted to monitor the CPU idle time.

Home Assistant supports a command line sensor, so I just needed to get the idle time out of the iostat command:

The idle time value was the 6th item on the 4th line. The AWK code to get only the idle time is:

~$ iostat | awk '{if (NR==4) print $6}'
96.92

AWK logic need to be in single quotes and curly brackets groups together statements. This logic says: if the Number of Record (NR) variable is 4 print the 6th item.

Once I was able to get the idle time I was able to add a sensor in the HA (/config/configuration.yaml) that could be monitored and alarmed:

 sensor:

 platform: command_line 
 name: Idle Time
 command: "iostat | awk 'NR==4' | awk '{print $6}'"
 unit_of_measurement: "%" 

Some Useful AWK Statements

There is an enhanced version of AWK, GAWK (GNU AWK) that might already be loaded on your Linux system. If you are on a Raspberry Pi you can install GAWK by:

sudo apt-get install gawk

There are some excellent tutorials on AWK below are some of commands that I’ve found useful:

substr(string,position,length) – get part of a string:

An example of substr could be used to get the CPU temperature from the sensors utility:

~$ sensors | grep CPU | awk '{print substr($2,2,4)}'
 44.0

The substr() command looks at the 2nd item (+44.0°C), and starts at the 2nd character and it gets 4 characters.

print() with if() – print based on conditions:

The AWK print statement can be used with an if statement to show a filtered list.

An example of this would be to filter the ps (snapshot of the current processes) command, and print only lines with a time showing:

~$ # SHOW ALL PROCESSES
~$ ps -e 
   PID TTY          TIME CMD
     1 ?        00:00:03 systemd
     2 ?        00:00:00 kthreadd
     4 ?        00:00:00 kworker/0:0H
     6 ?        00:00:00 mm_percpu_wq
     7 ?        00:00:00 ksoftirqd/0
     8 ?        00:01:10 rcu_sched
...
~$  # SHOW ONLY PROCESSES WITH TIME
~$ ps -e | awk '{if ($3 != "00:00:00") print $0}'
   PID TTY          TIME CMD
     1 ?        00:00:03 systemd
     8 ?        00:01:10 rcu_sched
    10 ?        00:00:06 migration/0
    15 ?        00:00:03 migration/1
...

systime() / strftime() – get time and format time:

These time functions allow you to add time stamps and then do formatting on the date/time string. I found this useful in logging and charting projects. An example to add a time stamp to the sensor’s CPU temperature would be:

$ sensors | grep CPU | awk '{print strftime("%H:%M:%S ",systime()) $1 $2 }'
 11:06:18 CPU:+45.0°C

Final Comments

I’ve found that learning a little bit of AWK has really paid off.

AWK supports a lot of functionality and it can be used to create full on scripting applications with user inputs, file I/O, math functions and shell commands, but despite all this I’ll stick to Python if things get complex.

Simulating the Mouse and Key Strokes

There are a number of tools that allow you to simulate mouse moments and key stokes. Applications for this could include:

  • testing of a user interface
  • communicating with an app where there isn’t an API
  • “Visual Macros” where a sequence of keys and entries are repeated

For this blog I will mostly focus on Linux but there are also tools in the Microsoft Windows world.

Microsoft Windows – Sendkey

Sendkey is a command line program that can be used in a batch file or Powerscript to do visual key and mouse automation.

Some useful short cut keys include:

  • Alt+Tab: Switch apps.
  • Alt+F4: Close apps.
  • Win+left arrow or Win+right arrow: Snap windows.
  • Win+Tab: Open the Task view.
  • Tab and Shift+Tab: Move backward and forward through options.
  • Ctrl+Esc: Open the Start menu.

The VBA Application object also supports the Sendkey method so it is possible to write scripts in standalone VBA or in Word or Excel.

Linux Mouse and Key Stroke Simulation

In the Linux environment there a number of choices, such as xautomation and xdotools.

I thought that the xdotools package was feature rich with a number of special functions for desktop and windows functions. The xautomation package was a little simpler and it has some Python wrappers.

I found that it was very useful to install wmctrl, this allows you to easily see which windows are running and it can set the active window with a substring (you don’t need the full window name).

To install xautomation and wmctrl in Ubuntu:

sudo apt-get install xautomation wmctrl

Use wmctrl -l to see which windows are open.

A specific window can have the focus based on a substring.

An example bash script to open a file (a Python file in Idle) and then write some text would be:

# Open a file, set it to the focus and write some text
testfile="dummy.py"

idle $testfile &

sleep 2 #Give Idle time to come up

wmctrl -a $testfile  #set the focus to dummy.py

xte 'str #this is a line from script'

Test a Visual Interface

As a second example I wanted to test a GUI program. For the GUI app I put a bug in the program so when the second button was pressed 10 times the app would close.

# simtest.py - two button test program
#   exit program if the 2nd button is pressed 10 times

import tkinter as Tkinter

top = Tkinter.Tk()
btcount = 0

def CallBack2():
    global btcount
    btcount += 1
    if btcount == 10:
         quit()
    label.config(text = btcount)

label = Tkinter.Label(top, text = "0" )
label.pack()

B1 = Tkinter.Button(top, width=50,text ="Button 1")
B1.pack()
dummy
B2 = Tkinter.Button(top, width=50, text ="Button 2", command = CallBack2)
B2.pack()
# bind the return key to the 2nd button
top.bind('<Return>', lambda event=None: B2.invoke())
top.mainloop()

The next step is to make a bash script that hits the two button indefinitely and then checks to see if the program is still running it. Finally if the program crashes the script needs to show some diagnostic information.

There are a lot of options on how to write this script, below is just one example:

# simkey2 - push buttons on a program and check app status

testfile="simtest.py"

# run the test program
python3 $testfile &

sleep 2 #give the program time to come up

wmctrl -a $testfile  #set the focus to the test file

# Loop until the test app crashes
while true; do
  xte  'key Tab' 'key Return'
  sleep 2
  # see if the app is still running
  isRun=$(ps -eo pid,cmd | grep $testfile | grep -v grep)

  # Quit if the app is no longer running
  #echo ${#isRun} $isRun 
  if [  ${#isRun} -eq 0 ]
  then
     clear
     echo "-----------------------------"
     echo "| "  $testfile  " crashed"
     echo "-----------------------------"
     break
  fi
  
done

Final Comments

Using simulated keyboard and mouse movements is a nice tool to have in your back pocket when you are looking at “closed” or older systems.

However there are some inherent problems when you try to simulate mouse and key boards such as:

  • You take over the PC. It can be challenging to do other functions
  • Doing a TAB to select the next enter-able area works great for “RETURN” key inputs, but it may not work for buttons that expect a mouse click.
  • Your testing script can not “see” the results so some thinking is required to determine if the operations at working correctly.

STOMP Protocol with RabbitMQ, Node-Red and Python

The STOMP (Simple Text Oriented Messaging Protocol) is a messaging system that is similar to MQTT (Message Queue Telemetric Transport) and AMQP (Advanced Message Queue Protocol).

STOMP uses a send and subscribe mechanism to pass messages and like other messaging systems, the STOMP server is called a broker. Open Source STOMP brokers are available with ActiveMQ and RabbitMQ.

In this blog I wanted to document my setup that used the STOMP plug-in to RabbitMQ and show examples of getting Python and Node-Red subscribing and sending data.

STOMP Specification logo

Why use STOMP?

MQTT and AMQP are very commonly used messaging protocol, so why use STOMP?

AMQP has a lot of great features but for many simple IoT (Internet of Things) applications AMQP may be overkill. Also AMQP is not supported with any mainstream Arduino libraries whereas Arduino Libraries exist for STOMP.

Both STOMP and MQTT are really light-weight, however STOMP has the advantage of being able to add headers and properties along with the message (a feature missing in MQTT but available in AMQP). So you could send a STOMP sensor message and include things like: quality, status, area, and comments along with the value.

STOMP messages can also be used on client side JavaScript web pages. This isn’t supported on AMQP but it possible using Websockets with MQTT.

 

RabbitMQ Setup with STOMP

RabbitMQ is a very popular open source messaging middle-ware. By default RabbitMQ loads AMQP (Advanced Message Queue Protocol) but it can also load MQTT and STOMP protocols. RabbitMQ offers Web Administration and an environment that allows different messaging protocols to intercommunicate.

RabbitMQ can be installed on Window, Linux, MacOS systems and there are also some cloud based offerings. For small systems lower end hardware like a Raspberry Pi can be used.  For complete RabbitMQ installation instructions see: https://www.rabbitmq.com/download.html . To install and run RabbitMQ on a Ubuntu system enter:

sudo apt-get update
sudo apt-get install rabbitmq-server
sudo service rabbitmq-server start

The next step is to add some plug-ins. For this project I loaded the STOMP plug-in and the Web Administration plug-in:

sudo rabbitmq-plugins enable rabbitmq_stomp
sudo rabbitmq-plugins enable rabbitmq_management

The rabbitmqctl command line tool allows you to configure and review the RabbitMQ server. To add a user admin1,  with password admin1, that has config, write and read rights for management and administrator access, enter:

sudo rabbitmqctl add_user admin1 admin1
sudo rabbitmqctl set_permissions -p / admin1 ".*" ".*" ".*"
sudo rabbitmqctl set_user_tags admin1 management administrator

At this point a super-user (admin1) has been defined and that user has been given rights to talk to all features of RabbitMQ including STOMP. See the STOMP documentation for STOMP specific user and system configuration.

After you’ve defined an administrative user the RabbitMQ web management plug-in can be accessed by: http://ip_address:15672 . The Overview tab should show STOMP running on the default port of 61613.

rabbitMQ_ov

For clarity it is good to define a RabbitMQ exchange that is to be used for STOMP. There are a lot of options but the easiest thing is to make a DIRECT exchange that is DURABLE (available after a restart).

rabbitMQ_ex

Python and STOMP

Python has a library component : stompy.py that is both a standalone STOMP testing client and a library component. To install the component:

 pip install stomp.py

Once loaded stomp.py can be used to connect to a server and test subscribing and send data. Below is an example connecting to host 192.168.0.105 with user: pete and password pete.

pete@lubuntu:~$ stomp -H 192.168.0.105 -U pete -W pete


> subscribe /exchange/stomp1/tag1
Subscribing to '/exchange/stomp1/tag1' with acknowledge set to 'auto', id set to '1'
> send /exchange/stomp1/tag1 hi from python
> 
message-id: T_1@@session-8xhCjj0giLAiOFg8E6OGPQ@@1
subscription: 1

hi from python
> 

For this example the exchange stomp1 was defined. The topic tag1 was dynamically defined and a message was sent and subscribe to. The topic (tag1) doesn’t need to be defined in RabbitMQ.

A Python STOMP test program would be:

 #  
 # stomp_test.py - test STOMP  
 #  
 import time  
 import sys  
   
 import stomp  
 import random  
   
 class MyListener(stomp.ConnectionListener):  
   def on_error(self, headers, message):  
     print('received an error "%s"' % message)  
   def on_message(self, headers, message):  
     print('received a message "%s"' % message)  
   
 # Define a STOMP connection and port  
 conn = stomp.Connection([("192.168.0.105", 61613)])  
 conn.set_listener('', MyListener())  
 conn.connect('pete', 'pete', wait=True) # define the username/password  
   
 # Setup a subscription  
 conn.subscribe(destination='/exchange/stomp1/tag1', id=1, ack='auto')  
   
 while True:  
   time.sleep(15) # send a random message every 15 seconds  
   conn.send(body=str(random.randint(1,11)), destination='/exchange/stomp1/tag1')  
   
 #conn.disconnect()  

 

Node-Red and STOMP

The Node-Red STOMP node can be installed using the menu option “Manage Palette”  or manually:

cd $HOME
cd ~/.node-red 
npm install node-red-node-stomp

A simple Node-Red test circuit would include:

  • an injector node – manually send in a test string
  • a STOMP output node – to send messages to a broker
  • a STOMP input node – subscribe to messages
  • a debug node – to show messages in the debug tab

nr_sample

The logic requires the STOMP node to be defined, and the STOMP nodes need to have a valid exchange and topic:

Node-Red can be used to configure Web dashboards that can show trending and charting of STOMP values.

 

Testing STOMP with RabbitMQ

Once I had both Python and Node-Red working with STOMP I was able to go back to my RabbitMQ and view the connections.

rabbitMQ_conn

It is also possible to use RabbitMQ to publish STOMP messages. In this example I included in the header: status, quality and units.

rabbitMQ_ex_pub

I then was able to check the message in Node-Red.

nr_msg_debug

 

Final Comments

Compared to AMQP and MQTT there aren’t nearly as many examples so I needed to do a little bit of experimentation. As an example I found that if I send and subscribe to a queue (use: /queue/my_queue)  this was only a 1-to-1 connection, if I had 2 subscribers each subscriber would only get 1/2 the messages. So I’m recommend using RabbitMQ exchanges for STOMP messaging.

This was a quick introduction to STOMP with Node-Red and Python. I still need to look at Arduino and Javascript connections.

 

 

 

Julia programming on a Raspberry Pi

Julia is a free and open-source general purpose programming language made specifically for scientific computing.

In this blog I wanted to document my Pi Julia interface testing.

Getting Started

Julia is supported on Windows, MacOS and Linux. See the download documentation for your specific system at : https://julialang.org/downloads/. To install Julia on a Raspberry Pi enter:

sudo apt install julia

To check your install, you can run Julia at the command line:

pi@raspberrypi:~ $ julia
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.0.3
 _/ |\__'_|_|_|\__'_|  |  Raspbian ⛬  julia/1.0.3+dfsg-4+rpi1
|__/                   |

julia> 

Julia packages can be installed at the command line by:

julia> using Pkg
julia> Pkg.add("some package")

Julia with Raspberry Pi GPIO

A Raspberry Pi package “PiGPIO” is installed by:

julia> using Pkg
julia> Pkg.add("PiGPIO")

Once the package is installed a daemon ( pigpiod) is created that the Julia script connects to. The pigpiod daemon is started by:

sudo pigpiod

Below is an example script that cycles a GPIO pin. For this example the GPIO pin is passed as a keyboard input. The readline() method returns a string, so parse() is used get an integer.

# pi1.jl - GPIO test 
using PiGPIO 

print("Enter the GPIO pin: ")

n = readline()
led_pin = parse(Int64,n) # convert the string input to an Integer

println("Pin used: $led_pin ")


#led_pin = 4 #use BCM pin 4 (physical pin 7) 
p=Pi() #connect to pigpiod daemon on localhost 
set_mode(p, led_pin, PiGPIO.OUTPUT) 
println("Julia is cycling an LED 5 times") 

try 
    for i in 1:5 # cycle 5 times 
        PiGPIO.write(p, led_pin, PiGPIO.HIGH) 
        sleep(0.5) 
        PiGPIO.write(p, led_pin, PiGPIO.LOW) 
        sleep(0.5) 
    end 
finally 
    println("Cleaning up!") 
    # if you wish to leave the pin as an input 
    #set_mode(p, led_pin, PiGPIO.INPUT) 
end

To run the code enter:

pi@raspberrypi:~ $ julia pi1.jl
Enter the GPIO pin: 4
Pin used: 4
[ Info: Successfully connected!
Julia is cycling an LED 5 times
Cleaning up!

Raspberry Pi Inputs

A simple GPIO read input programs is:

# pi2.jl - GPIO input test 

using PiGPIO

p = Pi() #connect to pigpiod daemon on localhost 

in_pin = 23
set_mode(p, in_pin, PiGPIO.INPUT)

println("\nInput status on Pin 23...")
println("hit Cntl-C to break out")

println(PiGPIO.read(p, in_pin))

while true
    println(PiGPIO.read(p, in_pin))
    sleep(1)
end

 

The output will look something like:

pi@raspberrypi: $ julia pi2.jl
[ Info: Successfully connected!

Input status on Pin 23...
hit Cntl-C to break out
1
1
0
0

Final Comments

As a Python user l found using Julia to be  frustrating, the documentation is weak and there aren’t the extensive libraries that I’m used to using in Python.

Neopixel Hats

There are some fun wearable projects that you can do with small Arduino modules like the Gemma ($9) and some neopixels. ($7).

 

Conductive thread is used to both secure the modules to the material and to do the wiring.

For this project we wanted to have a sequence of colour patterns, so we had:

  • a rainbow moving around in a circle
  • a red smiley face, that had a left/right smirk and then a wink
  • a rainbow moving around in a circle
  • a red smiley face, that had a left/right smirk and then a wink

However there a tons of possible patterns that could be used.

For the battery mounting there are a few options such as lipo batteries, coin batteries and small battery packs.

Below is some example code for the moving rainbow that we used.

Have Fun

#include <Adafruit_NeoPixel.h>

#define PIN 1
int theLED = 0;


Adafruit_NeoPixel strip = Adafruit_NeoPixel(12, PIN, NEO_GRB + NEO_KHZ800);

void setup() {
  strip.begin();
  strip.setBrightness(20);
  strip.show(); // Initialize all pixels to 'off'
}

void loop() {
  // Create a rainbow of colors that moves in a circle 

  int col;
  
     for (int i=0; i < strip.numPixels(); i++) {
        strip.setPixelColor(i, 0, 0, 255);
     }
    theLED = theLED + 1;
    if (theLED >= strip.numPixels()) {
      theLED = 0; 
    }
    strip.setPixelColor(theLED, 209, 31, 141);
    strip.setPixelColor(theLED - 1, 11, 214, 180);
    strip.setPixelColor(theLED - 2, 240, 210, 127);
    strip.setPixelColor(theLED - 3, 191, 127, 240);
    strip.show();
    delay (500);
}

Control Rasp Pi’s with Simple Lua GUIs

I was struggling to find a simple Lua graphic library. Love2D appears to be well regarded, but I wanted to find something that I could get up and running fast.

An old 1980’s graphic technology called curses has been available for years in most languages and I was familiar with it from C and Python.

In this blog I wanted to shared an example of using the Lua curses library to read and write Raspberry Pi general purpose I/O (GPIO).

Installing Lua

To install Lua on a Raspberry Pi:

sudo apt-get update
sudo apt-get install lua5.1
sudo apt-get install liblua5.1-0-dev -- development files, need by LuaRocks
sudo apt-get install lua-socket
sudo apt-get install luarocks -- package manager for Lua modules

sudo luarocks install luasocket

Lua has a package manager called luarocks, (this is similar to pip on Python), where you can install custom libraries or packages on the Pi.

There are a number of choices on how Lua can access Pi GPIO pin. I found that the lua-periphery library to be a reliable option. The Lua version of curses is not 100% compatible to the C version but it’s close.

To install these libraries enter:

sudo luarocks install lua-periphery
sudo luarocks install curses

Raspberry Pi Hardware

I used a Pimoroni Explorer Hat because it has some built in colored LEDs, but you could easily use some LEDs and resistors and wire your own equivalent setup.

 

For some details on how to use the Lua Raspberry Pi GPIO library see: https://funprojects.blog/2019/04/20/lua-and-raspberry-pi/

The Lua Curses App

My goal was to create a simple GUI with a title and a footer with the key commands, then show the values on the screen.

lua_curses_screen

To use colored text there are a few steps that are required:

  • enable color (curses.start_color())
  • define some color pairs (curses.init_pair)
  • create an attribute variable that is defined by a color pair(a_red = curses.color_pair(4))

Then use the attribute “ON” function to set the color  (stdscr:attron(a_red)).

The mvaddstr function is used to write text to position on the screen  object. (stdscr:mvaddstr(2, 5,”SET RASPBERRY PI LEDS” )).

Below is my code to setup 4 LED outputs, and use the keys 1,2,3 and 4 to write to these outputs. The “q” key is used to exit the code.

 -- A Lua curses example with some Raspberry Pi Data  
 -- Define Rasp Pi variables  
 local GPIO = require('periphery').GPIO  
 local gpio_in = GPIO(10, "in")  
 local led1 = GPIO(4,"out")  
 local led2 = GPIO(17,"out")  
 local led3 = GPIO(27,"out")  
 local led4 = GPIO(5,"out")  
 led1:write(1)  
 led2:write(1)  
 led3:write(1)  
 led4:write(1)  
 -- Define curses  
 local curses = require 'curses'  
 curses.initscr()  
 curses.echo(false) -- not noecho !  
 local stdscr = curses.stdscr() -- the screen object  
 -- setup color pairs and attribute variables  
 curses.start_color()  
 curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)  
 curses.init_pair(2, curses.COLOR_WHITE, curses.COLOR_BLACK)  
 curses.init_pair(3, curses.COLOR_BLUE, curses.COLOR_BLACK)  
 curses.init_pair(4, curses.COLOR_YELLOW, curses.COLOR_BLACK)  
 curses.init_pair(5, curses.COLOR_RED, curses.COLOR_BLACK)  
 curses.init_pair(6, curses.COLOR_GREEN, curses.COLOR_BLACK)  
 a_rw = curses.color_pair(1)  
 a_white = curses.color_pair(2)  
 a_blue = curses.color_pair(3)  
 a_yellow = curses.color_pair(4)  
 a_red = curses.color_pair(5)  
 a_green = curses.color_pair(6)  
 stdscr:clear()  
 -- Create a background  
 ncols = curses.cols()  
 nrows = curses.lines()  
  
 -- Create a top and bottom color strip  
 stdscr:attron(a_rw) -- set the fore/background colors  
 for i=0, (ncols - 1), 1 do -- write a top and bottom strip  
      stdscr:mvaddstr(0,i, " ")  
      stdscr:mvaddstr(nrows -1,i, " ")  
 end  
 stdscr:mvaddstr(0,0, " Curses Lua Dynamic Text Example")  
 stdscr:mvaddstr((nrows -1), 0, " Key Commands: q - to quit, 1,2,3,4 - to toggle LED")  
 -- Add the main screen static text  
 stdscr:attron(a_white) -- set the fore/background colors  
 stdscr:mvaddstr(2, 5,"SET RASPBERRY PI LEDS" )  
 for i=1,4,1 do   
      stdscr:mvaddstr(4+ i, 5, "LED " .. tostring(i) .. " : " )  
 end  
 stdscr:refresh()  
 local c = stdscr:getch ()  
 while c ~= 113 do -- 113 = q ,quit  
      if c == 49 then led1:write(not led1:read()) end  
      if c == 50 then led2:write(not led2:read()) end  
      if c == 51 then led3:write(not led3:read()) end  
      if c == 52 then led4:write(not led4:read()) end  
      -- show the inputs  
      stdscr:attron(a_blue)  
      stdscr:mvaddstr(5, 15, tostring(led1:read() ) .. " " )  
      stdscr:attron(a_yellow)  
      stdscr:mvaddstr(6, 15, tostring(led2:read() ) .. " " )  
      stdscr:attron(a_red)  
      stdscr:mvaddstr(7, 15, tostring(led3:read() ) .. " " )  
      stdscr:attron(a_green)  
      stdscr:mvaddstr(8, 15, tostring(led4:read() ) .. " " )  
      c = stdscr:getch ()  
 end  
 curses.endwin()  

Some Final Comments

Unfortunately I found the Lua curses documentation to be quite weak and there were very few examples.

My only major stumbling block was to find a stdscr.nodelay() function that allows the code to continue without waiting for a key stroke. This feature exists in the Python and C libraries.

Simple Terminal Interfaces

Typically our interfaces for projects use colorful web pages or custom GUIs. However there are many cases where a simple text interface is all that is required. This is especially true for SSH or remote connections from a Window’s client into a Raspberry Pi or Linux server.

In this blog I’d like to review a 1980’s technology called curses, with three examples. The first example will be simulated Rasp Pi scanning app in “C” and Python. The second and third examples will be in Python and they will show large text presentation and dynamic bars.

Python Curses

Python curses are standard in Python, and they include features such as:

  • support ASCII draw characters
  • basic color support
  • window and pad objects which can be written to and cleared independently

As a first example I wanted to have a colored background, a header and footer and some dynamic text.

curses_text

The first step is to define a curses main screen object (stdscr). The next step is to enable color and to create some color pairs. Using color pairs and the screen size (height, width = stdscr.getmaxyx()) it is possible to add a header and footer strip using the srtdscr.addstr command.

The stdscr.nodelay command allow the program to cycle until the stdscr.getch() call returns a key.

# curses_text.py - create a curses app with 10 dynamic values
#
import curses , time, random

# create a curses object
stdscr = curses.initscr()
height, width = stdscr.getmaxyx() # get the window size

# define two color pairs, 1- header/footer , 2 - dynamic text, 3 - background
curses.start_color()
curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)
curses.init_pair(3, curses.COLOR_WHITE, curses.COLOR_BLUE)

# Write a header and footer, first write colored strip, then write text
stdscr.bkgd(curses.color_pair(3))
stdscr.addstr(0, 0, " " * width,curses.color_pair(1) )
stdscr.addstr(height-1, 0, " " * (width - 1),curses.color_pair(1) )
stdscr.addstr(0, 0, " Curses Dynamic Text Example" ,curses.color_pair(1) )
stdscr.addstr(height-1, 0, " Key Commands : q - to quit " ,curses.color_pair(1) )
stdscr.addstr(3, 5, "RASPBERRY PI SIMULATED SENSOR VALUES" ,curses.A_BOLD )
stdscr.refresh()

# Cycle to update text. Enter a 'q' to quit
k = 0
stdscr.nodelay(1)
while (k != ord('q')):
# write 10 lines text with a label and then some random numbers
for i in range(1,11):
    stdscr.addstr(4+ i, 5, "Sensor " + str(i) + " : " ,curses.A_BOLD )
    stdscr.addstr(4+ i, 20, str(random.randint(10,99)) ,curses.color_pair(2) )
    time.sleep(2)
    k = stdscr.getch()

curses.endwin()

The simulated Pi values will refresh every  10 seconds until the “q” key is pushed and then the terminal setting are returned to normal (curses.endwin()) and the program exits.

“C” Curses Example

For this “C” example I used a Raspberry Pi. The curses library needs to be installed by:

 sudo apt-get install libncurses5-dev

The curses syntax is similar between “C” and Python but not 100%. For example in Python the addstr command includes a color pair reference, but in “C” this is not supported so an attribute on/off (attron/attroff) command is used to reference the color pair. Below is the “C” code:

/* c1.c - Basic Curses Example */

#include <curses.h>
#include <stdlib.h>
#include <unistd.h>

int main(void)
{
    int row, col, k;    
// Create a curses object and define color pairs
    initscr();
    getmaxyx(stdscr,row,col);
    start_color();
    init_pair(1,COLOR_RED,COLOR_WHITE);
    init_pair(2,COLOR_GREEN,COLOR_BLACK);
    init_pair(3,COLOR_WHITE,COLOR_BLUE);
    curs_set(0);
    noecho();
    //keypad(stdscr,TRUE);
    nodelay(stdscr, TRUE);
// Write a header and footer, first write colored strip, then write text
    bkgd(COLOR_PAIR(3));
    attron(COLOR_PAIR(1));
// Create a top and bottom color strip
    for (int i = 0; i < col; i++) {
        mvaddstr(0, i,  " ");
        mvaddstr(row-1, i,  " ");
    }
    mvaddstr(0, 0,  " Curses C Dynamic Text Example");
    mvaddstr(row-1, 0,  " Key Commands: q - to quit");
    attroff(COLOR_PAIR(1));   
    mvaddstr(2, 5,"RASPBERRY PI SIMULATED SENSOR VALUES" );
    refresh();
// Cycle with new values every 2 seconds until a q key (133) is entered    
    while (k != 113)
    {
        attroff(COLOR_PAIR(2));
        for (int i = 0; i < 10; i++) {
            mvprintw((4+i), 5,  " Sensor %d : ",i);
        }
        attron(COLOR_PAIR(2));
        for (int i = 0; i < 10; i++) {
            mvprintw((4+i), 20,  "%d",rand() %100);
        }
        k = getch();
        sleep(2);
    }
    endwin();
    exit(0);
}

To compile and run the program (c1.c) enter:

gcc -o c1 c1.c -lncurses
./c1

The “C” example should look very similar to the earlier Python example.

Figlet for Large Custom Text

Large Custom Text can be generated using the Python Figlet library.  Figlet has an extensive selection of text presentations and it uses standard ASCII character to generate the large text presentations. The Figlet library is installed by:

pip install pyfiglet

An example from the Python shell:

pyshell_figlet

For a Figlet example, I wanted to create a large heading and a large dynamic value.

curses_di

The Figlet library can be used to generate a string with user defined texted presented a large text-like format. A little bit of testing is required because the Figlet generated text can be 3,4,5 or more characters tall and the string needs to be added to very left end of the window.

# curses_di.py - show a large heading and large dynamic value
#
import curses, time
import pyfiglet, random

def get_io():
    global value1
    testvalue = str(random.randint(100,1000)/10) + " C"
    value1 = pyfiglet.figlet_format(testvalue, font = "starwars" )

# Create a string of text based on the Figlet font object
title = pyfiglet.figlet_format("Raspi Data", font = "small" ) 

stdscr = curses.initscr() # create a curses object
# Create a couple of color definitions
curses.start_color()
curses.init_pair(1, curses.COLOR_YELLOW, curses.COLOR_BLACK)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)

# Write the BIG TITLE text string
stdscr.addstr(1,0, title,curses.color_pair(1) )
stdscr.addstr(8,0, "Sensor 1: GPIO 7 Temperature Reading" ,curses.A_BOLD)

# Cycle getting new data, enter a 'q' to quit
stdscr.nodelay(1)
k = 0
while (k != ord('q')):
    get_io() # get the data values
    stdscr.addstr(10,0, value1,curses.color_pair(2) )
    stdscr.refresh()
    time.sleep(2)

    k = stdscr.getch()

curses.endwin()

I found that the the small and doom fonts worked well in my testing. To check out and test Figlet fonts online see:

http://patorjk.com/software/taag/#p=display&f=Slant&t=Dude%20what%20are%20you%20doing%20%3F

Curses Windows

By defining a curses window it is possible to clear and write to a window that it is independent from the background. The syntax to create a curses window object is:

mynewwindow = curses.newwin(height, width, begin_y, begin_x)

Windows are ideal for applications where multiple items such as Figlet objects are used. Below is an example with two large Figlet values.

Figlet2win


# Create a static 2 large values example
#
import curses, time
import pyfiglet, random
# Create a string of text based on the Figlet font object
title = pyfiglet.figlet_format("Weather Station 2", font = "small" )

stdscr = curses.initscr() # create a curses object
# Create a couple of color definitions
curses.start_color()
curses.init_pair(1, curses.COLOR_YELLOW, curses.COLOR_BLACK)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)

# Write the BIG TITLE text string
stdscr.addstr(1,0, title,curses.color_pair(1) )
stdscr.refresh()

win1 = curses.newwin(9, 44, 6, 4)
win1.addstr(8,0, "Sensor 1: Temperature Reading" ,curses.A_BOLD)

win2 = curses.newwin(9, 44, 6, 50)
win2.addstr(8,0, "Sensor 2: Humidity Reading" ,curses.A_BOLD)
value1 = pyfiglet.figlet_format("23 C", font = "doom" )
win1.addstr(0,0,value1,curses.color_pair(2) )
win1.refresh()
value2 = pyfiglet.figlet_format("35 %", font = "doom" )
win2.addstr(0,0, value2 ,curses.color_pair(2) )
win2.refresh()

# Hit any key to exit
stdscr.getch()
curses.endwin()

Dynamic Bars Example

For the Dynamic bars example I created a get_io function to simulate two real time data  values.

As a first step I created some background information such as headings, a header and a footer. By using the call: height, width = stdscr.getmaxyx() , I am able to position banners at the top and bottom of the terminal window. All of the background info is written to the stdscr object.

Two windows objects (win1 and win2) are used for the real time dynamic bars. Old bar data is removed using the win1.clear() and win2.clear() calls. Like the static example the dynamic bars are created by writing a fill character multiplied by the actual real time value (win1.addstr(1, 1, bar * value1) ). A window.refresh() command is used to show the changes.

The stdscr.getch() method is used to catch keyboard input, and the terminal program is exited when a quit character, “q” is entered.

The complete two dynamic bar program is shown below:


# Simple bar value interface
#
import curses
import time

# get_io is using random values, but a real I/O handler would be here
def get_io():
    import random
    global value1, value2
    value1 = random.randint(1,30)
    value2 = random.randint(1,30)

bar = '█' # an extended ASCII 'fill' character
stdscr = curses.initscr()
height, width = stdscr.getmaxyx() # get the window size
curses.start_color()
curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)
curses.init_pair(3, curses.COLOR_YELLOW, curses.COLOR_BLACK)

# layout the header and footer
stdscr.addstr(1,1, " " * (width -2),curses.color_pair(1) )
stdscr.addstr(1,15, "Raspberry Pi I/O",curses.color_pair(1) )
stdscr.hline(2,1,"_",width)
stdscr.addstr(height -1,1, " " * (width -2),curses.color_pair(1) )
stdscr.addstr(height -1,5, "Hit q to quit",curses.color_pair(1) )

# add some labels
stdscr.addstr(4,1, "Pi Sensor 1 :")
stdscr.addstr(8,1, "Pi Sensor 2 :")

# Define windows to be used for bar charts
win1 = curses.newwin(3, 32, 3, 15) # curses.newwin(height, width, begin_y, begin_x)
win2 = curses.newwin(3, 32, 7, 15) # curses.newwin(height, width, begin_y, begin_x)

# Use the 'q' key to quit
k = 0
while (k != ord('q')):
    get_io() # get the data values
    win1.clear()
    win1.border(0)
    win2.clear()
    win2.border(0)
# create bars bases on the returned values
    win1.addstr(1, 1, bar * value1, curses.color_pair(2))
    win1.refresh()
    win2.addstr(1, 1, bar * value2 , curses.color_pair(3))
    win2.refresh()
# add numeric values beside the bars
    stdscr.addstr(4,50, str(value1) + " Deg ",curses.A_BOLD )
    stdscr.addstr(8,50, str(value2) + " Deg ",curses.A_BOLD )
    stdscr.refresh()
    time.sleep(2)
    stdscr.nodelay(1)
    k = stdscr.getch() # look for a keyboard input, but don't wait

curses.endwin() # restore the terminal settings back to the original

cbars

For testing I used a random simulator for the data but the get_io function could be easily configured to connect to a Raspberry Pi or Arduino module.

The outline boxes in the window object could look strange if you are using a Window’s based SSH client like Putty. To create the problem in Putty’s settings, select: Window ->  Translations and use VSCII as the remote character set.

putty

Final Comments

Curses is definitely an ‘old school’ technology but it offers a simple solution for SSH and terminal based connections.