I have been using the BBC Micro:bit modules (~$20) with our kids to teach them basic hardware and software. The platform offers a block programming interface (very much like Scratch) and Python programming. The interesting thing is that the interface will toggle seamlessly between the two programming languages.
To add functionality to your projects Micro:bit supports extensions. This feature allows Arduino and Raspberry Pi sensors and devices to be connected to the basic module.
In this blog I wanted to:
Show a extensions example with deviices that are not directly available in the basic list
Comment on some limitations
Document how to add Micro:bit parts in Fritzing wire drawing tool
An Extension Example
Use the Extension menu item to add a new set of functionality to your Micro:bit’s project.
For this example I wanted to use some devices/sensors that I was using on my Arduino projects. These devices included:
It is important to note that the extension may not be readily available from the Microbit web page. For my project I did an Internet search to find the required github links. Once you have the URL it can be pasted into the Extension’s page:
Below is a picture of the Micro:bit with the three added devices
The Micro:bit logic used an on_start block to setup the pins for the TM1637 4-digit display, and initialize the OLED display.
The forever block:
queried the DHT11 sensor (on Pin 0)
showed the humidity on the Micro:bit display
showed the temperature on the TM1637 display
showed both the temperature and humidity on the 0.91″ OLED
cycled every 5 seconds
The block code can be viewed (or edited)in Python:
"""
Devices:
DHT11 Temperature/Humidity Sensor
TM1637 4-Digit Display
I2C 9.91" OLED Display
Show Temperature on TM1637
Show Humidity on Microbit screen
Show both Temperature and Humidity on the the OLED
"""
tm = TM1637.create(DigitalPin.P13, DigitalPin.P14, 7, 4)
MuseOLED.init()
def on_forever():
dht11_dht22.query_data(DHTtype.DHT11, DigitalPin.P0, True, False, True)
basic.show_string("H: " + str(dht11_dht22.read_data(dataType.HUMIDITY)) + " %")
tm.show_number(dht11_dht22.read_data(dataType.TEMPERATURE))
MuseOLED.clear()
MuseOLED.write_string("Temperature: " + str(dht11_dht22.read_data(dataType.TEMPERATURE)) + " C")
MuseOLED.new_line()
MuseOLED.write_string("Humidity: " + str(dht11_dht22.read_data(dataType.HUMIDITY)) + " %")
basic.pause(5000)
basic.forever(on_forever)
Limitations
Some of the limitations that I found were:
Not all Arduino sensors and devices were supported
Not all Arduino functionality is available with Micro:Bit. For example fonts on OLED devices.
Finding the correct extension can be tricky. For example searching 0.91 OLED doesn’t return any hits.
Some devices were supported in software, however they required 5V. A good example of this is the 2×16 LCD display
Documenting Wiring in Fritzing
Fritzing is an excellent free tool for wiring drawings (Note: for some platforms a donation might be required).
Woob offers a common interface for accessing a variety of different Internet data sources. So rather than using a specific API to access to weather and then another to access a job boards, the Woob API/tools can be used for both of these data sources.
Woob can be used as a:
Python library,
A command line tool, or by
Bash scripting
In this blog I wanted to document my Linux notes on:
basic command line tool usage
Bash/Python examples for getting weather data
Bash/Python examples for playing Internet streaming music
Note: WOOB is in development so features may change. Also there is a large list of application modules, but at present most are European based.
Getting Started
To install woob:
$ pip install woob
Once woob is installed a list of features can be shown by entering woob:
$ woob
usage: woob [--version] <command> [<args>]
Use one of this commands:
bands display bands and suggestions
bank manage bank accounts
bill get/download documents and bills
books manage rented books
bugtracker manage bug tracking issues
calendar see upcoming events
cinema search movies and persons around cinema
cli call a method on backends
config manage backends or register new accounts
contentedit manage websites content
dating interact with dating websites
debug debug backends
gallery browse and download web image galleries
gauge display sensors and gauges values
geolocip geolocalize IP addresses
housing search for housing
job search for a job
lyrics search and display song lyrics
money import bank accounts into Microsoft Money
msg send and receive message threads
parcel manage your parcels
paste post and get pastes from pastebins
pricecompare compare products
radio search, show or listen to radio stations
recipes search and consult recipes
repos manage a woob repository
rpg manage RPG data
shop obtain details and status of e-commerce orders
smtp daemon to send and check messages
subtitles search and download subtitles
torrent search and download torrents
translate translate text from one language to another
travel search for train stations and departures
video search and play videos
weather display weather and forecasts
For more information about a command, use:
$ man woob-<command>
or
$ woob <command> --help
Each features has a backend (database) associated with it. For example to define the weather.com backend for the weather command:
$ woob weather
Warning: there is currently no configured backend for weather
Do you want to configure backends? (Y/n): y
Available modules:
1) [ ] ilmatieteenlaitos Get forecasts from the Ilmatieteenlaitos.fi website
2) [ ] lameteoagricole lameteoagricole website
3) [ ] meteofrance Get forecasts from the MeteoFrance website
4) [ ] weather Get forecasts from weather.com
a) --all-- install all backends
q) --stop--
Select a backend to create (q to stop): 4
Backend "weather" successfully added.
If a backend needs to be removed, for example meteofrance, use the backend remove command:
$ wood weather backend remove meteofrance
To check which backends are defined, use the backend list command:
$ woob weather backend list
Enabled: weather
$ woob radio backend list
Enabled: freeteknomusic, somafm
Using the Weather Command
Woob offers a number of weather modules. A good generic option is weather.com.
To get started at the command line, enter: woob weather
$ woob weather
Welcome to weather v3.0
Copyright(C) 2010-2022 Romain Bignon
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.
Type "help" to display available commands.
Loaded backends: weather
weather> help
Weather Commands:
cities PATTERN
current CITY_ID
debug
forecasts CITY_ID
Woob Commands:
formatter [list | FORMATTER [COMMAND] | option OPTION_NAME [on | off]]
select [FIELD_NAME]... | "$direct" | "$full"
ls [-d] [-U] [PATH]
cd [PATH]
condition [EXPRESSION | off]
count [NUMBER | off]
backends [ACTION] [BACKEND_NAME]...
quit
logging [LEVEL]
Type "help <command>" for more info about a command.
The first step is to search for a city with the cities option. Once a city is found the id (1-20) is used to show the current weather and the future forecast.
The example below searches for Toronto. The first item (Toronto, Ontario, Canada) is selected (current 1) to show the present Toronto weather. Toronto’s future weather can be checked with the forecast command (forecast 1).
A scripting example to show the current and forecast weather for Toronto (the first item in the list):
$ # use the -n 1 option to get the first item
$ woob weather cities 'Toronto' -n 1
d8ccf908e3c4c748e232720575df7cdbca6e0f1b412bca8595d8a28d0c28e8bc@weather — Toronto, Ontario, Canada
$ # get the id (first string) and save to a variable
$ city_id=$(woob weather cities 'Toronto' -n 1 | awk '{print $1}')
$ # pass the city id to get the weather current
$ woob weather current $city_id
2022-12-08: -1 °C - 1030.2hPa (Rising) - humidity 73% - feels like -4 °C/24 °F - Fair
An example is to get the current weather for a place and show it as a Linux system tray notification:
$ # Create a notification note with current weather information
$ my_city="Sauble Beach"
$ city_id=$(woob weather cities "$my_city" -n 1 | awk '{print $1}')
$ notify-send -t 1000000 "$my_city" "$(woob weather current $city_id )
To show the long term forecast information in a Zenity dialog:
$ # Create an Info Dialog with weather forecast information
$ my_city="Sauble Beach"
$ city_id=$(woob weather cities "$my_city" -n 1 | awk '{print $1}')
$ wf=$(woob weather forecast $city_id)
$ zenity --width=650 --info --text="$wf" --title="$my_city"
Python Woob Weather Example
A Python example to select the first city in a list and then show the forecast would be:
#!/usr/bin/python3
#
# wweather.py - Search for weather forecast for a place
#
from woob.core import Woob
from woob.capabilities.weather import CapWeather
w=Woob()
w.load_backends(CapWeather)
# Find the id for a location
for city in w.iter_city_search("Sauble Beach"):
# stop at first hit
print(city.id, city.name)
break
# Get the Forecast for that location
for w0 in w.iter_forecast(city.id):
print("Date:", w0.date)
print("Forecast:", w0.text)
print("High:", w0.high)
print("Low", w0.low, "\n")
There are a number of radio modules to choose from. The first step is to add at least one module:
$ woob radio
Warning: there is currently no configured backend for radio
Do you want to configure backends? (Y/n): y
Available modules:
1) [ ] audioaddict Internet radios powered by audioaddict.com services
2) [ ] bandcamp Bandcamp music website
3) [ ] freeteknomusic freeteknomusic website
4) [ ] ina INA French TV video archives
5) [ ] nectarine Nectarine Demoscene Radio
6) [ ] nova Nova French radio
7) [ ] ouifm OÜI FM French radio
8) [ ] radiofrance Radios of Radio France: Inter, Info, Bleu, Culture, Musique, FIP, Le Mouv'
9) [ ] somafm SomaFM web radio
10) [ ] virginradio VirginRadio french radio
a) --all-- install all backends
q) --stop--
Select a backend to create (q to stop): 9
Backend "somafm" successfully added.
After one of more radio modules are added, you can search for radio stations that play a genre:
$ woob radio
Welcome to radio v3.0
Type "help" to display available commands.
Loaded backends: freeteknomusic, somafm
radio> search radio trance
id: thetrip@somafm
title: The Trip
description: Progressive house / trance. Tip top tunes.
current: B With U (Salinas' Dub Mix) - Tommyboy And Soultan Feat. Zara
streams: ['130Kbps/aac' ('http://somafm.com/thetrip130.pls'), 'fast/mp3' ('http://somafm.com/thetrip.pls'), '64Kbps/aacp' ('http://somafm.com/thetrip64.pls'), '32Kbps/aacp' ('http://somafm.com/thetrip32.pls')]
radio:/search> play 1
For this radio station search of “trance” only 1 station was found, so play 1 is used to listen to that station. (Note: if 3 were returned then you could listen to the third station by: play 3).
Bash Scripting for Internet Radio
There are some woob options that will make scripting a little easier. Some genres like “rock” or “ambient” will return too many hits, so the -n option can be set to limit the number of returned items. The -s option will return only selected fields.
$ # Show the id and description
$ # for the first 2 ambient station
$ woob radio search radio "ambient" -s id,description -n 2
id: deepspaceone@somafm
description: Deep ambient electronic, experimental and space music. For inner and outer space exploration.
id: groovesalad@somafm
description: A nicely chilled plate of ambient/downtempo beats and grooves.
An example script (wlisten.sh) takes a music genre entered on the command line and then finds and plays the first related station.
#!/bin/bash
#
# wlisten.sh - find a Woob Radio station and play it
#
echo -e "\nWoob Internet Radio Player\n"
echo -n "Enter a type of Music to listen to: "
read mtype
echo "Now playing : $mtype music ..."
woob radio play $(woob radio search radio "$mtype" -n 1 -s id | awk '{print $2}' )
To run this script (wlisten.sh) to play a reggae station:
$ bash wlisten.sh
Woob Internet Radio Player
Enter a type of Music to listen to: reggae
Now playing : reggae music ...
This example only uses the first returned station. By using a couple of Zenity list dialongs a Bash script can be created to present a list of preset genres, then the user can select a specific station.
#!/bin/bash
#
# wradio.sh - find a Woob Radio station and play it
#
mtype=$(zenity --title="Play Internet Radio" \
--list --height=500 --width=300 --column="Music Type" \
80s Ambient Dance House Jazz Pop Rock Reggae Top Trance)
# if a music type is selected then get stations
echo "Music Type: $mtype"
if [[ -n "$mtype" ]] ; then
stn=$(woob radio search radio $mtype -f radio_list | zenity --list \
--title="Select Radio Station" \
--column=station --column=description --width=900 --height=300)
# Get the station ID, its the 1st part of the station string
stn_id="${stn%%—*}"
# If the station string is not empty, play the station
if [[ -n "$stn_id" ]] ; then
echo "Station Id: $stn_id"
woob radio play $stn_id
fi
fi
Python Woob Radio Example
To get a selection of radio stations for a genre:
from woob.core import Woob
from woob.capabilities.radio import CapRadio
w = Woob()
w.load_backends(CapRadio)
genre="rock"
for stns in w.iter_radios_search(genre):
print(stns.id, stns.title, stns.description, "\n")
A Python Tkinter GUI example to select a genre and then play a radio station:
#!/usr/bin/python3
#
# wradio.py - Tkinter GUI for Woob Radio
#
from tkinter import *
from tkinter import ttk
import subprocess
from woob.core import Woob
from woob.capabilities.radio import CapRadio
w = Woob()
w.load_backends(CapRadio)
# Fill the tree with genre stations
def show_stations():
genre = cbo.get()
tree.delete(*tree.get_children())
# Insert Rows into tree based on found radio stations
for stns in w.iter_radios_search(genre):
tree.insert('','end',text=stns.id,values=(stns.id, stns.title, stns.description))
# Play selected station
def tree_getitem(a):
curItem = tree.focus()
stn_info=tree.item(curItem) # station is dictionary variable
thestn=stn_info['text'] # get the station id
p=subprocess.Popen(["killall", "woob"])
p=subprocess.Popen(["killall", "mpv"])
if thestn != "":
labelText.set("Playing Station: " + thestn)
p=subprocess.Popen(["woob", "radio", "play", thestn])
def stop_music():
labelText.set("")
p=subprocess.Popen(["killall", "woob"])
p=subprocess.Popen(["killall", "mpv"])
# Create an instance of tkinter frame
root = Tk()
genre="Rock"
root.title('Play Internet Radio Stations')
root.geometry('900x300')
# Create a listbox of station genres
label1 = ttk.Label(text = "Enter a genre: ").grid(row=0,column=0)
cbo = ttk.Combobox(root,height = 1)
genres= {"80s","Ambient", "Dance", "House","Jazz","Pop","Rock", "Reggae","Trance"}
cbo['value'] = [m for m in genres]
cbo.grid(row=0, column=1)
# Create a button to update station tree
bt_update = ttk.Button(root,text="Select Genre", command=show_stations)
bt_update.grid(row=0, column=2)
# Create a tree list with 3 columns
tree = ttk.Treeview(root, column=("Id","Station", "Desciption"), show='headings')
tree.column("#1", width=100)
tree.heading("#1", text="ID")
tree.column("#2", width=200)
tree.heading("#2", text="Station")
tree.column("#3", width=600)
tree.heading("#3", text="Description")
tree.grid(row=1, column=0, columnspan=3)
tree.bind('<ButtonRelease-1>', tree_getitem)
# Create a label to show station being played
labelText = StringVar()
label2 = ttk.Label(textvariable=labelText).grid(row=2,column=0)
# Add a button to start and stop music
bt_stop = ttk.Button(root,text="Stop Music", command=stop_music)
bt_stop.grid(row=2,column=1)
root.mainloop()
Summary
The Woob project has a lot of potential, however because it is still in the development stage it can be a little challenging to use.
Due to the lack of examples, I found that I was able to get up and running faster with Bash scripts rather than Python. To build a Bash script I would first test what I wanted to do in the online tool, then I would mimic this in Bash.
The Gemini protocol is a very simple and light Internet protocol.
Unlike HTML files that contain layers of tags, style sheets and Javascript, a Gemini document is a simple readable document.
In this blog I wanted to document:
Gemini servers and clients using only 1 line of Bash
How to use large ASCII text in Gemini documents
How to create simple bar charts
How to create Gemini CGI apps in Python
Getting Started
A Gemini document only supports a few statements and graphic images such as JPEG or PNG are not supported.
An example Gemini document with the common formatting options:
# Heading level 1 (H1)
## Heading level 2 (H2)
### Heading level 3 (H3)
=> testpage.gmi A link to another page.
> This line will show as a block-quote.
A list of items
* This is the first list item.
* This is another list item.
```
Code or ASCII Block
_ ___ ___ ___ ___ _____ _
/_\ / __|/ __|_ _|_ _| |_ _|__ __| |_
/ _ \\__ \ (__ | | | | | |/ -_|_-< _|
/_/ \_\___/\___|___|___| |_|\___/__/\__|
```
Within a Gemini browser this file would look like:
Content Type
The content type is used by browsers and applications to determine how to manage the requested file.
Typically the content type is managed by server, for example a web server will send a HTTP/1.0 200 OK prior to sending the HTML file.
For the Gemini protocol the content type is: 20 text/gemini . Depending on the Gemini server the user may need to be add the content type manually. (More about this in Bash and CGI servers).
Simple Bash Gemini Servers and Clients
For basic testing a one line Bash statement can be used for custom Gemini servers and clients.
The Gemini protocol typically uses SSL (Secure Sockets Layer) and TLS (Transport Layer Security) encryption so the Bash ncat utility is needed (Note: the nc command does not support SSL).
Below is an example of single Gemini request:
The Gemini server is defined by using the -l , listen option. When a client requests data, the cat statement is used with a pipe (|) to output the file testpage.gmi.
The Gemini client echo’s a “from PC” message with its request, this helps identify which client is requesting data.
A simple Bash ncat statement doesn’t manage the content type so a “20 text/gemini” line is added to the top of the test page.
Dynamic Bash Data
In the earlier example the Bash server is only supporting 1 request then exiting.
A while loop can be added to pass a Bash script to the ncat statement. Below is an example of Bash script (showdata.sh) that show CPU data using the vmstat utility:
#!/bin/bash
#
# showdata.sh - Output data for Gemini Bash Server
#
echo "20 text/gemini"
echo " "
echo "#VMSTAT"
echo " "
date +"%T"
echo " "
# set Gemini formating for ASCII
echo "\`\`\`"
#vmstat
top -n 1
echo "\`\`\`"
To make the script executable use the command: chmod +x showdata.sh
The Bash command to run this script as a Gemini server is:
while true; do ./showdata.sh | ncat -l -p 1965 --ssl; done
The earlier Bash Gemini client command can be used, or a Gemini browser/client app can be used. The handling of SSL/TLS encryption will vary with the client app. I used the very basic Zain app (https://gitgud.io/sathariel/zain) :
(Note: for the Zain client I needed to load tcl/tls, sudo apt-get install -y tcl-tls)
Using a 1 line Bash Gemini server is great for basic testing but I wouldn’t recommend if you want to connect to variety of different Gemini client applications.
Large ASCII Text
Gemini documents don’t support different font sizes, a workaround is to use the figlet tool to generate multi-line text strings. Figlet is installed on Ubuntu/Debian/Raspbian by:
sudo apt install figlet
Figlet has a number of font styles that use 2-5 line height characters:
When using ASCII headings the Gemini code formatting option should be used, and this has three backticks (“`) before and after the headings.
The earlier example can be modified to have ASCII headings:
#!/bin/bash
#
# showdata2.sh - Output Large Headings to a Gemini Bash Server
#
echo "20 text/gemini"
echo " "
echo "\`\`\`"
# Generate large text
figlet -f standard "Raspberry Pi"
figlet -f small -m 2 $(date +"%T")
# show CPU stats
vmstat
echo "\`\`\`"
Bar Charts
Horizontal ASCII bar charts can be created by using the printf statement with different ASCII fill characters. For example:
# show a label with bar and value text
#
label="temp"; val=20;
bar="printf '█%.0s' {1..$val}" ;
printf '\n%-5s ' $label; eval $bar
printf '░%.0s' {1..5} ;
printf ' 50 C\n'
temp ████████████████████░░░░░ 50 C
This bar logic can be using in a Raspberry Pi Stats page that looks at idle time and free space on the SD card:
#!/bin/bash
#
# pi_stats.sh - Output Pi Stats as a Gemini Page
#
echo "20 text/gemini"
echo " "
# Put the rest of the Gemini document into code block mode
echo "\`\`\`"
# Generate large text
figlet -f standard "Pi Stats"
# Show the time
echo "Time: $(date +'%T')"
echo ""
# Get idle time, scale 0-50
idle=$(top -n 1 | grep id | awk '{print $8}')
barsize=$(echo $idle | awk '{printf "%0.f" , $1/2}')
thebar="printf '█%.0s' {1..$barsize}"
graysize=$(expr 50 - $barsize)
thegray="printf '░%.0s' {1..$graysize}"
printf 'Idle Time '; eval $thebar; eval $thegray ; echo " $idle %"
echo ""
# Get free space on SD card, scale 0-50
freesp=$(df | grep root | awk '{printf "%0.f", $5/2}')
barsize=$(echo $freesp | awk '{printf "%0.f" , $1/2}')
thebar="printf '█%.0s' {1..$barsize}"
graysize=$(expr 50 - $barsize)
thegray="printf '░%.0s' {1..$graysize}"
printf 'Free Space '; eval $thebar; eval $thegray ; echo " $freesp %"
echo "\`\`\`"
To run this page use
while true; do ./pi_stats.sh | ncat -l -p 1965 --ssl; done
Python CGI Pages
There are a number of good Gemini servers, for my testing I used the Python based Jetforce server, it is installed by:
pip install jetforce
To run the Jetforce server it is important to define a home directory, the allowable hosts that can connect (0.0.0.0 is all IP4 nodes) and the server’s hostname:
# Start the jetforce Gemini server for all IP4 hosts
jetforce --dir /home/pi/temp --host "0.0.0.0" --hostname 192.168.0.105 &
# Start jetforce without hardcoding hostname
# jetforce --dir /home/pi/temp --host "0.0.0.0" --hostname $(hostname -I) &
By default CGI (Common Gateway Interface) files are defined in the directory cgi-bin which is under the home directory.
Unlike a one-line Bash server, Jetforce server will pass environment variables like QUERY_STRING and host and remote connection information.
CGI programs can be written in a variety of programming languages. For this example I wanted to pass Raspberry Pi BME280 temperature, pressure and humidity sensor information to a Gemini CGI page.
The CGI program was written in Python and I installed a bme280 and figlet library:
pip install RPi.bme280
pip install pyfiglet
The Python script (sersor2gmi.py) outputs a Gemini content type, a Figlet title and then the sensor data values:
#!//usr/bin/python3
#
# sersor2gmi.py - send BME280 sensor data to a Gemini page
#
import smbus2
import bme280
import pyfiglet
# find device address via: i2cdetect -y 1
port = 1
address = 0x77
bus = smbus2.SMBus(port)
calibration_params = bme280.load_calibration_params(bus, address)
# the sample method returns a compensated_reading object
data = bme280.sample(bus, address, calibration_params)
# Output a Gemini page
print("20 text/gemini") #Note: Some Gemini CGI servers may do this
print("```") # use code mode
#print(pyfiglet.figlet_format("Pi BME280 Data", font = "small"))
print(pyfiglet.figlet_format("Pi BME 280 Data", font = "small"))
print("Temperature:" + "{:5.1f}".format(data.temperature) + " C" )
print("Pressure: " + "{:5.1f}".format(data.pressure) + " kPa" )
print("Humidity: " + "{:5.1f}".format(data.humidity) + " %" )
print("```")
This file was added to the cgi-bin directory and it is made executeable (chmod +xsersor2gmi.py).
Below is the output seen in the Lagrange Gemini browser:
Final Comments
There are a variety of Gemini browsers for Windows, Android, Mac and Linux so if you’re looking for a quick and dirty internet solution Gemini might be a good solution.
I like how Gemini documents are totally readable, I can’t say the same for most web pages.
The one thing that I missed with Gemini pages is the ability of show nice charts, text based bars work for simple stuff but doing text based line charts is a little too ugly for me.
There are some great tutorials on SQL, SQLite and Matplotlib, however I wanted something super simple that I could use to teach my daughters with.
This blog documents my notes on what I used to teach them SQL, SQLite3 and then how to plot the results in Python. Yes it can only takes 6 lines of Python code to read an SQLite3 file and plot it, if you are doing 2 line charts with a title and legends its 9 lines.
Getting Started with SQL
There are some great SQL databases and SQL tools that can are quite user friendly.
For us I started with SQLite3 which is file based, no server component is required and it runs on all major OS’s and Raspberry Pi’s. There are a number of SQL Query builder tools but we found the DB Browser for SQLite gave us all the functionality we needed.
Check the install documentation for your hardware/OS, to install SQLite3 and DB Browser on Debian/Ubuntu/Raspberry Pi’s :
For testing there are a lot of excellent databases that can downloaded from Kaggle. These data files are in CSV format and they can be imported into a SQLite database file using the DB Browser.
A new database file (sars.db) was created and then the CSV was imported.
Create VIEWS to Simplify
SQL Views can be created to simply the field names, reduce the number of fields or add custom fields.
Use the “Execute SQL” tab to create a view. An example to create a view with simplified field names would be:
Views and SELECT queries can be generated to add custom fields. An example to extract the month from the date field and add 2 other fields (month number and month name):
Test SQL Queries
The DB Browser tool is good for testing out SQL queries and plot the data before moving to Python.
A query to find the worst 5 countries affected by SARS would:
select sum(deaths) sum_deaths, country from v_sars group
by Country having sum_deaths > 0 order by sum_deaths desc limit 5
Plotting in Python
There are number of ways to plot SQL data in Python. I found that the easiest way was to use Pandas dataframes.
The Python code to connect to SQLite3, run the SQL query (of sum of deaths vs. country) and plot the data is:
#
# sql_bar0.py - Sqlite to Bar Charts
#
import sqlite3, pandas , matplotlib.pyplot as plt
conn = sqlite3.connect("/media/pete/RED_4GB/sql/sars.db")
sql = """select sum(deaths) sum_deaths, country from v_sars group
by Country having sum_deaths > 0 order by sum_deaths desc limit 5"""
data = pandas.read_sql(sql, conn)
#x values: data.Country, y values: data.sum_deaths
plt.bar(data.Country, data.sum_deaths)
plt.title("SARS Death in 2003")
plt.show()
An example with 2 lines to shows the monthly deaths and cases would be:
#
# sql_line2.py - Sqlite to 2 Line Charts
#
import sqlite3, pandas , matplotlib.pyplot as plt
conn = sqlite3.connect("/media/pete/RED_4GB/sql/sars.db")
sql = """select s_month, sum(deaths) as sum_deaths, sum(cases) as sum_cases from v_month group by n_month"""
data = pandas.read_sql(sql, conn)
plt.plot(data.s_month,data.sum_deaths, label = "Deaths")
plt.plot(data.s_month,data.sum_cases, label = "Cases")
plt.legend()
plt.title("SARS Death in 2003")
plt.show()
Summary
By keeping the Python code simple we were able to focus on SQL queries.
Using the basic Python code the SQL connection we later changed from SQLite3 to MySQL or Progresql.
Industrial operations such as chemical refineries, power plants and mineral processing operations have quite different communications requirements than most IT installations. Some of the key industrial communications requirements include: security, multi-vendor connectivity, time tagging and quality indications.
To meet industrial requirements a communications standard called OPC (OLE for Process Control) was created. The original OPC design was based on Microsoft’s Object Linking and Embedding (OLE) and it quickly became the standard for communications between control systems consoles, historians and 3rd party applications.
The original OPC standard worked well but it had major limitations in the areas of Linux/embedded systems, routing across WANs, and new security concerns. To better address new industrial requirements the OPC UA, (Open Platform Communications Unified Architecture) standard was created.
In this article I will create an OPC UA server that will collect sensor data using Python and Node-Red, and the results will be shown in a Node-Red web dashboard.
Install Python OPC UA Server
There are a number of OPC UA open source servers to choose from.
For “C” development applications see the Open62541 project (https://open62541.org/), it offers a C99 architecture that runs on Windows, Linux, VxWorks, QNX, Android and a number of embedded systems.
For light weight quick testing OPC UA servers are available in Python and Node-Red.
The Free OPC-UA Library Project (https://github.com/FreeOpcUa) has a great selection of open source tools for people wishing to learn and play with OPC UA.
I keep things a little simple I will be using the python-opcua library which is a pure Python OPC-UA Server and client. (Note: a more complete Python OPCUA library, https://github.com/FreeOpcUa/opcua-asyncio, is available for more detailed work). Also an OPC-UA browser is a useful tool for monitoring OPC UA server and their tags. To load both of these libraries:
# Install the pure Python OPC-UA server and client
sudo apt install python-opcua
# Install the OPC UA client and the QT dependencies
sudo apt install PyQT*
pip3 install opcua-client
Simple Python OPC-UA Server
As a first project a simple OPC-UA server will be created to add OPC-UA tags and then simulate values.
The first step in getting this defined is to set an endpoint or network location where the OPC-UA server will be accessed from.
The default transport for OPC-UA is opc.tcp. The Python socket library can be used to determine a node’s IP address. (To simplify my code I also hard coded my IP address, opc.tcp://192.168.0.120:4841).
The OPC-UA structure is based on objects and files, and under an object or file tags are configured. Tags by default have properties like value, time stamp and status information, but other properties like instrument or alarm limits can be added.
Once a tag object is define, the set_value function is used to simulate the tag values.
# opcua_server1.py - Create an OPC UA server and simulate 2 tags
#
import opcua
import random
import time
s = opcua.Server()
s.set_server_name("OpcUa Test Server")
s.set_endpoint("opc.tcp://192.168.0.120:4841")
# Register the OPC-UA namespace
idx = s.register_namespace("http://192.168.0.120:4841")
# start the OPC UA server (no tags at this point)
s.start()
objects = s.get_objects_node()
# Define a Weather Station object with some tags
myobject = objects.add_object(idx, "Station")
# Add a Temperature tag with a value and range
myvar1 = myobject.add_variable(idx, "Temperature", 25)
myvar1.set_writable(writable=True)
# Add a Windspeed tag with a value and range
myvar2 = myobject.add_variable(idx, "Windspeed", 11)
myvar2.set_writable(writable=True)
# Create some simulated data
while True:
myvar1.set_value(random.randrange(25, 29))
myvar2.set_value(random.randrange(10, 20))
time.sleep(5)
The status of the OPC-UA server can be checked using the OPC-UA browser:
# start the Python OPC-UA browser client
opcua-client
Items within an OPC-UA server are defined by their name space index (ns) and their object index. The name space index is returned after an name space is register. An object’s index is defined when a new object is create. For this example the Windspeed tag has a NodeId of “ns-2;i=5”, or an index 5 on name space 2.
The opcua-client application can view real-time changes to a tag’s value using the subscription option.
In OPC the terms “tags” and “variables” are often used interchangeably. In the instrument lists the hardware signals are usually referred to as “tags”, but within the OPC UA server the term “variables” is used. The key difference is that a variable can also be an internal or soft point such as a counter.
The Python client app (station1.py) defines an OPC-UA client connection and then it uses the NodeId definition of the Temperature and Windspeed tags to get their values:
# station1.py - Put OPC-UA data into gauges
#
import tkinter as tk
import tk_tools
import opcua
# Connect to the OPC-UA server as a client
client = opcua.Client("opc.tcp://192.168.0.120:4841")
client.connect()
root = tk.Tk()
root.title("OPC-UA Weather Station 1")
# Create 2 gauge objects
gtemp = tk_tools.Gauge(root, height = 200, width = 400,
max_value=50, label='Temperature', unit='°C')
gtemp.pack()
gwind = tk_tools.Gauge(root, height = 200, width = 400,
max_value=100, label='Windspeed', unit='kph')
gwind.pack()
def update_gauge():
# update the gauges with the OPC-UA values every 1 second
gtemp.set_value(client.get_node("ns=2;i=2").get_value())
gwind.set_value(client.get_node("ns=2;i=5").get_value())
root.after(1000, update_gauge)
root.after(500, update_gauge)
root.mainloop()
XML Databases
In the earlier Python OPC-UA server example tags were dynamically added when the server was started. This method works fine for simple testing but it can be awkward for larger tag databases.
All industrial control vendors will have proprietary solutions to create OPC-UA tag databases from process control logic.
Users can also create their own tag databases using XML. The OPC-UA server tag database can be imported and exported to XML using the commands:
# to export from the online system to an XML file:
# where: s = opcua.Server()
s.export_xml_by_ns("mytags.xml")
# to import an XML file:
s.import_xml("mytags2.xml","")
The XML files can be viewed in a web browser, and unfortunately the format is a little ugly. The XML files have a header area with a large number of options.The Name Space Uris is the custom area that defines the OPC UA end point address.
After the header there are object and variable definitions (<AUVariable>). In these section the variable’s NodeID, tag name and description are defined.
The Free OPC-UA modeler that can help with the creation of XML tag databases. To install and run the Free OPC-UA modeler:
$ pip install opcua-modeler
$ opcua-modeler
The OPC-UA modeler will read existing XML files and then allow for objects, tags and properties to be inserted into the XML structure.
CSV to XML
A CSV file is an easy format for defining tag databases. For example a file mytags.csv could be defined with 3 fields; tagname, description and default value.
$ cat mytags.csv
# field: tag, description, default-value
TI-101,temperature at river, 25
PI-101,pressure at river, 14
A basic CSV to XML import tool can be created to meet your project requirements. There are a number of good programming options to do this migration. For my project I created a small Bash/AWK program to translate the 3 field CSV file to the required OPC-UA XML format.
The first awk section prints out the header information. The second awk section reads the input (CSV) text line by line and pulls out each of the three fields ($1, $2 and $3) and prints out the XML with these fields inserted in the output.
#!/usr/bin/bash
# csv2xml.sh - create an OPC UA XML file from CSV
#
# add the xml header info
awk ' BEGIN {
print "<?xml version=\"1.0\" encoding=\"utf-8\"?>"
print "<UANodeSet xmlns=\"http://opcfoundation.org/UA/2011/03/UANodeSet.xsd\""
print " xmlns:uax=\"http://opcfoundation.org/UA/2008/02/Types.xsd\""
print " xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\""
print " xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">"
print "<NamespaceUris>"
print " <Uri>http://192.168.0.120:4841</Uri>" ; # This address would be passed in
print "</NamespaceUris>"
}'
# Read the input CSV format and process to XML
awk ' {
FS="," ; # separate fields with a comma
# Skip any comment lines that start with a #
if ( substr($1,1,1) != "#" )
{
i = i+1 ; # increment the NodeID index
print "<UAVariable BrowseName=\"1:"$1"\" DataType=\"Int32\" NodeId=\"ns=1;i="i"\" ParentNodeId=\"i=85\">"
print " <DisplayName>"$1"</DisplayName>" ; # set the display name to the 1st field
print " <Description>"$2"</Description>" ; # set the description to the 2nd field
print " <References>"
print " <Reference IsForward=\"false\" ReferenceType=\"HasComponent\">i=85</Reference>"
print " </References>"
print " <Value>"
print " <uax:Int32>"$3"</uax:Int32>" ; # set the default value to the 3rd field
print " </Value>"
print "</UAVariable>"
}
}
END{ print "</UANodeSet>"} '
To run this script to read a CSV file (mytags.csv) and create an XML file (mytags.xml) :
cat mytags.csv | ./csv2xml.sh > mytags.xml
Node-Red OPC UA Server
There is a good OPC UA node (https://flows.nodered.org/node/node-red-contrib-opcua) that includes a server and most common OPC UA features. This node can be install within Node-Red using the “Manage Palette” option.
To setup a Node-Red OPC UA server and a browser, define a OPCUA server node to use the Node Red IP address and set a custom nodeset directory. For my example I set the directory to /home/pi/opcua and in this directory I copied the XML file that I created from CSV (mytags.xml) into.
The OPCUA Browser node will send messages directly into the debug pane. This browse node allows me to see the objects/variables that I defined in my XML file.
The next step is to look at writing and reading values.
The simplest way to communicate with an OPC UA server is to use an OpcUa Item node to define the NodeID and an OpcUa Client node to do some action. For the OpcUa Client node the End point address and an action needs to be defined.
In this example the pressure (PI-101) has a NodeID of “ns=5;i=2”, and this string is entered into the OpcUA item node. The OpcUA Client node uses a Write action. When a Write action is issued a Good or Bad status message is returned.
The OpcUa Client node supports a number of different actions. Rather than doing a Read action like in the Python client app, a Subscribe can be used. A Subscribe action will return a value whenever the value changes.
NodeRed Dashboards with the Python OPC UA Server
For the last example I will use the Python OPC UA server from the first example. The Temperature and WindSpeed will use the same simulation code, but an added Waveheight tag will be a manually entered value from Node-Red.
A Node-Red application that connects to the Python OPC UA server and presents that data in a Node-Red dashboard would be:
This example subscribes to two real-time inputs (Temperature and Windspeed) and presents the values in gauges. The OpcUA Item nodes define the OPC UA NodeId’s to be used.
All the OpcUa Client nodes will need their Endpoints defined to the Python OPC UA server address.
The subscribed data values are returned as a 2 item array (because the data type is a Int64). The Gauge node will only read the first payload array item, (which is 0) so a small function node copies the second payload item (msg.payload[1]) to the payload message:
// Copy the second payload array item to be the payload
// Note: msg.payload[0] = 0 and the Dashboard Gauge needs to use the value at payload[1]
msg.payload = msg.payload[1]
return msg;
For this example a manual input was included. The WaveHeight is subscribed to like the other tags, and the slider position is updated to its value. The slider can also be used to manually set the value by having the slider output passed to an OpcUa Client node with a WRITE action.
After the logic is complete the Deploy button will make the application live. The Node-Red dashboard can be viewed at: http://node-red-ip:1880/ui
Final Comments
This is a quick and dirty set of examples on how to use Python and Node-Red with OPC UA.
OPC UA has a ton of other features that can be implemented like : Alarms and Events and Historical Data.
Also it should be noted that most high end OPC UA servers support accessing the OPC UA items via their browse names. So instead of accessing a point using “ns=5;i=6” a browser name string can be used, such as “ns=5;s=MYTAGNAME”.
There are some nice Python packages like tk_tools, that can be used for IoT indicators and gauges.
My daughter and I had a project where we wanted to repurpose an old eReader to be a kitchen kiosk display. Unfortunately tk_tools doesn’t support Python 2.7, gray scale or larger text.
This blog documents how we made some simple update-able gauges using Python Tkinter Canvas objects that are supported in both Python 2.7 and 3.x .
Getting Started
Unfortunately the Python 2 and 3 Tkinter libaries are named differently (Tkinter in 2.7 vs tkinter in 3.x). If you are coding for both Python 2.7 and 3.x this gets messy, a simple workaround in your code is:
# Manage Python 2.7 and 3.x
#
import sys
# Check the version of Python and use the correct library
if sys.version_info[0] == 2:
import Tkinter
else:
import tkinter as Tkinter
Analog Clock
A Tkinter canvas supports a number of basic objects such as rectangles, circles, arcs, text, etc. The basic objects are positioned within the canvas space.
I found that as a first example an analog clock was a good place start. The first pass code for a clock with just the second hand would be:
# A Clock Second Hand Example
#
import tkinter as Tkinter # Python 3.x
import datetime
def update_sec():
#Reposition the second hand starting position
thesec = datetime.datetime.now().second
arcstart = 90 - thesec*6 #0 sec = 90deg
C.itemconfig(asec,start=arcstart) #pass the new start position
C.after(1000, update_sec)
# Create a canvas object with an oval face and a second hand
top = Tkinter.Tk()
C = Tkinter.Canvas(top, bg="silver", height=250, width=300)
C.pack()
coord = 10, 50, 240, 210
C.create_oval(coord, fill="white")
# Have the second hand start at the top (90 deg) with 1 deg arc
asec = C.create_arc(coord, start=90, extent=1, width=3)
C.after(1000, update_sec)
top.mainloop()
The key point is to get the id of the seconds hand arc (asec). The itemconfig method is then used to change the starting position of seconds hand arc (C.itemconfig(asec,start=arcstart) ).
The arc positioning is a little backwards, 0 degrees is at 3o’clock and then goes counter-clockwise.
The next step is to add narrow arcs for the minutes and hours. Also text could be used to digitally show the date and time. For the hour and minute hand I used different colours and thicknesses.
#
# A Clock Example
#
import tkinter as Tkinter # Python 3.x
from datetime import datetime
def update_sec():
# Position the hands
C.itemconfig(asec,start= 90 - datetime.now().second*6)
C.itemconfig(amin,start= 90 - datetime.now().minute*6)
C.itemconfig(ahour,start= 90 - datetime.now().hour*360/12)
C.itemconfig(dtime,text = datetime.now().strftime("%d/%m/%Y %H:%M:%S"))
C.after(1000, update_sec)
# Create a canvas object with an oval face and a second hand
top = Tkinter.Tk()
C = Tkinter.Canvas(top, bg="silver", height=250, width=300)
C.pack()
coord = 10, 50, 240, 210
C.create_oval(coord, fill="white")
# Have the second hand start at the top (90 deg) with 1 deg arc
asec = C.create_arc(coord, start=90, extent=1, width=2)
amin = C.create_arc(coord, start=90, extent=1, width=4, outline='blue')
ahour = C.create_arc(coord, start=90, extent=1, width=6, outline='red')
dtime = C.create_text(120,20, font="Times 16 bold", text="00:00:00")
C.after(1000, update_sec)
top.mainloop()
Gauges
There are a number of different types of gauges. My first example was a speedometer graph, that used an arc for both the background and the gauge needle:
#
# Use Canvas to create a basic gauge
#
from tkinter import *
import random
def update_gauge():
newvalue = random.randint(low_r,hi_r)
cnvs.itemconfig(id_text,text = str(newvalue) + " %")
# Rescale value to angle range (0%=120deg, 100%=30 deg)
angle = 120 * (hi_r - newvalue)/(hi_r - low_r) + 30
cnvs.itemconfig(id_needle,start = angle)
root.after(3000, update_gauge)
# Create Canvas objects
canvas_width = 400
canvas_height =300
root = Tk()
cnvs = Canvas(root, width=canvas_width, height=canvas_height)
cnvs.grid(row=2, column=1)
coord = 10, 50, 350, 350 #define the size of the gauge
low_r = 0 # chart low range
hi_r = 100 # chart hi range
# Create a background arc and a pointer (very narrow arc)
cnvs.create_arc(coord, start=30, extent=120, fill="white", width=2)
id_needle = cnvs.create_arc(coord, start= 119, extent=1, width=7)
# Add some labels
cnvs.create_text(180,20,font="Times 20 italic bold", text="Humidity")
cnvs.create_text(25,140,font="Times 12 bold", text=low_r)
cnvs.create_text(330,140,font="Times 12 bold", text=hi_r)
id_text = cnvs.create_text(170,210,font="Times 15 bold")
root.after(3000, update_gauge)
root.mainloop()
The basic gauge can be enhanced to have more value ranges and colour hihi/hi/low ranges:
#
# Use Canvas to create a basic gauge
#
from tkinter import *
import random
def update_gauge():
newvalue = random.randint(low_r,hi_r)
cnvs.itemconfig(id_text,text = str(newvalue) + " %")
# Rescale value to angle range (0%=120deg, 100%=30 deg)
angle = 120 * (hi_r - newvalue)/(hi_r - low_r) + 30
cnvs.itemconfig(id_needle,start = angle)
root.after(3000, update_gauge)
# Create Canvas objects
canvas_width = 400
canvas_height =300
root = Tk()
cnvs = Canvas(root, width=canvas_width, height=canvas_height)
cnvs.grid(row=2, column=1)
coord = 10, 50, 350, 350 #define the size of the gauge
low_r = 0 # chart low range
hi_r = 100 # chart hi range
# Create a background arc with a number of range lines
numpies = 8
for i in range(numpies):
cnvs.create_arc(coord, start=(i*(120/numpies) +30), extent=(120/numpies), fill="white", width=1)
# add hi/low bands
cnvs.create_arc(coord, start=30, extent=120, outline="green", style= "arc", width=40)
cnvs.create_arc(coord, start=30, extent=20, outline="red", style= "arc", width=40)
cnvs.create_arc(coord, start=50, extent=20, outline="yellow", style= "arc", width=40)
# add needle/value pointer
id_needle = cnvs.create_arc(coord, start= 119, extent=1, width=7)
# Add some labels
cnvs.create_text(180,15,font="Times 20 italic bold", text="Humidity")
cnvs.create_text(25,140,font="Times 12 bold", text=low_r)
cnvs.create_text(330,140,font="Times 12 bold", text=hi_r)
id_text = cnvs.create_text(170,210,font="Times 15 bold")
root.after(3000, update_gauge)
root.mainloop()
Our Final Project
Our final project had 4 gauges that were based on basic gauge code. Our Python app ran full screen on a Kobo eReader that we installed Debian Linux on. The app connected to our Home Assistant Pi and showed us our current weather conditions.
We had to tweek the basic code a little bit to account for the 800×600 screen size and grey scale graphics.
Summary
In this blog we only looked at some basic gauges, the Tkinter Canvas component can be used in a very variety of different applications such as: bar charts, real time charts, graphics etc.
littleBits is a set of electronic components that magnetically connect together. litteBits is geared towards the kids STEP market and it is available in many schools and libraries.
The littleBits company has done an excellent job making their product easy to use. There is a large variety of different “bit” modules and for Internet applications there is a Cloud Bit ($59).
I found that the Cloud Bit was very easy to get up and running, but I found it was expensive at $59 and somewhat limiting, for example you are only access 1-input and 1-output. So if you want to do 2 inputs/output you would need to purchase a second Cloud bit module.
In this blog I’d like to document how I used a $39 Arduino Bit to do 3-inputs and 3-outputs. I also had the code talk directly to a free Web Dashboard (AdaFruit).
littleBits Arduino Program
A set of commands needs to be setup between the littlebits Arduino module and the PC/Pi. In my Arduino program I referenced the ports A,B,C as inputs (on the left side), and D,E,F as outputs (on the right side).
The commands from the PC/Pi would be : reference_pin:value, for example D:255 would set the D (top left pin) at 100%. It’s important to note that Arduino inputs and outputs are scaled from 0-255.
For inputs the littleBits would send the results as pin: reference_pin:value, for example B:255 would be the result at full scale for the A0 input.
My test setup had:
A fork bit – this meant I only needed 1 power input source
A slider bit (0-1) on bit D0 (A)
A dimmer bit (0-255) on bit A0 (B)
A temperature bit on bit A1 (C)
An LED on bit d1 (D)
A number bit on D5 (E)
a bargraph bit on D9 (F)
Below is the littleBits Arduino program that managed the serial communications.
// littleBits_2_Dashboards - create a serial interface to read/write to a PC/Pi//
// Command from the littleBits: (A,B,C are the left pins)
// A:value <- for example B:24, pin A0 (2nd input) is 24
// Commands from the PC/Pi: (D,E,F are the right pins)// D:output <- for example E:128, set pin A0 to 50% (128/255)//Stringthecmd;Stringthevalue;Stringtheinput;charoutstring[3];voidsetup(){//define the littleBits right side pins 1,5 and 9 pinMode(1,OUTPUT);pinMode(5,OUTPUT);pinMode(9,OUTPUT);// define the littleBits left side inputspinMode(0,INPUT);pinMode(A0,INPUT);pinMode(A1,INPUT);Serial.begin(9600); // this needs to match the PC/Pi baud rate
}voidloop(){if(Serial.available()>0){thecmd=Serial.readStringUntil("\n");if(thecmd.length()>2){// ensure the msg size is big enoughthevalue=thecmd.substring(2);if(thecmd.startsWith("D")){analogWrite(1,thevalue.toInt());}if(thecmd.startsWith("E")){analogWrite(5,thevalue.toInt());}if(thecmd.startsWith("F")){analogWrite(9,thevalue.toInt());}}}// Try 3 different inputs: d0 = on/off , A0 = pot, A1 = temp sensorsprintf(outstring,"%d",digitalRead(0));Serial.write("A:");Serial.write(outstring);Serial.write("\n");sprintf(outstring,"%d",analogRead(A0));Serial.write("B:");Serial.write(outstring);Serial.write("\n");// A1 is an "i12" littleBits temperature sensorinttemp=analogRead(A1);temp=map(temp,0,1023,0,99);//rescale. Sensor range is 0-99 C or Fsprintf(outstring,"%d",temp);Serial.write("C:");Serial.write(outstring);Serial.write("\n");delay(5000);}
The Arduino IDE “Serial Monitor” can be used to view the output and set values.
Python on the PC or Raspberry Pi
The Arduino program will send input data for A,B,C every 5 seconds. This input can be seen in Python by:
#
# littleBits Read Test
#
import serial
ser = serial.Serial(port='/dev/ttyACM1', baudrate=9600) # format for Linux
#ser = serial.Serial(port='COM1', baudrate=9600) # format for Windows
while True:
inline = ser.readline()
inline = inline.decode() # make a string
pin = inline[0:1] # the first character is the pin
thevalue = inline[2:-1] # the value is between ":" and "\n"<span id="mce_SELREST_start" style="overflow:hidden;line-height:0;"></span>
print(pin,thevalue)
The output will look something like:
A 1
B 1023
C 21
To write commands from Python:
Write
#
# littleBits Write Test
#
import serial
ser = serial.Serial(port='/dev/ttyACM2', baudrate=9600) # format for Linux
#ser = serial.Serial(port='COM1', baudrate=9600) # format for Windows
while True:
print("\nWrite an output value to littleBit")
out = input("Enter pin:value, pin=A,B,C example: 'E:255' : ")
out = out.upper() + "\n"
out2 = out.encode('utf_8')
ser.write(out2)
Adafruit Dashboards
There are lots of good free dashboards. For this project I used the Adafruit site. To get started you will need to log in and create a free account.
I’ve bought a number of components from Adafruit. I think that they are an excellent company that goes out of their way to create great user guides and products.
The first step is to add some Adafruit tags that the code can read/write to.
In the Python code a couple of dictionaries (lb_inputs, lb_outputs) were created to link the littleBit references (A-F) with the Adafruit tags. Also two dictionaries (lb_inlast, lb_outlast) are used to minimize communications traffic so that only new values were written.
## Import standard python modulesimporttime,randomimportserial# import Adafruit IO REST clientfromAdafruit_IOimport Client, Feed, RequestError
ADAFRUIT_IO_USERNAME ="put_your_username_here"
ADAFRUIT_IO_KEY ="c039f24ecb6...xxxxx"
aio = Client(ADAFRUIT_IO_USERNAME, ADAFRUIT_IO_KEY)
# Create dictionaries of inputs, output, and last values
lb_inputs = {"A":"lb-slide", "B":"lb-dimmer","C": "lb-temp"}
lb_inlast = {"A":0, "B":0,"C": 0}
lb_outputs = {"D":"lb-led", "E":"lb-number", "F":"lb-bar"}
lb_outlast = {"D":0, "E":0,"F": 0}
# Setup the serial port connection
ser = serial.Serial(port='/dev/ttyACM1', baudrate=9600)
whileTrue:
# Get values from littleBits and write to the dashboard
inline = ser.readline()
inline = inline.decode() #inline should look like: A:125\n
pin = inline[0:1] # pin is the first character in string
thevalue = inline[2:-1] # value is between ":" and "\n"if lb_inlast[pin] != thevalue: # Write only new valuesprint(pin,thevalue, lb_inputs[pin])
ada_item = aio.feeds(lb_inputs[pin])
aio.send(ada_item.key,thevalue)
lb_inlast[pin] = thevalue
thetag ='lb-slide'# Write new dash values to littleBits if they've changedfor lbtag, dashtag in lb_outputs.items():
print (lbtag,dashtag)
thevalue = aio.receive(dashtag).value
if lb_outlast[lbtag] != thevalue: # Write only new values
outstr = lbtag +":"+ thevalue +"\n"print(outstr)
ser.write(outstr.encode('utf_8'))
lb_outlast[lbtag] = thevalue
time.sleep(2)
If everything is working correctly then new values should be written to in both directions. On the Adafruit Web site the Feed page should show the new values.
To make things look more presentable Adafruit Dashboards can be used.
Final Comments
In this project I used the Adafruit API, other good platforms would be IFTTT and Node-Red
Being a good dad I try and answer my kids science questions, but sometimes it’s really tough.
There is an awesome Python library called MetPy that can help with some of those challenging science and weather questions.
In this blog I’d like to introduce the MetPy library and show how to use it to solve questions like:
Can they make snow when it’s above freezing ?
How much thinner is the air in places like Denver?
How can you figure out what the wind chill is?
Getting Started with MetPy
To install MetPy:
pipinstallmetpy
One of nice things about MetPy is that it manages the scientific units, so variables are defined with their units. Below is a Python example where the units module is used for a simple temperature conversion. The temperature today is defined as 40 degF. The to()method can be used to convert the temperature to degC.
>>>> from metpy.units import units
>>> tempToday = [40] * units.degF
>>> tempToday.to(units.degC)
Quantity([4.44444444], 'degree_Celsius')
You can also do some interesting mixing of units in math calculations, for example you can add 6 feet and 4 meters:
MetPy has a very large selection of thermodynamic and weather related functions. In the next sections I will show how to use some of the these functions.
How can they make snow when it’s above freezing ?
Ski resort can make snow by forcing water and pressurized air through a “snow gun”. Making snow can be an expensive operation but it allows ski resorts to extend their season.
When you get a weather forecast the temperature is given as the ambient or dry bulb temperature. The wet bulb temperature takes the dry air temperature and relative humidity into account. The wet bulb temperature is always below the outside temperature. To start making snow a wet bulb temperature of -2.5°C or 27.5°F is required.
Metpy has a number of functions that can used to find humidity and wet bulb temperatures. The wet_bulb_temperature function will find the wet bulb temperature using the pressure, dry temperature and dew point.
Below is an example where the temperature is below freezing, but it’s not possible to make snow because the wet bulb temperature is only -0.6°C (not the required -2.5°C).
Knowing that -2.5°C (27.5°F) is the wet bulb temperature upper limit for snow making, the relative_humidity_wet_psychrometric function can be used to create a curve of humidity and dry temperature points where it is possible to make snow.
The code below iterates between -10 and 10 deg °C getting humidity values at a wet bulb temperature of -2.5°C.
#
# Find when you can make snow
#
import matplotlib.pyplot as plt
import metpy.calc
from metpy.units import units
print("Get temps vs. humidity")
print("-------------------")
plt_temp = []
plt_hum = []
for temp in range (-10,11): # Check dry temperatures between -10 - 10 C
dry_temp = [temp] * units.degC
# Get the relative humidity
rel_humid = metpy.calc.relative_humidity_wet_psychrometric(dry_temp, wet_temp,pres)
# Strip the humidity units for charting, and make a percent (0-100)
the_humid = rel_humid.to_tuple()[0] * 100
if (the_humid 0) : # Get valid points
plt_temp.append(temp) # append a valid temp
plt_hum.append(the_humid) # append a valid humidity
print (temp, the_humid)
fig, ax = plt.subplots()
ax.plot(plt_temp, plt_hum )
ax.set(xlabel='Temperature (C)', ylabel='Humidity (%)',
title='When you can make Snow')
ax.grid()
#fig.savefig("makesnow.png")
plt.show()
From the data we can see that it is possible to make snow when the temperature is above freezing and the humidity is low.
How much thinner is the air … in Denver?
We all know that the air is thinner when we’re up in an airplane, but how much thinner is it in Denver or Mexico City compared to New York City ?
Using the height_to_pressure_std function it is possible to get a pressure value based on altitude. The to() method can be used to convert the pressure to standard atmospheres.
Relative to New York City the air is about 18% thinner in Denver and 24% thinner in Mexico City.
Using the height_to_pressure_std function it is possible to create a chart of atmospheric pressure between sea level and the top of Mt. Everest (29,000 ft). At the top of Mt. Everest the air is 70% thinner than at sea level !!
#
# How much does the air thin as you climb ?
#
import matplotlib.pyplot as plt
import metpy.calc
from metpy.units import units
print("Get height vs. Atm Pressure")
print("---------------------------")
# create some plot variables
plt_ht = []
plt_press = []
# Check Atmospheric Pressure from sea level to Mt. Everest (29,000 ft) heights
for temp in range (0,30000,1000): # Check dry temperatures between -10 - 10 C
height = [temp] * units.feet
pressure = metpy.calc.height_to_pressure_std(height)
atm = pressure.to(units.atm)
print (height, atm)
plt_ht.append (height.to_tuple()[0]) # put the value into plt list
plt_press.append (atm.to_tuple()[0]) # put the value into plt list
fig, ax = plt.subplots()
ax.plot(plt_ht, plt_press )
ax.set(xlabel='Mountain Height (ft)', ylabel='Pressure (atm)',
title='How does the Pressure change Mountain Climbing?')
ax.grid()
fig.savefig("height_vs_press.png")
plt.show()
How can I figure out the Wind Chill ?
The MetPy windchill function will return a “feels like” temperature based on a wind speed and ambient temperature. For example an outside ambient temperature of 40 deg °F with a wind of 20 mph feel like 28 deg °F.
When the temperature starts going below -20 °C parents should be keep a close eye on their kids for frost bite. Below is a code example that shows a curve of -20 °C based on ambient temperature and wind.
#
# Wind Chill
#
import matplotlib.pyplot as plt
import metpy.calc
from metpy.units import units
# Create some plotting variables
plt_temp = []
plt_speed = []
for temp in range (0,-46,-1): # Check dry temperatures between -10 - 10 C
the_temp = [temp] * units.degC
for wind in range(1,61,1):
the_wind = [wind] * units.kph
windchill =
metpy.calc.windchill(the_temp,the_wind,face_level_winds=True)
# Select points with a wind chill around -20
if (windchill.to_tuple()[0]) = -20.1) :
plt_temp.append(temp)
plt_speed.append(wind)
fig, ax = plt.subplots()
ax.fill_between(plt_temp, plt_speed, label="-20 C - Wind Chill", color="red" )
ax.set(xlabel='Temperature (C)', ylabel='Wind Speed (kph)',
title='When is the Wind Chill -20C ?')
ax.grid()
fig.savefig("windchill.png")
plt.show()
Summary
MetPy didn’t solve all my kids questions but the Metpy library is an excellent for science questions around water and weather.
If you have a budding chemist or chemical engineer in your house try taking a look at the Python Thermo library.
There are a lot of Python web library choices, with each of the libraries offering different features and functions.
For simple Raspberry Pi Web application I was really impressed by the Python Bottle library. In this blog I’d like to document a Raspberry Pi rover project that I did using bottle.
Getting Started
To install the Python bottle library:
pip install bottle
There are some good tutorials for bottle. Probably the most important item is defining a decorator for a URL, and then linking the decorator to a function:
@route('/') # define a decorator for the start pagedef my_homefuntion() # call a function for the start page
return static_file("startpage.htm", root='') # call the start page
@route('/otherpage') # define a decorator for another pagedef my_otherpage_function() # call a function for the start page
# do some stuff...
It is not recommended to connect motors directly to Rasp Pi pin, for a few reasons:
larger motors require more power that a Pi GPIO pin can supply
power surges on motors can damage the Pi hardware
forward/backward motor directions require extra hardware
GPIO pins will only do ON/OFF, no variable speed settings.
There are a number of Raspberry Pi motor shield that can be used. For this project I used the Pimoroni Explorerhat Pro ($23). The Explorerhat has an excellent Python library that allows you to easily control the motor’s direction and speed.
Web Page
The goal for the web page (bottle1.htm) was to have 5 buttons for the controls and use AJAX to send the action request and then return the action code and display it on the page. The returned action would appear above the buttons ( in the lastcmd paragraph tag).
For this example the button push was sent in the header as a GET request. So a forward button push would be a myaction: forward header item. In previous projects I’ve used POST request with parameters, however I found that header items can make things a little simpler.
<!DOCTYPE html><html><head><title>Python Bottle Rover</title><metaname="viewport"content="width=device-width, initial-scale=1"><linkrel="icon"href="data:,"><style>html{font-family: Helvetica; display:inline-block; margin:0pxauto; text-align:center;}
h1{color:#0F3376; padding:2vh;}p{font-size:1.5rem;}
.button{display:inline-block; background-color:#4286f4;
border-radius:4px; color:white; font-size:30px; width:100%; height:75px}
.button2{background-color:green ;width:31%}
.stop{background-color:red; width:33%}
</style></head><script>function sendcmd(thecmd) {
// send the action as a header item var xhttp =new XMLHttpRequest();
xhttp.open("GET","/action" , true);
xhttp.setRequestHeader("myaction", thecmd);
xhttp.send()
xhttp.onreadystatechange =function() {
// Get the response and put on it the screenif (this.readyState ==4 ) {
document.getElementById("lastcmd").innerHTML ="Last Command:<b>"+xhttp.responseText;
}
}
}
</script><body><h2>Python Bottle Rover</h2><pid='lastcmd'></p><buttononclick="sendcmd('forward')"class="button">FORWARD</button><buttononclick="sendcmd('left')"class="button button2">LEFT</button><buttononclick="sendcmd('stop')"class="button stop">STOP</button><buttononclick="sendcmd('right')"class="button button2">RIGHT</button><buttononclick="sendcmd('backward')"class="button button">BACKWARD</button></body></html>
Bottle Rover App
For the rover app, there are two URL endpoints. The root (/) which would display the bottle1.htm page, and an action (/action) URL which would only be called from AJAX script when a button was pushed.
For this project the Raspberry Pi ip address was hardcoded into the code, for future projects dynamically getting the ip would be recommend. Also a web port of 8000 was used so as to not conflict with a dedicated web server (like Apache) that could be running on the Pi.
# Bottle2rover.py - web control for a RaspPi rover<span id="mce_SELREST_start" style="overflow:hidden;line-height:0;"></span>
#
from bottle import route, run, static_file, request
# Define RaspPi I/O pins
import explorerhat
# Send Action to Control Rover
def rover(action):
if action == "forward":
explorerhat.motor.one.speed(100)
explorerhat.motor.two.speed(100)
if action == "left":
explorerhat.motor.one.speed(100)
explorerhat.motor.two.speed(0)
if action == "right":
explorerhat.motor.one.speed(0)
explorerhat.motor.two.speed(100)
if action == "stop":
explorerhat.motor.one.speed(0)
explorerhat.motor.two.speed(0)
if action == "backward":
explorerhat.motor.one.speed(-50)
explorerhat.motor.two.speed(-50)
# use decorators to link the function to a url
@route('/')
def server_static():
return static_file("bottle1.htm", root='')
# Process an AJAX GET request, pass the action to the rocket launcher code
@route('/action')
def get_action():
action = request.headers.get('myaction')
print("Requested action: " + action)
rover(action)
return request.headers.get('myaction')
# Adjust to your required IP and port number
run(host = '192.168.0.106', port=8000, debug=False)
Final Comments
I was happily surprised how easy it was to get a Python bottle web app running. The bottle documentation was straightforward and I found that the code was quite lean.