Gemini Protocol for Lightweight Internet Apps

The Gemini protocol is a very simple and light Internet protocol.

Unlike HTML files that contain layers of tags, style sheets and Javascript, a Gemini document is a simple readable document.

In this blog I wanted to document:

  • Gemini servers and clients using only 1 line of Bash
  • How to use large ASCII text in Gemini documents
  • How to create simple bar charts
  • How to create Gemini CGI apps in Python

Getting Started

A Gemini document only supports a few statements and graphic images such as JPEG or PNG are not supported.

An example Gemini document with the common formatting options:

# Heading level 1 (H1) 
## Heading level 2 (H2)
### Heading level 3 (H3)

=> testpage.gmi A link to another page.

> This line will show as a block-quote. 

A list of items
* This is the first list item.
* This is another list item.

```
Code or ASCII Block 
   _   ___  ___ ___ ___   _____       _   
  /_\ / __|/ __|_ _|_ _| |_   _|__ __| |_ 
 / _ \\__ \ (__ | | | |    | |/ -_|_-<  _|
/_/ \_\___/\___|___|___|   |_|\___/__/\__|

```

Within a Gemini browser this file would look like:

Content Type

The content type is used by browsers and applications to determine how to manage the requested file.

Typically the content type is managed by server, for example a web server will send a HTTP/1.0 200 OK prior to sending the HTML file.

For the Gemini protocol the content type is: 20 text/gemini . Depending on the Gemini server the user may need to be add the content type manually. (More about this in Bash and CGI servers).

Simple Bash Gemini Servers and Clients

For basic testing a one line Bash statement can be used for custom Gemini servers and clients.

The Gemini protocol typically uses SSL (Secure Sockets Layer) and TLS (Transport Layer Security) encryption so the Bash ncat utility is needed (Note: the nc command does not support SSL).

Below is an example of single Gemini request:

The Gemini server is defined by using the -l , listen option. When a client requests data, the cat statement is used with a pipe (|) to output the file testpage.gmi.

The Gemini client echo’s a “from PC” message with its request, this helps identify which client is requesting data.

A simple Bash ncat statement doesn’t manage the content type so a “20 text/gemini” line is added to the top of the test page.

Dynamic Bash Data

In the earlier example the Bash server is only supporting 1 request then exiting.

A while loop can be added to pass a Bash script to the ncat statement. Below is an example of Bash script (showdata.sh) that show CPU data using the vmstat utility:

#!/bin/bash
#
# showdata.sh - Output data for Gemini Bash Server 
#
echo "20 text/gemini"
echo  " "
echo "#VMSTAT"
echo  " "
date +"%T"
echo  " "
# set Gemini formating for ASCII
echo  "\`\`\`"
#vmstat
top -n 1
echo  "\`\`\`"

To make the script executable use the command: chmod +x showdata.sh

The Bash command to run this script as a Gemini server is:

while true; do ./showdata.sh | ncat  -l -p 1965 --ssl; done

The earlier Bash Gemini client command can be used, or a Gemini browser/client app can be used. The handling of SSL/TLS encryption will vary with the client app. I used the very basic Zain app (https://gitgud.io/sathariel/zain) :

(Note: for the Zain client I needed to load tcl/tls, sudo apt-get install -y tcl-tls)

Using a 1 line Bash Gemini server is great for basic testing but I wouldn’t recommend if you want to connect to variety of different Gemini client applications.

Large ASCII Text

Gemini documents don’t support different font sizes, a workaround is to use the figlet tool to generate multi-line text strings. Figlet is installed on Ubuntu/Debian/Raspbian by:

sudo apt install figlet

Figlet has a number of font styles that use 2-5 line height characters:

When using ASCII headings the Gemini code formatting option should be used, and this has three backticks (“`) before and after the headings.

The earlier example can be modified to have ASCII headings:

#!/bin/bash
#
# showdata2.sh - Output Large Headings to a Gemini Bash Server 
#
echo "20 text/gemini"
echo  " "
echo  "\`\`\`"
# Generate large text 
figlet -f standard "Raspberry Pi"
figlet -f small -m 2  $(date +"%T")
# show CPU stats
vmstat
echo  "\`\`\`"

Bar Charts

Horizontal ASCII bar charts can be created by using the printf statement with different ASCII fill characters. For example:

# show a label with bar and value text 
#
label="temp"; val=20;
bar="printf '█%.0s' {1..$val}" ; 
printf '\n%-5s ' $label; eval $bar
printf '░%.0s' {1..5} ;
printf ' 50 C\n'

temp  ████████████████████░░░░░ 50 C

This bar logic can be using in a Raspberry Pi Stats page that looks at idle time and free space on the SD card:

#!/bin/bash
#
# pi_stats.sh - Output Pi Stats as a Gemini Page
#
echo "20 text/gemini"
echo  " "
# Put the rest of the Gemini document into code block mode
echo  "\`\`\`"
# Generate large text 
figlet -f standard "Pi Stats"

# Show the time
echo "Time: $(date +'%T')"
echo ""

# Get idle time, scale 0-50
idle=$(top -n 1 | grep id | awk '{print $8}')
barsize=$(echo $idle | awk '{printf "%0.f" , $1/2}')
thebar="printf '█%.0s' {1..$barsize}"
graysize=$(expr 50 - $barsize)
thegray="printf '░%.0s' {1..$graysize}"
printf 'Idle Time  '; eval $thebar; eval $thegray ; echo " $idle %"
echo  ""

# Get free space on SD card, scale 0-50
freesp=$(df | grep root | awk '{printf "%0.f", $5/2}')
barsize=$(echo $freesp | awk '{printf "%0.f" , $1/2}')
thebar="printf '█%.0s' {1..$barsize}"
graysize=$(expr 50 - $barsize)
thegray="printf '░%.0s' {1..$graysize}"
printf 'Free Space '; eval $thebar; eval $thegray ; echo " $freesp %"

echo  "\`\`\`"

To run this page use

while true; do ./pi_stats.sh | ncat  -l -p 1965 --ssl; done

Python CGI Pages

There are a number of good Gemini servers, for my testing I used the Python based Jetforce server, it is installed by:

pip install jetforce

To run the Jetforce server it is important to define a home directory, the allowable hosts that can connect (0.0.0.0 is all IP4 nodes) and the server’s hostname:

# Start the jetforce Gemini server for all IP4 hosts
jetforce --dir /home/pi/temp --host "0.0.0.0" --hostname 192.168.0.105 &
# Start jetforce without hardcoding hostname
# jetforce --dir /home/pi/temp --host "0.0.0.0" --hostname $(hostname -I) &

By default CGI (Common Gateway Interface) files are defined in the directory cgi-bin which is under the home directory.

Unlike a one-line Bash server, Jetforce server will pass environment variables like QUERY_STRING and host and remote connection information.

CGI programs can be written in a variety of programming languages. For this example I wanted to pass Raspberry Pi BME280 temperature, pressure and humidity sensor information to a Gemini CGI page.

The CGI program was written in Python and I installed a bme280 and figlet library:

pip install RPi.bme280
pip install pyfiglet

The Python script (sersor2gmi.py) outputs a Gemini content type, a Figlet title and then the sensor data values:

#!//usr/bin/python3
#
# sersor2gmi.py - send BME280 sensor data to a Gemini page
#
import smbus2
import bme280
import pyfiglet

# find device address via: i2cdetect -y 1
port = 1
address = 0x77
bus = smbus2.SMBus(port)

calibration_params = bme280.load_calibration_params(bus, address)

# the sample method returns a compensated_reading object
data = bme280.sample(bus, address, calibration_params)

# Output a Gemini page
print("20 text/gemini") #Note: Some Gemini CGI servers may do this
print("```") # use code mode
#print(pyfiglet.figlet_format("Pi BME280 Data", font = "small"))
print(pyfiglet.figlet_format("Pi BME 280 Data", font = "small"))

print("Temperature:" + "{:5.1f}".format(data.temperature) + " C" )
print("Pressure:   " + "{:5.1f}".format(data.pressure) + " kPa" )
print("Humidity:   " + "{:5.1f}".format(data.humidity) + " %" )
print("```") 

This file was added to the cgi-bin directory and it is made executeable (chmod +x sersor2gmi.py).

Below is the output seen in the Lagrange Gemini browser:

Final Comments

There are a variety of Gemini browsers for Windows, Android, Mac and Linux so if you’re looking for a quick and dirty internet solution Gemini might be a good solution.

I like how Gemini documents are totally readable, I can’t say the same for most web pages.

The one thing that I missed with Gemini pages is the ability of show nice charts, text based bars work for simple stuff but doing text based line charts is a little too ugly for me.

6 Lines of Python to Plot SQLite Data

There are some great tutorials on SQL, SQLite and Matplotlib, however I wanted something super simple that I could use to teach my daughters with.

This blog documents my notes on what I used to teach them SQL, SQLite3 and then how to plot the results in Python. Yes it can only takes 6 lines of Python code to read an SQLite3 file and plot it, if you are doing 2 line charts with a title and legends its 9 lines.

Getting Started with SQL

There are some great SQL databases and SQL tools that can are quite user friendly.

For us I started with SQLite3 which is file based, no server component is required and it runs on all major OS’s and Raspberry Pi’s. There are a number of SQL Query builder tools but we found the DB Browser for SQLite gave us all the functionality we needed.

Check the install documentation for your hardware/OS, to install SQLite3 and DB Browser on Debian/Ubuntu/Raspberry Pi’s :

sudo apt install sqlite3
sudo apt-get install sqlitebrowse

SQLite Databases

For testing there are a lot of excellent databases that can downloaded from Kaggle. These data files are in CSV format and they can be imported into a SQLite database file using the DB Browser.

For our first test we used the Kaggle SARS 2003 data set.

A new database file (sars.db) was created and then the CSV was imported.

Create VIEWS to Simplify

SQL Views can be created to simply the field names, reduce the number of fields or add custom fields.

Use the “Execute SQL” tab to create a view. An example to create a view with simplified field names would be:

Views and SELECT queries can be generated to add custom fields. An example to extract the month from the date field and add 2 other fields (month number and month name):

Test SQL Queries

The DB Browser tool is good for testing out SQL queries and plot the data before moving to Python.

A query to find the worst 5 countries affected by SARS would:

select sum(deaths) sum_deaths, country from v_sars group
  by Country having sum_deaths > 0 order by sum_deaths desc limit 5

Plotting in Python

There are number of ways to plot SQL data in Python. I found that the easiest way was to use Pandas dataframes.

To load the necessary Python libraries:

pip install sqlite3
pip install pandas
pip install matplotlib

The Python code to connect to SQLite3, run the SQL query (of sum of deaths vs. country) and plot the data is:

#
# sql_bar0.py - Sqlite to Bar Charts
#
import sqlite3, pandas , matplotlib.pyplot as plt

conn = sqlite3.connect("/media/pete/RED_4GB/sql/sars.db")

sql = """select sum(deaths) sum_deaths, country from v_sars group
  by Country having sum_deaths > 0 order by sum_deaths desc limit 5"""

data = pandas.read_sql(sql, conn)
#x values: data.Country,  y values: data.sum_deaths
plt.bar(data.Country, data.sum_deaths)
plt.title("SARS Death in 2003")
plt.show()

An example with 2 lines to shows the monthly deaths and cases would be:

#
# sql_line2.py - Sqlite to 2 Line Charts
#
import sqlite3, pandas , matplotlib.pyplot as plt

conn = sqlite3.connect("/media/pete/RED_4GB/sql/sars.db")

sql = """select s_month, sum(deaths) as sum_deaths, sum(cases) as sum_cases from v_month group by n_month"""

data = pandas.read_sql(sql, conn)

plt.plot(data.s_month,data.sum_deaths, label = "Deaths")
plt.plot(data.s_month,data.sum_cases, label = "Cases")
plt.legend()
plt.title("SARS Death in 2003")
plt.show()

Summary

By keeping the Python code simple we were able to focus on SQL queries.

Using the basic Python code the SQL connection we later changed from SQLite3 to MySQL or Progresql.

OPC UA Protocol with Python and Node-Red

Industrial operations such as chemical refineries, power plants and mineral processing operations have quite different communications requirements than most IT installations. Some of the key industrial communications requirements include: security, multi-vendor connectivity, time tagging and quality indications.

To meet industrial requirements a communications standard called OPC (OLE for Process Control) was created. The original OPC design was based on Microsoft’s Object Linking and Embedding (OLE) and it quickly became the standard for communications between control systems consoles, historians and 3rd party applications.

The original OPC standard worked well but it had major limitations in the areas of Linux/embedded systems, routing across WANs, and new security concerns. To better address new industrial requirements the OPC UA, (Open Platform Communications Unified Architecture) standard was created.

In this article I will create an OPC UA server that will collect sensor data using Python and Node-Red, and the results will be shown in a Node-Red web dashboard.

Install Python OPC UA Server

There are a number of OPC UA open source servers to choose from.

For “C” development applications see the Open62541 project (https://open62541.org/), it offers a C99 architecture that runs on Windows, Linux, VxWorks, QNX, Android and a number of embedded systems.

For light weight quick testing OPC UA servers are available in Python and Node-Red.

The Free OPC-UA Library Project (https://github.com/FreeOpcUa) has a great selection of open source tools for people wishing to learn and play with OPC UA.

I keep things a little simple I will be using the python-opcua library which is a pure Python OPC-UA Server and client. (Note: a more complete Python OPCUA library, https://github.com/FreeOpcUa/opcua-asyncio, is available for more detailed work). Also an OPC-UA browser is a useful tool for monitoring OPC UA server and their tags. To load both of these libraries:

# Install the pure Python OPC-UA server and client
sudo apt install python-opcua
# Install the OPC UA client and the QT dependencies
sudo apt install PyQT*
pip3 install opcua-client

Simple Python OPC-UA Server

As a first project a simple OPC-UA server will be created to add OPC-UA tags and then simulate values.

The first step in getting this defined is to set an endpoint or network location where the OPC-UA server will be accessed from.

The default transport for OPC-UA is opc.tcp. The Python socket library can be used to determine a node’s IP address. (To simplify my code I also hard coded my IP address, opc.tcp://192.168.0.120:4841).

The OPC-UA structure is based on objects and files, and under an object or file tags are configured. Tags by default have properties like value, time stamp and status information, but other properties like instrument or alarm limits can be added.

Once a tag object is define, the set_value function is used to simulate the tag values.

# opcua_server1.py - Create an OPC UA server and simulate 2 tags
#
import opcua
import random
import time
 
s = opcua.Server()
s.set_server_name("OpcUa Test Server")
s.set_endpoint("opc.tcp://192.168.0.120:4841")
  
# Register the OPC-UA namespace
idx = s.register_namespace("http://192.168.0.120:4841")
# start the OPC UA server (no tags at this point)  
s.start() 
  
objects = s.get_objects_node()
# Define a Weather Station object with some tags
myobject = objects.add_object(idx, "Station")
  
# Add a Temperature tag with a value and range
myvar1 = myobject.add_variable(idx, "Temperature", 25)
myvar1.set_writable(writable=True)
  
# Add a Windspeed tag with a value and range
myvar2 = myobject.add_variable(idx, "Windspeed", 11)
myvar2.set_writable(writable=True)
 
# Create some simulated data
while True:
    myvar1.set_value(random.randrange(25, 29))
    myvar2.set_value(random.randrange(10, 20))
    time.sleep(5)

The status of the OPC-UA server can be checked using the OPC-UA browser:

# start the Python OPC-UA browser client
opcua-client

Items within an OPC-UA server are defined by their name space index (ns) and their object index. The name space index is returned after an name space is register. An object’s index is defined when a new object is create. For this example the Windspeed tag has a NodeId of “ns-2;i=5”, or an index 5 on name space 2.

The opcua-client application can view real-time changes to a tag’s value using the subscription option.

In OPC the terms “tags” and “variables” are often used interchangeably. In the instrument lists the hardware signals are usually referred to as “tags”, but within the OPC UA server the term “variables” is used. The key difference is that a variable can also be an internal or soft point such as a counter.

Python OPC-UA Client App

For my Python client application I loaded up a simple gauge library (https://github.com/slightlynybbled/tk_tools):

pip install tk_tools

The Python client app (station1.py) defines an OPC-UA client connection and then it uses the NodeId definition of the Temperature and Windspeed tags to get their values:

# station1.py - Put OPC-UA data into gauges 
#
import tkinter as tk
import tk_tools
import opcua

# Connect to the OPC-UA server as a client
client = opcua.Client("opc.tcp://192.168.0.120:4841")
client.connect()

root = tk.Tk()
root.title("OPC-UA Weather Station 1")

# Create 2 gauge objects
gtemp = tk_tools.Gauge(root, height = 200, width = 400,
            max_value=50, label='Temperature', unit='°C')
gtemp.pack()
gwind = tk_tools.Gauge(root, height = 200, width = 400,
            max_value=100, label='Windspeed', unit='kph') 
gwind.pack()

def update_gauge():
    # update the gauges with the OPC-UA values every 1 second
    gtemp.set_value(client.get_node("ns=2;i=2").get_value())
    gwind.set_value(client.get_node("ns=2;i=5").get_value())
    root.after(1000, update_gauge)

root.after(500, update_gauge)

root.mainloop()

XML Databases

In the earlier Python OPC-UA server example tags were dynamically added when the server was started. This method works fine for simple testing but it can be awkward for larger tag databases.

All industrial control vendors will have proprietary solutions to create OPC-UA tag databases from process control logic.

Users can also create their own tag databases using XML. The OPC-UA server tag database can be imported and exported to XML using the commands:

# to export from the online system to an XML file:
# where: s = opcua.Server()
s.export_xml_by_ns("mytags.xml")
# to import an XML file:
s.import_xml("mytags2.xml","")

The XML files can be viewed in a web browser, and unfortunately the format is a little ugly. The XML files have a header area with a large number of options.The Name Space Uris is the custom area that defines the OPC UA end point address.

After the header there are object and variable definitions (<AUVariable>). In these section the variable’s NodeID, tag name and description are defined.

The Free OPC-UA modeler that can help with the creation of XML tag databases. To install and run the Free OPC-UA modeler:

$ pip install opcua-modeler
$ opcua-modeler

The OPC-UA modeler will read existing XML files and then allow for objects, tags and properties to be inserted into the XML structure.

CSV to XML

A CSV file is an easy format for defining tag databases. For example a file mytags.csv could be defined with 3 fields; tagname, description and default value.

$ cat mytags.csv
# field: tag, description, default-value
TI-101,temperature at river, 25
PI-101,pressure at river, 14

A basic CSV to XML import tool can be created to meet your project requirements. There are a number of good programming options to do this migration. For my project I created a small Bash/AWK program to translate the 3 field CSV file to the required OPC-UA XML format.

The first awk section prints out the header information. The second awk section reads the input (CSV) text line by line and pulls out each of the three fields ($1, $2 and $3) and prints out the XML with these fields inserted in the output.

#!/usr/bin/bash
# csv2xml.sh - create an OPC UA XML file from CSV
# 
 
# add the xml header info
awk ' BEGIN {
  print "<?xml version=\"1.0\" encoding=\"utf-8\"?>"
  print "<UANodeSet xmlns=\"http://opcfoundation.org/UA/2011/03/UANodeSet.xsd\"" 
  print "           xmlns:uax=\"http://opcfoundation.org/UA/2008/02/Types.xsd\""
  print "           xmlns:xsd=\"http://www.w3.org/2001/XMLSchema\"" 
  print "           xmlns:xsi=\"http://www.w3.org/2001/XMLSchema-instance\">"
  print "<NamespaceUris>"
  print "  <Uri>http://192.168.0.120:4841</Uri>" ; # This address would be passed in
  print "</NamespaceUris>"
}'

# Read the input CSV format and process to XML
awk ' {
   FS="," ; # separate fields with a comma
# Skip any comment lines that start with a #
  if ( substr($1,1,1) != "#" )
  {
    i = i+1 ; # increment the NodeID index
    print "<UAVariable BrowseName=\"1:"$1"\" DataType=\"Int32\" NodeId=\"ns=1;i="i"\" ParentNodeId=\"i=85\">"
    print "  <DisplayName>"$1"</DisplayName>" ; # set the display name to the 1st field
    print "  <Description>"$2"</Description>" ; # set the description to the 2nd field
    print "      <References>"
    print "        <Reference IsForward=\"false\" ReferenceType=\"HasComponent\">i=85</Reference>"
    print "      </References>"
    print "    <Value>"
    print "      <uax:Int32>"$3"</uax:Int32>" ; # set the default value to the 3rd field
    print "    </Value>"
    print "</UAVariable>"
  }   
}
END{ print "</UANodeSet>"} '

To run this script to read a CSV file (mytags.csv) and create an XML file (mytags.xml) :

cat mytags.csv | ./csv2xml.sh > mytags.xml

Node-Red OPC UA Server

There is a good OPC UA node (https://flows.nodered.org/node/node-red-contrib-opcua) that includes a server and most common OPC UA features. This node can be install within Node-Red using the “Manage Palette” option.

To setup a Node-Red OPC UA server and a browser, define a OPCUA server node to use the Node Red IP address and set a custom nodeset directory. For my example I set the directory to /home/pi/opcua and in this directory I copied the XML file that I created from CSV (mytags.xml) into.

The OPCUA Browser node will send messages directly into the debug pane. This browse node allows me to see the objects/variables that I defined in my XML file.

The next step is to look at writing and reading values.

The simplest way to communicate with an OPC UA server is to use an OpcUa Item node to define the NodeID and an OpcUa Client node to do some action. For the OpcUa Client node the End point address and an action needs to be defined.

In this example the pressure (PI-101) has a NodeID of “ns=5;i=2”, and this string is entered into the OpcUA item node. The OpcUA Client node uses a Write action. When a Write action is issued a Good or Bad status message is returned.

The OpcUa Client node supports a number of different actions. Rather than doing a Read action like in the Python client app, a Subscribe can be used. A Subscribe action will return a value whenever the value changes.

NodeRed Dashboards with the Python OPC UA Server

For the last example I will use the Python OPC UA server from the first example. The Temperature and WindSpeed will use the same simulation code, but an added Waveheight tag will be a manually entered value from Node-Red.

A Node-Red application that connects to the Python OPC UA server and presents that data in a Node-Red dashboard would be:

This example subscribes to two real-time inputs (Temperature and Windspeed) and presents the values in gauges. The OpcUA Item nodes define the OPC UA NodeId’s to be used.

All the OpcUa Client nodes will need their Endpoints defined to the Python OPC UA server address.

The subscribed data values are returned as a 2 item array (because the data type is a Int64). The Gauge node will only read the first payload array item, (which is 0) so a small function node copies the second payload item (msg.payload[1]) to the payload message:

// Copy the second payload array item to be the payload
//  Note: msg.payload[0] = 0 and the Dashboard Gauge needs to use the value at payload[1]
msg.payload = msg.payload[1]
return msg;

For this example a manual input was included. The WaveHeight is subscribed to like the other tags, and the slider position is updated to its value. The slider can also be used to manually set the value by having the slider output passed to an OpcUa Client node with a WRITE action.

After the logic is complete the Deploy button will make the application live. The Node-Red dashboard can be viewed at: http://node-red-ip:1880/ui

Final Comments

This is a quick and dirty set of examples on how to use Python and Node-Red with OPC UA.

OPC UA has a ton of other features that can be implemented like : Alarms and Events and Historical Data.

Also it should be noted that most high end OPC UA servers support accessing the OPC UA items via their browse names. So instead of accessing a point using “ns=5;i=6” a browser name string can be used, such as “ns=5;s=MYTAGNAME”.

Gauges in a Python Canvas

There are some nice Python packages like tk_tools, that can be used for IoT indicators and gauges.

My daughter and I had a project where we wanted to repurpose an old eReader to be a kitchen kiosk display. Unfortunately tk_tools doesn’t support Python 2.7, also and we needed to account for gray scale and larger text, so we needed to look at another solution.

This blog documents how we made some simple update-able gauges using Python Tkinter Canvas objects that are supported in both Python 2.7 and 3.x .

Getting Started

Unfortunately the Python 2 and 3 Tkinter libaries are named differently (Tkinter in 2.7 vs tkinter in 3.x). If you are coding for both Python 2.7 and 3.x this gets messy, a simple workaround in your code is:

# Manage Python 2.7 and 3.x
#
import sys
# Check the version of Python and use the correct library
if sys.version_info[0] == 2:
    import Tkinter
else:
    import tkinter as Tkinter

Analog Clock

A Tkinter canvas supports a number of basic objects such as rectangles, circles, arcs, text, etc. The basic objects are positioned within the canvas space.

I found that as a first example an analog clock was a good place start. The first pass code for a clock with just the second hand would be:

# A Clock Second Hand Example
#
import tkinter as Tkinter # Python 3.x
import datetime

def update_sec():
    #Reposition the second hand starting position
    thesec = datetime.datetime.now().second
    arcstart = 90 - thesec*6  #0 sec = 90deg
    C.itemconfig(asec,start=arcstart) #pass the new start position
    C.after(1000, update_sec)

# Create a canvas object with an oval face and a second hand
top = Tkinter.Tk()

C = Tkinter.Canvas(top, bg="silver", height=250, width=300)
C.pack()


coord = 10, 50, 240, 210 
C.create_oval(coord,  fill="white")
# Have the second hand start at the top (90 deg) with 1 deg arc
asec = C.create_arc(coord, start=90, extent=1, width=3)

C.after(1000, update_sec)
top.mainloop()

The key point is to get the id of the seconds hand arc (asec). The itemconfig method is then used to change the starting position of seconds hand arc (C.itemconfig(asec,start=arcstart) ).

The arc positioning is a little backwards, 0 degrees is at 3o’clock and then goes counter-clockwise.

The next step is to add narrow arcs for the minutes and hours. Also text could be used to digitally show the date and time. For the hour and minute hand I used different colours and thicknesses.

#
# A Clock Example
#
import tkinter as Tkinter # Python 3.x
from datetime import datetime

def update_sec():
    # Position the hands 
    C.itemconfig(asec,start= 90 - datetime.now().second*6)
    C.itemconfig(amin,start= 90 - datetime.now().minute*6)
    C.itemconfig(ahour,start= 90 - datetime.now().hour*360/12)
    C.itemconfig(dtime,text = datetime.now().strftime("%d/%m/%Y %H:%M:%S"))
    C.after(1000, update_sec)

# Create a canvas object with an oval face and a second hand
top = Tkinter.Tk()

C = Tkinter.Canvas(top, bg="silver", height=250, width=300)
C.pack()

coord = 10, 50, 240, 210
C.create_oval(coord,  fill="white")
# Have the second hand start at the top (90 deg) with 1 deg arc
asec = C.create_arc(coord, start=90, extent=1, width=2)
amin = C.create_arc(coord, start=90, extent=1, width=4, outline='blue')
ahour = C.create_arc(coord, start=90, extent=1, width=6, outline='red')
dtime = C.create_text(120,20, font="Times 16 bold", text="00:00:00")

C.after(1000, update_sec)
top.mainloop()

Gauges

There are a number of different types of gauges. My first example was a speedometer graph, that used an arc for both the background and the gauge needle:

#
# Use Canvas to create a basic gauge
#
from tkinter import *
import random

def update_gauge():
    newvalue = random.randint(low_r,hi_r)
    cnvs.itemconfig(id_text,text = str(newvalue) + " %")
    # Rescale value to angle range (0%=120deg, 100%=30 deg)
    angle = 120 * (hi_r - newvalue)/(hi_r - low_r) + 30
    cnvs.itemconfig(id_needle,start = angle)
    root.after(3000, update_gauge)

    
# Create Canvas objects    

canvas_width = 400
canvas_height =300

root = Tk()

cnvs = Canvas(root, width=canvas_width, height=canvas_height)
cnvs.grid(row=2, column=1)

coord = 10, 50, 350, 350 #define the size of the gauge
low_r = 0 # chart low range
hi_r = 100 # chart hi range

# Create a background arc and a pointer (very narrow arc)
cnvs.create_arc(coord, start=30, extent=120, fill="white",  width=2) 
id_needle = cnvs.create_arc(coord, start= 119, extent=1, width=7)

# Add some labels
cnvs.create_text(180,20,font="Times 20 italic bold", text="Humidity")
cnvs.create_text(25,140,font="Times 12 bold", text=low_r)
cnvs.create_text(330,140,font="Times 12 bold", text=hi_r)
id_text = cnvs.create_text(170,210,font="Times 15 bold")

root.after(3000, update_gauge)

root.mainloop()

The basic gauge can be enhanced to have more value ranges and colour hihi/hi/low ranges:

#
# Use Canvas to create a basic gauge
#
from tkinter import *
import random

def update_gauge():
    newvalue = random.randint(low_r,hi_r)
    cnvs.itemconfig(id_text,text = str(newvalue) + " %")
    # Rescale value to angle range (0%=120deg, 100%=30 deg)
    angle = 120 * (hi_r - newvalue)/(hi_r - low_r) + 30
    cnvs.itemconfig(id_needle,start = angle)
    root.after(3000, update_gauge)

    
# Create Canvas objects    

canvas_width = 400
canvas_height =300

root = Tk()

cnvs = Canvas(root, width=canvas_width, height=canvas_height)
cnvs.grid(row=2, column=1)

coord = 10, 50, 350, 350 #define the size of the gauge
low_r = 0 # chart low range
hi_r = 100 # chart hi range

# Create a background arc with a number of range lines
numpies = 8
for i in range(numpies):
    cnvs.create_arc(coord, start=(i*(120/numpies) +30), extent=(120/numpies), fill="white",  width=1)    

# add hi/low bands
cnvs.create_arc(coord, start=30, extent=120, outline="green", style= "arc", width=40)
cnvs.create_arc(coord, start=30, extent=20, outline="red", style= "arc", width=40)
cnvs.create_arc(coord, start=50, extent=20, outline="yellow", style= "arc", width=40)
# add needle/value pointer
id_needle = cnvs.create_arc(coord, start= 119, extent=1, width=7)

# Add some labels
cnvs.create_text(180,15,font="Times 20 italic bold", text="Humidity")
cnvs.create_text(25,140,font="Times 12 bold", text=low_r)
cnvs.create_text(330,140,font="Times 12 bold", text=hi_r)
id_text = cnvs.create_text(170,210,font="Times 15 bold")

root.after(3000, update_gauge)

root.mainloop()

Our Final Project

Our final project had 4 gauges that were based on basic gauge code. Our Python app ran full screen on a Kobo eReader that we installed Debian Linux on. The app connected to our Home Assistant Pi and showed us our current weather conditions.

We had to tweek the basic code a little bit to account for the 800×600 screen size and grey scale graphics.

Summary

In this blog we only looked at some basic gauges, the Tkinter Canvas component can be used in a very variety of different applications such as: bar charts, real time charts, graphics etc.

littleBit Dashboards (without Cloud Bits)

littleBits is a set of electronic components that magnetically connect together. litteBits is geared towards the kids STEP market and it is available in many schools and libraries.

The littleBits company has done an excellent job making their product easy to use. There is a large variety of different “bit” modules and for Internet applications there is a Cloud Bit ($59).

I found that the Cloud Bit was very easy to get up and running, but I found it was expensive at $59 and somewhat limiting, for example you are only access 1-input and 1-output. So if you want to do 2 inputs/output you would need to purchase a second Cloud bit module.

In this blog I’d like to document how I used a $39 Arduino Bit to do 3-inputs and 3-outputs. I also had the code talk directly to a free Web Dashboard (AdaFruit).

littleBits Arduino Program

A set of commands needs to be setup between the littlebits Arduino module and the PC/Pi. In my Arduino program I referenced the ports A,B,C as inputs (on the left side), and D,E,F as outputs (on the right side).

The commands from the PC/Pi would be : reference_pin:value, for example D:255 would set the D (top left pin) at 100%. It’s important to note that Arduino inputs and outputs are scaled from 0-255.

For inputs the littleBits would send the results as pin: reference_pin:value, for example B:255 would be the result at full scale for the A0 input.

ard_abc

My  test setup had:

  • A fork bit – this meant I only needed 1 power input source
  • A slider bit (0-1) on bit D0 (A)
  • A dimmer bit (0-255) on bit A0 (B)
  • A temperature bit on bit A1 (C)
  • An LED on bit d1 (D)
  • A number bit on D5 (E)
  • a bargraph bit on D9 (F)

lb_ard_setup

Below is the littleBits Arduino program that managed the serial communications.

// littleBits_2_Dashboards - create a serial interface to read/write to a PC/Pi
//
// Command from the littleBits: (A,B,C are the left pins) 
//  A:value <- for example B:24, pin A0 (2nd input) is 24 

// Commands from the PC/Pi: (D,E,F are the right pins)
//  D:output <- for example E:128, set pin A0 to 50% (128/255)
//
String thecmd; 
String thevalue;
String theinput;
char outstring[3];

void setup() {
  //define the littleBits right side pins 1,5 and 9 
  pinMode(1, OUTPUT);
  pinMode(5, OUTPUT);
  pinMode(9, OUTPUT);
  // define the littleBits left side inputs
  pinMode(0, INPUT);
  pinMode(A0, INPUT);
  pinMode(A1, INPUT);
  
  Serial.begin(9600); // this needs to match the PC/Pi baud rate
}
void loop() { 
  if (Serial.available() > 0) {
    thecmd = Serial.readStringUntil("\n"); 
    if (thecmd.length() > 2) { // ensure the msg size is big enough
      thevalue = thecmd.substring(2);
      if (thecmd.startsWith("D")) { analogWrite(1,thevalue.toInt()); }
      if (thecmd.startsWith("E")) { analogWrite(5,thevalue.toInt()); }
      if (thecmd.startsWith("F")) { analogWrite(9,thevalue.toInt()); }
    }     
  }
  // Try 3 different inputs: d0 = on/off , A0 = pot, A1 = temp sensor

  sprintf(outstring,"%d",digitalRead(0));
  Serial.write("A:");
  Serial.write(outstring);
  Serial.write("\n");

  sprintf(outstring,"%d",analogRead(A0));
  Serial.write("B:");
  Serial.write(outstring);
  Serial.write("\n");

// A1 is an "i12" littleBits temperature sensor
  int temp = analogRead(A1);
  temp = map(temp,0,1023,0,99); //rescale. Sensor range is 0-99 C or F
  sprintf(outstring,"%d",temp);
  Serial.write("C:");
  Serial.write(outstring);
  Serial.write("\n");
  

  delay(5000);
}

The Arduino IDE “Serial Monitor” can be used to view the output and set values.

msgbox

Python on the PC or Raspberry Pi

The Arduino program will send input data for A,B,C every 5 seconds. This input can be seen in Python by:

#
# littleBits Read Test
#
import serial

ser = serial.Serial(port='/dev/ttyACM1', baudrate=9600) # format for Linux
#ser = serial.Serial(port='COM1', baudrate=9600) # format for Windows

while True:
    inline = ser.readline()
    inline = inline.decode() # make a string
    pin = inline[0:1] # the first character is the pin
    thevalue = inline[2:-1] # the value is between ":" and "\n"<span id="mce_SELREST_start" style="overflow:hidden;line-height:0;"></span>
    print(pin,thevalue)

The output will look something like:

A  1
B  1023
C  21

To write commands from Python:

Write
#
# littleBits Write Test
#
import serial

ser = serial.Serial(port='/dev/ttyACM2', baudrate=9600) # format for Linux
#ser = serial.Serial(port='COM1', baudrate=9600) # format for Windows

while True:
    print("\nWrite an output value to littleBit")
    out = input("Enter pin:value, pin=A,B,C example: 'E:255' : ")
    out = out.upper() + "\n"
    out2 = out.encode('utf_8')
    ser.write(out2)

Adafruit Dashboards

There are lots of good free dashboards. For this project I used the Adafruit site. To get started you will need to log in and create a free account.

I’ve bought a number of components from Adafruit. I think that they are an excellent company that goes out of their way to create great user guides and products.

To get started with Adafruit Dashboards see: https://github.com/adafruit/Adafruit_IO_Python

The first step is to add some Adafruit tags that the code can read/write to.

Ada_feeds

In the Python code a couple of dictionaries (lb_inputs, lb_outputs)  were created to link the littleBit references (A-F) with the Adafruit tags. Also two dictionaries (lb_inlast, lb_outlast) are used to minimize communications traffic so that only new values were written.

#
# Import standard python modules
import time, random
import serial

# import Adafruit IO REST client
from Adafruit_IO import Client, Feed, RequestError

ADAFRUIT_IO_USERNAME = "put_your_username_here"
ADAFRUIT_IO_KEY = "c039f24ecb6...xxxxx"

aio = Client(ADAFRUIT_IO_USERNAME, ADAFRUIT_IO_KEY)

# Create dictionaries of inputs, output, and last values
lb_inputs = {"A":"lb-slide", "B":"lb-dimmer","C": "lb-temp"}
lb_inlast = {"A":0, "B":0,"C": 0}
lb_outputs = {"D":"lb-led", "E":"lb-number", "F":"lb-bar"}
lb_outlast = {"D":0, "E":0,"F": 0}

# Setup the serial port connection
ser = serial.Serial(port='/dev/ttyACM1', baudrate=9600)

while True:
    # Get values from littleBits and write to the dashboard
    inline = ser.readline()
    inline = inline.decode() #inline should look like: A:125\n
    pin = inline[0:1] # pin is the first character in string
    thevalue = inline[2:-1] # value is between ":" and "\n"
    if lb_inlast[pin] != thevalue: # Write only new values
        print(pin,thevalue, lb_inputs[pin])
        ada_item = aio.feeds(lb_inputs[pin])
        aio.send(ada_item.key,thevalue)
        lb_inlast[pin] = thevalue

    thetag = 'lb-slide'
    # Write new dash values to littleBits if they've changed
    for lbtag, dashtag in lb_outputs.items():
        print (lbtag,dashtag)
        thevalue = aio.receive(dashtag).value
        if lb_outlast[lbtag] != thevalue: # Write only new values
            outstr = lbtag + ":" + thevalue + "\n"
            print(outstr)
            ser.write(outstr.encode('utf_8'))
            lb_outlast[lbtag] = thevalue   

    time.sleep(2)

If everything is working correctly then new values should be written to in both directions. On the Adafruit Web site the Feed page should show the new values.

To make things look more presentable Adafruit Dashboards can be used.

ada_dash

Final Comments

In this project I used the Adafruit API, other good platforms would be IFTTT and Node-Red

Use MetPy to help answer your kid’s science questions

Being a good dad I try and answer my kids science questions, but sometimes it’s really tough.

There is an awesome Python library called MetPy that can help with some of those challenging science and weather questions.

In this blog I’d like to introduce the MetPy library and show how to use it to solve questions like:

  • Can they make snow when it’s above freezing ?
  • How much thinner is the air in places like Denver?
  • How can you figure out what the wind chill is?

Getting Started with MetPy

To install MetPy:

pip install metpy

One of nice things about MetPy is that it manages the scientific units, so variables  are defined with their units. Below is a Python example where the units module is used for a simple temperature conversion. The temperature today is defined as 40 degF. The to() method can be used to convert the temperature to degC.

>>>> from metpy.units import units
>>> tempToday = [40] * units.degF
>>> tempToday.to(units.degC)

Quantity([4.44444444], 'degree_Celsius')

You can also do some interesting mixing of units in math calculations, for example you can add 6 feet and 4 meters:

 >>> print( [6]*units.feet + [4] * units.m)
[19.123359580052494] foot

MetPy has a very large selection of thermodynamic and weather related functions. In the next sections I will show how to use some of the these functions.

How can they make snow when it’s above freezing ?

Ski resort can make snow by forcing water and pressurized air through a “snow gun”. Making snow can be an expensive operation but it allows ski resorts to extend their season.

snowgun

When you get a weather forecast the temperature is given as the ambient or dry bulb temperature. The wet bulb temperature takes the dry air temperature and relative humidity into account. The wet bulb temperature is always below the outside temperature. To start  making snow  a wet bulb temperature of -2.5°C or 27.5°F is required.

Metpy has a number of functions that can used to find humidity and wet bulb temperatures.  The wet_bulb_temperature function will find the wet bulb temperature using the pressure, dry temperature and dew point.

Below is an example where the temperature is below freezing, but it’s not possible to make snow because the wet bulb temperature is only -0.6°C  (not the required  -2.5°C).

>>> import metpy.calc
>>> 
>>> pressure = [101] * units.kPa
>>> temperature = [0.5] * units.degC
>>> dewpoint = [-2.5] * units.degC
>>> 
>>> metpy.calc.wet_bulb_temperature(pressure, temperature, dewpoint)

Quantity(-0.6491265444587265, 'degree_Celsius')

Knowing that -2.5°C (27.5°F) is the wet bulb temperature upper limit for snow making, the relative_humidity_wet_psychrometric function can be used to create a curve of humidity and dry temperature points where it is possible to make snow.

The code below iterates between -10 and 10 deg °C getting humidity values at a wet bulb temperature of -2.5°C.


#
# Find when you can make snow
#
import matplotlib.pyplot as plt
import metpy.calc
from metpy.units import units

print("Get temps vs. humidity")
print("-------------------")

plt_temp = []
plt_hum = []
for temp in range (-10,11): # Check dry temperatures between -10 - 10 C
    dry_temp = [temp] * units.degC
# Get the relative humidity
    rel_humid = metpy.calc.relative_humidity_wet_psychrometric(dry_temp, wet_temp,pres)
# Strip the humidity units for charting, and make a percent (0-100)
    the_humid = rel_humid.to_tuple()[0] * 100
    if (the_humid  0) : # Get valid points
        plt_temp.append(temp) # append a valid temp
        plt_hum.append(the_humid) # append a valid humidity
    print (temp, the_humid)

fig, ax = plt.subplots()
ax.plot(plt_temp, plt_hum )
ax.set(xlabel='Temperature (C)', ylabel='Humidity (%)',
title='When you can make Snow')
ax.grid()
#fig.savefig("makesnow.png")
plt.show()

makesnow

From the data we can see that it is possible to make snow when the temperature is above freezing and the humidity is low.

How much thinner is the air … in Denver?

We all know that the air is thinner when we’re up in an airplane, but how much thinner is it in Denver or  Mexico City compared to New York City ?

Using the height_to_pressure_std function it is possible to get a pressure value based on altitude. The to() method can be used to convert the pressure to standard atmospheres.

>>> import metpy.calc
>>> 
>>> New_York_alt = [33]*units.ft
>>> metpy.calc.height_to_pressure_std(New_York_alt).to(units.atm)
Quantity([0.99880745], 'standard_atmosphere')

>>> Denver_alt = [5280] * units.ft
>>> metpy.calc.height_to_pressure_std(Denver_alt).to(units.atm)
Quantity([0.82328412], 'standard_atmosphere')

>>> Mexico_city_alt = [7350]*units.ft
>>> metpy.calc.height_to_pressure_std(Mexico_city_alt).to(units.atm)
Quantity([0.76132418], 'standard_atmosphere')

Relative to New York City the air is about 18% thinner in Denver and 24% thinner in Mexico City.

Using the height_to_pressure_std function it is possible to create a chart of atmospheric pressure between sea level and the top of Mt. Everest (29,000 ft). At the top of Mt. Everest the air is 70% thinner than at sea level !!


#
# How much does the air thin as you climb ?
#
import matplotlib.pyplot as plt
import metpy.calc
from metpy.units import units
print("Get height vs. Atm Pressure")
print("---------------------------")
# create some plot variables
plt_ht = []
plt_press = []

# Check Atmospheric Pressure from sea level to Mt. Everest (29,000 ft) heights
for temp in range (0,30000,1000): # Check dry temperatures between -10 - 10 C
    height = [temp] * units.feet
    pressure = metpy.calc.height_to_pressure_std(height)
    atm = pressure.to(units.atm)
    print (height, atm)
    plt_ht.append (height.to_tuple()[0]) # put the value into plt list
    plt_press.append (atm.to_tuple()[0]) # put the value into plt list

fig, ax = plt.subplots()
ax.plot(plt_ht, plt_press )
ax.set(xlabel='Mountain Height (ft)', ylabel='Pressure (atm)',
title='How does the Pressure change Mountain Climbing?')
ax.grid()
fig.savefig("height_vs_press.png")
plt.show()

height_vs_press

How can I figure out the Wind Chill ?

The MetPy windchill function will return a “feels like” temperature based on a wind speed and ambient temperature. For example an outside ambient temperature of 40 deg °F with a wind of 20 mph feel like 28 deg °F.

>>> import metpy.calc
>>>
>>> temp = [40] * units.degF
>>> wind = [20] * units.mph
>>> metpy.calc.windchill(temp, wind, face_level_winds=True)
Quantity([28.42928573], 'degree_Fahrenheit')

When the temperature starts going below -20 °C parents should be keep a close eye on their kids for frost bite. Below is a code example that shows a curve of -20 °C based on ambient temperature and wind.


#
# Wind Chill
#
import matplotlib.pyplot as plt
import metpy.calc
from metpy.units import units

# Create some plotting variables
plt_temp = []
plt_speed = []

for temp in range (0,-46,-1): # Check dry temperatures between -10 - 10 C
    the_temp = [temp] * units.degC
    for wind in range(1,61,1):
        the_wind = [wind] * units.kph
        windchill =
        metpy.calc.windchill(the_temp,the_wind,face_level_winds=True)
# Select points with a wind chill around -20
        if (windchill.to_tuple()[0]) = -20.1) :
            plt_temp.append(temp)
            plt_speed.append(wind)

fig, ax = plt.subplots()
ax.fill_between(plt_temp, plt_speed, label="-20 C - Wind Chill", color="red" )

ax.set(xlabel='Temperature (C)', ylabel='Wind Speed (kph)',
title='When is the Wind Chill -20C ?')
ax.grid()
fig.savefig("windchill.png")
plt.show()

windchill

Summary

MetPy didn’t solve all my kids questions but the Metpy library is an excellent for science questions around water and weather.

If you have a budding chemist or chemical engineer in your house try taking a look at the Python Thermo library.

Pi Rover using a Bottle Web Framework

There are a lot of Python web library choices, with each of the libraries offering different features and functions.

For simple Raspberry Pi Web application I was really impressed by the Python Bottle library. In this blog I’d like to document a Raspberry Pi rover project that I did using bottle.

Getting Started

To install the Python bottle library:

pip install bottle

There are some good tutorials for bottle. Probably the most important item is defining a decorator for a URL, and then linking the decorator to a function:

@route('/') # define a decorator for the start page
def my_homefuntion() # call a function for the start page
   return static_file("startpage.htm", root='') # call the start page

@route('/otherpage') # define a decorator for another page 
def my_otherpage_function() # call a function for the start page
   # do some stuff...

For the RaspPi rover I used a low cost car chassis (~$15).

It is not recommended to connect motors directly to Rasp Pi pin, for a few reasons:

  • larger motors require more power that a Pi GPIO pin can supply
  • power surges on motors can damage the Pi hardware
  • forward/backward motor directions require extra hardware
  • GPIO pins will only do ON/OFF, no variable speed settings.

There are a number of Raspberry Pi motor shield that can be used. For this project I used the Pimoroni Explorerhat Pro ($23). The Explorerhat has an excellent Python library that allows you to easily control the motor’s direction and speed.

Web Page

The goal for the web page (bottle1.htm) was to have 5 buttons for the controls and use AJAX to send the action request and then return the action code and display it on the page. The returned action would appear above the buttons ( in the lastcmd  paragraph tag).

Screen_bottle

For this example the button push was sent in the header as a GET request. So a forward button push would be a myaction: forward header item. In previous projects I’ve used POST request with parameters, however I found that header items can make things a little simpler.

<!DOCTYPE html>
<html>
<head> 
<title>Python Bottle Rover</title> 
<meta name="viewport" content="width=device-width, initial-scale=1">
<link rel="icon" href="data:,"> 
<style>
  html{font-family: Helvetica; display:inline-block; margin: 0px auto; text-align: center;}
  h1{color: #0F3376; padding: 2vh;}p{font-size: 1.5rem;}
  .button{display: inline-block; background-color: #4286f4; 
  border-radius: 4px; color: white; font-size: 30px; width:100%; height: 75px}
  .button2{background-color: green ;width:31%}
  .stop{background-color: red; width:33%}
</style>
</head>
<script>
function sendcmd(thecmd) {
  // send the action as a header item 
  var xhttp = new XMLHttpRequest();
  xhttp.open("GET","/action" , true);
  xhttp.setRequestHeader("myaction", thecmd);
  xhttp.send()
  xhttp.onreadystatechange = function() {
	// Get the response and put on it the screen
	if (this.readyState == 4 ) {	
		document.getElementById("lastcmd").innerHTML = "Last Command:<b>" +xhttp.responseText;
	}
  }
}
</script>
 
<body>
<h2>Python Bottle Rover</h2> 
<p id='lastcmd'></p>
<button onclick="sendcmd('forward')" class="button">FORWARD</button>
<button onclick="sendcmd('left')" class="button button2">LEFT</button>
<button onclick="sendcmd('stop')" class="button stop">STOP</button>
<button onclick="sendcmd('right')" class="button button2" >RIGHT</button>
<button onclick="sendcmd('backward')" class="button button">BACKWARD</button>
  
</body>
</html>

 

Bottle Rover App

For the rover app, there are two URL endpoints. The root (/) which would display the bottle1.htm page, and an action (/action) URL which would only be called from AJAX script when a button was pushed.

For this project the Raspberry Pi ip address was hardcoded into the code, for future projects dynamically getting the ip would be recommend. Also a web port of 8000 was used so as to not conflict with a dedicated web server (like Apache) that could be running on the Pi.

# Bottle2rover.py - web control for a RaspPi rover<span id="mce_SELREST_start" style="overflow:hidden;line-height:0;"></span>
#
from bottle import route, run, static_file, request

# Define RaspPi I/O pins
import explorerhat

# Send Action to Control Rover
def rover(action):
  if action == "forward":
    explorerhat.motor.one.speed(100)
    explorerhat.motor.two.speed(100)
  if action == "left":
    explorerhat.motor.one.speed(100)
    explorerhat.motor.two.speed(0)
  if action == "right":
    explorerhat.motor.one.speed(0)
    explorerhat.motor.two.speed(100)
  if action == "stop":
    explorerhat.motor.one.speed(0)
    explorerhat.motor.two.speed(0)
  if action == "backward":
    explorerhat.motor.one.speed(-50)
    explorerhat.motor.two.speed(-50)

# use decorators to link the function to a url
@route('/')
def server_static():
  return static_file("bottle1.htm", root='')
# Process an AJAX GET request, pass the action to the rocket launcher code
@route('/action')
def get_action():
  action = request.headers.get('myaction')
  print("Requested action: " + action)
  rover(action)
  return request.headers.get('myaction')

# Adjust to your required IP and port number
run(host = '192.168.0.106', port=8000, debug=False)

Final Comments

I was happily surprised how easy it was to get a Python bottle web app running. The bottle documentation was straightforward and I found that the code was quite lean.

Get Jokes and Quotes

I was trying to get a database of recent jokes and quotes, unfortunately there aren’t a lot of really good downloadable sources that have new and relevant material.

Reddit is a social media site that has a lot of potential for building databases or lists of recent jokes and quotes.

In this blog I wanted to document how I created a list of recent jokes and quotes using the Python reddit library (praw) . I also included some filtering to help remove bad items.

Reddit

Reddit is a social media source that is free to use. To pull information from Reddit you will need to create an account and get API client ID and secret. To do this go to: https://www.reddit.com/prefs/apps/ and select edit in the developed applications area.

reddit_api_info

The Reddit Python library is called Praw, and it is installed by:

pip install praw

To use the Python Reddit library you will need:

  • your username
  • your password
  • your client ID and client secret

Reddit has a wide list of categories and once a category is selected you can sort the list by:  new, hot and trending.

reddit_jokes

Python Joke Example

An example of using Python to get the “hot” dad jokes from Reddit is below. For this example I included a bad_keywords list. Some trial and error will be required to remove some returned items

 # Python Reddit Example # Get top 10 dad jokes  
 # Some filtering is added to remove bad items   
   
 import praw  
 import re # use this is filter out bad items  
   
 # remove items with these words in them (could include a large list of swear words)  
 bad_keywords = ['sex','prostitute','shit','edit','remove','delete', 'repost','this sub']  
   
 reddit = praw.Reddit(client_id='xQsMfaHxxxxxxx',  
            client_secret='X8r62koQgVxxxxxxxx',  
            user_agent='myreddit', username='yourusername', password='xxxxxxx')  
                             
 i=0                            
 for submission in reddit.subreddit('dadjokes').hot(limit=10):  
     thestring = submission.title + " " + submission.selftext  
     if not re.compile('|'.join(bad_keywords),re.IGNORECASE).search(thestring):  
         i += 1  
         print(i,submission.title,"..." submission.selftext )  
   
   

 

The output will look something like:

1 What’s Beethoven doing in his grave ... De-composing
2 Why can’t Swiss cheese be part of a fat-free diet? ... It’s made with hole milk.
3 People always wonder how I come up with flaccid penis jokes so easily and I just respond back with... ... It's not that hard.
4 2020 is going to be a great year. ... I can see it so clearly.
5 I got kicked out of karaoke after singing “Danger Zone” nine times in a row. ... Too many Loggins attempts.
6 What did one snowman say to the other snowman? ... "Do you smell carrots?"
7 The store near me is having a sale on batteries. ... If you buy two packs, they'll throw in a pack of dead ones, free of charge.
8 Mr Ed just moved next door to me a few days ago. ... We’re neighbors now.

There are a number of other joke categories such as : yomamajokes, jokes, cleanjokes, greyjokes…

The reddit.subreddit object can have .hot, .new and .top calls.

For quotes see categories such as : quotes, showerthoughts

 

 

Pi Appliance

My goal was to make a Pi kitchen appliance that shows me the key things that I want to see and music I want to listen to while I’m getting my morning coffee. For this project I used a Rasp Pi with 2.8″ TFT touchscreen. These screens start at a round $15.

People’s morning interests will vary so in this blog I just wanted to highlight some of the issues that I needed to worked through. For me the main stumbling blocks were:

  • Hiding the top Rasp Pi menu bar
  • Creating a GUI that uses the full screen
  • Getting weather data
  • scraping web pages to extract what I need

Getting Started

There are some great Raspberry Pi TFT screens that come with buttons and cases. You will need to look at the documentation that comes with your screen, but a good reference is: https://learn.adafruit.com/adafruit-pitft-28-inch-resistive-touchscreen-display-raspberry-pitft_case

For my project I simply used some of my kids Lego blocks.

pi_kitch2

Remove the Pi Menu Bar

The Pi TFT screen isn’t super large, so I wanted to remove the Pi menu bar and run my application at full size.

tft_w_menu

To remove the menu bar tweek two files. First:

sudo nano /etc/xdg/lxsession/LXDE-pi/autostart

Comment out the line  (with #) :

@lxpanel --profile LXDE

Then do the same for:

nano /home/pi/.config/lxsession/LXDE-pi/autostart

After this reboot the system.

Create a Full Size App

There are a multitude of choices for a screen layout. I was looking for lines of text, with maybe the bottom line used for buttons. I found that 7 lines was a reasonable fit. To remove the Python Tkinter title I positioned the top of the screen above the physical screen position (-30 instead of 0).


# My Kitchen Appliance App
#
import urllib.request as urllib2
import tkinter as Tkinter
from tkinter.ttk import *

from tkinter.font import Font
from tkinter import messagebox
top = Tkinter.Tk()
top.title("My Kitchen Appliance")
top.geometry("320x240+-5+-30") # set screen size, left (-5) and top (-30)
top.resizable(False, False)
top.details_expanded = False

#Define the buttons
myfont = Font(family="Times New Roman Bold",size= 12) # Should try a few more sizes

tft_rows = 7 # try 7 rows of buttons
tftbutton = ['' for i in range(tft_rows)]
for i in range(tft_rows):
    tftbutton[i] = Tkinter.Button(top, text = "Line " + str(i+1), fg = "blue", bg = "white", anchor="w", width= 35, height= 1,font=myfont).grid(row=(i+1),column=1) # a buttpn arra

top.mainloop()

The Python GUI will look like this:

tft_7bttns

Get Weather Data

There are a number of good weather API’s. I used OpenWeather because I can use it in variety of apps like Node-Red. OpenWeather has a free user API but you should login and get an appid.

A Python example to get some current weather data for a city:


# get Open Weather (REST API) data
import requests

# api-endpoint

URL = "https://openweathermap.org/data/2.5/weather?q="
mycity = "burlington,CA"
myappid = "&appid=b6907d289e10d714a6e88b30761fae22"
# sending get request and saving the response as response object
fullURL = URL + mycity + myappid
r = requests.get(fullURL)

# extracting data in json format
data = r.json()

print (data)

# Check out the structure
#for index, value in enumerate(data):
# print(index, value)

# Show some weather data
print (data['weather'][0]['description'])
print (data['weather'][0]['main'])
print (str(int(data['main']['temp'])) + " C")
# convert wind speed from meters/sec to kph
print (str((data['wind']['speed'] * 3.6)) + " kph")

This code will give output such as:

$Python3 burlweather.py
{'coord': {'lon': -79.8, 'lat': 43.32}, 'weather': [{'id': 803, 'main': 
'Clouds', 'description': 'broken clouds', 'icon': '04n'}], 'base': 
'stations', 'main': {'temp': 5.81, 'pressure': 1014, 'humidity': 93, 
'temp_min': 3.33, 'temp_max': 7.78}, 'visibility': 24140, 'wind': 
{'speed': 2.1, 'deg': 50}, 'clouds': {'all': 75}, 'dt': 1574816701,
 'sys': {'type': 1, 'id': 818, 'country': 'CA', 'sunrise': 1574771158, 
'sunset': 1574804839}, 'timezone': -18000, 'id': 5911592, 'name': 'Burlington', 'cod': 200}
broken clouds
Clouds
5 C
7 kph

Scraping Web Pages

I wasn’t able to find an API for all the things I was after, so I need to scrape web pages. The Python Beautiful Soup library is a great for finding and grabbing stuff on web pages. To install it:

$ apt-get install python-bs4 (for Python 2)

$ apt-get install python3-bs4 (for Python 3)

I had an example where I wanted to find the ski lifts and runs open. I had the Web page but I needed to search the ugly HTML code.

ski_bs0

ski_bs

In the HTML code I found that the lift and run information is contained in a <p class=“open value” tag. Beautiful Soup allows you to make searches based on attributes. The results can be HTML code or the .text property will return the results as simple text (no HTML code).

The following Python code would search my URL and extract the number of lifts open:


$ python
Python 3.7.4
Type "help", "copyright", "credits" or "license" for more information.
>>> import urllib.request as urllib2
>>> from bs4 import BeautifulSoup
>>> theurl = 'https://www.onthesnow.ca/ontario/blue-mountain/skireport.html'
>>> page = urllib2.urlopen(theurl)
>>> soup = BeautifulSoup(page, 'html.parser')
>>> liftopen = soup.find("p", attrs={"class":"open value"})
>>> liftopen.text
'2 of 11'

Final Comments

There are a ton of different “Pi Appliance” applications that could be done. I hope that some of these hints that I’ve documented are helpful.

pi_kitch1