Micro:bit Extensions: Add extra sensors and devices

I have been using the BBC Micro:bit modules (~$20) with our kids to teach them basic hardware and software. The platform offers a block programming interface (very much like Scratch) and Python programming. The interesting thing is that the interface will toggle seamlessly between the two programming languages.

To add functionality to your projects Micro:bit supports extensions. This feature allows Arduino and Raspberry Pi sensors and devices to be connected to the basic module.

In this blog I wanted to:

  • Show a extensions example with deviices that are not directly available in the basic list
  • Comment on some limitations
  • Document how to add Micro:bit parts in Fritzing wire drawing tool

An Extension Example

Use the Extension menu item to add a new set of functionality to your Micro:bit’s project.

For this example I wanted to use some devices/sensors that I was using on my Arduino projects. These devices included:

It is important to note that the extension may not be readily available from the Microbit web page. For my project I did an Internet search to find the required github links. Once you have the URL it can be pasted into the Extension’s page:

Below is a picture of the Micro:bit with the three added devices

The Micro:bit logic used an on_start block to setup the pins for the TM1637 4-digit display, and initialize the OLED display.

The forever block:

  • queried the DHT11 sensor (on Pin 0)
  • showed the humidity on the Micro:bit display
  • showed the temperature on the TM1637 display
  • showed both the temperature and humidity on the 0.91″ OLED
  • cycled every 5 seconds

The block code can be viewed (or edited)in Python:

"""

Devices:

DHT11 Temperature/Humidity Sensor 

TM1637 4-Digit Display 

I2C 9.91" OLED Display

Show Temperature on TM1637

Show Humidity on Microbit screen

Show both Temperature and Humidity on the the OLED

"""
tm = TM1637.create(DigitalPin.P13, DigitalPin.P14, 7, 4)
MuseOLED.init()

def on_forever():
    dht11_dht22.query_data(DHTtype.DHT11, DigitalPin.P0, True, False, True)
    basic.show_string("H: " + str(dht11_dht22.read_data(dataType.HUMIDITY)) + " %")
    tm.show_number(dht11_dht22.read_data(dataType.TEMPERATURE))
    MuseOLED.clear()
    MuseOLED.write_string("Temperature: " + str(dht11_dht22.read_data(dataType.TEMPERATURE)) + " C")
    MuseOLED.new_line()
    MuseOLED.write_string("Humidity: " + str(dht11_dht22.read_data(dataType.HUMIDITY)) + " %")
    basic.pause(5000)
basic.forever(on_forever)

Limitations

Some of the limitations that I found were:

  • Not all Arduino sensors and devices were supported
  • Not all Arduino functionality is available with Micro:Bit. For example fonts on OLED devices.
  • Finding the correct extension can be tricky. For example searching 0.91 OLED doesn’t return any hits.
  • Some devices were supported in software, however they required 5V. A good example of this is the 2×16 LCD display

Documenting Wiring in Fritzing

Fritzing is an excellent free tool for wiring drawings (Note: for some platforms a donation might be required).

To add some Micro:bit parts to Fritzing see:

Once a parts file is downloaded it is imported into a “My Parts” grouping.

Summary

By adding extension you can greatly extend the usability of Micro:bits.

I found that for many simple projects block programming was quicker to create than Python, but it nice that the Python code gets autogenerated.

WOOB – Web Outside Of Browsers

The Web Outside of Browsers Project allows users to access Internet data in a generic way without using a Web Browser.

Woob offers a common interface for accessing a variety of different Internet data sources. So rather than using a specific API to access to weather and then another to access a job boards, the Woob API/tools can be used for both of these data sources.

Woob can be used as a:

  • Python library,
  • A command line tool, or by
  • Bash scripting

In this blog I wanted to document my Linux notes on:

  • basic command line tool usage
  • Bash/Python examples for getting weather data
  • Bash/Python examples for playing Internet streaming music

Note: WOOB is in development so features may change. Also there is a large list of application modules, but at present most are European based.

Getting Started

To install woob:

$ pip install woob

Once woob is installed a list of features can be shown by entering woob:

$ woob
usage: woob [--version] <command> [<args>]

Use one of this commands:
   bands           display bands and suggestions
   bank            manage bank accounts
   bill            get/download documents and bills
   books           manage rented books
   bugtracker      manage bug tracking issues
   calendar        see upcoming events
   cinema          search movies and persons around cinema
   cli             call a method on backends
   config          manage backends or register new accounts
   contentedit     manage websites content
   dating          interact with dating websites
   debug           debug backends
   gallery         browse and download web image galleries
   gauge           display sensors and gauges values
   geolocip        geolocalize IP addresses
   housing         search for housing
   job             search for a job
   lyrics          search and display song lyrics
   money           import bank accounts into Microsoft Money
   msg             send and receive message threads
   parcel          manage your parcels
   paste           post and get pastes from pastebins
   pricecompare    compare products
   radio           search, show or listen to radio stations
   recipes         search and consult recipes
   repos           manage a woob repository
   rpg             manage RPG data
   shop            obtain details and status of e-commerce orders
   smtp            daemon to send and check messages
   subtitles       search and download subtitles
   torrent         search and download torrents
   translate       translate text from one language to another
   travel          search for train stations and departures
   video           search and play videos
   weather         display weather and forecasts

For more information about a command, use:
   $ man woob-<command>
or
   $ woob <command> --help

Each features has a backend (database) associated with it. For example to define the weather.com backend for the weather command:

$ woob weather
Warning: there is currently no configured backend for weather
Do you want to configure backends? (Y/n): y

Available modules:
1) [ ] ilmatieteenlaitos   Get forecasts from the Ilmatieteenlaitos.fi website
2) [ ] lameteoagricole   lameteoagricole website
3) [ ] meteofrance       Get forecasts from the MeteoFrance website
4) [ ] weather           Get forecasts from weather.com
a) --all--               install all backends
q) --stop--

Select a backend to create (q to stop): 4
Backend "weather" successfully added.

If a backend needs to be removed, for example meteofrance, use the backend remove command:

$ wood weather backend remove meteofrance

To check which backends are defined, use the backend list command:

$ woob weather backend list
Enabled: weather

$ woob radio backend list
Enabled: freeteknomusic, somafm

Using the Weather Command

Woob offers a number of weather modules. A good generic option is weather.com.

To get started at the command line, enter: woob weather

$ woob weather 
Welcome to weather v3.0

Copyright(C) 2010-2022 Romain Bignon
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Lesser General Public License as published by
the Free Software Foundation, either version 3 of the License, or
(at your option) any later version.

Type "help" to display available commands.

Loaded backends: weather

weather> help
Weather Commands:
    cities PATTERN
    current CITY_ID
    debug
    forecasts CITY_ID

Woob Commands:
    formatter [list | FORMATTER [COMMAND] | option OPTION_NAME [on | off]]
    select [FIELD_NAME]... | "$direct" | "$full"
    ls [-d] [-U] [PATH]
    cd [PATH]
    condition [EXPRESSION | off]
    count [NUMBER | off]
    backends [ACTION] [BACKEND_NAME]...
    quit
    logging [LEVEL]

Type "help <command>" for more info about a command.

The first step is to search for a city with the cities option. Once a city is found the id (1-20) is used to show the current weather and the future forecast.

The example below searches for Toronto. The first item (Toronto, Ontario, Canada) is selected (current 1) to show the present Toronto weather. Toronto’s future weather can be checked with the forecast command (forecast 1).

Bash Scripting for Weather

The command line utility can be used with Bash script, (For more info on the interface).

A scripting example to show the current and forecast weather for Toronto (the first item in the list):

$ # use the -n 1 option to get the first item
$ woob weather cities 'Toronto' -n 1
d8ccf908e3c4c748e232720575df7cdbca6e0f1b412bca8595d8a28d0c28e8bc@weather — Toronto, Ontario, Canada

$ # get the id (first string) and save to a variable
$ city_id=$(woob weather cities 'Toronto' -n 1 | awk '{print $1}')

$ # pass the city id to get the weather current
$ woob weather current $city_id 
2022-12-08: -1 °C - 1030.2hPa (Rising) - humidity 73% - feels like -4 °C/24 °F - Fair

An example is to get the current weather for a place and show it as a Linux system tray notification:

$ # Create a notification note with current weather information
$ my_city="Sauble Beach"
$ city_id=$(woob weather cities "$my_city" -n 1 | awk '{print $1}')
$ notify-send -t 1000000 "$my_city" "$(woob weather current $city_id )

To show the long term forecast information in a Zenity dialog:

$ # Create an Info Dialog with weather forecast information
$ my_city="Sauble Beach"
$ city_id=$(woob weather cities "$my_city" -n 1 | awk '{print $1}')
$ wf=$(woob weather forecast $city_id)
$ zenity --width=650 --info --text="$wf" --title="$my_city"

Python Woob Weather Example

A Python example to select the first city in a list and then show the forecast would be:

#!/usr/bin/python3
#
# wweather.py - Search for weather forecast for a place
#
from woob.core import Woob
from woob.capabilities.weather import CapWeather

w=Woob()
w.load_backends(CapWeather)

# Find the id for a location
for city in w.iter_city_search("Sauble Beach"):
    # stop at first hit    
    print(city.id, city.name)
    break

# Get the Forecast for that location
for w0 in w.iter_forecast(city.id):
    print("Date:", w0.date)
    print("Forecast:", w0.text)
    print("High:", w0.high)
    print("Low", w0.low, "\n")

For more information on the Weather calls.

Using the Radio Command

There are a number of radio modules to choose from. The first step is to add at least one module:

$ woob radio
Warning: there is currently no configured backend for radio
Do you want to configure backends? (Y/n): y

Available modules:
1) [ ] audioaddict       Internet radios powered by audioaddict.com services
2) [ ] bandcamp          Bandcamp music website
3) [ ] freeteknomusic    freeteknomusic website
4) [ ] ina               INA French TV video archives
5) [ ] nectarine         Nectarine Demoscene Radio
6) [ ] nova              Nova French radio
7) [ ] ouifm             OÜI FM French radio
8) [ ] radiofrance       Radios of Radio France: Inter, Info, Bleu, Culture, Musique, FIP, Le Mouv'
9) [ ] somafm            SomaFM web radio
10) [ ] virginradio       VirginRadio french radio
a) --all--               install all backends
q) --stop--

Select a backend to create (q to stop): 9
Backend "somafm" successfully added.

After one of more radio modules are added, you can search for radio stations that play a genre:

$ woob radio
Welcome to radio v3.0

Type "help" to display available commands.

Loaded backends: freeteknomusic, somafm

radio> search radio trance
id: thetrip@somafm
title: The Trip
description: Progressive house / trance. Tip top tunes.
current: B With U (Salinas' Dub Mix) - Tommyboy And Soultan Feat. Zara
streams: ['130Kbps/aac' ('http://somafm.com/thetrip130.pls'), 'fast/mp3' ('http://somafm.com/thetrip.pls'), '64Kbps/aacp' ('http://somafm.com/thetrip64.pls'), '32Kbps/aacp' ('http://somafm.com/thetrip32.pls')]

radio:/search> play 1

For this radio station search of “trance” only 1 station was found, so play 1 is used to listen to that station. (Note: if 3 were returned then you could listen to the third station by: play 3).

Bash Scripting for Internet Radio

There are some woob options that will make scripting a little easier. Some genres like “rock” or “ambient” will return too many hits, so the -n option can be set to limit the number of returned items. The -s option will return only selected fields.

$ # Show the id and description 
$ #  for the first 2 ambient station 
$ woob radio search radio "ambient" -s id,description -n 2
id: deepspaceone@somafm
description: Deep ambient electronic, experimental and space music. For inner and outer space exploration.

id: groovesalad@somafm
description: A nicely chilled plate of ambient/downtempo beats and grooves.

An example script (wlisten.sh) takes a music genre entered on the command line and then finds and plays the first related station.

#!/bin/bash
#
# wlisten.sh - find a Woob Radio station and play it
#
echo -e "\nWoob Internet Radio Player\n"

echo -n "Enter a type of Music to listen to: " 
read mtype
echo "Now playing : $mtype music ..."
woob radio play $(woob radio search radio "$mtype" -n 1 -s id | awk '{print $2}' )

To run this script (wlisten.sh) to play a reggae station:

$ bash wlisten.sh

Woob Internet Radio Player

Enter a type of Music to listen to: reggae
Now playing : reggae music ...

This example only uses the first returned station. By using a couple of Zenity list dialongs a Bash script can be created to present a list of preset genres, then the user can select a specific station.

#!/bin/bash
#
# wradio.sh - find a Woob Radio station and play it
#
mtype=$(zenity --title="Play Internet Radio" \
	--list --height=500 --width=300 --column="Music Type" \
        80s Ambient Dance House Jazz Pop Rock Reggae Top Trance) 
# if a music type is selected then get stations
echo "Music Type: $mtype"
if [[ -n "$mtype" ]] ; then
  stn=$(woob radio search radio $mtype -f radio_list  | zenity --list \
    --title="Select Radio Station" \
    --column=station --column=description --width=900 --height=300)
  # Get the station ID, its the 1st part of the station string
  stn_id="${stn%%—*}"
  # If the station string is not empty, play the station
  if [[ -n "$stn_id" ]]  ; then
    echo "Station Id: $stn_id"
    woob radio play $stn_id
  fi
fi

Python Woob Radio Example

To get a selection of radio stations for a genre:

from woob.core import Woob
from woob.capabilities.radio import CapRadio

w = Woob()
w.load_backends(CapRadio)

genre="rock"

for stns in w.iter_radios_search(genre):
        print(stns.id, stns.title, stns.description, "\n")

See the Woob tech api docs for more details on the radio API.

A Python Tkinter GUI example to select a genre and then play a radio station:

#!/usr/bin/python3
#
# wradio.py - Tkinter GUI for Woob Radio    
#
from tkinter import *
from tkinter import ttk
import subprocess
from woob.core import Woob
from woob.capabilities.radio import CapRadio

w = Woob()
w.load_backends(CapRadio)

# Fill the tree with genre stations
def show_stations():
    genre = cbo.get()
    tree.delete(*tree.get_children())
    # Insert Rows into tree based on found radio stations
    for stns in w.iter_radios_search(genre):
        tree.insert('','end',text=stns.id,values=(stns.id, stns.title, stns.description))    

# Play selected station
def tree_getitem(a):
    curItem = tree.focus()
    stn_info=tree.item(curItem) # station is dictionary variable
    thestn=stn_info['text'] # get the station id
    p=subprocess.Popen(["killall", "woob"])
    p=subprocess.Popen(["killall", "mpv"])
    if thestn != "":
        labelText.set("Playing Station: " + thestn)
        p=subprocess.Popen(["woob", "radio", "play", thestn])

def stop_music():
    labelText.set("")
    p=subprocess.Popen(["killall", "woob"])
    p=subprocess.Popen(["killall", "mpv"])
    

# Create an instance of tkinter frame
root = Tk()
genre="Rock"
root.title('Play Internet Radio Stations')
root.geometry('900x300')

# Create a listbox of station genres
label1 = ttk.Label(text = "Enter a genre: ").grid(row=0,column=0)
cbo = ttk.Combobox(root,height = 1)
genres= {"80s","Ambient", "Dance", "House","Jazz","Pop","Rock", "Reggae","Trance"}
cbo['value'] = [m for m in genres]
cbo.grid(row=0, column=1)

# Create a button to update station tree
bt_update = ttk.Button(root,text="Select Genre", command=show_stations)
bt_update.grid(row=0, column=2)

# Create a tree list with 3 columns
tree = ttk.Treeview(root, column=("Id","Station", "Desciption"), show='headings') 
tree.column("#1", width=100)
tree.heading("#1", text="ID")
tree.column("#2", width=200)
tree.heading("#2", text="Station")
tree.column("#3", width=600)
tree.heading("#3", text="Description")
tree.grid(row=1, column=0, columnspan=3)
tree.bind('<ButtonRelease-1>', tree_getitem)

# Create a label to show station being played
labelText = StringVar()
label2 = ttk.Label(textvariable=labelText).grid(row=2,column=0)

# Add a button to start and stop music
bt_stop = ttk.Button(root,text="Stop Music", command=stop_music)
bt_stop.grid(row=2,column=1)

root.mainloop()
    

Summary

The Woob project has a lot of potential, however because it is still in the development stage it can be a little challenging to use.

Due to the lack of examples, I found that I was able to get up and running faster with Bash scripts rather than Python. To build a Bash script I would first test what I wanted to do in the online tool, then I would mimic this in Bash.

1-Line of Bash to add Apps to the System Tray

Putting your commonly used apps and scripts into the Linux system tray could be quite useful.

In this blog I’ll look at two approaches:

  • alltray – a command line utility to dock any program into the system tray
  • yad – (Yet Another Dialog) tool is a Bash GUI builder that also supports trays notifications.

The alltray approach is dead simple and it works for any linux application or script. The yad utility offers a little more functionality by adding the ability to create command line dialogs and it can dynamically change the tray icon, actions, menus and tool tip.

Alltray

To install alltray in Debian/Raspian/Ubuntu:

sudo apt install alltray

An simple alltray example that calls an xterm window with the top utility (to show top running processes):

# Use alltray to put a terminal window app in the the tray
#  usage: alltray [options] ["] <program_name> [parameters] ["]
#
alltray  "xterm -hold -T 'Top Processes' -e 'top'"

As a default alltray will use the icon of the command that is being called.

For this example I used xterm to open a new terminal with the options of: -hold (keep teminal open) , -t ( add title) and -e (execute a command).

If you want to change the font name and size, use the -fa and -fs options, for example:

xterm -hold -fa Monospace -fs 14  -T "Top Processes" -e  "top"

Alltray also supports a custom icon and right-click menus, an example of this would be:

# Show a tray item with a custom icon and right-click menu options
#   syntax for adding menus is: --menu "menu-label: command"
#
alltray  "xterm -hold -T 'Top Processes' -e  'top'" \
  -i /usr/share/icons/Adwaita/256x256/legacy/face-glasses.png \
  --menu "Disk Usage:xterm -hold -T 'df' -e 'df'" \
  --menu "Sensors:xterm -hold -T 'Sensors' -e 'sensors' " 

For this example a custom icon (face-glasses.png) is added along with 2 right-click menu options.

YAD – Yet Another Dialog

The yad utility is a command line dialog tool that supports a good selection of different dialog types, also yad can be configured for system tray applications without needing alltray.

For Debian/Raspian and Ubuntu systems yad can be installed by:

sudo apt install yad

Yad has a lot of options, (see the man pages or help for more details). To create a simple two button dialog:

# Show a simple YAD dialog
yad  --text="SOME TEXT"  --title="My Dialog"

To put this simple yad dialog on the system tray with a custom icon:

# Create a YAD system tray item to call a YALL dialog
#
yad --notification --image="gtk-execute" \ 
  --command="yad --text='SOME TEXT' --title='My Dialog' " \
  --text="My Tooltip"

Icons are fairly easy to manage using the yad tool: yad-icon-browser .

Like alltray, yad supports menus, below is a menu example:

# Create a YAD system tray item with a right-click menu
#
yad --notification --image="gtk-execute" \
 --command="yad --text='SOME TEXT' --title='My Dialog'" \
 --menu="Memory! yad --text='$(vmstat)' --title=VMSTAT \
        | Sensors! yad --text='$(sensors)' --title=Sensors \
        | USB ! yad --text='$(lsusb)' --title=USB \
        | Quit ! killall yad" \
 --text="My Tooltip"

The syntax for menus is:

menu=STRING
              STRING must be in the form:
              menu_label1[! action1[! icon1]]|label2[! action2[! icon2]]....   
              Menus are separated with `|' or --separator  argument.
              Menu items are separated with `!' or --item-separator argument.

For this menuing example I passed the output from command line tools like vmstat, sensors and lsusb to the yad –text parameter.

Unlike alltray, yad doesn’t have a built in quit menu option, but this functionality can be added with:

Quit ! killall yad

Remotely Change a YAD Tray Item

The yad notification option has a –listen parameter that allows commands to be sent from stdin (standard input) to yad in the form command:args. Possible commands are icon, tooltip, visible, action, menu and quit.

The yad stdio can be redirected a named pipe and this will enable other bash scripts to be able to send it commands. Below is a basic Bash script that creates a named pipe variable (mytraypipe=”/tmp/tray1.pipe”) and then it creates the named pipe if it doesn’t exist.

The Bash command: exec 1<> $mytraypipe , redirects stdio (file 1) to the named pipe. The final step is to call the yad with <&1 , to redirect the stdio into the command.

#!/bin/bash
#
# dyn_tray.sh - create a system tray item that can be modified
#             - write changes to the named pipe: $mytraypipe 
#
mytraypipe="/tmp/tray1.pipe"

# Make the pipe if required
if ! test -e "$mytraypipe"; then
  mkfifo $mytraypipe
fi

# redirect the stdio (file 1) to the named pipe
exec 1<> $mytraypipe

# create the notification icon
yad --notification                  \
    --listen                        \
    --image="emblem-colors-grey"              \
    --text="Dummy tooltip"   \
    --command="yad --text='Test Tray App' " <&1

Below is an example that changes the icon and tool tip for the earlier Bash script. This first step is to define a variable with the correct named pipe, after this commands can be send with an echo statement to the named pipe.

Summary

If you are looking for a quick way to pull together some commonly used apps and scripts both alltray and yad offer a simple 1 line of Bash solution.

If you want to dynamically change the tray item then yad is the tool for you.

Charts in 1 Line of Bash

There are some great charting packages out there, but for quick tests and playing around it’s nice to keep things simple. I wanted to do some basic Raspberry Pi charting and found that with the Gnuplot utility it took as little as 1 line of Bash to get the job done.

This blog shows some examples, probably the key points are:

  • Data can be piped to Gnuplot
  • Bash While-loops can be used to create real time/dynamic plots

Single Line Chart from a File

Gnuplot plot has a full scripting interface that allows users to create very complex presentations.

Gnuplot script can also be passed on the command line, with the options: -p (persist, spawn chart) and -e (execute command):

$ # Plot a file with 5 points, show file first
$ cat 5pnts.txt
2.7
3.5
4.1
3.9
5.6
$ # Plot the file as a line
$ gnuplot -p -e "plot '5pnts.txt' with lines title '5 Test Points' "

Multi-line Chart

The multi-line example is like the single line chart with the exception that the using statement is called for each plot line, 1:2 is the first line (x-axis column):(y-axis column) , and 1:3 is the second series line.

$ # Show the first 5 line of the data
$ head -n 5 data1.txt, Note: Gnuplot skips row starting with "#"
#t  Angle Error 
0.0	-14.7	3.6
1.0	8.6	    3.6
2.1	28.8	3.0
3.1	46.7	3.4
$
$ # plot, first column is x, next 2 columns are y series
$ gnuplot -p -e "plot 'data1.txt' \
  using 1:2 with lines title 'Angle','data1.txt' \
  using 1:3 with lines title 'Error'"

For-Loop Piped to a Line Chart

Rather than creating a text file, the output from a group of echo statements can be piped to Gnuplot. The ‘<cat’ definition is used for piped input.

$# Plot 10 random points
$ ( for i in `seq 1 10`; do echo "$RANDOM\n";  done )| \
  gnuplot -p -e "plot '<cat' with lines title '10 Random Pts'"

A more complex example would be to add a time on the X-axis and run a calculation at a sampled interval.

# Create a calc function
mycalc() {
  # Show the time and cpu idle time
  thetime=$(date +'%T')
  idle=$(top -n 1 | grep id | awk '{print $8}')
  echo "$thetime $idle\n"
}
# Run the calc function in the for-loop
# Define the y range, x scale as time
( for i in `seq 1 20`; do mycalc; sleep 1;  done )| \
  gnuplot -p -e "set yrange [50:100]; set xdata time; \
  set timefmt '%H:%M:%S';set format x '%H:%M:%S'; \
  plot '<cat' using 1:2 with lines title 'CPU Idle Time' "

Bar Charts from a File

For the bar chart example the plot options are defined in a variable. This example uses the iostat command to get the user and idle time usage. Awk is used to parse the 4th row and then get the individual values.

# Create a file with user and idle CPU usage
# Column: 1=bar position, 2=value, 3=label
iostat | awk '{if(NR==4) print "0 " $1 " user"}' > cpu0.dat
iostat | awk '{if(NR==4) print "1 " $3 " idle" }' >> cpu0.dat

# Define the options as a variable
options="set title 'CPU Diagnostics'; set boxwidth 0.5; \
  set style fill solid; "

# Plot the bars with labels, note: the label offset 
gnuplot -p -e "$options ;plot 'cpu0.dat' \
 using 1:2:xtic(3) with boxes notitle, \
 '' using 1:2:2 with labels font 'Helvetica,15' \
 offset 0.9,0.8  notitle" 

Bars with Dynamic Updates

A Bash while-loop can be used to dynamically get new data and pass it to Gnuplot.

Below is an example that refreshes a bar chart every 5 seconds. The refresh time is shown in the bar series title.

mycalc2() {
  # Show the time and user/cpu idle time
  # note: top gets instantaneous values, iostat uses averages
  top -n 1 | grep %Cpu | awk '{print "0 " $2 " user"}' > cpu0.dat
  top -n 1 | grep %Cpu | awk '{print "1 " $8 " idle"}' >> cpu0.dat
}
# define chart options
options="set title 'CPU Diagnostics'; set boxwidth 0.5; \
  set style fill solid; set yrange [0:100]; set xrange [-0.5:1.5]"

# create an infinite while loop to get data, and then plot
# note1: gnuplot needs a while loop to replot
# note2: use a big pause time in gnuplot or exiting will be tough 
( while :; do mycalc2; sleep 10;  done )| \
gnuplot -p -e "$options ; plot 'cpu0.dat' \
 using 1:2:xtic(3) with boxes \
 title \"$(date '+%T')\" ; while (1) { replot; pause 5 }" 

Summary

Using Gnuplot with the command line option allows for some quick and easy charting. For more detailed work try creating a Gnuplot script file.

Web Scraping with 1 Line of Bash

In the past Python with the Beautiful Soup library has been a great approach for my web scraping.

I was recently doing a small project and I was amazing at what one Bash statement would get me.

My approach was to use the text based Lynx browser and pipe the output to a grep search.

Below is an example where I used Lynx to dump the “Sunshine Village Snow Forecast” web page to find how much snow they had.

The Lynx Text Browser

The first step in web scraping is to get a web page into a searchable format.

I started out by looking at using cURL with the html2text tool, but I found that using the Lynx browser offered a one step solution with a cleaner text output.

To install Lynx on Raspian/Debian/Ubuntu use:

sudo apt install lynx

The lynx -dump option will output a web page to text with HTML tags and Javascript removed. It’s important to note that what you see on the page may not match the outputted text.

Below is an example where I wanted to get the new snow at Sunshine Village. On the web page Javascript is used to show the snow depth as either centimetres or inches, but on the text output both units and their values are shown.

Bash has a good selection of string manipulation tools. Below is an example to extract the first part of string to only show the snow in centimeters (cm):

$ theurl="https://www.snow-forecast.com/resorts/Sunshine/6day/mid"
$ thestr="New snow in Sunshine Village:"
$ # Create a variable with the result string from Lynx
$ newsnow=$(lynx -dump "$theurl" | grep "$thestr")
$ 
$ echo "$newsnow"
   New snow in Sunshine Village:  4.8cm1.9in on Fri 8th (after 3 PM)
$    
$ # Get the first part of the string before "cm"
$ # The %% gets the first part
$ echo "${newsnow%%cm*} cm"
   New snow in Sunshine Village:  4.8 cm

My Final App

We were going on a family ski trip and to get pumped I created a morning notification script that showed the new morning snow and the base.

#!/bin/bash
#
# skitrip.sh - show the Sunshine ski conditions in a notification
#
theurl="https://www.snow-forecast.com/resorts/Sunshine/6day/mid"

# Get the new snow depth
thestr="New snow in Sunshine Village:"
result=$(lynx -dump "$theurl" | grep "$thestr")
newsnow="${result%%cm*} cm"

# Get the base
thestr="Top Lift:"
base=$(lynx -dump "$theurl" | grep "$thestr")

# Show the results in a desktop notification
msg="$newsnow\n$base (base)"
icon="$HOME/Downloads/mountain.png"
notify-send -t 10000000 -i "$icon"  "Sunshine Ski Resort" "$msg"

The notify-send utility will put a message on a Linux desktop, another option could be to send a SMS message.

Summary

Scraping web pages can be tricky and the pages can change at anytime.

I found that Lynx worked on many pages but not all.

The grep utility is extremely useful and it offers a lot of interesting options, such as getting lines before or after the found string.

Text Interfaces with Whiptail and Dialog

Text interfaces are extremely useful when an MS-Window client is trying to connect into a Linux or Raspberry Pi node.

There are some good options for creating Bash text interfaces, two of the most popular are Whiptail and Dialog. Raspberry Pi users will be familiar with Pi configuration tool, raspi-config, which uses Whiptail to create all its menu and application screens.

Whiptail comes pre-installed on Raspbian and on many Linux images. Whiptail is a lighter version of the more complete Dialog package.

This blog will show some examples using Whiptail and Dialog.

Getting Started

For my work I preferred using Dialog, I found that it had many more options and a slightly easier interface than Whiptail. However for projects where I needed to distribute my code to other users or nodes I would keep things simple and use Whiptail.

To install Whiptail and Dialog on a Raspbian/Debian/Ubuntu:

sudo apt install whiptail
sudo apt install dialog

A simple Whiptail test to show the date/time in a message box:

whiptail --title "Time/Date" --msgbox  "$(date)" 0 0

This statement will clear the terminal screen and show a default background with message box centered in the window. A height and width of : 0 0 will autosize the message box.

Refreshing YESNO Dialog

A message box only has a single OK button. A yesno dialog has two button.

Whiptail only supports a basic yesno function. The Dialog utility supports changing the button labels and a timeout so the window can be automatically refreshed.

Below is an example that refreshed an yesno dialog every 5 seconds. The show_dlg function generates a random temperature and humidity valid, and then calls the dialog utility. The YES button, relabelled as “Refresh”, will manually force a refresh of the data and redraw the window. The NO button, relabelled as “Cancel” will close the script and clear the screen.

#!/usr/bin/bash
#
# dyesno.sh - A freshing yes/no dialog with simulated sensor data

YES=0
TIMEOUT=255

show_dlg() {  
   # simulate some data
   data1="Temperature: $(( ( RANDOM % 10 )  + 20 )) C";   
   data2="Humidity: $(( ( RANDOM % 10 )  + 30 )) %";   
   message="$data1\n\n$data2";   
   dialog --begin 2 2 --backtitle "Raspberry Pi Data - $(date +'%T' )" \
       --yes-label "Refresh" --no-label "Cancel" --timeout 5 \
       --title "DHT11 Sensor Results" --yesno  "$message" 8 30; 
} 

response=0
# Cycle is the response is YES button or the dialog timed out
while [ "$response" == "$YES" ] || [ "$response" == "$TIMEOUT" ]; do
  show_dlg
  response=$? ;# Get the output from the yesno dialog
done
clear

Radio Dialog to Toggle Pi GPIO Pins

A radio dialog allows a user to select one option from a list.

For this example I’m using 3 GPIO pins (0, 2, and 7) and the user can select one of these pins and toggle the output using the Raspberry Pi gpio utility.

The Dialog utility will output the item selected (0, 2 or 7), and the OK/Cancel button code (0=OK,1=Cancel).

#!/usr/bin/bash
#
# dradio1.sh - toggle a GPIO output pin
#
thepin=$(dialog --begin 2 2 --title "Toggle GPIO Pins" \
     --backtitle "Raspberry PI GPIO" --stdout \
     --radiolist "Select Pin:" 10 40 3 \
      0 "GPIO_0 - physical pin 11" off \
      2 "GPIO_2 - physical pin 13" off \
      7 "GPIO_7 - physical pin 7 " off )
clear
# Toggle if OK entered and a pin is selected
if [ "$?" == "0" ] && (( ${#thepin} > 0 )) ; then
  echo "Toggling (wPi) pin: $thepin"
  gpio toggle $thepin
  echo "Pin $thepin is: $(gpio read $thepin)"
fi   

Weather Station Form

A Form dialog can be used text with captions and allow user input to saved. Below is an example of a Weather station with view only and editable data.

#!/usr/bin/bash
#
# dform1.sh - Form to show data, and allow data entry

# Weather Sensor inputs (connect real inputs here)
wspeed="5 kph"
wdir="NW (350)"
wtemp="11 C"

# Show a dialog with viewonly and data entry values, save to a file
dialog --begin 2 2 --ok-label "Save" --backtitle "Pi Weather Station" \
     --title "North Beach" --stdout  \
     --form  "Data at : $(date +'%T' )"  12 65 0 \
	"Wind Speed :"      1 1 "$wspeed"  1 30 0 0 \
	"Wind Direction:"   2 1 "$wdir"    2 30 0 0 \
	"Water Temp:"       3 1 "$wtemp"   3 30 0 0 \
	"Beach Conditions:" 4 1 ""  	   4 30 30 30 \
	"Wildlife:"         5 1 ""  	   5 30 30 30 > beach.txt     
clear

The syntax for the form is:

--form text height width formheight [ label y x item y x flen ilen ]

where: y = line position
       x = position in line
    item = view only or editable data
    flen = field length , 0 = view only
    ilen = input length , 0 = view only

For this example the last 2 items (Beach Conditions and Wildlife) have a field and input length defined to 30 characters so data can be entered into these fields. If the OK button is selected the user entered data is saved to the file beach.txt.

Menu Example

Menuing applications are probably the most useful feature in the Whiptail and Dialog utilities.

Below is an System Information application, that has 3 options: node, disk space, and memory stats. Each of the menu items call a display_dialog function that presents the results of a Bash statement in a message box.

#!/bin/bash
#
# dmenu.sh - Bash Dialog Menu showing some system stats
#

# Display menu results in a msgbox
display_result() {
  dialog --title "$1" \
    --backtitle "System Information" \
    --no-collapse \
    --msgbox "$result" 0 0
}

while true; do
  selection=$(dialog --stdout \
    --backtitle "System Information" \
    --title "Key Features" \
    --clear \
    --cancel-label "Exit" \
    --menu "Please select:" 0 0 4 \
    "1" "Display Node Information" \
    "2" "Display Disk Space" \
    "3" "Display Memory Stats" \
     )
  exit_status=$?
  if [ $exit_status == 1 ] ; then
      clear
      exit
  fi
  case $selection in
    1 )
      result=$(echo "Hostname: $HOSTNAME"; uptime)
      display_result "System Information"
      ;;
    2 )
      result=$(df -h)
      display_result "Disk Space"
      ;;
    3 )
      result=$(vmstat --stats)
      display_result "Memory Stats"
      ;;
  esac
done

Changing Whiptail Default Colors

Whiptail uses the newt graphic library. A NEWT_COLORS variable can be created with custom colors. An example would be:

export NEWT_COLORS='
  window=,red
  border=white,red
  textbox=white,red
  button=black,white'
# to reset the color back to default use:
# unset NEWT_COLORS

A full definition of all the options and colors:

root                  root fg, bg
border                border fg, bg
window                window fg, bg
shadow                shadow fg, bg
title                 title fg, bg
button                button fg, bg
actbutton             active button fg, bg
checkbox              checkbox fg, bg
actcheckbox           active checkbox fg, bg
entry                 entry box fg, bg
label                 label fg, bg
listbox               listbox fg, bg
actlistbox            active listbox fg, bg
textbox               textbox fg, bg
acttextbox            active textbox fg, bg
helpline              help line
roottext              root text
emptyscale            scale full
fullscale             scale empty
disentry              disabled entry fg, bg
compactbutton         compact button fg, bg
actsellistbox         active & sel listbox
sellistbox            selected listbox

bg and fg can be:

color0  or black
color1  or red
color2  or green
color3  or brown
color4  or blue
color5  or magenta
color6  or cyan
color7  or lightgray
color8  or gray
color9  or brightred
color10 or brightgreen
color11 or yellow
color12 or brightblue
color13 or brightmagenta
color14 or brightcyan
color15 or white

Changing Default Dialog Colors

The custom Dialog colors are defined in the file: ~/.dialogrc

To create and edit this file:

dialog --create-rc  ~/.dialogrc
nano $HOME/.dialogrc

Within the ~/.dialogrc file, an important option is:

# Shadow dialog boxes? This also turns on color.
use_shadow = OFF

Dialog supports inline color (this isn’t supported in Whiptail) with the –colors option. Inline colors are defined with by “/Zx”:

ANSI colors:

/Z0 = black
/Z1 = red
/Z2 = green
/Z3 = yellow
/Z4 = blue
/Z5 = magenta
/Z6 = cyan
/Z7 = white

Other options:

/Zb = bold, /ZB = reset bold
/Zr = reverse, /ZR = reset reverse
/Zu = underline, /ZU = reset underline
/Zn = restore settings to normal

An example:

$ msg="Temperature: \Zb\Z3 28 \Zn Deg C"
$ dialog --title "Outside" --colors --msgbox  "$msg" 0 0 ; clear

Final Comments

I’m a big fan of Zenity and YAD X-Window dialog tools, and I found that it wasn’t a big transition to use Whiptail and Dialog.

It’s important to note that the Dialog option –stdout is needed if you want to pass the Dialog output to a variable. Passing the output from Whiptail is a little trickier, use: 3>&2 2>&1 1>&3

Xonsh: a Python and Bash Shell

Xonsh is a Python-powered, cross-platform shell language and command prompt, that works on all major systems including Linux, OSX, and Windows.

Bash scripts are fast and effective for small or batch applications. One of the limitations in Bash is it’s handling of math functions and floating point numbers.

Python is super popular for new programmers and it has a huge library of available functions.

For Raspberry Pi users Xonsh can offer a lot of opportunities to write some extremely lean scripts. Python can be used to connect to 3rd party devices and sensors and 1 line of Bash can be used for dialogs.

In this blog I will introduce Xonsh with some Raspberry Pi examples.

Getting Started

See the Xonsh documentation for installation directions on your specific system. For installation on Raspberry Pi/Ubuntu/Debian enter:

sudo apt install xonsh

To run the Xonsh, simply enter: xonsh

Out of the box Xonsh offers a configuration wizard and a tutorial.

Using Python

Python code can entered directly at the command line. The version of Python will depend on what is loaded on the base system. To check your version:

$ import sys
$ sys.version
'3.9.2 (default, Mar 12 2021, 04:06:34) \n[GCC 10.2.1 20210110]'

Like the interactive Python interface, print statements are not required to see the output:

$ 5 + 6
11
$ a=4;b=7
$ a+b
11
$ msg1="hi"
$ msg1 + " world"
'hi world'

Using Bash

Xonsh uses Python first, so an example with ls (the Bash list command):

$ # This first ls is used as a Bash list command
$ ls
Adafruit_DHT           LICENSE      README.md  dist      setup.py
Adafruit_DHT.egg-info  MANIFEST.in  build      examples  source
$ ls="this is a variable"
$ # Xonsh show the variable ls
$ ls
'this is a variable'

In the above example ls is first used as the Bash list command, but if a variable is defined with the same name, the variable is referenced.

Xonsh processes a Bash statement as a single line. This means that:

  • Bash for/while/if statements need to be all on one line
    • Remember to use white space between characters
  • Line extensions (with a “\”) are not supported
  • Bash functions are not supported.
    • You can write the function in Python instead of Bash

Using Python in Bash

Python statements are using within Bash by: @(Python statements). Below are two examples of using Python in Bash:

$ import sys
$ echo @(sys.version)
3.9.2 (default, Mar 12 2021, 04:06:34)
[GCC 10.2.1 20210110]
$ echo @("Answer=" +str(5+6))
Answer=11

Using Bash in Python

Bash variables can be used directly in Python, for example:

$ #Use Bash date and pass it to Python
$ now=$(date)
$ print("Now is: " + now)
Now is: Thu 24 Mar 2022 03:59:32 PM EDT

Pi Example: Show CPU Temp

Showing the Raspberry Pi’s CPU temperature is a simple example to show how Bash and Python can be used effectively together. Pi’s CPU temperature is stored internally as milli-DegC (1,000 times the degrees in Celsius). Floating point math in Bash is awkward, so this can be done in Python.

Below is a Xonsh script to read the CPU temperature, convert the value to one decimal point, and then present the result in a Bash/Zenity dialog:

#!/bin/xonsh
#
# temp2dlg.sh - use xonsh to get Pi temperature and show in a dialog
#

# Get the temperature with a Bash "cat" and variable
mtemp=$(cat /sys/class/thermal/thermal_zone0/temp)

# Use Python to convert a string of milli-DegC to DegC with 1 decimal
btemp=str(round(int(mtemp)/1000,1)) + " Deg C"
# Add some formatting
btemp="<span font='32' foreground='red'>" + btemp + "</span>"

# Use a Bash Zenity dialog to show the result
zenity --info --text=@(btemp) --title=Raspberry_PI_CPU_Temp  --width=300 &

To make the script executable and then run it:

$ chmod +x temp2dlg.sh
$ ./temp2dlg.sh

Raspberry Pi DHT11 Sensor Value

For the next project I used a DHT11 temperature/humidity sensor.

My goal for this project was to do a snapshot and present the data in a list dialog.

The code to read and present the data was only 7 lines. The Bash variable is passed between Bash statements just like a Python variable would be passed with the @(Python_statement) syntax. Really the only ugly part of the code was the long Zenity statement.

#!/bin/xonsh
#
# dht11_dlg.sh - using xonsh show DHT11 sensor data on a dialog
#
import time
import Adafruit_DHT
sensor=Adafruit_DHT.DHT11
gpio=17

humidity, temperature = Adafruit_DHT.read_retry(sensor, gpio)

thetime=$(date)

zenity --list --title=DHT11_Sensor_Data  --text=@(thetime) --column=Sensor --column=Value --column=Units Humidity @(humidity) "%" Temperature  @(temperature)  "Deg C"

Issues with Shells

Working between different shells and sub-shells can be a little confusing. I found that I occasionally got confused which shell I was working in. The ps command would tell me if xonsh is running:

I was able to pass an Xonsh script to a Bash script without any issues, but I found that for certain operations I needed to manually kill a Xonsh shell.

Below is an example using the DHT11 sensors, and the YAD command line dialog tool (install by: sudo apt install yad). This example had a Python function (show_data) that cycled every 2 seconds and piped the new sensor data to the dialog. Unfortunately the Xonsh shell was still running even after the dialog closed so I needed to use the pkill utility to terminate xonsh .

#!/bin/xonsh
#
# dht11_data.sh - using xonsh show DHT11 sensor data on a dialog
#
import time
import Adafruit_DHT
sensor=Adafruit_DHT.DHT11

gpio=17
# show_data - format sensor data for YAD multi-process bar format
def show_data():
	while True:
		time.sleep(2)
		humidity, temperature = Adafruit_DHT.read_retry(sensor, gpio)
		echo @("1#" + str(temperature) + " Deg C")
		echo @("1:" + str(temperature))
		echo @("2:#" + str(humidity) + " %")
		echo @("2:" + str(humidity))

echo "Showing realtime data..."
@(show_data)  | yad  --multi-progress --bar=Temperature --bar=Humidity --title=DHT11_Sensor_Data 
# exit and ensure that the subprocess is killed
echo "Exit and kill subprocess if running..."
pkill -f xonsh &

Summary

Xonsh has a lot of potential for users looking for simple scripting solutions.

For myself I’ll probably stick to either Bash or Python solutions, but I like that I have other options.

Gemini Protocol for Lightweight Internet Apps

The Gemini protocol is a very simple and light Internet protocol.

Unlike HTML files that contain layers of tags, style sheets and Javascript, a Gemini document is a simple readable document.

In this blog I wanted to document:

  • Gemini servers and clients using only 1 line of Bash
  • How to use large ASCII text in Gemini documents
  • How to create simple bar charts
  • How to create Gemini CGI apps in Python

Getting Started

A Gemini document only supports a few statements and graphic images such as JPEG or PNG are not supported.

An example Gemini document with the common formatting options:

# Heading level 1 (H1) 
## Heading level 2 (H2)
### Heading level 3 (H3)

=> testpage.gmi A link to another page.

> This line will show as a block-quote. 

A list of items
* This is the first list item.
* This is another list item.

```
Code or ASCII Block 
   _   ___  ___ ___ ___   _____       _   
  /_\ / __|/ __|_ _|_ _| |_   _|__ __| |_ 
 / _ \\__ \ (__ | | | |    | |/ -_|_-<  _|
/_/ \_\___/\___|___|___|   |_|\___/__/\__|

```

Within a Gemini browser this file would look like:

Content Type

The content type is used by browsers and applications to determine how to manage the requested file.

Typically the content type is managed by server, for example a web server will send a HTTP/1.0 200 OK prior to sending the HTML file.

For the Gemini protocol the content type is: 20 text/gemini . Depending on the Gemini server the user may need to be add the content type manually. (More about this in Bash and CGI servers).

Simple Bash Gemini Servers and Clients

For basic testing a one line Bash statement can be used for custom Gemini servers and clients.

The Gemini protocol typically uses SSL (Secure Sockets Layer) and TLS (Transport Layer Security) encryption so the Bash ncat utility is needed (Note: the nc command does not support SSL).

Below is an example of single Gemini request:

The Gemini server is defined by using the -l , listen option. When a client requests data, the cat statement is used with a pipe (|) to output the file testpage.gmi.

The Gemini client echo’s a “from PC” message with its request, this helps identify which client is requesting data.

A simple Bash ncat statement doesn’t manage the content type so a “20 text/gemini” line is added to the top of the test page.

Dynamic Bash Data

In the earlier example the Bash server is only supporting 1 request then exiting.

A while loop can be added to pass a Bash script to the ncat statement. Below is an example of Bash script (showdata.sh) that show CPU data using the vmstat utility:

#!/bin/bash
#
# showdata.sh - Output data for Gemini Bash Server 
#
echo "20 text/gemini"
echo  " "
echo "#VMSTAT"
echo  " "
date +"%T"
echo  " "
# set Gemini formating for ASCII
echo  "\`\`\`"
#vmstat
top -n 1
echo  "\`\`\`"

To make the script executable use the command: chmod +x showdata.sh

The Bash command to run this script as a Gemini server is:

while true; do ./showdata.sh | ncat  -l -p 1965 --ssl; done

The earlier Bash Gemini client command can be used, or a Gemini browser/client app can be used. The handling of SSL/TLS encryption will vary with the client app. I used the very basic Zain app (https://gitgud.io/sathariel/zain) :

(Note: for the Zain client I needed to load tcl/tls, sudo apt-get install -y tcl-tls)

Using a 1 line Bash Gemini server is great for basic testing but I wouldn’t recommend if you want to connect to variety of different Gemini client applications.

Large ASCII Text

Gemini documents don’t support different font sizes, a workaround is to use the figlet tool to generate multi-line text strings. Figlet is installed on Ubuntu/Debian/Raspbian by:

sudo apt install figlet

Figlet has a number of font styles that use 2-5 line height characters:

When using ASCII headings the Gemini code formatting option should be used, and this has three backticks (“`) before and after the headings.

The earlier example can be modified to have ASCII headings:

#!/bin/bash
#
# showdata2.sh - Output Large Headings to a Gemini Bash Server 
#
echo "20 text/gemini"
echo  " "
echo  "\`\`\`"
# Generate large text 
figlet -f standard "Raspberry Pi"
figlet -f small -m 2  $(date +"%T")
# show CPU stats
vmstat
echo  "\`\`\`"

Bar Charts

Horizontal ASCII bar charts can be created by using the printf statement with different ASCII fill characters. For example:

# show a label with bar and value text 
#
label="temp"; val=20;
bar="printf '█%.0s' {1..$val}" ; 
printf '\n%-5s ' $label; eval $bar
printf '░%.0s' {1..5} ;
printf ' 50 C\n'

temp  ████████████████████░░░░░ 50 C

This bar logic can be using in a Raspberry Pi Stats page that looks at idle time and free space on the SD card:

#!/bin/bash
#
# pi_stats.sh - Output Pi Stats as a Gemini Page
#
echo "20 text/gemini"
echo  " "
# Put the rest of the Gemini document into code block mode
echo  "\`\`\`"
# Generate large text 
figlet -f standard "Pi Stats"

# Show the time
echo "Time: $(date +'%T')"
echo ""

# Get idle time, scale 0-50
idle=$(top -n 1 | grep id | awk '{print $8}')
barsize=$(echo $idle | awk '{printf "%0.f" , $1/2}')
thebar="printf '█%.0s' {1..$barsize}"
graysize=$(expr 50 - $barsize)
thegray="printf '░%.0s' {1..$graysize}"
printf 'Idle Time  '; eval $thebar; eval $thegray ; echo " $idle %"
echo  ""

# Get free space on SD card, scale 0-50
freesp=$(df | grep root | awk '{printf "%0.f", $5/2}')
barsize=$(echo $freesp | awk '{printf "%0.f" , $1/2}')
thebar="printf '█%.0s' {1..$barsize}"
graysize=$(expr 50 - $barsize)
thegray="printf '░%.0s' {1..$graysize}"
printf 'Free Space '; eval $thebar; eval $thegray ; echo " $freesp %"

echo  "\`\`\`"

To run this page use

while true; do ./pi_stats.sh | ncat  -l -p 1965 --ssl; done

Python CGI Pages

There are a number of good Gemini servers, for my testing I used the Python based Jetforce server, it is installed by:

pip install jetforce

To run the Jetforce server it is important to define a home directory, the allowable hosts that can connect (0.0.0.0 is all IP4 nodes) and the server’s hostname:

# Start the jetforce Gemini server for all IP4 hosts
jetforce --dir /home/pi/temp --host "0.0.0.0" --hostname 192.168.0.105 &
# Start jetforce without hardcoding hostname
# jetforce --dir /home/pi/temp --host "0.0.0.0" --hostname $(hostname -I) &

By default CGI (Common Gateway Interface) files are defined in the directory cgi-bin which is under the home directory.

Unlike a one-line Bash server, Jetforce server will pass environment variables like QUERY_STRING and host and remote connection information.

CGI programs can be written in a variety of programming languages. For this example I wanted to pass Raspberry Pi BME280 temperature, pressure and humidity sensor information to a Gemini CGI page.

The CGI program was written in Python and I installed a bme280 and figlet library:

pip install RPi.bme280
pip install pyfiglet

The Python script (sersor2gmi.py) outputs a Gemini content type, a Figlet title and then the sensor data values:

#!//usr/bin/python3
#
# sersor2gmi.py - send BME280 sensor data to a Gemini page
#
import smbus2
import bme280
import pyfiglet

# find device address via: i2cdetect -y 1
port = 1
address = 0x77
bus = smbus2.SMBus(port)

calibration_params = bme280.load_calibration_params(bus, address)

# the sample method returns a compensated_reading object
data = bme280.sample(bus, address, calibration_params)

# Output a Gemini page
print("20 text/gemini") #Note: Some Gemini CGI servers may do this
print("```") # use code mode
#print(pyfiglet.figlet_format("Pi BME280 Data", font = "small"))
print(pyfiglet.figlet_format("Pi BME 280 Data", font = "small"))

print("Temperature:" + "{:5.1f}".format(data.temperature) + " C" )
print("Pressure:   " + "{:5.1f}".format(data.pressure) + " kPa" )
print("Humidity:   " + "{:5.1f}".format(data.humidity) + " %" )
print("```") 

This file was added to the cgi-bin directory and it is made executeable (chmod +x sersor2gmi.py).

Below is the output seen in the Lagrange Gemini browser:

Final Comments

There are a variety of Gemini browsers for Windows, Android, Mac and Linux so if you’re looking for a quick and dirty internet solution Gemini might be a good solution.

I like how Gemini documents are totally readable, I can’t say the same for most web pages.

The one thing that I missed with Gemini pages is the ability of show nice charts, text based bars work for simple stuff but doing text based line charts is a little too ugly for me.

SMath – A Free MathCAD Alternative

For engineering, physics and math user MathCAD is a great package for theoretical calculations and reports. Unfortunately MathCAD is rather expensive for students and causal users, and it also isn’t support in Linux.

SMath Studio is a free alternative that supports MathCAD files and it works in Linux.

In this blog I wanted to highlight some of my notes in using SMath Studio. Some of the key take away points are:

  • SMath offers a Worksheet approach that is far superior to Excel/LibreOffice for math problems.
    • Equations and matrices are shown like they were hand drawn
    • Units are included in the equations, and unit conversion is automatic
    • easy to read IF/THEN/ELSE, WHILE and FOR LOOP can be using within a Worksheet
  • SMath programming is done visually, (as opposed to writing code like Python or Matlab/Octave).
  • SMath support basic plotting but it is weak compared to other packages like Matplotlib.
  • SMath is good for visual reports, but Python would be a better solution for managing large amounts of data or when statistical calculations are needed.

Getting Started

SMath Studio is supported on Window, MacOS and Linux, see https://en.smath.com/ for installation files.

For Linux user the Mono interface will need to be loaded:

sudo apt install mono-devel
# then download : SMath Studio Desktop for Mono  
# extract to a folder
# to run SMath Studio in Linux:
./smathstudio_desktop_mono

The SMath Studio has a side panel of common functions and symbols. All the functions are listed in the fx dialog.

Freehand Calculations

Smath allow users to create complex freehand calculations using the side panel and some keyboard shortcuts. The arrow key moves the cursor to different sections of a large expression that need to be modified.

Variables and Units

Sheets are calculated from top to bottom, so it is important to define a variable before it is used. A variable is defined with a “:=“, the equal sign (“=“) is used to show the value of a variable.

For equations where units are used a dropdown of available units is shown. The default system is in metric so the conversion is also shown. Units will appear in blue.

The system will automatically manage the conversion between units, and there are some different presentations to show the steps in the conversion, fractions, or just the numeric value.

Show Solutions

Pages can be laid out to create clear presentations with variables that are passed to equations. Like with the units, equations can be shown with their intermediate values. Below is an example that shows a venturi flow rate calculation with the intermediate values and the conversion between Newtons (N), kilograms (kg) and meters (m) to a flow rate of cubic meters/sec.

Plotting and Advanced Functions

Below is a worksheet that allows a user to play with a 2nd order equation. SMath offers some graphical widgets such as: analog sliders, dropdown lists, up/down selector and toggle buttons. For this example I used three slider widgets allow users to change the constants (a,b,c). A plot is used to visually check if the equation has any X-intercepts. The SMath solve function will find X-intercepts. A small if statement is used to check if there are any intercepts. Finally an integration function calculates the area between the limits.

File I/O

SMath supports import/export data functions for: CSV, PDF, XLS, ODF, ODS and a few other. SQLite3 databases are also supported

Below is an example worksheet where a CSV file is imported, cleaned and then exported out to a new CSV file. The data is read in with the importData function to a matrix variable (rawdata).

A for loop iterates through each row data, and an if statement checks a row for positive values. The stack function appends valid rows of data to the cleandata matrix. The last step is to use the exportData_CSV function to write out the cleandata matrix.

Automating SMath

The Windows version of SMath Studio offers a number of command line options that can be used to save and print worksheets. Unfortunately these options don’t exist in the Linux (Mono) version.

As a work around it’s possible to use some keyboard automation tool in Linux to offer some interesting solutions. Below is a script to:

  • open SMath with a user defined worksheet
  • grab the SMath window focus
  • Save as PDF
  • Quit SMath
#!/usr/bin/bas
#
# auto_smath.sh - open
#
myfile="quad" ; # prefix of smath file
rm $myfile.pdf ; # remove existing PDF

# Open smath with a set file, quiet output 
./smathstudio_desktop_mono $myfile.sm  &
sleep 3

# Set focus to SMath
wmctrl -a "SMath Studio"

# Send keystroke to save as PDF
xdotool key alt+F 
sleep 1
for i in {1..4}
do
  xdotool key Down 
  sleep 1
done
xdotool key Return
sleep 1
xdotool key Tab 
sleep 1
xdotool key p
sleep 1
xdotool key Return
sleep 3 ; # Give time to save PDF before exiting

# Exit SMath
xdotool key alt+F4
echo "Done Running $myfile.sm ... saved to $myfile.pdf"

This script uses the wmctrl is get and manage the window focus, and xdotools to send keystrokes. They can be installed by:

sudo apt install wmctrl
sudo apt install xdotool

Some Final Thoughts

I wish I had SMath when I was in university it would have made things so much easier and reduced a lot of manual calculation errors.

It takes a few hours to get used to the keyboard short cuts and formatting.

A few times I forgot about the fact that sheets are calculated from top to bottom and this caused me to get some strange errors. If the software locks up use the following line to kill things:

ps -e | grep mono | awk '{system("sudo kill " $1 "  1>&-")'}