Simple Cross Platform Apps with Ren’Py

There are some excellent cross platform development tools out there. However for new programmers many of these tools can come with a steep learning curve.

Ren’Py is a visual novel engine that has been around for over 10 years. The Ren’Py software development kit (SDK) is supported on Linux, MacOS and Window and the Ren’Py final application can be built to run on Android, iOS Linux, MacOS, Window and HTML5.

Ren’Py uses a simple “Screen Language” so no previous programming experience is required. For more complex requirements Ren’Py supports Python.

The focus of Ren’Py is visual story telling and games, but new programmers can also create some simple apps that can be run on PCs, smart phones and web servers.

This blog will introduce Ren’Py with three examples:

  1. the start of visual novel with a couple of characters
  2. a tourist guide with graphic menus
  3. a dashboard with dynamic values and bars

Getting Started

If you want to do some playing of Ren’Py on a Raspberry Pi a light weight installation can be loaded by:

sudo apt-get install renpy

The apt-get version of Ren’Py however will not have any tutorials or extra build features so it is recommended that you go to the Ren’Py site for the complete installation directions.

The Ren’Py user interface will create new projects with all the required files and prompt the user for any additional requirements.

The script.rpy file contains all the logic for an application.

A Visual Novel

A visual novel requires some characters and background images.

Creating character drawings from scratch can be a lot of work, luckily there are some open source solutions such site as: https://charactercreator.org.

The character creator site allows for the creation of head, torso and head images. It also supports different facial expressions.

Below is a Ren’Py example showing a background, with a policeman and a suspect.

The script.rpy file contains the Screen Language code.

The first step is to define all the characters, in this game there two characters “cop” and “me”.

Ren’py uses labels to jump between code segments. The application begins are the start label (line10).

The image files are stored in game/images directory. Images can also be resized, rotated, moved or adjusted (Line 6). The image file cop_head.png can be referenced directly, and shown by:

show cop_head at truecenter

Dialog is shown by referencing the character and then the dialog text (lines 19-20).

By using hide and show statements different characters and backgrounds can presented. The applications ends with a return statement.

A Tourist Guide

For most graphic novels menus are required. These menus allow the application to have multiple outcomes or branches.

A Ren’Py menu is created by simply using a menu: statement. Each menu item is defined with button text and a jump statement. A jump statement is like an old-school BASIC GOTO statement, each jump has a label that the code will link to.

The example below is the start of a tourist guide for the Bruce Peninsula. The menu is called at the start of program. A menu item will jump to a specific section of the code using a label. Within a subsection, (like the beach), a new background and text is shown. After the user sees this information a jump start statement puts them back to the main menu.

For main menus it is probably cleaner to have subsections defined as separate label sections. However for smaller application it is possible to put the submenus and the display logic directly in the menu: logic.

Dynamic Screens

Ren’Py supports screens and the use of Python for applications that require more complex functions.

For the next example some CPU stats will be shown. The Linux sensors command will return the CPU and Ambient temperature:

$ sensors
dell_smm-virtual-0
Adapter: Virtual device
Processor Fan: 2691 RPM
CPU:            +53.0°C  
Ambient:        +41.0°C  

# use show Bash/AWK code to get temps
$ sensors | grep CPU | awk '{ printf "%d\n" , $2}'
53
$ sensors | grep Ambient | awk '{ printf "%d\n" , $2}'
41

A Ren’Py application can be created to show the CPU stats in a dynamic screen.

The code below defines a Ren’Py screen object and some Python code is used to shell out to the Bash/AWK statements.

A Ren’Py supports Python in either a single statement or as a python block. A single Python statement is define with a leading $, for example to set a variable: $biking = “YES”. A Python block is defined with a starting python: statement, then the code is indented from this statement.

The init python: statement is a code Python code block that is used to define (import) libraries and to initialize variables.

The Ren’Py screen object supports labels, text and bars that can be arranged or oriented in horizontal (hbox) or vertical (vbox) groupings.

# The script of the game goes in this file.

screen ts():

    python:
        now = datetime.now()
        nowtime = now.strftime("%H:%M:%S")
        ctemp = subprocess.check_output("sensors | grep CPU | awk '{ printf \"%d\" , $2}'", shell=True)
        atemp = subprocess.check_output("sensors | grep Ambient | awk '{ printf \"%d\" , $2}'", shell=True)
    
    frame:
        has vbox
        label "CPU Stats" text_size 120 
        
        text ""
        vbox:
            text "Time : [nowtime]" size 80
            text "Ambient temp: [atemp] C \n" size 80
            text "   CPU temp : [ctemp] C " size 60
        hbox:
            vbox:
                text "0 " size 40
            vbox:
                bar value atemp range 60 xalign 50 yalign 50 xmaximum 600 ymaximum 50 
            vbox:
                text " 60 " size 40


init python:

    import subprocess
    from datetime import datetime

label start:

    # Start with Weather screen

    show screen ts()
    # Cycle every 2 second to refresh the data

    define cycle = "True"

    while cycle == "True" :
        $ renpy.pause(2)

    return

Building a Final Application

The Ren’Py IDE Build option supports Windows and Linux apps directly. For Mac and Android builds the user will be prompted for some dependencies that will need to be loaded.

The HTML5 Build is still in beta, it appears to work well for standard graphic novel apps, but I found that it had some issues with Python library calls.

The HTML5 Build will create a separate library structure for the Ren’Py application which will need to be mapped into your Web Server configuration.

If you are looking to do some simple testing a Python standalone webserver can be run from the Ren’Py web directory:

# Run a python standalone server on port 8042
python3 -m http.server 8042

Final Thoughts

If you’re looking to create some simple cross-platform visual presentation apps Ren’Py is a good fit. Ren’Py is super easy to learn and don’t have to have strong programming skills.

Ren’Py supports simple screens that can have dynamic bars and text, however if you’re looking to incorporate gauges or line charts Ren’Py probably isn’t the best fit.

COBOL Programming in 1-hour

This morning I read about 60 year old COBOL being still alive and there was a need for programmers. I couldn’t believe it so I checked the job boards and I found 2 jobs in my immediate area.

I thought that I’d spend an hour or so looking into COBOL. Ok I’m definitely no way near an expert, but I was surprised how much I was able to pick-up.

Getting Started

Traditionally Cobol (Common Business Oriented Language) only ran on mainframe computers. Now luckily you can install a full working environment on Windows and Linux.

If you are working in Windows there are few options: NetCobol, GnuCobol, and the Hercules Emulator

I work primarily in Linux and found that OpenCobol was very easy to use. If you’re working in Ubuntu, Debian or on a Raspberry Pi enter:

sudo apt-get install open-cobol

An Input/Output Program

A Cobol program starts with 2 lines, and an IDENTIFICATION , and a PROGRAM-ID. Comments lines start with a *. All Cobol programming lines end with a period (.).

For this example 4 variables were defined in the DATA layout section. The PIC (picture) argument is used to define the type of variables X is alpha, and 9 is numeric.

*>  test2.cbl - get user input, and do output
*>	  
IDENTIFICATION DIVISION.
PROGRAM-ID. HELLO.

DATA DIVISION.
   WORKING-STORAGE SECTION.
   01 WS-ID PIC 9(3) VALUE 101.
   01 WS-STUDENT-NAME PIC X(25).
   01 WS-DEPT PIC X(25) VALUE 'Engineering'.
   01 WS-DATE PIC X(10).

PROCEDURE DIVISION.
   DISPLAY "Enter Your Name:".
   ACCEPT WS-STUDENT-NAME.
   ACCEPT WS-DATE FROM DATE.
   DISPLAY " ".
   DISPLAY "Name :  " WS-STUDENT-NAME " ID: " WS-ID.
   DISPLAY "Dept :  " WS-DEPT.
   DISPLAY "Date :  " WS-DATE.

STOP RUN.

For this example the ID and Dept are predefined. The WS-DATE is set by today’s DATE.

The WS-STUDENT-NAME is entered by the user.

To compile and run the program (ctest2.cbl) :

$ cobc -free -x -o ctest2 ctest2.cbl
pete@lubuntu:~/Writing/Blog/cobol$ ./ctest2
Enter Your Name:
Pete
 
Name :  Pete                      ID: 101
Dept :  Engineering              
Date :  210701    

Math

Cobol uses VERBs for math and conditional statements. Below is an example doing some basic math with an IF condition:

*> cmath.cbl - do some math with an IF condition
*>
IDENTIFICATION DIVISION.
PROGRAM-ID. HELLO.

DATA DIVISION.
   WORKING-STORAGE SECTION.
   01 A PIC 9(5) VALUE 10.
   01 B PIC 9(5) VALUE 20.
   01 C PIC 9(5) VALUE 0.
   01 D PIC 9(5) VALUE 0.
   01 E PIC 9(5) VALUE 0.
	  
PROCEDURE DIVISION.
   ADD A B TO C.
   MULTIPLY A BY B GIVING D.
   IF D > 100 THEN
      MOVE D TO E
   ELSE
	  MOVE 99 TO E
   END-IF.
   
   DISPLAY "C = " C.
   DISPLAY "D = " D.
   DISPLAY "E = " E.
STOP RUN.

Note for the IF statement there is no end of statement (.) until END-IF.

To compile and run this program:

$ cobc -free -x -o cmath cmath.cbl
$ ./cmath

C = 00030
D = 00200
E = 00200

File I/O – Read

Cobol has a number of read/write formats. The easiest is a fixed field sequential read/write.

For an example with input.txt , the STUDENT-ID is 5 characters and the name is 25.

20003 Albert Smith            
20004 Malika Jones            
20005 Bob Smith 

The Cobol code has a FILE section that defines the input fields. A WORKING-STORAGE section mimic the input file structure, with the exception of a end of File variable.

The STUDENT file object is written into the WS-STUDENT object until an END of file is set.

IDENTIFICATION DIVISION.
PROGRAM-ID. HELLO.

ENVIRONMENT DIVISION.
   INPUT-OUTPUT SECTION.
      FILE-CONTROL.
      SELECT STUDENT ASSIGN TO 'input.txt'
      ORGANIZATION IS LINE SEQUENTIAL.            

DATA DIVISION.
   FILE SECTION.
   FD STUDENT.
   01 STUDENT-FILE.
      05 STUDENT-ID PIC 9(5).
      05 NAME PIC A(25).

   WORKING-STORAGE SECTION.
   01 WS-STUDENT.
      05 WS-STUDENT-ID PIC 9(5).
      05 WS-NAME PIC A(25).
   01 WS-EOF PIC A(1). 

PROCEDURE DIVISION.
   OPEN INPUT STUDENT.
      PERFORM UNTIL WS-EOF='Y'
         READ STUDENT INTO WS-STUDENT
            AT END MOVE 'Y' TO WS-EOF
            NOT AT END DISPLAY WS-STUDENT
         END-READ
      END-PERFORM.
   CLOSE STUDENT.
STOP RUN.

File I/O – Append (Write)

Using the input.txt file, the code below (cwrite.cbl) will add 2 more records:

IDENTIFICATION DIVISION.
PROGRAM-ID. HELLO.

ENVIRONMENT DIVISION.
   INPUT-OUTPUT SECTION.
      FILE-CONTROL.
      SELECT STUDENT ASSIGN TO 'input.txt'
      ORGANIZATION IS LINE SEQUENTIAL. 
  
DATA DIVISION.  
    FILE SECTION.   
    FD STUDENT.
    01 STUDENT-FILE.
      05 STUDENT-ID PIC 9(5).
      05 NAME PIC A(25). 
 
  
PROCEDURE DIVISION.  
    DISPLAY 'WRITING TO A SEQUENTIAL FILE..'  
    OPEN EXTEND STUDENT.  
    MOVE '20006' TO STUDENT-ID.  
    MOVE ' Santa Claus' TO NAME.   
    WRITE STUDENT-FILE  
    END-WRITE. 
      
    MOVE '20007' TO STUDENT-ID.  
    MOVE ' Mary Christmas' TO NAME.  
    WRITE STUDENT-FILE  
    END-WRITE. 
      
    CLOSE STUDENT.  
STOP RUN.

To compile, run and check the output:

$ cobc -free -x -o main cwrite.cbl
$ ./main
WRITING TO A SEQUENTIAL FILE..
$ cat input.txt
20003 Albert Walker 
20004 Malika Jones
20005 Bob Cat
20006 Santa Claus
20007 Mary Christmas

Final Comments

After coding in other languages it becomes immediately obvious that Cobol would require many more lines of code than it would in Python.

Learning Cobol does not appear to be overly difficult, however there are a ton of little nuances that will need to be flushed out.

Re-purpose Your Old Home Router

We had some old routers at home that I thought that I’d see if I could re-purpose.

Some of the typical applications that can be done are:

  • A wireless repeater / access point
  • A remote switch
  • A print server for USB printers
  • A network storage device (with a portable USB drive)

A router can also be used as a low end application platform and it can do many projects that a Raspberry Pi might do. In this blog I will be looking at:

  • USB drives
  • Web Cams
  • USB Sensors
  • CGI Web Servers

Comparing a Router to a Raspberry Pi

Before getting started it’s useful to know how a re-purposed router stacks up against a Raspberry Pi.

For our router project we kept things super simple. We used command line apps and scripts rather than Python.

Pro’s of a Router:

  • Price. I got my routers used from $5-$10. A new Pi 3 or 4 is $35-$50.
  • Housing. A router is ready to be mounted
  • Network Ready. If you’re looking at networking functionality you are good to go.
  • Lots of Basic Apps. Most of key features are available.

Con’s of a Router:

  • Extremely Space Limited. Most of the routers have 32MB vs. 8+ GB of a Pi
    • Need to Pick Apps. You probably can’t load Python3, Apache and Samba at the same time.
    • Probably can’t run X-Windows.
  • Ash not Bash. The lighter weight Ash shell is supported not Bash. This may not be an issue.

Open Source Router Firmware

There are a number of open source firmware solutions that can put life back in old routers. OpenWRT and DD-WRT are the most popular packages, but there are others.

The first step is to determine if your old router is supported with one of these packages. It is important to note that many of the older routers only have 4MB of flash and 32MB of RAM, and this may not run or only marginally run OpenWRT or DD-WRT. The recommended requirements are 8+ MB of flash and 32+ MB of RAM.

Another consideration is USB support. A router without USB support can still be used as a web or application server but it will be missing external hardware integration.

For our router project we used OpenWRT.

Getting Started

How the re-purposed router will be used will effect the setup. It is important to ensure the re-purposed router does not effect other routers in your home LAN. Typically you’ll want to disable Router Features such as the DNS and DHCP server capabilities. In the OpenWRT web interface (Luci) software services can be disabled.

I did lots of playing and I bricked my router about a dozen times. Luckily it is usually possible to un-brick a router. I only permanently bricked one 1 router and it was with a bad DD-WRT firmware.

Un-brinking a Router

Often all you need to do is just reset your router and then connect directly to a LAN port to redo your configuration.

If this fails check the OpenWRT blog for any recommendations for your router. There are some excellent custom solutions but they are usually manufacturer specific.

For Netgear see nmrpflash , it’s an almost 100% sure proof un-brinking solution.

30-30-30 Hard Reset Rule

If resetting the router doesn’t work, the next set is the 30-30-30 Rule. This will typically work for all routers:

  • Press reset button for 30 seconds.
  • While holding, unplug router for 30 seconds
  • power on, still holding setup button 30 seconds.
  • Let go of reset button. Then try to reconfigure. (If this doesn’t work try another power down).

Adding Software Features

OpenWRT uses opkg the OpenWRT package manager. Software can be added either through the Luci web interface or manually in an SSH shell.

After a new install the opkg list of available packages will need to be update, this can be done from SSH by:

## update okpg is required to get info on new packages
opkg update

It is possible to use a USB drive to extend the router’s root file system, this would remove issues of running out space when installing software packages.

Add USB Support

Typically routers do not have a lot of available space. Use the df command to check space:

root@OpenWrt:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root                 2560      2560         0 100% /rom
tmpfs                    30060       228     29832   1% /tmp
/dev/mtdblock5           12160       488     11672   4% /overlay
overlayfs:/overlay       12160       488     11672   4% /
tmpfs                      512         0       512   0% /dev

OpenWRT supported software is loaded directly onto the router so some care is required when deciding which packages are to be installed.

Adding a USB drive can greatly help with the management of source files and backups.

The okpg (OpenWRT package manager) is used used to install software packages. To install USB drive support:

## update okpg is required to get info on new packages
opkg update
## get USB packages
opkg install block-mount e2fsprogs kmod-fs-ext4 kmod-usb-storage kmod-usb2 kmod-usb3

A useful command line USB tool is lsusb, however check how much space you have before installing it. To install lsusb enter:

opkg install usbutils

root@OpenWrt:~# lsusb
Bus 001 Device 003: ID 0a5c:bd17 Broadcom Corp. BCM43236 802.11abgn Wireless Adapter
Bus 001 Device 004: ID 413d:2107  
Bus 001 Device 006: ID 1871:0143 Aveo Technology Corp. 
Bus 001 Device 005: ID 058f:6387 Alcor Micro Corp. Flash Drive
Bus 001 Device 002: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

After the USB packages are loaded the router needs to be rebooted.

A new menu item called “Mount Points” under the “System” heading will be added to the Luci web interface. This option allows for easy addition and removal of USB drives.

If you are mounting more that 1 device, it is recommended to use the UUID label, otherwise there is potential that the mount points could be swapped between devices.

USB Web Cam

There are a number of different USB video solutions that are available. Some packa like Motion are full featured but they might require more resources than your router supports.

For light weight USB video support mjpeg streamer is good place to start. It can be installed by:

opkg install kmod-video-uvc mjpg-streamer

The configuration of the video is defined in: /etc/config/mjpg-streamer .

config mjpg-streamer 'core'
        # option enable '1' = on, '0'=off
        option enabled '1'
        option input 'uvc'
        option output 'http'
        option device '/dev/video0'
        option resolution '1280x1024'
        option yuv '0'
        option quality '80'
        option fps '5'
        option led 'auto'
        option www '/www/webcam'
        option port '8080'
        #option listen_ip '192.168.1.1'
        option username 'openwrt'
        option password 'openwrt'

The video service needs to be started:

## to start the service:
/etc/init.d/mjpg-streamer start
## to enable the service to start on boot
/etc/init.d/mjpg-streamer enable

The webcam is available on a router’s IP with the default port of 8080.

USB Sensors

Most sensors are 0-5 Volts (i.e. the typical Arduino sensor) but there are a few USB sensors.

I purchased a USB Thermometer (~$12 from Walmart).

The USB Thermometer acted like a human interface devices (HID)

A script in Ash (a light weight version of Bash) is used to get the temperature, and it required a few software packages:

opkg install kmod-usb-hid coreutils-printf xxd nano

The temper.sh script sends a query message to the HID address and then exacts the temperature from the returned binary data:

#!/bin/ash
#
# temper.sh - get temperature from a USB thermometer 
#
hid="/dev/hidraw1"
exec 5<> $hid
# send out query msg
echo -e '\x00\x01\x80\x33\x01\x00\x00\x00\x00\c' >&5
# get binary response
OUT=$(dd count=1 bs=8 <&5 2>/dev/null | xxd -p)
# characters 5-8 is the temp in hex x1000
HEX4=${OUT:4:4}
DVAL=$(printf '%d' "0x${HEX4}")
awk "BEGIN {print ($DVAL/100)}"

The script can then set to an executable or run thru ash:

root@OpenWrt:/usbstick# ash temper.sh
24.43
root@OpenWrt:/usbstick# chmod +x temper.sh
root@OpenWrt:/usbstick# ./temper.sh
24.43

CGI Web Server

The OpenWRT runs the Luci configuration interface on a light weight uhttp server. This web server can also be used for custom pages.

The uhttp web server will run Lua or CGI pages from the /www/cgi-bin directory.

Below is an Ash CGI example that runs the usb thermometer script and the vmstat command:

#!/bin/ash
#
# test.cgi - show the USB thermometer temperature and CPU stats 
#
echo "Content-type: text/html"
echo ""
echo "<!DOCTYPE html>
<html>
<head><title>Router Points</title>
</head>
<body>
<h1>Router CGI Page</h1><hr>"

echo "Temperature:  $(/littlehd/temper.sh) DegC"

echo "<hr><h3>Router CPU Stats</h3>"
echo "<pre>"
vmstat
echo "</pre>"

Final Comments

I found that re-purposing a router was an interesting project that could progress to a number of interesting side topics such as:

  • USB over IP – use an old router to remotely configure Arduino or ESP32 devices.
  • Remote network sniffer or traffic monitor
  • Print and Network Storage Servers

Bash with MQTT

I’m working on re-purposing an old router. One of my goals is to bring sensor and performance data from the router to a Home Assistant node. The router doesn’t not have a lot of space so I’d prefer to use Bash rather than Python for MQTT communications.

While I was working on the project I wanted to use some simple tools to view the data, unfortunately I wasn’t able to find any information on how to make a good MQTT client in Bash.

This blog documents how I used Bash to show bar charts of MQTT data.

Setup

The first step is to install the Mosquitto command line tools on the OpenWrt router:

# Update the package manager list
opkg update
# Install the MQTT client utiliy
opkg install mosquitto-client-nossl

When I have some time I’d like to look at making some more MQTT Bash scripts that could be used with file input.

The next step is to install the Mosquitto client on my Linux PC and Raspberry Pi:

sudo apt-get install mosquitto-clients

There are a number of MQTT brokers that can be used, both Home Assistant and Node Red have reliable brokers. The Mosquitto broker can also be loaded on a Linux, MacOS and Windows node.

Publish MQTT Data in Bash

The Mosquitto client can be used to both publish and subscribe to MQTT data.

For my router project, I wanted the: temperature (from a USB thermometer), idle time, % used space and available space. Below is the script that passed the data to the mosquitto_pub (publishing) tool:

#!/usr/ash
#
# mqtt_data.sh - send data to MQTT broker
#
# Get Data values
temp=$(/usbstick/temper.sh)
idle=$(vmstat | awk '{ if (NR==3) print $15}')
used=$(df | awk '{if (NR==4) printf "%.f\n", $5 }')
space=$(df | awk '{if (NR==4) printf "%.1f\n", $4/1000 }')

echo "$temp $idle $used $space"

# Publish Data
server="192.168.0.111"
pause=2

mosquitto_pub -h $server -t rtr_temp -m $temp
sleep $pause
mosquitto_pub -h $server -t rtr_idle -m $idle
sleep $pause
mosquitto_pub -h $server -t rtr_used -m $used
sleep $pause
mosquitto_pub -h $server -t rtr_space -m $space

I added a pause between each publish to let me watch the actions.

The cron utility can be used to schedule the running of the script. Once a minute is the fastest time available with cron, so if faster times are needed the script could cycle with a while loop.

Read/Subscribe to MQTT Data

The mosquitto_sub client tool can be used to read or subscribe to the data.

To look at the router points:

$ service="192.168.0.111"
$ mosquitto_sub -h $server -v -t rtr_idle -t rtr_temp -t rtr_used -t rtr_space

rtr_temp 26.25
rtr_idle 100
rtr_used 66
rtr_space 1.1

The -v (–verbose) option show output with the topic names and the values. Topics, -t option, can be put on the command line or passing in as a file.

For single point monitoring the Zenity utility can be used. This utility is typically preloaded on most Linux and Rasp Pi systems. A script to create a progress bar dialog with MQTT data is:

# Send MQTT values to a progress bar
( 
  while :; do
  msg=$(mosquitto_sub -h 192.168.0.111 -C 1 -t rtr_temp) 
  echo $msg
  echo "#$msg  Deg C"
  done
  ) | zenity --progress  --title="Router External Temperature"

Multipoint Progress Dialogs

The Zenity utility can only manage single point progress bars. For multiple point progress bars the YAD (Yet Another Dialog) package can be used.

To install YAD on Raspberry Pi’s and Ubuntu: sudo apt-get install yad

Below is some script that will show multiple points in a YAD dialog:

#!/usr/bash
#
# mqtt_bars.sh - Show multiple MQTT Topics on a Dialog
#
server="192.168.0.111"
topics=("rtr_idle" "rtr_temp" "rtr_used" )
scale=(100 40 100 )
units=( '%' 'degC' '%'  )
title="Router MQTT Points"

#Build topic and yad strings
yadstr=" "
for i in ${topics[@]}; do
  topstr="$topstr -t $i"
  yadstr="$yadstr --bar=$i"
done

echo "Press [CTRL+C] to stop..."
# Cycle thru 1 message at a time to YAD 
(
while : 
do 
  msg=$(mosquitto_sub -h $server -v -C 1 $topstr) 
  IFS=' ' read -a data <<< "$msg" 
  # match returned msg to order of bars, write value/label
  for i in "${!topics[@]} "
  do 
    if [ "${data[0]}" = "${topics[i]}" ]
    then 
      let j=i+1 ; # YAD indices start at 1
      # Rescale bar to defined scale
      barsize=$(bc <<< "${data[1]}*100/${scale[i]}")
      #barsize=100
      echo "$j:$barsize"
      echo  "$j:#${data[1]} ${units[i]}"
  fi
  done
done 
)  | yad --multi-progress $yadstr --title $title

Final Comments

There are probably lots of good 3rd party tools like Gnuplot that could be connected to mosquitto_sub to show real charts.

Bash Bar Charts for Text and Web Pages

This blog documents my notes on creating Bash text bar charts. The key topics are:

  • Create a simple horizontal bars (3 lines)
  • Create dynamic bars (~25 lines for an array of data values)
  • Add support for piped and command line data
  • Include options for: help, resizing, titles
  • Add colour and HTML support

A Horizontal Bar

To create a static bar an printf statements can be used. The seq {0..10} can be used to repeat an ASCII █ fill character 10 times.

$ printf 'LABEL: ' ; \
  printf '█%.0s' {1..10} ; \
  printf ' 10\n'

LABEL: ████████████████████ 10

Unfortunately the printf statement has some limitations on variable substitution. A simple workaround is to create a string and then eval it:

$ label="temp"; val=20;
$ bar="printf '█%.0s' {1..$val}" ; 
$ printf '\n%-5s ' $label; eval $bar ; printf ' %d\n' $val

temp  ████████████████████ 20

Coloured Bars

The tput setaf command can change the foreground, and tput setab is used for background colours. Colour codes are:

tput setab [1-7] # Set the background colour using ANSI escape
tput setaf [1-7] # Set the foreground colour using ANSI escape

Num  Colour    #define         R G B

0    black     COLOR_BLACK     0,0,0
1    red       COLOR_RED       1,0,0
2    green     COLOR_GREEN     0,1,0
3    yellow    COLOR_YELLOW    1,1,0
4    blue      COLOR_BLUE      0,0,1
5    magenta   COLOR_MAGENTA   1,0,1
6    cyan      COLOR_CYAN      0,1,1
7    white     COLOR_WHITE     1,1,1

To reset colours back to the defaults use: tput sgr0

An example to print a red bar and a stack of bars:

$ printf '\nLABEL: ' ; \
   tput setaf 1 ;\
   printf '█%.0s' {1..10} ; \
   printf ' 10\n'

LABEL: ██████████ 10
$ printf '\n 3 Stacked Bars: ' ; \
   tput setaf 1 ;\
   printf '█%.0s' {1..10} ; \
   tput setaf 2 ;\
   printf '█%.0s' {1..8} ; \
   tput setaf 4 ;\
   printf '█%.0s' {1..3} ; \
   printf ' 10+8+3=21\n'

 3 Stacked Bars: █████████████████████ 10+8+3=21

Dynamic Bars

The next step is to create a script that dynamically updates the bars. The tput clear command will clear the terminal screen keep the data and bars in the same location. The script below will dynamically show the CPU temperature, idle time and 2 random values with a 10 second update time.

#!/bin/bash
# 
# cpu_bars.sh - Show new data every 10 seconds
#
while :; do
    # Get data values
    CPUtemp=$(sensors | grep CPU | awk '{print substr($2,2,4)}')
    CPUidle=$(iostat | awk '{if (NR==4) print $6}')
    Random1=$((1+ $RANDOM % 100))
    Random2=$((1+ $RANDOM % 100))

    labels=( CPUtemp CPUidle Random1 Random2)
    values=( $CPUtemp $CPUidle $Random1 $Random2)
    units=( degC % psi mm)

    # Show a title
    tput clear
    printf " %10s " "" 
    tput setaf 7; tput smul;
    printf "%s\n\n" "Show CPU Data ($(date +%T'))"
    tput rmul;

    # cycle thru data and show a label, 
    for index in "${!labels[@]}"
    do
          tput setaf $(expr $index + 1); # don't use 0 (black) 
          printf " %10s " "${labels[index]}"
          eval "printf '█%.0s' {1..${values[index]%.*}}"
          printf " %s %s\n\n" ${values[index]} ${units[index]}
    done
    sleep 10
done

This script is run by: bash cpu_bars.sh .Typical output is below.

Bars in HTML

The ANSI colours are not supported in HTML, so instead HTML tags and style properties can be used.

It is important to use <pre> tags for Bash text output. Code to create two static bars in HTML would be:

$ (printf "<h1>A Bar from Bash</h1>\n" 
 printf "<pre><span style='font-size:24px;color:red'}>\n"
 printf 'LABEL1: ' ; printf '█%.0s' {1..10} ; printf ' 10\n'
 printf "</pre></span>\n") > bar1.htm

$ cat bar1.htm
<h1>A Bar from Bash</h1>
<pre><span style='font-size:24px;color:red'}>
LABEL1: ██████████ 10
</pre></span>

The script cpu_webbars.sh creates HTML output for an array of calculated values:

#!/bin/bash
# 
# cpu_webbars.sh - Show bars in HTML
#

# Get data values
CPUtemp=$(sensors | grep CPU | awk '{print substr($2,2,4)}')
CPUidle=$(iostat | awk '{if (NR==4) print $6}')
Random1=$((1+ $RANDOM % 100))
Random2=$((1+ $RANDOM % 100))

labels=( CPUtemp CPUidle Random1 Random2)
values=( $CPUtemp $CPUidle $Random1 $Random2)
units=( degC % psi mm)
colors=(red blue green magenta)

# Show a title
printf "<h1><center>Show CPU Data ($(date '+%T'))</center></h1>\n"
# cycle thru data and show a label, 
for index in "${!labels[@]}"
  do
  printf "<pre><span style='font-size:18px;color: ${colors[index]} '}>\n"
  printf " %10s " "${labels[index]}"
  eval "printf '█%.0s' {1..${values[index]%.*}}"
  printf " %s %s\n\n" ${values[index]} ${units[index]}
  printf "</pre></span>\n"
done

This script can be run and outputted to an file: bash cpu_webbars.sh > test.htm

Once the basic HTML output is working its possible to add headers and footers to make a more complete page:

header.htm > test.htm ; \
cat cpu_webbars.sh >> test.htm ; \
cat footer >> test.htm 

Piping Data to Bars

A script (hbar0.sh) will read the piped data and then create an array (data) of labels and values. The data array is cycled through and labels and bars are shown with a randomized colour:

#!/bin/bash
# hbar0.sh - Read in piped data  and plot bars
#     format: label,value;label2,value2;  and plot bars
#
input=$(< /dev/stdin) ; # read piped data
# remove new lines in files, and check for ";" after data pair 
input=$(echo $input | tr -d '\n')
IFS=';' read -r -a data <<< $input
printf "\n" 
for element in "${data[@]}"pete@lubuntu:~/Writing/Blog/text_bars
do 
  # make at array of each data element
  IFS=',' read -r -a datapt <<< $element
  # add a random color
  tput setaf $((1+ $RANDOM % 7))
  # print the label, bar and value
  printf " %10s " "${datapt##*[0]}"
  bar="printf '█%.0s' {1..${datapt[1]}}"
  eval $bar
  printf " %s\n\n" ${datapt[1]} 
  tput rmso ; # exit color mode   
done

The script can be tested with piped data:

$ echo "temp,33;pressure,44" | bash hbar0.sh

 temp       █████████████████████████████████ 33 

 pressure   ████████████████████████████████████████████ 44 

A data file can also be passed in using the cat command:

$ cat data0.txt 
temp,44;
humidity,33;
pressure,15;
wind spd,33;
wave ht,3;
$ cat data0.txt | bash hbar0.sh

       temp ████████████████████████████████████████████ 44

   humidity █████████████████████████████████ 33

   pressure ███████████████ 15

   wind spd █████████████████████████████████ 33

    wave ht ███ 3

Removing ANSI Colors

Terminal applications use ANSI color codes which unfortunately is not support on Web pages.

ANSI color codes can be removed from files and strings by:

# Strip out ANSI color codes:
cat infile | sed 's/\x1b\[[0-9;]*m//g' > outfile
$ echo "temp,33;pressure,44" | bash hbar0.sh > hbar0.txt
$ cat hbar0.txt

       temp █████████████████████████████████ 33

   pressure ████████████████████████████████████████████ 44

$ cat hbar0.txt | sed 's/\x1b\[[0-9;]*m//g'

       temp █████████████████████████████████ 33

   pressure ████████████████████████████████████████████ 44

A Final App

With the basics in place I was able to create an app that would support scaling, titles, units, custom colour and web output:

$ ./hbars
usage: hbars [data] [option] 
  -h --help     print this usage and exit
  -c --color    set color to all bars (default 7=white)
                 (0-grey,1-red,2=green,3=yellow,4=blue,5=magenta,6=cyan,7=white)
  -p --pretty   add different colors to bars (-c overrides)
  -t --title    top title
  -w --width    max width of bar (default 50)
  -W --Web      make output HTML formatted
  -f --fontsize fontsize for web output (default 24)

 examples:
   echo 'temp,33,C;pressure,14,psi' | ./hbars -t Weather -p -w 40 
   ./hbars -t Weather -p -w 40  'temp,33;pressure,14' 
   cat data.txt | ./hbars -W -f 24 -t 'Raspi Data' > data.htm

The code:

#!/bin/bash
#
# hbars.sh - show some text bars
#   pass data as:  label1,value1,unit1,color1; label2,value2,unit2,colour2; ....  
#
width=50
title=""
color=7
pretty=False
web=False
font=24

usage() { 
  echo "usage: hbars [data] [option] "
  echo "  -h --help     print this usage and exit"
  echo "  -c --color    set color to all bars (default 7=white)"
  echo "                 (0-grey,1-red,2=green,3=yellow,4=blue,5=magenta,6=cyan,7=white)"
  echo "  -p --pretty   add different colors to bars (-c overrides)"
  echo "  -t --title    top title"
  echo "  -w --width    max width of bar (default 50)"
  echo "  -W --Web      make output HTML formatted"
  echo "  -f --fontsize fontsize for web output (default 24)"
  echo ""
  echo " examples:"
  echo "   echo 'temp,33,C;pressure,14,psi' | ./hbars -t Weather -p -w 40 "
  echo "   ./hbars -t Weather -p -w 40  'temp,33;pressure,14' "
  echo "   cat data.txt | ./hbars -W -f 24 -t 'Raspi Data' > data.htm"
  echo ""

  exit 0
}
# Show help usage if no pipe data and no cmd line data
if [ -t 0 ]  && [ $# -eq 0 ] ; then
  usage
fi
# Check for command line options
while getopts "hpc:t:w:Wf:" arg; do
  case "$arg" in
    h) usage ;;
    c)  color=$OPTARG ;;
    p)  pretty=True; icolor=0 ;;
    t)  title=$OPTARG ;;
    w)  width=$OPTARG ;;
    W)  web=True;;
    f)  font=$OPTARG ;;
  esac
done
#------------------------------------------------
# Setup formatting for text, Web and color
# -----------------------------------------------
if [[ ${color} != 7 && ${pretty} = True ]]; then
  pretty=False
fi
colidx=0

setcolors() {
if [ $web = True ]; then
  colors=(gray red green yellow blue magenta cyan white)
  titlebold="echo '<h1>'"
  titlereset="echo '</h1>'"
  #color_set='echo "<span style=font-size:$(font)px  >" ' 
  #color_set="printf '<span  style=\"font-size:$(font)px;color:${colors[colidx]}\" >'" 
  color_set="printf '<pre><span style=\"color:${colors[colidx]} ; font-size:${font}px \" >'" 
  color_rs="echo '</span></pre>'"
else
  colors=(0 1 2 3 4 5 6 7 )
  titlebold="tput bold; tput smul"
  titlereset="tput rmul; tput rmso"
  color_set="tput setaf ${colors[colidx]}"
  color_rs="tput rmso"
fi
}
setcolors
#----------------------------------------------
# Get data, check if stdlin, file, if not assume string
#----------------------------------------------
if [ -p /dev/stdin ]; then
        lastarg=$(< /dev/stdin)
else
	lastarg=$(echo "${@: -1}")
	if test -f "$lastarg"; then
	  lastarg=$(<$lastarg)
	fi
fi
# Cleanup the input data
lastarg=$(echo $lastarg | sed 's/\n/;/g; s/  / /g; s/\t//g; s/;;/;/g')
IFS=';' read -r -a array <<< $lastarg

# ensure that there is some data
if [[ ${array} == 0 ]]; then
  echo "No data found"
  exit 0
fi
echo "input:$lastarg"
#exit 0
#------------------------------------
# Get max value and max label length
#------------------------------------
maxval=0
maxlbl=10
#echo "array:${array[@]}"
for element in "${array[@]}"
do 
  IFS=',' read -r -a datapt <<< $element
  if (( $(echo "$maxval < ${datapt[1]}" |bc -l) )); then
	maxval=${datapt[1]}
  fi
  if (( $(echo "$maxlbl < ${#datapt[0]}" |bc -l) )); then
	maxlbl=${#datapt[0]}
  fi
done
#---------------------------------
# Print Title - use bold/underline
#---------------------------------
if [[ ! -z $title ]]; then
  printf "\n %${maxlbl}s " " "
  eval $titlebold
  printf "%s" "${title}" ; printf "\n\n"
  eval $titlereset
fi
#------------------------------------
# Cycle thru data and build bar chart
#------------------------------------
for element in "${array[@]}"
do
# check data values
  IFS=',' read -r -a datapt <<< $element
  # check for empty records
  if [ ${#datapt[0]} = 0 ]; then 
    break
  fi
  label=${datapt[0]}
  if [[ ${label} != "-t*" ]]; then 
	  val=${datapt[1]}
	  sval=$(bc <<< "$width * $val / $maxval")

	  # add color, use 4th item if available
	  if [[ ${#datapt[@]} > 3 && $pretty = False ]]; then
		icolor=${datapt[3]}
	  fi
	  if [[ $pretty = True ]] ; then
		let colidx++
		if [ $colidx -gt 7 ]; then
		  let colidx=1
		fi
	  elif [[ ${#datapt[@]} > 3 ]]; then
		colidx=${datapt[3]}
	  else
	  	colidx=$color
	  fi
	  setcolors
          eval $color_set
          printf " %${maxlbl}s " "$label"
	  bar="printf '█%.0s' {1..$sval}"
	  eval $bar; 
	  # add value and units if available
	  units=""
	  if [[ ${#datapt[@]} > 2 ]]; then
		units=${datapt[2]}
	  fi
	  printf " %d %s\n\n" $val "$units"
	  eval $color_rs 
  fi
done

SQL Output to Bars

With the base code I was able to start doing some more complicated actions, like piping SQL SELECT output. Below is an example from Sqlite3. The first step is to format the SQL output to: label,value;

pete@lubuntu:$ sqlite3 $HOME/dbs/someuser.db "select fname,age from users limit 4"
Brooke|18
Leah|18
Pete|100
Fred|77
pete@lubuntu:$ sqlite3 $HOME/dbs/someuser.db "select fname,age from users limit 4" \
>  | tr  '|' ',' | tr '\n' ';'
Brooke,18;Leah,18;Pete,100;Fred,77;
pete@lubuntu:$ sqlite3 $HOME/dbs/someuser.db "select fname,age from users limit 4" \ > | tr '|' ',' | tr '\n' ';' | ./hbars -t "Sqlite3 Users" -p Sqlite3 Users Brooke █████████ 18 Leah █████████ 18 Pete ██████████████████████████████████████████████████ 100 Fred ██████████████████████████████████████ 77

Web Page with Bars

Support was added for fixed chart width, engineering units and custom line colours. HTML <center> tags were used on the title.

$ cat data.txt
temp,44,degC,1;
humidity,33,%,4;
air pressure,88,mm Hg,5;
rain/precipation,6,mm,6;

$ cat data.txt | ./hbars -W -f 24 -w 50 -t '<center>Raspi Data</center>' > data.htm

Final Comments

Horizontal bars are relatives easy to create in Bash. Unfortunately showing vertical bars and Line charts will require a different approach

TEMPer USB Temperature Sensor

I picked up a TEMPer USB temperature sensor from Walmart for about $12. My goal was to use it with my Home Assistant home automation system.

The model that I picked up supports Windows and it can be used directly in Excel. There is also integration with Home Assistant.

I found that the integration for Linux was tricky. This blog looks at how I got things working with a Raspberry Pi.

USB Connections

The lsusb command can be used to see which USB devices are connected:

pi@pi4:~ $ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 007: ID 413d:2107  
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

It is important to identify the ID model. The TEMPer USB model that I purchased was 413d:2107.

When I searched the Internet I found that people had created different packages but they were based on specific models. (Check your model).

temper.py

There is a Python library file that can be used on the TEMPer family of sensors. This library component is designed to be run directly as a Python file. It can be accessed by:

pip install temper-py
# Or get the file 
wget  https://raw.githubusercontent.com/ccwienk/temper/master/temper.py

Using the file directly will show the TEMPer device information (Note: use sudo):

$ sudo python3 temper.py
Bus 001 Dev 007 413d:2107 TEMPerGold_V3.1 29.31C 84.76F - - - -

$ sudo python3 temper.py --v
Firmware query: b'0186ff0100000000'
Firmware value: b'54454d506572476f6c645f56332e3120' TEMPerGold_V3.1 
Data value: b'80800b6d4e200000'
Bus 001 Dev 007 413d:2107 TEMPerGold_V3.1 29.25C 84.65F - - - -

$ sudo python3 temper.py --json
[
    {
        "vendorid": 16701,
        "productid": 8455,
        "manufacturer": "",
        "product": "",
        "busnum": 1,
        "devnum": 7,
        "devices": [
            "hidraw0",
            "hidraw1"
        ],
        "port": "1-1.4",
        "firmware": "TEMPerGold_V3.1",
        "hex_firmware": "54454d506572476f6c645f56332e3120",
        "hex_data": "80800b6d4e200000",
        "internal temperature": 29.25
    }
]

I found that the Python app was useful and it could be used on a variety of sensors.

Human Interface Devices

The TEMPer sensor acts like a Human Interface Device. When the TEMPer sensor is connected the dmesg utility can be used to show kernel information:

 $ dmesg -H
[May 1 21:22] usb 1-1.4: USB disconnect, device number 6
[May 1 21:23] usb 1-1.4: new full-speed USB device number 7 using xhci_hcd
[  +0.135008] usb 1-1.4: New USB device found, idVendor=413d, idProduct=2107, bcdDevice= 0.00
[  +0.000020] usb 1-1.4: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[  +0.011366] input: HID 413d:2107 as /devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:
[  +0.064445] hid-generic 0003:413D:2107.0009: input,hidraw0: USB HID v1.11 Keyboard [HID 413
[  +0.007181] hid-generic 0003:413D:2107.000A: hiddev96,hidraw1: USB HID v1.10 Device [HID 41
(END)

From this output I can see that TEMPer sensor is on the raw human interface device of hidraw1. (For this example the hidraw0 acts like a keyboard but it can’t be used to get temperature info). When I move the TEMPer sensor to a different PC I need to run dmesg again to recheck the hidraw device number, it could be 2,3,4 etc.

A Bash Script

There are some excellent links that got me about 95% of the way. The Bash script below will return a temperature value for my series of TEMPer USB thermometers.

#!/bin/bash
# 
# Get the temperature from a USB Temper Thermometer
#   
#   find the HID device from the kernel msg via dmesg
#   parse the line get HID device
hidstr=$(dmesg | grep -E  -m 1 'Device.*413d:2107')
# find the postion of the "hidraw" string
hidpos=$(echo $hidstr | grep -b -o hidraw  | awk 'BEGIN {FS=":"}{print $1}')

if [ $hidpos  = "" ]; then
  echo "No TEMPer device found"
else
  #get the hidraw device from the string, Note: offset of 1
  hidpos1=$((hidpos +1))
  hid=$(echo "/dev/${hidstr:hidpos1:7}")
  exec 5<> $hid
  # send out query msg
  echo -e '\x00\x01\x80\x33\x01\x00\x00\x00\x00\c' >&5
  # get binary response
  OUT=$(dd count=1 bs=8 <&5 2>/dev/null | xxd -p)
  # characters 5-8 is the temp in hex x1000
  HEX4=${OUT:4:4}
  DVAL=$((16#$HEX4))
  CTEMP=$(bc <<< "scale=2; $DVAL/100")
  echo $CTEMP 
fi

If you are always using the TEMPer in only one place you could simplify the script by hard coding the HID number. I wanted to move the sensor between devices so I added a check find the device.

Control USB Powered Devices

For home automation projects a Raspberry Pi offers a simple low cost approach for controlling a wide variety of devices. Typically these devices are either digitally wiring 0-5Volt devices like motion detectors, or wireless Ethernet devices like smart plugs, but a Raspberry Pi can also control USB powered devices, like USB fan and lights.

In this blog I will look at how to control and monitor Raspberry Pi USB port with two projects. The first project will use Node-Red to create a web dashboard to control USB lights. The second project will turn on USB cooling fans based on the PI’s CPU temperature.

Controlling USB Ports

There are a number of techniques to control USB ports, I found that one of easiest approach is to use the uhubctl utility. To load this utility:

sudo apt-get install libusb-1.0-0-dev
git clone https://github.com/mvp/uhubctl
cd uhubctl
make
sudo make install

The uhubctl utility can be used to view and control USB ports and ports smart USB hubs.

The lsusb command can be used to show connected USB devices

pi@raspberrypi:~ $ lsusb 
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp. Fast Ethernet Adapter
Bus 001 Device 002: ID 0424:9514 Standard Microsystems Corp. SMC9514 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

For the Raspberry Pi 3 and 4 the power on all USB ports is ganged together through port 2, so unfortunately it is not possible to power up/down an individual USB port.

The Pi 4 has two internal USB hubs. Hub 1-1 connects to all the USB port on the USB 2.10 standard. Hub 2 controls to all the ports on the USB 3.00 standard and the Ethernet jack.

The commands to turn on/off/toggle the USB ports and keep the Ethernet jack powered are:

sudo uhubctl -l 1-1 -p 2 -a on
sudo uhubctl -l 1-1 -p 2 -a off
sudo uhubctl -l 1-1 -p 2 -a toggle

The command will return the messages showing the current status, the power requested state and then the new status. To just do an action and remove feedback messages add a 1>&- at the end of the line.

Monitoring USB Power

The uhubctl utility can be used to check if the USB ports are powered. For Raspberry Pi’s the status of Port 2 is the power status for all the ports.

Using some Bash statements the power status can be parsed to just show off or power message. In the next example this Bash statement will be used to show the power status on a Node Red dashboard.

$ sudo uhubctl | grep 'Port 2' | awk '{print $4}'
off

Node Red USB Control Dashboard

Node Red is a visual programming tool that is included with the full desktop install of Raspberry Pi. If Node Red has not been installed see: https://nodered.org/docs/getting-started/raspberrypi

There are number of low cost USB lighting options that can be used with a Raspberry Pi, these include LED strips, eWire and small USB lights. Below is an example of littleBits eWire bit and a USB LED light.

A simple Node Red dashboard can be created that can: 1) turn Raspberry Pi USB ports on and off, and 2) checks the status of power on these ports. The logic would include: 2 dashboard buttons, one dashboard text and 2 exec nodes. The uhubctl utility can be used directly in the exec node.

The first exec node contains the Bash command to turn the USB ports on or off. The “on” or “off” string is sent from the dashboard buttons as a msg.payload this is appended to the command in the exec node. The output from the first exec node triggers the second exec node to get the latest USB port status.

The dashboard text node is configured to show the power status text as a large uppercase heading.

Once the logic is complete, the Deploy button on the right side of the menu bar will make the dashboard available to web clients at: https://raspberry_pi_address:1880/ui .

For my example I added an enhancement to include a countdown or sleep timer.

Raspberry Pi Cooling Fan

Raspberry Pi’s have a number of different cooling options. For this example I used a two littleBits fans that I placed on a littleBits mounting plate.

The vcgencmd measure_temp command will return the Pi CPU temperature. By adding some awk script it’s possible to parse out just the float value:

$ vcgencmd measure_temp 
temp=46.2'C
$ vcgencmd measure_temp | awk '{ print substr($1,6,4)}'
45.6

The bc (arbitrary precision calculator) command can be used with the math library (-l option) to check if one float is greater than another float number:

$ echo "33.4 > 36.1" | bc -l
0
$ echo "38.4 > 36.1" | bc -l
1

A simple script to loop through and check the temperature against a limit, and then turn on/off a fan would be:

#!/bin/bash
#
# Check the Pi temperature and turn on fan if too high
# 
tlim="46.0"
while :;
do
  tnow=$(vcgencmd measure_temp | awk '{ print substr($1,6,4)}')
  # check the CPU temp vs. the limit
  if (( $(echo "$tnow > $tlim" | bc -l ) )) ; then
     # CPU temp is above limit, turn on fan
     sudo uhubctl -l 1-1 -p 2 -a on 1>&-
  else
     # CPU temp is below limit, turn off fan
     sudo uhubctl -l 1-1 -p 2 -a off 1>&-
  fi
  sleep 10
done

This code can be enhanced to show when the fan needs to be turned on and off:

#!/bin/bash
#
# Check the Pi temperature and turn on fan if too high
#

# Start with Fan on
tlim="46.0"
fan="on"
sudo uhubctl -l 1-1 -p 2 -a $fan 1>&-

# Periodically check CPU temperature
while :;
do
  tnow=$(vcgencmd measure_temp | awk '{ print substr($1,6,4)}')

  if (( $(echo "$tnow > $tlim" | bc -l ) )) ; then
      if [ $fan = "off" ] ; then
    	fan="on"
	    echo "$(date +'%T')  Temp: $tnow - Turn fan $fan"
        sudo uhubctl -l 1-1 -p 2 -a $fan 1>&-
      fi
  else
      if [ $fan = "on" ] ; then
    	fan="off"
	    echo "$(date +'%T')  Temp: $tnow - Turn fan $fan"
        sudo uhubctl -l 1-1 -p 2 -a $fan 1>&-
      fi
  fi
  sleep 10
done

Power Control on other Controllers

A Raspberry Pi can control the power on other controllers, below is an example of a Pi 4 powering an Arduino UNO, an Arduino Nano (clone) and a BBC Micro:bit controller.

For external modules that don’t support Wifi or real time clocks a Raspberry Pi could be used as an easy way to power up/power down these external controllers.

It’s important to realize that a Raspberry Pi is not designed to power devices that have a high power requirement.

The Raspberry Pi 3 and 4 can have a maximum USB port output of 1200mA for all 4 ports combined, with no per-port limits (meaning, all 1200mA is available on a single port if no others are in use). This 1200mA limit assumes that the power is from a 2.5A power supply for the Pi3 or the 3A power supply for the Pi4.

If you connecting smart USB devices like devices like memory sticks or 3rd party controller, the device manufacture has a defined MaxPower rating that can be found once the device is connected.

The command lsusb -v will give a very long list of the full vendor information for all the connected devices. To get just the MaxPower for each device on the Raspberry Pi USB internal bus enter:

sudo lsusb -v  2>&- | grep -E  'Bus 00|MaxPower'

When this command is run with an Arduino Nano, Arduino UNO and a BBC Micro:bit and a memory stick, the total power requirements can be seen.

pi@pi4:~ $ sudo lsusb -v  2>&- | grep -E  'Bus 00|MaxPower'
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
    MaxPower                0mA
Bus 001 Device 004: ID 1a86:7523 QinHeng Electronics HL-340 USB-Serial adapter
    MaxPower               96mA
Bus 001 Device 003: ID 2341:0043 Arduino SA Uno R3 (CDC ACM)
    MaxPower              100mA
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
    MaxPower              100mA
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
    MaxPower                0mA

With this example the total used USB power is 296mA (0 +96+100+100+0), and this is within the Raspberry Pi specs.

A Bash command to calculate the total required bus power would be:

pi@pi4:~ $ (sudo lsusb -v 2>&- | grep MaxPower | grep -o -E '[0-9]+' ) | awk '{ sum += $1} END {print "\nTotal= " sum " mA"}' 

Total= 296 mA

Unfortunately devices like USB lights use the USB connection for power only so they do not appear in lsusb, so you’ll need to check the manufacturers literature for power requirements.

Final Comments

I prefer using direct wired GPIO pin connections or Wifi devices over using USB powered devices, however it’s nice to know that you have the USB option if you need it.

For kids projects that use littleBits or Micro:bits using a Pi with Node Red offers a nice way to remotely control them.

Automate Web Page Logins

To ensure a safe number of users during Covid we had to log into Web sites for access to activities like pools, gyms and ski hills. These precautions made sense, however the booking process was awkward and was easy to miss an activity if you weren’t signed up early enough.

Luckily there are some great Linux tools that can be used to automate web logins. In this blog I’ll look at two techniques:

  • Keyboard simulation using xte and xdotool
  • Python accessing the HTML on page using Selenium

Linux Keyboard Simulation

If you are working in MS-Windows take a look at SendKeys command.

In the Linux environment there a number of choices, such as xautomation and xdotool.

The xdotools package is feature rich with special functions for desktop and windows functions. The xautomation package was a little simpler and it focuses on keyboard and mouse simulation.

I found that it was very useful to install wmctrl, this allows you to easily find out which windows are running and it can set the active window with a substring (you don’t need the full window name).

To install xautomation and wmctrl in Ubuntu:

sudo apt-get install xautomation wmctrl

To install xdotool in Ubuntu:

sudo apt-get install xdotool

Log in Example with xte

To create an automation script you need to do the required steps once manually and document what is a Tab, an entry field or a Return key. Some trial and error work will probably be required for timing if you are working between pages links or with pages that have a slow call up time.

A good simple example is to try and log into Netflix.

The following Bash script will : 1) open a Chrome browser page, 2) set the focus to the page and 3) sent the correct tab, text and a return key.

#!/bin/bash
# netflix_login.sh - script logs into Netflix
#

url="https://www.netflix.com/ca/login"
email="my_email.com"
pwd="my_password"

chromium-browser $url & #open browser to

# wait for the page to open, the set focus to it
sleep 2
wmctrl -a "Netflix - Chromium"
sleep 1 # allow time to get focus before sending keys
xte "key Tab"
xte "key Tab"
xte "str $email"
xte "key Tab"
xte "str $pwd"
xte "key Tab"
xte "key Tab"
xte "key Return"

echo "Netflix Login Done..."

Use wmctrl -l to see which windows are open.

A specific window can have the focus based on a substring, with the -a options

The xte command uses a string to pass key, text and mouse actions to the active window.

Park Booking using xdotool

The xdotool syntax to send keystrokes and text is very similar to xte, but with a few extra features.

The park booking example is a little bit more complex because a booking time needs to be selected from a list. It might be possible to tab to the required time, but a safer way would be to do a search to find the required time.

Neither the xte nor the xdotool utilities support a search text function. A simple workaround is to use the web browser’s search function. To have keystrokes go to the result of the browser search it’s important to enable Caret navigation.

The Caret dialog is shown by entering F7. It’s important to note that the Caret Enable/Cancel or Yes/No buttons will vary between different browsers.

For the park booking page, the animation script needs to manage 8 entry fields. To keep things simple I’ll pass the date in the URL.

The important issue is to book a time slot, for this Control-F will call up the Search Dialog. After the required time is entered, the navigation moves to the selected time. The next step is to close the Search Dialog, this done by sending three tabs and a return key.

The final bash script is shown below. One of the useful features of xdotool is that it can do repeat key strokes with a delay between entries.

#!/bin/bash
# book10am.sh - make a 10:00 park booking
#
sdate="startDate=2021-04-23" #adjust the date
url="https://book.parkpassproject.com/book?inventoryGroup=1554186518&&inventory=1229284276&$sdate"

chromium-browser $url & #open browser to park booking page
sleep 5 # wait for browser to come up
wmctrl -a "Chromium"
sleep 2
# Turn on caret browsing
xdotool key F7
xdotool key Return
sleep 1

# tab to 'Time Slot' area 
tabcnt=8
xdotool key --repeat $tabcnt --delay 100 Tab

xdotool key Return
sleep 1

# Search for 10:00 time and select it
xdotool key ctrl+f 
xdotool type '10:00'
xdotool key Return
# Close find dialog and select time
xdotool key Tab Tab Tab Return Return

echo "Park Time Booking Complete"

Script Limitations with xdotool and xte

Using xdotool or xte is great for simple web page automation where the HTML form items are sequential and no special decision making is required.

Unfortunately I found that when I tried to book a park time on the weekend I started to see some limitation. During busy times if I tried to book by time the xte or xdotool could not determine if the time slot was full.

A simple workaround would be to search for the first ‘Available’ or ‘Not Busy’ slot but this doesn’t allow you to pick times that you like.

For projects that require some logic (like picking a good time from a list of time), Selenium with Python is an excellent fit.

For more complex web automation projects Selenium with Python is an excellent fit.

Installing Selenium with Python

Selenium is a portable framework for testing web applications, with server/client tools and IDE’s.

The Selenium WebDriver component sends commands from client APIs directly to a browser. There are webdriver components for Firefox, Google Chrome, Internet Explorer, Safari, Opera and Edge. Client API’s are available for C#, GO, Java, JavaScript, PhP, Python and Ruby.

For details on extra information on installation of the different webdrivers see: https://www.selenium.dev/downloads/

To install the Linux 32-bit Selenium Driver (geckodriver) for Firefox:

wget https://github.com/mozilla/geckodriver/releases/download/v0.29.1/geckodriver-v0.29.1-linux32.tar.gz
tar -xvzf geckodriver-v0.24.0-linux32.tar.gz
chmod +x geckodriver
sudo mv geckodriver /usr/local/bin

To install the Selenium library for Python:

pip install selenium

The big difference between using xte and selenium is that selenium can directly access the HTML code of the selected web page. The xte approach to accessing form items is to tab to them like you would with a keyboard. Selenium allows code to access an HTML item using its id name.

Selenium Log In Example

Like using xte the user needs to do some manual work before the script is written.

Once the required web page is open, the Web Developer Inspector tool can used to examine HTML code. To access the Inspector, Select Tools > Web Developer > Inspector from the top menu bar, or use the shortcut control-shift-C.

For the Netflix log in example the key HTML items are the “email or phone number” input and the the password input. Using the Inspector these id’s (or names) can be found.

The Python code to log into Netflix is:

#
# netflix_login.py - automate Netflix Login
#
from selenium import webdriver

url="https://www.netflix.com/ca/login"
email="my_email.com"
pwd="my_password"

browser = webdriver.Firefox()

browser.get(url)

# wait for page to refresh
browser.implicitly_wait(10) 

username = browser.find_element_by_id('id_userLoginId')
username.send_keys(email)

password = browser.find_element_by_id('id_password')
password.send_keys(pwd)

password.submit()

print("Login Complete")

When a web page is called it’s important to give the page some time to refresh. The implicity_wait(10) call will wait up to 10 seconds for a Selenium query. HTML Items can be found by either find_element_by_id() or by find_element_by_name(). The send_keys() method is used to pass text strings to <input> tags. Finally calling submit() will send all the form data to the requested action.

Using Selenium Searches

From the earlier park booking example we saw that xte had some limitations when a variable lists of options were presented. Luckily Selenium has a number of functions that can be used for searching HTML tags and text.

The first step is to manually open the web page and inspect the structure.

For this example the Inspector shows that each status entry in the list has a <div class=”jss97″>. with the Available items having a <div class=’jss100′> and the “Not Busy” items being a <div class=’jss100′>. The very top level is <div class=”jss94″>, this has both the times and the status messages.

Now that I know how the park times are defined I can check for the first “Not Busy”entry, or I can check a specific time and see if it’s busy or not.

Below is a code example that shows the first “Not Busy” time. To select a specific time/status entry use the click() method (itimes.click()).

Final Comments

Using simulated keyboard and mouse movements is a nice tool for automating simple web pages.

If you need to look at more complex web automation solutions check out Selenium, it has a Python library that allows you to access the DOM object of a web page.

Orange Pi – $2-$50 Raspberry Pi Competitor

Orange Pi is a low-cost Raspberry Pi competitor. It is developed by the Chinese Shenzhen Xunlong CO Software company. Just like the Raspberry Pi, Orange Pi is an Open Source project.

Orange Pi offers a wide variety of offerings, that start at $2 and go up from there.

I purchased an Orange Pi Lite for about $15, and I’ve been quite happy with it.

Compared to the Raspberry Pi, Orange Pi is:

  • less expensive. Faster for the price
  • has more hardware options and form factors

This blog documents some of my installations steps to get an Orange Pi (Lite) running with Raspberry Pi functionality.

Some Interesting OrangePi Offers

There are quite a few different modules that are available. Some of the ones that I found interesting are:

Orange Pi Zero – there a few model, the 256MB LTS ($2), to the Zero 2 ($30).

Orange Pi 3G-IOT-B – includes a built-in 3G SIM card support ($22). There are also 4G models.

Base Installation

Like the Rasp Pi there are a number of OS options that can be installed. I chose the Armbian OS, which seems to be the most popular.

There are a number of ways to put an image on an SD chip, I like: pi imager. Once an image has been installed, the command line can be used to get the basic things setup and installed

  ___  ____  _   _     _ _       
 / _ \|  _ \(_) | |   (_) |_ ___ 
| | | | |_) | | | |   | | __/ _ \
| |_| |  __/| | | |___| | ||  __/
 \___/|_|   |_| |_____|_|\__\___|
                                 
Welcome to Armbian buster with Linux 5.4.43-sunxi

System load:   0.00 0.00 0.00  	Up time:       21 min		
Memory usage:  41 % of 492MB  	IP:            192.168.0.113
CPU temp:      52°C           	
Usage of /:    16% of 15G    	

Last login: Sat Apr 17 12:28:07 2021 from 192.168.0.111

On Pi the config tool is called raspi-config, on the Ambian OS it’s : armbian-config

This tool is used to setup initial key things like: Wifi, Bluetooth, SSH, Desktop setting etc.

Depending on the OS that has been installed you may have everything you need or you may need to do further installs. For myself I needed to load “Pi type” features such as: the gpio utility, Python GPIO library and Node-Red.

I logged in as root for all my base installations.

GPIO (WiringPi) Utility

The gpio utility is super useful for manual seeing and setting GPIO pins.

To install gpio :

git clone https://github.com/orangepi-xunlong/WiringOP
cd WiringOP
chmod +x ./build
sudo ./build

The pinouts are different that the Rasp Pi, to see them enter: gpio readall

root@orangepilite:~# gpio readall
 +------+-----+----------+------+---+OrangePiH3+---+------+----------+-----+------+
 | GPIO | wPi |   Name   | Mode | V | Physical | V | Mode | Name     | wPi | GPIO |
 +------+-----+----------+------+---+----++----+---+------+----------+-----+------+
 |      |     |     3.3V |      |   |  1 || 2  |   |      | 5V       |     |      |
 |   12 |   0 |    SDA.0 |  OFF | 0 |  3 || 4  |   |      | 5V       |     |      |
 |   11 |   1 |    SCL.0 |  OFF | 0 |  5 || 6  |   |      | GND      |     |      |
 |    6 |   2 |      PA6 |  OFF | 0 |  7 || 8  | 0 | OFF  | TXD.3    | 3   | 13   |
 |      |     |      GND |      |   |  9 || 10 | 0 | OFF  | RXD.3    | 4   | 14   |
 |    1 |   5 |    RXD.2 |  OFF | 0 | 11 || 12 | 0 | OFF  | PD14     | 6   | 110  |
 |    0 |   7 |    TXD.2 |  OFF | 0 | 13 || 14 |   |      | GND      |     |      |
 |    3 |   8 |    CTS.2 |  OFF | 0 | 15 || 16 | 0 | OFF  | PC04     | 9   | 68   |
 |      |     |     3.3V |      |   | 17 || 18 | 0 | OFF  | PC07     | 10  | 71   |
 |   64 |  11 |   MOSI.0 |  OFF | 0 | 19 || 20 |   |      | GND      |     |      |
 |   65 |  12 |   MISO.0 |  OFF | 0 | 21 || 22 | 0 | OFF  | RTS.2    | 13  | 2    |
 |   66 |  14 |   SCLK.0 |  OFF | 0 | 23 || 24 | 0 | OFF  | CE.0     | 15  | 67   |
 |      |     |      GND |      |   | 25 || 26 | 0 | OFF  | PA21     | 16  | 21   |
 |   19 |  17 |    SDA.1 |  OFF | 0 | 27 || 28 | 0 | OFF  | SCL.1    | 18  | 18   |
 |    7 |  19 |     PA07 |  OFF | 0 | 29 || 30 |   |      | GND      |     |      |
 |    8 |  20 |     PA08 |  OFF | 0 | 31 || 32 | 0 | OFF  | RTS.1    | 21  | 200  |
 |    9 |  22 |     PA09 |  OFF | 0 | 33 || 34 |   |      | GND      |     |      |
 |   10 |  23 |     PA10 |  OFF | 0 | 35 || 36 | 0 | OFF  | CTS.1    | 24  | 201  |
 |   20 |  25 |     PA20 |  OFF | 0 | 37 || 38 | 0 | OFF  | TXD.1    | 26  | 198  |
 |      |     |      GND |      |   | 39 || 40 | 0 | OFF  | RXD.1    | 27  | 199  |
 +------+-----+----------+------+---+----++----+---+------+----------+-----+------+
 | GPIO | wPi |   Name   | Mode | V | Physical | V | Mode | Name     | wPi | GPIO |
 +------+-----+----------+------+---+OrangePiH3+---+------+----------+-----+------+

Python GPIO Library

The Raspberry Pi Python GPIO library as been ported to the Orange Pi and it’s called OPi.GPIO. To install it:

# if required install PIP for Python 2 and 3
apt install python-pip python3-pip

# install Orange Pi GPIO library for Python 2 and 3
pip install OPi.GPIO
pip3 install OPi.GPIO

A quick test from to test that things are working:

root@orangepilite:~# python3
Python 3.7.3 (default, Dec 20 2019, 18:57:59) 
[GCC 8.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> import OPi.GPIO as GPIO
>>> GPIO.setmode(GPIO.BOARD)
>>> GPIO.setup(12, GPIO.OUT)
>>> # set pin 12
... 
>>> GPIO.output(12, 1)
>>> GPIO.input(12)
1
>>> GPIO.cleanup()

Node-Red

To install Node-Red, (logged in as root), use the following command, and follow the prompts (don’t install Raspberry support). This install takes about 20 minutes and Node.js will also be added:

bash <(curl -sL https://raw.githubusercontent.com/node-red/linux-installers/master/deb/update-nodejs-and-nodered)

The next step is to install the Orange PI GPIO support:

cd $HOME/.node-red
npm install node-red-contrib-opi-gpio

To manual start Node-Red enter: node-red-start & . To manually stop Node-Red: node-red-stop. See the Node-Red documentation for more info.

A simple circuit to set and read GPIO would be:

Final Comments

A Orange Pi is awesome for generic or simple hardware projects.

If however you’re doing projects where you need tops, for motor controls or relays then I would still to a Raspberry Pi.