OpenPLC on a Raspberry Pi

For home automation projects there are a lot of good software packages like Home Assistant and Node Red that can be used to control and view sensors and devices.

If you are interested in looking at an industrial controls approach to your automation projects then OpenPLC is a good package to consider.

A PLC (Programmable Logic Controller) is an industrial hardened hardware device that manages I/O and logic using the IEC 61131-3 standard.

OpenPLC is open source software that runs on a Raspberry Pi, Linux or Windows PC and it offers users a great way to learn industrial control concepts, programming languages and communications protocols.

In this article I will create three small Raspberry Pi projects using the IEC 61131-3 ladder logic, function blocks and structure text programming languages. Finally I will have these projects pass their data via Modbus TCP to a Node Red dashboard.

Getting Started

The OpenPLC software comes in three packages, a logic editor, the runtime component, and a graphic builder. See https://www.openplcproject.com/getting-started/ for specific instructions for your installation.

For my installation I put the OpenPLC editor on my Ubuntu PC so that I could do remote configuration. I loaded the OpenPLC runtime on a Raspberry PI. The OpenPLC runtime web interface is used to load and monitor logic.

I didn’t install the OpenPLC graphic builder instead I used Node-Red Dashboards as my final user interface

OpenPLC has a good number of optional communications packages and slave I/O components. A typical layout could be as below.

For my application I created a project with three programs; a ladder program, a function block program and a structure text program. The Resource object (Res0) defines global variables that can be used by all programs, and the task cycle times. This is a small project so I put all the programs into the same task execution (task0). For larger project I might put all my digital logic into a fast task execution (20ms) and my analog logic into a slower task execution (250ms).

I setup the Raspberry Pi with a push button on pin 17 and an LED on pin 23.

On the Raspberry Pi GPIO pins are referenced using the IEC 61131-3 addressing. So the pushbutton at BCM pin 17 (physical pin 11) is addressed by %IX0.3 , an input bit on bus 0 at bit 3. The LED at BCM pin 23 (physical pin 16) is addressed by %QX0.2 , as output bit on bus 0 bit 2.

It’s important to note that OpenPLC has allocated all the left side (odd) pins as inputs and all the right side (even) pins as outputs.

Ladder Diagrams (LD)

Ladder logic was the first IEC 61131-3 programming languages, it was developed as a graphic representation for circuit diagrams of relay logic hardware. The term “ladder” comes from the fact that the logic looks a little like a ladder with the left side having a vertical power rail and a a vertical ground rail on the right side, then there are a series of horizontal lines or “rungs” wiring hardware components between the rails.

Most electricians feel very comfortable using Ladder logic and it is a good programming method for managing digital logic. If you come from a programming background Ladder logic may feel a little strange at first, for example to do AND / OR logic to light an LED would be:

For my Ladder program I wanted to light an LED for 3 seconds with a single push of a button. In the OpenPLC editor I referenced an external variable PB1 (it’s defined in Resource object Res0) and I created two local variables, LED2, my output LED and TOF0, an off delay timer.

IEC 61131-3 has a wide range functions that can be used in Ladder rungs. In this example a TOF function was inserted after the push button, and the time parameter is wired in variable.

Function Block Diagrams (FBD)

One of limitations of Ladder logic is that managing analog logic can be a little messy, for this reason Function Block Diagrams (FBD) was developed.

If you feel comfortable using graphic programming applications like Node-Red then you shouldn’t have any problems working in Function Block Diagrams.

For my FBD program I wanted to count the number of times the LED was lite and output the value to a Modbus hold register.

Like in the Ladder program the external PB1 variable is referenced. A new output CNT_FB is defined as an output word on bus 100, %QW100.

The FBD uses a Rising Edge Trigger (R_TRIG) to catch when the LED turns on. The output from R_TRIG is a boolean so the value is converted to an INT and added to the value of CNT_FB.

Structured Text (ST)

One of the advantages of Function Block Diagrams is that it is very readable and somewhat self documenting. The downside of FBD is that it can be messy for complex conditional logic.

Structured Text (ST) was developed as a programming option that can work along with the other 61131-3 languages. Structure Text is block structured and syntactically resembles Pascal.

For my Structured Text program I wanted to do the same functionality that was done in the earlier Function Block Diagram program. To do the same functionality in ST as the FBD programs it only took 3 lines of code vs. 5 Function Blocks.

In my ST program I added a simple IF condition to reset the push button counter if the value reached 1000.

It’s important to note that library functions such as R_TRIG are available in all the 61131-3 programming languages. It is also possible to create your own custom functions in one programming language and they can then we used in all the other languages.

Running OpenPLC Programs

After the three programs have been compiled and saved they can be install into the OpenPLC runtime application. To manually start the runtime application:

pi@pi4:~ $ cd OpenPLC_v3
pi@pi4:~/OpenPLC_v3 $ sudo ./start_openplc.sh &

The OpenPLC runtime will start a Web application on port 8080 on the Raspberry Pi. After logging into the web interface, the first step is to select the “Hardware” option and set the OpenPLC Hardware Layer to “Raspberry Pi”. Next select the “Programs” option and upload the OpenPLC configuration file. After a new configuration file is uploaded and compiled, the final step is to press the “Start PLC” button.

The “Monitoring” option can be used to view the status of variables in the PLC configuration.

Modbus with Node-Red

Modbus was the earliest and most common communication protocol used to connect Industrial devices together. Modbus can be used on serial interfaces (Modbus RTU) or on Ethernet networks (Modbus TCP), both are supported by OpenPLC.

Node-Red has a number of Modbus TCP nodes that can be used. I found that : node-red-contrib-modbustcp worked well for my application. New nodes can be added to Node-Red using the “Manage Palette” option.

A simple Node-Red application that can monitor the LED and counter statuses would use three modbustcp input nodes and a text and two numeric nodes.

The Modbus read call returns 16 bits of information, so a small function was created (“Only pass Item 0”) to change the msg payload to be just the first item in the array:

msg.payload = msg.payload[0];
return msg;

Modbus supports 4 object types; coils, discrete inputs, input registers and holding registers.

For this project the LED’s IEC addressing is %QX0.2 and this would be a coil at address 2. The Function Block counter (CNT_FB) address of %QW100 is a Hold Register of 100, (CNT_ST is a Hold Register of 0).

Modbus Writing from Node-Red

The Ladder logic program was updated to light the LED from either the push button or a hold register. The hold register (%QW1) is an integer so the value is converted to a boolean then “OR”-ed with the push button interface.

On Node-Red a slider node is used to pass a 0/1 to a modbus tcp output node, that write to hold register 1.

The Node-Red web dashboard is accessed at: http://your_rasp_pi:1880/ui/

Final Comments

OpenPLC is an excellent testing and teaching tool for industrial controls.

Elixir: Easy distributed programming

Creating distributed and concurrent applications doesn’t have to be difficult. Elixir allows hobbyist and new programmers to easily create projects that can work across multiple nodes.

Elixir is a general purpose programming language that runs on top of the Erlang Virtual Machine , which is known for running low-latency, distributed, and fault-tolerant systems.

In this article I will look at three projects that will use basic Elixir functions, (no custom project setup or imported libraries).

The first projects will do remote functions between a PC and a Raspberry Pi. The second project will use multi-node requests to get Pi statistics, and the final project will look at dynamic sharing of data between three nodes.

These projects will show that distributed projects don’t have to be complicated. Each of these projects will require only 10 – 25 lines of Elixir code.

Getting Started

See the Elixir site (https://elixir-lang.org/install.html) for your specific installation.

The Elixir installation will install the Erlang VM and three new executables: iex (Interactive Elixir shell), elixir (Elixir script runner) and elixirc (Elixir compiler).

For all the projects I will keep to standard Bash command line tools so no extra Erlang or Elixir libraries will be needed.

A good first example is use the interactive Elixir shell (iex) on a Raspberry PI to write to a GPIO (General Purpose Input/Output) pin.

The iex shell is opened and the Raspberry Pi gpio command line tool is called using the base Erlang :os.cmd function:

$ iex

iex> :os.cmd(:"gpio write 7 1")
[]
iex> :os.cmd(:"gpio read 7")   
'1\n'

Elixir can call Erlang functions, just remember to place a “:” in front of the Erlang function or custom variable.

Ok that was pretty basic, the next step is to control a Pi’s GPIO from a different node.

Remote Control GPIO

A two node network can be configured by defining a username with a node address and a common cookie between the two nodes.

For my setup I logged into the Rasp Pi iex shell with a name of pi3@192.168.0.105 and I used a cookie name of “pitest”. Then I logged into the PC iex session with a name of pete@192.168.0.120 and the same cookie name of “pitest”.

From my PC iex session I only need two lines of Elixir code to remotely write to a Pi GPIO pin (Figure 2). The first line connects to the Pi Elixir node, and the second line issues a remote procedural call to run an :os.cmd statement on the Pi node:

$ iex --name pete@192.168.0.120  --cookie pitest

iex> Node.connect :"pi3@192.168.0.105"
true
iex> :rpc.call(:"pi3@192.168.0.105",:os,:cmd ,[:"gpio write 7 1"]) 
[]

It’s important to note that the underlying Erlang VM on the Raspberry Pi manages the RPC request, and for this example no Elixir code was required on the Pi node.

A Remote GPIO Write Dialog

The next step is to create a simple way to select the GPIO pin and value to write.

Elixir tends to be used for backend applications, but there are a number of good web server options and an Erlang wx module is available.

One user interface approach is to use the Elixir IO module to do text console reads and writes. To get user input the IO.gets() function is called and IO.puts will write to the console. Variables can inserted into text strings by: #{the_variable}.

iex> thepin = IO.gets("Select the Pi GPIO pin: ")
Select the Pi GPIO pin: 7
"7\n"

iex> IO.puts "Selected GPIO pin: #{thepin}" 
Selected GPIO pin: 7

For simple dialogs I like to use the command line Zenity package. Zenity support a number of different dialog types and it is pre-loaded on Raspian and most Linux OS’s.

The zenity –form command can be configured to return a GPIO pin number and pin value.

An Elixir script (Zen2gpio.exs) can be created that launches a zenity form and passes the pin data an :rpc.call function.

For this script a module Zen2gpio is created with the function show_form.

As mentioned earlier Elixir strings can have variables values inserted into strings by using: #{the_variable}.

If the user enters data and presses OK, the result string will be passed and the dialog will reopen. Pressing the Cancel button will pass no data and the script will close.

#-------------
# Zen2gpio.exs - Use a Zenity form to set the GPIO pin and value input
#-------------
defmodule Zen2gpio do
  def show_form (pnode) do
    thecmd = "zenity --forms --title='Set Pi GPIO Pins' --separator=' ' --add-entry='GPIO Pin (0-26)' --add-entry='New Value (0-1)' "
    pininfo = :os.cmd(:"#{thecmd}")
    # If some data is entered in form, write to GPIO and refresh
    if byte_size("pininfo") > 0 do
      :rpc.call(:"pi3@192.168.0.105",:os,:cmd ,[:"gpio write #{pininfo}"]) 
      show_form (pnode)
    end
  end
end
 
# Connect to the pi node
pnode = :"p3@192.168.0.105"
Node.connect  pnode

# Show the dialog
Zen2gpio.show_form(pnode)

The Elixir script runner can launch the code with the common project cookie and a unique user name.

Multi-Node RPC Requests

The goal for the next project is to have a PC node query the Pi nodes for diagnostic information. This project is a little different than the earlier project in that a module is loaded on the Raspberry Pi’s to send back custom status messages.

The Raspberry PI’s CPU temperature can be obtained using the Bash command:

# Bash command to get the Pi CPU temperature
$ /opt/vc/bin/vcgencmd measure_temp
temp=42.3'C

This Bash command can be incorporated into a small Elixir script that is loaded and compiled on each of the Pi nodes. The script (PI_stats.ex) has a module PI_stats that contains a function cpu_temp.

The cpu_temp function returns a string containing the Pi node name and the output from the shell command to get the CPU temperature.

#-----------------------------
# PI_stats.ex - Get Some Stats
#-----------------------------
defmodule PI_stats do
  def cpu_temp() do
   "#{Node.self()}  #{:os.cmd(:"/opt/vc/bin/vcgencmd measure_temp")}"
  end
  # Add more diagnostics like: available RAM, idle time ...
end

The elexirc command is used to compile an Elixir scrip. After scripts are compiled their modules are available to iex shells that are called from that directory. Below is the code to compile and then test the PI_stats module:

## compile an Elixir script
$ elixirc PI_stats.ex

## test the PI_stats.cpu_temp function locally
$ iex --name pi3@192.168.0.105  --cookie pitest
...
iex> PI_stats.cpu_temp()
{"pi3@192.168.0.105 temp=47.8\'C\n'}

An Erlang :rpc.multicall function can be used on the PC node to retrieve the Pi CPU temperatures. The :rpc.multicall function is passed the node list, module name, function call and then any addition arguments:

iex> :rpc.multicall( [:"pi3@192.168.0.105", :"pi4@192.168.0.101"], PI_stats, :cpu_temp, [])

{["pi3@192.168.0.105  temp=47.2'C\n", "pi4@192.168.0.101  temp=43.8'C\n"], []}

On the PC node a script (get_temps.exs) is created to connect to the PI nodes and get the results from the RPC multi-call. To make the code more flexible all the Pi nodes are stored in a list (pinodes). The Eum.map function will iterate the pinode list and connect to each node.

For this example the results from the RPC multi-call are a little messy so an Enum.map and Enum.join functions are used format the results to one long string that is passed to a Zenity info dialog box.

#----------------------------------------
# get_temps.exs - get PI CPU temperatures
#  - show results on Zenity Dialog
#----------------------------------------                     
pinodes = [ :"pi3@192.168.0.105", :"pi4@192.168.0.101"] 
Enum.map(pinodes, fn x-> Node.connect x end)
 
# Get results from remote PI nodes
{result,_badnodes}  = :rpc.multicall( pinodes, PI_stats, :cpu_temp, [])
 
# Format the output for a Zenity info dialog
output = Enum.map(result, fn x -> x end) |> Enum.join
:os.cmd(:"zenity --info --text=\"#{output}\" --title='Pi Diagnostics'")  

Like the earlier project the Elixir script is run with the common project cookie. and a unique user name is used.

It’s important to note that once the PI_stats.ex script is compiled on the Pi nodes no other action is required, like in the first project the RPC request is processed by the underlying Erlang VM.

Data Sharing between Nodes

Elixir offers a number of different data storage options. For simple multi-node data sharing I found that the Erlang :mnesia package to be a good fit.

For this last project Mnesia is setup to have a shared schema between the three nodes.

The Pi nodes will populate tables with their GPIO pin status every 2 seconds.

On the PC the first project will be used to write to GPIO pins, and a new script we be created to monitor the status of the GPIO pins within a Mnesia shared table.

To create a shared or distributed schema Mnesia needs to be stopped on all the nodes.

The :mnesia.create_schema function will create a shared schema for all the listed nodes.

After the schema is created Mnesia needs to be restarted. The :rpc.multicall function is extremely useful when identical actions need to done on distributed nodes:

iex> allnodes = [ :"pete@192.168.0.120" , :"pi3@192.168.0.105", :"pi4@192.168.0.101"]
iex> :rpc.multicall( allnodes, :mnesia, :stop, [])
iex> :mnesia.create_schema(allnodes)
iex> :rpc.multicall( allnodes, :mnesia, :start, [])

If you already have an existing schema, it can be deleted with by :mnesia.delete_schema([node()]).

The next step is to add tables to the schema. A table of GPIO pin values for Raspberry Pi 3 (Pi3) is created by:

iex> :mnesia.create_table(Pi3,  [attributes: [ :gpio, :value] ] )

For nodes that are writing to a specific table, it is recommended that the table be defined as both a ram and disk copy. To do this log into that node and enter:

:mnesia.change_table_copy_type(Pi3, node(), :disc_copies)

For larger projects where multiple nodes are reading and writing into tables transactions statements should be used. For small projects where only one node is writing into a table a “dirty” read and write can be used. Below is an example of writing into table Pi3 a value of 1 for pin 4, and then reading the record back.

iex> :mnesia.dirty_write({Pi3, 4,1})                                
:ok                                    
iex> pin4val = :mnesia.dirty_read({Pi3, 4}) 
[{Pi3, 4, 1}]

Now that simple writes and reads can be done the next step is to create a script that continually populates the Pi3 table with GPIO pin values.

Populating GPIO Data into a Mneisa Table

The Elixir programming language has some interesting syntax features that allow you to write some efficient code. Two features that will streamline a table input function are anonymous and enumeration functions.

Anonymous functions use the “&” to create short hard. Below is a simple example and then a complex example to read a GPIO pin value and remove the trailing new line character:

iex> # A basic example
iex> sum = &(&1 + &2)
iex> sum.(2, 3)
5

iex> getpin=&(:os.cmd(:"gpio read #{(&1)} | tr -d \"\n\" ") )
iex> getpin.(7)
'1'

The function Enum.map can do complex “for-each” loops.

These two Elixir features can be used together to read 27 Raspberry Pi GPIO pins and then write data to a Mnesia table. Below is a script (Gpio3write.exs) that will write GPIO values into a Mnesia table every 2 seconds.

#---------------
# Gpio3write.exs - Write Pi 3 GPIO values into Mnesia every 2 seconds
#---------------
defmodule Gpio3write do
  def do_write do
    getpin=&(:os.cmd(:"gpio read #{(&1)} | tr -d \"\n\" ") )
    Enum.map(0..26, fn x-> :mnesia.dirty_write({Pi3, x, getpin.(x) }) end)
    :timer.sleep(2000)
    do_write()
  end
end
# Start Mnesia
:mnesia.start()
# Cycle every 2 seconds and write values
Gpio3write.do_write()

The script on the Pi node is started by:

elixir --name pi3@192.168.0.105 --cookie pitest Gpio3write.exs

Get Records from Mnesia

Unfortunately Mnesia does not support SQL syntax, but it does support some basic filters using the dirty_match_object :

iex># Get all the records, use :_ for "all", add a sort at the end

iex> :mnesia.dirty_match_object({Pi3, :_ ,:_})  |> Enum.sort
[
  {Pi3, 0, '0'},
  {Pi3, 1, '0'},
  {Pi3, 2, '1'},
...
]
iex># Get only Pi Values that are '1'
 :mnesia.dirty_match_object({Pi3, :_ ,'1'})  |> Enum.sort
[
  {Pi3, 2, '1'},
  {Pi3, 21, '1'}
]

A Zenity list dialog can be used to show table output in columns. Below is an example of a zenity –list command in Bash. The last argument in the command is the data string which fills in the defined columns. An –extra-button option is included, and its button text is returned when the button is pressed.

A script is created (Show_gpio.exs) that reads the Mnesia results. As in the earlier example Enum.map and Enum.join functions are used to format the results as one long string for a Zenity list dialog.

#-------------------
# Show_Show_gpio.exs - show Mnesia table in a Zenity list dialog
#-------------------
defmodule Show_gpio do
  def getdata() do
    result = :mnesia.dirty_match_object({Pi3, :_ ,:_}) |> Enum.sort
    # create a presentable string for output
    output = Enum.map(result, fn x -> "#{elem(x,1)} #{elem(x,2)} " end) |> Enum.join
    feedback = :os.cmd(:"zenity --list --title=Pi3_Table --text='Pin Values' --extra-button Refresh --column=Pin --column=Value #{output}")
    if ("#{feedback}" =~ "Refresh") do
      getdata()
    end
  end
end
# Start Mnesia
:mnesia.start()
# Wait for tables to update
:timer.sleep(1000)
# Show a Zenity list dialog. 
# Refresh button to continue, other buttons to quit
Show_gpio.getdata()

Like in the earlier examples the Elixir script runner is used to call the script with the list dialog.

To test that things are working, the first project (Zen2gpio.exs) can toggle GPIO pin values, and then the Show_gpio.exs dialog can be refreshed to check the new values.

When the final project is running we have an example of Elixir concurrency. The Pi3 node is populating a Mnesia table and it is handling remote GPIO writes and CPU temperature messages.

Final Comments

Elixir with the Erlang VM offers a lot of functionality out of the box.

The next steps from here would be to start looking at messaging between nodes and creating projects with imported Elixir libraries.

Simple Cross Platform Apps with Ren’Py

There are some excellent cross platform development tools out there. However for new programmers many of these tools can come with a steep learning curve.

Ren’Py is a visual novel engine that has been around for over 10 years. The Ren’Py software development kit (SDK) is supported on Linux, MacOS and Window and the Ren’Py final application can be built to run on Android, iOS Linux, MacOS, Window and HTML5.

Ren’Py uses a simple “Screen Language” so no previous programming experience is required. For more complex requirements Ren’Py supports Python.

The focus of Ren’Py is visual story telling and games, but new programmers can also create some simple apps that can be run on PCs, smart phones and web servers.

This blog will introduce Ren’Py with three examples:

  1. the start of visual novel with a couple of characters
  2. a tourist guide with graphic menus
  3. a dashboard with dynamic values and bars

Getting Started

If you want to do some playing of Ren’Py on a Raspberry Pi a light weight installation can be loaded by:

sudo apt-get install renpy

The apt-get version of Ren’Py however will not have any tutorials or extra build features so it is recommended that you go to the Ren’Py site for the complete installation directions.

The Ren’Py user interface will create new projects with all the required files and prompt the user for any additional requirements.

The script.rpy file contains all the logic for an application.

A Visual Novel

A visual novel requires some characters and background images.

Creating character drawings from scratch can be a lot of work, luckily there are some open source solutions such site as: https://charactercreator.org.

The character creator site allows for the creation of head, torso and head images. It also supports different facial expressions.

Below is a Ren’Py example showing a background, with a policeman and a suspect.

The script.rpy file contains the Screen Language code.

The first step is to define all the characters, in this game there two characters “cop” and “me”.

Ren’py uses labels to jump between code segments. The application begins are the start label (line10).

The image files are stored in game/images directory. Images can also be resized, rotated, moved or adjusted (Line 6). The image file cop_head.png can be referenced directly, and shown by:

show cop_head at truecenter

Dialog is shown by referencing the character and then the dialog text (lines 19-20).

By using hide and show statements different characters and backgrounds can presented. The applications ends with a return statement.

A Tourist Guide

For most graphic novels menus are required. These menus allow the application to have multiple outcomes or branches.

A Ren’Py menu is created by simply using a menu: statement. Each menu item is defined with button text and a jump statement. A jump statement is like an old-school BASIC GOTO statement, each jump has a label that the code will link to.

The example below is the start of a tourist guide for the Bruce Peninsula. The menu is called at the start of program. A menu item will jump to a specific section of the code using a label. Within a subsection, (like the beach), a new background and text is shown. After the user sees this information a jump start statement puts them back to the main menu.

For main menus it is probably cleaner to have subsections defined as separate label sections. However for smaller application it is possible to put the submenus and the display logic directly in the menu: logic.

Dynamic Screens

Ren’Py supports screens and the use of Python for applications that require more complex functions.

For the next example some CPU stats will be shown. The Linux sensors command will return the CPU and Ambient temperature:

$ sensors
dell_smm-virtual-0
Adapter: Virtual device
Processor Fan: 2691 RPM
CPU:            +53.0°C  
Ambient:        +41.0°C  

# use show Bash/AWK code to get temps
$ sensors | grep CPU | awk '{ printf "%d\n" , $2}'
53
$ sensors | grep Ambient | awk '{ printf "%d\n" , $2}'
41

A Ren’Py application can be created to show the CPU stats in a dynamic screen.

The code below defines a Ren’Py screen object and some Python code is used to shell out to the Bash/AWK statements.

A Ren’Py supports Python in either a single statement or as a python block. A single Python statement is define with a leading $, for example to set a variable: $biking = “YES”. A Python block is defined with a starting python: statement, then the code is indented from this statement.

The init python: statement is a code Python code block that is used to define (import) libraries and to initialize variables.

The Ren’Py screen object supports labels, text and bars that can be arranged or oriented in horizontal (hbox) or vertical (vbox) groupings.

# The script of the game goes in this file.

screen ts():

    python:
        now = datetime.now()
        nowtime = now.strftime("%H:%M:%S")
        ctemp = subprocess.check_output("sensors | grep CPU | awk '{ printf \"%d\" , $2}'", shell=True)
        atemp = subprocess.check_output("sensors | grep Ambient | awk '{ printf \"%d\" , $2}'", shell=True)
    
    frame:
        has vbox
        label "CPU Stats" text_size 120 
        
        text ""
        vbox:
            text "Time : [nowtime]" size 80
            text "Ambient temp: [atemp] C \n" size 80
            text "   CPU temp : [ctemp] C " size 60
        hbox:
            vbox:
                text "0 " size 40
            vbox:
                bar value atemp range 60 xalign 50 yalign 50 xmaximum 600 ymaximum 50 
            vbox:
                text " 60 " size 40


init python:

    import subprocess
    from datetime import datetime

label start:

    # Start with Weather screen

    show screen ts()
    # Cycle every 2 second to refresh the data

    define cycle = "True"

    while cycle == "True" :
        $ renpy.pause(2)

    return

Building a Final Application

The Ren’Py IDE Build option supports Windows and Linux apps directly. For Mac and Android builds the user will be prompted for some dependencies that will need to be loaded.

The HTML5 Build is still in beta, it appears to work well for standard graphic novel apps, but I found that it had some issues with Python library calls.

The HTML5 Build will create a separate library structure for the Ren’Py application which will need to be mapped into your Web Server configuration.

If you are looking to do some simple testing a Python standalone webserver can be run from the Ren’Py web directory:

# Run a python standalone server on port 8042
python3 -m http.server 8042

Final Thoughts

If you’re looking to create some simple cross-platform visual presentation apps Ren’Py is a good fit. Ren’Py is super easy to learn and don’t have to have strong programming skills.

Ren’Py supports simple screens that can have dynamic bars and text, however if you’re looking to incorporate gauges or line charts Ren’Py probably isn’t the best fit.

COBOL Programming in 1-hour

This morning I read about 60 year old COBOL being still alive and there was a need for programmers. I couldn’t believe it so I checked the job boards and I found 2 jobs in my immediate area.

I thought that I’d spend an hour or so looking into COBOL. Ok I’m definitely no way near an expert, but I was surprised how much I was able to pick-up.

Getting Started

Traditionally Cobol (Common Business Oriented Language) only ran on mainframe computers. Now luckily you can install a full working environment on Windows and Linux.

If you are working in Windows there are few options: NetCobol, GnuCobol, and the Hercules Emulator

I work primarily in Linux and found that OpenCobol was very easy to use. If you’re working in Ubuntu, Debian or on a Raspberry Pi enter:

sudo apt-get install open-cobol

An Input/Output Program

A Cobol program starts with 2 lines, and an IDENTIFICATION , and a PROGRAM-ID. Comments lines start with a *. All Cobol programming lines end with a period (.).

For this example 4 variables were defined in the DATA layout section. The PIC (picture) argument is used to define the type of variables X is alpha, and 9 is numeric.

*>  test2.cbl - get user input, and do output
*>	  
IDENTIFICATION DIVISION.
PROGRAM-ID. HELLO.

DATA DIVISION.
   WORKING-STORAGE SECTION.
   01 WS-ID PIC 9(3) VALUE 101.
   01 WS-STUDENT-NAME PIC X(25).
   01 WS-DEPT PIC X(25) VALUE 'Engineering'.
   01 WS-DATE PIC X(10).

PROCEDURE DIVISION.
   DISPLAY "Enter Your Name:".
   ACCEPT WS-STUDENT-NAME.
   ACCEPT WS-DATE FROM DATE.
   DISPLAY " ".
   DISPLAY "Name :  " WS-STUDENT-NAME " ID: " WS-ID.
   DISPLAY "Dept :  " WS-DEPT.
   DISPLAY "Date :  " WS-DATE.

STOP RUN.

For this example the ID and Dept are predefined. The WS-DATE is set by today’s DATE.

The WS-STUDENT-NAME is entered by the user.

To compile and run the program (ctest2.cbl) :

$ cobc -free -x -o ctest2 ctest2.cbl
pete@lubuntu:~/Writing/Blog/cobol$ ./ctest2
Enter Your Name:
Pete
 
Name :  Pete                      ID: 101
Dept :  Engineering              
Date :  210701    

Math

Cobol uses VERBs for math and conditional statements. Below is an example doing some basic math with an IF condition:

*> cmath.cbl - do some math with an IF condition
*>
IDENTIFICATION DIVISION.
PROGRAM-ID. HELLO.

DATA DIVISION.
   WORKING-STORAGE SECTION.
   01 A PIC 9(5) VALUE 10.
   01 B PIC 9(5) VALUE 20.
   01 C PIC 9(5) VALUE 0.
   01 D PIC 9(5) VALUE 0.
   01 E PIC 9(5) VALUE 0.
	  
PROCEDURE DIVISION.
   ADD A B TO C.
   MULTIPLY A BY B GIVING D.
   IF D > 100 THEN
      MOVE D TO E
   ELSE
	  MOVE 99 TO E
   END-IF.
   
   DISPLAY "C = " C.
   DISPLAY "D = " D.
   DISPLAY "E = " E.
STOP RUN.

Note for the IF statement there is no end of statement (.) until END-IF.

To compile and run this program:

$ cobc -free -x -o cmath cmath.cbl
$ ./cmath

C = 00030
D = 00200
E = 00200

File I/O – Read

Cobol has a number of read/write formats. The easiest is a fixed field sequential read/write.

For an example with input.txt , the STUDENT-ID is 5 characters and the name is 25.

20003 Albert Smith            
20004 Malika Jones            
20005 Bob Smith 

The Cobol code has a FILE section that defines the input fields. A WORKING-STORAGE section mimic the input file structure, with the exception of a end of File variable.

The STUDENT file object is written into the WS-STUDENT object until an END of file is set.

IDENTIFICATION DIVISION.
PROGRAM-ID. HELLO.

ENVIRONMENT DIVISION.
   INPUT-OUTPUT SECTION.
      FILE-CONTROL.
      SELECT STUDENT ASSIGN TO 'input.txt'
      ORGANIZATION IS LINE SEQUENTIAL.            

DATA DIVISION.
   FILE SECTION.
   FD STUDENT.
   01 STUDENT-FILE.
      05 STUDENT-ID PIC 9(5).
      05 NAME PIC A(25).

   WORKING-STORAGE SECTION.
   01 WS-STUDENT.
      05 WS-STUDENT-ID PIC 9(5).
      05 WS-NAME PIC A(25).
   01 WS-EOF PIC A(1). 

PROCEDURE DIVISION.
   OPEN INPUT STUDENT.
      PERFORM UNTIL WS-EOF='Y'
         READ STUDENT INTO WS-STUDENT
            AT END MOVE 'Y' TO WS-EOF
            NOT AT END DISPLAY WS-STUDENT
         END-READ
      END-PERFORM.
   CLOSE STUDENT.
STOP RUN.

File I/O – Append (Write)

Using the input.txt file, the code below (cwrite.cbl) will add 2 more records:

IDENTIFICATION DIVISION.
PROGRAM-ID. HELLO.

ENVIRONMENT DIVISION.
   INPUT-OUTPUT SECTION.
      FILE-CONTROL.
      SELECT STUDENT ASSIGN TO 'input.txt'
      ORGANIZATION IS LINE SEQUENTIAL. 
  
DATA DIVISION.  
    FILE SECTION.   
    FD STUDENT.
    01 STUDENT-FILE.
      05 STUDENT-ID PIC 9(5).
      05 NAME PIC A(25). 
 
  
PROCEDURE DIVISION.  
    DISPLAY 'WRITING TO A SEQUENTIAL FILE..'  
    OPEN EXTEND STUDENT.  
    MOVE '20006' TO STUDENT-ID.  
    MOVE ' Santa Claus' TO NAME.   
    WRITE STUDENT-FILE  
    END-WRITE. 
      
    MOVE '20007' TO STUDENT-ID.  
    MOVE ' Mary Christmas' TO NAME.  
    WRITE STUDENT-FILE  
    END-WRITE. 
      
    CLOSE STUDENT.  
STOP RUN.

To compile, run and check the output:

$ cobc -free -x -o main cwrite.cbl
$ ./main
WRITING TO A SEQUENTIAL FILE..
$ cat input.txt
20003 Albert Walker 
20004 Malika Jones
20005 Bob Cat
20006 Santa Claus
20007 Mary Christmas

Final Comments

After coding in other languages it becomes immediately obvious that Cobol would require many more lines of code than it would in Python.

Learning Cobol does not appear to be overly difficult, however there are a ton of little nuances that will need to be flushed out.

Re-purpose Your Old Home Router

We had some old routers at home that I thought that I’d see if I could re-purpose.

Some of the typical applications that can be done are:

  • A wireless repeater / access point
  • A remote switch
  • A print server for USB printers
  • A network storage device (with a portable USB drive)

A router can also be used as a low end application platform and it can do many projects that a Raspberry Pi might do. In this blog I will be looking at:

  • USB drives
  • Web Cams
  • USB Sensors
  • CGI Web Servers

Comparing a Router to a Raspberry Pi

Before getting started it’s useful to know how a re-purposed router stacks up against a Raspberry Pi.

For our router project we kept things super simple. We used command line apps and scripts rather than Python.

Pro’s of a Router:

  • Price. I got my routers used from $5-$10. A new Pi 3 or 4 is $35-$50.
  • Housing. A router is ready to be mounted
  • Network Ready. If you’re looking at networking functionality you are good to go.
  • Lots of Basic Apps. Most of key features are available.

Con’s of a Router:

  • Extremely Space Limited. Most of the routers have 32MB vs. 8+ GB of a Pi
    • Need to Pick Apps. You probably can’t load Python3, Apache and Samba at the same time.
    • Probably can’t run X-Windows.
  • Ash not Bash. The lighter weight Ash shell is supported not Bash. This may not be an issue.

Open Source Router Firmware

There are a number of open source firmware solutions that can put life back in old routers. OpenWRT and DD-WRT are the most popular packages, but there are others.

The first step is to determine if your old router is supported with one of these packages. It is important to note that many of the older routers only have 4MB of flash and 32MB of RAM, and this may not run or only marginally run OpenWRT or DD-WRT. The recommended requirements are 8+ MB of flash and 32+ MB of RAM.

Another consideration is USB support. A router without USB support can still be used as a web or application server but it will be missing external hardware integration.

For our router project we used OpenWRT.

Getting Started

How the re-purposed router will be used will effect the setup. It is important to ensure the re-purposed router does not effect other routers in your home LAN. Typically you’ll want to disable Router Features such as the DNS and DHCP server capabilities. In the OpenWRT web interface (Luci) software services can be disabled.

I did lots of playing and I bricked my router about a dozen times. Luckily it is usually possible to un-brick a router. I only permanently bricked one 1 router and it was with a bad DD-WRT firmware.

Un-brinking a Router

Often all you need to do is just reset your router and then connect directly to a LAN port to redo your configuration.

If this fails check the OpenWRT blog for any recommendations for your router. There are some excellent custom solutions but they are usually manufacturer specific.

For Netgear see nmrpflash , it’s an almost 100% sure proof un-brinking solution.

30-30-30 Hard Reset Rule

If resetting the router doesn’t work, the next set is the 30-30-30 Rule. This will typically work for all routers:

  • Press reset button for 30 seconds.
  • While holding, unplug router for 30 seconds
  • power on, still holding setup button 30 seconds.
  • Let go of reset button. Then try to reconfigure. (If this doesn’t work try another power down).

Adding Software Features

OpenWRT uses opkg the OpenWRT package manager. Software can be added either through the Luci web interface or manually in an SSH shell.

After a new install the opkg list of available packages will need to be update, this can be done from SSH by:

## update okpg is required to get info on new packages
opkg update

It is possible to use a USB drive to extend the router’s root file system, this would remove issues of running out space when installing software packages.

Add USB Support

Typically routers do not have a lot of available space. Use the df command to check space:

root@OpenWrt:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root                 2560      2560         0 100% /rom
tmpfs                    30060       228     29832   1% /tmp
/dev/mtdblock5           12160       488     11672   4% /overlay
overlayfs:/overlay       12160       488     11672   4% /
tmpfs                      512         0       512   0% /dev

OpenWRT supported software is loaded directly onto the router so some care is required when deciding which packages are to be installed.

Adding a USB drive can greatly help with the management of source files and backups.

The okpg (OpenWRT package manager) is used used to install software packages. To install USB drive support:

## update okpg is required to get info on new packages
opkg update
## get USB packages
opkg install block-mount e2fsprogs kmod-fs-ext4 kmod-usb-storage kmod-usb2 kmod-usb3

A useful command line USB tool is lsusb, however check how much space you have before installing it. To install lsusb enter:

opkg install usbutils

root@OpenWrt:~# lsusb
Bus 001 Device 003: ID 0a5c:bd17 Broadcom Corp. BCM43236 802.11abgn Wireless Adapter
Bus 001 Device 004: ID 413d:2107  
Bus 001 Device 006: ID 1871:0143 Aveo Technology Corp. 
Bus 001 Device 005: ID 058f:6387 Alcor Micro Corp. Flash Drive
Bus 001 Device 002: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

After the USB packages are loaded the router needs to be rebooted.

A new menu item called “Mount Points” under the “System” heading will be added to the Luci web interface. This option allows for easy addition and removal of USB drives.

If you are mounting more that 1 device, it is recommended to use the UUID label, otherwise there is potential that the mount points could be swapped between devices.

USB Web Cam

There are a number of different USB video solutions that are available. Some packa like Motion are full featured but they might require more resources than your router supports.

For light weight USB video support mjpeg streamer is good place to start. It can be installed by:

opkg install kmod-video-uvc mjpg-streamer

The configuration of the video is defined in: /etc/config/mjpg-streamer .

config mjpg-streamer 'core'
        # option enable '1' = on, '0'=off
        option enabled '1'
        option input 'uvc'
        option output 'http'
        option device '/dev/video0'
        option resolution '1280x1024'
        option yuv '0'
        option quality '80'
        option fps '5'
        option led 'auto'
        option www '/www/webcam'
        option port '8080'
        #option listen_ip '192.168.1.1'
        option username 'openwrt'
        option password 'openwrt'

The video service needs to be started:

## to start the service:
/etc/init.d/mjpg-streamer start
## to enable the service to start on boot
/etc/init.d/mjpg-streamer enable

The webcam is available on a router’s IP with the default port of 8080.

USB Sensors

Most sensors are 0-5 Volts (i.e. the typical Arduino sensor) but there are a few USB sensors.

I purchased a USB Thermometer (~$12 from Walmart).

The USB Thermometer acted like a human interface devices (HID)

A script in Ash (a light weight version of Bash) is used to get the temperature, and it required a few software packages:

opkg install kmod-usb-hid coreutils-printf xxd nano

The temper.sh script sends a query message to the HID address and then exacts the temperature from the returned binary data:

#!/bin/ash
#
# temper.sh - get temperature from a USB thermometer 
#
hid="/dev/hidraw1"
exec 5<> $hid
# send out query msg
echo -e '\x00\x01\x80\x33\x01\x00\x00\x00\x00\c' >&5
# get binary response
OUT=$(dd count=1 bs=8 <&5 2>/dev/null | xxd -p)
# characters 5-8 is the temp in hex x1000
HEX4=${OUT:4:4}
DVAL=$(printf '%d' "0x${HEX4}")
awk "BEGIN {print ($DVAL/100)}"

The script can then set to an executable or run thru ash:

root@OpenWrt:/usbstick# ash temper.sh
24.43
root@OpenWrt:/usbstick# chmod +x temper.sh
root@OpenWrt:/usbstick# ./temper.sh
24.43

CGI Web Server

The OpenWRT runs the Luci configuration interface on a light weight uhttp server. This web server can also be used for custom pages.

The uhttp web server will run Lua or CGI pages from the /www/cgi-bin directory.

Below is an Ash CGI example that runs the usb thermometer script and the vmstat command:

#!/bin/ash
#
# test.cgi - show the USB thermometer temperature and CPU stats 
#
echo "Content-type: text/html"
echo ""
echo "<!DOCTYPE html>
<html>
<head><title>Router Points</title>
</head>
<body>
<h1>Router CGI Page</h1><hr>"

echo "Temperature:  $(/littlehd/temper.sh) DegC"

echo "<hr><h3>Router CPU Stats</h3>"
echo "<pre>"
vmstat
echo "</pre>"

Final Comments

I found that re-purposing a router was an interesting project that could progress to a number of interesting side topics such as:

  • USB over IP – use an old router to remotely configure Arduino or ESP32 devices.
  • Remote network sniffer or traffic monitor
  • Print and Network Storage Servers

Bash with MQTT

I’m working on re-purposing an old router. One of my goals is to bring sensor and performance data from the router to a Home Assistant node. The router doesn’t not have a lot of space so I’d prefer to use Bash rather than Python for MQTT communications.

While I was working on the project I wanted to use some simple tools to view the data, unfortunately I wasn’t able to find any information on how to make a good MQTT client in Bash.

This blog documents how I used Bash to show bar charts of MQTT data.

Setup

The first step is to install the Mosquitto command line tools on the OpenWrt router:

# Update the package manager list
opkg update
# Install the MQTT client utiliy
opkg install mosquitto-client-nossl

When I have some time I’d like to look at making some more MQTT Bash scripts that could be used with file input.

The next step is to install the Mosquitto client on my Linux PC and Raspberry Pi:

sudo apt-get install mosquitto-clients

There are a number of MQTT brokers that can be used, both Home Assistant and Node Red have reliable brokers. The Mosquitto broker can also be loaded on a Linux, MacOS and Windows node.

Publish MQTT Data in Bash

The Mosquitto client can be used to both publish and subscribe to MQTT data.

For my router project, I wanted the: temperature (from a USB thermometer), idle time, % used space and available space. Below is the script that passed the data to the mosquitto_pub (publishing) tool:

#!/usr/ash
#
# mqtt_data.sh - send data to MQTT broker
#
# Get Data values
temp=$(/usbstick/temper.sh)
idle=$(vmstat | awk '{ if (NR==3) print $15}')
used=$(df | awk '{if (NR==4) printf "%.f\n", $5 }')
space=$(df | awk '{if (NR==4) printf "%.1f\n", $4/1000 }')

echo "$temp $idle $used $space"

# Publish Data
server="192.168.0.111"
pause=2

mosquitto_pub -h $server -t rtr_temp -m $temp
sleep $pause
mosquitto_pub -h $server -t rtr_idle -m $idle
sleep $pause
mosquitto_pub -h $server -t rtr_used -m $used
sleep $pause
mosquitto_pub -h $server -t rtr_space -m $space

I added a pause between each publish to let me watch the actions.

The cron utility can be used to schedule the running of the script. Once a minute is the fastest time available with cron, so if faster times are needed the script could cycle with a while loop.

Read/Subscribe to MQTT Data

The mosquitto_sub client tool can be used to read or subscribe to the data.

To look at the router points:

$ service="192.168.0.111"
$ mosquitto_sub -h $server -v -t rtr_idle -t rtr_temp -t rtr_used -t rtr_space

rtr_temp 26.25
rtr_idle 100
rtr_used 66
rtr_space 1.1

The -v (–verbose) option show output with the topic names and the values. Topics, -t option, can be put on the command line or passing in as a file.

For single point monitoring the Zenity utility can be used. This utility is typically preloaded on most Linux and Rasp Pi systems. A script to create a progress bar dialog with MQTT data is:

# Send MQTT values to a progress bar
( 
  while :; do
  msg=$(mosquitto_sub -h 192.168.0.111 -C 1 -t rtr_temp) 
  echo $msg
  echo "#$msg  Deg C"
  done
  ) | zenity --progress  --title="Router External Temperature"

Multipoint Progress Dialogs

The Zenity utility can only manage single point progress bars. For multiple point progress bars the YAD (Yet Another Dialog) package can be used.

To install YAD on Raspberry Pi’s and Ubuntu: sudo apt-get install yad

Below is some script that will show multiple points in a YAD dialog:

#!/usr/bash
#
# mqtt_bars.sh - Show multiple MQTT Topics on a Dialog
#
server="192.168.0.111"
topics=("rtr_idle" "rtr_temp" "rtr_used" )
scale=(100 40 100 )
units=( '%' 'degC' '%'  )
title="Router MQTT Points"

#Build topic and yad strings
yadstr=" "
for i in ${topics[@]}; do
  topstr="$topstr -t $i"
  yadstr="$yadstr --bar=$i"
done

echo "Press [CTRL+C] to stop..."
# Cycle thru 1 message at a time to YAD 
(
while : 
do 
  msg=$(mosquitto_sub -h $server -v -C 1 $topstr) 
  IFS=' ' read -a data <<< "$msg" 
  # match returned msg to order of bars, write value/label
  for i in "${!topics[@]} "
  do 
    if [ "${data[0]}" = "${topics[i]}" ]
    then 
      let j=i+1 ; # YAD indices start at 1
      # Rescale bar to defined scale
      barsize=$(bc <<< "${data[1]}*100/${scale[i]}")
      #barsize=100
      echo "$j:$barsize"
      echo  "$j:#${data[1]} ${units[i]}"
  fi
  done
done 
)  | yad --multi-progress $yadstr --title $title

Final Comments

There are probably lots of good 3rd party tools like Gnuplot that could be connected to mosquitto_sub to show real charts.

Bash Bar Charts for Text and Web Pages

This blog documents my notes on creating Bash text bar charts. The key topics are:

  • Create a simple horizontal bars (3 lines)
  • Create dynamic bars (~25 lines for an array of data values)
  • Add support for piped and command line data
  • Include options for: help, resizing, titles
  • Add colour and HTML support

A Horizontal Bar

To create a static bar an printf statements can be used. The seq {0..10} can be used to repeat an ASCII █ fill character 10 times.

$ printf 'LABEL: ' ; \
  printf '█%.0s' {1..10} ; \
  printf ' 10\n'

LABEL: ████████████████████ 10

Unfortunately the printf statement has some limitations on variable substitution. A simple workaround is to create a string and then eval it:

$ label="temp"; val=20;
$ bar="printf '█%.0s' {1..$val}" ; 
$ printf '\n%-5s ' $label; eval $bar ; printf ' %d\n' $val

temp  ████████████████████ 20

Coloured Bars

The tput setaf command can change the foreground, and tput setab is used for background colours. Colour codes are:

tput setab [1-7] # Set the background colour using ANSI escape
tput setaf [1-7] # Set the foreground colour using ANSI escape

Num  Colour    #define         R G B

0    black     COLOR_BLACK     0,0,0
1    red       COLOR_RED       1,0,0
2    green     COLOR_GREEN     0,1,0
3    yellow    COLOR_YELLOW    1,1,0
4    blue      COLOR_BLUE      0,0,1
5    magenta   COLOR_MAGENTA   1,0,1
6    cyan      COLOR_CYAN      0,1,1
7    white     COLOR_WHITE     1,1,1

To reset colours back to the defaults use: tput sgr0

An example to print a red bar and a stack of bars:

$ printf '\nLABEL: ' ; \
   tput setaf 1 ;\
   printf '█%.0s' {1..10} ; \
   printf ' 10\n'

LABEL: ██████████ 10
$ printf '\n 3 Stacked Bars: ' ; \
   tput setaf 1 ;\
   printf '█%.0s' {1..10} ; \
   tput setaf 2 ;\
   printf '█%.0s' {1..8} ; \
   tput setaf 4 ;\
   printf '█%.0s' {1..3} ; \
   printf ' 10+8+3=21\n'

 3 Stacked Bars: █████████████████████ 10+8+3=21

Dynamic Bars

The next step is to create a script that dynamically updates the bars. The tput clear command will clear the terminal screen keep the data and bars in the same location. The script below will dynamically show the CPU temperature, idle time and 2 random values with a 10 second update time.

#!/bin/bash
# 
# cpu_bars.sh - Show new data every 10 seconds
#
while :; do
    # Get data values
    CPUtemp=$(sensors | grep CPU | awk '{print substr($2,2,4)}')
    CPUidle=$(iostat | awk '{if (NR==4) print $6}')
    Random1=$((1+ $RANDOM % 100))
    Random2=$((1+ $RANDOM % 100))

    labels=( CPUtemp CPUidle Random1 Random2)
    values=( $CPUtemp $CPUidle $Random1 $Random2)
    units=( degC % psi mm)

    # Show a title
    tput clear
    printf " %10s " "" 
    tput setaf 7; tput smul;
    printf "%s\n\n" "Show CPU Data ($(date +%T'))"
    tput rmul;

    # cycle thru data and show a label, 
    for index in "${!labels[@]}"
    do
          tput setaf $(expr $index + 1); # don't use 0 (black) 
          printf " %10s " "${labels[index]}"
          eval "printf '█%.0s' {1..${values[index]%.*}}"
          printf " %s %s\n\n" ${values[index]} ${units[index]}
    done
    sleep 10
done

This script is run by: bash cpu_bars.sh .Typical output is below.

Bars in HTML

The ANSI colours are not supported in HTML, so instead HTML tags and style properties can be used.

It is important to use <pre> tags for Bash text output. Code to create two static bars in HTML would be:

$ (printf "<h1>A Bar from Bash</h1>\n" 
 printf "<pre><span style='font-size:24px;color:red'}>\n"
 printf 'LABEL1: ' ; printf '█%.0s' {1..10} ; printf ' 10\n'
 printf "</pre></span>\n") > bar1.htm

$ cat bar1.htm
<h1>A Bar from Bash</h1>
<pre><span style='font-size:24px;color:red'}>
LABEL1: ██████████ 10
</pre></span>

The script cpu_webbars.sh creates HTML output for an array of calculated values:

#!/bin/bash
# 
# cpu_webbars.sh - Show bars in HTML
#

# Get data values
CPUtemp=$(sensors | grep CPU | awk '{print substr($2,2,4)}')
CPUidle=$(iostat | awk '{if (NR==4) print $6}')
Random1=$((1+ $RANDOM % 100))
Random2=$((1+ $RANDOM % 100))

labels=( CPUtemp CPUidle Random1 Random2)
values=( $CPUtemp $CPUidle $Random1 $Random2)
units=( degC % psi mm)
colors=(red blue green magenta)

# Show a title
printf "<h1><center>Show CPU Data ($(date '+%T'))</center></h1>\n"
# cycle thru data and show a label, 
for index in "${!labels[@]}"
  do
  printf "<pre><span style='font-size:18px;color: ${colors[index]} '}>\n"
  printf " %10s " "${labels[index]}"
  eval "printf '█%.0s' {1..${values[index]%.*}}"
  printf " %s %s\n\n" ${values[index]} ${units[index]}
  printf "</pre></span>\n"
done

This script can be run and outputted to an file: bash cpu_webbars.sh > test.htm

Once the basic HTML output is working its possible to add headers and footers to make a more complete page:

header.htm > test.htm ; \
cat cpu_webbars.sh >> test.htm ; \
cat footer >> test.htm 

Piping Data to Bars

A script (hbar0.sh) will read the piped data and then create an array (data) of labels and values. The data array is cycled through and labels and bars are shown with a randomized colour:

#!/bin/bash
# hbar0.sh - Read in piped data  and plot bars
#     format: label,value;label2,value2;  and plot bars
#
input=$(< /dev/stdin) ; # read piped data
# remove new lines in files, and check for ";" after data pair 
input=$(echo $input | tr -d '\n')
IFS=';' read -r -a data <<< $input
printf "\n" 
for element in "${data[@]}"pete@lubuntu:~/Writing/Blog/text_bars
do 
  # make at array of each data element
  IFS=',' read -r -a datapt <<< $element
  # add a random color
  tput setaf $((1+ $RANDOM % 7))
  # print the label, bar and value
  printf " %10s " "${datapt##*[0]}"
  bar="printf '█%.0s' {1..${datapt[1]}}"
  eval $bar
  printf " %s\n\n" ${datapt[1]} 
  tput rmso ; # exit color mode   
done

The script can be tested with piped data:

$ echo "temp,33;pressure,44" | bash hbar0.sh

 temp       █████████████████████████████████ 33 

 pressure   ████████████████████████████████████████████ 44 

A data file can also be passed in using the cat command:

$ cat data0.txt 
temp,44;
humidity,33;
pressure,15;
wind spd,33;
wave ht,3;
$ cat data0.txt | bash hbar0.sh

       temp ████████████████████████████████████████████ 44

   humidity █████████████████████████████████ 33

   pressure ███████████████ 15

   wind spd █████████████████████████████████ 33

    wave ht ███ 3

Removing ANSI Colors

Terminal applications use ANSI color codes which unfortunately is not support on Web pages.

ANSI color codes can be removed from files and strings by:

# Strip out ANSI color codes:
cat infile | sed 's/\x1b\[[0-9;]*m//g' > outfile
$ echo "temp,33;pressure,44" | bash hbar0.sh > hbar0.txt
$ cat hbar0.txt

       temp █████████████████████████████████ 33

   pressure ████████████████████████████████████████████ 44

$ cat hbar0.txt | sed 's/\x1b\[[0-9;]*m//g'

       temp █████████████████████████████████ 33

   pressure ████████████████████████████████████████████ 44

A Final App

With the basics in place I was able to create an app that would support scaling, titles, units, custom colour and web output:

$ ./hbars
usage: hbars [data] [option] 
  -h --help     print this usage and exit
  -c --color    set color to all bars (default 7=white)
                 (0-grey,1-red,2=green,3=yellow,4=blue,5=magenta,6=cyan,7=white)
  -p --pretty   add different colors to bars (-c overrides)
  -t --title    top title
  -w --width    max width of bar (default 50)
  -W --Web      make output HTML formatted
  -f --fontsize fontsize for web output (default 24)

 examples:
   echo 'temp,33,C;pressure,14,psi' | ./hbars -t Weather -p -w 40 
   ./hbars -t Weather -p -w 40  'temp,33;pressure,14' 
   cat data.txt | ./hbars -W -f 24 -t 'Raspi Data' > data.htm

The code:

#!/bin/bash
#
# hbars.sh - show some text bars
#   pass data as:  label1,value1,unit1,color1; label2,value2,unit2,colour2; ....  
#
width=50
title=""
color=7
pretty=False
web=False
font=24

usage() { 
  echo "usage: hbars [data] [option] "
  echo "  -h --help     print this usage and exit"
  echo "  -c --color    set color to all bars (default 7=white)"
  echo "                 (0-grey,1-red,2=green,3=yellow,4=blue,5=magenta,6=cyan,7=white)"
  echo "  -p --pretty   add different colors to bars (-c overrides)"
  echo "  -t --title    top title"
  echo "  -w --width    max width of bar (default 50)"
  echo "  -W --Web      make output HTML formatted"
  echo "  -f --fontsize fontsize for web output (default 24)"
  echo ""
  echo " examples:"
  echo "   echo 'temp,33,C;pressure,14,psi' | ./hbars -t Weather -p -w 40 "
  echo "   ./hbars -t Weather -p -w 40  'temp,33;pressure,14' "
  echo "   cat data.txt | ./hbars -W -f 24 -t 'Raspi Data' > data.htm"
  echo ""

  exit 0
}
# Show help usage if no pipe data and no cmd line data
if [ -t 0 ]  && [ $# -eq 0 ] ; then
  usage
fi
# Check for command line options
while getopts "hpc:t:w:Wf:" arg; do
  case "$arg" in
    h) usage ;;
    c)  color=$OPTARG ;;
    p)  pretty=True; icolor=0 ;;
    t)  title=$OPTARG ;;
    w)  width=$OPTARG ;;
    W)  web=True;;
    f)  font=$OPTARG ;;
  esac
done
#------------------------------------------------
# Setup formatting for text, Web and color
# -----------------------------------------------
if [[ ${color} != 7 && ${pretty} = True ]]; then
  pretty=False
fi
colidx=0

setcolors() {
if [ $web = True ]; then
  colors=(gray red green yellow blue magenta cyan white)
  titlebold="echo '<h1>'"
  titlereset="echo '</h1>'"
  #color_set='echo "<span style=font-size:$(font)px  >" ' 
  #color_set="printf '<span  style=\"font-size:$(font)px;color:${colors[colidx]}\" >'" 
  color_set="printf '<pre><span style=\"color:${colors[colidx]} ; font-size:${font}px \" >'" 
  color_rs="echo '</span></pre>'"
else
  colors=(0 1 2 3 4 5 6 7 )
  titlebold="tput bold; tput smul"
  titlereset="tput rmul; tput rmso"
  color_set="tput setaf ${colors[colidx]}"
  color_rs="tput rmso"
fi
}
setcolors
#----------------------------------------------
# Get data, check if stdlin, file, if not assume string
#----------------------------------------------
if [ -p /dev/stdin ]; then
        lastarg=$(< /dev/stdin)
else
	lastarg=$(echo "${@: -1}")
	if test -f "$lastarg"; then
	  lastarg=$(<$lastarg)
	fi
fi
# Cleanup the input data
lastarg=$(echo $lastarg | sed 's/\n/;/g; s/  / /g; s/\t//g; s/;;/;/g')
IFS=';' read -r -a array <<< $lastarg

# ensure that there is some data
if [[ ${array} == 0 ]]; then
  echo "No data found"
  exit 0
fi
echo "input:$lastarg"
#exit 0
#------------------------------------
# Get max value and max label length
#------------------------------------
maxval=0
maxlbl=10
#echo "array:${array[@]}"
for element in "${array[@]}"
do 
  IFS=',' read -r -a datapt <<< $element
  if (( $(echo "$maxval < ${datapt[1]}" |bc -l) )); then
	maxval=${datapt[1]}
  fi
  if (( $(echo "$maxlbl < ${#datapt[0]}" |bc -l) )); then
	maxlbl=${#datapt[0]}
  fi
done
#---------------------------------
# Print Title - use bold/underline
#---------------------------------
if [[ ! -z $title ]]; then
  printf "\n %${maxlbl}s " " "
  eval $titlebold
  printf "%s" "${title}" ; printf "\n\n"
  eval $titlereset
fi
#------------------------------------
# Cycle thru data and build bar chart
#------------------------------------
for element in "${array[@]}"
do
# check data values
  IFS=',' read -r -a datapt <<< $element
  # check for empty records
  if [ ${#datapt[0]} = 0 ]; then 
    break
  fi
  label=${datapt[0]}
  if [[ ${label} != "-t*" ]]; then 
	  val=${datapt[1]}
	  sval=$(bc <<< "$width * $val / $maxval")

	  # add color, use 4th item if available
	  if [[ ${#datapt[@]} > 3 && $pretty = False ]]; then
		icolor=${datapt[3]}
	  fi
	  if [[ $pretty = True ]] ; then
		let colidx++
		if [ $colidx -gt 7 ]; then
		  let colidx=1
		fi
	  elif [[ ${#datapt[@]} > 3 ]]; then
		colidx=${datapt[3]}
	  else
	  	colidx=$color
	  fi
	  setcolors
          eval $color_set
          printf " %${maxlbl}s " "$label"
	  bar="printf '█%.0s' {1..$sval}"
	  eval $bar; 
	  # add value and units if available
	  units=""
	  if [[ ${#datapt[@]} > 2 ]]; then
		units=${datapt[2]}
	  fi
	  printf " %d %s\n\n" $val "$units"
	  eval $color_rs 
  fi
done

SQL Output to Bars

With the base code I was able to start doing some more complicated actions, like piping SQL SELECT output. Below is an example from Sqlite3. The first step is to format the SQL output to: label,value;

pete@lubuntu:$ sqlite3 $HOME/dbs/someuser.db "select fname,age from users limit 4"
Brooke|18
Leah|18
Pete|100
Fred|77
pete@lubuntu:$ sqlite3 $HOME/dbs/someuser.db "select fname,age from users limit 4" \
>  | tr  '|' ',' | tr '\n' ';'
Brooke,18;Leah,18;Pete,100;Fred,77;
pete@lubuntu:$ sqlite3 $HOME/dbs/someuser.db "select fname,age from users limit 4" \ > | tr '|' ',' | tr '\n' ';' | ./hbars -t "Sqlite3 Users" -p Sqlite3 Users Brooke █████████ 18 Leah █████████ 18 Pete ██████████████████████████████████████████████████ 100 Fred ██████████████████████████████████████ 77

Web Page with Bars

Support was added for fixed chart width, engineering units and custom line colours. HTML <center> tags were used on the title.

$ cat data.txt
temp,44,degC,1;
humidity,33,%,4;
air pressure,88,mm Hg,5;
rain/precipation,6,mm,6;

$ cat data.txt | ./hbars -W -f 24 -w 50 -t '<center>Raspi Data</center>' > data.htm

Final Comments

Horizontal bars are relatives easy to create in Bash. Unfortunately showing vertical bars and Line charts will require a different approach

TEMPer USB Temperature Sensor

I picked up a TEMPer USB temperature sensor from Walmart for about $12. My goal was to use it with my Home Assistant home automation system.

The model that I picked up supports Windows and it can be used directly in Excel. There is also integration with Home Assistant.

I found that the integration for Linux was tricky. This blog looks at how I got things working with a Raspberry Pi.

USB Connections

The lsusb command can be used to see which USB devices are connected:

pi@pi4:~ $ lsusb
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
Bus 001 Device 007: ID 413d:2107  
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

It is important to identify the ID model. The TEMPer USB model that I purchased was 413d:2107.

When I searched the Internet I found that people had created different packages but they were based on specific models. (Check your model).

temper.py

There is a Python library file that can be used on the TEMPer family of sensors. This library component is designed to be run directly as a Python file. It can be accessed by:

pip install temper-py
# Or get the file 
wget  https://raw.githubusercontent.com/ccwienk/temper/master/temper.py

Using the file directly will show the TEMPer device information (Note: use sudo):

$ sudo python3 temper.py
Bus 001 Dev 007 413d:2107 TEMPerGold_V3.1 29.31C 84.76F - - - -

$ sudo python3 temper.py --v
Firmware query: b'0186ff0100000000'
Firmware value: b'54454d506572476f6c645f56332e3120' TEMPerGold_V3.1 
Data value: b'80800b6d4e200000'
Bus 001 Dev 007 413d:2107 TEMPerGold_V3.1 29.25C 84.65F - - - -

$ sudo python3 temper.py --json
[
    {
        "vendorid": 16701,
        "productid": 8455,
        "manufacturer": "",
        "product": "",
        "busnum": 1,
        "devnum": 7,
        "devices": [
            "hidraw0",
            "hidraw1"
        ],
        "port": "1-1.4",
        "firmware": "TEMPerGold_V3.1",
        "hex_firmware": "54454d506572476f6c645f56332e3120",
        "hex_data": "80800b6d4e200000",
        "internal temperature": 29.25
    }
]

I found that the Python app was useful and it could be used on a variety of sensors.

Human Interface Devices

The TEMPer sensor acts like a Human Interface Device. When the TEMPer sensor is connected the dmesg utility can be used to show kernel information:

 $ dmesg -H
[May 1 21:22] usb 1-1.4: USB disconnect, device number 6
[May 1 21:23] usb 1-1.4: new full-speed USB device number 7 using xhci_hcd
[  +0.135008] usb 1-1.4: New USB device found, idVendor=413d, idProduct=2107, bcdDevice= 0.00
[  +0.000020] usb 1-1.4: New USB device strings: Mfr=0, Product=0, SerialNumber=0
[  +0.011366] input: HID 413d:2107 as /devices/platform/scb/fd500000.pcie/pci0000:00/0000:00:
[  +0.064445] hid-generic 0003:413D:2107.0009: input,hidraw0: USB HID v1.11 Keyboard [HID 413
[  +0.007181] hid-generic 0003:413D:2107.000A: hiddev96,hidraw1: USB HID v1.10 Device [HID 41
(END)

From this output I can see that TEMPer sensor is on the raw human interface device of hidraw1. (For this example the hidraw0 acts like a keyboard but it can’t be used to get temperature info). When I move the TEMPer sensor to a different PC I need to run dmesg again to recheck the hidraw device number, it could be 2,3,4 etc.

A Bash Script

There are some excellent links that got me about 95% of the way. The Bash script below will return a temperature value for my series of TEMPer USB thermometers.

#!/bin/bash
# 
# Get the temperature from a USB Temper Thermometer
#   
#   find the HID device from the kernel msg via dmesg
#   parse the line get HID device
hidstr=$(dmesg | grep -E  -m 1 'Device.*413d:2107')
# find the postion of the "hidraw" string
hidpos=$(echo $hidstr | grep -b -o hidraw  | awk 'BEGIN {FS=":"}{print $1}')

if [ $hidpos  = "" ]; then
  echo "No TEMPer device found"
else
  #get the hidraw device from the string, Note: offset of 1
  hidpos1=$((hidpos +1))
  hid=$(echo "/dev/${hidstr:hidpos1:7}")
  exec 5<> $hid
  # send out query msg
  echo -e '\x00\x01\x80\x33\x01\x00\x00\x00\x00\c' >&5
  # get binary response
  OUT=$(dd count=1 bs=8 <&5 2>/dev/null | xxd -p)
  # characters 5-8 is the temp in hex x1000
  HEX4=${OUT:4:4}
  DVAL=$((16#$HEX4))
  CTEMP=$(bc <<< "scale=2; $DVAL/100")
  echo $CTEMP 
fi

If you are always using the TEMPer in only one place you could simplify the script by hard coding the HID number. I wanted to move the sensor between devices so I added a check find the device.

Control USB Powered Devices

For home automation projects a Raspberry Pi offers a simple low cost approach for controlling a wide variety of devices. Typically these devices are either digitally wiring 0-5Volt devices like motion detectors, or wireless Ethernet devices like smart plugs, but a Raspberry Pi can also control USB powered devices, like USB fan and lights.

In this blog I will look at how to control and monitor Raspberry Pi USB port with two projects. The first project will use Node-Red to create a web dashboard to control USB lights. The second project will turn on USB cooling fans based on the PI’s CPU temperature.

Controlling USB Ports

There are a number of techniques to control USB ports, I found that one of easiest approach is to use the uhubctl utility. To load this utility:

sudo apt-get install libusb-1.0-0-dev
git clone https://github.com/mvp/uhubctl
cd uhubctl
make
sudo make install

The uhubctl utility can be used to view and control USB ports and ports smart USB hubs.

The lsusb command can be used to show connected USB devices

pi@raspberrypi:~ $ lsusb 
Bus 001 Device 003: ID 0424:ec00 Standard Microsystems Corp. Fast Ethernet Adapter
Bus 001 Device 002: ID 0424:9514 Standard Microsystems Corp. SMC9514 Hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

For the Raspberry Pi 3 and 4 the power on all USB ports is ganged together through port 2, so unfortunately it is not possible to power up/down an individual USB port.

The Pi 4 has two internal USB hubs. Hub 1-1 connects to all the USB port on the USB 2.10 standard. Hub 2 controls to all the ports on the USB 3.00 standard and the Ethernet jack.

The commands to turn on/off/toggle the USB ports and keep the Ethernet jack powered are:

sudo uhubctl -l 1-1 -p 2 -a on
sudo uhubctl -l 1-1 -p 2 -a off
sudo uhubctl -l 1-1 -p 2 -a toggle

The command will return the messages showing the current status, the power requested state and then the new status. To just do an action and remove feedback messages add a 1>&- at the end of the line.

Monitoring USB Power

The uhubctl utility can be used to check if the USB ports are powered. For Raspberry Pi’s the status of Port 2 is the power status for all the ports.

Using some Bash statements the power status can be parsed to just show off or power message. In the next example this Bash statement will be used to show the power status on a Node Red dashboard.

$ sudo uhubctl | grep 'Port 2' | awk '{print $4}'
off

Node Red USB Control Dashboard

Node Red is a visual programming tool that is included with the full desktop install of Raspberry Pi. If Node Red has not been installed see: https://nodered.org/docs/getting-started/raspberrypi

There are number of low cost USB lighting options that can be used with a Raspberry Pi, these include LED strips, eWire and small USB lights. Below is an example of littleBits eWire bit and a USB LED light.

A simple Node Red dashboard can be created that can: 1) turn Raspberry Pi USB ports on and off, and 2) checks the status of power on these ports. The logic would include: 2 dashboard buttons, one dashboard text and 2 exec nodes. The uhubctl utility can be used directly in the exec node.

The first exec node contains the Bash command to turn the USB ports on or off. The “on” or “off” string is sent from the dashboard buttons as a msg.payload this is appended to the command in the exec node. The output from the first exec node triggers the second exec node to get the latest USB port status.

The dashboard text node is configured to show the power status text as a large uppercase heading.

Once the logic is complete, the Deploy button on the right side of the menu bar will make the dashboard available to web clients at: https://raspberry_pi_address:1880/ui .

For my example I added an enhancement to include a countdown or sleep timer.

Raspberry Pi Cooling Fan

Raspberry Pi’s have a number of different cooling options. For this example I used a two littleBits fans that I placed on a littleBits mounting plate.

The vcgencmd measure_temp command will return the Pi CPU temperature. By adding some awk script it’s possible to parse out just the float value:

$ vcgencmd measure_temp 
temp=46.2'C
$ vcgencmd measure_temp | awk '{ print substr($1,6,4)}'
45.6

The bc (arbitrary precision calculator) command can be used with the math library (-l option) to check if one float is greater than another float number:

$ echo "33.4 > 36.1" | bc -l
0
$ echo "38.4 > 36.1" | bc -l
1

A simple script to loop through and check the temperature against a limit, and then turn on/off a fan would be:

#!/bin/bash
#
# Check the Pi temperature and turn on fan if too high
# 
tlim="46.0"
while :;
do
  tnow=$(vcgencmd measure_temp | awk '{ print substr($1,6,4)}')
  # check the CPU temp vs. the limit
  if (( $(echo "$tnow > $tlim" | bc -l ) )) ; then
     # CPU temp is above limit, turn on fan
     sudo uhubctl -l 1-1 -p 2 -a on 1>&-
  else
     # CPU temp is below limit, turn off fan
     sudo uhubctl -l 1-1 -p 2 -a off 1>&-
  fi
  sleep 10
done

This code can be enhanced to show when the fan needs to be turned on and off:

#!/bin/bash
#
# Check the Pi temperature and turn on fan if too high
#

# Start with Fan on
tlim="46.0"
fan="on"
sudo uhubctl -l 1-1 -p 2 -a $fan 1>&-

# Periodically check CPU temperature
while :;
do
  tnow=$(vcgencmd measure_temp | awk '{ print substr($1,6,4)}')

  if (( $(echo "$tnow > $tlim" | bc -l ) )) ; then
      if [ $fan = "off" ] ; then
    	fan="on"
	    echo "$(date +'%T')  Temp: $tnow - Turn fan $fan"
        sudo uhubctl -l 1-1 -p 2 -a $fan 1>&-
      fi
  else
      if [ $fan = "on" ] ; then
    	fan="off"
	    echo "$(date +'%T')  Temp: $tnow - Turn fan $fan"
        sudo uhubctl -l 1-1 -p 2 -a $fan 1>&-
      fi
  fi
  sleep 10
done

Power Control on other Controllers

A Raspberry Pi can control the power on other controllers, below is an example of a Pi 4 powering an Arduino UNO, an Arduino Nano (clone) and a BBC Micro:bit controller.

For external modules that don’t support Wifi or real time clocks a Raspberry Pi could be used as an easy way to power up/power down these external controllers.

It’s important to realize that a Raspberry Pi is not designed to power devices that have a high power requirement.

The Raspberry Pi 3 and 4 can have a maximum USB port output of 1200mA for all 4 ports combined, with no per-port limits (meaning, all 1200mA is available on a single port if no others are in use). This 1200mA limit assumes that the power is from a 2.5A power supply for the Pi3 or the 3A power supply for the Pi4.

If you connecting smart USB devices like devices like memory sticks or 3rd party controller, the device manufacture has a defined MaxPower rating that can be found once the device is connected.

The command lsusb -v will give a very long list of the full vendor information for all the connected devices. To get just the MaxPower for each device on the Raspberry Pi USB internal bus enter:

sudo lsusb -v  2>&- | grep -E  'Bus 00|MaxPower'

When this command is run with an Arduino Nano, Arduino UNO and a BBC Micro:bit and a memory stick, the total power requirements can be seen.

pi@pi4:~ $ sudo lsusb -v  2>&- | grep -E  'Bus 00|MaxPower'
Bus 002 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
    MaxPower                0mA
Bus 001 Device 004: ID 1a86:7523 QinHeng Electronics HL-340 USB-Serial adapter
    MaxPower               96mA
Bus 001 Device 003: ID 2341:0043 Arduino SA Uno R3 (CDC ACM)
    MaxPower              100mA
Bus 001 Device 002: ID 2109:3431 VIA Labs, Inc. Hub
    MaxPower              100mA
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
    MaxPower                0mA

With this example the total used USB power is 296mA (0 +96+100+100+0), and this is within the Raspberry Pi specs.

A Bash command to calculate the total required bus power would be:

pi@pi4:~ $ (sudo lsusb -v 2>&- | grep MaxPower | grep -o -E '[0-9]+' ) | awk '{ sum += $1} END {print "\nTotal= " sum " mA"}' 

Total= 296 mA

Unfortunately devices like USB lights use the USB connection for power only so they do not appear in lsusb, so you’ll need to check the manufacturers literature for power requirements.

Final Comments

I prefer using direct wired GPIO pin connections or Wifi devices over using USB powered devices, however it’s nice to know that you have the USB option if you need it.

For kids projects that use littleBits or Micro:bits using a Pi with Node Red offers a nice way to remotely control them.