Bash with MQTT

I’m working on re-purposing an old router. One of my goals is to bring sensor and performance data from the router to a Home Assistant node. The router doesn’t not have a lot of space so I’d prefer to use Bash rather than Python for MQTT communications.

While I was working on the project I wanted to use some simple tools to view the data, unfortunately I wasn’t able to find any information on how to make a good MQTT client in Bash.

This blog documents how I used Bash to show bar charts of MQTT data.

Setup

The first step is to install the Mosquitto command line tools on the OpenWrt router:

# Update the package manager list
opkg update
# Install the MQTT client utiliy
opkg install mosquitto-client-nossl

When I have some time I’d like to look at making some more MQTT Bash scripts that could be used with file input.

The next step is to install the Mosquitto client on my Linux PC and Raspberry Pi:

sudo apt-get install mosquitto-clients

There are a number of MQTT brokers that can be used, both Home Assistant and Node Red have reliable brokers. The Mosquitto broker can also be loaded on a Linux, MacOS and Windows node.

Publish MQTT Data in Bash

The Mosquitto client can be used to both publish and subscribe to MQTT data.

For my router project, I wanted the: temperature (from a USB thermometer), idle time, % used space and available space. Below is the script that passed the data to the mosquitto_pub (publishing) tool:

#!/usr/ash
#
# mqtt_data.sh - send data to MQTT broker
#
# Get Data values
temp=$(/usbstick/temper.sh)
idle=$(vmstat | awk '{ if (NR==3) print $15}')
used=$(df | awk '{if (NR==4) printf "%.f\n", $5 }')
space=$(df | awk '{if (NR==4) printf "%.1f\n", $4/1000 }')

echo "$temp $idle $used $space"

# Publish Data
server="192.168.0.111"
pause=2

mosquitto_pub -h $server -t rtr_temp -m $temp
sleep $pause
mosquitto_pub -h $server -t rtr_idle -m $idle
sleep $pause
mosquitto_pub -h $server -t rtr_used -m $used
sleep $pause
mosquitto_pub -h $server -t rtr_space -m $space

I added a pause between each publish to let me watch the actions.

The cron utility can be used to schedule the running of the script. Once a minute is the fastest time available with cron, so if faster times are needed the script could cycle with a while loop.

Read/Subscribe to MQTT Data

The mosquitto_sub client tool can be used to read or subscribe to the data.

To look at the router points:

$ service="192.168.0.111"
$ mosquitto_sub -h $server -v -t rtr_idle -t rtr_temp -t rtr_used -t rtr_space

rtr_temp 26.25
rtr_idle 100
rtr_used 66
rtr_space 1.1

The -v (–verbose) option show output with the topic names and the values. Topics, -t option, can be put on the command line or passing in as a file.

For single point monitoring the Zenity utility can be used. This utility is typically preloaded on most Linux and Rasp Pi systems. A script to create a progress bar dialog with MQTT data is:

# Send MQTT values to a progress bar
( 
  while :; do
  msg=$(mosquitto_sub -h 192.168.0.111 -C 1 -t rtr_temp) 
  echo $msg
  echo "#$msg  Deg C"
  done
  ) | zenity --progress  --title="Router External Temperature"

Multipoint Progress Dialogs

The Zenity utility can only manage single point progress bars. For multiple point progress bars the YAD (Yet Another Dialog) package can be used.

To install YAD on Raspberry Pi’s and Ubuntu: sudo apt-get install yad

Below is some script that will show multiple points in a YAD dialog:

#!/usr/bash
#
# mqtt_bars.sh - Show multiple MQTT Topics on a Dialog
#
server="192.168.0.111"
topics=("rtr_idle" "rtr_temp" "rtr_used" )
scale=(100 40 100 )
units=( '%' 'degC' '%'  )
title="Router MQTT Points"

#Build topic and yad strings
yadstr=" "
for i in ${topics[@]}; do
  topstr="$topstr -t $i"
  yadstr="$yadstr --bar=$i"
done

echo "Press [CTRL+C] to stop..."
# Cycle thru 1 message at a time to YAD 
(
while : 
do 
  msg=$(mosquitto_sub -h $server -v -C 1 $topstr) 
  IFS=' ' read -a data <<< "$msg" 
  # match returned msg to order of bars, write value/label
  for i in "${!topics[@]} "
  do 
    if [ "${data[0]}" = "${topics[i]}" ]
    then 
      let j=i+1 ; # YAD indices start at 1
      # Rescale bar to defined scale
      barsize=$(bc <<< "${data[1]}*100/${scale[i]}")
      #barsize=100
      echo "$j:$barsize"
      echo  "$j:#${data[1]} ${units[i]}"
  fi
  done
done 
)  | yad --multi-progress $yadstr --title $title

Final Comments

There are probably lots of good 3rd party tools like Gnuplot that could be connected to mosquitto_sub to show real charts.

Zenity: Command line Dialogs

Zenity is command line GUI creator that has been around since 2012, and it is pre-installed on most versions of Linux, including Raspberry PI’s. Zenity is also available for MacOS and Windows.

I came across it on a recent project and I wanted to document some of my code. My examples include:

  • CPU stats on a dialog – 1 line of Bash script
  • Show a web page in a dialog – 1 line
  • Create a 4 button PI Rover control – ~ 25 lines
  • Dynamic bar of CPU core temperature – 7 lines
  • Show CSV or SQL data in a list dialog – 1 line
  • Form to insert user data in an SQL database – ~7 lines

What is Zenity?

Zenity is a command line dialog creator. I found it pretty quick to pick up and it works well for simple Bash scripts. Zenity supports:

  • Basic forms
  • Calendar dialogs
  • Color selection dialogs
  • File Selection Dialogs
  • List Dialog
  • Message and Notification Dialog
  • Progress bars and Scales
  • Text Entry and Text Information Dialogs

Message Dialogs

Message dialogs can be created for errors, info, questions or warnings. The difference is the icon that shows up (and Ok/Cancel for the question dialog).

The Bash code to get the instantaneous CPU idle time would be:

pi@raspberrypi:~ $ # Run top once and look for the line with %Cpu
pi@raspberrypi:~ $ top -n 1 | grep Cpu
%Cpu(s):  7.4 us,  3.6 sy,  0.0 ni, 88.8 id,  0.3 wa,  0.0 hi,  0.0 si,  0.0 st

pi@raspberrypi:~ $ # Get the 8th item from the grep

pi@raspberrypi:~ $ top -n 1 | grep %Cpu | awk '{print $8}'
88.8

The bash code to show the CPU idle time in a info dialog is:

zenity --info --text=$(top -n 1 | grep %Cpu | awk '{print $8}') --title="CPU Idle Time"

Message Dialogs with Custom Font

Text font and size can be modified in message dialogs, using the Pango Markup Language syntax. Pongo is similar to HTML. The <span></span> set of tags is used to encode font and color definitions, For example:

zenity --warning --text='<span font="32" foreground="red">HIGH Temperature</span>' --title="HDD Check"

Unfortunately only Message Dialog texts can changed, so text in dialogs like list, scale, and progress can’t not have their fonts changed.

Web Pages in a Text-Info Dialog

Text or HTML files can be passed to a text-info dialog:

zenity --text-info --title="Background Reading" --html --url="https://developer.gnome.org"

A checkbox can be added, and the user feedback can be read by the bash script:

#!/bin/sh
# show web page in a dialog with a next step action
#
theurl="https://developer.gnome.org"

zenity --text-info --title="Background Reading" --html --url=$theurl \
       --checkbox="I read it...and I'm good to go"
rc=$?
echo $rc
case $rc in
    0)
        echo "Start some next step"
	# next step
	;;
    1)
        echo "Stop installation!"
	;;
   -1)
        echo "An unexpected error has occurred."
	;;
esac

It is important to note that the text-info dialog should be used for simple web pages. There is no Javascript support and web links will launch the default web browser with requested page.

Refreshing Message Dialogs

Of all the Zenity dialogs, only the Progress dialog supports a method to update text on an open dialog. A workaround is to use the –timeout option to close the dialog and then redisplay it with the new data.

rc=5
while [[ $rc -eq 5 ]];
do 
  zenity --info --text=$(date +'%S' ) \
  --title="Seconds Timer Test" --timeout=5 --ok-label Quit $ zenity --info \
  --text=$(date +'%S' )   --title="Seconds Timer Test" 2>/dev/null

  rc=$?	
  echo $rc
done 

This “timeout and redraw” method is ugly because the window always positions in the middle of the screen and this can be quite annoying. Unfortunately Zenity does not support any top/left positioning options.

The xdotool could be used find the zenity window id and then position it, but this would need to be done in another script. (It can’t be done in the same script because the zenity line doesn’t complete until either it times out or OK is pressed). The xdotool script would be:

pid=$(xdotool search -onlyvisible -name myzenitywindownname)
xdotool windowmove $pid 0 0

If you need dynamically updated text on a dialog I think that it would be best to use another tool, (Python, YAD etc.).

Info Dialog with Extra Buttons – Pi Rover Controls

It’s possible to add some extra button to an info dialog. Below is an example where a Raspberry Pi Rover is controlled with a zenity multi-button info dialog. Note: the pins will vary with your setupArduinoIt’s possible to add some extra button to an info dialog. Below is an example where a Raspberry Pi Rover is controlled with a zenity multi-button info dialog. Note: the pins will vary with your setup


#!/bin/bash
#
# rover.sh - Rover Controls with Multiple Button Dialog
# Define GPIO pins for the motors motorL=7 motorR=11 rc=1 # OK button return code =0 , all others =1 while [ $rc -eq 1 ]; do ans=$(zenity --info --title 'Drive a Rover' \ --text 'Motor Action' \ --ok-label Quit \ --extra-button FORWARD \ --extra-button STOP \ --extra-button LEFT \ --extra-button RIGHT \ ) rc=$? echo "${rc}-${ans}" echo $ans if [[ $ans = "FORWARD" ]] then echo "Running the Rover" gpio -1 write $motorL 1 ; gpio -1 write $motorR 1; elif [[ $ans = "STOP" ]] then echo "Stopping the Rover" gpio -1 write $motorL 0 ; gpio -1 write $motorR 0; elif [[ $ans = "LEFT" ]] then echo "Rover turning Left" gpio -1 write $motorL 1 ; gpio -1 write $motorR 0; elif [[ $ans = "RIGHT" ]] then echo "Rover turning RIGHT" gpio -1 write $motorL 0 ; gpio -1 write $motorR 1; fi done

The script can be run by: bash rover.sh

Below is the dialog and the rover.

Progress Bars – Show Dynamic Values

A Zenity progress dialog can show dynamic updates with scripts that define steps using sleep statements. When the step outputs a value the process bar is updated. The text on the progress dialog is changed by outputting a text string starting with a # character.

A 3-step example would be:

(
echo "33"; echo "# 1/3 done" ; sleep 5; \
echo "66"; echo "# 2/3 done" ; sleep 5; \
echo "100";echo "# Finished"  \
) | zenity --progress --title="3 step test"

The progress dialog can use a bash for or while statement. The progress dialog can be passed both new text and the value. A text string starting with # is interpreted as the new text. A number string is interpreted as the progress bar value.

Below is an example where a value is counted from 1 to 100:

( for i in `seq 1 100`; do echo $i; echo "# $i";  sleep 1; done ) | zenity --progress

The next thing I tested is a dialog that runs indefinitely (or until you hit “Control-C”). It’s import to note that the progress bar is from 0-100, so scaling your value may be required. An example of scaling a time from 0-60 would be:

echo "$(date +'%S')*100/60" | bc

An example to show seconds in a dialog would be:

#!/bin/bash
# show_sec.sh - progress dialog to show seconds
echo "Press [CTRL+C] to stop..." 
( 
  while :; do 
  echo "# $(date +'%S')" 
  # Scale 0-60 to 0-100 
  echo "$(date +'%S')*100/60" | bc
  sleep 1 
  done 
  ) | zenity --progress  --title="Show Time in Seconds"

To run this script: bash show_sec.sh

A more useful dialog would be to show the CPU temperature:

#!/bin/sh 
# show_cpu_temp.sh - Progress Dialog to show CPU temperature
# 
echo
 "Press [CTRL+C] to stop..."
(
while :; do 
  echo "# $(sensors | grep CPU)" 
  sensors | grep CPU | awk '{print substr($2,2,4) }' 
  sleep 5 
done ) | zenity --progress --title="CPU Temperature"  

List Dialog – Show CSV/SQL Data

If you are working with a simple known data set then the List Dialog might be a good fit.

The List Dialog expects the data to be a sequential list, so a 2 column example of static data would be:

zenity --list \
  --title="2 Column Example" \ 
  --column="Month" --column="Sales" \
   Jan 100 Feb 95 Mar 77 Apr 110 May 111

Text and CSV files can also be used in Zenity lists. The first step is to convert the file into a single column of data. This can be done with the tr statement. For the example below the comma (,) is replaced with a newline (\n) character:

$ cat lang.txt
Brazil,Brasilia,Portuguese
England,London,English
France,Paris,French
Germany,Berlin,German

$ cat lang.txt | tr ',' '\n'
Brazil
Brasilia
Portuguese
England
London
English
France
Paris
French
Germany
Berlin
German

Now the sequential data can be passed into a Zenity list:

cat lang.txt | tr ',' '\n' | zenity --list \
  --title="Country Info" \
  --column="Country" --column="Capital" --column="Language"

Once you have some Zenity and Bash basics down you can some fairly advanced operations. Below is a 1-line example that uses awk to parse out specific fields (1 and 3) and then the user selected output is echo-ed.

awk -F "\"*,\"*" '{print $1 "\n" $3}' pidata.csv  | \
  echo $(zenity --list --column="field1" --column="field3" --print-column=ALL)

Similarly to data from an SQL query can be show. Almost all SQL servers have a command line interface. The interface will vary from database to database, but an example with Sqlite would be:

(sqlite3 someuser.db "select fname,lname,age,job from users" ) | tr '|' '\n' | zenity --list \
  --title="My Database" \
  --column="first name" --column="last name" --column=age --column=job

Form Dialog – Insert SQL Data

The Forms Dialog allows for date, text and password inputs, and the result are passed as string (| is the default separator). A form example with output would be:

$ row=$(zenity --forms --title="Create user" --text="Add new user" \
   --add-entry="First Name" \    
   --add-entry="Last Name" \    
   --add-entry="Age" \    
   --add-entry="Job") ; echo $row

field1|field2|field3|field4

The next step is to format the form data into an SQL statement. The SQL INSERT syntax is:

INSERT INTO table (field1,field2…) VALUE (value1,value2…)

For the example above, field1|field2|field3|field4 needs to formatted to the values. This manipulation can be done by the bash sed command with search and replace (s) option:

$ row="field1|field2|field3|field4"
$ echo "'$row'" | sed "s/|/','/g"
'field1','field2','field3','field4'

The bash script to present the zenity form and input the data is below. The if statement is used to ensure that the cancel button wasn’t pressed. More if statement would probably be required for some data validation.

# zen_sqlin.sh - create a form to add a new user into a SQLite3 database
row=$(zenity --forms --title="Create user" --text="Add new user" \ --add-entry="First Name" \ --add-entry="Last Name" \ --add-entry="Age" \ --add-entry="Job") if [[ -n $row ]] # Some data found then indata=$(echo "'$row'" | sed "s/|/','/g") cmd="sqlite3 someuser.db \"INSERT INTO users (Fname,Lname,Age,Job) VALUES ($indata)\"" eval $cmd echo "Added data: $indata" fi

The script is run by: bash zen_sqlin.sh

Zenity (GTk) Warning Messages

Depending on your system you might see some Zenity warning messages such as:

Gtk-Message: 15:30:52.461: GtkDialog mapped without a transient parent. This is discouraged.

I never saw this on my Raspberry Pi but I did see it on my lubuntu system. To make things cleaner the warning can be piped to the null device:

$ zenity --info --text=$(date +'%S' )   --title="Seconds Timer Test" 2>/dev/null

Some Final Comments

I really just touched the surface on what zenity can do. For more info see some of the tutorials.

For simple stuff zenity works fine. If you’re looking for a more complete command line GUI tool try YAD, for myself I’ll stick to Python.

As a side note, there is a Python library for zenity. If you’re feeling comfortable with the bash version of zenity and you only need to do simple dialogs then this might be a good fit.