Bash Bar Charts for Text and Web Pages

This blog documents my notes on creating Bash text bar charts. The key topics are:

  • Create a simple horizontal bars (3 lines)
  • Create dynamic bars (~25 lines for an array of data values)
  • Add support for piped and command line data
  • Include options for: help, resizing, titles
  • Add colour and HTML support

A Horizontal Bar

To create a static bar an printf statements can be used. The seq {0..10} can be used to repeat an ASCII █ fill character 10 times.

$ printf 'LABEL: ' ; \
  printf '█%.0s' {1..10} ; \
  printf ' 10\n'

LABEL: ████████████████████ 10

Unfortunately the printf statement has some limitations on variable substitution. A simple workaround is to create a string and then eval it:

$ label="temp"; val=20;
$ bar="printf '█%.0s' {1..$val}" ; 
$ printf '\n%-5s ' $label; eval $bar ; printf ' %d\n' $val

temp  ████████████████████ 20

Coloured Bars

The tput setaf command can change the foreground, and tput setab is used for background colours. Colour codes are:

tput setab [1-7] # Set the background colour using ANSI escape
tput setaf [1-7] # Set the foreground colour using ANSI escape

Num  Colour    #define         R G B

0    black     COLOR_BLACK     0,0,0
1    red       COLOR_RED       1,0,0
2    green     COLOR_GREEN     0,1,0
3    yellow    COLOR_YELLOW    1,1,0
4    blue      COLOR_BLUE      0,0,1
5    magenta   COLOR_MAGENTA   1,0,1
6    cyan      COLOR_CYAN      0,1,1
7    white     COLOR_WHITE     1,1,1

To reset colours back to the defaults use: tput sgr0

An example to print a red bar and a stack of bars:

$ printf '\nLABEL: ' ; \
   tput setaf 1 ;\
   printf '█%.0s' {1..10} ; \
   printf ' 10\n'

LABEL: ██████████ 10
$ printf '\n 3 Stacked Bars: ' ; \
   tput setaf 1 ;\
   printf '█%.0s' {1..10} ; \
   tput setaf 2 ;\
   printf '█%.0s' {1..8} ; \
   tput setaf 4 ;\
   printf '█%.0s' {1..3} ; \
   printf ' 10+8+3=21\n'

 3 Stacked Bars: █████████████████████ 10+8+3=21

Dynamic Bars

The next step is to create a script that dynamically updates the bars. The tput clear command will clear the terminal screen keep the data and bars in the same location. The script below will dynamically show the CPU temperature, idle time and 2 random values with a 10 second update time.

#!/bin/bash
# 
# cpu_bars.sh - Show new data every 10 seconds
#
while :; do
    # Get data values
    CPUtemp=$(sensors | grep CPU | awk '{print substr($2,2,4)}')
    CPUidle=$(iostat | awk '{if (NR==4) print $6}')
    Random1=$((1+ $RANDOM % 100))
    Random2=$((1+ $RANDOM % 100))

    labels=( CPUtemp CPUidle Random1 Random2)
    values=( $CPUtemp $CPUidle $Random1 $Random2)
    units=( degC % psi mm)

    # Show a title
    tput clear
    printf " %10s " "" 
    tput setaf 7; tput smul;
    printf "%s\n\n" "Show CPU Data ($(date +%T'))"
    tput rmul;

    # cycle thru data and show a label, 
    for index in "${!labels[@]}"
    do
          tput setaf $(expr $index + 1); # don't use 0 (black) 
          printf " %10s " "${labels[index]}"
          eval "printf '█%.0s' {1..${values[index]%.*}}"
          printf " %s %s\n\n" ${values[index]} ${units[index]}
    done
    sleep 10
done

This script is run by: bash cpu_bars.sh .Typical output is below.

Bars in HTML

The ANSI colours are not supported in HTML, so instead HTML tags and style properties can be used.

It is important to use <pre> tags for Bash text output. Code to create two static bars in HTML would be:

$ (printf "<h1>A Bar from Bash</h1>\n" 
 printf "<pre><span style='font-size:24px;color:red'}>\n"
 printf 'LABEL1: ' ; printf '█%.0s' {1..10} ; printf ' 10\n'
 printf "</pre></span>\n") > bar1.htm

$ cat bar1.htm
<h1>A Bar from Bash</h1>
<pre><span style='font-size:24px;color:red'}>
LABEL1: ██████████ 10
</pre></span>

The script cpu_webbars.sh creates HTML output for an array of calculated values:

#!/bin/bash
# 
# cpu_webbars.sh - Show bars in HTML
#

# Get data values
CPUtemp=$(sensors | grep CPU | awk '{print substr($2,2,4)}')
CPUidle=$(iostat | awk '{if (NR==4) print $6}')
Random1=$((1+ $RANDOM % 100))
Random2=$((1+ $RANDOM % 100))

labels=( CPUtemp CPUidle Random1 Random2)
values=( $CPUtemp $CPUidle $Random1 $Random2)
units=( degC % psi mm)
colors=(red blue green magenta)

# Show a title
printf "<h1><center>Show CPU Data ($(date '+%T'))</center></h1>\n"
# cycle thru data and show a label, 
for index in "${!labels[@]}"
  do
  printf "<pre><span style='font-size:18px;color: ${colors[index]} '}>\n"
  printf " %10s " "${labels[index]}"
  eval "printf '█%.0s' {1..${values[index]%.*}}"
  printf " %s %s\n\n" ${values[index]} ${units[index]}
  printf "</pre></span>\n"
done

This script can be run and outputted to an file: bash cpu_webbars.sh > test.htm

Once the basic HTML output is working its possible to add headers and footers to make a more complete page:

header.htm > test.htm ; \
cat cpu_webbars.sh >> test.htm ; \
cat footer >> test.htm 

Piping Data to Bars

A script (hbar0.sh) will read the piped data and then create an array (data) of labels and values. The data array is cycled through and labels and bars are shown with a randomized colour:

#!/bin/bash
# hbar0.sh - Read in piped data  and plot bars
#     format: label,value;label2,value2;  and plot bars
#
input=$(< /dev/stdin) ; # read piped data
# remove new lines in files, and check for ";" after data pair 
input=$(echo $input | tr -d '\n')
IFS=';' read -r -a data <<< $input
printf "\n" 
for element in "${data[@]}"pete@lubuntu:~/Writing/Blog/text_bars
do 
  # make at array of each data element
  IFS=',' read -r -a datapt <<< $element
  # add a random color
  tput setaf $((1+ $RANDOM % 7))
  # print the label, bar and value
  printf " %10s " "${datapt##*[0]}"
  bar="printf '█%.0s' {1..${datapt[1]}}"
  eval $bar
  printf " %s\n\n" ${datapt[1]} 
  tput rmso ; # exit color mode   
done

The script can be tested with piped data:

$ echo "temp,33;pressure,44" | bash hbar0.sh

 temp       █████████████████████████████████ 33 

 pressure   ████████████████████████████████████████████ 44 

A data file can also be passed in using the cat command:

$ cat data0.txt 
temp,44;
humidity,33;
pressure,15;
wind spd,33;
wave ht,3;
$ cat data0.txt | bash hbar0.sh

       temp ████████████████████████████████████████████ 44

   humidity █████████████████████████████████ 33

   pressure ███████████████ 15

   wind spd █████████████████████████████████ 33

    wave ht ███ 3

Removing ANSI Colors

Terminal applications use ANSI color codes which unfortunately is not support on Web pages.

ANSI color codes can be removed from files and strings by:

# Strip out ANSI color codes:
cat infile | sed 's/\x1b\[[0-9;]*m//g' > outfile
$ echo "temp,33;pressure,44" | bash hbar0.sh > hbar0.txt
$ cat hbar0.txt

       temp █████████████████████████████████ 33

   pressure ████████████████████████████████████████████ 44

$ cat hbar0.txt | sed 's/\x1b\[[0-9;]*m//g'

       temp █████████████████████████████████ 33

   pressure ████████████████████████████████████████████ 44

A Final App

With the basics in place I was able to create an app that would support scaling, titles, units, custom colour and web output:

$ ./hbars
usage: hbars [data] [option] 
  -h --help     print this usage and exit
  -c --color    set color to all bars (default 7=white)
                 (0-grey,1-red,2=green,3=yellow,4=blue,5=magenta,6=cyan,7=white)
  -p --pretty   add different colors to bars (-c overrides)
  -t --title    top title
  -w --width    max width of bar (default 50)
  -W --Web      make output HTML formatted
  -f --fontsize fontsize for web output (default 24)

 examples:
   echo 'temp,33,C;pressure,14,psi' | ./hbars -t Weather -p -w 40 
   ./hbars -t Weather -p -w 40  'temp,33;pressure,14' 
   cat data.txt | ./hbars -W -f 24 -t 'Raspi Data' > data.htm

The code:

#!/bin/bash
#
# hbars.sh - show some text bars
#   pass data as:  label1,value1,unit1,color1; label2,value2,unit2,colour2; ....  
#
width=50
title=""
color=7
pretty=False
web=False
font=24

usage() { 
  echo "usage: hbars [data] [option] "
  echo "  -h --help     print this usage and exit"
  echo "  -c --color    set color to all bars (default 7=white)"
  echo "                 (0-grey,1-red,2=green,3=yellow,4=blue,5=magenta,6=cyan,7=white)"
  echo "  -p --pretty   add different colors to bars (-c overrides)"
  echo "  -t --title    top title"
  echo "  -w --width    max width of bar (default 50)"
  echo "  -W --Web      make output HTML formatted"
  echo "  -f --fontsize fontsize for web output (default 24)"
  echo ""
  echo " examples:"
  echo "   echo 'temp,33,C;pressure,14,psi' | ./hbars -t Weather -p -w 40 "
  echo "   ./hbars -t Weather -p -w 40  'temp,33;pressure,14' "
  echo "   cat data.txt | ./hbars -W -f 24 -t 'Raspi Data' > data.htm"
  echo ""

  exit 0
}
# Show help usage if no pipe data and no cmd line data
if [ -t 0 ]  && [ $# -eq 0 ] ; then
  usage
fi
# Check for command line options
while getopts "hpc:t:w:Wf:" arg; do
  case "$arg" in
    h) usage ;;
    c)  color=$OPTARG ;;
    p)  pretty=True; icolor=0 ;;
    t)  title=$OPTARG ;;
    w)  width=$OPTARG ;;
    W)  web=True;;
    f)  font=$OPTARG ;;
  esac
done
#------------------------------------------------
# Setup formatting for text, Web and color
# -----------------------------------------------
if [[ ${color} != 7 && ${pretty} = True ]]; then
  pretty=False
fi
colidx=0

setcolors() {
if [ $web = True ]; then
  colors=(gray red green yellow blue magenta cyan white)
  titlebold="echo '<h1>'"
  titlereset="echo '</h1>'"
  #color_set='echo "<span style=font-size:$(font)px  >" ' 
  #color_set="printf '<span  style=\"font-size:$(font)px;color:${colors[colidx]}\" >'" 
  color_set="printf '<pre><span style=\"color:${colors[colidx]} ; font-size:${font}px \" >'" 
  color_rs="echo '</span></pre>'"
else
  colors=(0 1 2 3 4 5 6 7 )
  titlebold="tput bold; tput smul"
  titlereset="tput rmul; tput rmso"
  color_set="tput setaf ${colors[colidx]}"
  color_rs="tput rmso"
fi
}
setcolors
#----------------------------------------------
# Get data, check if stdlin, file, if not assume string
#----------------------------------------------
if [ -p /dev/stdin ]; then
        lastarg=$(< /dev/stdin)
else
	lastarg=$(echo "${@: -1}")
	if test -f "$lastarg"; then
	  lastarg=$(<$lastarg)
	fi
fi
# Cleanup the input data
lastarg=$(echo $lastarg | sed 's/\n/;/g; s/  / /g; s/\t//g; s/;;/;/g')
IFS=';' read -r -a array <<< $lastarg

# ensure that there is some data
if [[ ${array} == 0 ]]; then
  echo "No data found"
  exit 0
fi
echo "input:$lastarg"
#exit 0
#------------------------------------
# Get max value and max label length
#------------------------------------
maxval=0
maxlbl=10
#echo "array:${array[@]}"
for element in "${array[@]}"
do 
  IFS=',' read -r -a datapt <<< $element
  if (( $(echo "$maxval < ${datapt[1]}" |bc -l) )); then
	maxval=${datapt[1]}
  fi
  if (( $(echo "$maxlbl < ${#datapt[0]}" |bc -l) )); then
	maxlbl=${#datapt[0]}
  fi
done
#---------------------------------
# Print Title - use bold/underline
#---------------------------------
if [[ ! -z $title ]]; then
  printf "\n %${maxlbl}s " " "
  eval $titlebold
  printf "%s" "${title}" ; printf "\n\n"
  eval $titlereset
fi
#------------------------------------
# Cycle thru data and build bar chart
#------------------------------------
for element in "${array[@]}"
do
# check data values
  IFS=',' read -r -a datapt <<< $element
  # check for empty records
  if [ ${#datapt[0]} = 0 ]; then 
    break
  fi
  label=${datapt[0]}
  if [[ ${label} != "-t*" ]]; then 
	  val=${datapt[1]}
	  sval=$(bc <<< "$width * $val / $maxval")

	  # add color, use 4th item if available
	  if [[ ${#datapt[@]} > 3 && $pretty = False ]]; then
		icolor=${datapt[3]}
	  fi
	  if [[ $pretty = True ]] ; then
		let colidx++
		if [ $colidx -gt 7 ]; then
		  let colidx=1
		fi
	  elif [[ ${#datapt[@]} > 3 ]]; then
		colidx=${datapt[3]}
	  else
	  	colidx=$color
	  fi
	  setcolors
          eval $color_set
          printf " %${maxlbl}s " "$label"
	  bar="printf '█%.0s' {1..$sval}"
	  eval $bar; 
	  # add value and units if available
	  units=""
	  if [[ ${#datapt[@]} > 2 ]]; then
		units=${datapt[2]}
	  fi
	  printf " %d %s\n\n" $val "$units"
	  eval $color_rs 
  fi
done

SQL Output to Bars

With the base code I was able to start doing some more complicated actions, like piping SQL SELECT output. Below is an example from Sqlite3. The first step is to format the SQL output to: label,value;

pete@lubuntu:$ sqlite3 $HOME/dbs/someuser.db "select fname,age from users limit 4"
Brooke|18
Leah|18
Pete|100
Fred|77
pete@lubuntu:$ sqlite3 $HOME/dbs/someuser.db "select fname,age from users limit 4" \
>  | tr  '|' ',' | tr '\n' ';'
Brooke,18;Leah,18;Pete,100;Fred,77;
pete@lubuntu:$ sqlite3 $HOME/dbs/someuser.db "select fname,age from users limit 4" \ > | tr '|' ',' | tr '\n' ';' | ./hbars -t "Sqlite3 Users" -p Sqlite3 Users Brooke █████████ 18 Leah █████████ 18 Pete ██████████████████████████████████████████████████ 100 Fred ██████████████████████████████████████ 77

Web Page with Bars

Support was added for fixed chart width, engineering units and custom line colours. HTML <center> tags were used on the title.

$ cat data.txt
temp,44,degC,1;
humidity,33,%,4;
air pressure,88,mm Hg,5;
rain/precipation,6,mm,6;

$ cat data.txt | ./hbars -W -f 24 -w 50 -t '<center>Raspi Data</center>' > data.htm

Final Comments

Horizontal bars are relatives easy to create in Bash. Unfortunately showing vertical bars and Line charts will require a different approach

A Web Server in 1 Line of Bash Code

For people who do a lot of work with command line tools or Bash code, having a Bash web server could be very handy.

I was really amazed that in one line of Bash code I was able to create web servers that could:

  • send the output from a bash command directly to a browser page
  • create diagnostic pages using standard Linux tools
  • create pages that view Rasp PI GPIO pins
  • create a page to toggle a PI GPIO pin

One Line Web Servers

There are number of 1 line web servers, these are minimal offerings that exist in most of the programming languages.

A Bash web server can be created using the nc or netcat, the networking utility:

while true; do { \
  echo -ne "HTTP/1.0 200 OK\r\nContent-Length: $(wc -c <index.htm)\r\n\r\n"; \
  cat index.htm; } | nc -l -p 8080 ; \ 
done

This Bash statement echo’s to port 8080, the output is an HTTP header with the file content length defined. The cat command is used to show the HTML file.

This 1 line Bash example shows a single page (index.htm) which isn’t overly useful, there are other web server options that would work much better.

Where a Bash web server really stands out is in its ability to execute command line utilities or scripts and send the results to a web client.

Bash Web Server Calling Bash Commands

The output from command line utilities like iostat can be sent directly to a web client:

while true;
  do echo -e "HTTP/1.1 200 OK\n\n$(iostat)" \
  | nc -l -k -p 8080 -q 1; 
done

There are 2 important options that need to be set on nc, and they are: -k (this keeps the connection open after the first connection) and -q 1 (this closes the connection after 1 seconds, so another connection can occur). Depending on the complexity of the script that is used the -q timing may need to be adjusted.

The web page for the iostat command will look like:

Multiple Commands with Headings

Comments and multiply command line utilities can be defined as a variable that can be passed to the Bash Web server.

The figlet utility can be used to create custom ASCII headings, this is useful if you want to keep things simple without using HTML syntax. To install figlet in Ubuntu enter: sudo apt-get install figlet .

An example of using figlet headings and the sensors and vmstat utility is:

title1=$(figlet Sensors)
cmd1=$(sensors | sed -e 's/\°/ /g') # browser has problem with degrees, so remove
title2=$(figlet VMStat)
cmd2=$(vmstat)
thebody="$title1\n$cmd1\n$title2\n$cmd2"

while true;
  do echo -e "HTTP/1.1 200 OK\n\n$thebody" \
  | nc -l -p 8080 -q 1; 
done

Bash Web Server with Raspberry Pi GPIO

For many Raspberry Pi projects monitoring the status of the GPIO (General Purpose Input/Output) pins is quite important.

The Raspberry Pi gpio utility can be used to show the present status with the readall option:

pi@raspberrypi:~/pete $ gpio readall
 +-----+-----+---------+------+---+---Pi 3B--+---+------+---------+-----+-----+
 | BCM | wPi |   Name  | Mode | V | Physical | V | Mode | Name    | wPi | BCM |
 +-----+-----+---------+------+---+----++----+---+------+---------+-----+-----+
 |     |     |    3.3v |      |   |  1 || 2  |   |      | 5v      |     |     |
 |   2 |   8 |   SDA.1 | ALT0 | 1 |  3 || 4  |   |      | 5v      |     |     |
 |   3 |   9 |   SCL.1 | ALT0 | 1 |  5 || 6  |   |      | 0v      |     |     |
 |   4 |   7 | GPIO. 7 |  OUT | 0 |  7 || 8  | 0 | IN   | TxD     | 15  | 14  |
 |     |     |      0v |      |   |  9 || 10 | 1 | IN   | RxD     | 16  | 15  |
 |  17 |   0 | GPIO. 0 |   IN | 0 | 11 || 12 | 0 | IN   | GPIO. 1 | 1   | 18  |
 |  27 |   2 | GPIO. 2 |  OUT | 0 | 13 || 14 |   |      | 0v      |     |     |
 |  22 |   3 | GPIO. 3 |   IN | 0 | 15 || 16 | 0 | IN   | GPIO. 4 | 4   | 23  |
 |     |     |    3.3v |      |   | 17 || 18 | 0 | IN   | GPIO. 5 | 5   | 24  |
 |  10 |  12 |    MOSI | ALT0 | 0 | 19 || 20 |   |      | 0v      |     |     |
 |   9 |  13 |    MISO | ALT0 | 0 | 21 || 22 | 0 | IN   | GPIO. 6 | 6   | 25  |
 |  11 |  14 |    SCLK | ALT0 | 0 | 23 || 24 | 1 | OUT  | CE0     | 10  | 8   |
 |     |     |      0v |      |   | 25 || 26 | 1 | OUT  | CE1     | 11  | 7   |
 |   0 |  30 |   SDA.0 |   IN | 1 | 27 || 28 | 1 | IN   | SCL.0   | 31  | 1   |
 |   5 |  21 | GPIO.21 |  OUT | 0 | 29 || 30 |   |      | 0v      |     |     |
 |   6 |  22 | GPIO.22 |  OUT | 0 | 31 || 32 | 0 | OUT  | GPIO.26 | 26  | 12  |
 |  13 |  23 | GPIO.23 |  OUT | 0 | 33 || 34 |   |      | 0v      |     |     |
 |  19 |  24 | GPIO.24 |  OUT | 0 | 35 || 36 | 0 | IN   | GPIO.27 | 27  | 16  |
 |  26 |  25 | GPIO.25 |  OUT | 0 | 37 || 38 | 0 | OUT  | GPIO.28 | 28  | 20  |
 |     |     |      0v |      |   | 39 || 40 | 0 | OUT  | GPIO.29 | 29  | 21  |
 +-----+-----+---------+------+---+----++----+---+------+---------+-----+-----+
 | BCM | wPi |   Name  | Mode | V | Physical | V | Mode | Name    | wPi | BCM |
 +-----+-----+---------+------+---+---Pi 3B--+---+------+---------+-----+-----+

Rather than passing the Bash commands as a string, a small Bash script (web_body.sh) file can be created. This file will show the time and then call the gpio readall command.

#!/bin/bash
# web_body.sh - Show the time and PI GPIO pins

date $T

echo "$(gpio readall)"

The online Bash web server command is:

while true; do { \
  echo -ne "HTTP/1.1 200 OK\r\n"; sh web_body.sh; } \
  | nc -l -k -q 2 8080; \
done

The Web page for this script will look something like:

Send GPIO Writes from the Address Bar

Client side GET requests can be simulated on the browser address bar.

For example entering gpio write 7 1 on the address bar sents the Bash Server a GET request.

The HTTP request encodes spaces, so a space appears as a %20.

Bash code can be added to look for specific messages. In this case the “gpio write 7 1” or “gpio write 7 0” message can be search for, and if it is found then that exact message can be executed.

The Bash code can now be modified to ; look for the “GET gpio” message, then decode any HTTP %20 characters to spaces, next parse out the string to get the gpio message, and finally execute the required command. The code is below:

while true; 
  do { echo -ne "HTTP/1.1 200 OK\r\n"; sh web_body.sh; } | \
  nc -l -k -q 5 8080 | grep "GET /gpio" | \
  sed -e 's/%20/ /g' | \
  eval $( awk '{print substr($0,6,15) }') ;
done

With the new code, the gpio write is executed and the result can be seen in the web page.

Create an HTML Form

Entering commands on the command line works but it’s crude, a better way is to create an HTML Form.

The Bash web server code can remain exact the same as in the earlier example.

The original script (web_body.sh) file can be modified to made the output HTML format and three forms can be included. The first and second form will define the GET actions to turn on or off the GPIO pin and the third form will be used to refresh the page to check for GPIO changes.

#!/bin/bash
# web_body.sh - Show the time and PI GPIO pins
#             - Use HTML instead of text output
#             - Add forms for GPIO on/off, and a refresh
echo "
<!DOCTYPE html><html><head>
</head><body>
<h1>Bash Commands in a Web Page</h1>
<h2>Toggle Pin 7 On/Off</h2>
<form action='gpio write 7 0'>
 <input type='submit' value='OFF'> 
</form> 
<form action='gpio write 7 1'>
 <input type='submit' value='ON'>
</form>
<form action=''>
 <input type='submit' value='Refresh Page'>
</form>
<pre>
"

date $T

echo "$(gpio readall)"

echo "</pre></body></html>"

The client web page is now:

After turning on or off the GPIO pin, a refresh of the web page is required to see the new status.

Final Commands

A Bash Web Server is a quick and easy solution for viewing the output from Bash scripts and commands. I especially like the fact that I don’t need to install any special software and I don’t need to write any HTML code.

It is important to note that the number of concurrent connections is very low, (1/sec if the nc -q option is 1).

A Bash Web Server supports client side GET and POST requests, however for complex requirements the Bash code could start to get messy quickly and it would probably be best to look at another solution.

Simple TCP/UDP Bash Apps

There are some great communication protocols (MQTT, RabbitMQ, ReDis …) that are excellent for passing data between nodes.

For applications where you only need to do simple communications a couple of lines of Bash can be used with TCP or UDP sockets.

In this blog I wanted to document UDP/TCP communications using Linux Bash commands to:

  • define periodic 1-way communications
  • use progress bars to show data from remote nodes
  • remotely send commands to a Raspberry Pi
  • setup simple TCP backdoors

NC (NetCat) – for TCP and UDP Connections

In theory you should be able to create a input read via something like:

echo $(read < /dev/udp/127.0.0.1/9999)

Then do a write using:

echo "some text" > /dev/udp/127.0.0.1/9999)

Unfortunately Linux device connections are not fully reliable, especially on the read or listening side. However the write component appears to be fairly solid. Luckily there is a solid solution using the nc (NetCat) command line utility. The nc utility is typically preloaded on most Linux systems.

The nc utility supports both UDP (-u option) and TCP (default) connections.

To setup a UDP listener, use the IP address of the listener node, and select the -k option to allow multiple connections to occur:

nc -u -l -k 192.168.0.111  9999

For this example the listener’s IP is: 192.168.0.111, and port 9999 is used.

To do manual writes from the command line, enter:

 nc -u  192.168.0.111  9999

To send data from a script there are two methods, either using nc or writing to the device:

# writing via nc, -w0 send only 1 message 
echo "456" | nc -u -w0 192.168.0.111  9999
# writing via device:
echo "456" > /dev/udp/192.168.0.111/9999

Multiple Writes and Zenity Progress Dialogs

Zenity is command line dialog utility that is typically preinstalled on most versions of Linux.

The data that is sent to the UDP listener can be piped to Zenity progress bar:

nc -u -l -k 192.168.0.111  9999 | zenity --progress --title="Remote Data"

A script to send seconds every second would be:

#!/bin/bash
echo "Press [CTRL+C] to stop..." 
( 
while : 
do
# $(date +'%S') seconds" | nc -u -w0 192.168.0.111 9999
 echo "$(date +'%S')" | nc -u -w0 192.168.0.111 9999
 sleep 1 
done 
)

When the script echos an integer the progress bar will be updated with the integer value. An echo string starting with a “#” will update the text above the bar.

The progress bar is from 0-100%, but the integer value can be re-scaled to make the information clearer. For example to re-scale 0-60 secs to 0-100:

echo "$(date +'%S')*100/60" | bc | nc -u -w0 192.168.0.111 9999

YAD – for Multiple Progress Bars

YAD (Yet Another Dialog) is a command line GUI utility that offers a little more functionality than Zenity. To install YAD on Raspberry Pi’s and Ubuntu: sudo apt-get install yad

A bash command with a UDP listener with YAD 2-bars would be:

nc -u -l -k 192.168.0.111  9999 | yad --multi-progress \
  --bar="CPU Idle" --bar="CPU Temp" --title="Remote CPU Info"

The CPU Idle Time can be found by:

top -n 1 | grep %Cpu | awk '{print $8}'
93.8

The CPU Temperature on a PC can be found by:

sensors | grep CPU | awk '{print substr($2,2,4)}'
44.0

A script to send the CPU Idle Time and Temperature to the UDP listener is:

#!/bin/bash
echo "Press [CTRL+C] to stop..." 
( 
while : 
do
 cpuidle=$(top -n 1 | grep %Cpu | awk '{print $8}') 
 echo "1:"$cpuidle | nc -u -w0 192.168.0.111 9999

 cputemp=$(sensors | grep CPU | awk '{print substr($2,2,4)}')
 echo "2:"$cputemp | nc -u -w0 192.168.0.111 9999
 echo "2:#"$cputemp" Deg C" | nc -u -w0 192.168.0.111 9999
 sleep 5 
done 
)

For YAD multiple progress bars, an echo of 1: is for bar 1, 2: is for bar 2 etc. Echo-ing “2:# ” updates the text for the 2nd bar.

NC is not the same on Rasp Pi

I found that on the Rasp Pi the nc listening functions would not pass any information to bash scripts. Manual mode still works to view messages, but the messages can’t be piped to other commands.

This mean that things like the Zenity and YAD progress bars would not work on a Raspberry Pi. For many applications this may not be a big problem because the Rasp Pi can still send information via nc.

NC vs. NCAT

The ncat utility is very similar to nc but it offers the ability to run commands. By default nc is preloaded on most systems, but ncat needs to be installed. Installing ncat will vary based on your OS.

The ncat utility allows you to make backdoors so be careful of its use.

To create a backdoor simply (via TCP), define the ncat -c (execute command option) to be /bin/bash:

ncat -l -k 192.168.0.108  9999 -c /bin/bash

If on a remote node you enter: ncat 192.168.0.108 , you can start typing commands that are run on the remote node with the results echoing back. Very cool for test system but super dangerous for real systems.

Remotely Toggle a Rasp Pi GPIO Pin

Rather than opening up the system totally fixed commands can be defined. For example to toggle pin 7 on a Rasp Pi. A listener script is run:

ncat -l -k 192.168.0.108  9999 -c "gpio toggle 7"

A remote button GUI script could be used in conjunction with the listener script to toggle the GPIO pin:

#!/bin/bash
#
# Toggle a Rasp Pi GPIO pin

rc=1 # OK button return code =0 , all others =1
while [ $rc -eq 1 ]; do
  ans=$(zenity --info --title 'Remote Connect to Pi' \
      --text 'Toggle GPIO Pin' \
      --ok-label Quit \
      --extra-button TOGGLE \
       )
  rc=$?
  echo "${rc}-${ans}"
  echo $ans
  if [[ $ans = "TOGGLE" ]]
  then
        echo "Toggle Pin"
        nc -w0 192.168.0.108  9999
  fi
done

Send a Command String to Run Remotely

For this example the Rasp Pi is setup to be TCP listener, and the command (-c option) is /bin/bash, so this allows the remote PC to send custom commands:

ncat -l -k 192.168.0.108  9999 -c /bin/bash

On the remote system a bash script is created with 2 buttons and the custom commands are sent to the Pi to run:

#!/bin/bash
#
# Toggle two Rasp Pi GPIO pins

rc=1 # OK button return code =0 , all others =1
while [ $rc -eq 1 ]; do
  ans=$(zenity --info --title 'Remote Connect to Pi' \
      --text 'Toggle GPIO Pins' \
      --ok-label Quit \
      --extra-button Pin2 \
      --extra-button Pin7 \
       )
  rc=$?
  echo "${rc}-${ans}"
  echo $ans
  if [[ $ans = "Pin2" ]]
  then
        echo "gpio toggle 2" | nc -w0 192.168.0.108  9999
  elif [[ $ans = "Pin7" ]]
  then
        echo "gpio toggle 7" | nc -w0 192.168.0.108  9999
  fi
done

On systems other than Rasp Pi, the nc command can also be used to run remote programs by:

nc -u -l -k 192.168.0.111  9999 | awk '{ system($1 " " $2 " " $3 " " $4)}'

Final Comments

In this blog I’ve kept things focused on TCP/UDP communications with bash script but you could easily include Arduino, Python and Node-Red as either clients or servers.

Zenity: Command line Dialogs

Zenity is command line GUI creator that has been around since 2012, and it is pre-installed on most versions of Linux, including Raspberry PI’s. Zenity is also available for MacOS and Windows.

I came across it on a recent project and I wanted to document some of my code. My examples include:

  • CPU stats on a dialog – 1 line of Bash script
  • Show a web page in a dialog – 1 line
  • Create a 4 button PI Rover control – ~ 25 lines
  • Dynamic bar of CPU core temperature – 7 lines
  • Show CSV or SQL data in a list dialog – 1 line
  • Form to insert user data in an SQL database – ~7 lines

What is Zenity?

Zenity is a command line dialog creator. I found it pretty quick to pick up and it works well for simple Bash scripts. Zenity supports:

  • Basic forms
  • Calendar dialogs
  • Color selection dialogs
  • File Selection Dialogs
  • List Dialog
  • Message and Notification Dialog
  • Progress bars and Scales
  • Text Entry and Text Information Dialogs

Message Dialogs

Message dialogs can be created for errors, info, questions or warnings. The difference is the icon that shows up (and Ok/Cancel for the question dialog).

The Bash code to get the instantaneous CPU idle time would be:

pi@raspberrypi:~ $ # Run top once and look for the line with %Cpu
pi@raspberrypi:~ $ top -n 1 | grep Cpu
%Cpu(s):  7.4 us,  3.6 sy,  0.0 ni, 88.8 id,  0.3 wa,  0.0 hi,  0.0 si,  0.0 st

pi@raspberrypi:~ $ # Get the 8th item from the grep

pi@raspberrypi:~ $ top -n 1 | grep %Cpu | awk '{print $8}'
88.8

The bash code to show the CPU idle time in a info dialog is:

zenity --info --text=$(top -n 1 | grep %Cpu | awk '{print $8}') --title="CPU Idle Time"

Message Dialogs with Custom Font

Text font and size can be modified in message dialogs, using the Pango Markup Language syntax. Pongo is similar to HTML. The <span></span> set of tags is used to encode font and color definitions, For example:

zenity --warning --text='<span font="32" foreground="red">HIGH Temperature</span>' --title="HDD Check"

Unfortunately only Message Dialog texts can changed, so text in dialogs like list, scale, and progress can’t not have their fonts changed.

Web Pages in a Text-Info Dialog

Text or HTML files can be passed to a text-info dialog:

zenity --text-info --title="Background Reading" --html --url="https://developer.gnome.org"

A checkbox can be added, and the user feedback can be read by the bash script:

#!/bin/sh
# show web page in a dialog with a next step action
#
theurl="https://developer.gnome.org"

zenity --text-info --title="Background Reading" --html --url=$theurl \
       --checkbox="I read it...and I'm good to go"
rc=$?
echo $rc
case $rc in
    0)
        echo "Start some next step"
	# next step
	;;
    1)
        echo "Stop installation!"
	;;
   -1)
        echo "An unexpected error has occurred."
	;;
esac

It is important to note that the text-info dialog should be used for simple web pages. There is no Javascript support and web links will launch the default web browser with requested page.

Refreshing Message Dialogs

Of all the Zenity dialogs, only the Progress dialog supports a method to update text on an open dialog. A workaround is to use the –timeout option to close the dialog and then redisplay it with the new data.

rc=5
while [[ $rc -eq 5 ]];
do 
  zenity --info --text=$(date +'%S' ) \
  --title="Seconds Timer Test" --timeout=5 --ok-label Quit $ zenity --info \
  --text=$(date +'%S' )   --title="Seconds Timer Test" 2>/dev/null

  rc=$?	
  echo $rc
done 

This “timeout and redraw” method is ugly because the window always positions in the middle of the screen and this can be quite annoying. Unfortunately Zenity does not support any top/left positioning options.

The xdotool could be used find the zenity window id and then position it, but this would need to be done in another script. (It can’t be done in the same script because the zenity line doesn’t complete until either it times out or OK is pressed). The xdotool script would be:

pid=$(xdotool search -onlyvisible -name myzenitywindownname)
xdotool windowmove $pid 0 0

If you need dynamically updated text on a dialog I think that it would be best to use another tool, (Python, YAD etc.).

Info Dialog with Extra Buttons – Pi Rover Controls

It’s possible to add some extra button to an info dialog. Below is an example where a Raspberry Pi Rover is controlled with a zenity multi-button info dialog. Note: the pins will vary with your setupArduinoIt’s possible to add some extra button to an info dialog. Below is an example where a Raspberry Pi Rover is controlled with a zenity multi-button info dialog. Note: the pins will vary with your setup


#!/bin/bash
#
# rover.sh - Rover Controls with Multiple Button Dialog
# Define GPIO pins for the motors motorL=7 motorR=11 rc=1 # OK button return code =0 , all others =1 while [ $rc -eq 1 ]; do ans=$(zenity --info --title 'Drive a Rover' \ --text 'Motor Action' \ --ok-label Quit \ --extra-button FORWARD \ --extra-button STOP \ --extra-button LEFT \ --extra-button RIGHT \ ) rc=$? echo "${rc}-${ans}" echo $ans if [[ $ans = "FORWARD" ]] then echo "Running the Rover" gpio -1 write $motorL 1 ; gpio -1 write $motorR 1; elif [[ $ans = "STOP" ]] then echo "Stopping the Rover" gpio -1 write $motorL 0 ; gpio -1 write $motorR 0; elif [[ $ans = "LEFT" ]] then echo "Rover turning Left" gpio -1 write $motorL 1 ; gpio -1 write $motorR 0; elif [[ $ans = "RIGHT" ]] then echo "Rover turning RIGHT" gpio -1 write $motorL 0 ; gpio -1 write $motorR 1; fi done

The script can be run by: bash rover.sh

Below is the dialog and the rover.

Progress Bars – Show Dynamic Values

A Zenity progress dialog can show dynamic updates with scripts that define steps using sleep statements. When the step outputs a value the process bar is updated. The text on the progress dialog is changed by outputting a text string starting with a # character.

A 3-step example would be:

(
echo "33"; echo "# 1/3 done" ; sleep 5; \
echo "66"; echo "# 2/3 done" ; sleep 5; \
echo "100";echo "# Finished"  \
) | zenity --progress --title="3 step test"

The progress dialog can use a bash for or while statement. The progress dialog can be passed both new text and the value. A text string starting with # is interpreted as the new text. A number string is interpreted as the progress bar value.

Below is an example where a value is counted from 1 to 100:

( for i in `seq 1 100`; do echo $i; echo "# $i";  sleep 1; done ) | zenity --progress

The next thing I tested is a dialog that runs indefinitely (or until you hit “Control-C”). It’s import to note that the progress bar is from 0-100, so scaling your value may be required. An example of scaling a time from 0-60 would be:

echo "$(date +'%S')*100/60" | bc

An example to show seconds in a dialog would be:

#!/bin/bash
# show_sec.sh - progress dialog to show seconds
echo "Press [CTRL+C] to stop..." 
( 
  while :; do 
  echo "# $(date +'%S')" 
  # Scale 0-60 to 0-100 
  echo "$(date +'%S')*100/60" | bc
  sleep 1 
  done 
  ) | zenity --progress  --title="Show Time in Seconds"

To run this script: bash show_sec.sh

A more useful dialog would be to show the CPU temperature:

#!/bin/sh 
# show_cpu_temp.sh - Progress Dialog to show CPU temperature
# 
echo
 "Press [CTRL+C] to stop..."
(
while :; do 
  echo "# $(sensors | grep CPU)" 
  sensors | grep CPU | awk '{print substr($2,2,4) }' 
  sleep 5 
done ) | zenity --progress --title="CPU Temperature"  

List Dialog – Show CSV/SQL Data

If you are working with a simple known data set then the List Dialog might be a good fit.

The List Dialog expects the data to be a sequential list, so a 2 column example of static data would be:

zenity --list \
  --title="2 Column Example" \ 
  --column="Month" --column="Sales" \
   Jan 100 Feb 95 Mar 77 Apr 110 May 111

Text and CSV files can also be used in Zenity lists. The first step is to convert the file into a single column of data. This can be done with the tr statement. For the example below the comma (,) is replaced with a newline (\n) character:

$ cat lang.txt
Brazil,Brasilia,Portuguese
England,London,English
France,Paris,French
Germany,Berlin,German

$ cat lang.txt | tr ',' '\n'
Brazil
Brasilia
Portuguese
England
London
English
France
Paris
French
Germany
Berlin
German

Now the sequential data can be passed into a Zenity list:

cat lang.txt | tr ',' '\n' | zenity --list \
  --title="Country Info" \
  --column="Country" --column="Capital" --column="Language"

Once you have some Zenity and Bash basics down you can some fairly advanced operations. Below is a 1-line example that uses awk to parse out specific fields (1 and 3) and then the user selected output is echo-ed.

awk -F "\"*,\"*" '{print $1 "\n" $3}' pidata.csv  | \
  echo $(zenity --list --column="field1" --column="field3" --print-column=ALL)

Similarly to data from an SQL query can be show. Almost all SQL servers have a command line interface. The interface will vary from database to database, but an example with Sqlite would be:

(sqlite3 someuser.db "select fname,lname,age,job from users" ) | tr '|' '\n' | zenity --list \
  --title="My Database" \
  --column="first name" --column="last name" --column=age --column=job

Form Dialog – Insert SQL Data

The Forms Dialog allows for date, text and password inputs, and the result are passed as string (| is the default separator). A form example with output would be:

$ row=$(zenity --forms --title="Create user" --text="Add new user" \
   --add-entry="First Name" \    
   --add-entry="Last Name" \    
   --add-entry="Age" \    
   --add-entry="Job") ; echo $row

field1|field2|field3|field4

The next step is to format the form data into an SQL statement. The SQL INSERT syntax is:

INSERT INTO table (field1,field2…) VALUE (value1,value2…)

For the example above, field1|field2|field3|field4 needs to formatted to the values. This manipulation can be done by the bash sed command with search and replace (s) option:

$ row="field1|field2|field3|field4"
$ echo "'$row'" | sed "s/|/','/g"
'field1','field2','field3','field4'

The bash script to present the zenity form and input the data is below. The if statement is used to ensure that the cancel button wasn’t pressed. More if statement would probably be required for some data validation.

# zen_sqlin.sh - create a form to add a new user into a SQLite3 database
row=$(zenity --forms --title="Create user" --text="Add new user" \ --add-entry="First Name" \ --add-entry="Last Name" \ --add-entry="Age" \ --add-entry="Job") if [[ -n $row ]] # Some data found then indata=$(echo "'$row'" | sed "s/|/','/g") cmd="sqlite3 someuser.db \"INSERT INTO users (Fname,Lname,Age,Job) VALUES ($indata)\"" eval $cmd echo "Added data: $indata" fi

The script is run by: bash zen_sqlin.sh

Zenity (GTk) Warning Messages

Depending on your system you might see some Zenity warning messages such as:

Gtk-Message: 15:30:52.461: GtkDialog mapped without a transient parent. This is discouraged.

I never saw this on my Raspberry Pi but I did see it on my lubuntu system. To make things cleaner the warning can be piped to the null device:

$ zenity --info --text=$(date +'%S' )   --title="Seconds Timer Test" 2>/dev/null

Some Final Comments

I really just touched the surface on what zenity can do. For more info see some of the tutorials.

For simple stuff zenity works fine. If you’re looking for a more complete command line GUI tool try YAD, for myself I’ll stick to Python.

As a side note, there is a Python library for zenity. If you’re feeling comfortable with the bash version of zenity and you only need to do simple dialogs then this might be a good fit.

Using AWK in Bash Scripts

In the past Python has been my go to language for quick scripts, however lately I’ve done a lot of projects where I’ve needed to use small Bash scripts.

I found that by adding a little bit of AWK to my Bash scripts I’ve been able to do something in one line of Bash/AWK that would of taken me multiple lines of Python.

AWK is named after it’s authors: Alfred Aho, Peter Weinberger, and Brian Kernighan, and it is an old school (1994) text extraction and reporting tool.

The nice thing about AWK is that you only need to learn a couple of commands to make it usefully.

A Bash/AWK Example

I was setting up a home automation system using a Raspberry Pi with Home Assistant. I was loading a lot of add-on components and I was worried that I might be overloading the Pi so I wanted to monitor the CPU idle time.

Home Assistant supports a command line sensor, so I just needed to get the idle time out of the iostat command:

The idle time value was the 6th item on the 4th line. The AWK code to get only the idle time is:

~$ iostat | awk '{if (NR==4) print $6}'
96.92

AWK logic need to be in single quotes and curly brackets groups together statements. This logic says: if the Number of Record (NR) variable is 4 print the 6th item.

Finding and Killing a Task

There are a number different approaches to doing. To find and kill a number of task:

#!/bin/bash
#
# stop_task.sh - stop a task
#
task1="edublocks"

echo "Stopping $task1..."
ps -e | grep -E $task1 | \
 awk '{print $1}' | xargs sudo kill -9 1>&-

Math on a Column of Data

Bash is easy to use for a row of data, but I find it tricky on columns of data.

As an example, to get the total size of all Sqlite files, ls – l *db is piped to an awk statement. This awk statement that has two parts the first part sums column five (sum +=$5), the END is used to do line-by-line, the second part prints the result.

pete@lubuntu:~/dbs$ ls -l *.db 
-rw-r--r-- 1 pete pete  323584 Apr 14  2020 ebola.db
-rw-r--r-- 1 pete pete 5124096 Apr 14  2020 netflix.db
-rw-r--r-- 1 pete pete   98304 Apr  8  2020 sars.db
-rw-r--r-- 1 pete pete  503808 Apr 14  2020 schools.db
-rw-r--r-- 1 pete pete  208896 Apr  8  2020 schools_old.db
-rw-r--r-- 1 pete pete    8192 Feb  8 20:11 someuser.db
pete@lubuntu:~/dbs$ ls -l *.db | awk '{sum +=$5 } END {print "Total= " sum}'
Total= 6266880

I used this approach to find how much power is being consumed on all my USB ports:

pete@lubuntu:~/dbs$ lsusb -v  2>&- | grep -E  'Bus 00|MaxPower'
Bus 002 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
    MaxPower                0mA
Bus 002 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
    MaxPower                0mA
Bus 001 Device 002: ID 8087:0024 Intel Corp. Integrated Rate Matching Hub
    MaxPower                0mA
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
    MaxPower                0mA
Bus 004 Device 001: ID 1d6b:0003 Linux Foundation 3.0 root hub
    MaxPower                0mA
Bus 003 Device 004: ID 413d:2107  
    MaxPower              100mA
Bus 003 Device 003: ID 04b3:310c IBM Corp. Wheel Mouse
    MaxPower              100mA
Bus 003 Device 002: ID 1a40:0101 Terminus Technology Inc. Hub
    MaxPower              100mA
Bus 003 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub
    MaxPower                0mA

pete@lubuntu
:~/dbs$ (lsusb -v 2>&- | grep MaxPower | grep -o -E '[0-9]+' ) | awk '{ sum += $1} END {print "\nTotal= " sum " mA"}' Total= 300 mA

Some Useful AWK Statements

There is an enhanced version of AWK, GAWK (GNU AWK) that might already be loaded on your Linux system. If you are on a Raspberry Pi you can install GAWK by:

sudo apt-get install gawk

There are some excellent tutorials on AWK below are some of commands that I’ve found useful:

substr(string,position,length) – get part of a string:

An example of substr could be used to get the CPU temperature from the sensors utility:

~$ sensors | grep CPU | awk '{print substr($2,2,4)}'
 44.0

The substr() command looks at the 2nd item (+44.0°C), and starts at the 2nd character and it gets 4 characters.

print() with if() – print based on conditions:

The AWK print statement can be used with an if statement to show a filtered list.

An example of this would be to filter the ps (snapshot of the current processes) command, and print only lines with a time showing:

~$ # SHOW ALL PROCESSES
~$ ps -e 
   PID TTY          TIME CMD
     1 ?        00:00:03 systemd
     2 ?        00:00:00 kthreadd
     4 ?        00:00:00 kworker/0:0H
     6 ?        00:00:00 mm_percpu_wq
     7 ?        00:00:00 ksoftirqd/0
     8 ?        00:01:10 rcu_sched
...
~$  # SHOW ONLY PROCESSES WITH TIME
~$ ps -e | awk '{if ($3 != "00:00:00") print $0}'
   PID TTY          TIME CMD
     1 ?        00:00:03 systemd
     8 ?        00:01:10 rcu_sched
    10 ?        00:00:06 migration/0
    15 ?        00:00:03 migration/1
...

systime() / strftime() – get time and format time:

These time functions allow you to add time stamps and then do formatting on the date/time string. I found this useful in logging and charting projects. An example to add a time stamp to the sensor’s CPU temperature would be:

$ sensors | grep CPU | awk '{print strftime("%H:%M:%S ",systime()) $1 $2 }'
 11:06:18 CPU:+45.0°C

Final Comments

I’ve found that learning a little bit of AWK has really paid off.

AWK supports a lot of functionality and it can be used to create full on scripting applications with user inputs, file I/O, math functions and shell commands, but despite all this I’ll stick to Python if things get complex.