Zenity: Command line Dialogs

Zenity is command line GUI creator that has been around since 2012, and it is pre-installed on most versions of Linux, including Raspberry PI’s. Zenity is also available for MacOS and Windows.

I came across it on a recent project and I wanted to document some of my code. My examples include:

  • CPU stats on a dialog – 1 line of Bash script
  • Show a web page in a dialog – 1 line
  • Create a 4 button PI Rover control – ~ 25 lines
  • Dynamic bar of CPU core temperature – 7 lines
  • Show CSV or SQL data in a list dialog – 1 line
  • Form to insert user data in an SQL database – ~7 lines

What is Zenity?

Zenity is a command line dialog creator. I found it pretty quick to pick up and it works well for simple Bash scripts. Zenity supports:

  • Basic forms
  • Calendar dialogs
  • Color selection dialogs
  • File Selection Dialogs
  • List Dialog
  • Message and Notification Dialog
  • Progress bars and Scales
  • Text Entry and Text Information Dialogs

Message Dialogs

Message dialogs can be created for errors, info, questions or warnings. The difference is the icon that shows up (and Ok/Cancel for the question dialog).

The Bash code to get the instantaneous CPU idle time would be:

pi@raspberrypi:~ $ # Run top once and look for the line with %Cpu
pi@raspberrypi:~ $ top -n 1 | grep Cpu
%Cpu(s):  7.4 us,  3.6 sy,  0.0 ni, 88.8 id,  0.3 wa,  0.0 hi,  0.0 si,  0.0 st

pi@raspberrypi:~ $ # Get the 8th item from the grep

pi@raspberrypi:~ $ top -n 1 | grep %Cpu | awk '{print $8}'
88.8

The bash code to show the CPU idle time in a info dialog is:

zenity --info --text=$(top -n 1 | grep %Cpu | awk '{print $8}') --title="CPU Idle Time"

Message Dialogs with Custom Font

Text font and size can be modified in message dialogs, using the Pango Markup Language syntax. Pongo is similar to HTML. The <span></span> set of tags is used to encode font and color definitions, For example:

zenity --warning --text='<span font="32" foreground="red">HIGH Temperature</span>' --title="HDD Check"

Unfortunately only Message Dialog texts can changed, so text in dialogs like list, scale, and progress can’t not have their fonts changed.

Web Pages in a Text-Info Dialog

Text or HTML files can be passed to a text-info dialog:

zenity --text-info --title="Background Reading" --html --url="https://developer.gnome.org"

A checkbox can be added, and the user feedback can be read by the bash script:

#!/bin/sh
# show web page in a dialog with a next step action
#
theurl="https://developer.gnome.org"

zenity --text-info --title="Background Reading" --html --url=$theurl \
       --checkbox="I read it...and I'm good to go"
rc=$?
echo $rc
case $rc in
    0)
        echo "Start some next step"
	# next step
	;;
    1)
        echo "Stop installation!"
	;;
   -1)
        echo "An unexpected error has occurred."
	;;
esac

It is important to note that the text-info dialog should be used for simple web pages, there is no Javascript support and web links will launch the default web browser with requested page.

Refreshing Message Dialogs

Of all the Zenity dialogs, only the Progress dialog supports a method to update text on an open dialog. A workaround is to use the –timeout option to close the dialog and then redisplay it with the new data.

rc=5
while [[ $rc -eq 5 ]];
do 
  zenity --info --text=$(date +'%S' ) \
  --title="Seconds Timer Test" --timeout=5 --ok-label Quit $ zenity --info \
  --text=$(date +'%S' )   --title="Seconds Timer Test" 2>/dev/null

  rc=$?	
  echo $rc
done 

This “timeout and redraw” method is ugly because the window always gets positioned in the middle of the screen and this can be quite annoying. Unfortunately Zenity does not support any top/left positioning options.

The xdotool can be used find the zenity window id and then reposition it, but this would need to be done in another script. (It can’t be done in the same script because the zenity line doesn’t complete until either it times out or OK is pressed). The xdotool script would be:

pid=$(xdotool search -onlyvisible -name myzenitywindownname)
xdotool windowmove $pid 0 0

If you need dynamically updated text on a dialog I think that it would be best to use another tool, (Python, YAD etc.).

Info Dialog with Extra Buttons – Pi Rover Controls

It’s possible to add some extra buttons to an info dialog. Below is an example where a Raspberry Pi Rover is controlled with a zenity multi-button info dialog. (Note: the pins will vary with your setup.)


#!/bin/bash
#
# rover.sh - Rover Controls with Multiple Button Dialog
# Define GPIO pins for the motors motorL=7 motorR=11 rc=1 # OK button return code =0 , all others =1 while [ $rc -eq 1 ]; do ans=$(zenity --info --title 'Drive a Rover' \ --text 'Motor Action' \ --ok-label Quit \ --extra-button FORWARD \ --extra-button STOP \ --extra-button LEFT \ --extra-button RIGHT \ ) rc=$? echo "${rc}-${ans}" echo $ans if [[ $ans = "FORWARD" ]] then echo "Running the Rover" gpio -1 write $motorL 1 ; gpio -1 write $motorR 1; elif [[ $ans = "STOP" ]] then echo "Stopping the Rover" gpio -1 write $motorL 0 ; gpio -1 write $motorR 0; elif [[ $ans = "LEFT" ]] then echo "Rover turning Left" gpio -1 write $motorL 1 ; gpio -1 write $motorR 0; elif [[ $ans = "RIGHT" ]] then echo "Rover turning RIGHT" gpio -1 write $motorL 0 ; gpio -1 write $motorR 1; fi done

The script can be run by: bash rover.sh

Below is the dialog and the rover.

Progress Bars – Show Dynamic Values

A Zenity progress dialog can show dynamic updates with scripts that define steps using sleep statements. When the step outputs a value the process bar is updated. The text on the progress dialog is changed by outputting a text string starting with a # character.

A 3-step example would be:

(
echo "33"; echo "# 1/3 done" ; sleep 5; \
echo "66"; echo "# 2/3 done" ; sleep 5; \
echo "100";echo "# Finished"  \
) | zenity --progress --title="3 step test"

The progress dialog can use a bash for or while statement. The progress dialog can be passed both a new text label and a value. A text string starting with # is interpreted as the new text label. A number string is interpreted as the progress bar value.

Below is an example where a value is counted from 1 to 100:

( for i in `seq 1 100`; do echo $i; echo "# $i";  sleep 1; done ) | zenity --progress

The next thing I tested is a dialog that runs indefinitely (or until you hit “Control-C”). It’s import to note that the progress bar is from 0-100, so scaling your value may be required. An example of scaling a time from 0-60 to 0-100 would be:

echo "$(date +'%S')*100/60" | bc

Using the above code to show seconds in a dialog would be:

#!/bin/bash
# show_sec.sh - progress dialog to show seconds
echo "Press [CTRL+C] to stop..." 
( 
  while :; do 
  echo "# $(date +'%S')" 
  # Scale 0-60 to 0-100 
  echo "$(date +'%S')*100/60" | bc
  sleep 1 
  done 
  ) | zenity --progress  --title="Show Time in Seconds"

To run this script: bash show_sec.sh

A more useful dialog would be to show the CPU temperature:

#!/bin/sh 
# show_cpu_temp.sh - Progress Dialog to show CPU temperature
# 
echo
 "Press [CTRL+C] to stop..."
(
while :; do 
  echo "# $(sensors | grep CPU)" 
  sensors | grep CPU | awk '{print substr($2,2,4) }' 
  sleep 5 
done ) | zenity --progress --title="CPU Temperature"  

List Dialog – Show CSV/SQL Data

If you are working with a simple known data set then the List Dialog might be a good fit.

The List Dialog expects the data to be a sequential list, so a 2 column example of static data would be:

zenity --list \
  --title="2 Column Example" \ 
  --column="Month" --column="Sales" \
   Jan 100 Feb 95 Mar 77 Apr 110 May 111

Text and CSV files can also be used in Zenity lists. The first step is to convert the file into a single column of data. This can be done with the tr statement. For the example below the comma (,) is replaced with a newline (\n) character:

$ cat lang.txt
Brazil,Brasilia,Portuguese
England,London,English
France,Paris,French
Germany,Berlin,German

$ cat lang.txt | tr ',' '\n'
Brazil
Brasilia
Portuguese
England
London
English
France
Paris
French
Germany
Berlin
German

Now the sequential data can be passed into a Zenity list:

cat lang.txt | tr ',' '\n' | zenity --list \
  --title="Country Info" \
  --column="Country" --column="Capital" --column="Language"

Once you have some Zenity and Bash basics down you can do some fairly advanced operations. Below is a 1-line example that uses awk to parse out specific fields (1 and 3) and then the user selected output is echo-ed.

awk -F "\"*,\"*" '{print $1 "\n" $3}' pidata.csv  | \
  echo $(zenity --list --column="field1" --column="field3" --print-column=ALL)

Almost all SQL servers have a command line interface. The output from an SQL query can be formatted to a “zenity friendly” form. The interface will vary from database to database, but an example with Sqlite would be:

(sqlite3 someuser.db "select fname,lname,age,job from users" ) | tr '|' '\n' | zenity --list \
  --title="My Database" \
  --column="first name" --column="last name" --column=age --column=job

Form Dialog – Insert SQL Data

The Forms Dialog allows for date, text and password inputs, and the result are passed as string (| is the default separator). A form example with output would be:

$ row=$(zenity --forms --title="Create user" --text="Add new user" \
   --add-entry="First Name" \    
   --add-entry="Last Name" \    
   --add-entry="Age" \    
   --add-entry="Job") ; echo $row

field1|field2|field3|field4

The next step is to format the form data into an SQL statement. The SQL INSERT syntax is:

INSERT INTO table (field1,field2…) VALUE (value1,value2…)

For the example above, field1|field2|field3|field4 needs to formatted to the values. This manipulation can be done by the bash sed command with search and replace (s) option:

$ row="field1|field2|field3|field4"
$ echo "'$row'" | sed "s/|/','/g"
'field1','field2','field3','field4'

The bash script to present the zenity form and input the data is below. The if statement is used to ensure that the cancel button wasn’t pressed. More if statements would probably be required for some data validation.

# zen_sqlin.sh - create a form to add a new user into a SQLite3 database
row=$(zenity --forms --title="Create user" --text="Add new user" \ --add-entry="First Name" \ --add-entry="Last Name" \ --add-entry="Age" \ --add-entry="Job") if [[ -n $row ]] # Some data found then indata=$(echo "'$row'" | sed "s/|/','/g") cmd="sqlite3 someuser.db \"INSERT INTO users (Fname,Lname,Age,Job) VALUES ($indata)\"" eval $cmd echo "Added data: $indata" fi

The script is run by: bash zen_sqlin.sh

Zenity (GTk) Warning Messages

Depending on your system you might see some Zenity warning messages such as:

Gtk-Message: 15:30:52.461: GtkDialog mapped without a transient parent. This is discouraged.

I never saw this on my Raspberry Pi but I did see it on my lubuntu system. To make things cleaner the warning can be piped to the null device:

$ zenity --info --text=$(date +'%S' )   --title="Seconds Timer Test" 2>/dev/null

Some Final Comments

I really just touched the surface on what zenity can do. For more info see some of the tutorials.

For simple stuff zenity works fine. If you’re looking for a more complete command line GUI tool try YAD, for myself I’ll stick to Python.

As a side note, there is a Python library for zenity. If you’re feeling comfortable with the bash version of zenity and you only need to do simple dialogs then this might be a good fit.

littleBit Dashboards (without Cloud Bits)

littleBits is a set of electronic components that magnetically connect together. litteBits is geared towards the kids STEP market and it is available in many schools and libraries.

The littleBits company has done an excellent job making their product easy to use. There is a large variety of different “bit” modules and for Internet applications there is a Cloud Bit ($59).

I found that the Cloud Bit was very easy to get up and running, but I found it was expensive at $59 and somewhat limiting, for example you are only access 1-input and 1-output. So if you want to do 2 inputs/output you would need to purchase a second Cloud bit module.

In this blog I’d like to document how I used a $39 Arduino Bit to do 3-inputs and 3-outputs. I also had the code talk directly to a free Web Dashboard (AdaFruit).

littleBits Arduino Program

A set of commands needs to be setup between the littlebits Arduino module and the PC/Pi. In my Arduino program I referenced the ports A,B,C as inputs (on the left side), and D,E,F as outputs (on the right side).

The commands from the PC/Pi would be : reference_pin:value, for example D:255 would set the D (top left pin) at 100%. It’s important to note that Arduino inputs and outputs are scaled from 0-255.

For inputs the littleBits would send the results as pin: reference_pin:value, for example B:255 would be the result at full scale for the A0 input.

ard_abc

My  test setup had:

  • A fork bit – this meant I only needed 1 power input source
  • A slider bit (0-1) on bit D0 (A)
  • A dimmer bit (0-255) on bit A0 (B)
  • A temperature bit on bit A1 (C)
  • An LED on bit d1 (D)
  • A number bit on D5 (E)
  • a bargraph bit on D9 (F)

lb_ard_setup

Below is the littleBits Arduino program that managed the serial communications.

// littleBits_2_Dashboards - create a serial interface to read/write to a PC/Pi
//
// Command from the littleBits: (A,B,C are the left pins) 
//  A:value <- for example B:24, pin A0 (2nd input) is 24 

// Commands from the PC/Pi: (D,E,F are the right pins)
//  D:output <- for example E:128, set pin A0 to 50% (128/255)
//
String thecmd; 
String thevalue;
String theinput;
char outstring[3];

void setup() {
  //define the littleBits right side pins 1,5 and 9 
  pinMode(1, OUTPUT);
  pinMode(5, OUTPUT);
  pinMode(9, OUTPUT);
  // define the littleBits left side inputs
  pinMode(0, INPUT);
  pinMode(A0, INPUT);
  pinMode(A1, INPUT);
  
  Serial.begin(9600); // this needs to match the PC/Pi baud rate
}
void loop() { 
  if (Serial.available() > 0) {
    thecmd = Serial.readStringUntil("\n"); 
    if (thecmd.length() > 2) { // ensure the msg size is big enough
      thevalue = thecmd.substring(2);
      if (thecmd.startsWith("D")) { analogWrite(1,thevalue.toInt()); }
      if (thecmd.startsWith("E")) { analogWrite(5,thevalue.toInt()); }
      if (thecmd.startsWith("F")) { analogWrite(9,thevalue.toInt()); }
    }     
  }
  // Try 3 different inputs: d0 = on/off , A0 = pot, A1 = temp sensor

  sprintf(outstring,"%d",digitalRead(0));
  Serial.write("A:");
  Serial.write(outstring);
  Serial.write("\n");

  sprintf(outstring,"%d",analogRead(A0));
  Serial.write("B:");
  Serial.write(outstring);
  Serial.write("\n");

// A1 is an "i12" littleBits temperature sensor
  int temp = analogRead(A1);
  temp = map(temp,0,1023,0,99); //rescale. Sensor range is 0-99 C or F
  sprintf(outstring,"%d",temp);
  Serial.write("C:");
  Serial.write(outstring);
  Serial.write("\n");
  

  delay(5000);
}

The Arduino IDE “Serial Monitor” can be used to view the output and set values.

msgbox

Python on the PC or Raspberry Pi

The Arduino program will send input data for A,B,C every 5 seconds. This input can be seen in Python by:

#
# littleBits Read Test
#
import serial

ser = serial.Serial(port='/dev/ttyACM1', baudrate=9600) # format for Linux
#ser = serial.Serial(port='COM1', baudrate=9600) # format for Windows

while True:
    inline = ser.readline()
    inline = inline.decode() # make a string
    pin = inline[0:1] # the first character is the pin
    thevalue = inline[2:-1] # the value is between ":" and "\n"<span id="mce_SELREST_start" style="overflow:hidden;line-height:0;"></span>
    print(pin,thevalue)

The output will look something like:

A  1
B  1023
C  21

To write commands from Python:

Write
#
# littleBits Write Test
#
import serial

ser = serial.Serial(port='/dev/ttyACM2', baudrate=9600) # format for Linux
#ser = serial.Serial(port='COM1', baudrate=9600) # format for Windows

while True:
    print("\nWrite an output value to littleBit")
    out = input("Enter pin:value, pin=A,B,C example: 'E:255' : ")
    out = out.upper() + "\n"
    out2 = out.encode('utf_8')
    ser.write(out2)

Adafruit Dashboards

There are lots of good free dashboards. For this project I used the Adafruit site. To get started you will need to log in and create a free account.

I’ve bought a number of components from Adafruit. I think that they are an excellent company that goes out of their way to create great user guides and products.

To get started with Adafruit Dashboards see: https://github.com/adafruit/Adafruit_IO_Python

The first step is to add some Adafruit tags that the code can read/write to.

Ada_feeds

In the Python code a couple of dictionaries (lb_inputs, lb_outputs)  were created to link the littleBit references (A-F) with the Adafruit tags. Also two dictionaries (lb_inlast, lb_outlast) are used to minimize communications traffic so that only new values were written.

#
# Import standard python modules
import time, random
import serial

# import Adafruit IO REST client
from Adafruit_IO import Client, Feed, RequestError

ADAFRUIT_IO_USERNAME = "put_your_username_here"
ADAFRUIT_IO_KEY = "c039f24ecb6...xxxxx"

aio = Client(ADAFRUIT_IO_USERNAME, ADAFRUIT_IO_KEY)

# Create dictionaries of inputs, output, and last values
lb_inputs = {"A":"lb-slide", "B":"lb-dimmer","C": "lb-temp"}
lb_inlast = {"A":0, "B":0,"C": 0}
lb_outputs = {"D":"lb-led", "E":"lb-number", "F":"lb-bar"}
lb_outlast = {"D":0, "E":0,"F": 0}

# Setup the serial port connection
ser = serial.Serial(port='/dev/ttyACM1', baudrate=9600)

while True:
    # Get values from littleBits and write to the dashboard
    inline = ser.readline()
    inline = inline.decode() #inline should look like: A:125\n
    pin = inline[0:1] # pin is the first character in string
    thevalue = inline[2:-1] # value is between ":" and "\n"
    if lb_inlast[pin] != thevalue: # Write only new values
        print(pin,thevalue, lb_inputs[pin])
        ada_item = aio.feeds(lb_inputs[pin])
        aio.send(ada_item.key,thevalue)
        lb_inlast[pin] = thevalue

    thetag = 'lb-slide'
    # Write new dash values to littleBits if they've changed
    for lbtag, dashtag in lb_outputs.items():
        print (lbtag,dashtag)
        thevalue = aio.receive(dashtag).value
        if lb_outlast[lbtag] != thevalue: # Write only new values
            outstr = lbtag + ":" + thevalue + "\n"
            print(outstr)
            ser.write(outstr.encode('utf_8'))
            lb_outlast[lbtag] = thevalue   

    time.sleep(2)

If everything is working correctly then new values should be written to in both directions. On the Adafruit Web site the Feed page should show the new values.

To make things look more presentable Adafruit Dashboards can be used.

ada_dash

Final Comments

In this project I used the Adafruit API, other good platforms would be IFTTT and Node-Red

Micro:bits and Node-Red

BBC Micro Bit, (micro:bit) is an open source hardware ARM-based embedded system designed by the BBC for use in computer education in the UK. The device is half the size of a credit card and has an ARM Cortex-M0 processor, accelerometer and magnetometer sensors, Bluetooth and USB connectivity, a display consisting of 25 LEDs, and two programmable button.

Depending on where you purchase it the price ranges between $15-$20. So this is a very attractive module for the beginning programmer.

The micro:bit module has 2 buttons to interface to it and a small 5×5 LED screen. This is good for small tests but its a little limiting.

For the most part micro:bit is a standalone unit so in this blog I wanted to show how to put micro:bits information on to a Node-Red web dashboard that could be viewed from a smart phone, tablet or PC.

mp_nr_overview

Micro:bits Setup

The micro:bits has a USB connection that can be used for communications to PCs or Raspberry Pi’s. For my setup I used a Raspberry Pi Zero W, with a microUSB-to-USB adapter to connect into the micro:bit.

mp_pi_setup

The micro:bit can be programmed via a nice Web Interface, for details see: https://microbit.org/guide/quick/. For this application I programmed with blocks.

My logic had the temperature and light sensor values written out ever 10 seconds, in the format of: T=xxx, L=xxx, I used a comma separator between the data pieces. Button presses were sent as either A=1, or B=1, .

mp_usb_logic

 

Node-Red Setup

Node-Red is pre-install on the Raspberry Pi image, if you want to use a PC instead see the Node-Red installation documentation.

A Node-Red has a Serial port component (https://flows.nodered.org/node/node-red-node-serialport) that can be loaded manually or via the Palette Manager.

The first step is to insert a serial input node and define the serial interface. Double-click on the serial input node and edit the serial connection. The interface will vary with your setup but node-red will show a list of possible USB ports. The default baud rate of the micro:bits USB port is 115200. I used a timeout of 200ms to get the messages, but you could also look for a terminating character (the comma “,” could be used).

nr_serial_edit

The logic used 4 Javascript function nodes to parse the micro:bit message

nr_serial_logic

“Get Temp Value” Function:


// Pull out the temperature
//
var themsg = msg.payload;

if (themsg.indexOf("T=") > -1) {

var msgitems = themsg.split(",");

var temp = msgitems[0];
temp = temp.substring(2,4)
msg.payload = temp;
return msg;
}

“Get Light Value” Function:

// Pull out the Light Sensor Value
//
var themsg = msg.payload;</pre>
if (themsg.indexOf("T=") &gt; -1) {

var msgitems = themsg.split(",");

var light = msgitems[1];

light = light.substring(2,5)

msg.payload = light;

return msg;

}

“Check Button A” Function:

// If the message is Button A pressed
// "A=1,"
if (msg.payload == "A=1,") {
msg.payload =  1;
return msg;
}

“Check Button B” Function:

// If the message is Button B pressed
// "B=1,"
if (msg.payload == "B=1,") {
msg.payload =  1;
return msg;
}

Chart nodes are used to show the results. (Note: you’ll need to create a dashboard name).

For the button presses a 1-0 transition is needed after a button press, otherwise the chart will always show a value of 1. The 0-1 transition is done using a trigger node.

The final web dashboard is available at: http://your_node_red_ip:1880/UI.

mp_screen

Final Comments

The next step will be to add the ability to have Node-Red write values to the micro:bit. This would be done with the Node-Red serial output node. Micro:bit’s have a serial read function that would then process the command.

Apache Kafka with Node-Red

Apache Kafka is a distributed streaming and messaging system. There are a number of other excellent messaging systems such as RabbitMQ and MQTT. Where Kafka is being recognized is in the areas of high volume performance, clustering and reliability.

Like RabbitMQ and MQTT, Kafka messaging are defined as topics. Topics can be produced (published) and consumed (subscribed). Where Kafka differs is in the storage of messages. Kafka stores all produced topic messages up until a defined time out.

Node-Red is an open source visual programming tool that connects to Raspberry Pi hardware and it has web dashboards that can be used for Internet of Things presentations.

In this blog I would like to look at using Node-Red with Kafka for Internet of Things type of applications.

Getting Started

Kafka can be loaded on a variety of Unix platforms and Windows.  A Java installation is required for Kafka to run, and it can be installed on an Ubuntu system by:

apt-get install default-jdk

For Kafka downloads and installation instructions see: https://kafka.apache.org/quickstart. Once the software is installed and running there a number of command line utilities in the Kafka bin directory that allow you to do some testing.

To test writing messages to a topic called iot_test1, use the kafka-console-producer.sh  command and enter some data (use Control-C to exit):

bin/kafka-console-producer.sh --broker-list localhost:9092 --topic iot_test1
11
22
33

To read back and listen for messages:

 bin/kafka-console-consumer.sh --bootstrap-server localhost:9092 --topic iot_test1 --from-beginning
11
22
33

The Kafka server is configured in the /config/server.properties  file. A couple of the things that I tweeked in this file were:

# advertised the Kafka server node ip
advertised.listeners=PLAINTEXT://192.168.0.116:9092
# allow topics to be deleted
delete.topic.enable=true

Node-Red

Node-Red is a web browser based visual programming tool, that allows users to create logic by “wiring” node blocks together.  Node-Red has a rich set of add-on components that includes things such as: Raspberry Pi hardware, Web Dash boards, email, Tweeter, SMS etc.

Node-Red has been pre-installed on Raspbian since 2015. For full installation instructions see:  https://nodered.org/#get-started

To add a Node-Red component select the “Palette Manager”, and in the Install tab search for kafka. I found that the node-red-contrib-kafka-manager component to be reliable (but there are others to try).

For my test example I wanted to create a dashboard input that could be adjusted. Then read back the data from the Kafka server and show the result in a gauge.

This logic uses:

  • Kafka Consumer Group – to read a topic(s) from a Kafka server
  • Dashboard Gauge – to show the value
  • Dashboard Slider – allows a user to select a numeric number
  • Kafka Producer – sends a topic message to the Kafka server

nodered_kafka Double-click on the Kafka nodes and in the ‘edit configuration’ dialog create and define a Kafka broker (or server). Also add the topic that you wish to read/write to.

kafka_consume

Double-click on the gauge and slider nodes and define a Dashboard group. Also adjust the labels, range and sizing to meet your requirements.

kafka_gauge

After the logic is complete hit the Deploy button to run the logic. The web dashboard is available at: http://your_node_red_ip:1880/ui.

kafka_phone

Final Comment

I found Node-Red and Kafka to be easy to use in a simple standalone environment. However when I tried to connect to a Cloud based Kafka service (https://www.cloudkarafka.com/) I quickly realized that there is a security component that needs to be defined in Node-Red. Depending on the cloud service that is used some serious testing will probably be required.

 

Simple Terminal Interfaces

Typically our interfaces for projects use colorful web pages or custom GUIs. However there are many cases where a simple text interface is all that is required. This is especially true for SSH or remote connections from a Window’s client into a Raspberry Pi or Linux server.

In this blog I’d like to review a 1980’s technology called curses, with three examples. The first example will be simulated Rasp Pi scanning app in “C” and Python. The second and third examples will be in Python and they will show large text presentation and dynamic bars.

Python Curses

Python curses are standard in Python, and they include features such as:

  • support ASCII draw characters
  • basic color support
  • window and pad objects which can be written to and cleared independently

As a first example I wanted to have a colored background, a header and footer and some dynamic text.

curses_text

The first step is to define a curses main screen object (stdscr). The next step is to enable color and to create some color pairs. Using color pairs and the screen size (height, width = stdscr.getmaxyx()) it is possible to add a header and footer strip using the srtdscr.addstr command.

The stdscr.nodelay command allow the program to cycle until the stdscr.getch() call returns a key.

# curses_text.py - create a curses app with 10 dynamic values
#
import curses , time, random

# create a curses object
stdscr = curses.initscr()
height, width = stdscr.getmaxyx() # get the window size

# define two color pairs, 1- header/footer , 2 - dynamic text, 3 - background
curses.start_color()
curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)
curses.init_pair(3, curses.COLOR_WHITE, curses.COLOR_BLUE)

# Write a header and footer, first write colored strip, then write text
stdscr.bkgd(curses.color_pair(3))
stdscr.addstr(0, 0, " " * width,curses.color_pair(1) )
stdscr.addstr(height-1, 0, " " * (width - 1),curses.color_pair(1) )
stdscr.addstr(0, 0, " Curses Dynamic Text Example" ,curses.color_pair(1) )
stdscr.addstr(height-1, 0, " Key Commands : q - to quit " ,curses.color_pair(1) )
stdscr.addstr(3, 5, "RASPBERRY PI SIMULATED SENSOR VALUES" ,curses.A_BOLD )
stdscr.refresh()

# Cycle to update text. Enter a 'q' to quit
k = 0
stdscr.nodelay(1)
while (k != ord('q')):
# write 10 lines text with a label and then some random numbers
for i in range(1,11):
    stdscr.addstr(4+ i, 5, "Sensor " + str(i) + " : " ,curses.A_BOLD )
    stdscr.addstr(4+ i, 20, str(random.randint(10,99)) ,curses.color_pair(2) )
    time.sleep(2)
    k = stdscr.getch()

curses.endwin()

The simulated Pi values will refresh every  10 seconds until the “q” key is pushed and then the terminal setting are returned to normal (curses.endwin()) and the program exits.

“C” Curses Example

For this “C” example I used a Raspberry Pi. The curses library needs to be installed by:

 sudo apt-get install libncurses5-dev

The curses syntax is similar between “C” and Python but not 100%. For example in Python the addstr command includes a color pair reference, but in “C” this is not supported so an attribute on/off (attron/attroff) command is used to reference the color pair. Below is the “C” code:

/* c1.c - Basic Curses Example */

#include <curses.h>
#include <stdlib.h>
#include <unistd.h>

int main(void)
{
    int row, col, k;    
// Create a curses object and define color pairs
    initscr();
    getmaxyx(stdscr,row,col);
    start_color();
    init_pair(1,COLOR_RED,COLOR_WHITE);
    init_pair(2,COLOR_GREEN,COLOR_BLACK);
    init_pair(3,COLOR_WHITE,COLOR_BLUE);
    curs_set(0);
    noecho();
    //keypad(stdscr,TRUE);
    nodelay(stdscr, TRUE);
// Write a header and footer, first write colored strip, then write text
    bkgd(COLOR_PAIR(3));
    attron(COLOR_PAIR(1));
// Create a top and bottom color strip
    for (int i = 0; i < col; i++) {
        mvaddstr(0, i,  " ");
        mvaddstr(row-1, i,  " ");
    }
    mvaddstr(0, 0,  " Curses C Dynamic Text Example");
    mvaddstr(row-1, 0,  " Key Commands: q - to quit");
    attroff(COLOR_PAIR(1));   
    mvaddstr(2, 5,"RASPBERRY PI SIMULATED SENSOR VALUES" );
    refresh();
// Cycle with new values every 2 seconds until a q key (133) is entered    
    while (k != 113)
    {
        attroff(COLOR_PAIR(2));
        for (int i = 0; i < 10; i++) {
            mvprintw((4+i), 5,  " Sensor %d : ",i);
        }
        attron(COLOR_PAIR(2));
        for (int i = 0; i < 10; i++) {
            mvprintw((4+i), 20,  "%d",rand() %100);
        }
        k = getch();
        sleep(2);
    }
    endwin();
    exit(0);
}

To compile and run the program (c1.c) enter:

gcc -o c1 c1.c -lncurses
./c1

The “C” example should look very similar to the earlier Python example.

Figlet for Large Custom Text

Large Custom Text can be generated using the Python Figlet library.  Figlet has an extensive selection of text presentations and it uses standard ASCII character to generate the large text presentations. The Figlet library is installed by:

pip install pyfiglet

An example from the Python shell:

pyshell_figlet

For a Figlet example, I wanted to create a large heading and a large dynamic value.

curses_di

The Figlet library can be used to generate a string with user defined texted presented a large text-like format. A little bit of testing is required because the Figlet generated text can be 3,4,5 or more characters tall and the string needs to be added to very left end of the window.

# curses_di.py - show a large heading and large dynamic value
#
import curses, time
import pyfiglet, random

def get_io():
    global value1
    testvalue = str(random.randint(100,1000)/10) + " C"
    value1 = pyfiglet.figlet_format(testvalue, font = "starwars" )

# Create a string of text based on the Figlet font object
title = pyfiglet.figlet_format("Raspi Data", font = "small" ) 

stdscr = curses.initscr() # create a curses object
# Create a couple of color definitions
curses.start_color()
curses.init_pair(1, curses.COLOR_YELLOW, curses.COLOR_BLACK)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)

# Write the BIG TITLE text string
stdscr.addstr(1,0, title,curses.color_pair(1) )
stdscr.addstr(8,0, "Sensor 1: GPIO 7 Temperature Reading" ,curses.A_BOLD)

# Cycle getting new data, enter a 'q' to quit
stdscr.nodelay(1)
k = 0
while (k != ord('q')):
    get_io() # get the data values
    stdscr.addstr(10,0, value1,curses.color_pair(2) )
    stdscr.refresh()
    time.sleep(2)

    k = stdscr.getch()

curses.endwin()

I found that the the small and doom fonts worked well in my testing. To check out and test Figlet fonts online see:

http://patorjk.com/software/taag/#p=display&f=Slant&t=Dude%20what%20are%20you%20doing%20%3F

Curses Windows

By defining a curses window it is possible to clear and write to a window that it is independent from the background. The syntax to create a curses window object is:

mynewwindow = curses.newwin(height, width, begin_y, begin_x)

Windows are ideal for applications where multiple items such as Figlet objects are used. Below is an example with two large Figlet values.

Figlet2win


# Create a static 2 large values example
#
import curses, time
import pyfiglet, random
# Create a string of text based on the Figlet font object
title = pyfiglet.figlet_format("Weather Station 2", font = "small" )

stdscr = curses.initscr() # create a curses object
# Create a couple of color definitions
curses.start_color()
curses.init_pair(1, curses.COLOR_YELLOW, curses.COLOR_BLACK)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)

# Write the BIG TITLE text string
stdscr.addstr(1,0, title,curses.color_pair(1) )
stdscr.refresh()

win1 = curses.newwin(9, 44, 6, 4)
win1.addstr(8,0, "Sensor 1: Temperature Reading" ,curses.A_BOLD)

win2 = curses.newwin(9, 44, 6, 50)
win2.addstr(8,0, "Sensor 2: Humidity Reading" ,curses.A_BOLD)
value1 = pyfiglet.figlet_format("23 C", font = "doom" )
win1.addstr(0,0,value1,curses.color_pair(2) )
win1.refresh()
value2 = pyfiglet.figlet_format("35 %", font = "doom" )
win2.addstr(0,0, value2 ,curses.color_pair(2) )
win2.refresh()

# Hit any key to exit
stdscr.getch()
curses.endwin()

Dynamic Bars Example

For the Dynamic bars example I created a get_io function to simulate two real time data  values.

As a first step I created some background information such as headings, a header and a footer. By using the call: height, width = stdscr.getmaxyx() , I am able to position banners at the top and bottom of the terminal window. All of the background info is written to the stdscr object.

Two windows objects (win1 and win2) are used for the real time dynamic bars. Old bar data is removed using the win1.clear() and win2.clear() calls. Like the static example the dynamic bars are created by writing a fill character multiplied by the actual real time value (win1.addstr(1, 1, bar * value1) ). A window.refresh() command is used to show the changes.

The stdscr.getch() method is used to catch keyboard input, and the terminal program is exited when a quit character, “q” is entered.

The complete two dynamic bar program is shown below:


# Simple bar value interface
#
import curses
import time

# get_io is using random values, but a real I/O handler would be here
def get_io():
    import random
    global value1, value2
    value1 = random.randint(1,30)
    value2 = random.randint(1,30)

bar = '█' # an extended ASCII 'fill' character
stdscr = curses.initscr()
height, width = stdscr.getmaxyx() # get the window size
curses.start_color()
curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)
curses.init_pair(3, curses.COLOR_YELLOW, curses.COLOR_BLACK)

# layout the header and footer
stdscr.addstr(1,1, " " * (width -2),curses.color_pair(1) )
stdscr.addstr(1,15, "Raspberry Pi I/O",curses.color_pair(1) )
stdscr.hline(2,1,"_",width)
stdscr.addstr(height -1,1, " " * (width -2),curses.color_pair(1) )
stdscr.addstr(height -1,5, "Hit q to quit",curses.color_pair(1) )

# add some labels
stdscr.addstr(4,1, "Pi Sensor 1 :")
stdscr.addstr(8,1, "Pi Sensor 2 :")

# Define windows to be used for bar charts
win1 = curses.newwin(3, 32, 3, 15) # curses.newwin(height, width, begin_y, begin_x)
win2 = curses.newwin(3, 32, 7, 15) # curses.newwin(height, width, begin_y, begin_x)

# Use the 'q' key to quit
k = 0
while (k != ord('q')):
    get_io() # get the data values
    win1.clear()
    win1.border(0)
    win2.clear()
    win2.border(0)
# create bars bases on the returned values
    win1.addstr(1, 1, bar * value1, curses.color_pair(2))
    win1.refresh()
    win2.addstr(1, 1, bar * value2 , curses.color_pair(3))
    win2.refresh()
# add numeric values beside the bars
    stdscr.addstr(4,50, str(value1) + " Deg ",curses.A_BOLD )
    stdscr.addstr(8,50, str(value2) + " Deg ",curses.A_BOLD )
    stdscr.refresh()
    time.sleep(2)
    stdscr.nodelay(1)
    k = stdscr.getch() # look for a keyboard input, but don't wait

curses.endwin() # restore the terminal settings back to the original

cbars

For testing I used a random simulator for the data but the get_io function could be easily configured to connect to a Raspberry Pi or Arduino module.

The outline boxes in the window object could look strange if you are using a Window’s based SSH client like Putty. To create the problem in Putty’s settings, select: Window ->  Translations and use VSCII as the remote character set.

putty

Final Comments

Curses is definitely an ‘old school’ technology but it offers a simple solution for SSH and terminal based connections.

Pi Sailboat

My daughters and I have built a number of boat projects with an assortment of Arduino, ESP-8266, Bluetooth and RFI components. I believe that this version using a Raspberry Pi and NodeRed offers one of the simplest solutions. This sailboat used a basic catamaran design with a Raspberry Pi mounting inside a waterproof container. Using NodeRed dashboards you can control the sailboat’s rudder from a smart phone. The complete NodeRed logic consisted of only 6 nodes.

Building the Sailboat

There are a lot of different building materials that you could choose from. K’Nex construction pieces are lighter than either Lego or Meccano and they allow you to create reasonably large structures with a minimal number of pieces. If you do not have access to K’Nex pieces then popsicle sticks and some card board would offer a good low cost solution.

To build the sailboat we used:
• K’Nex building pieces
• 4 plastic bottles
• 1 small plastic container with a lid
• String
• Duct tape
• Garbage bag
• Low torque servo
• Raspberry Pi Zero W or Pi 3
• Small USB phone charger

The base of the sailboat was a rectangular structure with 16 down facing K’Nex pieces that allowed plastic bottles to be duct taped in place.

boat_bottom

A few K’Nex pieces were used to create a compartment for the servo, and wire was used to secure the servo in place. A rudder was built by screwing a small piece of wood into the servo arm.

servobox

A garbage bag was cut to the required size and taped to the mast. The boom had a swivel connection to the mast and guide ropes were connected to both the boom and mast.

sailboat_details

Servo and Rudder Setup

Only very low torque servos can connected directly to Rasberry Pi GPIO pins.

Pi_servo_wiring

An example of a low torque servo would be the TowerPro SG90 ($4) that has a torque of 25.00 oz-in (1.80 kg-cm). If you have larger torque servos you will need to either use a custom Raspberry Pi servo hat (there are some good ones on the market), or you will need to use a separate power and ground circuit for the servo.

The wiringPi tool gpio can be used to control the servo. This package is pre-install in the Raspbian image, or it can be manually installed by:

sudo apt-get install -y wiringpi

Servos typically want a pulse frequency of 50 Hz, and the Raspberry Pi PWM (Pulse Width Modulation) pins have a frequency of 19200 Hz, so some range definitions and scaling is required:

gpio -g mode 18 pwm #define pin 18 as the PWM pin
gpio pwm-ms #use 'mark space' mode 
gpio pwmc 192 # set freq as 19200
gpio pwmr 2000 # use a range of 2000

The gpio pwm commands are not persistent after a reboot. A simple solution for this is to put these commands in the Pi user login file of: $HOME/.bash_login.

After the pwm setup commands are run you need to do some manual testing to define your different rudder (servo) positions (Figure 6), such as “Hard Left”, “Hard Right”, “Easy Left”, “Easy Right” and “Straight”. The pwr timing numbers will vary based on your requirements and servo arm positioning, for our sailboat we used:

gpio -g pwm 18 200 #straight
gpio -g pwm 18 260 #hard left
gpio -g pwm 18 140 #hard right
gpio -g pwm 18 230 #easy left
gpio -g pwm 18 170 #easy right

servo_settings

NodeRed Logic and Dashboards

NodeRed is pre-installed on the Raspbian image, but it will need to be set to autostart on a Pi reboot:  sudo systemctl enable nodered.service

NodeRed has a web configuration interface that is accessed by: http://localhost:1880 or http://pi_ip_address:1880.

On the options button (far right), by selecting: View -> Dashboard , you can define and change the web dashboard layouts.

dashboard

To create logic, nodes are selected from the left node panel and dragged and dropped on to the center flow panel. Logic flow are then created by clicking and joining together different inputs and outputs on the nodes. If a dashboard node is dropped on the flow panel it will be added to the default web dashboard. The gpio -g pwm commands can be called using the exec node. The button dashboard node will pass the defined payload value, for example a “Hard Left” 260 is passed when the button is pushed. The button’s payload value will be appended to the exec command to make a complete gpio -g pwm servo position command.

nodered

Once you’ve completed your logic setup press the Deploy button on the top right to make your configuration live and ready to test.

The final step is to enable a smart phone or tablet to connect to the Raspberry Pi, this can be done by either making the Raspberry Pi a WiFi access point or by tethering the Pi to a cell phone. There are some great guides on how to setup a Raspberry Pi as an access point. For this project the simple tethering method was used. Once the Pi is tethered to a phone, the PI’s IP address can be obtained from the hotspot users list.

pi_address

The NodeRed dashboard is accessed on your phone by: http://pi_ip_address:1880/ui .

nodered_ui

Assuming that everything is connected correctly you should be able to control the sailboard with your phone.

Summary

Once you’ve mastered the basic NodeRed and sailboat construction other projects such as motor boats, iceboats, airboats are possible.

airboat

 

 

 

Pi/Node-Red Car

The goal of the Pi/Node-Red car project was to create a small vehicle that can be controlled from a smart phone . For the project we used:

  • 1 Car chassis for Arduino ($15)
  • 1 Pimoroni Explorer HAT Pro  ($23)
  • 1 Portable microUSB charger
  • 1 USB WiFi Adapter
  • 4 short alligator clips and 4 connectors
  • Duct tape

The Arduino car chassis may require a small amount of assembly. Rather than soldering connections we like to use short alligator clips. It is not recommended to wire DC motors directly to a Raspberry Pi so the Pimoroni Explorer HAT Pro is used to connect the 2 DC motors.

The Raspberry Pi and the portable microUSB charger are secured to the top of the car chassis with duct tape. The left motor is wired to the motor 1 connectors on the Explorer Hat, and the right motor is wired to motor 2 connectors. Note you may have to do a little trial and error on the Explorer HAT “+” and “-” motor connections to get both wheels spinning in a forward direction.

The Explorer HAT Node-Red library is installed by:

 cd $HOME/.node-red
npm install node-red-dashboard 

The Web dashboard presentation is configured in the “dashboard” tab. For this example we create 2 groups: a control group to drive the vehicle, and a light group to turn on the Explorer Pro lights. Use the “+group” button to add a group, and the “edit” to change an existing group.
dash_conf

To control a motor, an “Explorer HAT” node and a dashboard button node are dropped and connected together. All the configuration is done in the button node . The button configure options are:

  • the group the button will appear in (Controls)
  • the size of the button (3×1 = 50% of width and narrow)
  • Topic, motor.one or motor.twois used for motor control
  • Payload, -100 = reverse, 0=stop, 100 = forward

Control_conf

The Explorer HAT has 4 colored LEDs. To toggle the LEDS, the topic is light.color with 1=ON, and 0=OFF . We thought that it would be fun to also add some Web dashboard button to control the colored lights.

light_conf

The Node-Red dashboard user interface is accessed by: ipaddress:1880/UI, so for example 192.168.1.102:1880/ui. Below is a picture that shows the final vehicle logic and the Web dashboard.

 

final_logic2