Simple Terminal Interfaces

Typically our interfaces for projects use colorful web pages or custom GUIs. However there are many cases where a simple text interface is all that is required. This is especially true for SSH or remote connections from a Window’s client into a Raspberry Pi or Linux server.

In this blog I’d like to review a 1980’s technology called curses, with three examples. The first example will be simulated Rasp Pi scanning app in “C” and Python. The second and third examples will be in Python and they will show large text presentation and dynamic bars.

Python Curses

Python curses are standard in Python, and they include features such as:

  • support ASCII draw characters
  • basic color support
  • window and pad objects which can be written to and cleared independently

As a first example I wanted to have a colored background, a header and footer and some dynamic text.

curses_text

The first step is to define a curses main screen object (stdscr). The next step is to enable color and to create some color pairs. Using color pairs and the screen size (height, width = stdscr.getmaxyx()) it is possible to add a header and footer strip using the srtdscr.addstr command.

The stdscr.nodelay command allow the program to cycle until the stdscr.getch() call returns a key.

# curses_text.py - create a curses app with 10 dynamic values
#
import curses , time, random

# create a curses object
stdscr = curses.initscr()
height, width = stdscr.getmaxyx() # get the window size

# define two color pairs, 1- header/footer , 2 - dynamic text, 3 - background
curses.start_color()
curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)
curses.init_pair(3, curses.COLOR_WHITE, curses.COLOR_BLUE)

# Write a header and footer, first write colored strip, then write text
stdscr.bkgd(curses.color_pair(3))
stdscr.addstr(0, 0, " " * width,curses.color_pair(1) )
stdscr.addstr(height-1, 0, " " * (width - 1),curses.color_pair(1) )
stdscr.addstr(0, 0, " Curses Dynamic Text Example" ,curses.color_pair(1) )
stdscr.addstr(height-1, 0, " Key Commands : q - to quit " ,curses.color_pair(1) )
stdscr.addstr(3, 5, "RASPBERRY PI SIMULATED SENSOR VALUES" ,curses.A_BOLD )
stdscr.refresh()

# Cycle to update text. Enter a 'q' to quit
k = 0
stdscr.nodelay(1)
while (k != ord('q')):
# write 10 lines text with a label and then some random numbers
for i in range(1,11):
    stdscr.addstr(4+ i, 5, "Sensor " + str(i) + " : " ,curses.A_BOLD )
    stdscr.addstr(4+ i, 20, str(random.randint(10,99)) ,curses.color_pair(2) )
    time.sleep(2)
    k = stdscr.getch()

curses.endwin()

The simulated Pi values will refresh every  10 seconds until the “q” key is pushed and then the terminal setting are returned to normal (curses.endwin()) and the program exits.

“C” Curses Example

For this “C” example I used a Raspberry Pi. The curses library needs to be installed by:

 sudo apt-get install libncurses5-dev

The curses syntax is similar between “C” and Python but not 100%. For example in Python the addstr command includes a color pair reference, but in “C” this is not supported so an attribute on/off (attron/attroff) command is used to reference the color pair. Below is the “C” code:

/* c1.c - Basic Curses Example */

#include <curses.h>
#include <stdlib.h>
#include <unistd.h>

int main(void)
{
    int row, col, k;    
// Create a curses object and define color pairs
    initscr();
    getmaxyx(stdscr,row,col);
    start_color();
    init_pair(1,COLOR_RED,COLOR_WHITE);
    init_pair(2,COLOR_GREEN,COLOR_BLACK);
    init_pair(3,COLOR_WHITE,COLOR_BLUE);
    curs_set(0);
    noecho();
    //keypad(stdscr,TRUE);
    nodelay(stdscr, TRUE);
// Write a header and footer, first write colored strip, then write text
    bkgd(COLOR_PAIR(3));
    attron(COLOR_PAIR(1));
// Create a top and bottom color strip
    for (int i = 0; i < col; i++) {
        mvaddstr(0, i,  " ");
        mvaddstr(row-1, i,  " ");
    }
    mvaddstr(0, 0,  " Curses C Dynamic Text Example");
    mvaddstr(row-1, 0,  " Key Commands: q - to quit");
    attroff(COLOR_PAIR(1));   
    mvaddstr(2, 5,"RASPBERRY PI SIMULATED SENSOR VALUES" );
    refresh();
// Cycle with new values every 2 seconds until a q key (133) is entered    
    while (k != 113)
    {
        attroff(COLOR_PAIR(2));
        for (int i = 0; i < 10; i++) {
            mvprintw((4+i), 5,  " Sensor %d : ",i);
        }
        attron(COLOR_PAIR(2));
        for (int i = 0; i < 10; i++) {
            mvprintw((4+i), 20,  "%d",rand() %100);
        }
        k = getch();
        sleep(2);
    }
    endwin();
    exit(0);
}

To compile and run the program (c1.c) enter:

gcc -o c1 c1.c -lncurses
./c1

The “C” example should look very similar to the earlier Python example.

Figlet for Large Custom Text

Large Custom Text can be generated using the Python Figlet library.  Figlet has an extensive selection of text presentations and it uses standard ASCII character to generate the large text presentations. The Figlet library is installed by:

pip install pyfiglet

An example from the Python shell:

pyshell_figlet

For a Figlet example, I wanted to create a large heading and a large dynamic value.

curses_di

The Figlet library can be used to generate a string with user defined texted presented a large text-like format. A little bit of testing is required because the Figlet generated text can be 3,4,5 or more characters tall and the string needs to be added to very left end of the window.

# curses_di.py - show a large heading and large dynamic value
#
import curses, time
import pyfiglet, random

def get_io():
    global value1
    testvalue = str(random.randint(100,1000)/10) + " C"
    value1 = pyfiglet.figlet_format(testvalue, font = "starwars" )

# Create a string of text based on the Figlet font object
title = pyfiglet.figlet_format("Raspi Data", font = "small" ) 

stdscr = curses.initscr() # create a curses object
# Create a couple of color definitions
curses.start_color()
curses.init_pair(1, curses.COLOR_YELLOW, curses.COLOR_BLACK)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)

# Write the BIG TITLE text string
stdscr.addstr(1,0, title,curses.color_pair(1) )
stdscr.addstr(8,0, "Sensor 1: GPIO 7 Temperature Reading" ,curses.A_BOLD)

# Cycle getting new data, enter a 'q' to quit
stdscr.nodelay(1)
k = 0
while (k != ord('q')):
    get_io() # get the data values
    stdscr.addstr(10,0, value1,curses.color_pair(2) )
    stdscr.refresh()
    time.sleep(2)

    k = stdscr.getch()

curses.endwin()

I found that the the small and doom fonts worked well in my testing. To check out and test Figlet fonts online see:

http://patorjk.com/software/taag/#p=display&f=Slant&t=Dude%20what%20are%20you%20doing%20%3F

Curses Windows

By defining a curses window it is possible to clear and write to a window that it is independent from the background. The syntax to create a curses window object is:

mynewwindow = curses.newwin(height, width, begin_y, begin_x)

Windows are ideal for applications where multiple items such as Figlet objects are used. Below is an example with two large Figlet values.

Figlet2win


# Create a static 2 large values example
#
import curses, time
import pyfiglet, random
# Create a string of text based on the Figlet font object
title = pyfiglet.figlet_format("Weather Station 2", font = "small" )

stdscr = curses.initscr() # create a curses object
# Create a couple of color definitions
curses.start_color()
curses.init_pair(1, curses.COLOR_YELLOW, curses.COLOR_BLACK)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)

# Write the BIG TITLE text string
stdscr.addstr(1,0, title,curses.color_pair(1) )
stdscr.refresh()

win1 = curses.newwin(9, 44, 6, 4)
win1.addstr(8,0, "Sensor 1: Temperature Reading" ,curses.A_BOLD)

win2 = curses.newwin(9, 44, 6, 50)
win2.addstr(8,0, "Sensor 2: Humidity Reading" ,curses.A_BOLD)
value1 = pyfiglet.figlet_format("23 C", font = "doom" )
win1.addstr(0,0,value1,curses.color_pair(2) )
win1.refresh()
value2 = pyfiglet.figlet_format("35 %", font = "doom" )
win2.addstr(0,0, value2 ,curses.color_pair(2) )
win2.refresh()

# Hit any key to exit
stdscr.getch()
curses.endwin()

Dynamic Bars Example

For the Dynamic bars example I created a get_io function to simulate two real time data  values.

As a first step I created some background information such as headings, a header and a footer. By using the call: height, width = stdscr.getmaxyx() , I am able to position banners at the top and bottom of the terminal window. All of the background info is written to the stdscr object.

Two windows objects (win1 and win2) are used for the real time dynamic bars. Old bar data is removed using the win1.clear() and win2.clear() calls. Like the static example the dynamic bars are created by writing a fill character multiplied by the actual real time value (win1.addstr(1, 1, bar * value1) ). A window.refresh() command is used to show the changes.

The stdscr.getch() method is used to catch keyboard input, and the terminal program is exited when a quit character, “q” is entered.

The complete two dynamic bar program is shown below:


# Simple bar value interface
#
import curses
import time

# get_io is using random values, but a real I/O handler would be here
def get_io():
    import random
    global value1, value2
    value1 = random.randint(1,30)
    value2 = random.randint(1,30)

bar = '█' # an extended ASCII 'fill' character
stdscr = curses.initscr()
height, width = stdscr.getmaxyx() # get the window size
curses.start_color()
curses.init_pair(1, curses.COLOR_RED, curses.COLOR_WHITE)
curses.init_pair(2, curses.COLOR_GREEN, curses.COLOR_BLACK)
curses.init_pair(3, curses.COLOR_YELLOW, curses.COLOR_BLACK)

# layout the header and footer
stdscr.addstr(1,1, " " * (width -2),curses.color_pair(1) )
stdscr.addstr(1,15, "Raspberry Pi I/O",curses.color_pair(1) )
stdscr.hline(2,1,"_",width)
stdscr.addstr(height -1,1, " " * (width -2),curses.color_pair(1) )
stdscr.addstr(height -1,5, "Hit q to quit",curses.color_pair(1) )

# add some labels
stdscr.addstr(4,1, "Pi Sensor 1 :")
stdscr.addstr(8,1, "Pi Sensor 2 :")

# Define windows to be used for bar charts
win1 = curses.newwin(3, 32, 3, 15) # curses.newwin(height, width, begin_y, begin_x)
win2 = curses.newwin(3, 32, 7, 15) # curses.newwin(height, width, begin_y, begin_x)

# Use the 'q' key to quit
k = 0
while (k != ord('q')):
    get_io() # get the data values
    win1.clear()
    win1.border(0)
    win2.clear()
    win2.border(0)
# create bars bases on the returned values
    win1.addstr(1, 1, bar * value1, curses.color_pair(2))
    win1.refresh()
    win2.addstr(1, 1, bar * value2 , curses.color_pair(3))
    win2.refresh()
# add numeric values beside the bars
    stdscr.addstr(4,50, str(value1) + " Deg ",curses.A_BOLD )
    stdscr.addstr(8,50, str(value2) + " Deg ",curses.A_BOLD )
    stdscr.refresh()
    time.sleep(2)
    stdscr.nodelay(1)
    k = stdscr.getch() # look for a keyboard input, but don't wait

curses.endwin() # restore the terminal settings back to the original

cbars

For testing I used a random simulator for the data but the get_io function could be easily configured to connect to a Raspberry Pi or Arduino module.

The outline boxes in the window object could look strange if you are using a Window’s based SSH client like Putty. To create the problem in Putty’s settings, select: Window ->  Translations and use VSCII as the remote character set.

putty

Final Comments

Curses is definitely an ‘old school’ technology but it offers a simple solution for SSH and terminal based connections.

Pi with Neopixels/Simulated Neopixels

Neopixels are addressable full-colour RGB LEDs that come in a variety of different arrangements. Ranging from single LEDs, to matrix arrays and a variety of sewable components that can be used on wearable products.

dif_neopixels

Neopixels were originally made available for Arduino projects, but now there are also Python libraries for Raspberry Pi’s.

In this blog I will be looking at setting up neopixels components on Raspberry Pi’s, and then I will show some “soft” neopixel layouts using the Python Tkinter graphic library.

Getting Started

To load the Raspberry Pi neopixel libary comes from the nice people at Adafruit, and it is loaded by:

sudo pip3 install rpi_ws281x adafruit-circuitpython-neopixel

It is important to note that neopixels can draw a lot of power so consider looking at using external 5V power for projects with a lot of LEDs. The LED power consumption is based on:

  • How many neopixel LEDs are lit at one time, and
  • What the intensity of the LEDs is.

A few other import points are:

  • not all neopixel strips are the same. Different strips will vary greatly from the LED intensity, and more importantly on the RGB vs. GRB addressing.
  • NeoPixels must be connected to D10, D12, D18 or D21 to work
  • For NeoPixels to work on Raspberry Pi, you must run the code as root

The neopixels are wired with 3 pins : 5V (VCC), GND and signal in. The default signal in wires to Pi pin 18. Neopixel component can be connected in series with data in and out connectors.

data_in_out

Below is an example that will set all the LEDs to a light magenta and then it will cycle one LED to a brighter RED. The overall neopixel string has a 10% brightness.


# Python neopixel example to cycle an LEDs

import board
import neopixel
import time

ORDER = neopixel.RGB  # or neopixel.GRB
numpixels = 12

# Create a pixel object with 12 pixels and low intensity
pixels = neopixel.NeoPixel(board.D18,numpixels , brightness=0.10, auto_write=True, pixel_order=ORDER)
while True:
for i in range(numpixels):
   pixels.fill((10, 0, 40)) # fill all pixels in light magenta
   pixels[i] = (80,0,0) # fill one pixel in brighter red
   time.sleep(1)

Depending on the type and manufacturer of the neopixels the result will look a little different. Some trial and error testing will be required to determine if the strips are RGB or GRB.

 

Simulated Neopixels

If you don’t have neopixels or if what to simulate neopixels then the Python Tkinter graphic library can be used to create a variety of different arrangements. For my testing I create two arrangements: a strip and a matrix.

The important things that I learned was how to create a array object that could simulate the neopixel object. To do this in Python:


import tkinter as tk

root = tk.Tk()
root.title("Soft NeoPixel Strip")

numleds = 25

# Create an array that can be used later in Tkinter

ledstrip = ['' for i in range(numleds)]

for i in range(numleds):
   ledstrip[i] = tk.Label(root,relief='raised',width=3 ) # a label array
   ledstrip[i].grid(row = 0, column = i) # position the labels is a horizontal row
root.mainloop()

Simulated Strip Neopixel

Below is an example of a soft “strip” neopixel application with a demo function.

py_neo_Strip

# Python Neopixel Single Strip Presentation
#
import tkinter as tk

numleds = 25

theled = 0

def stringdemo():
    # move a coloured LED around the string
    global theled
    ledstrip[theled].configure(background= 'white')
    theled = theled + 1
    if theled >= numleds:
        theled = 0
    ledstrip[theled].configure(background= 'sky blue')
    root.after(500, stringdemo)
    
root = tk.Tk()
root.title("Soft NeoPixel Strip")

# create an LED object 
ledstrip = ['' for i in range(numleds)]

# put the LED object into a horizontal strip
for i in range(numleds):
    ledstrip[i] = tk.Label(root,width=2,height=1,relief='raised',background = 'white')
    ledstrip[i].grid(row = 0, column = (i+1))

root.after(500, stringdemo) #start a demo

root.mainloop()

Simulated Matrix Neopixels

Below is an example of a soft matrix neopixel application.

py_neo_Matrix

# Python Neopixel Matrix Presentation
#
import tkinter as tk

numleds = 100
rowcnt = 10
colcnt = int (numleds/rowcnt)

theled = 0

def stringdemo():
    # move a coloured LED around the string
    global theled
    ledstrip[theled].configure(background= 'dark gray')
    theled = theled + 1
    if theled >= numleds:
        theled = 0
    ledstrip[theled].configure(background= 'red')
    root.after(500, stringdemo)
    
    
root = tk.Tk()
root.title("Soft NeoPixel Matrix")

# create LED object
ledstrip = ['' for i in range(numleds)]

# put the LED object into a grid
for i in range(rowcnt):
    for j in range(colcnt):
        ledstrip[theled] = tk.Label(root,width=4,height=2,relief='raised',background = 'dark gray')
        ledstrip[theled].grid(row = i, column = j)
        theled = theled + 1

theled = 0 #reset the led index for the demo
root.after(500, stringdemo)

root.mainloop()

Summary

Neopixels can be used on custom lighting applicatons, for example I used them on a water fountain project.

Given a choice I would recommend using Arduino hardware over the Raspberry Pi hardware for neopixel projects. I found that the Arduino neopixel library to be much more stable and considerably faster than the Pi version.

Small Standalone Databases for IoT

For many IoT projects you want to historically save your data, however if you’re only saving a small amount of data using a database server such as MySQL can be overkill.

I wanted to come up with a way to save my data and be able to port it between projects and hardware.  My goal was to use Python because it has the best Raspberry Pi support. I looked at some options like:

  • CSV files – simple but filtering and sorting isn’t so easy
  • dBase files – good “old school” approach, however it’s no longer support in Excel and MS Access so viewing the data offline is a little ugly
  • Python Pickle and PickleDB – great for saving game scores but no real sorting
  • SQLite – excellent light weight database

For my light weight Python applications I was really happy with TinyDB. TinyDB offers:

  • a JSON storage interface, so you could easily migrate to MongoDB (or equivalent)
  • pure Python with no dependencies,  (so it runs on all OS’s and versions of Python)
  • supports queries and searches

Getting Started with TinyDB

To install TinyDB :

pip install tinydb

TinyDB is standalone so there are no server components. Your Python code talks directly to one of more JSON files, or you can also create multiple tables in one JSON.

Below is a Python example that create a JSON file with two tables.


# Use TinyDB to create a JSON file with 2 tables
#
from tinydb import TinyDB, Query

db = TinyDB('db2.json')
db.purge_tables() # Clear out the file's old data

table = db.table('Moisture')

table.insert({"date":"2019-05-06","time":"16:00","moisture":200})
table.insert({"date":"2019-05-06","time":"16:10","moisture":210})
table.insert({"date":"2019-05-06","time":"16:20","moisture":220})

table2 = db.table('Light')

table2.insert({"date":"2019-06-06","time":"16:00","light":30})
table2.insert({"date":"2019-06-06","time":"16:10","light":35})
table2.insert({"date":"2019-06-06","time":"16:20","light":36})

db.close()

#
import json

with open('db2.json') as f:
data = json.load(f)

print("JSON file with 2 tables\n")
print(json.dumps(data, indent = 4, sort_keys=True))

The JSON file will have a format like:

json_2tables

A Test Example

For a test example I wanted to simulate a month’s worth of 5 minute data. Then I would try some queries to show the data.

 
# Create a month's worth of 10 minute simulated JSON data
#
# Use a format of:
#    "Pi_1": {
#        "1": {
#            "date": "2019-04-01",
#            "time": "01:00",
#            "moisture": 577,
#            "light": 30
#        }

from tinydb import TinyDB, Query
import random

db = TinyDB('/writing/blog/tinydb/db1.json')
db.purge_tables()
table = db.table('Pi_1')

for iday in range(1,31):
    print("day: ",iday)
    for ihour in range(0,24):
        print("\thour: ", ihour)
        for imin in range(0,56,5):
            thedate = "2019-04-{:02}".format(iday)
            thetime = "{:02}:{:02}".format(ihour, imin)
            table.insert({"date": thedate,"time": thetime ,"moisture": random.randint(200,600),"light":  random.randint(20,100)})


print ("Finished updating JSON file\n\n")
db.close(

Once the JSON data file was created I used a simple Python Tkinter application to query the data by day. My goal was to see how easy it was to query the data and return results. Below is a my TinyDB Tkinter viewing application and code. The bottom slider is used to select the day in the month.

dbtiny_data


# Use a Tkinter 
# Create a dialog that uses TinyDB to query 1 day of data
# from a JSON file
#
from tinydb import TinyDB, Query
import tinydb 
import tkinter as tk

db = TinyDB('db1.json')

# get a selected days data
def sel():
   thedate = "2019-04-{:02}".format(int(var.get()))
   print(thedate)
   table = db.table('Pi_1') 
   data1 = table.search(Query()['date'] == thedate)
   print(data1)
   if data_list.size() > 0:
       data_list.delete(1,data_list.size() -1 )
   for items in data1:
       data_list.insert(data_list.size()+ 1,
                        items['date'] + " " + items['time'] +
                        " -  Moisture : " + str(items['moisture']) +
                        " -  Light : " + str(items['light']) )


root = tk.Tk()
root.title("DBtiny - Daily Data")

var = tk.DoubleVar()
data_list = tk.Listbox(root,height=20, width= 60)
data_list.grid(row=1,column=1)
                     
day_scale = tk.Scale( root,variable = var, from_=1,to=30,orient='horizontal',length=300 ).grid(row=2,column=1)

button1 = tk.Button(root, text="Get Daily Values", command=sel).grid(row=3,column=1)

root.mainloop()

Summary

TinyDB is a good solution for small IoT projects where I’m using Python and I’m not collecting a lot of data.

My next step would be to look at presenting the JSON data in Node-Red.

 

ODROID – A Raspberry Pi Alternative

The ODROID is a series of single-board computers manufactured by Hardkernel in South Korea. The ODROID-C1+ ($35) and the ODROID-C2 ($46) have a form factor similar to the Raspberry Pi 3. The higher end ODROID-XU($59) which is around 5 times faster than the Pi 3 has a signifigently different board layout.

For my testing I looked at the ODROID-C2 ($46), it is a little more expensive than a Pi 3 but the literature states that it’s 2-3 times faster.

My goal was to see if I could use the ODROID-C2 for some typical Raspberry Pi applications. In this blog I will be looking at doing C, Python and NodeRed programming from a Pi user perspective.

OD_PI

I’ve been happy with the functionality and openness of the Raspberry Pi platform, however I find its desktop performance to be sluggish. For only a few dollars more than a Pi 3 the ODROID-C2 CPU, RAM and GPU specs are impressive.

ODROID-C2 / Raspberry Pi 3 Hardware Comparison
Odroid C2 Raspberry Pi 3
CPU Amlogic S905 SoC
4 x ARM Cortex-A53 1.5GHz
64bit ARMv8 Architecture @28nm
Broadcom BCM2837
4 x ARM Cortex-A53 1.2Ghz
64bit ARMv7 Architecture @40nm
RAM 2GB 32bit DDR3 912MHz 1GB 32bit LPDDR2 450MHz
GPU 3 x ARM Mali-450 MP 700MHz 1 x VideoCore IV 250MHz
USB 4 Ports 4 Ports
Ethernet / LAN 10 / 100 / 1000 Mbit/s 10 / 100 Mbit/s
Built in WiFi No Yes
Built in Bluetooth No Yes
IR Receiver Built in Needs add-on
I/O Expansion 40 + 7 pin port
GPIO / UART / I2C / I2S / ADC
40 pin port
GPIO / UART / SPI / I2S
Camera Input USB 720p MIPI CSI 1080p
List Price (US) $46 $35

First Impressions

The ODROID-C2 is almost the same footprint as the Raspberry Pi 3 but not exactly. I found that because the microSD mounting is different some, but not all, of my Pi cases could be used .

lego_case

When you are designing your projects it is important to note that the ODROID-C2 does not have a built in Wifi or Bluetooth adapters, so you’ll need wired connections or USB adapters. Like some of the Orange Pi modules the ODROID-C2 has a built-in IR connection.

ODROID-C2 can be loaded with Ubuntu, Arch Linux and Android images. For my testing I used the Armbian 5.40 Ubuntu desktop and the performance was signifigently faster than my Raspberry Pi 3 Raspian desktop. I could definitely see ODROID-C2 being used as a low cost Web browser station.

The ODROID-C2 images are quite lean, so you will need to go the ODROID Wiki, https://wiki.odroid.com, for instructions on loading additional software components.

The ODROID-C2 has a 40 pin General Purpose Input/Output (GPIO) arrangement like the Pi 3, so it is possible to use Pi prototyping hats on the ODROID-C2 .

pi_hats

There are some noticeable differences in the pin definitions between the two modules, so for this reason I didn’t risk using any of my intelligent Pi hats on the ODROID-C2. The gpio command line tool can be used to view the pin definitions:

Od_readall

The Raspberry Pi GPIO names are in the range of 2 to 27, whereas the ODROID-C2 GPIO ranges are in the 200’s, because of this don’t expect to be able to run all your Raspberry Pi code “as is” on the ODROID-C2.

Unlike the Arduino the Raspberry Pi platform has no built in support for analog inputs. I got pretty excited when I noticed that the ODROID-C2 had two built in Analog-to-Digital Converter (ADC) pins (AIN.1 on pin 37 and AIN.0 on pin 40). However after some investigation I found that these pins had virtually no example code and they only support 1.8 volts. Most of my analog input sensors require 3.3V or 5V so I’m not sure how often these ADC pins will be used.

Python Applications

The ODROID-C2 Wiki references the RPi.GPIO and wiringpi Python libraries. I tested both of these libraries and I found that standard reads and writes worked, but neither of these libraries supported the callback functions like the Raspberry Pi versions.

For existing Pi projects where you are using callback functions for rising and/or falling digital signals (like intrusion alarms) you will need to do some re-coding with a polling method. It’s also important to note that the ODROID RPi.GPIO library is a little confusing because it uses the Pi pin names and not the ODROID pin names, so for example ODROID-C2 physical pin 7 is referenced as GPIO.04 (as on a PI) and not GPIO.249 (the ODROID-C2 name). Below is simple Python example that polls for a button press and then toggles an LED output when a button press is caught.

import RPi.GPIO as GPIO

GPIO.setmode(GPIO.BCM)

button = 4 # physical pin 7, PI GPIO.04, ODROID-C2 GPIO.249
led = 17 # physical pin 11, PI GPIO.17, ODROID-C2 GPIO.247
GPIO.setup(led, GPIO.OUT, initial=GPIO.LOW)
GPIO.setup(button, GPIO.IN, pull_up_down = GPIO.PUD_UP)

print ('Wait for button...')
while True:
    if GPIO.input(button) == 1:
        GPIO.output(led,0)
    else:
    GPIO.output(led,1)
    print "button pressed"

There are some excellent Python libraries that are designed to work with the Raspberry Pi. However it will require some trial and error to determine which libraries will and won’t work with the ODROID-C2.

I tried testing the DHT11 temperature and humidity sensor with the ODROID-C2, and unfortunately the library had a “Segmentation Error” when I tried running an example.

Node-Red

NodeRed can be installed on ODROID-C2 by using the manual install instructions for Raspbian at nodered.org. This install procedure will add a start item to the desktop Application menu, but due to hardware differences the Raspberry Pi GPIO input/output components will not load.

To read and write to Raspberry Pi GPIO a simple workaround is to use exec nodes to call the gpio utility.

Odroid_NodeRed

The command line syntax for writing a gpio output is: gpio write pin state, and for reading it is: gpio read pin. One of the limitations of this workaround is that you will need to add a polling mechanism, luckily there are some good scheduling nodes such as Big Timer that can be used.

C Applications

Programming in C is fairly well documented and an ODROID “C Tinkering Kit” is sold separately. The wiringPi library is used in C applications.

Below is a C version of the Python example above. It is important to note that these 2 examples talk to the same physical pins but the C wiringPi library uses the ODROID-C2 wPi numbers and the Python RPi.GPIO library uses the Pi BCM numbers.

// led.c - Read/Write "C" example for an Odroid-C2
//
#include  <wiringPi.h>

int main(void)
{
    wiringPiSetup();
    int led = 0;
    int button = 7;
    pinMode(led, OUTPUT);
    pinMode(button, INPUT);
 
    for (;;)
    {
        if (digitalRead(button) == 1) {
           digitalWrite(led, LOW); 
        }
	else
	{
           digitalWrite(led, HIGH); 
        }
    }
    return 0;
}

To compile and run this program:

$ gcc -o led led.c -lwiringPi -lpthread
$ sudo ./led

Summary

I liked that I could reuse some of my Pi cases and prototyping hats with the ODROID-C2.

As a PI user I found that coding in C, Python and NodeRed on the ODROID-C2 was fairly easy, but there were many limitation compared to the Pi platform. The ODROID Wiki had the key product documentation, but it was no where near the incredibly rich documentation that exists with Raspberry Pi modules.

There are a some excellent Python libraries and PI hardware add-ons that support a variety of sensors and I/O applications, these may or not work with the ODROID hardware.

During the development cycle of a project it is nice to have a faster interface, but typically my final projects do not need any high end processing and they can often run on low end Raspberry Pi 1 modules. So for now I would stick to a Raspberry Pi for GPIO/hardware type projects.

I enjoying playing with the ODROID-C2 and for projects requiring higher performance such as video servers, graphic or Web applications then the ODROID-C2 module is definitely worth considering.

Pi Thermometer using Python Turtles

Python Turtles are a great way to start kids in programming. Turtles offer a simple step-by-step graphical presentation that has tons of tutorials and examples.

Turtles can also be used on Raspberry Pi projects. In this blog I wanted to look at a Turtle example that reads a temperature sensor and graphically shows the result as an “old style” thermometer.

Getting Started

Python Turtles is probably already  loaded on your system, if not enter:

pip install turtles

The turtle library opens a graphic screen with the very center of the screen being (0,0). This is a little different than many other graphic systems (like PyGame) where the top left  is (0,0).

Different turtle objects can be defined and moved around the screen. A useful feature of turtles is that you can clear all the drawing from one turtle without effecting what the other turtles have done. (Note: this is useful in this thermometer example where we can have a static background turtle and a dynamic turtle that updates with new information).

Below is an example with 3 turtles. The first turtle (t1) is set to red and then moved forward, left, forward and then sent home. The second turtle (t2) is set to purple and given a turtle symbol. The third turtle (t3) is set to green and then moved to a position and a thick circle is drawn.


from turtle import *

setup(500, 400)
Screen()
title(" 3 Turtles")

# First red turtle goes forward, left, forward and back home
t1=Turtle()
t1.color("red")
t1.forward(100)
t1.left(90)
t1.forward(100)
t1.home()

# Second purple turtle has a turtle shape
t2=Turtle()
t2.shape("turtle")
t2.color("purple")
t2.right(45)
t2.forward(100)
t2.left(90)
t2.forward(100)

# Third green turtle goes to a location and makes a thick circle
t3=Turtle()
t3.color("green")
t3.pensize(10)
t3.up()
t3.goto(-100,-10)
t3.down()
t3.circle(80)

turtles1

Drawing an “Old Style” Thermometer

For this project we wanted to draw an “old style” mercury thermometer, with a bulb of red mercury at the bottom and a tube above it.

therm_bg

Using the simple turtle commands like : move, left, right, forward etc. is great to learn but it can be awkward for more difficult drawings.

A more efficient approach is to define an array of x,y coordinates and then move to each position in the array. For example the upper tube can be drawn by:


# Define an array of x,y coordinates for the top tube
outline = ((0,-50),(25,-50),(25,210),(-25,210),(-25,-50),(0,-50))
for pos in outline: # move to each tube x,y point
       thermo.goto(pos)
       thermo.pendown()

Circles are created using a turtle.circle(width) command. To fill an object or a group of objects a turtle.begin_fill() and a turtle.end_fill() set of command is used.

For our example the filled circle for the bulb is created by:


# put the pen up and move to the circle starting
thermo.penup()
thermo.goto(0,-137)
thermo.pendown()
thermo.pensize(5)
thermo.color("black","red")
# draw the circle with fill
thermo.begin_fill()
thermo.circle(50)
thermo.end_fill()

The complete code to draw the complete thermometer background would be:


# Create a background for an "old style" thermometer
from turtle import Turtle,Screen, mainloop
import random, time

# Define a Turtle object
thermo = Turtle()
thermo.penup()
thermo.hideturtle()
thermo.pensize(5)

# Define an array of x,y coordinates for the top tube
outline = ((0,-50),(25,-50),(25,210),(-25,210),(-25,-50),(0,-50))
for pos in outline: # move to each tube x,y point
       thermo.goto(pos)
       thermo.pendown()
# put the pen up and move to the circle starting
thermo.penup()
thermo.goto(0,-137)
thermo.pendown()
thermo.pensize(5)
thermo.color("black","red")
# draw the circle with fill
thermo.begin_fill()
thermo.circle(50)
thermo.end_fill()

mainloop()

Raspberry Pi Hardware Setup

There are a number of different temperature sensors that can be used. For our example we used a low cost ($5) DHT11 temperature/humidity sensor. The DHT11 sensor that we used had 3 pins, (Signal, 5V and GND), and we wired the Signal pin to the Pi physical pin 7.

DHT11-Wiring-Diagram

There is a DHT temperature/sensor Python library that is installed by:

sudo pip install Adafruit_DHT

A Python DHT test program would be:


#!/usr/bin/python
import sys
import Adafruit_DHT

sensor_type = 11 # sensor type could also be 22, for DHT22

dht_pin = 4 # Note: BCM pin 4 = physical pin 7

humidity, temperature = Adafruit_DHT.read_retry(sensor_type, dht_pin)

print( "Temp: ", temperature, " deg C")

print( "humidity: ", humidity, " %")

Turtle Thermometer

Now for the final project we can start pulling things together.

For the thermometer project we used 2 turtles, a static background turtle (thermo) and a dynamic turtle (bar). The bar turtle is cleared and redrawn in the drawbar() function.

A screen object wn is used to resize the window and add a title.

For testing a random integer can be used. This is also useful for checking the 0-40C range of the bar.

The full code and an screen shot are below:


from turtle import Turtle,Screen, mainloop
import random, time
import Adafruit_DHT

# Update the temperature bar height and value
def drawbar(temp):
       top = (-50 + 260 * temp/40)
       boutline = ((0,-50),(20,-50),(20,top),(-20,top),(-20,-50),(0,-50))
       bar.penup()
       bar.clear()  # clear the old bar and text
       bar.begin_fill()
       for pos in boutline:
              bar.goto(pos)
       bar.end_fill()
       bar.goto(30,top)
       bar.write(str(temp) + " C",font=("Arial",24, "bold"))

# Setup a default screen size and Title
wn = Screen()
wn.setup(width = 500, height = 500)
wn.title("RaspPi Temperature Sensor")

# define a static thermo backgroup object and a dynamic bar object
thermo = Turtle()
thermo.penup()
bar = Turtle()
bar.color("red")
bar.hideturtle()

# define an array for the top tube
outline = ((0,-50),(25,-50),(25,210),(-25,210),(-25,-50),(0,-50))
thermo.hideturtle()
thermo.pensize(5)
for pos in outline:
       thermo.goto(pos)
       thermo.pendown()

# add some temperature labels
thermo.penup()
thermo.goto(50,-50)
thermo.write("0 C")
thermo.goto(50,210)
thermo.write("40 C")

# draw the filled bulb at the bottom
thermo.goto(0,-137)
thermo.pendown()
thermo.pensize(5)
thermo.color("black","red")
thermo.begin_fill()
thermo.circle(50)
thermo.end_fill()

# Update the temperature
while True:
       humidity, temperature = Adafruit_DHT.read_retry(11, 4)
       # use a random number for testing
       #temperature = random.randint(0,40)
       drawbar(temperature)
       time.sleep(5)

PI_thermo

Final Comments

Compared to other Python graphic libraries (like PyGame, Tkinter or Qt) Turtle graphics can be slow and perhaps limiting, but for kids Turtles projects are super easy to configure. If you are doing simple stuff Turtles requires a lot less code than the other graphic libraries (keyboard input is a good example of this).

There are a lot of possible fun Raspberry Pi projects that can be done with Turtle. Some of the other projects that we have done include:

  • use a Wii remote to draw pictures
  • create a Turtle drawing as you drive a rover (show the path)

DynamoDB with Python and JavaScript

Amazon Web Services (AWS) is a probably the best known Cloud services offering on the Internet.  At first glance AWS is a quite intimidating, learning about all the different features of AWS could be a full time job. For this blog I wanted to introduce one component of AWS called DynamoDB.

DynamoDB is a NoSQL database that could be connected to sensor hub devices such as Raspberry Pi’s for Internet of Things (IoT) solutions.

aws_services

Getting Started with DynamoDB

The first step to getting started with AWS is to create an account. You can open a free “testing” account but unfortunately you’ll need to show that you’re serious by including a credit card number.

After creating an account you can sign into the AWS Management Console (https://aws.amazon.com/console/). With a single user account it is possible to run different services in different locations. It is important to note that not all services are available in all locations, so select a location that offers all the services that you need.

AWS_Region

After a region is selected go to the Services menu item and search for DynamoDB, next select “create table”. A DynamoDB table has a NoSQL database structure, so this means that all that is required is a table name and a primary key. A primary key allows rows of data to have a way to be connected together.

For my IoT table example I created a primary key called signalname. (If you plan to have a very large data set and you need extremely high speed response you might want to consider defining multiple keys, such as location or sensor type).

create_table

Once the table is created, you can view and modify data.

the_table

The “Items” tab is used to add, view and delete data. Note, the viewing of updates is not automatic, so you need to press the “refresh” button periodically.

mytag1

Users and Security

The AWS user security is quite detailed. To get started select your account name on the menu bar and “My Security Credentials“.  The Groups item allows you to create security classifications for activities such as admin, viewing, etc.

New users can be defined and added into security groups.

user_security

When a new users is created an Access Key ID and a Secret Access Key is generated. The Secret Access Key is only shown when the user is created so ensure that you save it, and/or download the CSV file.

newuser

After an Access key ID and Secret Access Key is created for a user, you can start  programming interfaces to DynamoDB.

DynamoDB and Python

The Python AWS library is called boto3 and it is installed  in Windows by :

pip install boto3

and in Linux by:

sudo pip install boto3

The AWS security can be either added directly into the Python (or JavaScript) application or it can reside on the node or PC that you are working on. Both methods have pro’s and con’s. To add the credentials locally, you’ll need to install the AWS command line tool (awscli) and then enter your specific information:

$ sudo pip install awscli
$ aws configure
AWS Access Key ID [None]: AKIAIDxxxxx
AWS Secret Access Key [None]: kzzeko8Fxxxxxxxxxxxxxxxxx
Default region name [None]: Central
Default output format [None]:

A Python example to write and read a row into DynamoDB with the credentials in the program could be:

import boto3
import time

thesec = int(time.time() % 60)

mysession = boto3.session.Session(
    aws_access_key_id= 'AKIAID7xxxxxxxxx',
    aws_secret_access_key='kzzeko8xxxxxxxxxxxxxxxx')
db = mysession.resource('dynamodb', region_name='ca-central-1')

table = db.Table('my_iot_burl1')

response = table.put_item(
   Item={
        'signal': 'mytag1',
        'tagdesc': 'test tag1',
        'tagvalue': thesec
    })

response = table.get_item(Key={'signal': 'mytag1'})
print(thesec)
print(response)
print(response['Item']['tagvalue'])

If the credentials are locally used then the code to write data would be:

import boto3

dynamodb = boto3.resource('dynamodb')
table = dynamodb.Table('users01')

table.put_item(
   Item={
        'signal': 'mytag1',
        'tagdesc': 'test tag1',
        'tagvalue': 16
    }
)

The response comes back as JSON data. For this example the result would be:

16
{‘Item’: {‘tagvalue’: Decimal(’16’), ‘tagdesc’: ‘test tag1’, ‘signal’: ‘mytag1’}, ‘ResponseMetadata’: {‘RequestId’: ’66M7R4CVCACA4KNA24LM90KG0NVV4KQNSO5AEMVJF66Q9ASUAAJG’, ‘HTTPStatusCode’: 200, ‘HTTPHeaders’: {‘server’: ‘Server’, ‘date’: ‘Fri, 15 Feb 2019 16:29:48 GMT’, ‘content-type’: ‘application/x-amz-json-1.0’, ‘content-length’: ’84’, ‘connection’: ‘keep-alive’, ‘x-amzn-requestid’: ’66M7R4CVCACA4KNA24LM90KG0NVV4KQNSO5AEMVJF66Q9ASUAAJG’, ‘x-amz-crc32’: ‘220267538’}, ‘RetryAttempts’: 0}}
16

For large amounts of data being written to DynamoDB it is possible to use a batch writing function:

import boto3
import time

db = boto3.resource('dynamodb', region_name='ca-central-1')

table = db.Table('my_iot_burl1')

thesec = int(time.time() % 60)

with table.batch_writer() as batch:
    for i in range(1,11):
        batch.put_item(
            Item={
                'signal': 'mytag' + str(i),
                'tagdesc': 'test tag'+ str(i),
                'tagvalue': thesec,
                'status': 'NORMAL'
            }
        )

The AWS console can be used to verify that the data was written in.

Writing to DynamoDB will clear the existing row with a completely new set of data. If you only want to update a field in a row then the update_item call is used. In the example below only the sensor tagvalue is updated and all other fields are left unchanged.

import boto3

mysession = boto3.session.Session(
    aws_access_key_id= 'AKIAIDxxxxxxxxx',
    aws_secret_access_key='kzzekxxxxxxxxxxxxxxx'
)

db = mysession.resource('dynamodb', region_name='ca-central-1')

table = db.Table('my_iot_burl1')

# set the value for mytag1 to bet 29
table.update_item(
    Key={ 'signal': 'mytag1' },
    UpdateExpression='SET tagvalue = :newval1',
    ExpressionAttributeValues={
        ':newval1': 29
    }
)

Python “Scan the Data” Example

The boto3 library supports the ability to scan the data based on some filtering criteria. The filter syntax is: Attr(‘the field’).eq(‘thevalue’).

The following comparison operators are available:

eq | ne | le | lt | ge | gt | not_null | null | contains | not_contains | begins_with | in | between

Multiple statements can be created with | (OR) , & (AND) operators. Below is an example that scans the database for rows with an alarm status or with a tag value greater than 100.

scan_view

import boto3
from boto3.dynamodb.conditions import Key, Attr

db = boto3.resource('dynamodb', region_name='ca-central-1')

table = db.Table('my_iot_burl1')

response = table.scan(
    FilterExpression=Attr('status').eq('ALARM') | Attr('tagvalue').gt(100)
)

#print (response)
for i in response['Items']:
    print(i)
    print(i['signal'], "=", i['tagvalue'])

The results were:

{‘tagvalue’: Decimal(‘150’), ‘signal’: ‘mytag2’, ‘tagdesc’: ‘test tag2’, ‘status’: ‘NORMAL’}
mytag2 = 150
{‘tagvalue’: Decimal(‘999’), ‘signal’: ‘mytag1’, ‘tagdesc’: ‘test tag1’, ‘status’: ‘ALARM‘}
mytag1 = 999

JavaScript

For standard HTML/JavaScript projects the credentials are included in the source file. This obviously has some serious security issues, so it’s recommended that you define a low level user that can only do a small subset of functions.

Below is a simple example that can read our DynamoDB table and it can write a new row with an updated value.

<html>
<head>
<script src="https://sdk.amazonaws.com/js/aws-sdk-2.7.16.min.js"></script>

<script>
AWS.config.update({
  region: "ca-central-1",
  accessKeyId: "AKIAID7xxxxxxxxxxxx",
  secretAccessKey: "kzzeko8xxxxxxxxxxxxx"
});

var docClient = new AWS.DynamoDB.DocumentClient();

function createItem() {
  var d = new Date();
  var n = d.getSeconds();
    var params = {
        TableName :"my_iot_burl1",
        Item:{
      'signal': 'mytag4',
      'desc': 'test tag4',
      'info': {
        'value':n,
        'quality': "questionable"
        }
    }
    };
    docClient.put(params, function(err, data) {
        if (err) {
            document.getElementById('textarea').innerHTML = "Unable to add item: " + "\n" + JSON.stringify(err, undefined, 2);
        } else {
            document.getElementById('textarea').innerHTML = "PutItem succeeded: " + "\n" + JSON.stringify(params,undefined, 2);
        }
    });
}

function readItem() {
    var table = "my_iot_burl1";
    var signal = 'mytag4';
    var params = {
        TableName: table,
        Key:{'signal' : 'mytag4' }
    };
    docClient.get(params, function(err, data) {
        if (err) {
            document.getElementById('textarea2').innerHTML = "Unable to read item: " + "\n" + JSON.stringify(err, undefined, 2);
        } else {
            document.getElementById('textarea2').innerHTML = "GetItem succeeded: " + "\n" + JSON.stringify(data, undefined, 2);
        }
    });
}

</script>
</head>

<body>
<input id="createItem" type="button" value="Create Item" onclick="createItem();" />
<br><br>
<textarea readonly id= "textarea" style="width:400px; height:300px"></textarea>
<hr>
<input id="readItem" type="button" value="Read Item" onclick="readItem();" />
<br><br>
<textarea readonly id= "textarea2" style="width:400px; height:300px"></textarea>
</body>
</html>

aws_html

Summary

On first impressions I was pretty overwhelmed using Amazon Web Services. However after some time playing with it I found that it wasn’t unlike many other Web services that are available.

DynamoDB can be used to communicate with sensor values from a Raspberry Pi, but unfortunately there are no mainstream libraries for Arduino projects. AWS does offer an MQTT solution that can be used to update a DynamoDB table.

For small IoT projects I feel that AWS is overkill, but for larger projects I would definitely consider using it.

MicroPython Air Boat

My daughter’s and I have built a number of air boat projects, but this time I thought that I’d try it using MicroPython. MicroPython is a lean and efficient implementation of the Python 3 programming language that includes a small subset of the Python standard library and is optimised to run on microcontrollers.

For this project my goal was to create a MicroPython application that was a standalone WiFi access point, and its used a small web server for controlling the air boat fans.

Hardware

The hardware that I used on this project included:

• 1 – ESP8266 module ($5-$10)
• 2 – L9110 Fans ($5 each)
• 1 – Uno proto shield (optional) or use a small breadboard
• 9V battery or portable phone charger
• K-Nex blocks
• small plastic box
• 2 – plastic water bottles
• duct tape

MicroPython is supported on a number of Wifi enabled ESP32 and ESP8266 modules. These modules are well priced with the NodeMCU modules starting around $5. It’s important to note that most of the ESP-8266 modules also support Lua and Arduino C/C++ programming.

nodemcu

For this project I used an Wemos ESP-8266 module, that comes in a Arduino Uno form factor. I like the Wemos modules because I can used my Uno proto shields and these modules supports 5V DC, as opposed to the typical MicroPython modules that only support 3.3V. The fans will work with 3.3V but they generate a lot more wind power at 5V.

The L9110 fans are designed for Arduino projects and they only cost about $5. These fans have four pins: VCC, GND, INA (direction), and INB (on/off). For this project I only used forward spinning fans, so only pin INB was used.

L9110_fan

Boat Construction

For the boat frame I used K’Nex pieces because they are light weight and sturdy, however there a lot of other materials that could be used. Water bottles were used for flotation, and duct tape was used to secure the bottles to the frame. To help protect the electronics a small plastic snack container was used. I wire wrapped the fans to the boat frame.

airboat

MicroPython Setup

There are a few choices for MicroPython development. I found that the uPyCraft IDE offered a nice integrated environment (Figure 5), and there are some excellent tutorials to help you get started.

upycraft

For MicroPython projects you typically create 2 Python applications, a boot.py and a main.py file. I like to equate this to Arduino where you have a setup() function and a loop() function. For this example I wanted to keep the documentation simple so I put everything in the boot.py file, but it’s recommended to use main.py for larger projects.

Creating an Access Point

The picture below shows how to setup an access point in just a few lines of code. For this example the access point is called ‘ESP32’ with a password of ‘12345678’. When the code is run you will be able to see when a remote user connects and disconnects to the access point.

access_point

MicroPython Web Server

After getting the Access Point working the next step is to create a Web server. This is a simple Web server project so I embedded the HTML content into my Python code, however for a more complex application I would definitely have my web pages as files that are independent from the code.

For the Web Server example application , we start with the access point connection code, then we setup a socket on port 80. The HTTP request/response sequence is passed through a function called web_page(request). In web_page(request) the code looks for keywords in the HREF request.

# MicroPython boot.py - Access Point Web Server

try:
  import usocket as socket
except:
  import socket

import network

station = network.WLAN(network.AP_IF)
station.active(True)
station.config(essid='ESP32')
station.config(authmode=3,password='12345678')

while station.isconnected() == False:
  pass

print('Connection successful')
print(station.ifconfig())

def web_page(request):
  
  fans_state = ""
  if request.find('/?forward') > 0:
    fans_state="Going Forward"
  if request.find('/?Stopped') > 0:
    fans_state="Stopped" 
  
  html = """<html><head> <title>Ice Boat Web Server</title> 
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="icon" href="data:,"> <style>
  html{font-family: Helvetica; display:inline-block; margin: 0px auto; text-align: center;}
  h1{color: #0F3376; padding: 2vh;}p{font-size: 1.5rem;}
  .button{display: inline-block; background-color: #e7bd3b; border: none; 
  border-radius: 4px; color: white; text-decoration: none; font-size: 30px; width:100%}
  </style></head>
  <body> <h1>Ice Boat Web Server</h1> 
  <p>ICE Boat : <strong>""" + fans_state + """</strong></p>
  <p><a href='/?forward'><button class="button">Forward</button></a></p>
  <p><a href='/?stop'><button class="button button">STOP</button></a></p>
  
  </body></html>"""
  
  return html

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('', 80))
s.listen(5)

while True:
  conn, addr = s.accept()
  print('Got a connection from %s' % str(addr))
  request = conn.recv(1024)
  request = str(request)
  print('The Content = %s' % request)
  response = web_page(request)
  conn.send(response)
  conn.close()

The embedded HTML code passes keywords using anchor tags, for example: <a href=’/?forward’>. Often we use mobile frameworks like Bootstrap to help with formatting, however because we’re running a standalone access point we need to manually define all our style codes.

To access our MicroPython web server, use the address: 192.168.4.1. This is the default address (in the access point setup you can change this). When our web server is running we should be able to toggle the stop/forward states.

screen2

Writing Outputs

The MicroPython command line interface is good way to test outputs. To access the command line interface, use the connect button and then enter “Control-C”.

To manage pins on the hardware, machine library is used :

from machine import Pin

Then pin objects can be defined as either inputs or outputs:

Pin14 = Pin(14, Pin.IN)
Pin5 = Pin(5, Pin.OUT)

The value of a pin is read using: pinobject.value(), and set with: pinobject.value(thevalue) .

motor_test

Final Application

Now that we have all the pieces working independently we can pull it all together. For the final code I’ve defined two fans (motorR and motorL), and a fan control function is called from the web requests.

# MicroPython boot.py - Access Point and Airboat Web Controls
import time
try:
  import usocket as socket
except:
  import socket

from machine import Pin
import network

# Define the left, right and back fans pin
motorR = Pin(12,Pin.OUT)
motorL = Pin(4,Pin.OUT)

# start with fans stopped, Note: my FANs are 0=run
motorR.value(1)
motorL.value(1)


station = network.WLAN(network.AP_IF)
station.active(True)
station.config(essid='ESP32')
station.config(authmode=3,password='12345678')

while station.isconnected() == False:
  pass

print('Connection successful')
print(station.ifconfig())

def fancontrol(left,right):
  motorL.value(left)
  motorR.value(right)

def web_page(request):
  
  fans_state = "Stopped"
  if request.find('/?forward') > 0:
    fans_state="Going Forward"
    fancontrol(0,0)
  if request.find('/?left') > 0:
    fans_state="Going Left"
    fancontrol(1,0)
  if request.find('/?right') > 0:
    fans_state="Going Right" 
    fancontrol(0,1)
  if request.find('/?stop') > 0:
    fans_state="Stopped"
    fancontrol(1,1)
  
  html = """<html><head> <title>Air Boat Web Server</title> 
  <meta name="viewport" content="width=device-width, initial-scale=1">
  <link rel="icon" href="data:,"> <style>
  html{font-family: Helvetica; display:inline-block; margin: 0px auto; text-align: center;}
  h1{color: #0F3376; padding: 2vh;}p{font-size: 1.5rem;}
  .button{display: inline-block; background-color: #e7bd3b; border: none; 
  border-radius: 4px; color: white; text-decoration: none; font-size: 30px; width:100%}
  .button2{background-color: #4286f4; width:49%}
  </style></head>
  <body> <h1>Air Boat Web Server</h1> 
  <p>Air Boat : <strong>""" + fans_state + """</strong></p>
  <p><a href='/?forward'><button class="button">Forward</button></a></p>
  <p><a href='/?left'><button class="button button2">LEFT</button></a>
  <a href='/?right'><button class="button button2" >RIGHT</button></a></p>
  <p><a href='/?stop'><button class="button button">STOP</button></a></p>
  
  </body></html>"""
  
  return html

s = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
s.bind(('', 80))
s.listen(5)

while True:
  conn, addr = s.accept()
  print('Got a connection from %s' % str(addr))
  request = conn.recv(1024)
  request = str(request)
  print('The Content = %s' % request)
  response = web_page(request)
  conn.send(response)
  conn.close()

airboat_moving

Summary

You’ll be quite surprised about how fast even 2 fans will move the air boat. Balancing the direction of the fans or adding a simple rudder may be required to ensure that it keeps a straight forward direction.

I’ve done the same project on the exact same hardware in Anduino C++ and I would say the response speed is similar but the Python code might be slightly leaner. I found that the MicroPython IDE wasn’t as robust as the Arduino IDE, but I really enjoyed doing interactive Python testing from a command prompt.

I won’t be giving up on Arduino C++, but I can definitely see a place for MicroPython, especially for projects with lots of string manipulation.

 

X Windows from Raspberry Pi to Android

So you’ve done a cool graphic app on your Raspberry Pi wouldn’t it be nice to see it on your Android phone ? Well luckily there are a couple of X Server apps that you can run on your Android phone that allows you to see your Pi graphic apps.

X Server Apps for Android

When you do a search for “x servers” on Android Play Store, you will get a few results.

apps_xservers

X Server XSDL from pelya, has good reviews and I found it to be fairly robust. My only comment was I wasn’t sure how to clear the background help statements, and it made the screen look a little messy.

X server from Darkside is a beta release. I preferred the background and SSH integration over XSDL, however this app did not always close the window when the code requested it.

Both options are free and open source.

Getting Started with X Windows

There are two main ways for X Windows to be run on a remote system. The first method is that the Window is sent from the Pi (or any Linux/X Windows system) to the Android phone. The second method is what you typically do on a Windows PC using an SSH terminal program like  Putty; you SSH (Secure Shell) in to the Rasp Pi and then when you run an app it open on your remote node.

xwindow_overview

Calling windows from a PC is pretty easy but calling windows from an Android phone is challenging and slow because of the keyboard. The two X Servers apps do not have SSH components built-in but there are some Android SSH client apps that can be run independently or integrated with the Darkside X app.

Sending an X Window from a Raspberry Pi or Linux node is rather straightforward. The sending location of an X window is defined by the DISPLAY session variable. To check its value enter:

~$ echo $DISPLAY
localhost:10.0

A typical results will be something like: localhost:10.0 .

To change the DISPLAY variable enter the ipaddress:window_number.  In my case the phone’s IP is 192.168.0.102, and the window number is 0, so in a terminal on the PI the command is:

export DISPLAY=192.168.0.102:0

This will now direct all X Windows in my terminal session to my phone. To test things we can send the Python IDE to the phone by:

$ idle

x_idle

Unfortunately the Android X Server app does not support moving or resizing of X Windows, so the app is pinned to the top left. Luckily if we are writing Python apps we can set the sizing and positioning when the app starts up.

Python Tkinter

The tkinter graphic library allows you to create a simple graphic front end to your Python projects.

The following code will update a label with the time every second. The geometry setting is used to move the window (300 left, and 100 down).

# test1.py - Show the time every second
#
import tkinter as tk
from datetime import datetime

root = tk.Tk()
root.geometry("+300+100")

def Updatetime():
label['text'] = datetime.now().strftime("%H:%M:%S")
root.after(1000,Updatetime)

def CallBack():
root.destroy()

label = tk.Label( root, text='', font = "Aria 72 bold" )
label.pack()

B = tk.Button(root, text ="Quit", font = "Aria 72 bold", command = CallBack)

B.pack()

root.after(1000,Updatetime)
root.mainloop()

To set the X window to my phone and run this Python application enter:

~$ export DISPLAY=192.168.0.102:0
~$ python test1.py

test1

Final Comment

I found that Python 3 was much more solid than Python 2. For example a Pi real time bars example worked perfectly on Python3, but on Python2 the bar wouldn’t update at all.

x_bars

 

 

 

 

 

 

 

Data Mine News Headlines

Python offers some amazing tools to do text based searching, sorting and analysis.

In this blog I wanted to look at grabbing a large group of news headlines and then do some Natural Language Processing (NLP) in Python to try and find out “Who’s in the News”.

For this example I’ll be using the Reddit News, but any News feed like Twitter, CNN, BBC etc. could be used.

Reddit

Reddit is a social media source that is free to use. To pull information from Reddit you will need to create an account and get API client ID and secret. To do this go to: https://www.reddit.com/prefs/apps/ and select edit in the developed applications area.

reddit_api_info

The Reddit Python library is called Praw, and it is installed by:

pip install praw

An example to connect to the Reddit API and list the newest 4 News headlines would be:

# red1.py - Python Reddit Example 
# Get 4 Latest News Headlines
import praw

# Update with your client info
reddit = praw.Reddit(client_id='xQsMxxxxxxxx',
            client_secret='X8r62xxxxxxxxxxxx',
            user_agent='myreddit', username='yourname', password='xxxx')
					 
i=0					 
for submission in reddit.subreddit('news').new(limit=4):
	i += 1
	print(i,submission.title)

Running this code will show something like:

> python red1.py
1 De Blasio Unveils Health Care Plan for Undocumented and Low-Income New Yorkers
2 Kentucky teacher seen dragging student with autism down hall pleads not guilty
3 Homeless man allegedly involved in GoFundMe scam arrested in Philadelphia
4 Government shutdown stops FDA food safety inspections

The output from the program can be checked with the Reddit’s web page:

reddit_new4

When you are looking at Reddit it’s important to note that there are a number of different topics that could be queried. For example /inthenews is different than the /news.

Natural Language Processing

There are some amazing tools to allow you to manipulate and view the textual data. Pandas is a fast in memory data management library that supports sorting, querying and viewing of data. Spacy is a NLP tool. These two libraries are loaded by:

pip install pandas
pip install spacy

As a first example we’ll look at some text and we’ll use spacy to analyze what each word in the sentence is:

# Spacy test to get work types
#
import pandas as pd
import spacy

# Use the English core small web dictionary file
nlp = spacy.load('en_core_web_sm')
nlp.entity

# load some sample text into Spacy
doc = nlp('Astronomers in Canada have revealed details of 
       mysterious signals emanating from a distant galaxy')

print(doc,"\n")

# list the text and show the word type
for w in doc:
	print (w,w.pos_)

The output from this will:

>python red2.py
Astronomers in Canada have revealed details of mysterious signals emanating from a distant galaxy

Astronomers NOUN
in ADP
Canada PROPN
have VERB
revealed VERB
details NOUN
of ADP
mysterious ADJ
signals NOUN
emanating VERB
from ADP
a DET
distant ADJ
galaxy NOUN

Spacy will identify the words by their word type, like Astronomers NOUN. 

The proper nouns (PROPN) like Canada can be even further filtered to the type of noun, in this case location (GPE).

If you are only interested in proper nouns then it is possible to get the actual type of noun, for example: person, location, organization, work of art, date etc. To get the proper nouns the doc.ent object is queried.

# red3.py - Spacy test to get noun types
#
import pandas as pd
import spacy

nlp = spacy.load('en_core_web_sm')
nlp.entity

doc = nlp('Fred Smith and John Doe live in Toronto and they work for the Toronto Raptors.')

print(doc,"\n")

stuff = []
for w in doc.ents:
	print(w.text,w.label_)

The output for this is:

>python red3.py
Fred Smith and John Doe live in Toronto and they work for the Toronto Raptors.

Fred Smith PERSON
John Doe PERSON
Toronto GPE
the Toronto Raptors ORG

 

Pandas – to query/group and count

The Pandas library is extremely useful for doing statistical and data manipulation type functions.  If we expand our earlier example to include Pandas we can do some querying/grouping and counting

# NLP with Pandas data frames for queries/grouping/counting
#
import pandas as pd
import spacy

nlp = spacy.load('en_core_web_sm')
nlp.entity

doc = nlp('Fred Smith and John Doe live in Toronto and they work for the Toronto Raptors.')

print(doc,"\n")

stuff = []
for w in doc.ents:
	print(w.text,w.label_)
	stuff.append([w.text,w.label_])

# define a struction        
dflabel = ['keyword','wordtype']
# load a list into a Panda data frame with our structure
df = pd.DataFrame(stuff, columns=dflabel)

# print our data frame
print (df.head(n=50))

# create a new data frame with only the wordtype PERSON, then group and count it
names = df.query("wordtype=='PERSON'").groupby('keyword').count().sort_values(by='wordtype',ascending=False)

print (names.head(n=50))

The results for this would be:

Fred Smith and John Doe live in Toronto and they work for the Toronto Raptors.

Fred Smith PERSON
John Doe PERSON
Toronto GPE
the Toronto Raptors ORG

keyword wordtype
0 Fred Smith PERSON
1 John Doe PERSON
2 Toronto GPE
3 the Toronto Raptors ORG

          wordtype
keyword
Fred Smith 1
John Doe 1

Getting “Who’s in the News” from Reddit

Now we’re ready to put the pieces together. In this next example we’ll use the /inthenews Reddit section, and we’ll query 750 new items. From the results we’ll look at the people who are making the news.

# red_2_names.py - Get top names in Reddit "inthenews" 
#
import pandas as pd
import spacy
import praw

# Modify for your reddit id
reddit = praw.Reddit(client_id='xQsMfXXX',
                     client_secret='X8r62koQxxxxxxxx',
                     user_agent='myreddit', username='yourname', password='xxxx')
					 
thedata = ""
i=0
for submission in reddit.subreddit('inthenews').new(limit=750):
        i += 1
        #print(i,submission.title)
        thedata = thedata + submission.title
        
	
nlp = spacy.load('en_core_web_sm')
nlp.entity
doc = nlp(thedata)

# Create a list of keywords and wordtypes
stuff = []
dflabel = ['keyword','wordtype']
for w in doc.ents:
	stuff.append([w.text,w.label_])
#print(stuff)
	
df = pd.DataFrame(stuff, columns=dflabel)	
names = df.query("wordtype=='PERSON'").groupby('keyword').count().sort_values(by='wordtype',ascending=False)

print ("Who is making the news?\n")
print (names.head(n=10))

This example will generate results similar to the following:

python red_2_names.py
Who is making the news?

 wordtype
keyword
Trump 14
Ocasio-Cortez 6
Donald Trump 4
Cohen 3
Nancy Pelosi 2
Jeff Bezos 2
Ronald Reagan 2
Brown 2
Jayme Closs 2
Jared Kushner 2

The query could be changed to look at other nouns like locations, organization or all topics.

Final Comments

With just a small amount of code it is possible to do some amazing things. However like any statistical project we can improve the data set, for example entire news articles could be imported rather than just the headlines.