Gemini Protocol for Lightweight Internet Apps

The Gemini protocol is a very simple and light Internet protocol.

Unlike HTML files that contain layers of tags, style sheets and Javascript, a Gemini document is a simple readable document.

In this blog I wanted to document:

  • Gemini servers and clients using only 1 line of Bash
  • How to use large ASCII text in Gemini documents
  • How to create simple bar charts
  • How to create Gemini CGI apps in Python

Getting Started

A Gemini document only supports a few statements and graphic images such as JPEG or PNG are not supported.

An example Gemini document with the common formatting options:

# Heading level 1 (H1) 
## Heading level 2 (H2)
### Heading level 3 (H3)

=> testpage.gmi A link to another page.

> This line will show as a block-quote. 

A list of items
* This is the first list item.
* This is another list item.

```
Code or ASCII Block 
   _   ___  ___ ___ ___   _____       _   
  /_\ / __|/ __|_ _|_ _| |_   _|__ __| |_ 
 / _ \\__ \ (__ | | | |    | |/ -_|_-<  _|
/_/ \_\___/\___|___|___|   |_|\___/__/\__|

```

Within a Gemini browser this file would look like:

Content Type

The content type is used by browsers and applications to determine how to manage the requested file.

Typically the content type is managed by server, for example a web server will send a HTTP/1.0 200 OK prior to sending the HTML file.

For the Gemini protocol the content type is: 20 text/gemini . Depending on the Gemini server the user may need to be add the content type manually. (More about this in Bash and CGI servers).

Simple Bash Gemini Servers and Clients

For basic testing a one line Bash statement can be used for custom Gemini servers and clients.

The Gemini protocol typically uses SSL (Secure Sockets Layer) and TLS (Transport Layer Security) encryption so the Bash ncat utility is needed (Note: the nc command does not support SSL).

Below is an example of single Gemini request:

The Gemini server is defined by using the -l , listen option. When a client requests data, the cat statement is used with a pipe (|) to output the file testpage.gmi.

The Gemini client echo’s a “from PC” message with its request, this helps identify which client is requesting data.

A simple Bash ncat statement doesn’t manage the content type so a “20 text/gemini” line is added to the top of the test page.

Dynamic Bash Data

In the earlier example the Bash server is only supporting 1 request then exiting.

A while loop can be added to pass a Bash script to the ncat statement. Below is an example of Bash script (showdata.sh) that show CPU data using the vmstat utility:

#!/bin/bash
#
# showdata.sh - Output data for Gemini Bash Server 
#
echo "20 text/gemini"
echo  " "
echo "#VMSTAT"
echo  " "
date +"%T"
echo  " "
# set Gemini formating for ASCII
echo  "\`\`\`"
#vmstat
top -n 1
echo  "\`\`\`"

To make the script executable use the command: chmod +x showdata.sh

The Bash command to run this script as a Gemini server is:

while true; do ./showdata.sh | ncat  -l -p 1965 --ssl; done

The earlier Bash Gemini client command can be used, or a Gemini browser/client app can be used. The handling of SSL/TLS encryption will vary with the client app. I used the very basic Zain app (https://gitgud.io/sathariel/zain) :

(Note: for the Zain client I needed to load tcl/tls, sudo apt-get install -y tcl-tls)

Using a 1 line Bash Gemini server is great for basic testing but I wouldn’t recommend if you want to connect to variety of different Gemini client applications.

Large ASCII Text

Gemini documents don’t support different font sizes, a workaround is to use the figlet tool to generate multi-line text strings. Figlet is installed on Ubuntu/Debian/Raspbian by:

sudo apt install figlet

Figlet has a number of font styles that use 2-5 line height characters:

When using ASCII headings the Gemini code formatting option should be used, and this has three backticks (“`) before and after the headings.

The earlier example can be modified to have ASCII headings:

#!/bin/bash
#
# showdata2.sh - Output Large Headings to a Gemini Bash Server 
#
echo "20 text/gemini"
echo  " "
echo  "\`\`\`"
# Generate large text 
figlet -f standard "Raspberry Pi"
figlet -f small -m 2  $(date +"%T")
# show CPU stats
vmstat
echo  "\`\`\`"

Bar Charts

Horizontal ASCII bar charts can be created by using the printf statement with different ASCII fill characters. For example:

# show a label with bar and value text 
#
label="temp"; val=20;
bar="printf '█%.0s' {1..$val}" ; 
printf '\n%-5s ' $label; eval $bar
printf '░%.0s' {1..5} ;
printf ' 50 C\n'

temp  ████████████████████░░░░░ 50 C

This bar logic can be using in a Raspberry Pi Stats page that looks at idle time and free space on the SD card:

#!/bin/bash
#
# pi_stats.sh - Output Pi Stats as a Gemini Page
#
echo "20 text/gemini"
echo  " "
# Put the rest of the Gemini document into code block mode
echo  "\`\`\`"
# Generate large text 
figlet -f standard "Pi Stats"

# Show the time
echo "Time: $(date +'%T')"
echo ""

# Get idle time, scale 0-50
idle=$(top -n 1 | grep id | awk '{print $8}')
barsize=$(echo $idle | awk '{printf "%0.f" , $1/2}')
thebar="printf '█%.0s' {1..$barsize}"
graysize=$(expr 50 - $barsize)
thegray="printf '░%.0s' {1..$graysize}"
printf 'Idle Time  '; eval $thebar; eval $thegray ; echo " $idle %"
echo  ""

# Get free space on SD card, scale 0-50
freesp=$(df | grep root | awk '{printf "%0.f", $5/2}')
barsize=$(echo $freesp | awk '{printf "%0.f" , $1/2}')
thebar="printf '█%.0s' {1..$barsize}"
graysize=$(expr 50 - $barsize)
thegray="printf '░%.0s' {1..$graysize}"
printf 'Free Space '; eval $thebar; eval $thegray ; echo " $freesp %"

echo  "\`\`\`"

To run this page use

while true; do ./pi_stats.sh | ncat  -l -p 1965 --ssl; done

Python CGI Pages

There are a number of good Gemini servers, for my testing I used the Python based Jetforce server, it is installed by:

pip install jetforce

To run the Jetforce server it is important to define a home directory, the allowable hosts that can connect (0.0.0.0 is all IP4 nodes) and the server’s hostname:

# Start the jetforce Gemini server for all IP4 hosts
jetforce --dir /home/pi/temp --host "0.0.0.0" --hostname 192.168.0.105 &
# Start jetforce without hardcoding hostname
# jetforce --dir /home/pi/temp --host "0.0.0.0" --hostname $(hostname -I) &

By default CGI (Common Gateway Interface) files are defined in the directory cgi-bin which is under the home directory.

Unlike a one-line Bash server, Jetforce server will pass environment variables like QUERY_STRING and host and remote connection information.

CGI programs can be written in a variety of programming languages. For this example I wanted to pass Raspberry Pi BME280 temperature, pressure and humidity sensor information to a Gemini CGI page.

The CGI program was written in Python and I installed a bme280 and figlet library:

pip install RPi.bme280
pip install pyfiglet

The Python script (sersor2gmi.py) outputs a Gemini content type, a Figlet title and then the sensor data values:

#!//usr/bin/python3
#
# sersor2gmi.py - send BME280 sensor data to a Gemini page
#
import smbus2
import bme280
import pyfiglet

# find device address via: i2cdetect -y 1
port = 1
address = 0x77
bus = smbus2.SMBus(port)

calibration_params = bme280.load_calibration_params(bus, address)

# the sample method returns a compensated_reading object
data = bme280.sample(bus, address, calibration_params)

# Output a Gemini page
print("20 text/gemini") #Note: Some Gemini CGI servers may do this
print("```") # use code mode
#print(pyfiglet.figlet_format("Pi BME280 Data", font = "small"))
print(pyfiglet.figlet_format("Pi BME 280 Data", font = "small"))

print("Temperature:" + "{:5.1f}".format(data.temperature) + " C" )
print("Pressure:   " + "{:5.1f}".format(data.pressure) + " kPa" )
print("Humidity:   " + "{:5.1f}".format(data.humidity) + " %" )
print("```") 

This file was added to the cgi-bin directory and it is made executeable (chmod +x sersor2gmi.py).

Below is the output seen in the Lagrange Gemini browser:

Final Comments

There are a variety of Gemini browsers for Windows, Android, Mac and Linux so if you’re looking for a quick and dirty internet solution Gemini might be a good solution.

I like how Gemini documents are totally readable, I can’t say the same for most web pages.

The one thing that I missed with Gemini pages is the ability of show nice charts, text based bars work for simple stuff but doing text based line charts is a little too ugly for me.

Re-purpose Your Old Home Router

We had some old routers at home that I thought that I’d see if I could re-purpose.

Some of the typical applications that can be done are:

  • A wireless repeater / access point
  • A remote switch
  • A print server for USB printers
  • A network storage device (with a portable USB drive)

A router can also be used as a low end application platform and it can do many projects that a Raspberry Pi might do. In this blog I will be looking at:

  • USB drives
  • Web Cams
  • USB Sensors
  • USB Connections to Arduino/Micro:bit devices
  • CGI Web Servers

Comparing a Router to a Raspberry Pi

Before getting started it’s useful to know how a re-purposed router stacks up against a Raspberry Pi.

For our router project we kept things super simple. We used command line apps and scripts rather than Python.

Pro’s of a Router:

  • Price. I got my routers used from $5-$10. A new Pi 3 or 4 is $35-$50.
  • Housing. A router is ready to be mounted
  • Network Ready. If you’re looking at networking functionality you are good to go.
  • Lots of Basic Apps. Most of key features are available.

Con’s of a Router:

  • Extremely Space Limited. Most of the routers have 32MB vs. 8+ GB of a Pi
    • Need to Pick Apps. You probably can’t load Python3, Apache and Samba at the same time.
    • Probably can’t run X-Windows.
  • Ash not Bash. The lighter weight Ash shell is supported not Bash. This may not be an issue.

Open Source Router Firmware

There are a number of open source firmware solutions that can put life back in old routers. OpenWRT and DD-WRT are the most popular packages, but there are others.

The first step is to determine if your old router is supported with one of these packages. It is important to note that many of the older routers only have 4MB of flash and 32MB of RAM, and this may not run or only marginally run OpenWRT or DD-WRT. The recommended requirements are 8+ MB of flash and 32+ MB of RAM.

Another consideration is USB support. A router without USB support can still be used as a web or application server but it will be missing external hardware integration.

For our router project we used OpenWRT.

Getting Started

How the re-purposed router will be used will effect the setup. It is important to ensure the re-purposed router does not effect other routers in your home LAN. Typically you’ll want to disable Router Features such as the DNS and DHCP server capabilities. In the OpenWRT web interface (Luci) software services can be disabled.

I did lots of playing and I bricked my router about a dozen times. Luckily it is usually possible to un-brick a router. I only permanently bricked one 1 router and it was with a bad DD-WRT firmware.

Un-brinking a Router

Often all you need to do is just reset your router and then connect directly to a LAN port to redo your configuration.

If this fails check the OpenWRT blog for any recommendations for your router. There are some excellent custom solutions but they are usually manufacturer specific.

For Netgear see nmrpflash , it’s an almost 100% sure proof un-brinking solution.

30-30-30 Hard Reset Rule

If resetting the router doesn’t work, the next set is the 30-30-30 Rule. This will typically work for all routers:

  • Press reset button for 30 seconds.
  • While holding, unplug router for 30 seconds
  • power on, still holding setup button 30 seconds.
  • Let go of reset button. Then try to reconfigure. (If this doesn’t work try another power down).

Adding Software Features

OpenWRT uses opkg the OpenWRT package manager. Software can be added either through the Luci web interface or manually in an SSH shell.

After a new install the opkg list of available packages will need to be update, this can be done from SSH by:

## update okpg is required to get info on new packages
opkg update

It is possible to use a USB drive to extend the router’s root file system, this would remove issues of running out space when installing software packages.

Add USB Support

Typically routers do not have a lot of available space. Use the df command to check space:

root@OpenWrt:~# df
Filesystem           1K-blocks      Used Available Use% Mounted on
/dev/root                 2560      2560         0 100% /rom
tmpfs                    30060       228     29832   1% /tmp
/dev/mtdblock5           12160       488     11672   4% /overlay
overlayfs:/overlay       12160       488     11672   4% /
tmpfs                      512         0       512   0% /dev

OpenWRT supported software is loaded directly onto the router so some care is required when deciding which packages are to be installed.

Adding a USB drive can greatly help with the management of source files and backups.

The okpg (OpenWRT package manager) is used used to install software packages. To install USB drive support:

## update okpg is required to get info on new packages
opkg update
## get USB packages
opkg install block-mount e2fsprogs kmod-fs-ext4 kmod-usb-storage kmod-usb2 kmod-usb3

A useful command line USB tool is lsusb, however check how much space you have before installing it. To install lsusb enter:

opkg install usbutils

root@OpenWrt:~# lsusb
Bus 001 Device 003: ID 0a5c:bd17 Broadcom Corp. BCM43236 802.11abgn Wireless Adapter
Bus 001 Device 004: ID 413d:2107  
Bus 001 Device 006: ID 1871:0143 Aveo Technology Corp. 
Bus 001 Device 005: ID 058f:6387 Alcor Micro Corp. Flash Drive
Bus 001 Device 002: ID 05e3:0610 Genesys Logic, Inc. 4-port hub
Bus 001 Device 001: ID 1d6b:0002 Linux Foundation 2.0 root hub

After the USB packages are loaded the router needs to be rebooted.

A new menu item called “Mount Points” under the “System” heading will be added to the Luci web interface. This option allows for easy addition and removal of USB drives.

If you are mounting more that 1 device, it is recommended to use the UUID label, otherwise there is potential that the mount points could be swapped between devices.

USB-Serial Connections to Arduino/Micro:Bit Controllers

A router doesn’t have externally exposed GPIO (General Purpose Input/Output) pins like a Raspberry Pi or an Arduino. Some routers do have internal GPIO pins but you’ll need to soldered to these pins to expose them.

A good workaround to this problem is to use some older micro-controllers. Low end modules like the Arduino Nano, littleBit Arduino Bit or a BBC Micro:bit can be directly connected to sensors and the data can be passed with the USB-to-Serial interface.

To enable USB-to-Serial communications in OpenWRT:

## add USB-Serial packages
opkg install kmod-usb-serial kmod-usb-acm
## add terminal config package
opkg install coreutils-stty

For our home project we used a BBC Micro:bit. The Micro:bit is very user friendly for coding, it only took 5 blocks to send the on-board temperature reading and to show the value on the front panel.

The OpenWRT firmware runs a light version of the Bash shell called Ash. Below is an Ash script that will connect to the Micro:bit USB port and print out the temperature data:

#!/bin/ash
#
# microbit.sh - reads microbit temperature
#  
# set terminal speed
stty -F /dev/ttyACM0 115200 
# read USB-Serial device (/dev/ttyACM0)
while read -r line < /dev/ttyACM0; do
  echo " Temp: $line DegC"
done

Once the sensor data has been passed to the router, the data could be passed to other network nodes using light weight protocols like MQTT or memcached. Both of these options support Bash/Ash interfaces.

USB Web Cam

There are a number of different USB video solutions that are available. Some packa like Motion are full featured but they might require more resources than your router supports.

For light weight USB video support mjpeg streamer is good place to start. It can be installed by:

opkg install kmod-video-uvc mjpg-streamer

The configuration of the video is defined in: /etc/config/mjpg-streamer .

config mjpg-streamer 'core'
        # option enable '1' = on, '0'=off
        option enabled '1'
        option input 'uvc'
        option output 'http'
        option device '/dev/video0'
        option resolution '1280x1024'
        option yuv '0'
        option quality '80'
        option fps '5'
        option led 'auto'
        option www '/www/webcam'
        option port '8080'
        #option listen_ip '192.168.1.1'
        option username 'openwrt'
        option password 'openwrt'

The video service needs to be started:

## to start the service:
/etc/init.d/mjpg-streamer start
## to enable the service to start on boot
/etc/init.d/mjpg-streamer enable

The webcam is available on a router’s IP with the default port of 8080.

USB Sensors

Most sensors are 0-5 Volts (i.e. the typical Arduino sensor) but there are a few USB sensors.

I purchased a USB Thermometer (~$12 from Walmart).

The USB Thermometer acted like a human interface devices (HID)

A script in Ash (a light weight version of Bash) is used to get the temperature, and it required a few software packages:

opkg install kmod-usb-hid coreutils-printf xxd nano

The temper.sh script sends a query message to the HID address and then exacts the temperature from the returned binary data:

#!/bin/ash
#
# temper.sh - get temperature from a USB thermometer 
#
hid="/dev/hidraw1"
exec 5<> $hid
# send out query msg
echo -e '\x00\x01\x80\x33\x01\x00\x00\x00\x00\c' >&5
# get binary response
OUT=$(dd count=1 bs=8 <&5 2>/dev/null | xxd -p)
# characters 5-8 is the temp in hex x1000
HEX4=${OUT:4:4}
DVAL=$(printf '%d' "0x${HEX4}")
awk "BEGIN {print ($DVAL/100)}"

The script can then set to an executable or run thru ash:

root@OpenWrt:/usbstick# ash temper.sh
24.43
root@OpenWrt:/usbstick# chmod +x temper.sh
root@OpenWrt:/usbstick# ./temper.sh
24.43

CGI Web Server

The OpenWRT runs the Luci configuration interface on a light weight uhttp server. This web server can also be used for custom pages.

The uhttp web server will run Lua or CGI pages from the /www/cgi-bin directory.

Below is an Ash CGI example that runs the usb thermometer script and the vmstat command:

#!/bin/ash
#
# test.cgi - show the USB thermometer temperature and CPU stats 
#
echo "Content-type: text/html"
echo ""
echo "<!DOCTYPE html>
<html>
<head><title>Router Points</title>
</head>
<body>
<h1>Router CGI Page</h1><hr>"

echo "Temperature:  $(/littlehd/temper.sh) DegC"

echo "<hr><h3>Router CPU Stats</h3>"
echo "<pre>"
vmstat
echo "</pre>"

Final Comments

I found that re-purposing a router was an interesting project that could progress to a number of interesting side topics such as:

  • USB over IP – use an old router to remotely configure Arduino or ESP32 devices.
  • Remote network sniffer or traffic monitor
  • Print and Network Storage Servers