For many projects an SQL database is overkill for simple storage of values. As an alternate solution there are a number of excellent distributed in-memory caching systems that can be used.
Two of the most popular in-memory caching systems are Redis and Memcached.
I’m big fan of Redis, and I’ve enjoyed doing projects with it. Redis offer some awesome speed with a small footprint, and it has many features that make it even superior to a messaging system like MQTT.
However if you’re looking for something 100% dead simple you should take a look a memcached, it has a super simple setup and with only a dozen commands so you’ll get up and running in no time.
There are API’s in all the common programming languages. It only takes 1 line of Bash to read or write to memchached.
Getting Started
Memached can be installed on all major OS’s. To install it on Ubuntu/Raspberry Pi:
sudo apt-get install memcached
If you’re using Docker there are some lightweight memcached images (89MB) that can be used. To run the memcached docker image the –net host option should be used:
$ sudo docker run -it --net host memcached
The -it (interactive) option could be useful if there are any errors kicked out.
See the man pages for a full description of memcached options.
Telnet can be used to enter manual enter commands (port 11211 is the default):
$ telnet localhost 11211
Trying 127.0.0.1...
Connected to localhost.
Escape character is '^]'.
version
VERSION 1.6.12
lru_crawler metadump all
key=mynum exp=-1 la=1642363105 cas=2 fetch=no cls=1 size=66
END
quit
Connection closed by foreign host.
For more information on the commands and Telnet.
Bash Interface to Memcached
There are lot of help guides for other Python, PHP, GO etc., but not a lot on how to use Bash.
The Bash nc command can be used to read and write to socket. The -q option will close the socket after 0 seconds (after the echo command is sent). The default port is 11211 but this can be changed along with adding some security options. The first example is to use the stats command:
$ echo "stats" | nc -q 0 192.168.0.120 11211
STAT pid 1
STAT uptime 722
STAT time 1642359428
STAT version 1.6.12
...
END
$ # Find the number of current key items
$ echo "stats" | nc -q 0 192.168.0.120 11211 | grep curr_items
STAT curr_items 1
Set/Get a Key and Value
To set an in-memory key-value store, the syntax is:
set mykey <flags> <ttl> <size>
value
The flags option is typical set to 0. The ttl “time to live” is in seconds, a ttl of 0 is indefinite.The size of the value also needs to be define. (This is taken care of in the Python, PHP… libraries). It’s important to note that a newline (\r\n) is required before the value.
To set a key to a variable mynum with a value of 55 and a indefinite time to live:
$ # hard code the arguments in a set key command
$ # Note: echo -e is used to pass the \r\n for new lines
$ echo -e "set mynum 0 0 2 \r\n55\r" | nc -q 0 127.0.0.1 11211
STORED
A more flexible approach would be to pass variables:
$ # Set a new key/value pair, with no flags and indefinite timeout
$ ipp="192.168.0.120 11211"
$ thekey="mykey1"
$ thenum=55
$ numsize=${#thenum}
$ # send command
$ echo -e "set $thekey 0 0 $numsize\r\n$thenum\r" | nc -q 0 $ipp
STORED
Get a key/value
The get key command returns 3 lines with the value being the 2nd line. Some awk code can be used to parse out just the value.
$ # Get a new key/value pair
$ ipp="192.168.0.120 11211"
$ thekey="mykey1"
$
$ echo "get $thekey" | nc -q 0 $ipp
VALUE mykey1 0 2
60
END
$ # Get just the 2nd line with the value
$ echo "get $thekey" | nc -q 0 $ipp | awk '{if (NR == 2) print $0}'
55
$
$ # Store the result in a variable
$ mykey1=$(echo "get $thekey" | nc -q 0 $ipp | awk '{if (NR == 2) print $0}')
$ echo "mykey1 = $mykey1"
mykey1 = 55
Increment/Decrement a Value
The inc / decr commands will increase or decrease a numeric stored key value by a defined amount:
$ #incr/decr a value's number
#
ipp="192.168.0.120 11211"
thekey="mykey1"
# Get starting value
echo "get mynum" | nc -q 0 $ipp | awk '{if (NR == 2) print $0}'
55
thediff=10; # increase the value (only positive values)
# incr and show the value
echo "incr $thekey $thediff" | nc -q 0 $ipp
65
thediff=5; # decrease the value (only positive values)
# incr and show the value
echo "decr $thekey $thediff" | nc -q 0 $ipp
60
Prepending and Appending
Memcached does not have queue or list functionality, if you need this take a look at Redis.
Below is an example script that creates a diagnostic log in a key/value. The append command adds msg text to the end of the overall string.
#!/usr/bin/bash
#
# diagmsg.sh - append msgs to a memcached variable string
#
ipp="192.168.0.120 11211"
msg="1:00 - Base Software Loaded\r"
size=${#msg}
echo -e "set mymsg 0 0 $size\r\n$msg\n\r" | nc -q 0 $ipp
# Create an array of diagnostics
diagmsgs=("2:00 - System Started\r" "2:15 - Getting Data\r" "2:30 - Backing up\r")
# Append array to value text
for msg in "${diagmsgs[@]}"
do
size=${#msg}
#echo "Size: $size $msg";
echo -e "append mymsg 0 0 $size\r\n$msg\n\r" | nc -q 0 $ipp;
done
# Show the result
echo -e "\nDiagnostic Results\n"
echo "get mymsg" | nc -q 0 $ipp
The results of this script would be:
$ bash diagmsg.sh
STORED
STORED
STORED
STORED
Diagnostic Results
VALUE mymsg 0 92
1:00 - Base Software Loaded
2:00 - System Started
2:15 - Getting Data
2:30 - Backing up
END
Final Thought
I played with the Python and PHP library and they were quite easy to use.
By adding a Bash interface to memcache it allows me to use programs like Octave/Matlib where a native interface isn’t available (just use the System call and pass the Bash code).
Sample Code: PHP
PHP example:
<?php
// memc.php - test memcached
//
// Load the PHP memcached component:
//
// sudo apt-get install php-memcached
$memcache = new Memcached();
$memcache->addServer('localhost', 11211) or die ("Could not connect");
$version = $memcache->getVersion();
var_dump( $version);
$tmp_object = new stdClass;
$tmp_object->str_attr = 'test';
$tmp_object->int_attr = 123;
$memcache->set('key', $tmp_object);
echo "Store data in the cache (data will expire in 10 seconds)<br/>\n";
$get_result = $memcache->get('key');
echo "Data from the cache:<br/>\n";
var_dump($get_result);
?>
Sample Code: Python
Python example:
$ # Install Python memcache library
$ python3 -m pip install pymemcache
$ # Example set/get
$ python3
Python 3.8.10 (default, Nov 26 2021, 20:14:08)
[GCC 9.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> from pymemcache.client import base
>>> client = base.Client(('localhost', 11211))
>>> client.set('mykey','teststring')
True
>>> client.get('mykey')
b'teststring'
>>> client.set('mynum',44)
True
>>> client.get('mynum')
b'44'
Fun Tech Projects 
One thought on “Memcached: fast lightweight network cache”