Home Assistant History on Node-Red Charts

Home Assistant is an open source home automation platform that can monitor and control smart home devices and it integrates with many of other common systems.

HA_demo

Home Assistant installation is targeted for Raspberry Pi’s but other hardware options are available.

I was very impressed how easy it was to install Home Assistant and get a basic home integration system up and running.

There is a huge number of integration solutions (1500+) that connect to most of the mainstream products. However if you want to do some custom programming with connections to Arduino or other Raspberry Pi or PCs there isn’t an easy “out of the box” solution.  To solve this requirement Home Assistant has included Node-Red as an add-on.

Node-RED is a visual programming tool for wiring together hardware devices, APIs and online services.

For information how to install Node-Red on Home Assistant see the HA documentation. (I wrote a blog for my installation).

Home Assistant History

The default installation of Home Assistant has history enabled, and the data is stored in local SQLite database (home-assistant_v2.db) within your configuration directory unless the recorder integration is set up differently.

Charts of sensor history can be show in the Home Assistant Overview pages or as dialogs.

Using Node-Red with HA History

The Node-Red installation has a number of Home Assistant nodes that allow sensors data to be read and created. For viewing history the HA get history node is used.

A simple manual test circuit to get history would have: an injector, a get_history and a debug node.

By double-clicking on the get history node its configuration can be defined. For this example a sensor entity id of sensor.cpu_temp is used with a 10 minute (10m) time scale. When the injector is toggled the debug window will show an array to data and time entries.

3 Button History Chart

The logic to create a 3 button history chart would use: 3 dashboard buttons, 3 get history nodes, a Javascript function, and a dashboard chart node.

The Node Red chart node typically does local historical storage within the node itself. However the chart node can also be used to read external data and show a line chart of the data,nothing is stored locally (more info on this).

A javascript function node can be used to format the data into the required form:

//
// Format the HA History results to match the charts JSON format
//

var series = ["HA Values"];
var labels = ["Data Values"];
var data = "[[";
var thetime;
 
for (var i=0; i < msg.payload.length; i++) {
    thetime = (msg.payload[i].last_changed); // Note: check your format?
    data += '{ "x": "' + thetime + '", "y":' + msg.payload[i].state + '}';
    if (i < (msg.payload.length - 1)) {
        data += ","
    } else {
        data += "]]"
    }
}
var jsondata = JSON.parse(data);
msg.payload = [{"series": series, "data": jsondata, "labels": labels}];
return msg;

The payload data will be in the format of:

If everything is formatted correctly the Node Red dashboard page should look like:

For a more flexible presentation it would be good to use adjustable time periods.

Final Thoughts

For simple historical storage I found that the built-in History worked fine, however if you’re looking for custom long term storage then the InfluxDB add-on to HA might be a better solution.

X3D: Create 3D Graphics with Animation

There are some great 3D graphic packages out there. Unfortunately a lot of these packages have a pretty large learning curve. If you have some simple requirements and you want to get up and running fast X3D might be a good fit.

Extensible 3D (X3D) is a standard for the creation of 3D graphics and it’s an extension to the basic HTML code. X3D is supported on all major browser and if you’re familiar with Javascript and HTML syntax the learning curve is pretty quick.

This blog documents my notes and testing. I was looking at some X3D basics such as:

  • Create simple buttons with onclick events
  • Create viewpoints
  • Create inline models (for re-use)
  • Create a two button / two pump simulation
  • Create a tank filling simulation

Getting Started

There are some great tools like Unity3D and Adobe Maya that can be used to create 3D models and worlds.

For a first example, a button will be create from scratch. An onclick event will be used to change the colour and text.

<html> 
   <head>
    <meta http-equiv="X-UA-Compatible" content="IE=edge"/> 
     <title>Button Example</title> 
     <script type='text/javascript' src='https://www.x3dom.org/download/x3dom.js'> </script> 
     <link rel='stylesheet' type='text/css' href='https://www.x3dom.org/download/x3dom.css'></link>
     <script language="javascript">
		// Toggle Button colour and text
		function on_off() {
			if (document.getElementById('btn_col').getAttribute('diffuseColor') ==  '1 0 0') {
				document.getElementById('btn_col').setAttribute('diffuseColor', '0 1 0');
				document.getElementById('btn_text').setAttribute('string', 'On');
			} else {
				document.getElementById('btn_col').setAttribute('diffuseColor', '1 0 0');
				document.getElementById('btn_text').setAttribute('string', 'Off');			
			}
		}
	 </script>
	 
   </head> 
   <body> 
     <h1>Button Example</h1> 
 	
	 <x3d  width='100%' height='100%'> 
  
	   	<scene>
			<Viewpoint position="1.62945 0.74856 3.91868" orientation="-0.63681 0.76606 0.08730 0.50487" 
	zNear="2.56413" zFar="6.15946" centerOfRotation="0.00000 0.00000 0.00000" fieldOfView="0.78540" description="defaultX3DViewpointNode"></Viewpoint>
			<Shape>							
				<Appearance>
					<Material diffuseColor="0.8 0.8 0.8" />
				</Appearance>
	   			<Box size="1,1,1"/>
			</Shape>
			
			<Transform translation='0 0 0.4'>
				<Shape onclick="on_off();">
					<Appearance>
						<Material id="btn_col" diffuseColor="1 0 0" />
					</Appearance>

					<Sphere radius="0.4">
				</Shape>
			</Transform>
				
			<Transform translation='0 0 0.81'>					
				<Shape onclick="on_off();">
					<Appearance>
						<Material diffuseColor="0 0 0" />
					</Appearance>
					<Text id="btn_text" string="Off">
						<FontStyle size='0.25' />
					</Text>
				</Shape>
			</Transform>

		</scene> 
	</x3d> 
  </body> 
</html> 

The X3D viewer is built into all modern Web browsers. The x3d tag defines the size of the view space with a scene tag inside.

For this example there are 3 shapes, a box at the default co-ordinates, a sphere positioned in front of the box (on z-axis) and text in front of the sphere. The Transform tag is used to position items. Color is configured using a Material tag (as part of the Appearance tag).

The getAttribute and setAttribute function calls are used to adjust X3D items.

Navigation and Viewpoints

While the scene is being viewed the mouse and keyboard can be used to navigate around the scene.

Viewpoints can be created using the X3D debug mode:

  • Enter “d” – for debug mode,
  • use the mouse and keyboard to get a view that you like
  • Enter “v” – to see the viewpoint tag (copy this into your code)

It is useful to create multiple viewpoints and then have a mechanism to toggle between them. This can be done by giving the different viewpoints an ID:

<Viewpoint id="vdefault" position="1.62945 0.74856 3.91868" orientation="-0.63681 0.76606 0.08730 0.50487" 
	zNear="2.56413" zFar="6.15946" centerOfRotation="0.00000 0.00000 0.00000" fieldOfView="0.78540" description="defaultX3DViewpointNode">
</Viewpoint>

<Viewpoint id="top" position="1.07778 2.85553 3.04234" orientation="-0.89982 0.36019 0.24613 0.94283" 
	zNear="2.56413" zFar="6.15946" centerOfRotation="0.00000 0.00000 0.00000" fieldOfView="0.78540" description="defaultX3DViewpointNode">
</Viewpoint>

<Viewpoint id="bottom" position="1.89463 -2.26667 3.13753" orientation="0.68467 0.72282 0.09357 0.64533" 
	zNear="2.56413" zFar="6.15946" centerOfRotation="0.00000 0.00000 0.00000" fieldOfView="0.78540" description="defaultX3DViewpointNode">
</Viewpoint>

The next step is to have buttons, and use an onlick event to enable the “set_bind” attribute:

<button onclick="document.getElementById('vdefault').setAttribute('set_bind','true');" > Default View</button>

<button onclick="document.getElementById('top').setAttribute('set_bind','true');"> Top View</button>

<button onclick="document.getElementById('bottom').setAttribute('set_bind','true');"> Bottom View</button>  

Re-using Models with Inline Files

As the next example a pump model will be created, and then it will be called twice. The button example will also be called twice to control the pumps.

The first step is to create a basic pump.

<html> 
   <head>
    <meta http-equiv="X-UA-Compatible" content="IE=edge"/> 
     <title>Basic Pump</title> 
     <script type='text/javascript' src='https://www.x3dom.org/download/x3dom.js'> </script> 
     <link rel='stylesheet' type='text/css' href='https://www.x3dom.org/download/x3dom.css'></link> 
   </head> 
   <body> 
	 <x3d  width='800px' height='600px'>   
	   <scene>
		<Group DEF = "hpipe" description= "Horizonal pipe">
			<transform rotation="0 0 1 1.5708" translation='-0.5 -0.35 0'> 
				<Shape description= "Outer pipe" >
					<Appearance>
						<Material id="pumpcol" diffuseColor='0.8,0.8,0.8'   transparency='0' ></Material>  
					</Appearance>
					<Cylinder  radius="0.15" height="1"></Cylinder>
				</Shape>		   
			</transform>						
		</Group>
		   		   
		<transform  rotation="1 0 0 1.5708" translation='0 0 0'> 
			<Shape description= "pump body" >
				<Appearance>
					<Material id="pumpbcol" diffuseColor='0.8,0.8,0.8'   transparency='0' ></Material>  
				</Appearance>
				<Cylinder  radius="0.5" height="0.5"></Cylinder>
			</Shape>		   
		</transform>
		<transform  rotation="1 0 0 1.5708" translation='0 0 0'> 
			<Shape description= "pump body inner piece" >
				<Appearance>
					<Material id="pumpbcol" diffuseColor='0,0,0'   transparency='0' ></Material>  
				</Appearance>
				<Cylinder  radius="0.25" height="0.51"></Cylinder>
			</Shape>		   
		</transform>

		<Transform translation="1 0.7 0"> 
			<Group USE="hpipe"/>
		</Transform>
	</scene> 
	</x3d> 
   </body> 
</html> 

This pump model has a Group tag that is used to define a horizontal pipe,<Group DEF = “hpipe” >, that is used as the in coming pipe on the pump. This grouped tag is re-used for the out going pipe, <Group USE=”hpipe”> .

An Inline X3D is not the same as the HTML version of the X3D. The inline file:

  • Starts with : <?xml version=”1.0 encoding=”UTF=8?>
  • Has no: <html> or <body> tags
  • Has an <X3D>, <Scene> with </Scene> and </X3D> tags

The X3D inline file (pump_mod.x3d) is as follows:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE X3D PUBLIC "ISO//Web3D//DTD X3D 3.0//EN" "http://www.web3d.org/specifications/x3d-3.0.dtd">
<X3D version="3.0" profile="Immersive" xmlns:xsd="http://www.w3.org/2001/XMLSchema-instance" xsd:noNamespaceSchemaLocation="http://www.web3d.org/specifications/x3d-3.0.xsd">
	<head>
		<meta name="filename" content="Deer.x3d" />
		<meta name="generator" content="Blender 2.70 (sub 0)" />
	</head>
	<Scene>
 
<Group id="pipe1" DEF = "hpipe" description= "Horizonal pipe">
	<transform rotation="0 0 1 1.5708" translation='-0.5 -0.35 0'> 
		<Shape description= "Outer pipe" >
			<Appearance>
				<Material id="pumpcol" diffuseColor='0.8,0.8,0.8'   transparency='0' ></Material>  
			</Appearance>
			<Cylinder  radius="0.15" height="1"></Cylinder>
		</Shape>		   
	</transform>			
</Group>
<transform  rotation="1 0 0 1.5708" translation='0 0 0'> 
	<Shape description= "pump body" >
		<Appearance>
			<Material id="pumpbcol" diffuseColor='0.8,0.8,0.8'   transparency='0' ></Material>  
		</Appearance>
		<Cylinder  radius="0.5" height="0.5"></Cylinder>
	</Shape>		   
</transform>
<transform  rotation="1 0 0 1.5708" translation='0 0 0'> 
	<Shape description= "pump body inner piece" >
		<Appearance>
			<Material id="pumpbcol" diffuseColor='0,0,0'   transparency='0' ></Material>  
		</Appearance>
		<Cylinder  radius="0.25" height="0.51"></Cylinder>
	</Shape>		   
</transform>

<Transform translation="1 0.7 0"> 
	<Group id="pipe2" USE="hpipe"/>
</Transform>

  </Scene> 	 
</X3D> 

Inline files can only be used with Web Servers, they can not be called locally (with a file://).

An example of inline buttons and pumps is below:

<html>
<head>
    <meta http-equiv="X-UA-Compatible" content="IE=edge"/>
    <title>Pump Example</title>
    <script type='text/javascript' src='https://www.x3dom.org/download/x3dom.js'> </script>
    <link rel='stylesheet' type='text/css' href='https://www.x3dom.org/download/x3dom.css'/>
     <script language="javascript">
		function toggle(butid, pumpid) {
            // turn on/off a pump and change the button color and text
			if (document.getElementById(pumpid + '__pumpcol').getAttribute('diffuseColor') ==  '0.8,0.8,0.8') {
				document.getElementById(pumpid + '__pumpcol').setAttribute('diffuseColor', '0 1 0');
				document.getElementById(pumpid + '__pumpbcol').setAttribute('diffuseColor', '0 1 0');
                document.getElementById(butid + '__btn_col').setAttribute('diffuseColor', '0 1 0');
				document.getElementById(butid + '__btn_text').setAttribute('string', 'On');
			} else {
				document.getElementById(pumpid + '__pumpcol').setAttribute('diffuseColor', '0.8,0.8,0.8');
				document.getElementById(pumpid + '__pumpbcol').setAttribute('diffuseColor', '0.8,0.8,0.8');	
                document.getElementById(butid + '__btn_col').setAttribute('diffuseColor', '1 0 0');
				document.getElementById(butid + '__btn_text').setAttribute('string', 'Off');	
			}
		}
	 </script>
    
</head>
<body>
<h1>Button/Pump Control</h1>
<x3d runtimeEnabled="True" width='800px' height='800px'>
    <Scene>
        <Viewpoint position="0.23317 2.51382 5.21442" orientation="-0.99984 0.00995 -0.01482 0.71962" 
             zNear="2.86396" zFar="10.49846" centerOfRotation="0.00000 0.00000 0.00000" fieldOfView="0.78540" >
        </Viewpoint>
        
        <Transform translation='-1 0 0'> 
           <inline nameSpaceName="pump1"   url="pump_mod.x3d" /> 
        </Transform>    
        <Transform translation= '1 0 0'> 
           <inline nameSpaceName="pump2"   url="pump_mod.x3d" /> 
        </Transform>

        <Billboard axisOfRotation='0 0 0' >

            <Transform translation='-1 -2 0'> 
               <inline nameSpaceName="button1" onclick="toggle('button1','pump1')"  url="button.x3d" /> 
            </Transform>
            <Transform translation='1 -2 0'> 
               <inline nameSpaceName="button2" onclick="toggle('button2','pump2')"  url="button.x3d" /> 
            </Transform>

        </Billboard>
    </Scene>

</x3d>
</body>
</html>

The inline nameSpaceName is used to access id’s in the inline models. The nameSpaceName precedes all of the ids in the top level code, so an id of pumpcol in the nameSpaceName of pump1 is accessed as pump1__pumpcol.

A Billboard tag is used for positioning of the buttons. A Billboard keeps its contents facing forward even if the rest of the scene is rotated. Billboards are excellent for scene controls, gauges etc.

An inline example with the pump and button files can be accessed at: http://metcalfepete.alwaysdata.net/pump2.htm

Timers

X3D files can use the standard Javascript setTimeout() function.

As an test I created a tank filling and emptying example (http://metcalfepete.alwaysdata.net/tank.htm). The filling/emptying function (fill_tank()) is called every second and the liquid level (ilevel) and liquid height is adjusted either up or down.

The pipes and the tank can be made semi-tranparent with: transparency=’0.6′ (where 0=solid, 1=fully transparent).

The liquid in the pipes is not shown by: render=”False”

The full code for this example is:

<html> 
   <head>
    <meta http-equiv="X-UA-Compatible" content="IE=edge"/> 
     <title>Tank Example</title> 
     <script type='text/javascript' src='https://www.x3dom.org/download/x3dom.js'> </script> 
     <link rel='stylesheet' type='text/css' href='https://www.x3dom.org/download/x3dom.css'></link> 
     <script language="javascript">
		var ipipe = 0;
		var ilevel = 0;
		var ito = 0; //Timeout variable
		 
		function fill_tank() {
		 	if (ito != 0 ) { clearTimeout(ito);}
		 	document.getElementById('in_liq').setAttribute('render', 'True');
		 	document.getElementById('out_liq').setAttribute('render', 'False');
		 	document.getElementById('sliq').setAttribute('render', 'True');
		 	if (ilevel < 1.8) {
				ilevel = ilevel + 0.1;
				var ypos = -1 + (ilevel/2);
				document.getElementById("tliq").setAttribute('translation', "1 " + ypos.toString() + " 0");
				document.getElementById('liquid').setAttribute('height', ilevel.toString());
				var pfill = 100*ilevel/1.8
				document.getElementById('tank101').setAttribute('string', 'Filling: ' + parseInt(pfill) + '%');
				ito = setTimeout(fill_tank,1000);
	 
			} else {
				document.getElementById('in_liq').setAttribute('render', 'False');
		 		document.getElementById('tank101').setAttribute('string', 'Tank Full');			 
			}
		}
		function empty_tank() {
			if (ito != 0 ) { clearTimeout(ito);}
		 	document.getElementById('in_liq').setAttribute('render', 'False');
		 	document.getElementById('out_liq').setAttribute('render', 'True');
		 	document.getElementById('sliq').setAttribute('render', 'True');
		 	if (ilevel >= 0.1) {
				ilevel = ilevel - 0.1;
				var ypos = -1 + (ilevel/2);
				document.getElementById("tliq").setAttribute('translation', "1 " + ypos.toString() + " 0");
				document.getElementById('liquid').setAttribute('height', ilevel.toString());
				var pfill = 100*ilevel/1.8
				document.getElementById('tank101').setAttribute('string', 'Emptying: ' + parseInt(pfill) + '%');
				ito = setTimeout(empty_tank,1000);
	 
			} else {
				document.getElementById('out_liq').setAttribute('render', 'False');
		 		document.getElementById('tank101').setAttribute('string', 'Tank Empty');			 
			}
		}	
	 </script>
   </head> 
   <body> 
     <h1>Storage Tank </h1> 
 
	 <x3d  width='1200px' height='600px'> 
  
	   <scene>
		<Viewpoint  id="frontview" position="0.87500 -0.21796 4.11873" orientation="0.00000 -1.00000 0.00000 0.03041" 
	      zNear="1.45981" zFar="10.47667" centerOfRotation="1.00000 0.00000 0.00000" fieldOfView="0.78540">
		 </Viewpoint>

		<Viewpoint id="topview" position="2.64724 1.92990 3.25410" orientation="-0.74251 0.64386 0.18471 0.71156" 
	      zNear="1.45981" zFar="10.47667" centerOfRotation="1.00000 0.00000 0.00000" fieldOfView="0.78540" >
		</Viewpoint>
		   
		<Group id="pipe1" DEF = "hpipe" description= "Horizonal pipe">
			<transform rotation="0 0 1 1.5708" translation='-1 0.5 0'> 
				<Shape description= "Outer pipe" >
					<Appearance>
						<Material diffuseColor='0.8,0.8,0.8'   transparency='0.6' ></Material>  
					</Appearance>
					<Cylinder  radius="0.15" height="2.3"></Cylinder>
				</Shape>		   
			</transform>
		</Group>
		<transform rotation="0 0 1 1.5708" translation='-1 0.5 0'> 
			<Shape id="in_liq" description= "Liquid in pipe" render="False">
				<Appearance>
					<Material diffuseColor='1.0 0 1.0'   transparency='0' ></Material>  
				</Appearance>
				<Cylinder  radius="0.10" height="2.3"></Cylinder>
			</Shape>		   
		</transform>

		<Transform translation="4 -1.25 0"> 
			<Group id="pipe2" USE="hpipe"/>
		</Transform>
		   
		<Transform rotation="0 0 1 1.5708" translation='3 -0.75 0'> 
			<Shape id="out_liq" description= "Liquid in pipe" render="False">
				<Appearance>
					<Material diffuseColor='1.0 0 1.0'   transparency='0' ></Material>  
				</Appearance>
				<Cylinder  radius="0.10" height="2.3"></Cylinder>			
			</Shape>		   			
		</Transform>
		   
		<transform translation='1 0 0'> 
			<Shape  DEF="tank" id="tank" >
				<Appearance>
					<Material diffuseColor='0.8,0.8,0.8'   transparency='0.6' ></Material>  
				</Appearance>
				<Cylinder  radius="1" height="2"></Cylinder>
			</Shape>		   
		</transform>  
		   
		<transform id="tliq" translation='1 -0.9 0'> 
			<Shape DEF="liquid" id="sliq" render="False">
				<Appearance>
					<material diffuseColor="1.0 0 1.0" transparency='0'></material> 
				</Appearance>
				<Cylinder id="liquid" radius="0.9" height="0" render="False" ></Cylinder>
			</Shape>		   
		</transform> 	   

		   
		<Transform translation='1 0 1'>
		  <Shape>
			<!-- Text -->
			<Text id="tank101" length='0' maxExtent='2' lit="false" string='"Tank 101"'>
			   <FontStyle size='0.25' />
			</Text>
			<Appearance>
				<material diffuseColor="0 0 0" specularColor="0 0 0" transparency='0'></material> 
			</Appearance>
		  </Shape>
		</Transform>		   
		   
	</scene> 
	</x3d> 
   <button onclick="fill_tank()" >Fill Tank</button>
   <button onclick="empty_tank()">Empty Tank</button>
   <button onclick="document.getElementById('frontview').setAttribute('set_bind','true');">Front View</button>
   <button onclick="document.getElementById('topview').setAttribute('set_bind','true');">Top View</button> 

   </body> 
</html> 

Final Thoughts

I just scratched the surface on X3D, and I found that it’s a huge topic. For sure that biggest issue is building the models.

It would be interesting to connect IoT data to a X3D module using MQTT or REST calls.

Gauges in a Python Canvas

There are some nice Python packages like tk_tools, that can be used for IoT indicators and gauges.

My daughter and I had a project where we wanted to repurpose an old eReader to be a kitchen kiosk display. Unfortunately tk_tools doesn’t support Python 2.7, gray scale or larger text.

This blog documents how we made some simple update-able gauges using Python Tkinter Canvas objects that are supported in both Python 2.7 and 3.x .

Getting Started

Unfortunately the Python 2 and 3 Tkinter libaries are named differently (Tkinter in 2.7 vs tkinter in 3.x). If you are coding for both Python 2.7 and 3.x this gets messy, a simple workaround in your code is:

# Manage Python 2.7 and 3.x
#
import sys
# Check the version of Python and use the correct library
if sys.version_info[0] == 2:
    import Tkinter
else:
    import tkinter as Tkinter

Analog Clock

A Tkinter canvas supports a number of basic objects such as rectangles, circles, arcs, text, etc. The basic objects are positioned within the canvas space.

I found that as a first example an analog clock was a good place start. The first pass code for a clock with just the second hand would be:

# A Clock Second Hand Example
#
import tkinter as Tkinter # Python 3.x
import datetime

def update_sec():
    #Reposition the second hand starting position
    thesec = datetime.datetime.now().second
    arcstart = 90 - thesec*6  #0 sec = 90deg
    C.itemconfig(asec,start=arcstart) #pass the new start position
    C.after(1000, update_sec)

# Create a canvas object with an oval face and a second hand
top = Tkinter.Tk()

C = Tkinter.Canvas(top, bg="silver", height=250, width=300)
C.pack()


coord = 10, 50, 240, 210 
C.create_oval(coord,  fill="white")
# Have the second hand start at the top (90 deg) with 1 deg arc
asec = C.create_arc(coord, start=90, extent=1, width=3)

C.after(1000, update_sec)
top.mainloop()

The key point is to get the id of the seconds hand arc (asec). The itemconfig method is then used to change the starting position of seconds hand arc (C.itemconfig(asec,start=arcstart) ).

The arc positioning is a little backwards, 0 degrees is at 3o’clock and then goes counter-clockwise.

The next step is to add narrow arcs for the minutes and hours. Also text could be used to digitally show the date and time. For the hour and minute hand I used different colours and thicknesses.

#
# A Clock Example
#
import tkinter as Tkinter # Python 3.x
from datetime import datetime

def update_sec():
    # Position the hands 
    C.itemconfig(asec,start= 90 - datetime.now().second*6)
    C.itemconfig(amin,start= 90 - datetime.now().minute*6)
    C.itemconfig(ahour,start= 90 - datetime.now().hour*360/12)
    C.itemconfig(dtime,text = datetime.now().strftime("%d/%m/%Y %H:%M:%S"))
    C.after(1000, update_sec)

# Create a canvas object with an oval face and a second hand
top = Tkinter.Tk()

C = Tkinter.Canvas(top, bg="silver", height=250, width=300)
C.pack()

coord = 10, 50, 240, 210
C.create_oval(coord,  fill="white")
# Have the second hand start at the top (90 deg) with 1 deg arc
asec = C.create_arc(coord, start=90, extent=1, width=2)
amin = C.create_arc(coord, start=90, extent=1, width=4, outline='blue')
ahour = C.create_arc(coord, start=90, extent=1, width=6, outline='red')
dtime = C.create_text(120,20, font="Times 16 bold", text="00:00:00")

C.after(1000, update_sec)
top.mainloop()

Gauges

There are a number of different types of gauges. My first example was a speedometer graph, that used an arc for both the background and the gauge needle:

#
# Use Canvas to create a basic gauge
#
from tkinter import *
import random

def update_gauge():
    newvalue = random.randint(low_r,hi_r)
    cnvs.itemconfig(id_text,text = str(newvalue) + " %")
    # Rescale value to angle range (0%=120deg, 100%=30 deg)
    angle = 120 * (hi_r - newvalue)/(hi_r - low_r) + 30
    cnvs.itemconfig(id_needle,start = angle)
    root.after(3000, update_gauge)

    
# Create Canvas objects    

canvas_width = 400
canvas_height =300

root = Tk()

cnvs = Canvas(root, width=canvas_width, height=canvas_height)
cnvs.grid(row=2, column=1)

coord = 10, 50, 350, 350 #define the size of the gauge
low_r = 0 # chart low range
hi_r = 100 # chart hi range

# Create a background arc and a pointer (very narrow arc)
cnvs.create_arc(coord, start=30, extent=120, fill="white",  width=2) 
id_needle = cnvs.create_arc(coord, start= 119, extent=1, width=7)

# Add some labels
cnvs.create_text(180,20,font="Times 20 italic bold", text="Humidity")
cnvs.create_text(25,140,font="Times 12 bold", text=low_r)
cnvs.create_text(330,140,font="Times 12 bold", text=hi_r)
id_text = cnvs.create_text(170,210,font="Times 15 bold")

root.after(3000, update_gauge)

root.mainloop()

The basic gauge can be enhanced to have more value ranges and colour hihi/hi/low ranges:

#
# Use Canvas to create a basic gauge
#
from tkinter import *
import random

def update_gauge():
    newvalue = random.randint(low_r,hi_r)
    cnvs.itemconfig(id_text,text = str(newvalue) + " %")
    # Rescale value to angle range (0%=120deg, 100%=30 deg)
    angle = 120 * (hi_r - newvalue)/(hi_r - low_r) + 30
    cnvs.itemconfig(id_needle,start = angle)
    root.after(3000, update_gauge)

    
# Create Canvas objects    

canvas_width = 400
canvas_height =300

root = Tk()

cnvs = Canvas(root, width=canvas_width, height=canvas_height)
cnvs.grid(row=2, column=1)

coord = 10, 50, 350, 350 #define the size of the gauge
low_r = 0 # chart low range
hi_r = 100 # chart hi range

# Create a background arc with a number of range lines
numpies = 8
for i in range(numpies):
    cnvs.create_arc(coord, start=(i*(120/numpies) +30), extent=(120/numpies), fill="white",  width=1)    

# add hi/low bands
cnvs.create_arc(coord, start=30, extent=120, outline="green", style= "arc", width=40)
cnvs.create_arc(coord, start=30, extent=20, outline="red", style= "arc", width=40)
cnvs.create_arc(coord, start=50, extent=20, outline="yellow", style= "arc", width=40)
# add needle/value pointer
id_needle = cnvs.create_arc(coord, start= 119, extent=1, width=7)

# Add some labels
cnvs.create_text(180,15,font="Times 20 italic bold", text="Humidity")
cnvs.create_text(25,140,font="Times 12 bold", text=low_r)
cnvs.create_text(330,140,font="Times 12 bold", text=hi_r)
id_text = cnvs.create_text(170,210,font="Times 15 bold")

root.after(3000, update_gauge)

root.mainloop()

Our Final Project

Our final project had 4 gauges that were based on basic gauge code. Our Python app ran full screen on a Kobo eReader that we installed Debian Linux on. The app connected to our Home Assistant Pi and showed us our current weather conditions.

We had to tweek the basic code a little bit to account for the 800×600 screen size and grey scale graphics.

Summary

In this blog we only looked at some basic gauges, the Tkinter Canvas component can be used in a very variety of different applications such as: bar charts, real time charts, graphics etc.

Home Assistant (REST) API

There are a few methods to communicate with Home Assistant. In this blog I wanted to document my notes on using the REST API.

Getting Started

I found that loading the File Editor Add-on made configuration changes quite easy. To load the File Editor, select the Supervisor item, then Add-on Store:

With the File Editor option you be able to modify your /config/configuration.yaml file. For this you’ll need to add an api: statement. I also added a command line sensor that shows the Raspberry Pi CPU idle time. I did this so that I could see a dynamic analog value:

After I made these changes I restarted my HA application, by the “Configuration” -> “Server Controls”.

Next I needed to create a user token. Select your user and then click on “Create Token” in the Long-Lived Access Tokens section. This token is very long and it only is shown once, so copy and paste it somewhere save.

Access the REST API with CURL

Curl is a command line utility that exists on Linux, Mac OS and Windows. I work in Linux mostly and it’s pre-installed with Ubuntu. If you’re working in Windows you’ll need to install CURL.

Getting started with curl isn’t required and you got straight to programming in your favourite language, however I found that it was usefully testing things out in curl before I did any programming.

The REST API is essentially an HTTP URL with some headers and parameters passed to it. For a full definition see the HA API document. The key items in REST API are:

  • Request type – GET or POST (note: there are other types)
  • Authorization – this is where the user token is passed
  • Data – is used for setting and defining tags
  • URL – the Home Assistant URL and the end point (option to view or set)

To check that the HA API is running a curl GET command can used with the endpoint of /api/.

$ curl -X GET -H "Authorization: Bearer eyJ0eXAiO....zLjc"   http://192.168.0.103:8123/api/

{"message": "API running."}

The user token is super long so your can use the \ character to break up your command. For example:

curl -X GET \
   -H "Authorization: Bearer eyJ0eXAiOiJKV......zLjc" \
   http://192.168.0.106:8123/api/states

Read a Specific HA Item

To get a specific HA item you’ll need to know its entity id. This can found by looking at the “Configuration” -> “Entities” page:

For my example I created a sensor called Idle Time, its entity id is: sensor.idle_time.

A curl GET command with the endpoint of /states/sensor.idle_time will return information on this sensor. The full curl command and the results would look like:

$ curl -X GET   -H "Authorization: Bearer eyJ0eXAiOiJKV1Q....zLjc"  \   http://192.168.0.103:8123/api/states/sensor.idle_time

{"entity_id": "sensor.idle_time", "state": "98.45", "attributes": {"unit_of_measurement": "%", "friendly_name": "Idle Time"}, "last_changed": "2020-12-12T17:34:10.472304+00:00", "last_updated": "2020-12-12T17:34:10.472304+00:00", "context": {"id": "351548f602f5a3887ff09f26903712bc", "parent_id": null, "user_id": null}}

Write to a New HA Item

A new or dynamic items can be created and written to remotely using a POST command with the definitions included in the data section. An example to create an entity called myput1 with a value of 88.6 would be:

curl -X POST \
   -H "Authorization: Bearer eyJ0eXAiOi....zLjc" \
   -H "Content-Type: application/json" \
   -d '{"state":"88.6", "attributes": {"unit_of_measurement": "%", "friendly_name": "Remote Input 1"}}' \
   http://192.168.0.103:8123/api/states/sensor.myinput1

This new entity is now available to HA and shown on the dashboard.

Write to a Switch

If you have a writeable device such as a switch you can use the REST to remotely control it.

For myself I have a Wemo switch with an entity name of : switch.switch1.

To control the switch the entity id is passed in the data section and the endpoint uses either a turn_on or turn_off parameter.

curl -X POST \
   -H "Authorization: Bearer eyJ0eXAiOiJ....zLjc" \
   -H "Content-Type: application/json" \
   -d '{"entity_id": "switch.switch1"}' \
   http://192.168.0.103:8123/api/services/switch/turn_on

Python and the HA API

Python can parse the JSON responses from the reading a sensor value:

from requests import get
import json

url = "http://192.168.0.103:8123/api/states/sensor.idle_time"
token = "eyJ0eXAiOiJK...zLjc"

headers = {
    "Authorization": "Bearer " + token,
    "content-type": "application/json",
}

response = get(url, headers=headers)

print("Rest API Response\n")
print(response.text)

# Create a json variable
jdata = json.loads(response.text)

print("\nJSON values\n")
for i in jdata:
    print(i, jdata[i])

The output will be something like:

Rest API Response

 {"entity_id": "sensor.idle_time", "state": "98.46", "attributes": {"unit_of_measurement": "%", "friendly_name": "Idle Time"}, "last_changed": "2020-12-12T19:29:10.655530+00:00", "last_updated": "2020-12-12T19:29:10.655530+00:00", "context": {"id": "2509c01cadb9e5b0681fa22d914e7b10", "parent_id": null, "user_id": null}}

 JSON values

 entity_id sensor.idle_time
 state 98.46
 attributes {'unit_of_measurement': '%', 'friendly_name': 'Idle Time'}
 last_changed 2020-12-12T19:29:10.655530+00:00
 last_updated 2020-12-12T19:29:10.655530+00:00
 context {'id': '2509c01cadb9e5b0681fa22d914e7b10', 'parent_id': None, 'user_id': None}

To write a value to myinput1 in Home Assistant:

from requests import post
import json

url = "http://192.168.0.103:8123/api/states/sensor.myinput1"
token = "eyJ0eXAiOiJKV1QiLCJhbGciOiJIUzI1NiJ9.eyJpc3MiOiJkMDI2YjAxY2VkZWU0M2E1OWY1NmI1OTM2OGU1NmI0OSIsImlhdCI6MTYwNzc5Mzc0NCwiZXhwIjoxOTIzMTUzNzQ0fQ.qEKVKdadxNWp249H3s_nmKyzQMIu5WDQkS9hiT-zLjc"

mydata = '{"state":"99.3", "attributes": {"unit_of_measurement": "%", "friendly_name": "Remote Input 1"}}'

headers = {
    "Authorization": "Bearer " + token,
    "content-type": "application/json",
}

response = post(url, headers=headers,data =mydata)

print("Rest API Response\n")
print(response.text)

# Create a json variable
jdata = json.loads(response.text)


print("\nJSON values\n")
for i in jdata:
    print(i, jdata[i])

The output will look something like:

Rest API Response

 {"entity_id": "sensor.myinput1", "state": "99.3", "attributes": {"unit_of_measurement": "%", "friendly_name": "Remote Input 1"}, "last_changed": "2020-12-12T20:40:05.797256+00:00", "last_updated": "2020-12-12T20:40:05.797256+00:00", "context": {"id": "31b422d02db41cde94470ebae7fac48c", "parent_id": null, "user_id": "1392a10c7bbb4cf0891a7f8a351740c7"}}

 JSON values

 entity_id sensor.myinput1
 state 99.3
 attributes {'unit_of_measurement': '%', 'friendly_name': 'Remote Input 1'}
 last_changed 2020-12-12T20:40:05.797256+00:00
 last_updated 2020-12-12T20:40:05.797256+00:00
 context {'id': '31b422d02db41cde94470ebae7fac48c', 'parent_id': None, 'user_id': '1392a10c7bbb4cf0891a7f8a351740c7'}

Final Comments

A REST API interface allows foreign devices such as PCs, Raspberry Pi and Arduino module to be used as remote I/O devices.

There are REST client HTTP libraries that are available for the Arduino, however it might be cleaner to implement an MQTT interface instead.

GnuPlot: Realtime Plots in 20 lines

There are some excellent charting and plotting packages but if you’re like me you sometimes want to do a quick test plot to capture some realtime data.

Gnuplot has been around for quite awhile and I was happily surprised at what it can do. I was amazed that I could make real-time bar and line charts in only 20 lines of scripting code. (For simple Bash applications it is possible to create charts in just 1 line.)

In this blog I’ll introduce Gnuplot and show two examples. The first will show the status Raspberry Pi I/O pins, and the second example will be a line chart of CPU diagnostics.

Getting Started

Gnuplot can be installed on Linux, Windows, and Mac. To install Gnuplot on Ubuntu:

sudo apt-get install gnuplot

Gnuplot is typically run as a command line utility, but it can also be run manually. When Gnuplot is run manually the charting instructions and data values can be inserted directly. Below is an example where 4 sets of data points are plotted in a line chart.

pi@raspberrypi: $ gnuplot

gnuplot> $Mydata << EOD
# Now enter some data                                
2 1
3 1.5
4 2.1
5 3.3
EOD
gnuplot> plot $Mydata with line

Data block names must begin with a $ character, which distinguishes them from other types of persistent variables. The end-of-data delimiter (EOD in the example) may be any sequence of characters. For this example the plot command will use the data in the $Mydata variable and create a line chart.

A Static Bar Chart

A simple but useful example of Gnuplot would be to show the realtime status of the Raspberry Pi General Purpose Input/Output (GPIO) pins as a bar chart.

A bar chart presentation can be created using a data file of:

# gpio.dat - data file for GPIO pin values
# column1 = chart position, column2 = heading, column3 = value
0 GPIO2 0
1 GPI03 1
2 GPI04 1
#....

To plot a bar chart the fill style and bar width needs to be defined. The plot using 1:3:xtic(2) argument will make the first column in the data file the x-position, the third column the y-value and the x-labels be the second column. The interactive commands to plot the file:

pi@raspberrypi:~/pete/gnuplot $ gnuplot 

gnuplot> set style fill solid
gnuplot> set boxwidth 0.5
gnuplot> plot "gpio.dat" using 1:3:xtic(2) with boxes title ""

Real-Time Bar Chart of PI GPIO

The previous example used a manually created gpio.dat data file. The status of GPIO pins can be found using the gpio command line utility. To get the status of GPIO pin 9:

gpio read 9

By adding some Bash and awk script it is possible to create a gpio.dat file:

$ gpio read 9
1
$ gpio read 9 | awk '{ print "9 GPIO9 " $1 }'
9 GPIO9 1
$ gpio read 9 | awk '{ print "9 GPIO9 " $1 }' > gpio.dat
$ cat gpio.dat
9 GPIO9 1

To make a dynamic bar chart I created a Gnuplot script file (gpio_bars.txt). The Gnuplot scripting language is quite powerful, and it support a wide range of functions and control statements.

Rather than manually adding lines for each GPIO pin status a for loop can iterate from pins 2-29. A system command is used to run the gpio utility and bash commands. To refresh the data a replot and a pause command is used.

# Create Dynamic bar that read GPIO pins every 5 seconds
#
set title "PI GPIO Data"
set boxwidth 0.5
set style fill solid

# Create a dummy file to get started
system "echo '0 GPIO2 1' > gpio.dat"

plot "gpio.dat" using 1:3:xtic(2) with boxes title ""

while (1) {  # make a new 'gpio.dat' every cycle with fresh data
  system "echo '' > gpio.dat"
  do for [i=2:29] {
    j = i-2 # put first GPIO pin at position 0
    system "gpio read " .i.  "  | awk '{ print  \"" . j . " GPIO" . i . " \" $1 }' >> gpio.dat
  }
  replot
  pause 5
}

To run this script enter:

gnuplot -persist gpio_bars.txt

A Simple Line Chart

A bar chart presentation can be created using a data file of:

# GPU.dat - a time stamp with two data points
18:48:30 51.0 49.0
18:48:40 50.5 49.5
18:48:45 51.5 49.0
18:48:50 50.0 50.5
18:48:55 50.5 49.5

The interactive Gnuplot commands to show a line chart of this data would be:

pi@raspberrypi: $ gnuplot

gnuplot> set xdata time
gnuplot> set timefmt "%H:%M:%S"
gnuplot> set format x "%H:%M:%S"
gnuplot> plot "gpu.dat" using 1:2 with line title "GPU temp" ,\
>"gpu.dat" using 1:3:3 with line title "CPU temp"  

A few extra lines are needed in the Gnuplot script. First the plot needs to know that the x-axis is time data, and it needs to know the format of the time data. Multiple data points can be plotted at the same time, and the using argument tells Gnuplot the x:y-point data. (If the data file had a third point the using reference would be 1:4:4 ).

A Real-Time Line Chart

Linux has a lot of useful command line trouble shooting tools, one of these is the sensors utility that allows users to get fan speed and temperatures.

$ sensors
dell_smm-virtual-0
Adapter: Virtual device
Processor Fan: 2676 RPM
CPU: +47.0°C
Ambient: +38.0°C
SODIMM: +37.0°C
...

Using some Bash and Awk commands it is possible to get the fan speed and CPU temperature:

$ sensors | grep RPM
Processor Fan: 2685 RPM
$ sensors | grep RPM | awk '{print $3}'
2685

$ sensors | grep CPU
CPU: +50.0°C
$ sensors | grep CPU | awk '{print $2}'
+48.0°C
$ sensors | grep CPU | awk '{print substr($2,2,4)}'
48.0

Awk supports a systime() call to return the present date/time, and a strftime() call to customize the presentation of the time/date. (Note: a Raspberry Pi might need to have gawk installed to get this functionality, use “sudo apt-get install gawk”).

The next step is to format the sensor output with a timestamp:

$ sensors |grep RPM | awk '{print strftime("%H:%M:%S ", systime()) $3}'

10:26:46 2687

$sensors | grep CPU | awk '{print strftime(\"%H:%M:%S \",systime()) substr($2,2,4)}'

10:27:46 49.0

Now that a time and value string can be generated a Gnuplot script can be created (line_fan_cpu.txt) to show real time data.

To make the Bash code a little easier two data files are created, fan.dat and cpu.dat.

The plot has to account for different scale ranges, so y2range and y2label definitions are used. The final script addition is to include an axes (x1y2 or x1y2) to each plot point, this lines up the data value to the right or left y-axis.

The complete Gnuplot script to show fan speed and CPU temperature is only 20 lines of code!

# Create a Plot or User and System CPU Usage, update every 5 seconds
#
set title "GnuPlot - Fan Speed  and CPU Temperature"
set yrange [2650:2700] 
set ylabel "Fan Speed"
set y2range [43:49] 
set y2label "CPU Temp (C)"
set y2tics
set xdata time
set timefmt "%H:%M:%S"
set format x "%H:%M:%S"

system "sensors |grep RPM | awk '{print strftime(\"%H:%M:%S \", systime()) $3}' > fan.dat"
system "sensors | grep CPU | awk '{print strftime(\"%H:%M:%S \",systime()) substr($2,2,4)}'  > cpu.dat"

plot "fan.dat" using 1:2  with lines axes x1y1 title "fan speed (RPM)",  "cpu.dat" using 1:2 with lines axes x1y2 title "CPU Temp (C)"
while (1) {
	pause 5
	system "sensors |grep RPM | awk '{print strftime(\"%H:%M:%S \", systime()) $3}' >> fan.dat"
	system "sensors | grep CPU | awk '{print strftime(\"%H:%M:%S \", systime()) substr($2,2,4)}'  >> cpu.dat"
	replot
}

To run this script enter: $ gnuplot -persist line_fan_cpu.txt

Final Comments

I won’t give up using plotting packages like MatPlotlib or ggplot, but I was very impressed how easy it was to create real-time plots using Gnuplot.

Manipulating the Bash/awk script can be a little complex but it’s incredible useful to be able to use output from almost any command line utility in Gnuplot.

Gnuplot can plot a very large number of data points, but it makes sense to a tail command to only show the latest x-number of points.

Animated Node-Red Graphics with MQTT and SVG

There is a new SVG (Scalar Vector Graphics) node that is available for Node-Red  Dashboards. This node allows for animated Node-Red graphics that can be viewed on a smart phone.

In this blog I wanted to document an example of integrating MQTT messaging to SVG animated graphics.

Getting Started

If you’re unfamiliar with SVG graphics that are some good tutorials. For my own reference I wrote some notes on SVG and Javascript integration.

The Web Dashboards should be at the latest version,  you can do this in the “Manage Palette” option. The node-red-contrib-ui-svg node can be install from the “Manage Palette” option, or manually by:

cd ~/.node-red)
npm install node-red-contrib-ui-svg

There is some excellent documentation on this node.

The SVG Graphic node has a built-in SVG graphic editor or SVG code can be pasted directly into the “SVG Source” tab.

svg_source_edit

For my project I used industrial SVG examples from: https://www.opto22.com/support/resources-tools/demos/svg-image-library

The SVG editor is useful for identified and defining SVG items that are you’d like to animate.

svg_editor_ids

For this project the solar panels (id=panels) and the output (id=watts) were to be dynamically updated from MQTT.

SVG items can  be dynamically update by: 1)  the “Input Bind” tab in the node’s definition or, 2) as an input message.

An example using the input message approach would be:

In this example the colour of the panels is set to green using an injector and function node.

 

MQTT with SVG Graphics

There are some good MQTT brokers, such as Mosquitto that can be used. Node-Red also has a MQTT broker node (MOSCA) that is easy to install.

The earlier test logic can be adapted to connect MQTT inputs:

svg_mqtt_example

For this example two MQTT tags were used: 1) watts, and 2) panel_status.

MQTT testing can be done with the MQTT client tools, that are installed by:

 sudo apt-get install mosquitto mosquitto-clients -y

From a terminal MQTT tags can be published to a broker  (-h servername) with a topic (-t thetopic) and a message (-m themessage) :

mosquitto_pub -h 192.168.0.111 -t watts -m "123"
mosquitto_pub -h 192.168.0.111 -t panel_status -m "gold"

In Node-Red the watts text  is updated by the function node code of:


// Pass the MQTT payload
// and update the text, hardcode the units
//

msg.payload = {
"command": "update_text",
"selector": "#watts",
"textContent": (msg.payload + " C")
}

return msg;

The panels color is changed by the function node code of:

//
// Pass the MQTT payload
// as a fill color attribute
//

msg.payload = {
"command": "update_style",
"selector": "#panels",
"attributeName": "fill",
"attributeValue": msg.payload
}

return msg;

Final Comments

Drawing SVG graphics from scratch is awkward, but there are some great Internet examples of pre-build SVG graphics. All it takes is a bit of time to find the graphic items that you need to animate.

In this blog I looked at making SVG files dynamic. It is also possible to put “hot links” on the SVG file to call URLs or to send messages back to Node-Red.

 

 

Home Assistant with Node-Red

Home Assistant is an open source home automation platform that can monitor and control smart home devices and it integrates with many of other common systems.

HA_demo

Home Assistant installation is targeted for Raspberry Pi’s but other hardware options are available.

I was very impressed how easy it was to install Home Assistant and get a basic home integration system up and running.

There is a huge number of integration solutions (1500+) that connect to most of the mainstream products. However if you want to do some custom Arduino or Raspberry Pi connections there isn’t an easy “out of the box” solution.  To solve this requirement Home Assistant has included Node-Red as an add-on.

Node-RED is a visual programming tool for wiring together hardware devices, APIs and online services.

I found that getting the Node-Red integration was a little tricky. This blog will show how to get Node-Red integration working and it includes a simple simulator circuit.

Getting Started

The installation instructions are very straightforward. I would recommend using a wired connection for your Raspberry Pi. A wireless network connection is 100% possible but it is not in the base installation directions.

After the basic installation is complete, add-ons can be installed under the Supervisor->Dashboard. I would recommend installing “File editor” and “Terminal & SSH” add-ons along with Node-Red.

ha_addons

I found that the Node-Red installed without any problems but it required some configuration changes before it would run.

In the Node Red add-on you will need to add a credential_secret and a password.

nodered_config

If Node-Red doesn’t start look at the log for errors (it’s at the bottom of the same page).

nodered_log

The base Node-Red installation has a very good selection of pre-installed nodes. If you wish to add more nodes see the “Manage Pallet” option that is accessed from the top right options icon.

At this stage Node-Red is somewhat standalone and it is not fully integrated with Home Assistant.

Integrating Node-Red with Home Assistant

The directions and files for Node-Red integration  can be downloaded to your PC.

ha_nr_int_files

Specifically you want to custom_components/nodered directory and files, which will need to be moved to the Raspberry Pi. The Home Assistant “File editor” add-on can be used to create Pi directories and move files from your PC.

nodered_files

The following directories and file should now exist:

/root/config/custom_components/nodered/__init__.py
/root/config/custom_components/nodered/__pycache__
/root/config/custom_components/nodered/binary_sensor.py
/root/config/custom_components/nodered/config_flow.py
/root/config/custom_components/nodered/const.py
/root/config/custom_components/nodered/discovery.py
/root/config/custom_components/nodered/manifest.json
/root/config/custom_components/nodered/save.txt
/root/config/custom_components/nodered/sensor.py
/root/config/custom_components/nodered/services.yaml
/root/config/custom_components/nodered/switch.py
/root/config/custom_components/nodered/websocket.py

/root/config/custom_components/nodered/.translations/en.json

Once this is complete Home Assistant will need to be restarted.

Including Node-Red Integrations

The next step is to create sensors and switches in Node-Red that can be accessed in Home Assistant. Below is a simple circuit that sends a random number (0-100) to a HA entity.

This logic uses a Big Timer node, that generates a pulse every minute from the middle output pin. An injector node allow you to force a new value. A random node will output a new random number whenever the Big Timer or Inject nodes are triggered.

nr_circuit

Double-click on the HA entity to configure the HA server and other properties.

nr_entity_config

Once the logic is complete click the “Deploy” button to make the logic active.

Node-Red integration is enabled by adding it in the Configuration->Integration page.

nr_new_int

nr_entities

Overview Dashboard with Node-Red Data

The final step is to modify the Overview Dashboard to include the Node-Red Entity.

For this example I added a gauge component using the Orange-Plus at the bottom right of this Configure UI page.

config_ui

On the live Overview page it is possible to click on the gauge card and get more information about this sensor.

HA_overview

Final Thoughts

Home Assistant is a very well structured home automation solution that offers a number of excellent approaches to bring in data.

Node-Red is a very flexible programming environment that help expands connectivity to Arduino, Raspberry Pi and other 3rd party services that are not in the base Home Assistant software.

For information on how to connect an Arduino module to Node-Red see:

Arduino talking MQTT to Node-Red

Arduino talking TCP to Node-Red and Python

 

SVG Web Animation

There are a number of good Web animation techniques that are available. Flash animation support is being discontinued in 2020, but luckily there are some good alternatives, such as HTML 5 Canvas, Scalar Vector Graphics (SVG),  and even CSS (Cascading Style Sheets) can be used .

HTML 5 Canvas draws 2D graphics, on the fly with a JavaScript. The canvas is rendered pixel by pixel. In canvas, once the graphic is drawn, it is forgotten by the browser. If its position should be changed, the entire scene needs to be redrawn, including any objects that might have been covered by the graphic.

SVG is a language for describing 2D graphics in XML, which means that every element is available within the SVG DOM. In SVG, each drawn shape is remembered as an object. If attributes of an SVG object are changed, the browser can automatically re-render the shape.

In this blog I wanted to document my SVG testing.

Using SVG

SVG can be used inline within an HTML document. Below is an example with a rectangle, circle and some text.

<!DOCTYPE html>
<html>
<body>

<svg width="400" height="400">
  <rect width="100" height="100" fill= "blue" />
  <circle cx="150" cy="50" r="40" fill="green" />
  <text x="10" y="120" fill="red">SOME SVG TEXT</text>
</svg>

</body>
</html>

This will generate a web like:

svg1

An SVG element has a width and height property and all the drawing elements are positioned within this area.

Dynamic SVG Bar

A bar can be created using a rectangle () element. Javascript is used to dynamically adjust the top and height of the rectangle.

<!DOCTYPE html>
<html>
<head>
<script type="text/JavaScript">

setTimeout(updatebar, 2000);

function updatebar() {
// 
  var newvalue = (Math.random() * 300); // get a random integer 0-100
  if (newvalue > 250) {
    document.getElementById("bar1").style.fill = "red";
  } else {
    document.getElementById("bar1").style.fill = "green";
  }
  newvalue = Math.round(newvalue);
  var newheight = newvalue.toString();
  var num = 300 - newvalue - 5;
  var newy = num.toString();  

  document.getElementById("bar1").setAttribute("height",newheight);
  document.getElementById("bar1").setAttribute("y",newy);
  document.getElementById("bartext").innerHTML = "value: " + newheight ;
  setTimeout(updatebar, 2000);
}
</script>
</head>
<body>
<h2> SVG Bar Example</h2>
<svg width="400" height="400" >
  <rect x="0" y="0" width="160" height="300" style="fill:white;stroke:black;stroke-width:4;" />
  <rect id="bar1" x="5" y="145" width="150" height="150" style="fill:green;stroke:black;stroke-width:2;" /> 
  <text id="bartext" x=30 y = 320 style="font-family:verdana;font-weight:bold">value: 50 </text>
</svg>

</body>
</html>

svg_bar

Animate Graphics

There some SVG drawing packages such as: https://www.drawsvg.org/drawsvg.html

There are also some free libraries, I used: https://www.opto22.com/support/resources-tools/demos/svg-image-library to animate a flow valve.

I needed to do a little bit of trial and error but I was able to add a text element to the bottom of the SVG that could be animated with Javascript. Below is the SVG file:

<svg x="0px" y="0px" viewBox="0 0 100 100" xml:space="preserve"><svg><g>
<g id="shape">
  <circle fill="#999999" cx="50.093" cy="32.75" r="32.75"></circle>
  <rect x="0.176" y="71.667" fill="#999999" width="99.833" height="26.833"></rect>
  <rect x="44.926" y="63" fill="#999999" width="10.333" height="10.5"></rect>
  <rect y="70" fill="#999999" width="4.5" height="30"></rect> 
  <rect x="95.509" y="70" fill="#999999" width="4.5" height="30"></rect>
</g>
<g id="linear">
  <linearGradient class="linear" id="SVGID_1_" gradientUnits="userSpaceOnUse" x1="50.0926" y1="94.5113" x2="50.0926" y2="75.6554">
    <stop offset="0" style="stop-color:#E6E6E6"></stop>
    <stop offset="1" style="stop-color:#000000"></stop>
  </linearGradient>
  <polygon class="linear" fill="url(#SVGID_1_)" points="70.031,75.655 60.041,79.079 40.144,79.079 30.154,75.655 0.176,75.655 0.176,94.511 
    29.289,94.511 40.144,90.791 60.041,90.791 70.896,94.511 100.009,94.511 100.009,75.655   ">
</g>
<g id="light">
  <circle fill="#E6E6E6" cx="50.093" cy="32.75" r="29.074"></circle>
  <path fill="#E6E6E6" d="M44.833,71.5v-6.409c0.606,0.096,1.217,0.179,1.833,0.241V71.5H44.833z"></path>
</g>
<g id="hlight">
  <path fill="#FFFFFF" d="M18.343,33.75C18.343,15.663,33.005,1,51.093,1c8.789,0,16.763,3.469,22.646,9.104
    C67.777,3.881,59.391,0,50.093,0c-18.087,0-32.75,14.663-32.75,32.75c0,9.298,3.881,17.685,10.104,23.646
    C21.812,50.514,18.343,42.539,18.343,33.75z">
  <rect x="26.093" y="21.474" fill="#FFFFFF" width="48" height="23.053"></rect>
</g>
<g id="shadow">
  <path fill="#000000" d="M72.738,9.104c5.635,5.882,9.104,13.857,9.104,22.646c0,18.087-14.663,32.75-32.75,32.75
    c-8.789,0-16.763-3.469-22.646-9.104C32.408,61.619,40.795,65.5,50.093,65.5c18.087,0,32.75-14.663,32.75-32.75
    C82.843,23.452,78.961,15.066,72.738,9.104z">
  <path d="M55.259,71.5v-6.409c-0.606,0.096-1.217,0.179-1.833,0.241V71.5H55.259z"></path>
  <polygon points="27.235,22.589 74.093,22.589 74.093,21.474 26.093,21.474 26.093,44.526 27.235,44.526  "></polygon>
  <path d="M22.018,33.75c0-16.057,13.017-29.074,29.074-29.074c7.774,0,14.83,3.057,20.047,8.028
    c-5.296-5.558-12.764-9.028-21.047-9.028c-16.057,0-29.074,13.017-29.074,29.074c0,8.283,3.469,15.751,9.028,21.047
    C25.076,48.58,22.018,41.524,22.018,33.75z">
</g>
</g>
<text id="flowval" name="flowval" x = "40" y = "40" fill= "green" style="font-family:verdana;font-weight:bold"> 00 </text></svg>
</svg>

External SVG files can be used in a Web page by:

  •  <img – as an image file. (Only good for static or self-contained SVG files)
  • <frame – as a frame.
  • <object – as a object reference within an existing web document
  • <embed – as an embedded element

Unfortunately I found that Chrome only supports inline SVG with Javascript. Microsoft Edge and Firefox worked with both the object and embed tags.

Below is the code that I used to animate the text on the flow valve:

<!DOCTYPE html>
<html>
<head>
<script type="text/JavaScript">

setTimeout(updatebar, 2000);

function updatebar() {
// 
  var randvalue = (Math.random() * 100); // get a random integer 0-100
  var flowvol = Math.round(randvalue.toString());
  // set the text in the svg object   
  var svgobj = document.getElementById("flowmeter");
  var svgdoc =  svgobj.getSVGDocument();
  svgdoc.getElementById("flowval").textContent =  flowvol ;
  
  setTimeout(updatebar, 2000);
}
</script>
</head>
<body>
<h2> SVG Flow Meter</h2>
<embed id="flowmeter" height= '50%' width='100%' src="flowmeter.svg" >

</body>
</html>

flowmeter_1

Final Comments

I found that it can be a little challenging to tweek existing SVG library examples. The browser “development tools” can be used to find and test SVG elements. Below is an example where I found the solar panel element that I need to animate.

svg_devtools

For 100% cross-browser capability it would be recommended to put all the SVG code in one document.

The next step will be to use a Raspberry Pi Web app and some AJAX calls to animate the data.

Pi Charts with PySimpleGUI

There are a lot of good charting packages available for IoT and Rasperry Pi projects.

The PySimpleGUI python libary stands out in its ability to have the same code for both a local GUI and a Web interface. PySimpleGUI isn’t focused as a charting package but it has the canvas and graph elements that allow you to create real time bar charts and real time trend charts.

Getting Started with Graph Elements

For some background on PySimpleGUI see: PySimpleGUI – quick and easy interfaces

The graph element can have different co-ordinate orientations, for example the center can be (0,0) or the bottom left can be (0,0). A graph element is created with the syntax:

Graph(canvas_size, graph_bottom_left, graph_top_right …)

An example with 2 different co-ordinate orientations would be:


# A basic PySimpleGUI graph example
import PySimpleGUI as sg

bcols = ['blue','red','green']
myfont = "Ariel 18"

gtop = sg.Graph((200,200), (0,0),(200,200),background_color="white")
gcenter = sg.Graph((200,200), (-100,-100), (100,100),background_color="white")

layout = [[sg.Text('Graph: 0,0 at bottom left',font=myfont)],
[gtop],
[sg.Text('Graph: 0,0 at center',font=myfont)],
[gcenter],
[sg.Exit()]]

window = sg.Window('Graph Example', layout)

# Write text and lines
event, values = window.read(timeout=0)

gtop.draw_text(text="(0,0)", location=(0,0))
gcenter.draw_text(text="(0,0)", location=(0,0))

gtop.draw_text(text="(50,50)", location=(50,50))
gcenter.draw_text(text="(50,50)", location=(50,50))

gtop.draw_line((0,0),(50,50))
gcenter.draw_line((0,0),(50,50))

# Wait for a key to exit
window.read()

window.close()

sg_chart04

Bar Charts

Bar charts can be created using the graph.draw_rectangle() method. Below is an example that takes a command line input to toggle between a tkinter local interface and a web interface. This example has 3 input points that are scanned every 2 seconds.


import sys
import random

# Pass any command line argument for Web use
if len(sys.argv) &gt; 1: # if there is use the Web Interface
    import PySimpleGUIWeb as sg
    mode = "web"
    mysize = (20,2)
else: # default uses the tkinter GUI
    import PySimpleGUI as sg
    mode = "tkinter"
    mysize = (12,1)
BAR_WIDTH = 150
BAR_SPACING = 175
EDGE_OFFSET = 3
GRAPH_SIZE = (500,500)
DATA_SIZE = (500,500)

bcols = ['blue','red','green']
myfont = "Ariel 18"

#update with your ip
myip = '192.168.0.107'

graph = sg.Graph(GRAPH_SIZE, (0,0), DATA_SIZE)

layout = [[sg.Text('Pi Sensor Values',font=myfont)],
  [graph],
  [sg.Text('PI Tag 1',text_color=bcols[0],font=myfont,size= mysize ),
  sg.Text('PI Tag 2',text_color=bcols[1],font=myfont,size= mysize ),
  sg.Text('PI Tag 3',text_color=bcols[2],font=myfont,size= mysize)],
  [sg.Exit()]]

if mode == "web":
    window = sg.Window('Real Time Charts', layout,web_ip=myip, web_port = 8888, web_start_browser=False)
else:
    window = sg.Window('Real Time Charts', layout)
while True:
    event, values = window.read(timeout=2000)
    if event in (None, 'Exit'):
        break

    graph.erase()
    for i in range(3):
# Random value are used. Add interface to Pi sensors here:
        graph_value = random.randint(0, 400)
        graph.draw_rectangle(top_left=(i * BAR_SPACING + EDGE_OFFSET, graph_value),
        bottom_right=(i * BAR_SPACING + EDGE_OFFSET + BAR_WIDTH, 0), fill_color=bcols[i])
        <span id="mce_SELREST_start" style="overflow:hidden;line-height:0;"></span>graph.draw_text(text=str(graph_value), location=(i*BAR_SPACING+EDGE_OFFSET+15, graph_value+10),color=bcols[i],font=myfont)

window.close()

The presentation between the tkinter and Web interface is almost identical, but not 100% some tweeking on text sizing is required.

Real Time Trend Charts

It is important to note that the PySimpleGUIWeb is still in development so there may be some compatibility issues when trying to toggle between the tkinter and Web versions.

Below is an example that will create a realtime chart.

sg_realtimechart

#!/usr/bin/env python

import array
from datetime import datetime
import sys
import random

# Pass any command line argument for Web use
if len(sys.argv) > 1: # if there is use the Web Interface
    import PySimpleGUIWeb as sg
    mode = "web"
    mysize = (20,2)
else: # default uses the tkinter GUI
    import PySimpleGUI as sg
    mode = "tkinter"
    mysize = (12,1)

from threading import Thread

STEP_SIZE = 1  # can adjust for more data saved than shown
SAMPLES = 100 # number of point shown on the chart
SAMPLE_MAX = 100 # high limit of data points
CANVAS_SIZE = (1000, 600)
LABEL_SIZE = (1000,20)

# create an array of time and data value
pt_values = []
pt_times = []
for i in range(SAMPLES+1): 
    pt_values.append("")
    pt_times.append("")
    

def main():

    timebar = sg.Graph(LABEL_SIZE, (0, 0),(SAMPLES, 20), background_color='white', key='times')
    graph = sg.Graph(CANVAS_SIZE, (0, 0), (SAMPLES, SAMPLE_MAX),
          background_color='black', key='graph')

    layout = [
        [sg.Quit(button_color=('white', 'red')),sg.Button(button_text="Print Log", button_color=('white', 'green'),key="log"),
         sg.Text('     ',  key='output')],
        [graph],
        [timebar],       
    ]
    if mode == 'web':
        window = sg.Window('Pi Trend Chart', layout,
                       web_ip='192.168.0.107', web_port = 8888, web_start_browser=False)
    else:
        window = sg.Window('Pi Trend Chart', layout)

    graph = window['graph']
    output = window['output']

    i = 0
    prev_x, prev_y = 0, 0

    while True: # the Event Loop
  
        event, values = window.read(timeout=1000)
        if event in ('Quit', None):  # always give ths user a way out
            break
        if event in ('log'): # print the recorded time/data arrays
            print("\nReal Time Data\n")
            for j in range(SAMPLES+1):
                if pt_times[j] == "": #only print updated info
                    break
                print (pt_times[j], pt_values[j])

        # Get data point and time
        data_pt = random.randint(0, 100)
        now = datetime.now()
        now_time = now.strftime("%H:%M:%S")
        # update the point arrays
        pt_values[i] = data_pt       
        pt_times[i] = str(now_time) 

        window['output'].update(data_pt)
        
        if data_pt > SAMPLE_MAX:
            data_pt = SAMPLE_MAX
        new_x, new_y = i, data_pt

        if i >= SAMPLES:
            # shift graph over if full of data
            graph.move(-STEP_SIZE, 0)
            prev_x = prev_x - STEP_SIZE
            # shift the array data points
            for i in range(SAMPLES):
                pt_values[i] = pt_values[i+1]
                pt_times[i] = pt_times[i+1]
                    
        graph.draw_line((prev_x, prev_y), (new_x, new_y), color='red')
        
        prev_x, prev_y = new_x, new_y
        i += STEP_SIZE if i < SAMPLES else 0
    
        
        timebar.erase()
        # add a scrolling time value 
        time_x = i
        timebar.draw_text(text=str(now_time), location=((time_x - 2),7) )
        # add some extra times
        if i >= SAMPLES:
            timebar.draw_text(text=pt_times[int(SAMPLES * 0.25)], location=((int(time_x * 0.25) - 2),7) )
            timebar.draw_text(text=pt_times[int(SAMPLES * 0.5)], location=((int(time_x * 0.5) - 2),7) )
            timebar.draw_text(text=pt_times[int(SAMPLES * 0.75)], location=((int(time_x *0.75) - 2),7) )
        if i > 10:
            timebar.draw_text(text=pt_times[1], location=( 2,7) )
        
    window.close()


if __name__ == '__main__':
    main()

Final Comment

If you are looking at doing some charting and you want to have both a local and a web interface then PySimpleGUI and PySimpleGUIWeb will be something that you should take a look at.