Author Archives: Sean

Reading a smart meter with a webcam

Yeah I know – the title, right? “What the faq is he doing with a webcam if he has a smart meter?” A contemporary UK “smart meter” is very smart: it holds your data hostage. What I had installed recently is an electronic meter with communications adapter and a little head unit to show me near-instantaneous consumption data. As soon as the meter was fitted, I went looking for the actual data that must exist at least temporarily to allow the head unit to work. The executive summary: the data is not accessible to the homeowner.

Buy a Software Defined Radio for fun, but not this

I didn’t go looking for the data straight away, I had a quick look at the head unit and the meter to see if there was anyway of connecting to them or exporting the data. There are none. “The meter must be communicating with the head unit somehow, maybe I can sniff / connect the same way?” I thought, and read several articles suggesting ‘Software Defined Radio’ would give me access to not only my meter’s data, but those of similar meters in my neighbourhood. This is either not a thing in the UK or was only briefly a thing with early models of even-less-smart meter, broadcasting your data in an easily accessible way being incompatible with privacy.

The smart meter

The standards to which UK smart meters must adhere pretty much preclude any easy tapping or sniffing of data. In fact, it appears the only way to gain access to your own smart meter data is to arrange with your supplier to send the data to them or to a 3rd party selected by your supplier, so they may offer you your own data ‘as a service’. This is a miserable shortcoming of the smart meter standard, which should be rectified as soon as possible. It must be possible for a reasonably competent homeowner to gain direct and private access to their own consumption data.

That little head unit is okay for instantaneous power and a running total, but when it gets to £20, where did the money go? There are no clues. This falls far short of any definition of ‘smart’. I believe it should be illegal to call a device ‘smart’ unless its smartness is self-contained. If you have a device that can deliver its “smart” behaviour while disconnected from the Internet, that’s a smart device. If it needs to communicate with a service hosted elsewhere, the device itself shouldn’t be marketed as anything other than a proxy, agent or facade.

The government’s publications for smart meters seem at first glance to have the intent of giving the homeowner control over their data. Where they fall short is in describing only how the homeowner grants access to their data to suppliers and third parties: there’s nothing to describe making the data accessible to the homeowner. In the government’s standards for smart meters, the best you’ll find is a circular buffer “capable of storing four hours of UTC date and time stamped six minute Consumption data”. When you can see your head unit updating the ‘live’ consumption every few seconds and billing periods are rarely less than 1 month, 40 records is effin’ risible! Thanks gov!

Where the webcam comes in

Light on the meter with impulses per kWh legend

Previously, I’d read a meter with a webcam by looking at an old-school meter’s analogue dials and ‘recognising’ the digits. This time there was something a bit more ‘digital’ accessible to the webcam: a blinking red light. Next to the light is a legend that says “1,000 impulses per kWh”. To my pedantic mind, that looks like the light flashes every time one Wh (Watt-hour) is consumed, or 3.6kWs (kilowatt-seconds).

The average power consumption over the interval from one flash to the next is 3.6kWs divided by the duration of the interval. For example, if the light flashes once per second, the average power consumption is 3.6kW (3.6kWs / 1s). Another example – if the light flashes 1,000 times over an hour (the legend’s “1kWh” over an hour) the interval is 3.6s and the average power is 1kW. If you fire up an 11kW ‘power shower’ you’d expect to see about 3 blinks per second, and with everything off at night time, you’d expect some quite long delays between blinks.

The scheme

The basic scheme is to take a picture of the meter with the webcam, crop out everything but the square holding the light, get an average colour for that square and (waves hands) detect when the light switches from off to on. The interval between off-on transitions can then be used to compute average power over the interval. The equipment is same as last time: my venerable Dell Mini 9 running Ubuntu 18.04 and a cheap-and-cheerful Logitech C250 webcam.

For a first shot I used uvccapture and ImageMagick from a shell script to generate a stream of RGB values for the average pixel color, then wrote a little utility that converted these to a datetime and average power pair. This worked, but not well: the capture rate was 3-5 frames per second. Even this was useful though – it was obvious from the first few hours’ data that “Child number one” was taking a half hour power shower compared to just a few minutes for everyone else. I’ve lost that data, so there’s no real “steaming gun” in the data that follows.

Here’s a reasonably representative 24 hours collected with a little Java program (attached below) driving the webcam at 50 frames per second and logging timestamps and power:

You can see from most of the night that there’s a baseline around 50W – I suspect that’s mostly low-power devices we don’t turn off overnight – our Internet router and other devices with electronic displays. The nearly-1kW ‘pulses’ between midnight and 7am are from an electric underfloor heating circuit. It’s interesting to see it spend about twice as much time off as on, so its average power is somewhere around 300W. The UFH is turned off in the morning. Some spikes visible through the day are kitchen devices being used at the same time. The kettle is around 2kW, there’s a 950W microwave, a toaster around 1kW – none of which run for very long.

Parent number one was off-shift that day, so the slightly increased baseline through the afternoon could have involved Netflix. Children numbers one and two arrive home from school either side of sunset in the late afternoon, so there’s a bit more power being consumed by lights, anime and Minecraft. Cooking / tidying up around dinner results in some 2.5kW pulses from an ‘instant’ hot water tap on the kitchen sink which has a small tank of its own. The big spikes late in the evening are from a ‘power shower’ which seems to be about 5kW. They’re mixed in with some added electric heating in the bathroom.

There’s also a regular ‘pock’ of about 300-400W which becomes more obvious around the evening ablutions (and on damp days!). I believe this is the dehumidifier (with humidistat) in our bathroom which is doing a pretty good job (there’s no convenient place to put an extractor and the bathroom has multiple damp-inducing construction issues which we plan to fix … one day) but perhaps costing more to run than we expected.

This quality of data tells me almost exactly what’s responsible for my power bill. It’s not perfect – I suspect the webcam misses the occasional blink from the meter and there may also be spurious false-positive readings recorded which manifest themselves as horrifying 20-30kW spikes! Effectively polling the meter blink also quantises the data in a way that produces equally-ish spaced horizontal stripes of points around a reading. I’ve got a tiny microcontroller and photocell on order which I’m hoping will allow me to stick (temporarily) something on the face of the meter which will much more precisely capture (by interrupt rather than poll) the timing of the blink’s off-to-on transition.

What I’ve collected with the webcam is around about the quality and quantity of data I expected from my smart meter, so I’m bitterly disappointed by what was installed. I have to say that what’s installed is exactly what the standard says should be installed, so my disappointment is reserved wholly for the government. We shouldn’t have to try this hard to get data about us from devices which are recording us in our homes.

My electricity supplier in the UK is Bulb, whose online presence and customer features seem a little bit more savvy and easier to user than those of some better-known competitors. I picked them on the basis of price from uswitch.com. They have a referral scheme, so if you like this article please use my Bulb referrer link – thanks!

Adafruit Bluefruit LE Micro into Wowwee Robosapien

I bought an Arduino Micro for a project a little while ago, found it easy to develop with and wondered where else I could use one. There are two Robosapiens in our house, so one of them went under the knife for a brain transplant. I chose an Adafruit Bluefruit LE Micro this time, so I could disconnect the USB cable and control the Robosapien over Bluetooth.

The Robosapien’s getting on in years and resources explaining what’s inside it are disappearing from the WWW. Markcra’s Robosapien pages are an invaluable starting point, though I never did find a complete list of pinouts for the U2 controller IC. Here’s the list I used while adding the wire extensions for the prototyping board:

U2 pin Connected to
P10 IR
P11 Finger/toe/heel switches left
P12 Grip switch left
P13 Shoulder switch left
P14 Finger/toe/heel switches right
P15 Grip switch right
P16 Shoulder switch right
P17 Microphone
P20 LED
P21 LED
P22 LED
P23 LED
P24 LED
P25 LED
P26 LED
P27 LED
P30 Grip motor open left
P31 Grip motor close left
V01 Speaker
V02 ?
P32 Shoulder motor left
P33 Shoulder motor leftexpecting to be able to
P34 Grip motor open right
P35 Grip motor open right
P36 Shoulder motor right
P37 Shoulder motor right
P40 Waist left
P41 Waist right
P42 Leg motor left
P43 Leg motor left
P44 Leg motor right
P45 Leg motor right
P46 ?
P47 ?

“Left” and “right” are from the Robosapien’s point of view. With 31 lines of I/O before any enhancements might be considered, I had to add some extra I/O in the form of 2 MCP23S17 SPI I/O Expanders. The Bluefruit LE Micro uses SPI to communicate with its Bluetooth module, so adding the MCP23S17s was straightforward. I failed to get a single CS (Chip Select) line to work with HAEN (Hardware Address ENable) set on the MCP23S17s, but one CS pin on the master per MCP23S17 works a treat.

Prototyping board close up

Prototyping board close up

The Arduino code is very simple – it reads a character from either the USB serial port or the Bluetooth UART into a command buffer. ‘Commands’ on the modified Robosapien are trivial – motors and LEDs can be turned on and off,  delays of 0-999ms can be inserted and triggers for low/high on the switches can be waited for. For example, “s+” is “left shoulder up”, “S-” (lower/upper case for left/right!) is “right shoulder down”, “w0” is “waist motor off”. “L=” is right leg motor ‘brake’ (high signal to both sides of motor driver, though the brake effect is very weak). “e000” is all LEDs on. “e255” is all LEDs off. There are 8 LEDs. “W500” inserts a delay of 500ms before the next command is executed. “tb1” is “trigger when left button signal is high”. “q” turns off all motors.

Behaviour Send text string
Funky walk w+W300s+S-l-L+w-W360s-S+l+L-w+W360s+S-l-L+w-W360s-S+l+L-w+W360s+S-l-L+w-W360s-S+l+L-w+W360s+S-l-L+w-W360s-S+l+L-w+W360s+S-l-L+w-W360s-S+l+L-w+W360s+S-l-L+w-W360s-S+l+L-w+W360l0L0s0S0W500q
Turn in place w+l+L-W075l-L+W075w-W075l+L-W150l-L+W075w+W075l+L-W150l-L+W075w-W075l+L-W150l-L+W075w+W075l+L-W150l-L+W075w-W075l+L-W150l-L+W075w+W075l+L-W150l-L+W075w-W075l+L-W150l-L+W075w+W075l+L-W150l-L+W075q
‘High’ kick w+h+H-s+S+W999W600qS-tS1S=l-W999W150L+W150L-W150L+W150L-W150L+W150L-W150L+W150L-W150L+W150q

Source code for Arduino IDE is here: robosapienarduino.tar.gz

While the brain transplant is a success, losing the behavioural repertoire and vocalisations means there’s plenty still to do to return the Robosapien to full health. It being an old (nearly ten years?) toy, I’m wary of physically modding it further – such as adding contact sensors to its feet – and using it more in case I break its mechanisms.

The Robosapien has lived up to its claims of “made to be hackable”, but PCB-hacking is fraught with dangers. I’d like to see “made-to-hack” toys have their toy-behaviour controllers on pluggable daughter boards, exposing well-documented headers for owners who want to try their hand at brain surgery.

Having successfully operated on one patient, I’m looking forward to using the Adafruit Bluefruit LE Micro on new candidates… and there’s an unloved, apparently-hard-of-hearing Zoomer in the toy cupboard downstairs. “Zoomer! Here boy! Whassis? Screwdriver! Roll over!”

Add a search engine to Ubuntu Touch browser

You can’t add search engines with the browser GUI, perhaps because Ubuntu Touch is a Work-In-Progress and it’s still early days. Adding search engines to browsers still has patchy support from major browsers, even though the OpenSearch standard is donkey’s years old. I installed Ubuntu Touch on my Nexus 4 after experiencing the “Sleep Of Death” too many times and wanted to use my Firtl search proxy in Touch’s browser.

I located the configuration files (they’re OpenSearch XML description documents, which is handy!) by using grep -r to search for one of the existing search engine names. The search engine description documents are located (on my phone) in

/usr/share/webbrowser-app/webbrowser/searchengines

The / filesystem appears to be mounted read-only and must be remounted read-write to make changes. I did this with the command

mount -o remount,rw /

Copy your search engine’s OpenSearch description document into this directory and next time you attempt to set Ubuntu Touch’s web browser’s search engine, you should see the new search engine in the list. Don’t forget to remount / read-only.

Happy Searching!

Parametric 3D printer designs with OpenSCAD

xyzprintingdavinci1.0I bought an entry-level 3D Printer from PrintME3D recently – an XYZprinting DaVinci 1.0 and I’m impressed. I’ve printed nothing fancy – mostly brackets, clips, motor couplings and a few replacement parts for toys and tools.

I’ve been using OpenSCAD and wanted to  put my simple designs online. Some of my OpenSCAD designs might be useful for other people but with different dimensions, so I made a simple parameter form for each design with a preview image to show the effect of changes.

The preview image is linked to a simple 3D viewer powered by JSC3D, which uses STL produced on the server using the modified parameters from the form. The images are of an adjustable, flexible clip-on bracket to hold a rear mudguard in place with an M5 bolt on a Giant MTX bicycle belonging to my daughter.

Bicycle+rear+mudguard+stay+bracket+-+two+piece

OpenSCAD rendering of one half of symmetrical adjustable rear mudguard bracket

Bicycle+rear+mudguard+stay+bracket+-+two+piece-3D

JSC3D interactive rendering of STL file

The XYZPrinting DaVinci 1.0 is great for an entry model 3D printer, once Repetier (so it can be used with a Linux PC) has been installed and trial-and-error has established good temperatures for the bed and extruder and levelled the bed. After producing many satisfactory-to-good prints, I find it occasionally goes through bad patches of the first layer not adhering to the bed. If poor adhesion happens in only one small patch, it’s not always fatal for the print. I suspect occasional poor adhesion is due to the bed going out of level, possibly due to the vigour with which I remove the previous print’s baked-on glue.

Validating host names – top level domains (TLDs)

For a recent project I needed to update a hard-coded list of TLDs. Keeping it fresh seemed like a good idea, so I took IANA’s list of TLDs (IANA TLDs also available as text here) and made a little API service to provide the data in different formats:

ICANN provides the TLDs in upper case, but the Firtl API lower-cases them for convenience.