I’d like to start off a conversation about SD Lifespan. How can we make our SBC projects run for longer? In comments I've seen elsewhere, people seem to think it is ok that a Pi may well fail within a year due to SD - I don't think that is even REMOTELY acceptable unless you're making a novelty games machine.  People make a big deal about the reliability of Linux - not a great deal of use if the entire file system will come to a halt in a year...

I've never suffered this problem - my first heating system issue has appeared after more than a year's continuous use (and that includes doing lots of experiments on the same system).  You may have seen comments in earlier blog entries about this – for the first time ever I recently suffered a dead SD on one of my Raspberry Pi projects – stone, cold dead – read only and NOTHING on the Pi or my PC could encourage the SD to write again.

It has been said that some cheap SDs are not as large as they seem and as soon as you exceed use beyond their ACTUAL size – the chips become read-only.  I’ve yet to test this out but TKaiser has suggested testing all new SDs and in a previous comment has recommended SanDisk Extreme Plus.

The test program H2TESTW is widely available for free. I’m testing my first 16GB disk now – looks like it will take 20 minutes but as no user interaction is needed… time well spent.

In here you will find questions and opinions. In the comments hopefully you will find some resolution – lots of bright people read this blog and I’m hoping they have solutions rather than opinions.

If you read on the web about the subject of eMMC and SD and USB memory – it is hard to tell what is hard science and what is opinion.

For example there are blogs suggesting that instead of relying on SD, use a USB memory stick. I have trouble with this as the technology is similar. Why should a USB stick last any longer than an SD.

You’ll see reference to eMMC – there can be no doubt that eMMC (usually an internal module or chip) is usually faster than SD – but does it LAST any longer – some say yes, some say no. To be sure it is less convenient to back up compared to an SD you can simply pull out and replicate!

Then there is the hard disk. I have a natural tendency to think that a spinning disk has to be less reliable than solid state memory but every experience I have says the opposite. I could not tell you the last time a hard disk went bad on me. Of course – they tend to be more expensive – and they are very much larger than SD.

The general idea is that you can READ SD as often as you want but there is a limit sometimes described as 10,000 write cycles, sometimes describes as 10 times that amount. I suspect the latter and that there is just a lot of old information out there.

Then there is WEAR LEVELLING wherein some SDs have a chip inside that helps prevent a single location being written to, too many times – knowledge on this seems to be akin to witchcraft. WHICH manufacturers use this in WHICH SDs and HOW effective is it? I’ve not found a single source of information on the subject that is up to date and verified.

Today I read about putting some directories into RAM.

In the /etc/fstab file you can add for example

tmpfs /var/log tmpfs defaults,noatime,nosuid,mode=0755,size=100m 0 0

Works a treat but for one tiny item – Apache would not start up!

Several people have mentioned RAMLOG – but from what I can see –that no longer works with Jessie (the problem of old material hanging around on the web. This looks modern – and is reasonably straightforward to install – takes just a couple of minutes. https://github.com/azlux/log2ram – I installed it – and it works at treat. The default action is to update the disk every hour  - but moving the file “log2ram” from /etc/cron.hourly to /etc/cron.daily to me makes more sense.

So many questions – so many potentially wrong answers. See comments about actual number of writes to SD – would you believe any given location (not the one you see but the REAL location) could be as low as 1000s rather than 10s of thousands  – I had no IDEA it was that low).

On the subject of power supplies, in the comments you’ll find code for testing the likes of the Raspberry Pi – as there are registers in the Pi which pick up voltage issues… I was horrified how easy a long USB lead would allow the the Pi to work – but continually to register issues.

In testing – I found comments from TKaiser useful – then when wondering about the CPU frequency I found THIS article – and the associated script useful..

http://megakemp.com/2013/02/26/adventures-in-overclocking-a-raspberry-pi/

So already we see a need to reduce writes, only use good, tested SDs, use good good supplies with short leads. Not new, not rocket science but I am seeing some good science behind the need for this and look forward to reading more of your educated comments.

Keep the comments coming!

A Little Test

In the process of this discussion, TKaiser supplied us with a little script to return some information about power from the likes of the Pi2 or Pi3. This was intended to be used as a command line tool – repeating until told otherwise. Well, I like REPORTS…

I took out the loop section so as to return a single line of information – and that can conveniently be run in an EXEC node in Node-Red

I changed the script to simplify output – if someone can tell me how to produce output without “’C” and “V” so we have just numbers coming out – would be nice… I called this tk2.sh (changing permissions – don’t forget) and ran that inside an EXEC node in Node-Red…

     Maxfreq=$(( $(awk '{printf ("%0.0f",$1/1000); }'  </sys/devices/system/cpu/cpu0/cpufreq/scaling_max_freq) -15 ))
    Health=$(perl -e "printf \"%19b\n\", $(vcgencmd get_throttled | cut -f2 -d=)")
    Temp=$(vcgencmd measure_temp | cut -f2 -d= | tr -d C | tr -d \')
    RealClockspeed=$(vcgencmd measure_clock arm | awk -F"=" '{printf ("%0.0f",$2/1000000); }' )
    SysFSClockspeed=$(awk '{printf ("%0.0f",$1/1000); }' </sys/devices/system/cpu/cpu0/cpufreq/scaling_cur_freq)
    CoreVoltage=$(vcgencmd measure_volts | cut -f2 -d= | sed 's/000//' | tr -d V)
    if [ ${RealClockspeed} -ge ${Maxfreq} ]; then
        echo -e "${Temp}$(printf "%5s" ${SysFSClockspeed}) $(printf "%019d" ${Health}) ${CoreVoltage}"
    else
        echo -e "${Temp}$(printf "%5s" ${RealClockspeed}) $(printf "%019d" ${Health}) ${CoreVoltage}"
    fi

(If you see a question mark above - the word is sampling)

As you can see we have some space-delimited values! If you look at the bitfield, semi-permanent recordings of issues are on the left (most significant bits) while on-going issues are on the right. Extracting from TKaiser’s notes..

The bits on the right are:

0: under-voltage

1: arm frequency capped

2: currently throttled

And corresponding on the left:

16: under-voltage has occurred

17: arm frequency capped has occurred

18: throttling has occurred

It is easy enough to break this down..

Here is another version where I have split up the values

The first is the input – the second is the split version – the same except they are now in 4 different places

var reading=msg.payload.split(" ");
msg.payload=reading[0] + " " + reading[1] + " " + reading[2] + " " + reading[3];
return msg;

And from there you can do what you like with the data of course – one idea might be to read every minute and turn that string bitfield into an integer,  totalling up errors in the lower bits (you could just read bits of the string to achieve the same thing) … after a period send off a report email…

No need to report over-heating as the governor should take care of that – however – min-max summary in the email might be nice while testing.

The post A Question of Lifespan appeared first on Scargill's Tech Blog.

Following on from an initial comment by Keith Howell in the blog about NGROK, Antonio and I were chatting today about TKaiser’s little testing script and I wanted to share something with him. Skype of course is useless at this unless it’s a very small file - but we’ve used a tool called HFS in the past which opens up a simple file-sharing web-based system on your PC, meant for short term use of course.

Well, you can then of course give someone else access but that’s a pain as you have to go doing port redirects on your router – a little OTT for a simple one-off file sharing.

Combining two little utilities makes short work of this.

HFS lets you collaborate on files… while NGROK opens up the server port to the outside world without going anywhere near your router. Above you see the local version of HFS – and you can drag and drop files into that file area on the left… below you see the web version in this case on port 83 but any port will do as long as it isn’t in use.

If you look at the first image – top left where the house and the backslash is – you can add username and password for access by a simple right-click.

So – now you can go mess with your router and do a port redirect so someone can access this to grab files… OR open a command box in Windows and…

ngrok http 83

At which point the command line will show you the URL for someone to type in order to get the web page above! That’s it.  If you want to do stuff like secure access (HTTPS) and more you have to pay for NGROK but for this simple version – it’s all free.  Whenever you use NGROK you are given a unique access address so this is really meant for just short term use. Note again – use of port 83 here is arbitrary.

Both of these packages are installation-free – just stick them somewhere convenient like the desktop.

https://ngrok.com/download  (mac, Windows, Linux)

http://www.rejetto.com/hfs/?f=dl

The post File Share Quickies appeared first on Scargill's Tech Blog.

As a once-dedicated C programmer (I learned with the K&R book) I originally dismissed Javascript as an interpreted toy (great for websites of course), that is until I started playing with Node-Red.

At that point, after much struggle, I started to get reasonably good with it and once I got used to the string handling, C by comparison started to feel annoyingly awkward when dealing with strings and objects, despite it’s vastly superior speed.

Some time ago I looked at Espruino in a bored moment – saw that it was very much under-developed – didn’t even handle WIFI at the time – a bit pointless on an ESP8266 -  and dismissed it.

Fast forward to this week.  I had set in place some copying of files from here to Spain which, according to FTP was going to take an hour. I could not tell you why but suddenly I got the urge to go see what was happening with Espruino.

So – I believe the model here is that they developed JavaScript for a range of devices – including the ESP8266 and their funding comes from selling the boards and from contributions. I can’t help with boards right now as I have some of my own but I could not resist grabbing the code and sticking it into one of my ESP12 boards to see how they’re coming along. 

Regular readers will know that I like to revisit projects, most of the time great ideas unless they really take off, end up in the bin but just sometimes they go on to be something better.

So off I went to the site. As I’m using their software for free – here’s a push for them – take a look here  – maybe it is worth looking at one of their boards?  Anyway, I grabbed their general ESP code – and using the horrible ESPTOOL but the much prettier ESP8266FLASHER.EXE  I blew the modules into my 4MB ESP-12-E module.  I grabbed the code and followed instructions on this page.

Node the bulk of the code at 0x1000 – don’t make the mistake I made of making that 0x10000 – it won’t work and will cause hours of fun. All 4 files are in their download along with instructions.

I grabbed their web IDE (Espruino IDE) to test the code at 115K baud…. my own serial terminal does as well but theirs is prettier. For the life of my I could not get it to start up properly in their IDE – but in my terminal, on hitting reset on the ESP8266 I got this.

_____                 _

|   __|___ ___ ___ _ _|_|___ ___

|   __|_ -| . |  _| | | |   | . |

|_____|___|  _|_| |___|_|_|_|___|

          |_| http://espruino.com

1v91 Copyright 2016 G.Williams

Espruino is Open Source. Our work is supported

only by sales of official boards and donations:

http://espruino.com/Donate

Flash map 4MB:512/512, manuf 0xe0 chip 0x4016

>

Now I have to say two things – for a complete beginner I found the documentation totally confusing – making reference to LED1 which doesn’t exist and other things – but it turns out you have to read again more carefully. Secondly when I went onto their forums totally confused – two guys came in immediately and helped me.

Once I had the board responding which ultimately did not take a lot of effort – I went back to their IDE and the board was then responsive – I still don’t know why this is – but just so you know if it happens to you – you can talk to the thing with a serial terminal.

Here’s the first issue I encountered…

Every time I hit enter – the interpreter would come back with undefined. I could not fathom this out until someone pointed out that the interpreter always returns either a value – or if no value is defined, undefined…. kind of makes sense though not what you’d expect from, say, a BASIC interpreter.  Once I’d gotten to grips with this I got more ambitious.

1+2

The answer to that was “=3” – seems fair enough.

If I reset the board however I could get nothing out of their IDE – I went back to my simple serial terminal – no problem.

My next problem was following examples – one example referred to flashing a light called LED1 – turns out this doesn’t exist in their ESP8266 implementation but the ports can be expressed as numbers where 0 is GPIO0.

var  on = false;
setInterval(function() {
  on = !on;
  digitalWrite(0,on);
}, 200);

I entered the above – perfectly valid JavaScript – and blow me – it WORKED!

Of course – on resetting this would be lost unless you typed “save()” or “reset()” to clear everything out. Running that timer command with different values WITHOUT hitting reset would result in multiple timers.

So – reset – then run that code then save… and the flashing light survives power off!

Espruino does not sadly handle hardware PWM – a shame as Espressif HAVE implemented this at quite reasonable resolution (14 bits) but then their implementation is not perfect – turn it off and that timer keeps on ticking anyway – making a mess of other high speed events like SPI. So maybe that’s why they implemented software PWM – which, well, works!

Some things you should know – they’ve not implemented use of GPIO16 and space is limited at least at first glance – 12K for program storage (including OTA they only seem to use 1 Meg so that does leave nearly 3MB available on an ESP12)… you’ll find the limitations here – but to be fair many of those are ESP8266 limitations – no DAC for example – well of course there isn’t – the ESP8266 doesn’t HAVE a DAC. So the list of limitations is not as bad as it seems by any means. I’ve no idea why they did not implement GPIO16 as a simple output – I use it all the time.

What they DO have are libraries for a good number of peripherals so you might well want to take a look.

Speed? Well, they use the ESP8266 at 160Mhz by default (and that’s fine – I don’t know why we normally stick with 80Mhz given that the extra speed doesn’t take much more power) and claim that for general I/O the unit should be fast enough for general purpose use.

I used example code to connect to my WIFI – no problem – but when I tried to MQTT – I came unstuck  -their source library for MQTT simply would not load successfully.

I read that you had to put the REQUIRE into the right hand side of their IDE to load the module -  I did that and it worked but I really would like to understand how to load the module WITHOUT their IDE – as mentioned above I’m not too happy about it’s behaviour from power up.

So – armed with the MQTT library it was as simple as…

mqtt=require("MQTT").connect({
  host: "192.168.0.20",
  username: "admin",
  password: "mypass"
});

mqtt.subscribe("test/espruino");

mqtt.on('publish', function (pub) {
    console.log("topic: "+pub.topic);
    console.log("message: "+pub.message);
  });

I fired many messages at it from an MQTT publisher and sure enough  - they came in.  If one can get both WIFI and MQTT running on this interpreter and if it stays up reliably then maybe here’s yet another way to write code for the ESP8266.

To test, I disconnected the WIFI access point – and reconnected – a message came up to say MQTT was disconnected.. but on re-applying power to the WIFI access point… nothing.   When I ran the MQTT code again it connected (indicating that the WIFI connection automatically reconnected itself). What is needed now is a check for broken connection and indefinite MQTT reconnect and resubscribe attempts until it is back in running order.  If anyone has done this already – please do let us know.

http://forum.espruino.com/conversations/303129/

Lots of modules…

http://www.espruino.com/modules/

I can see this 12k storage for code being a REAL bottleneck – I do hope that gets sorted…

The post Espruino For You Sir appeared first on Scargill's Tech Blog.

I’m looking forward to trying out the new ESP12-S – this one apparently has a better antenna – and by the look of the schematic (see page 15 – thanks Alexander) the resistors on reset, GPI))0,2 and 15 are already there.

That should simplify layout somewhat!

ai-thinker.com - www.ai-thinker.com/uploadfile/2016/0909/20160909073755898.pdf

Hmm, board layout with 0.1 pins, 3v3 reg with decent heatsink area underneath, maybe 3 mosfets…

So this should work with existing layouts. I’ve kept back an IoTBEAR adaptor and ordered a couple of these from Ebay sub- £2 each.

The post ESP12S appeared first on Scargill's Tech Blog.

As Maureen and I start our summer migration to southern Spain, don’t expect anything new in here until around 17th – but of course – people are commenting constantly on various blog entries so I do hope you’re registered so you won’t miss anything.

I’m quiet excited, when we get to Spain, as well as the usual house-fixing chores, I have a new solar panel to fit (it is currently in the back of the car with a very expensive piece of plywood protecting it from several months-worth of “stuff” we’re talking over there. The plywood is likely to be re-purposed as a charging bench when I get there).

I’m looking forward to further delving into Espruino and the possibilities therein, I’ve more stuff to review and no doubt Antonio and I will continue to bring new developments to “the script” – which for the benefit of new readers is a comprehensive set of tools sitting on top of Debian/Raspbian/Ubuntu for working with IOT – most of which springs up from my original home control project.

I plan to do some more work with 433Mhz peripherals and RFLink and of course, sunbathing. Anyway, all of that to come – thanks for your recent contributions, I and no doubt others have learned a lot in the past few weeks, especially about making our SBC projects a bit more reliable.

For now, I’m about to go off on my travels and as you are no doubt aware, updating WordPress in the car is just – well, a non-starter really so if you’re expecting answers from me – a little patience please. Our trip will take 4 days in total, armed with cats and most of the gadgets I will need in the coming months.

The post Summer Migration appeared first on Scargill's Tech Blog.

I was recently given the opportunity thanks to a good friend, to check out the RTK.GPIO board, a small PCB with microUSB socket which promises to emulate the IO of a Raspberry Pi.

The board, sure enough has a Raspberry Pi connector looking remarkably similar to the Pi with 3v3 and 5v outputs.  The Kickstarter project advertises “an easy way to be able to add a Raspberry Pi GPIO header to your desktop or laptop.

The project managed to gain pledges of over £10k  which is pretty amazing for a simple board with little more than a processor and interface chip.

Claiming correctly that the Raspberry Pi add-on market is huge, the project states that there is an add-on board or HAT for “almost every electronics project for the Raspberry Pi”.  That kind of sets expectations for the board, sadly, a little too high.

Headlines like that set the pulse racing – could we test almost everything Pi I/O related on a PC? Well, no. Read on and you see that the IO is “basic” -  and you can do basic I/O – i.e. flashing a few lights. Well, you can do that on the BBC Micro-Bit – a horrible little “educational” board which can also do very little (why they didn’t go for the ESP8266 and Arduino-compatible software is beyond me given the massive user bases and hence support).

The Kickstarter page compares this simple IO board to Arduino, Raspberry Pi Zero and BBC Micro-Bit  with large user-bases. An Arduino Chinese copy is supposed to be £8-10 (wrong, Nano boards are available at under £2 inc postage and immediately programmable – possibly a tad more with the connectors pre-soldered). I won’t go into the others – you get the point.

Near the bottom of the Kickstarter page is a link to the software.  At the bottom of THAT page is a link to the LATEST software – except that it isn’t – it’s a 404 page actually. The actual software page, despite being quite old now (2014) has some recent updates but is not entirely error-free.

There is a reference to stretch goals including I2c but it is left a little vague as to what happened there (given the age of the board you’d think this would be updated).

The software is in two halves, code for the actual board and Python for the PC. I was hoping for some I2c examples but as far as I could tell most of the examples were variations on the same thing – flashing a few lights on and off. One example goes a little further and programs up an LCD display (again simple I/O) which might be marginally entertaining but again is merely using simple I/O

Had this board featured i2c and other serial protocols, opening up possibilities to control a wide range of devices, then I could see it being worth having on the bench for beginners… but to flash a few lights, for me, a cheap Chinese Arduino beats this hands down with a decent IDE that is easy for beginners and opens up a whole world of possibilities long after your first flashing light– whether you are using Linux (including Raspberry Pi) or a PC. For £1.15 or slightly more for the soldered version and the free Arduino IDE you get WAY more for beginners and advanced programmers alike.

Some time ago I was after extending an ESP8266 and looked at various ways to do this – one of which was simply to turn an Arduino into an I2c slave to control  ports – input and output along with analog input and PWM out. I wasn’t at the time thinking of an educational tool but the code therein was simple and would take very little time to convert the code to running off the serial port of virtually any PC, making it even easier to use.

If anyone wants to run with that and turn it into an “educational tool” – by all means.

If it is Raspberry Pi you’re after, well the Pi Zero WIFI is around the same price as the RTK.GPIO, it easy to use with the vast number of example programs out there and with WIFI built in is way more powerful.

The post RTK.GPIO Board appeared first on Scargill's Tech Blog.

Mongoose OS is an open source Operating System for the IOT, supporting ESP32, ESP8266, STM32, TI CC3200 (Amazon AWS IoT integrated). Code in C or JavaScript.  Time to give it a go!

It has been quiet here in the cave this week because, after a winter of sitting on my bum programming, I decided to do some work involving bags of sand and large paving tiles. Needless to say after a day of that, I was out of action for another day.  I’m still walking around very carefully in case I’ve torn something inside – but that didn’t dent my enthusiasm this morning when I checked my diary which said I had promised to give Mongoose OS a go.

So the context for this is ESP8266 – as a veteran C programmer who has recently taken on Node-Red like a new religion, I’m starting to really like programming in Javascript. Up to now,  the interpreted languages I’ve seen for the ESP8266 seem to eat up all the RAM – or have other limitations – so I keep looking. I played with another Javascript last week which seemed to have very little storage for programs…and marked Mongoose down to see if it was any good.

So – Mongoose – on a Windows 10 PC at least (they support other operating systems but I only have the one), you go to their site and download a program called MOS.EXE.   I put it on my desktop.  It wants a folder for its stuff so I made one on the desktop called “mos”.  You may choose to do things differently.

I followed the instructions after telling it what chip I’m using… from the command line on the desktop (don’t worry this is not command line stuff generally)

mos init --arch ESP8266

So that worked and created some files and directories under the mos directory I had manually added.

Next job was to run mos.exe with no parameters – this would bring up a web interface – from where I could pick a port for my FTDI which was sitting there with an ESP-12 board hanging on the end of it waiting to be lobotomised.

Well, that didn’t work, I tried all sorts to no avail until I tried contacting the author and in suggesting a debug he showed a command line with the port number on it.

Following their instructions but with a port number mentioned I blew the boot code onto the chip.

from the mos directory:

..\mos --port COM4 flash

In my case when doing the above I held GPIO0 low on reset to put the board into programming mode – it may be that whatever board you are using handles that using DTR… which Mongoose supports – kind of like the Arduino IDE does.

Then I ran the MOS utility from the command line (see below) WITH the port number and lo and behold the web page came up – talked to the board and I was up and running..

..\mos -–port COM4 ui

I realised looking at the web page console log messages that this was now working – but of course it did not know about my WIFI – I broke out of that command line and….

mos --port COM4 wifi mywifiname mywifipass

That sorted THAT out … back to the UI – as you can see despite referring to the wrong PORT – the UI was indeed talking to the board.

From there on it was plain sailing – I tried their example of a button press triggering an MQTT message… realising there was no reference to my MQTT broker in their script – in “prototyping mode”  I hit the configuration tab in the web UI and in the middle of THAT… was the MQTT setup.

No reference to ports so I put my MQTT address as “192.168.1.19:1883” – added username and password and saved the config – the UI uploaded that to the board and rebooted it. I could see in the console message that all was working well.

Here it is from boot upwards  - I’ve bolded up bits you might find interesting including the considerable amount of RAM left! Their example button used GPIO0 – I changed it to GPIO2. Looks like it knows about the time as well.

 ets Jan  8 2013,rst cause:1, boot mode:(3,0)

load 0x40100000, len 1720, room 16 
tail 8
chksum 0x56
load 0x3ffe8000, len 784, room 0 
tail 0
chksum 0x61
csum 0x61    u  jUJ B  TTTTj-kVVU   U (TT U    U     TT U+UZ I U TT U UT  E  ]Q  u M, V      K    u  V UP        o  r  o|   l  $`   s p   mode :
esp_mgos_init2       mjs_base 1.0 (20170421-141840/???)
esp_mgos_init2       Mongoose OS Firmware 2017042114 (20170421-141840/???)
esp_mgos_init2       SDK 2.0.0(656edbf), RAM: 51984 total, 51068 free
esp_print_reset_info Reset cause: 4 (soft reset)
fs_momode : sta(18:fe:34:d2:3a:a0)
add if0
unt             Mounting FS: 131072 @ 0xdb000
mgos_sys_config_init MAC: 1AFE34D23AA0
mgos_sys_config_init WDT: 30 seconds
mgos_wifi_set_mode   WiFi mode: STA
mgos_wifi_setup_sta  WiFi STA: Connecting to wififorus
mgos_i2c_create      I2C GPIO init ok (SDA: 12, SCL: 14)
mgos_sys_config_init HTTP server started on [80]
mg_rpc_channel_mqtt  0x3ffeff8c esp8266_D23AA0/rpc/#
mg_rpc_channel_uart  0x3fff002c UART0
Flash button is configured on GPIO pin  2 
Press the flash button now! 
mgos_app_init        mJS memory stat: before init: 46872 after init: 45920 after init.js: 41348
mgos_init            Init done, RAM: 41348 free, 39184 min free
scandone
aid 31
cnt 
_wifi_on_change_ Wifi: connected
ip:192.168.1.38,mask:255.255.255.0,gw:192.168.1.1
mgos_wifi_on_change_ WiFi: ready, IP 192.168.1.38, DNS 208.122.23.23
mqtt_global_reconnec MQTT connecting after 1859 ms
mgos_sntp_query      SNTP query to pool.ntp.org
mgos_sntp_ev         SNTP reply from 195.43.138.123: time 1492863022.756811, local 11.453939, delta 1492863011.302872
mqtt_global_connect  MQTT connecting to 192.168.1.19:1883
mgos_mqtt_ev         MQTT Connect (1)
mgos_mqtt_ev         MQTT CONNACK 0
mgos_mqtt_ev         Subscribing to 'esp8266_D23AA0/rpc'
mgos_mqtt_ev         Subscribing to 'esp8266_D23AA0/rpc/#'
pm open,type:2 0

I ran their button_mqtt example with this setup – worked a treat as I could see on the console log and also on my MQTT client set up to monitor this board.

Is any of this any use? After all we can program these boards in Arduino IDE or in C using the SDK and you can write BIG programs in C without worrying too much about storage?

Well, I’m only at the start of this but if you look at their site it looks like they handle OTA, temperature sensors and other stuff already and once you get past the hurdle I had (you may not have any of the problems I had getting started) – it takes seconds to try a sample program and if you are comfortable with Javascript – you might find this useful.

Couple of things – I tried disconnecting and reconnecting the MQTT broker – the board performed perfectly, I did NOT  try disconnecting and reconnecting the WIFI. Also there is only one access point name – a backup one would be nice… also I note that I2C is set up automatically on GPIO12 and GPIO14, so not as flexible there as doing your own thing in C – however – this is about ease of use if you’re into Javascript -  so this is not a definitive “yes this is a winner” but more just an introduction.  Comments welcome.

The post Mongoose OS appeared first on Scargill's Tech Blog.

Regular readers will know that I am an Ivona fan. In Node-Red I use the free Ivona service to provide high quality speech for my Raspberry Pi in Node-Red at the heart of my home control setup. Well, Ivona is now defunct. Amazon Polly is a replacement.

I’ll clarify that, Ivona is SOON to become defunct and you can’t create new accounts. The Amazon Polly system, is for most purposes a replacement for Ivona.

So – if you go to the Ivona site – you will see the reference to Amazon on the front page. The short, sharp answer is: Polly works, it is effectively free and it is as good as or better than Ivona. Read on.

So the Amazon system “Polly” works via an account. I have an Amazon Developer account and when I tried to add Polly – it said I didn’t have the right permissions – so – I added user Pete to my account and made him part of the Polly group – and that didn’t work either – then I noted something about payment and realised I’d not put any payment details in – I did that – and all of a sudden the thing came to life and I got the only things I needed – my user ID and secret password.

DON’T PANIC about payment – there is a free tier of up to (wait for it) 5 million characters per month for the first 12 months then $4 per million characters – by which point you probably won’t need any – read on) – for my purposes there is not a hope in hell I’ll ever reach the free limit. In addition  - the way I use it is the way they seem to want you to  – download a phrase as a file (MP3) and save it with a meaningful file name.  Next time you want that phrase – check to see if the file already exists – if so, play it, if not, get a new file from Amazon. In a typical use case that I might have, once the messages are used once there is very little chance of me needing to download anything and hence NO chance of incurring charges at least in the first year.

There are no doubt more elegant ways to do this than calling a command line from Node-Red and sometime someone will write a node to do it – might even be me – but right now this works perfectly and as far as I know it is the only published solution for Node-Red and Polly. If I’m wrong please do tell.

I’m assuming you have your credentials – don’t worry about location – they don’t have a location for England but all it means is to tell the code which server to use. Ireland works for me and it is working from here in Spain.

You need to grab the command line code – I used this on a Pi2.

sudo pip install awscli

Once that was in I used:

aws configure

to set up the user ID, secret key and location which I’d already set up on the Amazon site.

That done I tried this:

aws polly synthesize-speech --output-format mp3 --voice-id Amy --text "Hello my name is peter." peter.mp3

The resulting file was sitting in the /home/pi directory – this used the voice Amy (British female) to store a phrase into peter.mp3.  Good for testing but as you’ll see the final solution is much better.

The rest is about queuing messages, storing them with meaningful names, playing them back and making sure you don’t re-record a phrase you have already recorded. If you don’t like Amy – use another voice. If you want different voices for different phrases then you could incorporate the name into the filename (I’ll leave that to the reader).  If you want to add sound effects – just put .MP3 files in the relevant folder with your sound effects and call them by name.

Looking at the above diagram, a test inject passes  what you want Polly to say in the payload.

The first function looks to see if the payload has something in it and if so it pushes that onto a stack. The code then looks to see if speech is busy – if not and if there is something on the stack, it checks – if it is an mp3 file it sends the file to the MP3 player. If it is not an mp3, it looks to see if you’ve already created an mp3 for that speech, if so it plays that file, otherwise it passes the message onto Amazon to create the file – which is then played back.

It would have been nice to process new speech while playing something else back but that would get more complicated, involving more flags. As it stands this is easy to understand. You can fire in more speech or .MP3 files while one is playing and they will simply be queued.

You clearly need your Amazon account setup and Node-Red for this – you also need MPG123 player. Both Node-Red and MPG123 are in my standard script.

Here is the code I used in each of those functions…. the MPG123 exec node simply has mpg123 for the command and the append payload ticked. The AWS exec node has  aws for the command and the append payload ticked.

Here is the code for the three yellow function nodes:

if (typeof context.arr == "undefined" || !(context.arr instanceof Array)) context.arr = [];
if (typeof global.get("speech_busy") == "undefined") global.set("speech_busy", 0);

if (msg.payload !== "") context.arr.push(msg.payload);
if ((global.get("speech_busy") === 0) && (context.arr.length)) {
    msg.payload = context.arr.shift();
    global.set("speech_busy", 1);
    if (msg.payload.indexOf(".mp3") == -1) {
        var fs = global.get("fs");
        var mess = msg.payload;
        var messfile = mess.toLowerCase();
        messfile = messfile.replace(/[.,\/#!$%\^&\*;:{}=\-_`~()]/g, "");
        messfile = messfile.replace(/ /g, "_");
        
        if (fs.existsSync("/usr/audio/" + messfile + ".mp3")) {
            msg.payload = "/usr/audio/" + messfile + ".mp3";
            return [null, msg];
        }
        else {
            var voice = "Amy";
            msg.payload = 'polly synthesize-speech --output-format mp3 --voice-id ' + voice + ' --text "' + mess + '" /usr/audio/' + messfile + '.mp3';
            global.set("speech", messfile);
            return [msg, null];
        }
                    
    }
    return [null, msg]; // mp3 or synth        
}

// now play function node
msg.payload="/usr/audio/" + global.get("speech") + ".mp3";
return msg;

// clr busy function node
context.global.speech_busy=0;
msg.payload=""; return msg;

That was version 1. But ultimately I wanted new speech to be processed by Amazon WHILE a previously recorded item was playing (assuming a previously recorded item was playing and the next item had to be created).

Several hours later I came up with this – it appears to work!

and here is the latest code – probably not QUITE as straightforward to read – but when you run it – indicators on the EXEC functions (brown) show clearly that the software is able to play a recorded message while fetching a new one. Could do with some extreme testing.

if (typeof context.arr == "undefined" || !(context.arr instanceof Array)) context.arr = [];
if (typeof global.get("speech_busy") == "undefined") global.set("speech_busy", 0);
if (typeof global.get("create_speech_busy") == "undefined") global.set("create_speech_busy", 0);

if (msg.payload !== "") context.arr.push(msg.payload);
if (context.arr.length) {
    msg.payload = context.arr.shift();
    if (msg.payload.indexOf(".mp3") == -1) {
        var fs = global.get("fs");
        var mess = msg.payload;
        var messfile = mess.toLowerCase();
        messfile = messfile.replace(/[.,\/#!$%\^&\*;:{}=\-_`~()]/g, "");
        messfile = messfile.replace(/ /g, "_");
        messfile = "/usr/audio/" + messfile + ".mp3";

        if (fs.existsSync(messfile)) {
            if (global.get("speech_busy")==1) { context.arr.unshift(msg.payload);  return [null, null]; }
            else { global.set("speech_busy", 1); msg.payload = messfile; return [null, msg]; }
        }
        else {
            if (global.get("create_speech_busy")) { context.arr.unshift(msg.payload);  return [null, null]; } else
            {
            context.arr.unshift(msg.payload);
            global.set("create_speech_busy",1);
            var voice = "Amy";
            msg.payload = 'polly synthesize-speech --output-format mp3 --voice-id ' + voice + ' --text "' + mess + '" ' + messfile;
            return [msg, null];
            }
        }
    }
    if (global.get("speech_busy")==1) context.arr.unshift(msg.payload); 
    else {  global.set("speech_busy", 1); return [null, msg]; } // mp3 or synth        
}

and here, the two small function nodes

// clear creating
global.set("create_speech_busy",0);
msg.payload="";
return msg;

// clear playing
global.set("speech_busy",0);
msg.payload=""; return msg;

The post Amazon Polly Speech appeared first on Scargill's Tech Blog.

You may well have read my review of the NanoPi NEO back in August 2016 – a nice little H3 unit available in two versions, 256Meg and 512Meg RAM. Well, the new unit has an H5 64-bit processor and comes with 512MB RAM and although there is only one USB connector – there are three USBs available. Size is 40mm square! The NEO 2, however is more expensive than the original at $15 + post etc.

There are lots of add-ons available for it including WIFI, a TTL-RS232 module (just an FTDI but cheap enough), power dock, prototype board etc etc – you can read about it here so I won’t go into too much detail. Personally I wonder about the logic of bringing out add-ons for such a small board – would you not be better off buying more complete board in the first place? Where this board scores is size – it really is small.

So – nice looking - I’m not sure why they chose hardwired Ethernet over WIFI – I’d have gone the other way and kept the height down – but there you are. The existing case for the NEO does not fit the new model incidentally.

The board DOES have fast Ethernet which is a plus – but then if you were going to use it as a NAS the 512Meg RAM might be a limitation?

So you can log into the board via the FTDI or by normal micro-USB serial. At this point I headed off to the WIKI to get the operating system image..

One thing that worried me – and I’ve complained about this before – the requirements are:  A NEO2 (obviously), a micro TFT card, a power supply and “a computer running Ubuntu 14.04 64 bit” -  FRIENDLYARM  - YOU MUST BE JOKING! Most of us in the west have Microsoft Windows PCs. I do have one PC running Linux – it is running MINT and I’ve no intention of changing that just for one new board!!!

I was a little disappointed to find that the website has only Ubuntu Core available – unlike the M3 and M1 both of which have excellent DEBIAN images which work straight out of the box. Again – this fascination with Ubuntu – a bit disappointing to those of us who like Debian and have used it on previous boards.

At the time of writing, the ARMBIAN site had only nightly releases available and they refer to WIFI – so not exactly customised to this board which by default does not have WIFI !!

So, despite desire for Debian, I downloaded the Ubuntu Core software from the FriendlyArm WIKI site. The image was a mere 244MB in size so took no time to grab. Well, it would have been very quick but their sharing site is not the fastest in the world. I blew the image file with Win32 Disk Imager as usual and plugged it into the NEO 2 board. From what I could read, this would expand on first use and might take some time.

Power on and both the green and blue lights flashed for a moment then the green went full on – and the blue continued to flash. A quick check with Advanced IP Scanner showed the board sitting at 192.168.0.193.

I opened winSCP as usual and I tried logging in as root with password fa.

No problem – straight in. But I realised when running my normal script that SUDO does not exist on this version of the operating system – so that has now been added to the script so the extra half hour starting again was not entirely wasted.

Having ran the script and that having created a PI user – I went off to log in as user PI – loaded and ran the script again. While all of this was running the processor was running at 53c, so depending on use, the heatsink may not be needed after all.

However – it THEN turned out that Friendlyarm were using 15.10 – which is ANCIENT – so -

sudo apt-get update
sudo apt-get upgrade
sudo apt-get dist-upgrade
sudo apt-get install update-manager-core
sudo do-release-upgrade

All of which went swimmingly well – until – the BOOT area ran out of space.

So it looks like, as things stand, you can’t upgrade to Ubuntu 16.04 – and as their Ubuntu is currently the only option  - I have no option but to say this isn’t a lot of use until better software comes out or that boot area can be increased on the command line without needing a doctorate. Even EASUS won’t let me resize the larger partition to make space for a bigger BOOT area.

I cannot tell you how many times I have impressed on various companies – you cannot just put out hardware without software support – and Ubuntu 16.04 (if you have to use Ubuntu) is pretty standard stuff.

At this point I was out of options – but I remembered that Armbian do have a nightly build – I decided to give that a go. I downloaded the image, put it into the NEO2 – and rebooted.

The green light came on – but not the flashing blue. I waited. “nanopineo2.lan” appeared on my network – this time as 192.168.0.194.

I logged in as root with password 1234 and it seemed to come up – but would not stay – it would immediately disappear… so I fastened the serial interface up – no-where near as convenient… and logged in. Once that initial login was done – I could log in with winSCP as usual.

Several hours of testing for myself and Antonio later….

An updated script working with the Armbian Nightly build of 27/03/2017 works perfectly with this – there is a temperature monitor built in and at the peak of installation the temperature of the board sits at 63c.  Based on that I would assume that a heatsink is not needed for general control use. Do not try the original script. This one is called “the script”. FriendlyArm have also written to assure me that Ubuntu 16.04 will soon be available.

And yes the board is fast. Total Time to install the script was 46 minutes – as against several hours on the Raspberry Pi Zero WIFI !!

Update 30/03/2017 – I’m glad in a way that I had hassle with operating systems as it focussed Antonio and I to improve the script. However, true to their word, in the early hours of the morning (probably daylight for them) FriendlyArm came back to me with an updated Ubuntu Core 16.04 – here it is.

https://m.mediafire.com/folder/b7f7bt3s240uw

Interestingly, at the start of the PI part of the setup, it asked me for my PI username – then again part way through the Node-Red setup, near the end of that section, it did the same again.  This normally does NOT happen.  Also, the script took over an hour to run, 25% or so longer than the Armbian nightly build time, which was 45 minutes. Perhaps some tweaking is still needed! Temperature (without any heatsink) never got above 60c.

I don’t know how long that will remain valid so you’d best check their site for details. The install did however, once again work perfectly. Need to sort out that need for a password in the middle though – something up there.

30/04/2017 - You'll see in the comments that there is an update for the operating system here -  so today I tried that out on a 16GB Sandisk Ultra SD. The image expands to over 3GB and installed without issue. I have however noted some issues with the utility program as I could not get the unit to set up for 3.5mm jack audio. I got an error message.  There were a couple of other minor niggles which I've reported and no doubt will be fixed.  I've also noted that for the first time, they've created a PI user but when I used that for the script, it kept asking me for the password which was marginally annoying.  The script ran in 1hr 34 mins and during that time the temperature (without heatsink) crept up to 62c. I think it is fair to say I'd want to use this board ventilated or with a heatsink on it.  FriendlyArm do a nice heatsink but it does add a few mm to the thickness of the unit.

The unit still asked me for password so I ran:

sudo visudo

and changed the sudo line to read

%sudo   ALL=(ALL) NOPASSWD: ALL

That sorted that out. Annoying but not that critical.

A brief check of my script and everything worked - so at least there are no negative changes and far less messing about than there was when I started this blog entry.  I tried a typical Chinese WIFI dongle known to work with the M3 and similar boards – although IFCONFIG was aware that something was plugged in, no WLAN messages came up.  GPIO apparently is on the way. 

I think a little more time is needed before we can say this is a winner.  I never understand why companies release boards without full GPIO support when the openly available competition such as the Raspberry Pi has support by the shedload.

The post NanoPi Neo 2 appeared first on Scargill's Tech Blog.

This entry is all about InfluxDB, Grafana and a cute little Bluetooth garden sensor!

Some time ago a pal of mine and I did some swaps of spare kit and I ended up with a Mi Flora sensor.  A rather pleasing looking device,  the MI Flora has a 3v lithium button cell and can transmit by Bluetooth LE, the light level, temperature, battery level and moisture.

Battery is supposed to last for months. I’ve provided an AliExpress link above which puts the units at under £8 Inc. postage. No doubt that price will vary dramatically depending on where you chose to spend your money. Amazon charge more like £13.

So having pulled off the protective tab which meant the battery would work, I put the unit to one side as I had no idea how to read it.

A week or so later in Spain, with poor weather my pal Jay came on to ask me if I’d used it. I had not – and so we decided to give it a go. As it turned out I had some cheap Bluetooth 4 dongles I’d bought on a whim last year. I plugged one into my Raspberry Pi 2. The program “hcitool” lets you scan for Bluetooth devices and as it happens it was already installed on the Pi. From the command prompt:

sudo hcitool lescan

Instant success as the Mi Flora device with it’s MAC address was revealed. I made a note of the MAC address.

I would need Python 3 to experiment – but this was already installed on the Pi and some code to get the information from the unit. This code turned out to be ok with a little modification. I ended up creating a folder called /home/pi/myflora and dumping the lot in there.

Jay had modified the program to send MQTT. That of course needed the PAHO client. In order to do that I needed something called PIP.

sudo apt-get install python3-pip

sudo pip3 install paho-mqtt

Note NOT pip (which works for Python 2.7) but pip3.

I then added to the MI flora directory a test program which Jay sent me called bathroom.py – presumably because he’d put the sensor in his bathroom – I adjusted the output slightly as I was putting my unit in the garden

import sys
import paho.mqtt.client as mqtt

from miflora.miflora_poller import MiFloraPoller, \
MI_CONDUCTIVITY, MI_MOISTURE, MI_LIGHT, MI_TEMPERATURE, MI_BATTERY

poller = MiFloraPoller("C4:7C:8D:32:08:63")
print("Getting data from Mi Flora")
print("FW: {}".format(poller.firmware_version()))
print("Name: {}".format(poller.name()))
print("Temperature: {}".format(poller.parameter_value(MI_TEMPERATURE)))
print("Moisture: {}".format(poller.parameter_value(MI_MOISTURE)))
print("Light: {}".format(poller.parameter_value(MI_LIGHT)))
print("Conductivity: {}".format(poller.parameter_value(MI_CONDUCTIVITY)))
print("Battery: {}".format(poller.parameter_value(MI_BATTERY)))

# Publishing the results to MQTT
mqttc = mqtt.Client("miflorabathroom")
mqttc.username_pw_set("admin","xxxx")
mqttc.connect("192.168.1.19", 1883)
mqttc.publish("miflora/garden", "{ \"battery\" : " + str(poller.parameter_value(MI_BATTERY)) + ", \"version\" : \"" + str(poller.firmware_version()) + "\", \"sunlight\" : " + str(poller.parameter_value(MI_LIGHT)) + ", \"temperature\" : " + str(poller.parameter_value(MI_TEMPERATURE)) + ", \"moisture\" : " + str(poller.parameter_value(MI_MOISTURE)) + ", \"fertility\" : " + str(poller.parameter_value(MI_CONDUCTIVITY)) + " }")
mqttc.loop(2)
# End of MQTT section

So the idea was that running “python3 garden.py” would send some MQTT to my Mosquitto MQTT broker – and indeed it did (change MAC address for the MI Flora and MQTT details in the above).  Subscribing to “miflora/garden” did the trick – I could fire off the program and with in seconds I’d see the resulting JSON package coming out into my MQTT-Spy client.

Next job was to get Node-Red talking to the MI Flora – I used an INJECT node, set to retrigger every 15 minutes – firing off to an EXEC node which contained nothing more than “python3 /home/pi/miflora/bathroom.py”

That worked – so now I had a regular supply of data in the form of an easy to use JSON package.

I split the package up – and fed it into a Node-Red Dashboard graph – and that was my first disappointment. If you fire more than 4 items into those graphs – the popup fails to show all of them – and the graphs are not very good anyway. I think a lot more work needs to go into that.

I do use GROVESTREAMS on-line (which is REALLY flexible) to store data remotely but there’s a limit as to how far back you can read for free – that was not the case when I first started using them and wrote the Node-Red-Contrib-Grove node to make it all easy.

So now I was on a search for a decent graphics package that would run on the Raspberry Pi (or similar). NOT as easy as it sound,  unless you’re very easily pleased when it comes to graphics.

After a lot of experimenting I decided to give Grafana a shot. I think if I started writing this blog again, I would would have Grafana in the title because this really, as it turns out, is the star of the show.

Having seen a few screenshots, I had visions of populating  a SQLITE 3 database and firing that data into Grafana.

Erm, no. Despite (wait for it) using SQLITE internally to store users and dashboards, the package does not actually support SQLITE for data !!!  I've explained that now, to save you some confusion later. When I did the install, it kept referring to SQLITE and could I HELL figure out how to set up a database for my data – that’s why.

As a Pi user (that means SD use so I’m not keen on MySQL as SD has limited WRITE capability and I worry about databases shortening the life of the SD - see the later article on SD life - very informative) the options for data sources did not look appealing to me at all, the least offensive being INFLUX and that is the one I settled on – partly as there is a node for it available for Node-Red.  Despite the pretty website this is still very preliminary at version 1.02. Influx doesn’t use terms like tables and fields which kept me going for a couple of hours – but it is good at storing time-stamped data – indeed you don’t even do the time-stamping. You create an empty database and start firing data at the DB in pairs – stream name – and data value – it really is very simple once you get started. I guess you could think of it as a single table with pairs of data-name and data-value.  I went down a few dead ends and discovered if you got your stream names wrong – good luck renaming them as INFLUX does not support renaming!

Anyway, it ‘s all very easy once you get started. I did things backwards and installed Grafana first. Here’s what I did – no guarantee it will work for you.

You might want to consider reading this before using InfluxDB on the Pi for long-term use - there are very few build options for small SBCs and this article gives the impression it likes more RAM than we would normally have on the likes of the Pi. I do have to say though that it is working just fine on mine after several days of operation. Comments welcome.

I grabbed the Grafana file for my Pi from here. https://github.com/fg2it/grafana-on-raspberry

sudo dpkg -i grafana_4.1.2-1487023783_armhf.deb
sudo /bin/systemctl daemon-reload
sudo /bin/systemctl enable grafana-server

That was Grafana up and running on port 3000 – user name admin, password admin to actually DO anything – and quickly change that password!!! I did that, went into the ini file and disabled user registration as the admin can do that and opened up port 3000 redirect in my router so I could access this from the outside world. That all worked well.

wget http://ftp.us.debian.org/debian/pool/main/i/influxdb/influxdb_1.0.2+dfsg1-1_armhf.deb

sudo dpkg -i influxdb_1.0.2+dfsg1-1_armhf.deb

That set up influx and that ran without further ado at port 8083. I did NOT open that up to the outside world. In there I created a new empty database called “logger”

In Grafana – which supports influx without any changes, I followed Engineer John’s instructions to set up the data-source and a new Dashboard.

So the next thing – would be to get some data.  I played with this for some time – should I keep incoming data source names simple – as you can’t rename them? Then Jay pointed out that in Grafana you can ALIAS names. That did it. So in influxDB node for Node-Red, you fire in a payload – and depending where you want to put your data – you use msg.measurement.. Yes, exactly, what’s wrong with msg.topic you might ask. So here is my little MI Flora MQTT node firing values into InfluxDB

And that yellow function block…

node.status({fill:"blue",shape:"dot",text: msg.payload});

tmp=JSON.parse(msg.payload);

msg.measurement="battery"; msg.payload=tmp.battery; node.send(msg);
msg.measurement="sunlight"; msg.payload=tmp.sunlight; node.send(msg);
msg.measurement="temperature"; msg.payload=tmp.temperature; node.send(msg);
msg.measurement="moisture"; msg.payload=tmp.moisture; node.send(msg);
msg.measurement="fertility"; msg.payload=tmp.fertility; node.send(msg);

Yes, I know – far better ways to do this – but it was my first stab at it and it worked. For general use however I added an incoming function that merely copies msg.topic into msg.measurement and that means any single data-source coming in via MQTT can be blasted straight into InFluxDB.

Yes, that’s write – you don’t create tables or fields – you just send in field names and data and that’s it!

I also fed in some data from my general sensors – with horrible topics like pergola/lighting but of course thanks to the Grafana alias ability – names like that would not show up in Dashboards.

Having stored up some data it was time to get it into graphs – I took the VERY simplest approach of showing lines as I have at this point no idea what some of the Grafana functions do – I DO know that smoothing the data might be fun as Grafana relies on the data-source to do that and Influx doesn’t do that.  What IS nice is that you don’t have to worry about things like “I want the last 5 minute’s worth” or “show me the last month’s worth” as Grafana does all of that for you – from here  it gets easy.

New dashboard and under metrics ADD ROW – takes an hour or so to get to grips with this..

I’ve expanded the top one here – I have 3 lines in that one graph – see the blank ALIAS BY  - if you want to change the source name (in this case battery) put something in there.

And that  is how I got to this point…

So top left you see battery status from my deep discharge battery and  the two MI Flora units, then moisture levels from the two units – light levels – and on the right centre – a whole host of temperature readings, overall humidity and finally just to play with – the UK temperature split off and displayed differently.

There’s a lot to take in – and I am as you can see NOT an expert at this yet so before firing questions back – you might want to spend some time looking at the links on this blog entry and using Google. If you think another database is better by all means write in to say why – and if you think there’s a better easy-to-fit-in-Pi graphing system, preferable one that works with SQLITE – again do let us know. I started all of this with zero information late afternoon yesterday and had it all working by the end of last night – so it’s not THAT big a deal even if my clumsy writing makes it look that way.

Not entirely sure what’s going on the this battery level on my first day – but it did occur to me that the unit is intended for putting into plant-pots – not sitting outside in the sopping rain so I’ve covered it in cling-film!! As we’re off back to the UK for a few weeks,  I don’t want to come back to a rusted heap.  Note the Mi Flora battery at the top – seems to be self-recovering!!

Update: the little Bluetooth sensors have now been running for months and the battery levels remain fine – the Bluetooth range however is crap – best to have a Bluetooth dongle nearby to talk to them – don’t even think about two thicknesses of wall!

Update 27/03/2017: I've been hitting issues with out of bounds temperature data – i.e. a couple of duff readings crept in - with values like 2000 and -64  which of course flattened the data completely. I realised quickly that InfluxDB has no DELETE function and as far as I could tell - things like comparing data with a value made the graph line disappear.

I put something up on the web and someone came back and asked if I was using InfluxDB 1.2 which apparently has a DELETE function. Well, I wasn't - I was way back at 1.01 - worse - the same folder where that came from only went up to 1.1

However, thanks to THIS helpful chap - I easily updated to Influx 1.2 without the issue he described. I STILL can't add something like "where value < 100" but maybe I can delete ??? Anyone with more info on this please by all means chip in...

Update 18/04/2017: I've just fitted the second of these units and also fixed a minor error in the code above.  All that is needed for a second unit is a copy of the program with different MAC address and different MQTT publish address. Sure you could use the same one twice with command line parameters but I took the easy option.

Bluetooth RANGE is becoming an issue. The Pi is in my office - breezeblock wall then the garden - the second unit I tried maybe 15ft or so from the office wall - not a chance.  Clearly BT4 is good for power, not so good for range.

So - there's a nice job for an ESP32... BT in - WIFI out.  A relay!

Update 30/04/2017: At the time of writing, InfluxDB is up to version 1.2.2 and if you follow the link I gave above but instead of 1.2.0  use 1.2.1 in the two relevant links - you will get the latest version of InfluxDB. Similarly I found Grafana 4.20 - same place as before but called https://bintray.com/fg2it/deb/download_file?file_path=main%2Fg%2Fgrafana_4.2.0_armhf.deb

I just repeated the Grafana install routine to update it using the new simpler name grafana_4.2.0_armhf.deb - all worked perfectly.

I'm pretty happy with Grafana and I now know how to delete data points (by time) but if you really want to use SQLITE and do your own thing - take a look at the excellent work that Csongor Varga has done - https://www.youtube.com/watch?v=nkKf26oKzhQ

If you like this post – please share a link to it by social media, by email with friends or on your website.
More readers means more feedback means more answers for all of us. Thank you!

The post Garden Data Collection appeared first on Scargill's Tech Blog.

I said I would not touch the ESP32 until the price came down and true to form, I just received my first ESP32 from AliExpress. Still a little more expensive than I’d like at £5.35 (inc postage came to £6.67) but I felt it was time to make a start. Bear in mind that a similar-looking ESP8266 board is half this price.

The ESP32-S board is as you would expect from a Node-MCU-type board, a simple USB plug in job with 0.1” centre holes along with reset (EN) and programming  buttons.  Unlike the ESP8266, you can see the startup messages coming up at 115k board and when in programming mode the board says “waiting to download” – which is very civilised.

I’m waiting to hear back from the Mongoose people as I had no success getting this board to work with their Javascript software but in the meanwhile I thought I’d give the Arduino IDE a go.

As far as I can tell there is no board level install for the Arduino IDE and ESP32 yet but I found that the Sparkfun guide worked well on my existing Arduino 1.69 IDE.   I had to install GIT on Windows to make this work – but that took mere seconds, all in all an utterly painless experience.

Despite mentioning the reset and programming buttons, you don’t need to touch any of those with the IDE – just load in your program and press the UPLOAD button on the IDE and you’re all set.

I took the more or less standard Blink example and changed the port to pin 5.

int ledPin = 5;

void setup()
{
    pinMode(ledPin, OUTPUT);
    Serial.begin(115200);
}

void loop()
{
    Serial.println("Hello, world!");
    digitalWrite(ledPin, HIGH);
    delay(500);
    digitalWrite(ledPin, LOW);
    delay(500);
}

You don’t get any simpler than this – I checked which ports were available on the IDE then plugged in the ESP32 board  and tried again – COM6 appeared out of nowhere so I selected that. I also selected the board “Node32s” which appeared in the Arduino IDE once I’d done the Sparkfun install.

And that, ladies and gents, was it. I plugged a LED+resistor to ground and GPIO5 and it flashed reliably while putting out the serial message “Hello, world!”.

Now, the novice might say “so – I can do that with an ESP8266…  but I always like to try something really simple first to prove that the board and software were both working.

I borrowed this diagram for my board from the excellent free book by Neil Kolban “Kolban-ESP32”. So, why look at the ESP32 when the ESP8266 is such a great board? Well, how about this… 34 GPIO lines, 7A/D convertor inputs, WiFi AND Bluetooth, 520K RAM and 3 UARTs – that good enough for starters? Oh and 3x SPI and 2x I2c.  The ESP32 is also capable of running in deep sleep at 20uA.

Of course all of this sounds good but whether all of that is easy to use is another matter – remember the ESP8266 has I2c but it doesn’t work so you have to use a software version!

In the IDE, ESP32-specific examples were, well, almost non-existent – so I tried one of my ESP8266 examples with WIFI and MQTT. Hey – there’s nothing like going in at the deep end and I absolutely did not expect results!!!

My code is a work in progress and not shown here – but it is a first stab for the ESP8266 at getting my normal WIFI+MQTT+various commands software running in the Arduino environment – a half-hearted project I’ve been working on in the background and includes OTA.  To be honest I was just wasting time trying this out because of the differences between the ESP8266 and the ESP32.  I set the compile going and of course, it did not know about the ESP8266 WIFI or the ESP8266 OTA code. I looked at an example in the Sparkfun page referenced above and they used “#include WiFi.h” – so I changed my include to that – and commented out the OTA stuff.

Well, blow me, it compiled – and better - when I pressed upload in the IDE – the next thing I saw on the serial monitor was it connecting to my WIFI and then to my MQTT broker!!! I had no real commands in the code yet but if I fired a dummy message out to MQTT it is supposed to come back with “Did not understand” which means WIFI and both MQTT publish and subscribe work.

Well, it worked – and sent the message back, with no modifications  - I’m impressed. Within a couple of minutes I had my usual {out5:1} working and from there, well, the sky is the limit.

As far as I can tell (someone tell me if I’m wrong) we’re still stuck with a maximum of 1 GB of FLASH for programs and that little demo used 40% of it – I’m assuming most of that is libraries. RAM on the other hand – well, there’s TONS of that. Unlike the Arduino IDE when using ESP8266 which give you lots of options for FLASH, I could find no options for the ESP32 – which might account for the 1GB limit. I’ll keep looking.

Assuming the price continues to drop I can see a great future for this little device.

The post ESP32 First Steps appeared first on Scargill's Tech Blog.

These are just thoughts – hopefully someone is going to write in and tell me I have it all wrong.

As regular readers will know I have reviewed (and messed with) MANY small Pi-type boards including the Raspberry Pi, Banana Pi, NanoPi etc etc.  I’ve also had a go with some power supplies and even made up my own solutions at board level. We’ve had discussions about the issues with FLASH and minimising writes and so mentally it all seems to be coming together. What is needed for our home control central control projects is:

A cheap SBC with enough RAM to be able to lose some for LOG2RAM and buffering for datbases etc to utterly minimise writes to FLASH – so ideally maybe once per day, stuff in RAM would be written to FLASH.

To go with the above then – clearly a power supply is needed that will not fail – i.e. with battery backup.

Sure, if you through enough money at this – there are solutions for everything but at a budget?

Firstly, RAM – just not enough – some of these boards have 256Meg and 512 Meg  - enough for normal use but if you start stealing some of it to run Log2RAM and RAM-buffering databases (RAMDISK) then you start to run out – it seems to be that 2GB of RAM would be a nice figure yet I’m only aware of a small number of these boards that have this much – if memory serves me well, the Odroid C2 – and a board I’ve only just recently heard of – the FriendlyArm NanoPi K2. There will be one or two others I’ve long since forgotten.  The Raspberry Pi people for example apparently have no intention of moving to 2GB in the near future. We see people using these boards for media management yet if you look at the notes from the designers of such software they usually suggest 2GB minimum RAM!  I don’t do media management but our chats about FLASH have me using Log2RAM and RAMDISK and depending on your needs you could be looking at losing 512Meg or your precious RAM.

Power supplies – especially important given the above - recent testing has shown that we don’t seem to have a single low cost unit available that can handle the battery going down to zero – and recovering gracefully while the the load is connected or having to press a button or some impractical manual interjection – and bearing in mind that several of the better boards need  2amps+.   Looking back through the blog there is not one – there’s one on Kickstarter but that is now way, way late in appearing and for all we know THAT might not do the job.

What are people’s thoughts – am I talking nonsense? Am I missing some products? What do you think? Do people have low-cost solutions I’m not aware of?

The post Thoughts on RAM appeared first on Scargill's Tech Blog.

This could be the shorted blog I’ve ever written! We’ve been talking about ESP8266 recently and as I was running out of boards, I sent off for some of these from AliExpress – my favourite store next to Ebay.  At a total cost of just over £3 they had to be worth a go.

So on the LEFT you see the reset button – on the right you see the FLASH button – this might indicate that normal Arduino-style programming doesn’t work… so I put it to test with a sample BLINK program.

When the board comes on – the normal blue light on the ESP12 flickers – nothing unexpected there.  I tried programming from the Arduino IDE – nothing - mind you I was expecting that. I pressed the reset and FLASH buttons together and let go of the RESET then the FLASH button – I started the programming – the blue light flashed – success.  As I was using the standard Arduino blink sketch, I tied my LED+RESISTOR to ground and GPIO13 and sure enough – the light flashed.

Just to be sure as these WERE cheap, I tried all three of them. No problems at all.

for the board itself – the lettering on the top is utterly messed up – someone wasn’t thinking when they did the screen print – however the lettering UNDERNEATH is just fine. The regulator used is one of the larger ones and so should handle a little extra 3v3 if this is needed for any peripheral boards. I’d not recommend driving the board from 12v however!  I did my testing powering the board from the microUSB connector. No problem. There IS another LED on the board but it didn’t do anything when programming – no doubt at some point I’ll be doing something and it will light up at which point I’ll update the blog.

And so there it is – next I have some WEMOS D1 Mini boards on the way from Ebay.

The post Cheap NodeMCU Boards appeared first on Scargill's Tech Blog.

I’m sure most of us who play with the likes of Raspberry Pis and other SBC boards as well as other devices, have looked at uninterruptible supplies at one point or another – I often use the little battery powered chargers you get for phones – and until recently I thought that was a universal solution – having powered my Pi in Spain for many months now on one of these and we get awful power cuts there – no problem.

04/May 2017: This is an update to the original article as I found a problem!

Then someone pointed out to me that SOME of these units turn off the output when charging. Well of course they don’t ADVERTISE that – so that’s a problem if you’re looking for cheap options.

So today I received from BANGGOOD a cheap circuit board that promises to do the job. As it happened my one and only Lithium in a torch left in storage over the winter had kicked the bucket… but I remembered I had a spare but dead battery pack for one of my laptops – and proceeded to open it up. Sure enough half a dozen Lithiums in there.

I took one out that looked clean – dead. Put it on my big power supply for a couple of minutes at 2 amps – it looks ok.  So now I’ve attached it to this little board and plugged the board into a decent USB supply. one of the little chips is getting a little hot as it pumps power into the board but I think it will be fine (update once the battery ha some charge this cooled off – then running a FriendlyArm SBC nothing at all got warm).   So after it had been on for a while to give the battery a chance to get some power, I plugged an SBC into it’s output. You can see all of this in the picture above. Worked a treat and the little board settled into a blue-flashing-light mode after a few seconds.   I repeatedly disconnected the incoming power – no difference, the SBC just kept going.

So if you want something new to play with…. and you’re aware of the dangers of going within 100 miles of Lithium batteries – you might want to give this a shot. At £2.37 inc shipping – you can’t really go wrong provided you give it a damn good test before doing anything serious with it.  I think I may make a little bank of them with the remaining Lithium batteries from my power pack.

http://www.banggood.com/37V-Liion-Battery-Mini-USB-To-USB-A-Power-Apply-Module-p-928948.html

But there is a catch! I only just discovered this...  If the battery is TOTALLY DEAD and you start things up - nothing happens - the battery doesn't charge - the load does not come on. Disconnect the load for a short time and the battery immediately starts to charge and you can now reconnect the load.

This got me to thinking about a solution and in keeping with the ultra-low costs above, here is my ultra-low cost solution..

An Arduino Nano with relay (total cost around £3 from AliExpress) can control the output of the unit via the relay. The analog input (A0) can attach to the battery while the NANO is driven by the output of the unit.  The ultra-low current consumption of the NANO when the relay is off should prevent issues....  so what is needed is this - when the power gets low, this unit should alert the Pi to shut down..  it does this with a GPIO bit (D11 in this case) and optionally via serial - a nice way to do this would be to output a JSON package.

Here is an example of testing the little board from power up..

{"voltage":0.62,"status":"init"}
{"voltage":5.00,"status":"init"}
{"voltage":5.00,"status":"starting"}
{"voltage":5.00,"status":"starting"}
{"voltage":5.00,"status":"starting"}
{"voltage":5.00,"status":"active"}
{"voltage":5.00,"status":"active"}
{"voltage":0.96,"status":"warning"}
{"voltage":1.08,"status":"warning"}
{"voltage":0.94,"status":"standby"}
{"voltage":0.97,"status":"standby"}
{"voltage":1.14,"status":"standby"}

So the output message is init - and stays that way - the power is seen to be adequate - and a timer starts - after X seconds the relay turns on. Next, the battery is seen to be low - a timer starts and a warning GPIO bit is turned on  - the Pi could depending on your software, choose to use either the serial signal or the digital input - and shuts itself down. The output then goes low and stays low until the battery voltage rises significantly. You HAVE to have "hysteresis" in there - i.e. a difference between ON trigger voltage and OFF trigger voltage as the battery is going to recover a little the instant you turn off the load.

Here is the software I've put together to play with the idea - it seems fine - you may wish to change voltages and timings (or not).

#define LIGHT 12
#define RELAY 11
#define WARNING 10

#define WAITING 30
#define ONDELAY 20
#include <stdarg.h>

uint32_t mymillis = 0;
uint8_t lightState = 0;
uint8_t state = 0; // 0 is power up, 1 is on, 2 is turning off
float vol;
uint8_t onDelay = 0;

uint16_t lightOnSet = 2;
uint16_t lightOffSet = 1022;

float offVoltage = 2.4;
float onVoltage = 3.2;

char *theState = "init";

float voltage()
{
  // read the input on analog pin 0
  int sensorValue = analogRead(A0);
  // Convert the analog reading (which goes from 0 - 1023) to a voltage (0 - 5V):
  float vols = sensorValue * (5.0 / 1023.0);
  // print out the value you read:
  return (vols);
}

void p(char *fmt, ...)
{
  char buf[128]; // resulting string limited to 128 chars
  va_list args;
  va_start(args, fmt);
  vsnprintf(buf, 128, fmt, args);
  va_end(args);
  Serial.print(buf);
}
void setup()
{
  // put your setup code here, to run once:
  Serial.begin(115200);
  digitalWrite(LIGHT, LOW);
  digitalWrite(RELAY, LOW);
  digitalWrite(WARNING, LOW);
  pinMode(LIGHT, OUTPUT);
  pinMode(RELAY, OUTPUT);
  pinMode(WARNING, OUTPUT);
}

void loop()
{
  // put your main code here, to run repeatedly:

  if (mymillis <= millis())
  {
    vol = voltage();
    p("{\"voltage\":");
    Serial.print(vol);
    p(",\"status\":\"%s\"}\r\n", theState); // sadly printf here does not support floats
    if (state == 0)
    {
      if (vol > onVoltage)
      {
        if (onDelay == 0)
        {
          onDelay = ONDELAY;
          theState = "starting";
        }
      }
    }
    else if (state == 1)
    {
      if (vol < offVoltage)
      {
        state = WAITING + 2;
        theState = "warning";
      }
    }
    if (state > 2)
    {
      digitalWrite(WARNING, HIGH);
      state--;
      if (state == 2)
      {
        state = 0;
        digitalWrite(RELAY, LOW);
        theState = "standby";
      }
    }
    else
      digitalWrite(WARNING, LOW);

    if (vol > onVoltage)
    {
      if (onDelay)
      {
        if (--onDelay == 0)
        {
          state = 1;
          theState = "active";
        }
      }
    }
else
  {
      if (onDelay)
      {
        onDelay=0; theState = "standby"; state=0;
      }    
  }
  
    if (lightState == 0)
    {
      lightState = 1;
      digitalWrite(LIGHT, HIGH);
      switch (state)
      {
      case 0:
        if (onDelay)
        {
          lightOnSet = 80;
          lightOffSet = 944;
        }
        else
        {
          lightOnSet = 1;
          lightOffSet = 1023;
        }
        digitalWrite(RELAY, LOW);
        break; // battery is low
      case 1:
        lightOnSet = 980;
        lightOffSet = 44;
        digitalWrite(RELAY, HIGH);
        break; // battery is ok
      default:
        lightOnSet = 512;
        lightOffSet = 512;
        break; // intermediate state;
      }
      mymillis = millis() + lightOnSet;
    }
    else
    {
      lightState = 0;
      digitalWrite(LIGHT, LOW);
      mymillis = millis() + lightOffSet;
    }
  }
}

Of course you could easily add an LCD display to this - a good cheap, low current one that needs 5 of the available wires and can EASILY be added to this circuit is here. This works - I just left it out to make life easy.

Now then there is an important point here - the NANO in this case is running on 5v... and the Pi will be 3v3 - so in the case of the WARNING OUTPUT and the SERIAL OUTPUT if you use one, the other or both you will need LEVEL SHIFTING - the simplest being a pair of resistors - that is a resistor from the NANO output to the Pi and a resistor from that PI pin to ground - as you don't want 5v going into the Pi - the maths is simple - I'd make the first resistor say, 10k and the second (grounded) resistor, say 15k.  Experiment.

I'm experimenting with my enlarged version of the above code including a backlit Nokia display..  https://bitbucket.org/snippets/scargill/xn497

Have fun.

The post Uninterruptable Supplies appeared first on Scargill's Tech Blog.

I struggled with the title for this one as I figured if I put /sys/class/anything in the title – everyone would run away as I usually do.

This entry came about because of my utter annoyance that so many little Linux SBCs out there come with ZERO or very little support for GPIO.

Let’s take as a working example the NEO2 from FriendlyArm (and I’m not speaking out of turn – they know my views about not supporting this stuff).  Lovely little board – and today I was playing with my extremely un-elegant power supply backup solution (see previous blogs) and realised I had no idea how to access the ports.  It looks great in the advert:

• GPIO1: 2.54mm pitch 24 pin-header, compatible with Raspberry Pi's GPIO pin1 - pin 24. It includes UART, SPI, I2C, IO etc
• GPIO2: 2.54mm pitch 12 pin-header. It includes USB, IR receiver, I2S, IO etc

Lovely – grab your favourite Raspberry Pi utility – I like GPIO – the command line utility – or maybe the nodes in Node-Red – and your off and playing with GPIO.

I’m joking of course – none of that stuff works on these other boards though one or two have “equivalents”!  Sometimes the manufacturer provides utilities. I remember one provided some C code – lovely – then told me it only works for ROOT users… you know – the user you are not supposed to be !!! I can only get ONE of the four serial ports to actually work in Node-Red even thought it reports they are there (not permissions – tried that).

Anyway, this evening I sat down with a large glass of artificial strawberry drink and started reading.

There is a directory in most of these boards in Linux called /sys/class/gpio

So I went looking – sure enough the NEO2 had this – and a couple of files and 2 shortcuts to GPIO that meant absolutely nothing to me.

So here’s what I’ve learned – just by messing about mind you.

You have to expose the GPIO – so let’s take GPIO0 as a working example.

As root user:

echo 0 > /sys/class/gpio/export

This made a GPIO0 magically appear.

echo out > /sys/class/gpio/gpio0/direction

This made the GPIO an output

echo 1 > /sys/class/gpio/gpio0/value

This made the output go high.

Sounds simple – but which I/O – I made 2 variations of that last command – one with 1, one with 0 – and played them back and forth while sticking a LED on the various potential IO pins on the NEO2. Voila – a lot simpler than I was expecting it turns out that GPIO0 was PA6 on the NEO2.

Now I was getting excited – but here’s the thing – I needed to control this output from Node-Red – and my Node-Red is user PI and you can’t DO this command as user Pi. You could use SUDO but SUDO doesn’t work with echo. 

pi@neo2:~$ echo 1 >  /sys/class/gpio/gpio0/value
-bash: /sys/class/gpio/gpio0/value: Permission denied

Arggghh. But simple? Use Sudo

pi@neo2:~$ sudo echo 1 >  /sys/class/gpio/gpio0/value
-bash: /sys/class/gpio/gpio0/value: Permission denied

Aaarrrrrggghhh.

So I spoke to my friend Antonio and he came up with TEE.

echo 1 | sudo tee /sys/class/gpio/gpio0/value

Aaaaaah. WORKS – so now I put this into an EXEC node in Node-Red and tested – WORKS.

As you can imagine the next step would be to waste hours trying to find out which ports you can use – but then I realised that at least in this case  - FriendlyArm supply the Linux GPIO names! I tried GPIO2 – bingo! Nothing is permanent – so a little code in Node-Red would need to check if a port had been used before – and set it up as an output first if not – that’s easy – a lookup table for meaningful names and a bit of substitution in a function would also be easy.

Could I get this to work with inputs? SURE – almost the same starting with “in” instead of “out” assuming you’ve already exposed gpio0...

echo in | sudo tee /sys/class/gpio/gpio0/direction

The next part was sheer guesswork

cat /sys/class/gpio/gpio0/value

Sure enough – this returned the value of that port – either 1 or 0. Would it work in Node-Red EXEC?

YUP! Works a treat. So you could write something for this outside of Node-Red but the latter is a nice friendly environment – and the use of a global would let you check if the input or output was already in use – this is readily adapted to other gpio-software-challenged boards.

This is course is trivial – it does not answer how to use peripheral features such as I2c etc. but if you’ve been struggling just to get the very basic input and output – perhaps that part of your struggle is now over.

Someday soon I hope to have proper drivers for the ports on this and other devices – but then someday soon I hope to win the pools. Best not wait around too long for either.

Meanwhile I have my input and now I can set about detecting low power – and shutting the device down gracefully – just need to write a little code to set up  that input automatically.

So – first things first – here’s a setup I’ve made for the NEO2 – a simple text file called “pins.def” (for want of a better name) in the /home/pi directory (for want of a better place).

{ "0":"in","203":"in","6":"out","67":"out"}

A simple piece of json defining the ports I’m interested in – and their direction.

And here is my test rig in Node-Red

One of those ports – GPIO6 is an output and is a good test…(pin12 on the dual connector on the NEO2).

Simple enough – on power up the injector node fires off anything to the yellow function node which sets up the ports by exporting them and setting their direction.

var fs = global.get("fs");
var fl="/home/pi/pins.def";

       try {
                fs.accessSync(fl, fs.F_OK);
                // If file exists pull out the JSON string
                fs.readFile(fl, 'utf8', function (err,data) {
                  if (err) {

                        }
                    var pins=JSON.parse(data);
                    for (var port in pins) {

                           msg.payload=port + " | sudo tee /sys/class/gpio/export"; node.send(msg);
                           msg.payload=pins[port] + " | sudo tee /sys/class/gpio/gpio" + port + "/direction"; node.send(msg);

                       }                   
                });
            } catch (e)
                {
                }

Insert your own error stuff as desired.

That’s it – the tests simply fire off the right info as described above to the EXEC node – I’ve tested this from reboot – it works.

Now – remember the point of this – I wanted to be able to check a port to see if I should gracefully power down the Pi if the port was low? Well, I can check that every 10 seconds like this.

Of COURSE I can make this more graceful. So every 10 seconds – I trigger the cat command on GPIO0 – RBE merely outputs only on change – probably not needed.. then a simple function returns ONLY if the output is zero…. and shuts down the NEO2.

Easy – you could adapt this to any setup. 

Not my finest code – but it works. Thanks to this – my little test rig (see the NEO2 article) is now up and running with graceful NEO2 shutdown.

Time to go to the market and enjoy a little sunshine – secure in the knowledge that my little board won’t come to any harm, barring a flood.

The post GPIO the Hard Way appeared first on Scargill's Tech Blog.

The code that forms a series of articles on this blog has now not been updated for some time – simply because it “just works” and has been doing so reliably for many months. However, I’ve been wanting to add bits to it and my problem was I was running dangerously low on iRAM. This would not affect the stack or reliability but would just stop me from continuously adding code.

Well, thanks to Espressif that is now history. The latest Esp8266 SDK, released just days ago, fixes a number of things on the ESP8266 – you can find the SDK here – you will need it if you wish to compile code but of course ROMS are available as usual.  The upshot for me is the return of nearly 800 bytes of iRAM which is something of a breath of fresh air.

The software has been updated to version 1.8.5, the code is available here…  and the SDK here.

And if you’re wondering what on EARTH I’m talking about… if you’ve seen “the script” – this is what it was put together for in the first place.

Blowing the home control 2016 code

Home control 2016

Extract from the manual (which is the best source of info).

The post ESP8266 Code Update appeared first on Scargill's Tech Blog.

Some time ago I wrote about HighCharts – a wonderful set of Javascript tools for doing graphs – and I gave an example of getting the graphs into Node-Red.  I’ll not repeat that as you have the link above.  At the time I’d not added in databases, I’ve since made some VERY simple code to bring in the charts from SQLITE -  I have to say it isn’t the fastest code in the world so I won’t embarrass myself by replicating it here. But at the time, my only limit was Node-Red Dashboard – there was as far as I could tell no way to get full screen or anything like it on a PC  -  until I took the plunge today and ask the right people – turns out that you can set an individual page in Node-Red Dashboard to WAY past the normal 6 blocks width simply by altering the size of a GROUP used only on that page.

So given block sizes set by default to 48px per block – by merely changing the maximum width in blocks you can easily get to full screen size for a given page – while leaving others alone. The result, using HighCharts (their JavaScript is free for personal use – you pay for commercial use)… below you see info I’ve been collecting in SQLITE3.

In the above image you see multiple lines, with missing point filling, from 27th April until now – taken from a single SQLITE table where I have different topics for different lines and I store the value and date (and that’s about it) – I call the table “timeseries”.  So I pull out what I need, sorted into topics – and fill in the info required by HighCharts – the best way to pick this up is to read my blog item above and go have a play on their site.

And so it was that I got the full screen I wanted and one of the guys in SLACK said “have you tried PLOTLY?”  Well, I went off to their site and this was all commercial and you could pay so much a month – didn’t much impress so I went back into the conversation only to be told I was looking in the wrong place, had I noticed their OPEN SOURCE JAVASCRIPT?

I hadn’t of course but here’s the link.  Free for you and I  to use.  Better – it is also easy to experiment with it online as they use CODEPEN for demos.  So click on that CodePen link and you’ll see some code you can mess with and this result…

You will see JS code you can mess with to change values – one of the things I like is smoothed lines.  So where you see “mode: ‘lines+markers’, add after that…

line: {shape: 'spline'},

and VOILA…

Note that the blue line is now nice and rounded… and it was that simple – I spotted that line in another example. Try altering the data. So if you put this and a link to the library (see the HighCharts example) you should be able to add this to a Node-Red Dashboard template – and from there it is a simple matter to add dynamic data.

Another line you can add in there is essential for me – as someone who occasionally loses data due to power outages or whatever…

connectgaps: true,

That does what you think it does! Here’s another way to play with Plotly - https://plot.ly/create/

I’m quite happy with Grafana… but you never know – this time next week they may turn commercial (or not) and Plotly looks like another contender for ultra-nice graphing with relative ease (I didn’t say just “ease” because for some folks none of this is easy – but it is do-able).

In practice:  I took this example from Codepen -  and inserted it into a template – well, it didn’t work did it. Turns out there were two things wrong – the default DIV – even with an ID attached – would not work – and I’m not 100% sure why but the remote CDN would not work either.

SO – I simply grabbed the minified .JS file from the CDN (put the address in a browser, save the result in a file) – transferred the .JS file to my /myjs/ directory on my Raspberry Pi (read the HighCharts and other blog entries in here for the reason I do that) and made a new simple div – voila – it worked.

Here is the relevant code to put into the template..

<script src="/myjs/plotly-latest.min.js"></script>
   
<div id="plots" style="min-width: 100%; height: 100%; margin: 0 auto"></div>

Follow that with the script and make sure that the call at the bottom refers to “plots” – or whatever you call your div. I tried fixed pixel values in the div as well as 100% – all work as expected. Better still – when I viewed the page on my phone (Android 6) it worked extremely well.

Of course that is simple, static data but of course that isn’t the same as injecting data into the graph from, say a database. Also the time/date X axis right now is actually a string – so lines with different dates end up all over the place.

I fixed that in this little test – here is the content of the TEMPLATE

<script src="/myjs/plotly-latest.min.js"></script>
<div id="plots" style="min-width: 800px; height: 400px; margin: 0 auto"></div>
<script>
   (function(scope){
        scope.$watch('msg', function(msg) {
            Plotly.newPlot('plots', msg.data, layout);
        });
    })(scope);

var layout = {
  xaxis: {
    type: 'date',
    title: 'Time'
  },
  yaxis: {
    title: 'Value'
  },
  title: 'Connect the Gaps Between Data',
  showlegend: true
};
</script>       

And a function to feed this with values – something that could be expanded to multiple lines and any number of values?

var dd=[];
dd[0] = {
  name: "The first trace",
  x: ['2012-01-01', '2012-02-01','2012-03-01','2012-04-01'],
  y: [55,33,11,1],
  mode: 'lines+markers',
  line: {shape: 'spline'},
  connectgaps: true
};

dd[1] = {
  name: "The OTHER trace",
  x: ['2012-01-03', '2012-02-01','2012-03-01','2012-04-01'],
  y: [22,43,66,77],
  mode: 'lines',
  line: {shape: 'spline'},
  connectgaps: true
};

msg.data = [];

msg.data[0]=dd[0];
msg.data[1]=dd[1];

return msg;

So there it is – Plotly – taking in arbitrary values from an arbitrary source and displaying on demand – the next step from this would be to change that function node to pull in stuff from a database.  For now I’ll leave that to the reader.

Have fun.

The post A Quick Plotly appeared first on Scargill's Tech Blog.

I’ve covered the FriendlyArm NEO and NEO2 boards here before now and generally been in favour – but for one little item in common with many other boards – little or no IO support. 

Updated 17/05/2017: I promised I would write to FriendlyArm and ask for WIRINGPI and that I did. In the meanwhile if you take a look at the relevant blog entry you’ll see I made a start at using GPIO the hard way and even found a WIRINGOP (OP – Orange Pi) variation that would do basic IO but that was about it.

Days later I received an email to say that FriendlyArm have enabled WIRINGNP (as they call it) by default in both the NEO and NEO2.  I rushed to do the updates they suggested for existing setups but I hit a couple of snags (and reported my problem) – but they’ve also brought out a new ROM with the updates incorporated so I tried that on the H5-based NEO2. They have a WIKI on the subject. There are C examples for setting up soft PWM etc etc   - but here I am only concerned with the GPIO utility that runs as user Pi and hence can easily be called from Node-Red or indeed – the command line.

The Ubuntu command line is pretty much what you might expect on any similar “Pi” and certainly it runs my script so I had no trouble testing the new WIRINGNP as they call it.  I was particularly interested in the GPIO program which a non-root user can run from the command line.

Using their npi-config menu-driven command line tool I told the NEO2 I want to use PWM – you can also set audio, the welcome message, serial, I2c, SPI and PWM enable/disable. At that point the board rebooted with the new settings.

Now-  I’m terribly excited, the serial setup allows for enabling and disabling serial ports 0-3 (yes, FOUR serial ports) with serial port 1 enabled – we’d discussed this by email and this is what I was expecting – I didn’t expect a nice, easy menu setup for this – well done if it works…

After some diversion as they’ve chosen to refer to physical pins as against wPi or BCM pin designations -  I eventually realised that pin 38 (on a separate 4 pin connector) is the hardware PWM pin

gpio mode 1 pwm
gpio pwm-bal
gpio pwmr 1024
gpio mode 38 pwm
gpio pwm 38 100

The first four lines are setup and I discovered that line 1 was necessary despite being of no relevance to this board (1 on the Raspberry Pi is the PWM pin) – the setup for pin 38 follows as the second last line before setting the PWM value to 100  (out of 0-1024). A handy LED on pin 38 confirmed that the hardware PWM on that pin works.  I've asked FriendlyArm to check. Also if the LED load on this pin is too heavy,  reboot does not work but that I think is more down to the H5 processor. 

Note also there is some kind of screen debug output when you run these commands and the C example – again I’ve reported that back to FriendlyArm as you don’t get this on the Rasperry Pi.

You can’t do any other kind of PWM on the other pins using GPIO – but that also applies to the Raspberry Pi.

For simple GPIO – let’s say pin 12

gpio mode 12 out
gpio write 12 1

Works a treat.

FriendlyArm also have conveniently made a user PI – so you don’t even have to create that though they persist on their site in assuming quite often that you are user ROOT – I have suggested they think about that as people are usually encouraged NOT to run as root. Running my script is as simple as dropping it into the PI directory – giving it execute permissions and running it. I ran my script to install Node-Red etc., so I could check out the serial ports.  Serial port 0 is normally used for debug and FriendlyArm suggested making serial port 1 available by default.

As you can see in the image of my Node-Red setup for the serial ports (standard serial node) – I hooked up all 4 serial ports and in the case of serial port 1 – I shorted the input and output pins together. As you would expect – injecting into the ports brought back failures in 3 of them as they’re disabled – but in the second case – the timestamp was sent back. Perfect. I’m out of time but quietly confident the rest would work if I needed them. So in just this one respect – compare this to the Raspberry Pi 3 – it has Bluetooth and that conflicts with the serial port – you don’t actually have a spare serial port – here you have three.

At the end of this – the little NEO2 is going to end up talking to an Arduino Mega running that 433Mhz RFLink board as well as lots of other tasks.

I have yet to test SPI and I2c but as it stands – these boards are in very select company in getting support from their own manufacturer!

The following day I received an update. We’d discussed the various naming systems for pins – physical – ie. the actual pin number, wPI (Wiring Pi) and the BCM numbering schemes.  I suggested physical was probably best – but when the update came they’d implemented both for those programming in C.

//using wiringPi Pin Number
int pin = 18;             
wiringPiSetup ();             

  //using Physical Pin Number
int pin = 38;             
wiringPiSetupPhys();                                          

  //using BCM Pin Number
int pin = 5;
wiringPiSetupGpio();

Well, I thought that was quite satisfactory!  Here’s a little test I tried – on physical pin 38…

such a program needs to be compiled as such:

gcc -Wall -o flashtest flashtest.c -lwiringPi -lpthread

Needs to be run as root  - WHY!!!!!!!

Anyway, sudo ./flashtest works – and I tested in the Node-Red EXEC function – it works in there (again using sudo  - no big deal). So it should be possible to make little C programs to do just about anything you want with GPIO and call them!

More later – I’m hoping they’ll get rid of some debug stuff in there but this according to the WIKI can be setup for both the NEO (cheap) and the NEO2 (more powerful).

The post NEO REBOOT appeared first on Scargill's Tech Blog.

Quite a while ago now I did a project based on the little Nextion displays, resistive touch-LCD displays which run on serial commands and come with an editor to make simply touch displays for even the simplest of micros.

Well, that project has proven to capture a lot of interest and I have one of those displays running 24-7 in a holiday rental property – and it has been doing that for a couple of years now.  I’ve not done anything new with the displays for a while but this morning, a new generation display turned up for me to take a look at.

The NX8048K070_011C is a new 7” capacitive display complete with case (also available without the case) from Itead – the company who also produce the Sonoff units we’ve discussed many times in this blog.

The unit runs on 5v at 550ma and is to use the company’s own words, a powerful 7.0'' HMI TFT capacitive multi-touch display with a beautiful enclosure, with 32MB Flash data storage space, 1024 byte EEPROM, 8192 byte RAM, and GPIO supported.

The “GPIO supported means there’s an 8-bit port connector. There is also an SD connector for storing info and a small 4-way connector for power, serial in and serial out.

So first things first – the advert headlines with $88 but in fact that is for the resistive version – for the capacitive version, add $20 for a total of $108 + shipping. The shipping via normal e-parcel was $11.40  (I’m only using dollars here because the ad is in dollars). DHL shipping is a horrifying $57 – but then I would not use DHL even if they threatened to cut off my toes. I STILL think the best thing ITEAD have done is the original Sonoff – and I say that because of the price. Right now they are going at $4.85, which for a boxed unit controlling the mains via WIFI is just stunning. The later units are in better boxes and are more powerful but by the time you reach $20 each the price to control a lamp is starting to get a little less enticing.

Anyway, I digress. So the Nextion displays work in a strange way – you don’t write directly to them like a typical Raspberry Pi display – but then you don’t have that overhead either. Once set up – which you do with the free “Nextion Editor” – the display merely needs serial text commands to do updates hence the simple 4-wire interface.

I’d rather hoped the rather nice display case would have some kind of feet – but instead it comes complete with 4 wingnuts which come complete with elongated bolts for mounting into existing installations.

One of the most annoying features of the original Nextions was the SD – you could not actually store data in it – that has now been addressed along with supplying real time clock, GPIO (well, on an ESP8266 you lose 2 wires to run the display so it can be useful to get that and more back) and of course the smoother capacitive touch display.

The actual display itself is only 800*480 which means you can see the dots but from any distance at all this is fine. The design of the case means it would easily fit into a cavity in a control panel and the whole thing is quite sturdy. I do wonder however why they made such a large case compared to the actual size of the LCD itself – as you can see in the photo.

As if to demonstrate the new features, when I powered up the Nextion display – it showed a swipe as you might find on modern smartphones – once I’d swiped, I was asked for a password (1111) – I put that in and the display proceeded to demonstrate the time and date. Might have made for a more posh demo if it hadn’t been set to 2044 – but never mind.

So, simple operation, nice easy, light touch compared to the older (but cheaper) resistive displays)  but at the end of the day, ease of use or otherwise comes down to the Nextion Editor – because they have done their own thing you really need to be able to use the editor to set the display up.

I downloaded the latest Nextion editor (free) on my Windows PC. After all this time it is still 0.47 – you might think considering the amount of time the Nextions have been out they would at least be to 1.0 – but then perhaps one should not judge a book by it’s cover.

It did occur to me that just MAYBE you could fit a little ESP8266 or NANO into that case – but on opening – no – not unless you cut off any connectors.   Oh, and I found out why the clock was wrong – there is indeed a battery connector on the board – but no battery was fitted – well, that accounts for that!

At this point, rather than going through all the fine detail of the , here’s a video.

Also take a look at my Nextion WIFI Touch Display -  the size is different but the general idea is the same. I’d like to hear from anyone who’s been using these displays for any length of time to hear what they think of the current editor – and what do you think of the price (please don’t write in to say you’ve used an old tablet  - already had that discussion). Is this good – or is there something better out there?

The post Big Nextion appeared first on Scargill's Tech Blog.

What, not ANOTHER board from FriendlyArm? And yes, you’d be right to wonder why I’m doing a bunch of articles on them right now – well, it just so happens I got a box of stuff all at once so I thought I’d get them all out of the way at once.

So – why another board? If you’ve read my various blogs you will know that regardless of what operating system these SBC board from various places are supposed to come with,  I usually end up putting these boards onto Debian or more decently command-line Ubuntu and running my script on them.

The only exception were a couple of earlier boards from these guys which ended up as media centres – Kodi etc.  The T3 boards.  Well, they’re good but compared to a good phone, most of these Android installations are either low resolution, slow speed or both.  Indeed on a daily basis I end up apologising to the wife as the Android crawls along. I like to blame old versions of Android but ultimately most of these SBC boards are not as fast as a good phone. I often say in conversation – if only they’d use the chips used in phones…

And so it was that I opened up the K2 board from FriendlyArm and the big claim is 4K video. I had to give it a go, didn’t I, so I downloaded the Android setup from their site.

Now this board has a number of special features.  Firstly it has an infra-red input and they supply an infra-red handset – so no messing around – it just “works”.  Another feature which I don’t think any of the other boards manage – this board is size-compatible with the Raspberry Pi 2 and 3 – and that means – yes, you can use any of the millions of Raspberry Pi cases out there – and there are some beauties!!!  Nice.

Even then I wasn’t that optimistic about Android – after all I already have a media centre running and it takes some setting up – did I really want to go through all that again?

Ok, well I’ll cut a long story short.. YES, ABSOLUTELY. This is lovely. So the board has 2GB RAM which is more than any Raspberry Pi has ever had or apparently ever will. It has a standard microSD and I JUST happen to have a 64GB microSD which will give BAGS of room for movies.

And so I guess you’d expect me to rant on for ages about various features? Well, I’m not. It works and works extremely well. I’ve tried YouTube and Chrome searching for high quality movies and everything I throw at the board runs flat out – maximum settings I can see on Youtube are 1080p and the quality is excellent – my monitor isn’t high enough res for true 4K but watching the smooth videos I have no doubt the claim pans out. The remote works a treat. The only thing that is missing is a 3.5mm jack which would have been good as my HD computer monitor doesn’t have a loudspeaker attached!!

I say it is fast – it is but if you REALLY want to go to town there’s an eMMC connector on the board… personally I’m happy with the SD.

You can look up the specs online on the link above, but in short, an Amlogic S905 with a Mali graphics processor, 2GB RAM, Gigabit Ethernet, WIFI, BLUETOOTH, audio via HDMI or Bluetooth,  Oh, my headphones – hang on… testing…RIGHT – that sorts the audio – the Bluetooth works a treat with my little GEAR4 speakers – no problem.

I’m running this with the heatsink attached – which also has a fan but a fan is always a daft idea in a media centre… and… yes, I think you can likely get away without needing the fan… the fins are cool enough to touch continuously.  Right – I’m off to find a nice box for this.

Oh and it has a proper power connector but I just plugged in the microUSB connector and that’s working a treat.

```

The post NanoPi K2 appeared first on Scargill's Tech Blog.