Since this one works, I ordered a few more sensors and microcontrollers from the same manufacturers and I soldered them

I now have a bunch of these little devices around the house, and I am very happy with how they extend the range

As long as I place enough of them along the way, I can sprinkle them wherever I want and they’ll self organize, and one of them reports all their readings to an SQLite database on a RaspberryPi Zero

They are on their own WiFi, which doesn’t matter for security since I’ve written their program myself, but still feels great

Haven’t done statistics and graphs yet

#electronics #AirQuality #DIY

Latest news is that I got my brain into embedded thinking mode

I changed the way I determine if a sensor reading is worthy of archiving

The way I set up my probes, each #ESP8266 send its readings as fast as it can (that’s every 3s) to the entire mesh and the root node, the #ESP32, forwards them across WiFi networks over HTTP to my #RaspberryPi Zero W

The rpi0 then decides what readings to keep and what to toss

The naïve strat

The way I was deciding what to toss was, until yesterday, an SQL query doing a GROUP BY on the node sending the readings and checking the MAX date for its readings

The database file is already several dozen MB in size and growing by almost that much daily (I’m stress testing my system),

Even with a proper SQL index, the SDCard has to read a lot of information on every incoming sensor reading (SQLite is a file based system, there is nothing in RAM), just to decide what to keep and what to toss. That translates into lots of unnecessary i/o of all kinds, faster SDCard wear, and general latency unpleasantness

The rpi0 CPU was 40% to 90% all the time, according to htop

I’m way too used to PostgreSQL…

The embedded-friendly strat

My new strat is simply creating a file on disk for each sensor, changing its modified timestamp and using that to determine when its readings were archived last

That’s just a handful of empty files on disk and I’m only changing their metadata, not their content. They’re using next to nothing in terms of space. Checking their metadata is also really fast, and possibly cached by the OS and/or my web server.

Now every HTTP request fits within 4ms if the previous reading for the same sensor was recent, or at most 80ms when the database gets a write

The rpi0 CPU is now back to low levels, between 7% and 12% most of the time

#electronics #diy

This morning my ISP had an outage and I restarted their router, which happens to power my rpi0 with the web server for this project

And I forgot to restart that web server, so I lost all data from today

But before that, it was performing beautifully. There were 844 records for each node since last night, like clockwork. I happen to have cleared my previous data last night, so all the nodes were running and already on a stable mesh at the time.

Each sensor has been broadcasting its readings every three seconds, at which point my server decides to ignore most readings until the 1 minute mark (per sensor) and I’m happy that not one minute was lost for half a day of recording

I am less happy that after the web server was restarted, none of the microcontrollers found it by themselves. It looks like the entire mesh collapsed because the root node couldn’t bridge both networks again.

I understand that the root node didn’t restart the bridge between both WiFi (I didn’t set it up that way), but I would have thought that the iot mesh should have continued independently? Perhaps because it is declared as the root node? I’ll have to test this more.

I reset the ESP32 root node and it picked up the home WiFi fine, but none of the probes connected to it again (this mesh should have self healed within minutes, at most)

I went around the house to reset each ESP8266 node and yes, that is when the mesh self organized again and I started seeing the web server collecting readings again, as their BME680 sensor finished its self calibration routine

#meshnetwork #AirQuality #esp32 #esp8266 #bme680 #raspberrypi

I’m very pleased with the general responsiveness of my little project

Pictured in the attached video is the bridge #esp32 node, the one that waits for data from the #mesh and forwards it to the storage device

The blinks represent data being forwarded from one WiFi to the other

Seeing 6 blinks within 3s means that all the probes have called home in time before their next sensor reading

Today I soldered a few more of my custom air quality thingies, flashed them with the exact same program as the other ones, gave them a USB adapter and plugged them in their own spot in the house

They automatically found their path in the mesh, then they started sharing and recording their readings, which automatically appeared in my dashboard, and all I had to do was rename them there

In the picture, the number in the parentheses is the number of readings for the day (more or less one per minute) and all bars and lines represent average readings for the day; I have one graph per room to get the timelines

Humidity and temperature are on the left axis, all the other are on the right axis

(Looks like the one in the bedroom crashed again)

The one in the attic is missing however long it took me to solder the new things, since that’s the one I unplugged to use as a template

#electronics #bme680 #esp8266

Attached is a video of the #esp32 happily blinking its payloads across WiFi networks to the #RaspberryPi Zero, for storage in the #sqlite database

Thousands of readings per day (about 1300 each sensor so far, today), all sensors being within 15 readings of each other, I’d call that stable

I’m happy to notice that when a board fails, it eventually recovers on its own (it takes several hours, though)

I need to change the storage mechanism

I’m doing backups of my readings database from the Raspberry Pi Zero to my PC, and what used to take 4 MB of space early on (in September) now takes 60 MB

Obviously even with indexes, SQLite can’t be expected to read this from flash storage to present my graphs in a timely manner

I guess I can double store: once in a daily database I can use to present my graphs, so that SQLite only ever has to read from tiny databases, and a second time in a single database meant for larger analysis (off the rpi)

The migration will be painful 😂

#RaspberryPi #SQLite

There is a minor issue with my IoT setup, and that is with DNS queries: they are absolutely flooding my PiHole logs 😅

Apparently, every time my gateway ESP32 forwards any readings to the Pi for storage, a new query reaches my local DNS resolver (which happens to be the same Raspberry Pi Zero); the ESP32 doesn’t seem to cache entries

At a rate of 1 reading every 3 seconds for each node, and just under a dozen of those…

Trying to tail pihole logs is a lost endeavor at the moment

I guess one solution would be with a dedicated IoT WiFi and its own DNS resolver?

#iot #pihole #esp32

I just added documentation to the README for my DIY mesh network of air quality sensors

• what this project does
• list of hardware parts & costs
• how to build

Comments are welcome! This is still just a personal project

https://github.com/GuillaumeRossolini/griotte

#iot #esp32 #esp8266 #RaspberryPi #arduino #mesh #AirQuality

Latest in my little project: build a separate network of the same, that I can ship to someone else as a plug and play thing

I had lots and lots of network issues so I reworked the code a lot, I added keepalive messages and reduced memory leaks as much as I knew how

Pictured is from yesterday’s database, 7 nodes and only one reboot so that’s great

There are still things that I don’t understand, for example there are spots that I can’t place a node in? Like why can I place nodes at the edges of a room but not in the middle? Or why not directly across the wall from a working node? I have one in my office and another in the next room, but I can’t place one in the hallway in between? So weird

[edit] query can be found at https://github.com/GuillaumeRossolini/griotte/blob/main/app/stats1.sql

#DiyAirQualityMonitor #sqlite #arduino