Since this one works, I ordered a few more sensors and microcontrollers from the same manufacturers and I soldered them

I now have a bunch of these little devices around the house, and I am very happy with how they extend the range

As long as I place enough of them along the way, I can sprinkle them wherever I want and they’ll self organize, and one of them reports all their readings to an SQLite database on a RaspberryPi Zero

They are on their own WiFi, which doesn’t matter for security since I’ve written their program myself, but still feels great

Haven’t done statistics and graphs yet

#electronics #AirQuality #DIY

Latest news is that I got my brain into embedded thinking mode

I changed the way I determine if a sensor reading is worthy of archiving

The way I set up my probes, each #ESP8266 send its readings as fast as it can (that’s every 3s) to the entire mesh and the root node, the #ESP32, forwards them across WiFi networks over HTTP to my #RaspberryPi Zero W

The rpi0 then decides what readings to keep and what to toss

The naïve strat

The way I was deciding what to toss was, until yesterday, an SQL query doing a GROUP BY on the node sending the readings and checking the MAX date for its readings

The database file is already several dozen MB in size and growing by almost that much daily (I’m stress testing my system),

Even with a proper SQL index, the SDCard has to read a lot of information on every incoming sensor reading (SQLite is a file based system, there is nothing in RAM), just to decide what to keep and what to toss. That translates into lots of unnecessary i/o of all kinds, faster SDCard wear, and general latency unpleasantness

The rpi0 CPU was 40% to 90% all the time, according to htop

I’m way too used to PostgreSQL…

The embedded-friendly strat

My new strat is simply creating a file on disk for each sensor, changing its modified timestamp and using that to determine when its readings were archived last

That’s just a handful of empty files on disk and I’m only changing their metadata, not their content. They’re using next to nothing in terms of space. Checking their metadata is also really fast, and possibly cached by the OS and/or my web server.

Now every HTTP request fits within 4ms if the previous reading for the same sensor was recent, or at most 80ms when the database gets a write

The rpi0 CPU is now back to low levels, between 7% and 12% most of the time

#electronics #diy

This morning my ISP had an outage and I restarted their router, which happens to power my rpi0 with the web server for this project

And I forgot to restart that web server, so I lost all data from today

But before that, it was performing beautifully. There were 844 records for each node since last night, like clockwork. I happen to have cleared my previous data last night, so all the nodes were running and already on a stable mesh at the time.

Each sensor has been broadcasting its readings every three seconds, at which point my server decides to ignore most readings until the 1 minute mark (per sensor) and I’m happy that not one minute was lost for half a day of recording

I am less happy that after the web server was restarted, none of the microcontrollers found it by themselves. It looks like the entire mesh collapsed because the root node couldn’t bridge both networks again.

I understand that the root node didn’t restart the bridge between both WiFi (I didn’t set it up that way), but I would have thought that the iot mesh should have continued independently? Perhaps because it is declared as the root node? I’ll have to test this more.

I reset the ESP32 root node and it picked up the home WiFi fine, but none of the probes connected to it again (this mesh should have self healed within minutes, at most)

I went around the house to reset each ESP8266 node and yes, that is when the mesh self organized again and I started seeing the web server collecting readings again, as their BME680 sensor finished its self calibration routine

#meshnetwork #AirQuality #esp32 #esp8266 #bme680 #raspberrypi

I’m very pleased with the general responsiveness of my little project

Pictured in the attached video is the bridge #esp32 node, the one that waits for data from the #mesh and forwards it to the storage device

The blinks represent data being forwarded from one WiFi to the other

Seeing 6 blinks within 3s means that all the probes have called home in time before their next sensor reading

Today I soldered a few more of my custom air quality thingies, flashed them with the exact same program as the other ones, gave them a USB adapter and plugged them in their own spot in the house

They automatically found their path in the mesh, then they started sharing and recording their readings, which automatically appeared in my dashboard, and all I had to do was rename them there

In the picture, the number in the parentheses is the number of readings for the day (more or less one per minute) and all bars and lines represent average readings for the day; I have one graph per room to get the timelines

Humidity and temperature are on the left axis, all the other are on the right axis

(Looks like the one in the bedroom crashed again)

The one in the attic is missing however long it took me to solder the new things, since that’s the one I unplugged to use as a template

#electronics #bme680 #esp8266

Attached is a video of the #esp32 happily blinking its payloads across WiFi networks to the #RaspberryPi Zero, for storage in the #sqlite database

Thousands of readings per day (about 1300 each sensor so far, today), all sensors being within 15 readings of each other, I’d call that stable

I’m happy to notice that when a board fails, it eventually recovers on its own (it takes several hours, though)

This weekend I finally caved and used #ChatGPT [1] to improve a program I’m working on [2]

The project

To recap the above thread this is a part of, I’m trying to build a mesh network of microcontrollers (ESP8266), each with an air quality sensor and they broadcast their readings so that a different microcontroller (ESP32) can forward them to a Raspberry Pi Zero for storage, statistics, visualization etc.

I have about a dozen of nodes and the network has started to fall apart (it was fine when I had 6), so I’m trying to find workarounds but I am unfamiliar with C++. This is where ChatGPT comes in.

The experience wasn’t smooth. It felt a little like chatting with a junior dev, fresh out of school and full of ideas about how a happy path algorithm should be written or how an ideal library should interface, but less concerned with the code in front of them.

How it started

I started by sharing the public URL for the .ino file (it’s an Arduino IDE project) on my GitHub and it looks like the bot used it as a search rather than a link, and it scanned the top results but didn’t find my code. It told me so in so many words. I had to share the link to the raw code instead. No biggie but a bit underwhelming.

(As an aside, I find it funny that the bot’s default search engine, Bing, was unable to locate a file from a public repository on GitHub based on its full URL. Both of these services are operated by Microsoft.)

Then, having just seen the code and after I expressed my desire for a code review, the bot proceeded to explain to me generalities of programming and best practice. Meh.

First code review

After a third prompt from me, it finally read my code but still provided several comments about things that aren’t in my program. I don’t use a display anywhere, so why the advice about OLED? (I know why, that was in one of the results in its first search, the one where it told me it hadn’t found my code). Of course I am indeed using the data with an output, mine being the mesh rather than a screen, but… Meh.

Out of the six initial suggestions, only the one about the macro was useful; although with the amount of memory at my disposal, that particular optimization wouldn’t matter.

Of the other recommendations, none applied to my program and the bot kept suggesting their suboptimal variant as fixes later on, contradicting itself numerous times. Every time I corrected its mistake, it praised me for my insight and gave my a bullet list of pros and cons, which felt a little patronizing, and then it proceeded to make the same mistake a few prompts later. And again later on.

General impressions

The bot failed to refactor my code at any point, instead giving me basic unfinished examples modified to fit the current recommendation. One of the things I was worried about was my use of the JSON library and how strings are manipulated. After all, this is C++ and with a headless device that won’t tell me anything when it crashes. I tried a bunch of times to get help from the bot, failed and moved on.

Every time I challenged one of the bot’s suggestions, it came back to me with an apology and that after further review my code didn’t need that particular change or with a generic phrase to not even try giving me a firm answer.

The one suggestion that looked like it could improve things, about deep sleep, ended up being a non starter because that would power down the sensor and I would have to restart the calibration phase on wake-up (that’s about 5 minutes for this sensor). The bot didn’t know to check what consequences the change would have.

Mixing software and hardware

This part was interesting. I was challenging the bot’s assumptions about deep sleep and how this specific sensor would behave in that case, and I told the bot to go read the code of the library for the sensor. I had to prompt this twice because the first time it came back with guesses. It looks like it did actually search for it, scanned the top results and came back with apparently more insight into the inner workings of the library. It didn’t find the authoritative source that I was using, but that’s a tough ask even for a human so I won’t hold this against it (it did manage to find two reasonable sources). Not meh and actually a decent attempt.

What the bot told me it found, was validating my existing bias so I didn’t keep pushing. I also didn’t go and verify by myself.

Trying variations on earlier prompts

I tried again to get the bot to review my code for errors. As was becoming the norm, at my first prompt I got only generic answers; when prompted again the bot had apparently forgotten all about my code; and when I reminded it that it did have access to my code, then it finally obliged. Its answer was predictably full of extensive praise about each thing that I did correctly (in fact I was surprised that there were no errors, but given that, the bot’s response was predictable). I’d have been fine with a short list of common errors it checked for and didn’t find, but ok.

Then I tried to have the bot refactor my code in a few ways. I wanted to see if it could suggest code to send the data over the mesh in one format while writing on Serial in another format. I already knew it was bad at rewriting existing code.

That was a fiesta of all the advice it had given me this far, and it applied the bad variant every time. It knew by then of my preference for the “guard” style but ignored it; it made several copies of each scalar; it didn’t rewrite my code at all, instead using some made up example; it kept using functions that it had warned me against.

Spite

I tried trolling the bot by suggesting deep sleep again, but it can’t do sarcasm and it tried its best to give me points any time it could, so it didn’t contradict me and praised me again for my insight.

I also tried to see if a pointed question with no actual consequence would get flagged but alas, the bot helpfully (not!) provided a pro/con list to help me decide for myself, not realizing how futile the decision was. This one wasn’t about sarcasm, it was about comparing two identical options and telling me this in no uncertain terms.

Brainstorming

This is how I generally find this kind of bot useful: tossing around ideas, having it formulate them in several ways to help me solidify my own understanding of them.

I still tried to get it to suggest code with the libraries I am using, but I stopped insisting on refactoring my actual code. I also gave up on fine tuning the code according to the code standards established earlier in the conversation. At this point I have zero faith in their Memories product before even trying it 😂

(Perhaps an error I made here was in not telling the bot to forget the first search results)

The bot did make some assertions that look like they can hold water and I’ll have to verify them.

At one point I failed to see a correct expression in the suggested code, and when I told the bot that it was missing that expression, it didn’t call me out on it. I had to read the new answer and figure out that it was the exact same suggestion a second time. That’s annoying.

Getting an idea that might be good

I eventually saw that when trying to reconnect to the WiFi, with gradual increments on the delay inbetween attempts, eventually we get to a delay that is so long that perhaps the microcontroller staying awake doesn’t make sense.

For example when I’m doing maintenance on the Raspberry Pi or on the root node, none of the sensor readings matter for a while. They can try reconnecting a few times on signal failure in case it’s just a blip or a reboot, but if they fail for too long, they should feel free to go in power saving mode and try again much later.

So I gave the bot a basic description of this algorithm and it provided code that we iterated on for a bit. At one point I gave up trying with words and I pasted code myself, which earned me deep praise again and a line by line explanation of how great my code was. Of course the bot was eager to improve on this anyway, but logic isn’t its forte…

Parting thoughts

Now I need to iterate on this on my own, since in my case the nodes with a sensor have no concept of my home WiFi and they can’t reach the rpi0, only the root node can do that. So that’s the only microcontroller that would save power in the event that my home WiFi went down. Big improvement! /s

As for the bot itself, while it got better at conversing since earlier versions, and it learned how to search the web, fetch content and parse it really fast, and it can produce code that sometimes works, it’s still much less useful than a coding partner of any amount of experience.

I can’t gauge its trustworthiness or reliability, even when it’s sharing the links that it scanned, without reading through the materials myself.

It still can’t keep its facts straight, although it has access to API documentation and/or actual source code, and it rarely knows when it doesn’t have enough information.

[1] https://chatgpt.com/share/670d8875-dc8c-8007-97b1-9938b82ed838
[2] https://github.com/GuillaumeRossolini/griotte