Dev Log: May 24 Wrap-up

Overview

Today was all about making my microcontroller setup a lot smarter and, more importantly, easier to maintain. I finally tackled setting up Over-the-Air (OTA) firmware updates and wired up some decent remote logging so I don't have to keep plugging a USB cable into the board every time I want to see what is going wrong.

What I Worked On

Building a GitHub-powered OTA pipeline

I wanted a clean way to push firmware updates without physically accessing the device. I ended up writing an OTA update routine that checks our real-time database for new version tags and pulls the corresponding compiled binary directly from GitHub Releases.

The logic itself is pretty straightforward using the native Update library, but managing the network state to do this securely took some trial and error. I also patched some annoying Wi-Fi credential handling bugs—specifically making sure the device doesn't panic if a transient disconnect happens versus an actual authentication failure.

Wrestling with ESP32 heap fragmentation

This was the real headache of the day. When I first got the OTA code running alongside our standard database syncing, the device kept randomly crashing during the update cycle.

It turns out I was hitting a massive wall with memory. Each secure TLS session (NetworkClientSecure) grabs about 40 KB of heap. Since the ESP32 only has about 160 KB of usable heap to begin with, spinning up separate clients for database calls, GitHub API checks, and binary downloads was completely exhausting the memory and fragmenting the heap.

I solved this by refactoring the network layer to share a single, reusable secure client across all HTTPS operations. By keeping a single TLS session alive and reusing it, I managed to keep peak memory usage incredibly low and predictable, preventing the dreaded out-of-memory crashes.

Adding remote logging

Once the OTA pipeline was stable, I realized I desperately needed a way to monitor the device's health from afar. I integrated a secure, structured remote logging setup using NetworkClientSecure to stream logs directly to an external dashboard service.

Now, things like boot cycles, Wi-Fi status changes, database commands, and OTA updates are all captured. To keep the memory footprint and binary size as small as possible, I used a little trick: I configured the secure client to skip strict certificate validation (setInsecure()). It’s a trade-off, but for a home hobby device, saving precious RAM and flash storage is absolutely worth it. I also added some compile-time safety guards so the build will fail immediately if I ever forget to define the logging tokens in my local config.

Working with hardware is a great reality check. In web development, we rarely think twice about spinning up another HTTP client or throwing a few extra megabytes at a problem. On a tiny microcontroller, you have to fight for every single kilobyte, and honestly, it makes the victory feel a lot sweeter.

Wrapping Up

The device is now sitting comfortably on version 1.0.4, updating itself seamlessly, and screaming its logs to a dashboard where I can actually see them. Tomorrow, I'm hoping to get back to writing some actual control logic now that the core infrastructure is solid.