This week was about the parts of a system nobody sees until they break. The bigger piece of work was a resilience fix in camera-s3-stream, a shell-based upload pipeline I run on my own server, where the upload loop had been failing without a trace. Alongside it, a small but overdue maintenance release for CivoCloudManager. Both are the kind of work that decides whether a system still behaves six months after launch.
Silent failures: the camera upload loop
This is a shell-based system that streams footage from IP cameras to S3 via rclone, with a dead-man’s-switch for tamper detection. I noticed that the upload loop was occasionally stopping without logging anything useful. The cause was a combination of two things: a full tmpfs partition causing mv to fail silently, and the discovery cache returning stale camera entries after a forced re-scan because I never invalidated it. The upload loop now catches transient I/O errors and retries instead of dying quietly, and the discovery script flushes the disk cache when called with --force. Neither fix is complicated. What bothered me is that this had been running in production, on my actual home server, with silent failures for longer than I want to admit. A monitoring gap I should have closed sooner.
CivoCloudManager 2.1.1: maintenance, not features
One commit: sync updated legal documents from the legal repo, add a Support tab to the settings panel, bump the build to 2.1.1. The Support tab was the only thing users had written in about recently: there was no obvious way to reach me from inside the app. It is a small quality-of-life gap, and I should have caught it at launch.
What This Looks Like From the Outside
Reliability fixes and maintenance releases are not the kinds of things that make for impressive screenshots. But they are exactly what a well-built product looks like under the hood. If you are considering having an app built and want to know what “done properly” means in practice, the services page has the longer answer. The short answer is: the parts no one sees are usually the ones that matter most six months later.
CivoCloudManager is open source if you want to see how a zero-dependency Swift 6 macOS app is structured: github.com/marcelrgberger/civo-cloud-manager.