Replication Is Not a Backup: Designing Storage by Recovery Intent

Three copies of a corrupted database are three corrupted databases. Replication protects availability when hardware fails; it does not protect against bad writes, accidental deletion, a compromised administrator, or a faulty migration.

I organize homelab storage around recovery intent instead of application names. Every data set should answer five questions: Can it be rebuilt? How current must recovery be? How long can restoration take? What independent copy exists? How will I prove the restore worked?

Tier one: desired state in Git

Kubernetes manifests, scripts, generated dashboards, alert rules, and runbooks belong in version control. They are reviewed, reproducible, and safe to distribute within the private repository.

Secrets, private keys, kubeconfigs, databases, mail, uploaded files, and media do not belong there. Git is a recovery source for configuration, not a convenient bucket for everything important.

Tier two: replicated application state

Longhorn replicates critical volumes across cluster nodes. CloudNativePG provides database-native high availability for PostgreSQL services. This tier keeps applications online through a node or disk failure and supports controlled maintenance.

Replication settings are chosen by importance. A replaceable model cache should not consume the same protection as a password vault or business database.

Tier three: bulk data

Large personal files and media use a separate NAS-backed tier. Application metadata remains independent so the database can be backed up, verified, and restored without copying every large object in the same operation.

This separation also makes failure modes clearer. A disconnected bulk share should not corrupt an application database, and a database restore should not rewrite unchanged originals.

Tier four: independent backups

Backups are written as completed archives rather than thousands of small live files. Sensitive sets are encrypted before they leave the host. Backup age, size, and success are monitored.

The backup destination is not treated as infallible. Retained migration artifacts and cold copies remain until the replacement has survived normal operations and a deliberate restore exercise.

Tier five: disposable caches

Downloaded models, thumbnails, transcodes, indexes, and other derivatives can often be recreated. Labeling them as disposable avoids expensive backup jobs and makes rebuild procedures honest.

Disposable does not mean unmonitored. A cache that fills a disk can still take down a service, so capacity and time-to-full metrics matter.

Verification closes the loop

A successful backup command is evidence that bytes were written, not that the application can recover. Restore procedures check the restored service, record counts, file manifests, permissions, and—when appropriate—cryptographic hashes.

This model has made storage decisions much easier. Instead of asking “where should this volume go?” I ask “what failure am I protecting against, and what evidence will prove recovery?”