Kilimo Hakika Infrastructure
Three years running the servers behind everything a small agribusiness ships
Technical Consultant, Infrastructure and Software Development · Jun 2023 - Jul 2026 · Kilimo Hakika Limited (Contract)
What the contract actually is
Since June 2023 I have run Kilimo Hakika's server infrastructure. Not a project with an end date. The estate that everything else in this portfolio for them sits on top of: the public website, the Food and Climate Tech Network site, and the internal project tracker.
The company has no in-house ops. When something goes down at nine on a Sunday, that is me.
The headline number from their side is roughly 1,400 US dollars a year less than the managed hosting options that were priced against it. That comparison is only fair because it includes my time, and I would rather state it that way than pretend self-hosting is free.
One reverse proxy, several unrelated sites
Multiple applications share a single Caddy instance on one VPS. No application publishes a public port. Caddy is the only thing listening to the internet, reaching each app by container name on an internal Docker network, and issuing and renewing TLS certificates automatically. No certbot, no cron job for renewals, no expired-certificate incidents.
Running several projects through one proxy creates a specific trap, and I walked into it. Docker Compose registers a service's key as a DNS alias on every network it joins. Two projects that each sensibly name their application service app will collide the moment they share a network, and routing breaks for both in a way that looks like neither project's fault. Every service now carries a project-prefixed key, and the reason is written in a comment beside it, because the fix is invisible six months later and looks like pointless verbosity.
A related consequence worth being deliberate about: Caddy loads exactly one configuration file, mounted from one repository. That means one project's repo holds the routing for sites that live in other repos. It is slightly odd, and the alternative is worse, so it is documented rather than tidied away. A git pull in the wrong repo should not be able to silently drop another site's routing.
A network with no public door
The internal project tracker holds the company's commercial pipeline: which tenders they are bidding for, at what value, against what deadline. That does not belong on the public internet behind a login form.
It runs behind a self-hosted Headscale network, the open-source control server for Tailscale. Only enrolled devices can reach it at all. The coordination server is the single publicly resolvable name in that setup; the tracker itself resolves to nothing from outside.
Staff onboarding is self-service on Windows, Mac and Android. iOS and iPadOS need a manual registration step, which is a platform limitation rather than a design choice, and it is in the runbook so it does not get rediscovered each time someone gets a new phone.
Rolling out the access-control policy needed sequencing. Applying restrictive rules before every device carries its tag drops the whole fleet off the network simultaneously, including the machine you are administering from. It went out in three phases: declare tags permissively, tag every node, then tighten to the final policy. One detail cost me real time and is now the loudest warning in that document: sending the reload signal re-reads the policy file but not the main config file, so the first time you point it at a policy path the reload reports success while loading nothing at all.
Certificates on the private network come from an internal certificate authority, so every staff device has to install and trust that root. The runbook covers the trust settings for each platform and includes a check for CA drift.
Backups, and actually restoring one
Nightly compressed Postgres dumps, weekly media snapshots, thirty-day retention, pruning on a schedule.
Backups you have never restored are a hypothesis. I ran a restore into a clean database and confirmed the row counts came back before calling it done. It is the least interesting hour of work on this list and the only one that matters when something goes wrong.
Being straight about the current state: one of the two deployments still writes its backups to the same server it is backing up. The rotation and the restore path work, the off-server copy is not configured yet. That is the next job on this contract and the runbook says so in capital letters rather than implying it is finished.
Deployment, and the trap in the environment file
Deploys are a documented sequence rather than a pipeline. For an estate this size, a CI system would be another thing to maintain and another thing to break.
What does exist is a written cutover checklist, and it leads with the landmine. Media URLs are stored as absolute paths in the database, so changing the public domain fixes future uploads and silently breaks every image already on the site until the existing rows are rewritten too. The exact SQL to do it is in the checklist, because I am not going to remember it and neither is anyone who inherits this.
Second entry on the same list: after editing an environment file, restarting a container is not enough. It reuses the existing environment and the change appears to do nothing. You have to recreate it. That one is worth fifteen minutes of confusion every single time it is forgotten.
Why this is written down at all
Most of the value here is not the configuration. It is that the runbooks exist: the network document, the deploy checklist, the restore procedure, the per-platform onboarding steps, the specific gotchas that cost an afternoon each.
A one-person infrastructure contract is a bus-factor problem by definition. Written procedure is the only thing that turns it into something another engineer could pick up, and it is the part that usually gets skipped.