Skip to content
ENNO
← Back to blog
·3 min read·

From Docker Compose to Infrastructure as Code with OpenTofu

My homelab runs from two Git repos: OpenTofu for the VMs, DNS and tunnels, and a small deploy script for 15 Compose stacks. How each one works, and the two and a half months one of them was quietly wrong.

From Docker Compose to Infrastructure as Code with OpenTofu

For a long time, deploying something in my homelab meant SSH into the box, git pull, docker compose up, and hope I remembered what else that service needed. There was no record of who changed what, because the answer was always me, at some hour I couldn't remember.

These days nothing gets onto the homelab without going through Git. It's two repos, and they do different jobs.

Diagram of two pipelines. homelab-iac (OpenTofu, 4 stacks, 5 VMs): branch and PR, tofu plan posted as a PR comment, I comment /apply after reading the plan, merge only once apply is green. A daily drift check plans against main. homelab-stacks (Docker Compose, 15 stacks): push to main, deploy.sh deploys only the stacks that changed, Infisical secrets filtered to the vars the compose file references, rsync and health check on the target VM.
Two repos, and both pipelines run on my own box.

The machines: OpenTofu

The first repo describes what exists: the Proxmox VMs, Cloudflare DNS and tunnels, the Zero Trust apps in front of them, the B2 bucket that holds Tofu's own state. I went with OpenTofu, the open-source fork of Terraform. The main reason over Ansible was state. Tofu knows what's already there and works out the difference, which was exactly the part I'd been doing in my head.

It's split into four stacks (foundation, network, compute and scm) that apply in that order and don't read each other's state. Anything one stack needs from another goes through Infisical. A VM is one entry in a map:

hcl
vms = { "infra" = { cores = 8 memory = 12288 disk_size = "150G" pool = "infrastructure" tags = ["docker", "traefik", "tofu-managed"] } # media, dev, agents ... }

and a module loops over it with for_each. Five VMs at the moment, all cloned from the same Debian template with cloud-init.

A change is a PR. A webhook runs tofu plan and posts the output as a comment, so I can read it on my phone. If it looks right I comment /apply, and only once the apply is green does branch protection let me merge. Apply-before-merge sounds backwards, but it means main always matches what's actually running. Nothing on main is a change that hasn't happened yet.

The pipeline runs on my infrastructure VM through a small webhook listener (adnanh/webhook), not GitHub Actions. I didn't want a cloud runner holding the keys to my Proxmox API. There's also a job at six every morning that plans every stack against main and pings Discord if anything has drifted. So far it's been 26 PRs and 13 /apply comments, which is about how often the actual machines change.

The containers: a deploy script

The second repo is what runs on those VMs, 15 Docker Compose stacks. Tofu would be overkill here, so it's plain push-to-deploy. A push to main hits a webhook, and deploy.sh works out which stacks changed and deploys only those. Each stack has a small file saying where it goes:

yaml
host: <vm-address> deploy_path: /opt/stacks/arr health_check: curl -sf http://localhost:8989/ping infisical: - env: dev path: /arr

Secrets were the part that took longest to get right. The script fetches the stack's Infisical folder, then keeps only the variables the compose file actually references:

bash
needed_vars=$(grep -oP '\$\{(\w+)' "$compose_file" | sed 's/\${//' | sort -u) # ... fetch secrets as JSON from Infisical ... if echo "$needed_vars" | grep -qx "$key"; then echo "$line" >> "$env_file" fi

So a stack only gets what it asks for, and the log says how many were kept and how many were filtered out. Nothing sensitive goes in either repo.

Where it went wrong anyway

The pipeline only helps if you use it. My portfolio site moved to a pre-built image from GitHub's registry in March, and I updated the box by hand instead of the stacks repo. A couple of secrets went straight into the .env on the server too. For about two and a half months the repo still described the old build-from-source setup, while the server ran something else.

Nothing broke, which was the problem. The first normal deploy of that stack would have put the old config back, taken the site down and dropped a secret the site needs. I found it in June while wiring up mail for the site's contact form, moved the secrets into Infisical and brought the compose file back in line. Since then the rule is simple. If I'm about to edit a file on a server, the change goes in the repo instead.

The drift check covers the Tofu side. The compose side still relies on me, and that's the bit I'd fix next.

#opentofu#infrastructure-as-code#docker#homelab#devops