Homelab Infra: Ephemeral k3s Test Clusters
Homelab Infra
Purpose
Terraform + Ansible + Gitea Actions pipeline meant to spin up an ephemeral, disposable k3s test cluster (Proxmox LXC) on push to an app repo’s test branch, and tear it back down once the job finishes — a throwaway cluster per test run instead of a shared, always-on staging environment.
Layout
1
2
3
4
5
6
terraform/ LXC provisioning (clones from a golden template)
ansible/ k3s install (fork of techno_tim/k3s-ansible, trimmed)
k8s-bootstrap/ Traefik + TLS + Middleware, applied after k3s comes up
.gitea/workflows/
ephemeral-k3s-test.yml reusable workflow, called by app repos
reaper.yml scheduled orphan cleanup (every 30 min)
terraform/ clones 3 privileged LXCs from a pre-built golden Proxmox template (terraform/scripts/build-golden-template.sh). ansible/ then installs k3s across them as 1 master + 2 agents, using a trimmed fork of techno_tim/k3s-ansible. k8s-bootstrap/ applies Traefik, a TLS store, and middleware once the cluster is up. The Gitea Actions side is a reusable workflow (ephemeral-k3s-test.yml) that app repos call on push to a test branch, plus a reaper.yml scheduled job that sweeps any orphaned clusters every 30 minutes as a fallback.
Current status
The core provisioning path has been proven end-to-end by hand against the real Proxmox host: golden template build → terraform apply (3 privileged LXCs clone in ~60s) → ansible-playbook site.yml → a fully healthy 1-master + 2-agent k3s cluster (all nodes Ready, all core + MetalLB pods Running) → terraform destroy with zero orphans left behind.
Getting there surfaced real bugs that a design review wouldn’t have caught — among them, MetalLB’s speaker DaemonSet needing a memberlist Secret that nothing in MetalLB’s own manifest or the Ansible collection creates (agents otherwise hang indefinitely joining the cluster), and kube-vip having to be disabled outright since the collection’s own bootstrap cleanup deletes the VIP manifest before k3s’s manifest controller applies it — not needed anyway with only one master.
Not yet done: k8s-bootstrap/ hasn’t been applied against a real cluster yet, the dedicated Gitea Actions runner isn’t provisioned, and neither ephemeral-k3s-test.yml nor reaper.yml has been triggered — not even via a manual workflow_dispatch. This is infrastructure that’s been validated manually through full cluster bring-up and teardown; the CI wiring that would make it trigger automatically on a git push is still in progress.