Skip to main content

Labyrinth

lab.yml describes a two-node Proxmox cluster. One command builds its containers, routing, SSO, firewall rules and dashboards.

  • OpenTofu + Ansible
  • Infrastructure as Code
  • Solo Project
just apply
$ just apply
generate   tfvars · inventory · traefik · authentik · prometheus
tofu       containers, DNS records, firewall rules, tailnet enrolment
ansible    mounts, packages, services
lab converged

Machines rebuilt from version control

I used to configure containers by hand. Deploying meant remembering which machine had spare cores, which reverse-proxy config needed a new hostname, and which firewall rule I had opened and forgotten. Labyrinth describes the machines in version control and rebuilds them without patching by hand. It has roughly 7,300 lines of OpenTofu, Ansible and Python. CI validates every change to the description before it reaches the hardware.

How it works

There are three things I edit: lab.yml, which says where things run; a directory per stack, which says what a thing is; and an encrypted vault for secrets. Everything downstream is generated: Terraform variables, the Ansible inventory, Traefik routes, Authentik blueprints, DNS records, firewall rules and Prometheus scrape targets. A hand edit to any of those means the generator has a bug.

OpenTofu makes containers exist and owns the state that lives off the cluster; Ansible takes them from empty container to running service. The split is deliberate, and it is the reason a node can be wiped and come back the same.

yaml
# lab.yml — the single source of truth
cluster:
  name: void

nodes:
  leviathan:          # HP DL380 G9 — bulk storage, most stacks
    datastore: local-lvm
  erebus:             # consumer build with a GPU — media and hot tier
    datastore: local-zfs

hosts:
  ingress:
    node: leviathan
    stack: ingress
    cores: 2
    memory: 2048
  jellyfin:
    node: erebus
    stack: jellyfin
    cores: 4
    memory: 8192
    gpu: true

The build generates the topology view from lab.yml, so it stays in step with what is deployed.

The generated topology view: a service map showing how stacks relate, with routes through ingress, SSO via authentik, VPN egress and cross-stack dependencies, each host panel listing its stack and routes.

Six groups of services

Each stack has one directory, one service and one container. Its metadata defines routing, SSO, firewall openings and storage mounts. Adding a service means adding a folder instead of editing six files. Thirteen stacks are running today in six groups.

  • Ingress

    Traefik sits behind CrowdSec. Every cross-host route terminates at Traefik and is generated from the hostname declared by its stack.

  • Identity

    Authentik provides single sign-on through forward auth on the routes. Stack metadata generates its providers and applications.

  • Monitoring

    Prometheus, Grafana and Loki use scrape targets from the generated host inventory. New containers are monitored as soon as they exist.

  • VPN gateway

    Downloaders can reach the internet only through the VPN gateway. If the tunnel goes down, their traffic stops.

  • Media

    Jellyfin is installed natively to use the GPU for transcoding. Everyone else in the house makes requests through Jellyseerr.

  • Fleet tooling

    Portainer shows logs and stats. apt-cacher-ng caches package updates for thirteen containers, and every build regenerates the docs site from lab.yml.

What it runs on

The cluster has two very different machines: a decade-old rack server with a lot of disks and a consumer desktop with a GPU. lab.yml assigns each service to a node, making that mismatch a scheduling detail I do not have to remember.

Hardware

ClusterTwo-node Proxmox VE cluster, LXC-first
Nodes
  • HP DL380 G9: bulk storage and most stacks
  • Consumer build with an RTX 2070: media and the hot tier
StorageZFS: raidz2 across the server’s SAS disks for bulk, NVMe pools for hot data
Network
  • UniFi Dream Machine, VLAN-segmented
  • Tailscale for remote access, with tag-based ACLs
ToolchainOpenTofu, Ansible, Python generators, CI on every change

Worth stealing the pattern

The repository is private. lab.yml describes the specific hardware in my house, so it would need changes to deploy elsewhere. The overall structure applies elsewhere, and I am happy to walk through the generator, the stack contract and the parts that resisted being declarative.

Ask me about it