Two changes to the control server, plus the fallout on the hosts. OIDC via Authentik, so `tailscale up --login-server ...` opens a browser instead of needing a pre-auth key. This is a second Authentik application, separate from headplane's, with headscale's own /oidc/callback redirect. Headless hosts keep using pre-auth keys. Note that headscale runs OIDC discovery at startup and a failure is FATAL -- pointing `issuer` at an application that does not exist yet means the control server will not boot, so verify the discovery document before deploying. override_local_dns = false, because the upstream default of true replaces resolv.conf with 100.100.100.100 on every node. That silently broke the LAN's `.sol` names -- pihole serves those and the global nameservers return NXDOMAIN for them -- and took ad blocking down with them. It also made each node's entire DNS depend on tailscaled, which is what had forced --accept-dns=false onto neptun and mercury individually; both of those workarounds are now removed, and with MagicDNS resolving properly again neptun no longer needs its hardcoded /etc/hosts pin for jupiter. Also serves jellyfin and seerr from jupiter, matching the ports they already use on its LAN vhosts, and rotates the tailnet pre-auth keys. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
homelab
Flake-based NixOS config. Hosts: jupiter (ZimaBlade, NAS + services),
neptun (netcup VPS: public reverse proxy, Authentik, headscale),
mercury (Raspberry Pi 3B+, DNS/DHCP), terra (desktop).
Structure
flake.nix # inputs + nixosConfigurations (jupiter, neptun, kexec, ...)
common.nix # shared base: user, ssh, nix, firewall, timezone
services/ # one reusable module per service, by category
media/ jellyfin, audiobookshelf, the *arrs, sabnzbd, seerr, ...
network/ caddy, samba, avahi, pihole, unbound
vpn/ tailscale, headscale (control server), headplane (its web UI)
identity/ authentik (OIDC provider, from the authentik-nix flake)
dev/ gitea
desktop/ hyprland
containers.nix # podman backend, shared across categories
hosts/
jupiter/ # ZimaBlade NAS
configuration.nix # host bits + imports common + the services it runs
disk-config.nix # disko: eMMC partitions
hardware-configuration.nix
secrets.nix # sops-nix wiring
vm.nix # VirtualBox test image (jupiter-vbox)
neptun/ # netcup public reverse proxy + tailnet node
configuration.nix disk-config.nix hardware-configuration.nix secrets.nix
secrets/ # age-encrypted sops files, one per host
scripts/ # deploy, edit_secrets
Hosts compose by importing common.nix + whichever services/* modules they
run. Each service module opens its own firewall ports.
Test in VirtualBox (no hardware needed)
nix build .#nixosConfigurations.jupiter-vbox.config.system.build.virtualBoxOVA
VBoxManage import result/*.ova --vsys 0 --vmname jupiter-vbox
VBoxManage startvm jupiter-vbox --type headless
Login darman / test. Forward ports with VBoxManage modifyvm ... --natpf1.
First install on the ZimaBlade — nixos-anywhere + disko
Wipes the OS disk and installs the flake over SSH. No USB needed if the box already runs Linux (ZimaOS) reachable by root SSH — nixos-anywhere kexecs into an installer, partitions via disko, installs.
⚠️ The OS disk in
disk-config.nixis WIPED. Setdeviceto the OS disk ONLY (by-id). Back up / physically identify the NAS data disk first — it must NOT appear in disko.lsblk -o NAME,SERIAL,SIZE,MODELto identify.
- Set the real OS disk id in
hosts/jupiter/disk-config.nix(ls -l /dev/disk/by-id), and the data-disk mount inconfiguration.nix. - Add your login SSH pubkey to
users.users.darman.openssh.authorizedKeys.keys. - Set the real samba password:
export SOPS_AGE_KEY_FILE=~/.config/sops/age/keys.txt nix shell nixpkgs#sops -c sops secrets/jupiter.yaml # edit, commit - Stage the pre-generated host key so sops can decrypt on boot #1
(private key lives off-repo in
~/.config/homelab/jupiter/):install -Dm600 ~/.config/homelab/jupiter/ssh_host_ed25519_key \ /tmp/extra/etc/ssh/ssh_host_ed25519_key install -Dm644 ~/.config/homelab/jupiter/ssh_host_ed25519_key.pub \ /tmp/extra/etc/ssh/ssh_host_ed25519_key.pub - Run from your laptop:
nix run github:nix-community/nixos-anywhere -- \ --flake .#jupiter \ --extra-files /tmp/extra \ --generate-hardware-config nixos-generate-config ./hosts/jupiter/hardware-configuration.nix \ --target-host root@<zimablade-ip>--extra-filesplants the host key before first boot (its age identity is already a recipient in.sops.yaml, so/run/secrets/samba_passworddecrypts on boot #1).--generate-hardware-configpulls the target's real kernel modules into the placeholder. Commit the result. Reboot into NixOS.
Manual alternative (USB ISO): boot installer, disko the disk, then
nixos-install --flake .#jupiter.
Deploy (the ./deploy wrapper)
All arguments mandatory — no default host, no default config.
./deploy kexec <host> # headless kexec into a RAM installer (RO-root box)
./deploy install <config> <host> # first install; wipes OS disk, ships host key
./deploy switch <config> <host> # rebuild + activate on a running host
./deploy boot|test <config> <host> # stage for next boot / activate without boot entry
./deploy image <config> # build an SD-card image (mercury)
./deploy flash <config> <dev> # build SD image, write it, drop the sops age key
switch/boot/test prompt for darman's password (wheelNeedsPassword).
<config> is a nixosConfigurations name (jupiter, neptun). Its pre-generated
SSH host key lives at ~/.config/homelab/<config>/ssh_host_ed25519_key.
Examples:
./deploy switch jupiter jupiter.sol
./deploy install neptun 159.195.64.117
Rollback: nixos-rebuild switch --rollback on the host, or pick a prior
generation at boot.
Post-deploy steps (per host)
Things the flake cannot do for you. Skipping these leaves a host that builds and boots but doesn't work.
Every host, immediately after a first install
ssh darman@<host> sudo -v # DO NOT SKIP
users.mutableUsers is true, so /etc/shadow is written once, when the
user is created. If the sops secret wasn't readable at that moment the account
gets ! (locked) permanently — deploy switch will never fix it, because the
activation script only sets a password for users not already in /etc/shadow.
Combined with wheelNeedsPassword = true and PermitRootLogin = "no" that
means no way to escalate, and recovery is physical: netcup's rescue system for
neptun, or pulling the SD card for mercury. Verify sudo while you still have
another way in.
neptun (netcup VPS)
- Edge firewall. In netcup's panel, inbound
ACCEPTfor TCP 22/80/443/2222 and a rule accepting inbound UDP. The firewall is stateless: without the UDP rule every DNS and NTP reply is dropped, and nothing on the box reports an error — it looks like headscale crash-looping on its DERP fetch and Caddy failing ACME.grep -A1 '^Udp:' /proc/net/snmpshowingInDatagrams 0is the tell. Rules apply on VM restart, not on save. This is safe:nixos-fwis stateful and default-deny, so it remains the real policy. Also open UDP 3478 (STUN) and 41641 (tailscale direct). - Authentik creates
akadminon first start; log in athttps://auth.mgaction.townwithauthentik_bootstrap_passwordfrom sops. The username is hardcoded upstream and the bootstrap runs once — later changes to the env vars are ignored. - Bootstrap the tailnet (headscale starts with an empty database):
Put that key in every host's sops file as
sudo headscale users create darman sudo headscale preauthkeys create --user darman --reusable --expiration 24htailscale_authkeyand rebuild. - Headplane API key — defaults to 90d, after which headplane silently stops
listing nodes:
sudo headscale apikeys create --expiration 999d # -> headplane_headscale_api_key - Headplane OIDC. In Authentik create an OAuth2/OpenID provider
(confidential, redirect
https://vpn.mgaction.town/admin/oidc/callback, a signing key must be selected or discovery exposes no JWKS) and an application with slugheadplane— the slug is what makes the issuer.../application/o/headplane/inservices/vpn/headplane.nix. Client ID goes in that file, client secret into sops. - Pin jupiter's tailnet address.
networking.hostsinhosts/neptun/configuration.nixcarries jupiter's IP, because neptun runs--accept-dns=falseand cannot use MagicDNS. Until it matchesheadscale nodes list, theabs.andgit.vhosts fail to proxy.
jupiter
chown -R gitea:gitea /mnt/data/AppData/giteaafter the first deploy (the repos were copied in over CIFS asdarman:users).- Re-enrolling after the headscale database was recreated:
tailscaledkeeps its old node key and reportsRunning, and the autoconnect unit exits early on that state without ever sending the new pre-auth key. Force it:sudo tailscale logout && sudo systemctl restart tailscaled-autoconnect
mercury (Raspberry Pi 3B+)
./deploy flash mercury /dev/sdXwrites the dedicated age key to the root partition. Without~/.config/homelab/mercury/age.txtit silently skips that step and no secret decrypts on the box — checkls /run/secretsafter first boot.- It boots from an SD card, so config changes are
./deploy switch mercury <ip>(an aarch64 build — needsextra-platforms+ binfmt on the laptop, see the gotchas inCLAUDE.md) rather than a reflash. - Suspect the card first when binaries crash with
Illegal instructionor services fail inexplicably. Failing flash returns corrupt data with no I/O errors indmesg:A card that has corrupted one path will corrupt more. Replace it and reflash; only pihole's runtime state (query history, dynamic leases) is lost — the static leases are declarative.sudo nix-store --verify --check-contents # add --repair to fix
Adding a service
Copy the whoami block in oci-containers.containers, swap image/ports/volumes.
Native NixOS module exists for many apps (Nextcloud, Jellyfin, Grafana...) —
prefer services.<app> over a container when available. Add a caddy
virtualHosts block to expose it.
Notes
- Backend is Podman with
dockerCompat—dockerCLI works, no daemon. - Samba keeps its own password DB.
services.sambanever sets it; a systemd oneshot (samba-smbpasswd) provisions it. Host reads the password from/run/secrets/samba_password(sops-nix); the VM falls back to plaintext/etc/samba/smb-password. - Secrets:
secrets/jupiter.yamlis age-encrypted (safe to commit) to two recipients in.sops.yaml— the admin key (edit on laptop,~/.config/sops/age/keys.txt) and the jupiter host key (derived from its SSH host key viassh-to-age, decrypts at runtime). Private keys live off-repo and are gitignored. Rotate/add recipients withsops updatekeys. - Data disk: plain
fileSystems."/mnt/data"in configuration.nix — kept out of disko so it is never formatted. Reference byby-id/by-uuid. system.stateVersion=26.05, install-time schema. Do NOT bump on upgrades.- Terraform is not used: a single bare-metal box has no provider API. disko + nixos-anywhere cover provisioning natively.