Two changes to the control server, plus the fallout on the hosts.
OIDC via Authentik, so `tailscale up --login-server ...` opens a browser
instead of needing a pre-auth key. This is a second Authentik application,
separate from headplane's, with headscale's own /oidc/callback redirect.
Headless hosts keep using pre-auth keys. Note that headscale runs OIDC
discovery at startup and a failure is FATAL -- pointing `issuer` at an
application that does not exist yet means the control server will not
boot, so verify the discovery document before deploying.
override_local_dns = false, because the upstream default of true replaces
resolv.conf with 100.100.100.100 on every node. That silently broke the
LAN's `.sol` names -- pihole serves those and the global nameservers
return NXDOMAIN for them -- and took ad blocking down with them. It also
made each node's entire DNS depend on tailscaled, which is what had
forced --accept-dns=false onto neptun and mercury individually; both of
those workarounds are now removed, and with MagicDNS resolving properly
again neptun no longer needs its hardcoded /etc/hosts pin for jupiter.
Also serves jellyfin and seerr from jupiter, matching the ports they
already use on its LAN vhosts, and rotates the tailnet pre-auth keys.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
The Authentik provider and application now exist (slug "headplane", which
is what makes the issuer .../application/o/headplane/), so the client ID
is a real value rather than a placeholder, and the client secret and
headscale API key are in sops.
The tailscale pre-auth keys for neptun and jupiter are rotated because
the tailnet was recreated from scratch: the old headscale database went
with the VPS's OS disk, so every key issued against it is meaningless to
the new control server.
Note the headscale API key defaults to a 90d expiry. When it lapses
headplane stops listing nodes with no obvious cause -- `headscale apikeys
list` shows the date.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
By default headscale fetches https://controlplane.tailscale.com/derpmap/default
at startup and treats failure as fatal, so it cannot boot when that URL is
unreachable. A self-hosted control plane that will not start without
Tailscale's infrastructure rather misses the point of self-hosting -- and
it crash-looped for exactly that reason while neptun had no DNS.
Enable the embedded DERP server on region 999 and drop the upstream map.
The relay rides Caddy on :443, which is why that vhost already sets
flush_interval -1; only STUN needs a port of its own.
Verified against headscale 0.28.0 before committing: it starts clean with
urls = [], registers "DERP region: {RegionID:999 ...}" pointing at
vpn.mgaction.town with DERPPort 443, and brings up STUN.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Probing the live Debian VPS turned up three mismatches between what it
serves and what this config declares:
- git.mgaction.town had no vhost at all. Gitea's web UI and HTTPS clones
are public today; only its SSH side (the :2222 socat forward) had been
ported, so a deploy would have taken the web side offline.
- Audiobookshelf is served as abs.mgaction.town, not the longer
audiobookshelf.mgaction.town this config used. The mobile app is
configured with the short name.
- The apex returns 200 from Caddy. Left unserved deliberately, so it now
gets Caddy's default 404; noted in a comment so it doesn't look like an
oversight next time.
Gitea's ROOT_URL was http:// while Caddy terminates TLS for that name.
Gitea builds absolute URLs from it, so clone buttons, redirects and
webhooks were handing out downgraded links.
Also record that defaultGateway6 is confirmed rather than assumed --
`ip -6 route show default` on the VPS gives "default via fe80::1 dev
eth0 metric 1024 onlink".
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
nixpkgs only carries Zitadel 2.71, which predates the login-v2 split and
cannot take a v3/v4 database (its migrations are forward-only), so the
instance running on the old Debian VPS could never have moved onto it.
authentik-nix ships 2026.5.4 and tracks upstream closely.
The authentik-nix input deliberately does not follow our nixpkgs, per
upstream's warning that overriding it breaks their pinned python
dependency set. That costs a second nixpkgs in the lock, so add
nix-community's Cachix to common.nix -- without it the closure is ~400
local derivations (npm, rust, python). The laptop that runs
scripts/deploy needs the same two lines in /etc/nix/nix.custom.conf.
Authentik's own module creates the database and orders its units against
postgresql.target, and recent versions need no redis, so the wiring is
just the module plus a secret. Pin postgresql explicitly so that editing
system.stateVersion can never silently demand a pg_upgrade of the
identity store.
Secret ownership is not uniform and the difference matters: authentik
and caddy take a systemd EnvironmentFile, which PID 1 reads as root
before dropping privileges, so root:root 0400 is correct. Headplane
opens its secret paths itself while already running as the headscale
user, so those three need an explicit owner or they fail to start.
Also on neptun:
- Pass Caddy's ACME account email through the same EnvironmentFile
mechanism and reference it with the Caddyfile {$VAR} placeholder.
services.caddy.email would render the address into the world-readable
store.
- Stop accepting MagicDNS from our own control server. headscale pushes
override_local_dns, so joining the tailnet would point neptun's
resolv.conf at a MagicDNS served by the tailscaled neptun itself hosts
-- a tailscaled failure would then also take out DNS, ACME renewal and
finally the certs for the control server every other node needs in
order to recover.
- Give headplane a writable DNS extra-records file. Its view of
headscale's config stays read-only, which is the right outcome for a
declarative box; records are data rather than config.
- Require a password for sudo. Deploys become interactive, but darman's
key is otherwise the only thing between the public internet and root.
- Enable zram (8 GB, and disko leaves no room for a swap device), and let
tailscaled-autoconnect retry instead of failing permanently when the
control server isn't up yet on a first boot.
networking.hosts still carries a PLACEHOLDER address for jupiter --
replace it from `headscale nodes list` once jupiter first enrols.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
Group service modules by category (media, network, vpn, identity,
dev, desktop) to make the growing services/ dir easier to navigate.
containers.nix stays at the top level since it's a shared backend,
not a single-category service.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Shorten verbose multi-paragraph comments to essentials, and drop a
stale claim in common.nix that jupiter kept its own copy of the base
config (it now imports common.nix directly).
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Path-route Headplane under /admin on the same vhost as headscale instead of
its own subdomain - Caddy handle blocks split on the prefix, headscale gets
everything else. base_url drops to the site root since Headplane appends
/admin (and the OIDC callback path) itself.
Wire Zitadel as the OIDC provider. client_id/client_secret/the headscale
API key can't be real until Zitadel and headscale are actually deployed and
an application/key exist, so those are REPLACE_ME placeholders for now
(documented in services/headplane.nix) - direct API-key login stays enabled
as a fallback so this can't lock anyone out in the meantime.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Headscale is the tailnet control server every host's services/tailscale.nix
already points at (--login-server=https://vpn.mgaction.town). MagicDNS
base_domain "hosts.mgaction.town" matches the "jupiter.hosts.mgaction.town"
names already used in this repo's Caddy vhosts.
Headplane is its web UI, running as headscale's own user (native process
integration, no container). No OIDC wired up - log in with a headscale API
key generated on the box. Both proxied through Caddy; headscale's vhost
needs flush_interval -1 since its node-update endpoint is a long-poll.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Local Postgres, peer-authed over the unix socket (the "zitadel" role is
granted createdb+createrole and doubles as both the runtime and bootstrap
DB user - no password anywhere). TLS terminates at Caddy; Zitadel listens
on localhost:8080 and is proxied at auth.mgaction.town.
Master key and admin bootstrap password come from sops - the admin
password specifically needs the sops.templates -> rendered-file route
(services.zitadel.steps would leak it into the world-readable Nix store),
same pattern as mercury's pihole.env.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- sabnzbd, prowlarr, sonarr, radarr, clonarr, seerr, cinephage, mediamanager
services, wired into jupiter with LAN Caddy vhosts.
- Gitea: migrated the old ZimaOS docker instance's data (sqlite db, 4 repos,
no LFS objects) into the NixOS module's default stateDir layout. HTTP via
Caddy; git SSH on its own built-in server at :2222 (not :222 - the unpriv
gitea user can't bind <1024).
- mediamanager-nix flake input for the mediamanager service.
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
- services/pihole.nix: official pihole/pihole:2026.07.2 via podman, host net,
caps NET_ADMIN/NET_RAW/SYS_NICE/CHOWN; FTLCONF_* env config (upstream unbound,
DHCP 50-200, static lease jupiter, .sol domain, local records)
- unbound: resolveLocalQueries=false (was hijacking resolv.conf to :53 -> boot
DNS deadlock; the real root cause of the earlier failures too)
- password via sops FTLCONF env file; /var/lib/pihole created via tmpfiles
- VM-verified: mercury.sol/jupiter.sol/external all resolve, 0 restarts
mercury is the DHCP server (no lease of its own) so its name wasn't resolvable.
Add explicit dns.hosts A records. VM-verified: both resolve, external unaffected.
- per-host darman_password (distinct hash) in secrets/{jupiter,vps,mercury}.yaml
-> hashedPasswordFile; different console password per host (ssh still key-only)
- mercury: dedicated age key (on boot partition post-flash), sops-nix wired
- AdGuard: module has no secret hook + writable config -> mutableSettings=true,
admin password set via web setup on first boot (never in repo/store)
- services/{samba,avahi,audiobookshelf,containers,caddy,tailscale}.nix
- common.nix grows firewall base + timezone; hosts import what they need
- jupiter/vm/vps import service modules; drop the jupiter/services.nix monolith
- each module opens its own firewall ports; caddy/tailscale shared by hosts
- verified: jupiter/vps/vbox eval + jupiter builds, config equivalent
darmanandClaude Opus 4.8