deploy: make the local reinstall path actually work, and fail closed

The `install <config> localhost` auto-path added in 0ea9020 could not have
completed. Fixed three blockers plus the guard that was silently not guarding.

Inside installer-iso the run died before doing anything:
  - systemd sets no $HOME for a service without User= (SetLoginEnvironment=
    defaults to false), and this script runs under `set -u`, so it aborted on
    the bare $HOME with "unbound variable". Added $KEYDIR + Environment=HOME.
  - the host key it needs to seed /etc/ssh isn't on the ISO at all — that is
    built from a public repo and carries no credentials on purpose. It now
    travels on the boot partition, located via homelab.keypart=<PARTUUID> on
    the kernel cmdline, and dies with the disko wipe minutes later. Without
    it sops can't decrypt on boot #1 and mutableUsers locks darman for good.
  - installation-cd-minimal leaves experimental-features unset, so both
    `nix run` and `nixos-install --flake` failed. (The nixos-images kexec
    installer sets them itself, which is why the same branch worked after
    kexec-local but not from the ISO.)

The staging-dir guard passed everything on btrfs: findmnt prints the
subvolume as /dev/sdb2[/@], lsblk can't open that, and an empty parent was
treated as "different disk" — so it allowed staging the iso on the very disk
disko then wiped. terra's current CachyOS root is exactly that layout. Now
uses --nofsroot, resolves EVERY whole-disk ancestor (LVM/RAID span several:
/mnt/ssd_01 -> sdd+sde), and treats "can't tell" as a hard error. btrfs
staging is refused outright — stage-1 mounts a btrfs volume's top level, so
an iso inside a subvolume is unreachable.

findiso= lost its leading slash whenever the staging mountpoint was /,
giving /findisovar/tmp/x.iso and an emergency shell after the reboot.

Also:
  - confirm before rebooting, like flash/kexec-local already do; --yes skips
    it and is what the ISO passes itself
  - $BOOT from `bootctl --print-boot-path`, not a hardcoded /boot
  - free-space checks on both target partitions before the ~1GB copy
  - `nix run .#disko` / `.#nixos-anywhere` from locked inputs instead of
    github:... master-of-the-day, resolved while a disk is being wiped
  - one_match warns instead of silently taking [0]; require_tracked covers
    every hosts/<config>/*.nix; flash traps its mount
  - drop nixos-images' `inputs.nixpkgs.follows` — that input doesn't exist,
    it only printed a warning on every nix command

Verified: the prepare path exercised under stubs against this box's real
disks (btrfs-on-OS-disk, tmpfs, LVM, subdirectory), shellcheck clean, all
six configs evaluate, checks.kexec-local still passes.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
2026-07-24 02:25:14 +02:00
co-authored by Claude Opus 4.8
parent 0ea90200b4
commit e538788907
5 changed files with 418 additions and 97 deletions
+38 -14
View File
@@ -118,26 +118,50 @@ self-elevates:
./scripts/deploy install terra localhost
```
`scripts/deploy` detects it isn't already inside a live installer (checks
`uname -n`) and instead: builds `installer-iso`'s kernel + initrd + iso image,
asks where to stage the iso file (never auto-picks — the wrong disk here is
destroyed mid-install; `HOMELAB_INSTALLER_STAGE_DIR` skips the prompt for
scripted use), refuses if that resolves to the same disk `disk-config.nix` is
about to wipe, copies the kernel/initrd to the ESP, writes a systemd-boot
**one-shot** entry (`bootctl set-oneshot` — reverts to the normal default
after this one boot, even if it fails) with `findiso=` + `homelab.install=terra`
on its kernel cmdline, and reboots — a real `systemctl reboot`, not kexec.
`uname -n`) and instead:
1. Asks where to stage the iso file — never auto-picks, because the wrong disk
here is destroyed mid-install (`HOMELAB_INSTALLER_STAGE_DIR` skips the
prompt for scripted use). It **refuses** if that path resolves to a disk
`disk-config.nix` is about to wipe, if it can't work out which physical disk
the path is on at all (fail-closed — LVM and RAID can span several), or if
it's on btrfs (stage-1 mounts a btrfs volume's *top level*, so a path inside
a subvolume never resolves and you boot to an emergency shell). On terra,
`/mnt/hdd_01` is the right answer; the CachyOS root is btrfs on the OS disk
and is rejected on both counts.
2. Prints exactly what it is about to do — OS disk, staging disk, boot entry —
and waits for you to type `yes`. `--yes` skips it; that is what the ISO
passes when it re-runs the command itself.
3. Builds `installer-iso`'s kernel + initrd + iso image, checks both target
partitions have room, then copies the kernel/initrd **and terra's
pre-generated ssh host key** to the boot partition (found via
`bootctl --print-boot-path`, not assumed to be `/boot`) and the iso to the
staging dir.
4. Writes a systemd-boot **one-shot** entry (`bootctl set-oneshot` — reverts to
the normal default after this one boot, even if it fails) with `findiso=` +
`homelab.install=terra` + `homelab.keypart=<PARTUUID>` on its kernel
cmdline, and reboots — a real `systemctl reboot`, not kexec.
The booted installer clones the repo (`homelab-checkout.service`, needs
network — it's public now, no credentials involved) and then
`homelab-auto-install.service` reads `homelab.install=terra` back off
`/proc/cmdline` and re-runs the exact same `./scripts/deploy install terra
localhost` itself — now genuinely inside the installer, so it takes the
disko + `nixos-install` branch instead of preparing again. Ships terra's
pre-generated host key first so `/run/secrets/*` decrypts on boot #1. The
whole thing is unattended after the initial reboot; ssh into
`/proc/cmdline`, mounts `homelab.keypart=` to pick terra's host key back up
into `/root/.config/homelab/terra/`, and re-runs the exact same
`./scripts/deploy install terra localhost` itself — now genuinely inside the
installer, so it takes the disko + `nixos-install` branch instead of preparing
again. That key is what seeds `/etc/ssh` on the new system, which is what lets
`/run/secrets/*` decrypt on boot #1; it has to travel this way because the ISO
is built from a **public** repo and deliberately carries no credentials. The
copy on the boot partition dies with the disko wipe minutes later.
The whole thing is unattended after the initial reboot; ssh into
`homelab-installer` (same pubkey as the ISO everywhere else) to watch
progress — `journalctl -u homelab-checkout -u homelab-auto-install -f`.
> The checkout step resolves `git.mgaction.town`, which goes through mercury's
> pihole on the LAN. If mercury is down, the installer boots fine but never
> gets the repo — fix DNS and `systemctl restart homelab-checkout`.
When it's done, reboot again into the freshly installed NixOS. Then, same as
any other host:
```
@@ -155,7 +179,7 @@ All arguments mandatory — no default host, no default config.
```
./deploy kexec <config> <host> # headless kexec into a RAM installer (RO-root box)
./deploy install <config> <host> # first install; wipes OS disk, ships host key
./deploy install <config> <host> [--yes] # first install; wipes OS disk, ships host key
./deploy switch <config> <host> # rebuild + activate on a running host
./deploy boot|test <config> <host> # stage for next boot / activate without boot entry
./deploy image <config> # build an SD-card image (mercury)