deploy: make the local reinstall path actually work, and fail closed
The `install <config> localhost` auto-path added in 0ea9020 could not have
completed. Fixed three blockers plus the guard that was silently not guarding.
Inside installer-iso the run died before doing anything:
- systemd sets no $HOME for a service without User= (SetLoginEnvironment=
defaults to false), and this script runs under `set -u`, so it aborted on
the bare $HOME with "unbound variable". Added $KEYDIR + Environment=HOME.
- the host key it needs to seed /etc/ssh isn't on the ISO at all — that is
built from a public repo and carries no credentials on purpose. It now
travels on the boot partition, located via homelab.keypart=<PARTUUID> on
the kernel cmdline, and dies with the disko wipe minutes later. Without
it sops can't decrypt on boot #1 and mutableUsers locks darman for good.
- installation-cd-minimal leaves experimental-features unset, so both
`nix run` and `nixos-install --flake` failed. (The nixos-images kexec
installer sets them itself, which is why the same branch worked after
kexec-local but not from the ISO.)
The staging-dir guard passed everything on btrfs: findmnt prints the
subvolume as /dev/sdb2[/@], lsblk can't open that, and an empty parent was
treated as "different disk" — so it allowed staging the iso on the very disk
disko then wiped. terra's current CachyOS root is exactly that layout. Now
uses --nofsroot, resolves EVERY whole-disk ancestor (LVM/RAID span several:
/mnt/ssd_01 -> sdd+sde), and treats "can't tell" as a hard error. btrfs
staging is refused outright — stage-1 mounts a btrfs volume's top level, so
an iso inside a subvolume is unreachable.
findiso= lost its leading slash whenever the staging mountpoint was /,
giving /findisovar/tmp/x.iso and an emergency shell after the reboot.
Also:
- confirm before rebooting, like flash/kexec-local already do; --yes skips
it and is what the ISO passes itself
- $BOOT from `bootctl --print-boot-path`, not a hardcoded /boot
- free-space checks on both target partitions before the ~1GB copy
- `nix run .#disko` / `.#nixos-anywhere` from locked inputs instead of
github:... master-of-the-day, resolved while a disk is being wiped
- one_match warns instead of silently taking [0]; require_tracked covers
every hosts/<config>/*.nix; flash traps its mount
- drop nixos-images' `inputs.nixpkgs.follows` — that input doesn't exist,
it only printed a warning on every nix command
Verified: the prepare path exercised under stubs against this box's real
disks (btrfs-on-OS-disk, tmpfs, LVM, subdirectory), shellcheck clean, all
six configs evaluate, checks.kexec-local still passes.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
@@ -125,11 +125,42 @@ kept its ssh host key. Run it after ANY change to the kexec paths.
|
|||||||
loop: it reads `homelab.install=<config>` back off `/proc/cmdline` (set by the prepare
|
loop: it reads `homelab.install=<config>` back off `/proc/cmdline` (set by the prepare
|
||||||
step) and re-runs the identical command itself once `homelab-checkout.service` has
|
step) and re-runs the identical command itself once `homelab-checkout.service` has
|
||||||
cloned the repo — the whole reinstall is one command and unattended after the first
|
cloned the repo — the whole reinstall is one command and unattended after the first
|
||||||
reboot. It always ASKS where to stage the iso file (never auto-picks — the wrong disk
|
reboot. It confirms (type `yes`) before rebooting, like `flash`/`kexec-local`; `--yes`
|
||||||
here is destroyed mid-install) and refuses if that turns out to be the disk
|
skips that and is what the ISO passes itself. It always ASKS where to stage the iso
|
||||||
`disk-config.nix` is about to wipe; `HOMELAB_INSTALLER_STAGE_DIR` skips the prompt for
|
file (never auto-picks — the wrong disk here is destroyed mid-install);
|
||||||
scripted use. Both this and `kexec-local` self-elevate via `sudo` (`require_root()`)
|
`HOMELAB_INSTALLER_STAGE_DIR` skips the prompt for scripted use. Both this and
|
||||||
rather than requiring you to prefix the command yourself.
|
`kexec-local` self-elevate via `sudo` (`require_root()`) rather than requiring you to
|
||||||
|
prefix the command yourself.
|
||||||
|
- **The staging-dir guard must fail CLOSED, and `findmnt` needs `--nofsroot`**: on btrfs
|
||||||
|
`findmnt -no SOURCE` prints `/dev/sdb2[/@]`, which `lsblk` cannot open, so a naive
|
||||||
|
parent-device lookup comes back empty. Treating empty as "different disk" silently
|
||||||
|
allowed staging the iso on the very disk disko then wiped — terra's CachyOS root is
|
||||||
|
exactly that layout, so it hit the live case. `disks_backing()` (`lsblk -rnso
|
||||||
|
NAME,TYPE`) returns EVERY whole-disk ancestor because LVM/RAID can span several
|
||||||
|
(`/mnt/ssd_01` → sdd + sde), and an empty result is a hard error, not a pass. btrfs
|
||||||
|
staging is refused outright: stage-1 mounts a btrfs volume's TOP level, so an iso
|
||||||
|
inside a subvolume is unreachable via `findiso=`.
|
||||||
|
- **`findiso=` must keep its leading slash**: stage-1 tests `[ -e "/findiso$isoPath" ]`,
|
||||||
|
so stripping the mountpoint prefix off a stagedir whose mountpoint is `/` yields
|
||||||
|
`var/tmp/x.iso` → `/findisovar/tmp/x.iso` → emergency shell, after you have already
|
||||||
|
rebooted out of the working OS.
|
||||||
|
- **The auto-install needs the host key shipped to it, and a `$HOME`**: the ISO is built
|
||||||
|
from a public repo with no credentials, so `local_install_prepare_and_reboot()` copies
|
||||||
|
the key onto the boot partition and passes that partition's PARTUUID as
|
||||||
|
`homelab.keypart=`; the service mounts it and drops the key in
|
||||||
|
`/root/.config/homelab/<config>/` before running the install. Also, systemd does NOT
|
||||||
|
set `$HOME` for a system service without `User=` (`SetLoginEnvironment=` defaults to
|
||||||
|
false), and `scripts/deploy` runs under `set -u` — hence `$KEYDIR` instead of a bare
|
||||||
|
`$HOME`, plus `Environment=HOME=/root` on the unit.
|
||||||
|
- **`installer-iso` must enable `experimental-features` itself.** `installation-cd-minimal`
|
||||||
|
leaves them unset, so `nix run` and `nixos-install --flake` both die with "experimental
|
||||||
|
Nix feature 'nix-command' is disabled". The nixos-images `kexec` installer sets them
|
||||||
|
itself, which is why the same `install <config> localhost` branch worked after
|
||||||
|
`kexec-local` but not from the ISO.
|
||||||
|
- **disko/nixos-anywhere run as `nix run .#disko` / `.#nixos-anywhere`**, from this
|
||||||
|
flake's locked inputs — not `nix run github:...`. They execute while a disk is being
|
||||||
|
wiped, so the revision must be the reviewed one in `flake.lock`, and it has to resolve
|
||||||
|
without network.
|
||||||
- **disko wipes only the OS disk** named in `hosts/<h>/disk-config.nix`; data disks are
|
- **disko wipes only the OS disk** named in `hosts/<h>/disk-config.nix`; data disks are
|
||||||
plain `fileSystems` in `configuration.nix`.
|
plain `fileSystems` in `configuration.nix`.
|
||||||
- `nixos-anywhere`/kexec needs a writable root; **ZimaOS root is read-only**, hence the
|
- `nixos-anywhere`/kexec needs a writable root; **ZimaOS root is read-only**, hence the
|
||||||
|
|||||||
@@ -118,26 +118,50 @@ self-elevates:
|
|||||||
./scripts/deploy install terra localhost
|
./scripts/deploy install terra localhost
|
||||||
```
|
```
|
||||||
`scripts/deploy` detects it isn't already inside a live installer (checks
|
`scripts/deploy` detects it isn't already inside a live installer (checks
|
||||||
`uname -n`) and instead: builds `installer-iso`'s kernel + initrd + iso image,
|
`uname -n`) and instead:
|
||||||
asks where to stage the iso file (never auto-picks — the wrong disk here is
|
|
||||||
destroyed mid-install; `HOMELAB_INSTALLER_STAGE_DIR` skips the prompt for
|
1. Asks where to stage the iso file — never auto-picks, because the wrong disk
|
||||||
scripted use), refuses if that resolves to the same disk `disk-config.nix` is
|
here is destroyed mid-install (`HOMELAB_INSTALLER_STAGE_DIR` skips the
|
||||||
about to wipe, copies the kernel/initrd to the ESP, writes a systemd-boot
|
prompt for scripted use). It **refuses** if that path resolves to a disk
|
||||||
**one-shot** entry (`bootctl set-oneshot` — reverts to the normal default
|
`disk-config.nix` is about to wipe, if it can't work out which physical disk
|
||||||
after this one boot, even if it fails) with `findiso=` + `homelab.install=terra`
|
the path is on at all (fail-closed — LVM and RAID can span several), or if
|
||||||
on its kernel cmdline, and reboots — a real `systemctl reboot`, not kexec.
|
it's on btrfs (stage-1 mounts a btrfs volume's *top level*, so a path inside
|
||||||
|
a subvolume never resolves and you boot to an emergency shell). On terra,
|
||||||
|
`/mnt/hdd_01` is the right answer; the CachyOS root is btrfs on the OS disk
|
||||||
|
and is rejected on both counts.
|
||||||
|
2. Prints exactly what it is about to do — OS disk, staging disk, boot entry —
|
||||||
|
and waits for you to type `yes`. `--yes` skips it; that is what the ISO
|
||||||
|
passes when it re-runs the command itself.
|
||||||
|
3. Builds `installer-iso`'s kernel + initrd + iso image, checks both target
|
||||||
|
partitions have room, then copies the kernel/initrd **and terra's
|
||||||
|
pre-generated ssh host key** to the boot partition (found via
|
||||||
|
`bootctl --print-boot-path`, not assumed to be `/boot`) and the iso to the
|
||||||
|
staging dir.
|
||||||
|
4. Writes a systemd-boot **one-shot** entry (`bootctl set-oneshot` — reverts to
|
||||||
|
the normal default after this one boot, even if it fails) with `findiso=` +
|
||||||
|
`homelab.install=terra` + `homelab.keypart=<PARTUUID>` on its kernel
|
||||||
|
cmdline, and reboots — a real `systemctl reboot`, not kexec.
|
||||||
|
|
||||||
The booted installer clones the repo (`homelab-checkout.service`, needs
|
The booted installer clones the repo (`homelab-checkout.service`, needs
|
||||||
network — it's public now, no credentials involved) and then
|
network — it's public now, no credentials involved) and then
|
||||||
`homelab-auto-install.service` reads `homelab.install=terra` back off
|
`homelab-auto-install.service` reads `homelab.install=terra` back off
|
||||||
`/proc/cmdline` and re-runs the exact same `./scripts/deploy install terra
|
`/proc/cmdline`, mounts `homelab.keypart=` to pick terra's host key back up
|
||||||
localhost` itself — now genuinely inside the installer, so it takes the
|
into `/root/.config/homelab/terra/`, and re-runs the exact same
|
||||||
disko + `nixos-install` branch instead of preparing again. Ships terra's
|
`./scripts/deploy install terra localhost` itself — now genuinely inside the
|
||||||
pre-generated host key first so `/run/secrets/*` decrypts on boot #1. The
|
installer, so it takes the disko + `nixos-install` branch instead of preparing
|
||||||
whole thing is unattended after the initial reboot; ssh into
|
again. That key is what seeds `/etc/ssh` on the new system, which is what lets
|
||||||
|
`/run/secrets/*` decrypt on boot #1; it has to travel this way because the ISO
|
||||||
|
is built from a **public** repo and deliberately carries no credentials. The
|
||||||
|
copy on the boot partition dies with the disko wipe minutes later.
|
||||||
|
|
||||||
|
The whole thing is unattended after the initial reboot; ssh into
|
||||||
`homelab-installer` (same pubkey as the ISO everywhere else) to watch
|
`homelab-installer` (same pubkey as the ISO everywhere else) to watch
|
||||||
progress — `journalctl -u homelab-checkout -u homelab-auto-install -f`.
|
progress — `journalctl -u homelab-checkout -u homelab-auto-install -f`.
|
||||||
|
|
||||||
|
> The checkout step resolves `git.mgaction.town`, which goes through mercury's
|
||||||
|
> pihole on the LAN. If mercury is down, the installer boots fine but never
|
||||||
|
> gets the repo — fix DNS and `systemctl restart homelab-checkout`.
|
||||||
|
|
||||||
When it's done, reboot again into the freshly installed NixOS. Then, same as
|
When it's done, reboot again into the freshly installed NixOS. Then, same as
|
||||||
any other host:
|
any other host:
|
||||||
```
|
```
|
||||||
@@ -155,7 +179,7 @@ All arguments mandatory — no default host, no default config.
|
|||||||
|
|
||||||
```
|
```
|
||||||
./deploy kexec <config> <host> # headless kexec into a RAM installer (RO-root box)
|
./deploy kexec <config> <host> # headless kexec into a RAM installer (RO-root box)
|
||||||
./deploy install <config> <host> # first install; wipes OS disk, ships host key
|
./deploy install <config> <host> [--yes] # first install; wipes OS disk, ships host key
|
||||||
./deploy switch <config> <host> # rebuild + activate on a running host
|
./deploy switch <config> <host> # rebuild + activate on a running host
|
||||||
./deploy boot|test <config> <host> # stage for next boot / activate without boot entry
|
./deploy boot|test <config> <host> # stage for next boot / activate without boot entry
|
||||||
./deploy image <config> # build an SD-card image (mercury)
|
./deploy image <config> # build an SD-card image (mercury)
|
||||||
|
|||||||
Generated
+77
-18
@@ -179,9 +179,63 @@
|
|||||||
"type": "github"
|
"type": "github"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"nix-vm-test": {
|
||||||
|
"inputs": {
|
||||||
|
"nixpkgs": [
|
||||||
|
"nixos-anywhere",
|
||||||
|
"nixpkgs"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"locked": {
|
||||||
|
"lastModified": 1783265394,
|
||||||
|
"narHash": "sha256-cq4YfNFGYzp0NItZP8tEC7xUI8OSgY4fj75AU/NSaPM=",
|
||||||
|
"owner": "numtide",
|
||||||
|
"repo": "nix-vm-test",
|
||||||
|
"rev": "1a587212d2ac8b669c6c32499015f996506b6ba5",
|
||||||
|
"type": "github"
|
||||||
|
},
|
||||||
|
"original": {
|
||||||
|
"owner": "numtide",
|
||||||
|
"repo": "nix-vm-test",
|
||||||
|
"type": "github"
|
||||||
|
}
|
||||||
|
},
|
||||||
|
"nixos-anywhere": {
|
||||||
|
"inputs": {
|
||||||
|
"disko": [
|
||||||
|
"disko"
|
||||||
|
],
|
||||||
|
"nix-vm-test": "nix-vm-test",
|
||||||
|
"nixos-images": [
|
||||||
|
"nixos-images"
|
||||||
|
],
|
||||||
|
"nixos-stable": [
|
||||||
|
"nixpkgs"
|
||||||
|
],
|
||||||
|
"nixpkgs": [
|
||||||
|
"nixpkgs"
|
||||||
|
],
|
||||||
|
"treefmt-nix": "treefmt-nix"
|
||||||
|
},
|
||||||
|
"locked": {
|
||||||
|
"lastModified": 1784704708,
|
||||||
|
"narHash": "sha256-xuzg2Xfalr1X5ah5IPgqB8Vyif+2je1qkCtGxCB+rm0=",
|
||||||
|
"owner": "nix-community",
|
||||||
|
"repo": "nixos-anywhere",
|
||||||
|
"rev": "91fc9b70fc295258c366cce8627efb6f185fd9fb",
|
||||||
|
"type": "github"
|
||||||
|
},
|
||||||
|
"original": {
|
||||||
|
"owner": "nix-community",
|
||||||
|
"repo": "nixos-anywhere",
|
||||||
|
"type": "github"
|
||||||
|
}
|
||||||
|
},
|
||||||
"nixos-images": {
|
"nixos-images": {
|
||||||
"inputs": {
|
"inputs": {
|
||||||
"nixos-stable": "nixos-stable",
|
"nixos-stable": [
|
||||||
|
"nixpkgs"
|
||||||
|
],
|
||||||
"nixos-unstable": "nixos-unstable"
|
"nixos-unstable": "nixos-unstable"
|
||||||
},
|
},
|
||||||
"locked": {
|
"locked": {
|
||||||
@@ -198,23 +252,6 @@
|
|||||||
"type": "github"
|
"type": "github"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
"nixos-stable": {
|
|
||||||
"locked": {
|
|
||||||
"lastModified": 1783389287,
|
|
||||||
"narHash": "sha256-0xIy4dVLqq47rA+mRy0hXDfjhQd4E5PoIns/RmB7nR4=",
|
|
||||||
"ref": "nixos-26.05",
|
|
||||||
"rev": "0ad6f47ea4fe188f4bc8f0380f93ae8523337c6c",
|
|
||||||
"shallow": true,
|
|
||||||
"type": "git",
|
|
||||||
"url": "https://github.com/NixOS/nixpkgs"
|
|
||||||
},
|
|
||||||
"original": {
|
|
||||||
"ref": "nixos-26.05",
|
|
||||||
"shallow": true,
|
|
||||||
"type": "git",
|
|
||||||
"url": "https://github.com/NixOS/nixpkgs"
|
|
||||||
}
|
|
||||||
},
|
|
||||||
"nixos-unstable": {
|
"nixos-unstable": {
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1782175435,
|
"lastModified": 1782175435,
|
||||||
@@ -421,6 +458,7 @@
|
|||||||
"disko": "disko",
|
"disko": "disko",
|
||||||
"home-manager": "home-manager",
|
"home-manager": "home-manager",
|
||||||
"mediamanager-nix": "mediamanager-nix",
|
"mediamanager-nix": "mediamanager-nix",
|
||||||
|
"nixos-anywhere": "nixos-anywhere",
|
||||||
"nixos-images": "nixos-images",
|
"nixos-images": "nixos-images",
|
||||||
"nixpkgs": "nixpkgs_2",
|
"nixpkgs": "nixpkgs_2",
|
||||||
"nixpkgs-unstable": "nixpkgs-unstable",
|
"nixpkgs-unstable": "nixpkgs-unstable",
|
||||||
@@ -480,6 +518,27 @@
|
|||||||
"url": "ssh://gitea@git.mgaction.town:2222/darman/TOME.git"
|
"url": "ssh://gitea@git.mgaction.town:2222/darman/TOME.git"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
"treefmt-nix": {
|
||||||
|
"inputs": {
|
||||||
|
"nixpkgs": [
|
||||||
|
"nixos-anywhere",
|
||||||
|
"nixpkgs"
|
||||||
|
]
|
||||||
|
},
|
||||||
|
"locked": {
|
||||||
|
"lastModified": 1784369104,
|
||||||
|
"narHash": "sha256-47cxbcZODibHv3rELFQ9vZly0vUNkND/atn/U7HLeb0=",
|
||||||
|
"owner": "numtide",
|
||||||
|
"repo": "treefmt-nix",
|
||||||
|
"rev": "df3c0640565d04a0261253cdd89fce78ec50168a",
|
||||||
|
"type": "github"
|
||||||
|
},
|
||||||
|
"original": {
|
||||||
|
"owner": "numtide",
|
||||||
|
"repo": "treefmt-nix",
|
||||||
|
"type": "github"
|
||||||
|
}
|
||||||
|
},
|
||||||
"uv2nix": {
|
"uv2nix": {
|
||||||
"inputs": {
|
"inputs": {
|
||||||
"nixpkgs": [
|
"nixpkgs": [
|
||||||
|
|||||||
@@ -16,9 +16,24 @@
|
|||||||
url = "github:Mic92/sops-nix";
|
url = "github:Mic92/sops-nix";
|
||||||
inputs.nixpkgs.follows = "nixpkgs";
|
inputs.nixpkgs.follows = "nixpkgs";
|
||||||
};
|
};
|
||||||
|
# NB: no `inputs.nixpkgs.follows` here — nixos-images has no `nixpkgs`
|
||||||
|
# input (it takes nixos-stable / nixos-unstable), so declaring one only
|
||||||
|
# printed "override for a non-existent input" on every nix command.
|
||||||
nixos-images = {
|
nixos-images = {
|
||||||
url = "github:nix-community/nixos-images";
|
url = "github:nix-community/nixos-images";
|
||||||
|
inputs.nixos-stable.follows = "nixpkgs";
|
||||||
|
};
|
||||||
|
# Pinned as an input rather than `nix run github:...` from scripts/deploy.
|
||||||
|
# Both it and disko run at the exact moment a disk is being wiped, so the
|
||||||
|
# revision has to come from flake.lock — reviewed, reproducible, and
|
||||||
|
# resolvable from the local store — instead of whatever upstream master
|
||||||
|
# happens to be that day (which also fails outright with no network).
|
||||||
|
nixos-anywhere = {
|
||||||
|
url = "github:nix-community/nixos-anywhere";
|
||||||
inputs.nixpkgs.follows = "nixpkgs";
|
inputs.nixpkgs.follows = "nixpkgs";
|
||||||
|
inputs.nixos-stable.follows = "nixpkgs"; # 26.05 already IS stable
|
||||||
|
inputs.disko.follows = "disko";
|
||||||
|
inputs.nixos-images.follows = "nixos-images";
|
||||||
};
|
};
|
||||||
home-manager = {
|
home-manager = {
|
||||||
url = "github:nix-community/home-manager/release-26.05";
|
url = "github:nix-community/home-manager/release-26.05";
|
||||||
@@ -48,15 +63,23 @@
|
|||||||
};
|
};
|
||||||
};
|
};
|
||||||
|
|
||||||
outputs = { self, nixpkgs, disko, sops-nix, nixos-images, home-manager, mediamanager-nix, authentik-nix, ... }@inputs:
|
outputs = { self, nixpkgs, disko, nixos-anywhere, sops-nix, nixos-images, home-manager, mediamanager-nix, authentik-nix, ... }@inputs:
|
||||||
let
|
let
|
||||||
system = "x86_64-linux";
|
system = "x86_64-linux";
|
||||||
in
|
in
|
||||||
{
|
{
|
||||||
packages.${system}.tome = nixpkgs.legacyPackages.${system}.callPackage ./pkgs/tome.nix {
|
packages.${system} = {
|
||||||
|
tome = nixpkgs.legacyPackages.${system}.callPackage ./pkgs/tome.nix {
|
||||||
src = inputs.tome;
|
src = inputs.tome;
|
||||||
};
|
};
|
||||||
|
|
||||||
|
# Re-exported so `./scripts/deploy` can run them as `nix run .#disko` /
|
||||||
|
# `nix run .#nixos-anywhere`, at the revision flake.lock pins. See the
|
||||||
|
# nixos-anywhere input above for why that matters.
|
||||||
|
disko = disko.packages.${system}.disko;
|
||||||
|
nixos-anywhere = nixos-anywhere.packages.${system}.nixos-anywhere;
|
||||||
|
};
|
||||||
|
|
||||||
nixosConfigurations = {
|
nixosConfigurations = {
|
||||||
# Real host — install on the ZimaBlade.
|
# Real host — install on the ZimaBlade.
|
||||||
# disko owns the OS-disk partitioning + filesystems (see disk-config.nix).
|
# disko owns the OS-disk partitioning + filesystems (see disk-config.nix).
|
||||||
@@ -185,6 +208,16 @@
|
|||||||
networking.hostName = "homelab-installer";
|
networking.hostName = "homelab-installer";
|
||||||
environment.systemPackages = [ pkgs.git ];
|
environment.systemPackages = [ pkgs.git ];
|
||||||
|
|
||||||
|
# installation-cd-minimal leaves experimental-features unset, so
|
||||||
|
# the ISO's nix.conf has no `nix-command`/`flakes` at all (unlike
|
||||||
|
# the nixos-images kexec installer, which sets
|
||||||
|
# extra-experimental-features itself — which is why the same
|
||||||
|
# `install <config> localhost` branch works after kexec-local but
|
||||||
|
# not here). Without this, both `nix run .#disko` and
|
||||||
|
# `nixos-install --flake` die with "experimental Nix feature
|
||||||
|
# 'nix-command' is disabled".
|
||||||
|
nix.settings.experimental-features = [ "nix-command" "flakes" ];
|
||||||
|
|
||||||
# Fresh clone of a PUBLIC repo — no credentials baked into the
|
# Fresh clone of a PUBLIC repo — no credentials baked into the
|
||||||
# ISO. require_tracked() in scripts/deploy still works fine here
|
# ISO. require_tracked() in scripts/deploy still works fine here
|
||||||
# (this IS a real git checkout, unlike the old baked-`self`
|
# (this IS a real git checkout, unlike the old baked-`self`
|
||||||
@@ -219,17 +252,51 @@
|
|||||||
after = [ "homelab-checkout.service" ];
|
after = [ "homelab-checkout.service" ];
|
||||||
requires = [ "homelab-checkout.service" ];
|
requires = [ "homelab-checkout.service" ];
|
||||||
wantedBy = [ "multi-user.target" ];
|
wantedBy = [ "multi-user.target" ];
|
||||||
path = [ pkgs.gnugrep ];
|
path = [ pkgs.gnugrep pkgs.util-linux pkgs.coreutils pkgs.git ];
|
||||||
serviceConfig.Type = "oneshot";
|
serviceConfig.Type = "oneshot";
|
||||||
|
# systemd does NOT set $HOME for a system service without User=
|
||||||
|
# (systemd.exec(5): SetLoginEnvironment= "defaults to true if
|
||||||
|
# User=, DynamicUser= or PAMName= are set, false otherwise"), and
|
||||||
|
# scripts/deploy runs under `set -u`. Without this the whole
|
||||||
|
# unattended run died on the bare $HOME expansion with an
|
||||||
|
# "unbound variable" that reads like a bug in the script.
|
||||||
|
serviceConfig.Environment = "HOME=/root";
|
||||||
script = ''
|
script = ''
|
||||||
cfg=$(grep -o 'homelab\.install=[^ ]*' /proc/cmdline | cut -d= -f2 || true)
|
cfg=$(grep -o 'homelab\.install=[^ ]*' /proc/cmdline | cut -d= -f2 || true)
|
||||||
if [ -z "$cfg" ]; then
|
if [ -z "$cfg" ]; then
|
||||||
echo "no homelab.install= on the kernel cmdline — nothing to auto-install"
|
echo "no homelab.install= on the kernel cmdline — nothing to auto-install"
|
||||||
exit 0
|
exit 0
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# The host key scripts/deploy seeds /etc/ssh with (so sops can
|
||||||
|
# decrypt on boot #1) cannot live in this ISO: it is built from
|
||||||
|
# a PUBLIC repo and the private keys are deliberately off-repo.
|
||||||
|
# local_install_prepare_and_reboot() therefore drops it on the
|
||||||
|
# boot partition and passes that partition's PARTUUID here.
|
||||||
|
# That copy dies with the disko wipe a few minutes later.
|
||||||
|
keypart=$(grep -o 'homelab\.keypart=[^ ]*' /proc/cmdline | cut -d= -f2 || true)
|
||||||
|
if [ -n "$keypart" ]; then
|
||||||
|
mkdir -p /run/homelab-key
|
||||||
|
if mount -o ro "/dev/disk/by-partuuid/$keypart" /run/homelab-key; then
|
||||||
|
src=/run/homelab-key/homelab-installer
|
||||||
|
if [ -f "$src/ssh_host_ed25519_key" ]; then
|
||||||
|
echo "picking up $cfg's host key from PARTUUID=$keypart"
|
||||||
|
install -Dm600 "$src/ssh_host_ed25519_key" \
|
||||||
|
"/root/.config/homelab/$cfg/ssh_host_ed25519_key"
|
||||||
|
install -Dm644 "$src/ssh_host_ed25519_key.pub" \
|
||||||
|
"/root/.config/homelab/$cfg/ssh_host_ed25519_key.pub"
|
||||||
|
else
|
||||||
|
echo "warning: no host key at $src — the install will refuse" >&2
|
||||||
|
fi
|
||||||
|
umount /run/homelab-key
|
||||||
|
else
|
||||||
|
echo "warning: could not mount PARTUUID=$keypart for the host key" >&2
|
||||||
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
echo "auto-installing $cfg (homelab.install= on the kernel cmdline)"
|
echo "auto-installing $cfg (homelab.install= on the kernel cmdline)"
|
||||||
cd /root/homelab
|
cd /root/homelab
|
||||||
exec ./scripts/deploy install "$cfg" localhost
|
exec ./scripts/deploy install "$cfg" localhost --yes
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
})
|
})
|
||||||
|
|||||||
+195
-55
@@ -17,7 +17,8 @@
|
|||||||
# /var/tmp) must be exec-capable and hold
|
# /var/tmp) must be exec-capable and hold
|
||||||
# ~3x the tarball.
|
# ~3x the tarball.
|
||||||
# Then run `install <config> localhost`.
|
# Then run `install <config> localhost`.
|
||||||
# ./deploy install <config> <host> first install. Wipes the OS disk. Ships the
|
# ./deploy install <config> <host> [--yes]
|
||||||
|
# first install. Wipes the OS disk. Ships the
|
||||||
# host's sops key. <host>=localhost/127.0.0.1
|
# host's sops key. <host>=localhost/127.0.0.1
|
||||||
# skips nixos-anywhere/ssh and runs disko +
|
# skips nixos-anywhere/ssh and runs disko +
|
||||||
# nixos-install directly against /mnt — but
|
# nixos-install directly against /mnt — but
|
||||||
@@ -27,16 +28,21 @@
|
|||||||
# Run from the REAL running OS instead (e.g.
|
# Run from the REAL running OS instead (e.g.
|
||||||
# a box where kexec-local doesn't work),
|
# a box where kexec-local doesn't work),
|
||||||
# it builds installer-iso, stages its
|
# it builds installer-iso, stages its
|
||||||
# kernel/initrd on the ESP + the iso file on
|
# kernel/initrd + the host key on the boot
|
||||||
# a non-OS-disk partition, sets a systemd-boot
|
# partition and the iso file on a non-OS-disk
|
||||||
# one-shot entry with homelab.install=<config>
|
# partition, sets a systemd-boot one-shot
|
||||||
# on its kernel cmdline, and reboots — a real
|
# entry with homelab.install=<config> +
|
||||||
# ACPI reboot, not a kexec jump. The booted
|
# homelab.keypart=<PARTUUID> on its kernel
|
||||||
# installer's homelab-auto-install.service
|
# cmdline, and reboots — a real ACPI reboot,
|
||||||
# reads that cmdline param and re-runs this
|
# not a kexec jump. The booted installer's
|
||||||
# exact command itself once its repo checkout
|
# homelab-auto-install.service reads those
|
||||||
# (homelab-checkout.service) succeeds, finishing
|
# cmdline params, picks the host key back up
|
||||||
# the install unattended. See CLAUDE.md.
|
# and re-runs this exact command itself once
|
||||||
|
# its repo checkout (homelab-checkout.service)
|
||||||
|
# succeeds, finishing the install unattended.
|
||||||
|
# It confirms before rebooting; --yes skips
|
||||||
|
# that (it is what the ISO passes itself).
|
||||||
|
# See CLAUDE.md.
|
||||||
# ./deploy switch <config> <host> rebuild + activate on a running host.
|
# ./deploy switch <config> <host> rebuild + activate on a running host.
|
||||||
# ./deploy boot <config> <host> stage for next boot, don't activate now.
|
# ./deploy boot <config> <host> stage for next boot, don't activate now.
|
||||||
# ./deploy test <config> <host> activate without adding a boot entry.
|
# ./deploy test <config> <host> activate without adding a boot entry.
|
||||||
@@ -75,6 +81,17 @@ REPO="$(git -C "$SCRIPT_DIR" rev-parse --show-toplevel 2>/dev/null || dirname "$
|
|||||||
cd "$REPO"
|
cd "$REPO"
|
||||||
export PATH="/nix/var/nix/profiles/default/bin:$PATH"
|
export PATH="/nix/var/nix/profiles/default/bin:$PATH"
|
||||||
|
|
||||||
|
# Off-repo material keyed by <config>: pre-generated SSH host keys (install)
|
||||||
|
# and per-config sops age keys (flash).
|
||||||
|
#
|
||||||
|
# Resolved defensively rather than as a bare $HOME, because this script also
|
||||||
|
# runs from installer-iso's homelab-auto-install.service, and systemd does not
|
||||||
|
# set $HOME for a system service without User= (systemd.exec(5):
|
||||||
|
# SetLoginEnvironment= "defaults to true if User=, DynamicUser= or PAMName= are
|
||||||
|
# set, false otherwise"). Under `set -u` that aborted the whole unattended run
|
||||||
|
# with an "unbound variable" that read like a bug in this script.
|
||||||
|
KEYDIR="${HOMELAB_KEY_DIR:-${HOME:-/root}/.config/homelab}"
|
||||||
|
|
||||||
die() { echo "error: $*" >&2; exit 1; }
|
die() { echo "error: $*" >&2; exit 1; }
|
||||||
|
|
||||||
need() { command -v "$1" >/dev/null 2>&1 || die "missing required tool: $1"; }
|
need() { command -v "$1" >/dev/null 2>&1 || die "missing required tool: $1"; }
|
||||||
@@ -85,6 +102,11 @@ need() { command -v "$1" >/dev/null 2>&1 || die "missing required tool: $1"; }
|
|||||||
require_root() {
|
require_root() {
|
||||||
[ "$(id -u)" = 0 ] && return 0
|
[ "$(id -u)" = 0 ] && return 0
|
||||||
echo ">> $1 needs root — re-executing under sudo" >&2
|
echo ">> $1 needs root — re-executing under sudo" >&2
|
||||||
|
# $KEYDIR is derived from $HOME, and whether sudo carries $HOME across
|
||||||
|
# depends on the local sudoers policy (env_reset/always_set_home). Pin the
|
||||||
|
# resolved value so the re-exec looks for host keys where the invoking user
|
||||||
|
# has them, not under /root.
|
||||||
|
export HOMELAB_KEY_DIR="$KEYDIR"
|
||||||
exec sudo -E -- "$SCRIPT_PATH" "${SCRIPT_ARGS[@]}"
|
exec sudo -E -- "$SCRIPT_PATH" "${SCRIPT_ARGS[@]}"
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -95,9 +117,21 @@ one_match() {
|
|||||||
local what="$1"; shift
|
local what="$1"; shift
|
||||||
local f=("$@") # caller expands the glob (nullglob is on)
|
local f=("$@") # caller expands the glob (nullglob is on)
|
||||||
[ "${#f[@]}" -gt 0 ] || die "no $what found — did the build actually produce one?"
|
[ "${#f[@]}" -gt 0 ] || die "no $what found — did the build actually produce one?"
|
||||||
|
# Say so instead of silently taking [0]: a stale result-sd/ symlink from an
|
||||||
|
# earlier config is exactly how you flash the wrong image without a word.
|
||||||
|
[ "${#f[@]}" -eq 1 ] \
|
||||||
|
|| echo ">> warning: ${#f[@]} candidates for $what, using ${f[0]} (rm the stale ones)" >&2
|
||||||
printf '%s\n' "${f[0]}"
|
printf '%s\n' "${f[0]}"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# Every whole-disk device backing a block device or a mounted path, one per
|
||||||
|
# line. LVM/RAID/LUKS can sit on several at once (verified on terra:
|
||||||
|
# /mnt/ssd_01 -> sdd AND sde), so a single lookup is not enough. Empty output
|
||||||
|
# means "could not determine" — which callers must treat as unsafe, not as OK.
|
||||||
|
disks_backing() {
|
||||||
|
lsblk -rnso NAME,TYPE "$1" 2>/dev/null | awk '$2 == "disk" { print "/dev/" $1 }'
|
||||||
|
}
|
||||||
|
|
||||||
# Sets tb / cpio / bbox — the kexec tarball plus the static cpio+gzip that
|
# Sets tb / cpio / bbox — the kexec tarball plus the static cpio+gzip that
|
||||||
# kexec-run.sh needs on PATH to rebuild its initrd.
|
# kexec-run.sh needs on PATH to rebuild its initrd.
|
||||||
#
|
#
|
||||||
@@ -137,23 +171,38 @@ is_live_installer() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
# `install <config> localhost` run on a REAL running OS (not already inside a
|
# `install <config> localhost` run on a REAL running OS (not already inside a
|
||||||
# live installer): builds installer-iso, stages its kernel/initrd + iso file
|
# live installer): builds installer-iso, stages its kernel/initrd + the host's
|
||||||
# locally, points a systemd-boot one-shot entry at them with
|
# pre-generated ssh key on the boot partition and the iso file on a non-OS
|
||||||
# homelab.install=<config> on the kernel cmdline, and reboots — a real ACPI
|
# disk, points a systemd-boot one-shot entry at them with
|
||||||
# reboot through firmware POST, deliberately NOT a kexec jump (see terra's
|
# homelab.install=<config> + homelab.keypart=<PARTUUID> on the kernel cmdline,
|
||||||
# kexec-local gotcha in CLAUDE.md). The booted installer's
|
# and reboots — a real ACPI reboot through firmware POST, deliberately NOT a
|
||||||
# homelab-auto-install.service reads that cmdline param and re-runs this exact
|
# kexec jump (see terra's kexec-local gotcha in CLAUDE.md). The booted
|
||||||
# `install <config> localhost` command itself (now genuinely inside the
|
# installer's homelab-auto-install.service reads those params, picks the host
|
||||||
# installer) once homelab-checkout.service has fetched the repo, finishing the
|
# key back up and re-runs this exact `install <config> localhost` command
|
||||||
# job unattended.
|
# itself (now genuinely inside the installer) once homelab-checkout.service has
|
||||||
|
# fetched the repo, finishing the job unattended.
|
||||||
local_install_prepare_and_reboot() {
|
local_install_prepare_and_reboot() {
|
||||||
local config="$1"
|
local config="$1" hostkey="$2" assume_yes="$3"
|
||||||
require_root "preparing a local reinstall"
|
require_root "preparing a local reinstall"
|
||||||
[ -d /sys/firmware/efi ] || die "not booted UEFI — the one-shot boot entry needs systemd-boot"
|
[ -d /sys/firmware/efi ] || die "not booted UEFI — the one-shot boot entry needs systemd-boot"
|
||||||
need bootctl
|
need bootctl
|
||||||
need nix
|
need nix
|
||||||
need lsblk
|
need lsblk
|
||||||
need findmnt
|
need findmnt
|
||||||
|
need awk
|
||||||
|
need realpath
|
||||||
|
need stat
|
||||||
|
need df
|
||||||
|
|
||||||
|
# systemd-boot keeps its entries on $BOOT — the XBOOTLDR partition when there
|
||||||
|
# is one, the ESP otherwise — which is not always /boot. Hardcoding /boot on
|
||||||
|
# a box that mounts its ESP elsewhere just creates a directory on the root
|
||||||
|
# filesystem, and then reboots into an entry the firmware never sees.
|
||||||
|
local boot
|
||||||
|
boot="$(bootctl --print-boot-path 2>/dev/null)" \
|
||||||
|
|| die "bootctl couldn't locate the boot partition — is systemd-boot installed here?"
|
||||||
|
[ -d "$boot/loader/entries" ] \
|
||||||
|
|| die "$boot/loader/entries doesn't exist — systemd-boot isn't installed on this box"
|
||||||
|
|
||||||
# No default/auto-picked location — the wrong disk here is destroyed
|
# No default/auto-picked location — the wrong disk here is destroyed
|
||||||
# mid-install (see the OS-disk check below), so this always asks rather
|
# mid-install (see the OS-disk check below), so this always asks rather
|
||||||
@@ -168,48 +217,124 @@ local_install_prepare_and_reboot() {
|
|||||||
[ -n "$stagedir" ] || die "no staging path given"
|
[ -n "$stagedir" ] || die "no staging path given"
|
||||||
[ -d "$stagedir" ] \
|
[ -d "$stagedir" ] \
|
||||||
|| die "staging dir $stagedir doesn't exist — needs to be an existing partition that is NOT the OS disk being wiped"
|
|| die "staging dir $stagedir doesn't exist — needs to be an existing partition that is NOT the OS disk being wiped"
|
||||||
|
# Absolute + symlink-free: findiso= below is computed by stripping the
|
||||||
|
# mountpoint prefix off this, and a relative answer at the prompt would
|
||||||
|
# produce a path the initrd can never resolve.
|
||||||
|
stagedir="$(realpath "$stagedir")"
|
||||||
|
|
||||||
# Refuse if the staging partition turns out to live on the same disk
|
# Refuse if the staging partition turns out to live on the same disk
|
||||||
# disko is about to wipe — the iso file (and the running installer
|
# disko is about to wipe — the iso file (and the running installer
|
||||||
# loopback-mounted from it) would be destroyed mid-install.
|
# loopback-mounted from it) would be destroyed mid-install.
|
||||||
local osdisk osdisk_real stage_src stage_pkname stage_disk_real
|
local osdisk osdisk_real stage_src stage_fstype stage_disks d
|
||||||
osdisk="$(nix eval --raw ".#nixosConfigurations.$config.config.disko.devices.disk" \
|
osdisk="$(nix eval --raw ".#nixosConfigurations.$config.config.disko.devices.disk" \
|
||||||
--apply 'd: (builtins.head (builtins.attrValues d)).device' 2>/dev/null)" \
|
--apply 'd: (builtins.head (builtins.attrValues d)).device' 2>/dev/null)" \
|
||||||
|| die "couldn't read the OS disk device from hosts/$config/disk-config.nix"
|
|| die "couldn't read the OS disk device from hosts/$config/disk-config.nix"
|
||||||
osdisk_real="$(readlink -f "$osdisk")"
|
osdisk_real="$(readlink -f "$osdisk")"
|
||||||
stage_src="$(findmnt -no SOURCE --target "$stagedir")" \
|
|
||||||
|
# --nofsroot matters: on btrfs, findmnt prints the subvolume as
|
||||||
|
# `/dev/sdb2[/@]`, which is not a path lsblk can open. Without it the lookup
|
||||||
|
# came back empty and the guard below was skipped entirely — i.e. it silently
|
||||||
|
# allowed staging on the very disk about to be wiped. terra's current
|
||||||
|
# CachyOS root is exactly that layout.
|
||||||
|
stage_src="$(findmnt -no SOURCE --nofsroot --target "$stagedir")" \
|
||||||
|| die "$stagedir doesn't resolve to a mounted filesystem"
|
|| die "$stagedir doesn't resolve to a mounted filesystem"
|
||||||
stage_pkname="$(lsblk -no PKNAME "$stage_src" 2>/dev/null || true)"
|
# `|| true` so the explicit check below is what reports the problem: lsblk
|
||||||
if [ -n "$stage_pkname" ]; then
|
# exits nonzero on a device it can't parse, and under `set -e` + pipefail a
|
||||||
stage_disk_real="$(readlink -f "/dev/$stage_pkname")"
|
# bare assignment from a failing substitution kills the script silently,
|
||||||
[ "$stage_disk_real" = "$osdisk_real" ] \
|
# right past the fail-closed message.
|
||||||
&& die "$stagedir is on the OS disk ($osdisk) that install would wipe — re-run and pick a different disk"
|
stage_disks="$(disks_backing "$stage_src" || true)"
|
||||||
|
# Fail closed. "Couldn't determine the disk" is not "different disk".
|
||||||
|
[ -n "$stage_disks" ] \
|
||||||
|
|| die "couldn't determine which physical disk $stagedir ($stage_src) is on — refusing to guess, since being wrong destroys the install mid-flight"
|
||||||
|
for d in $stage_disks; do
|
||||||
|
if [ "$d" = "$osdisk_real" ]; then
|
||||||
|
die "$stagedir is on the OS disk ($osdisk -> $osdisk_real) that install would wipe — re-run and pick a different disk"
|
||||||
|
fi
|
||||||
|
done
|
||||||
|
|
||||||
|
# stage-1 resolves findiso= by mounting each blkid-visible partition and
|
||||||
|
# testing `-e /findiso$isoPath` (nixos/modules/system/boot/stage-1-init.sh).
|
||||||
|
# For btrfs it mounts the volume's TOP level, so a path that lives inside a
|
||||||
|
# subvolume (/@/...) is simply not there and the box boots to an emergency
|
||||||
|
# shell — after it has already rebooted out of the working OS.
|
||||||
|
stage_fstype="$(findmnt -no FSTYPE --target "$stagedir")"
|
||||||
|
[ "$stage_fstype" != btrfs ] \
|
||||||
|
|| die "$stagedir is btrfs: findiso= mounts the volume's top level, so a path inside a subvolume never resolves. Stage on a non-btrfs partition (ext4/vfat/ntfs)."
|
||||||
|
|
||||||
|
# Last chance to back out. This is the most destructive command in the
|
||||||
|
# script — it reboots the machine you are typing at and the wipe that
|
||||||
|
# follows is unattended — so it confirms just like `flash` and `kexec-local`
|
||||||
|
# do, both of which are less final than this.
|
||||||
|
if [ "$assume_yes" != "--yes" ]; then
|
||||||
|
echo ">> about to REINSTALL this machine from scratch:"
|
||||||
|
echo " hostname: $(uname -n)"
|
||||||
|
echo " config: $config"
|
||||||
|
echo " OS disk: $osdisk"
|
||||||
|
echo " -> $osdisk_real ** WIPED, unattended, after the reboot **"
|
||||||
|
# Unquoted on purpose: collapses the one-per-line list onto one line.
|
||||||
|
echo " staging: $stagedir (on $(echo $stage_disks))"
|
||||||
|
echo " entry: $boot/loader/entries/homelab-installer.conf (one-shot)"
|
||||||
|
read -rp ">> type 'yes' to build the installer, reboot into it and wipe $osdisk_real: " ok
|
||||||
|
[ "$ok" = yes ] || die "aborted"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
echo ">> building installer-iso (kernel + initrd + iso image)"
|
echo ">> building installer-iso (kernel + initrd + iso image)"
|
||||||
local kernel initrd isodir iso mnt_point iso_relpath
|
local kernel initrd isodir iso mnt_point iso_relpath boot_src boot_partuuid
|
||||||
kernel="$(nix build --no-link --print-out-paths .#nixosConfigurations.installer-iso.config.system.build.kernel)/bzImage"
|
kernel="$(nix build --no-link --print-out-paths .#nixosConfigurations.installer-iso.config.system.build.kernel)/bzImage"
|
||||||
initrd="$(nix build --no-link --print-out-paths .#nixosConfigurations.installer-iso.config.system.build.initialRamdisk)/initrd"
|
initrd="$(nix build --no-link --print-out-paths .#nixosConfigurations.installer-iso.config.system.build.initialRamdisk)/initrd"
|
||||||
isodir="$(nix build --no-link --print-out-paths .#nixosConfigurations.installer-iso.config.system.build.isoImage)"
|
isodir="$(nix build --no-link --print-out-paths .#nixosConfigurations.installer-iso.config.system.build.isoImage)"
|
||||||
iso="$(one_match 'installer iso' "$isodir"/iso/*.iso)"
|
iso="$(one_match 'installer iso' "$isodir"/iso/*.iso)"
|
||||||
|
|
||||||
echo ">> staging kernel/initrd on the ESP, iso image on $stagedir"
|
# A short write is not visible until the reboot, when findiso finds a
|
||||||
install -Dm644 "$kernel" /boot/homelab-installer/bzImage
|
# truncated iso and drops to an emergency shell. Check first — `install`
|
||||||
install -Dm644 "$initrd" /boot/homelab-installer/initrd
|
# prints no progress and the iso is ~1GB.
|
||||||
|
local need_stage need_boot avail_stage avail_boot
|
||||||
|
need_stage="$(stat -Lc %s "$iso")"
|
||||||
|
need_boot="$(( $(stat -Lc %s "$kernel") + $(stat -Lc %s "$initrd") + $(stat -Lc %s "$hostkey") ))"
|
||||||
|
avail_stage="$(df -B1 --output=avail "$stagedir" | tail -1 | tr -d ' ')"
|
||||||
|
avail_boot="$(df -B1 --output=avail "$boot" | tail -1 | tr -d ' ')"
|
||||||
|
[ "$avail_stage" -ge "$(( need_stage + 64 * 1024 * 1024 ))" ] \
|
||||||
|
|| die "$stagedir has $(( avail_stage / 1024 / 1024 ))MB free, the iso needs $(( need_stage / 1024 / 1024 ))MB — pick another partition"
|
||||||
|
[ "$avail_boot" -ge "$(( need_boot + 16 * 1024 * 1024 ))" ] \
|
||||||
|
|| die "$boot has $(( avail_boot / 1024 / 1024 ))MB free, kernel+initrd need $(( need_boot / 1024 / 1024 ))MB"
|
||||||
|
|
||||||
|
echo ">> staging kernel/initrd/host key on $boot, iso image on $stagedir"
|
||||||
|
install -Dm644 "$kernel" "$boot/homelab-installer/bzImage"
|
||||||
|
install -Dm644 "$initrd" "$boot/homelab-installer/initrd"
|
||||||
install -Dm644 "$iso" "$stagedir/homelab-installer.iso"
|
install -Dm644 "$iso" "$stagedir/homelab-installer.iso"
|
||||||
|
|
||||||
# findiso= is a path relative to whatever partition the initrd finds it on
|
# The ISO is built from a PUBLIC repo and deliberately carries no
|
||||||
# (it mounts every blkid-visible partition looking for it) — not necessarily
|
# credentials, so the host key has to travel with the staged installer or
|
||||||
# relative to `/`, if $stagedir is a subdirectory of a bigger filesystem
|
# the auto-install run has nothing to seed /etc/ssh with — and without that,
|
||||||
# rather than a mountpoint itself.
|
# sops can't decrypt on boot #1, /etc/shadow gets written once with a locked
|
||||||
mnt_point="$(findmnt -no TARGET --target "$stagedir")"
|
# darman, and no later `deploy switch` can fix it (README).
|
||||||
iso_relpath="${stagedir#"$mnt_point"}/homelab-installer.iso"
|
#
|
||||||
|
# $boot lives on the OS disk, so disko destroys this copy minutes later. The
|
||||||
|
# mode is advisory on vfat (permissions come from the mount's fmask, 0077 on
|
||||||
|
# a NixOS/systemd-boot ESP) — it is the wipe, not the mode, doing the work.
|
||||||
|
install -Dm600 "$hostkey" "$boot/homelab-installer/ssh_host_ed25519_key"
|
||||||
|
install -Dm644 "$hostkey.pub" "$boot/homelab-installer/ssh_host_ed25519_key.pub"
|
||||||
|
boot_src="$(findmnt -no SOURCE --nofsroot --target "$boot")" \
|
||||||
|
|| die "couldn't resolve $boot to a device"
|
||||||
|
boot_partuuid="$(lsblk -no PARTUUID "$boot_src" 2>/dev/null | head -1 | tr -d ' ' || true)"
|
||||||
|
[ -n "$boot_partuuid" ] \
|
||||||
|
|| die "couldn't read a PARTUUID for $boot ($boot_src) — the installer needs it to find the host key"
|
||||||
|
|
||||||
cat >/boot/loader/entries/homelab-installer.conf <<EOF
|
# findiso= is a path relative to whatever partition the initrd finds it on
|
||||||
|
# (it mounts every blkid-visible partition looking for it), not to `/`, if
|
||||||
|
# $stagedir is a subdirectory of a bigger filesystem rather than a mountpoint
|
||||||
|
# itself. It must KEEP its leading slash: stage-1 tests `-e /findiso$isoPath`,
|
||||||
|
# so a bare `var/tmp/x.iso` becomes `/findisovar/tmp/x.iso` and never matches.
|
||||||
|
# Prefixing then squeezing handles both ends: stagedir == the mountpoint
|
||||||
|
# (strip leaves "") and mnt_point == "/" (strip leaves a relative path).
|
||||||
|
mnt_point="$(findmnt -no TARGET --target "$stagedir")"
|
||||||
|
iso_relpath="$(printf '/%s/%s' "${stagedir#"$mnt_point"}" homelab-installer.iso | tr -s /)"
|
||||||
|
|
||||||
|
cat >"$boot/loader/entries/homelab-installer.conf" <<EOF
|
||||||
title Homelab Installer ($config, findiso)
|
title Homelab Installer ($config, findiso)
|
||||||
linux /homelab-installer/bzImage
|
linux /homelab-installer/bzImage
|
||||||
initrd /homelab-installer/initrd
|
initrd /homelab-installer/initrd
|
||||||
options nohibernate root=fstab loglevel=4 lsm=landlock,yama,bpf findiso=$iso_relpath homelab.install=$config
|
options nohibernate root=fstab loglevel=4 lsm=landlock,yama,bpf findiso=$iso_relpath homelab.install=$config homelab.keypart=$boot_partuuid
|
||||||
EOF
|
EOF
|
||||||
|
|
||||||
echo ">> one-shot boot into the installer, then rebooting — it will finish this install itself"
|
echo ">> one-shot boot into the installer, then rebooting — it will finish this install itself"
|
||||||
@@ -221,14 +346,20 @@ EOF
|
|||||||
# invisible to `nix build`/`nixos-install`, which then fails obscurely or builds
|
# invisible to `nix build`/`nixos-install`, which then fails obscurely or builds
|
||||||
# a stale config. Check before doing anything destructive.
|
# a stale config. Check before doing anything destructive.
|
||||||
require_tracked() {
|
require_tracked() {
|
||||||
local config="$1" cfgfile="hosts/$1/configuration.nix"
|
local config="$1" cfgfile="hosts/$1/configuration.nix" f
|
||||||
[ -e "$cfgfile" ] || die "no $cfgfile in the repo"
|
[ -e "$cfgfile" ] || die "no $cfgfile in the repo"
|
||||||
# No .git at all (e.g. a tarball export of the repo, no working tree) means
|
# No .git at all (e.g. a tarball export of the repo, no working tree), or no
|
||||||
# there's nothing that CAN be untracked — nothing to check. Only skip on a
|
# git binary, means there's nothing that CAN be untracked — nothing to check.
|
||||||
# MISSING .git, not on any other git failure.
|
# Only skip on that, not on any other git failure.
|
||||||
|
command -v git >/dev/null 2>&1 || return 0
|
||||||
git -C "$REPO" rev-parse --is-inside-work-tree >/dev/null 2>&1 || return 0
|
git -C "$REPO" rev-parse --is-inside-work-tree >/dev/null 2>&1 || return 0
|
||||||
git -C "$REPO" ls-files --error-unmatch "$cfgfile" >/dev/null 2>&1 \
|
# Every .nix in hosts/<config>/, not just configuration.nix: an untracked
|
||||||
|| die "$cfgfile is untracked — 'git add hosts/$config' first (flakes ignore untracked files)"
|
# disk-config.nix is exactly as invisible to the flake, and it is the file
|
||||||
|
# that decides which disk gets wiped.
|
||||||
|
for f in "hosts/$config"/*.nix; do
|
||||||
|
git -C "$REPO" ls-files --error-unmatch "$f" >/dev/null 2>&1 \
|
||||||
|
|| die "$f is untracked — 'git add hosts/$config' first (flakes ignore untracked files)"
|
||||||
|
done
|
||||||
}
|
}
|
||||||
|
|
||||||
# The password field of a Proton Pass item ("--field password" prints the bare
|
# The password field of a Proton Pass item ("--field password" prints the bare
|
||||||
@@ -469,9 +600,12 @@ case "$cmd" in
|
|||||||
;;
|
;;
|
||||||
|
|
||||||
install)
|
install)
|
||||||
config="${2:-}"; host="${3:-}"
|
config="${2:-}"; host="${3:-}"; assume_yes="${4:-}"
|
||||||
{ [ -n "$config" ] && [ -n "$host" ]; } || die "usage: ./deploy install <config> <host>"
|
{ [ -n "$config" ] && [ -n "$host" ]; } || die "usage: ./deploy install <config> <host> [--yes]"
|
||||||
hostkey="$HOME/.config/homelab/$config/ssh_host_ed25519_key"
|
# $KEYDIR, not a bare $HOME — see its definition. This same check runs
|
||||||
|
# inside installer-iso, where homelab-auto-install.service has no $HOME and
|
||||||
|
# has just dropped the key into /root/.config/homelab/<config>/.
|
||||||
|
hostkey="$KEYDIR/$config/ssh_host_ed25519_key"
|
||||||
[ -f "$hostkey" ] || die "missing host key: $hostkey"
|
[ -f "$hostkey" ] || die "missing host key: $hostkey"
|
||||||
[ -d "./hosts/$config" ] || die "no ./hosts/$config directory in the repo"
|
[ -d "./hosts/$config" ] || die "no ./hosts/$config directory in the repo"
|
||||||
require_tracked "$config"
|
require_tracked "$config"
|
||||||
@@ -481,7 +615,7 @@ case "$cmd" in
|
|||||||
# Not already inside a live installer: build one, stage it, one-shot
|
# Not already inside a live installer: build one, stage it, one-shot
|
||||||
# boot into it, and let it finish this exact command itself. See
|
# boot into it, and let it finish this exact command itself. See
|
||||||
# local_install_prepare_and_reboot above and CLAUDE.md.
|
# local_install_prepare_and_reboot above and CLAUDE.md.
|
||||||
local_install_prepare_and_reboot "$config"
|
local_install_prepare_and_reboot "$config" "$hostkey" "$assume_yes"
|
||||||
exit 0
|
exit 0
|
||||||
fi
|
fi
|
||||||
|
|
||||||
@@ -492,7 +626,10 @@ case "$cmd" in
|
|||||||
[ -f "./hosts/$config/disk-config.nix" ] || die "no ./hosts/$config/disk-config.nix"
|
[ -f "./hosts/$config/disk-config.nix" ] || die "no ./hosts/$config/disk-config.nix"
|
||||||
|
|
||||||
echo ">> disko .#$config onto this box's OS disk (WILL be wiped)"
|
echo ">> disko .#$config onto this box's OS disk (WILL be wiped)"
|
||||||
nix run github:nix-community/disko -- \
|
# `.#disko`, not github:nix-community/disko — the revision comes from this
|
||||||
|
# repo's flake.lock rather than upstream master-of-the-day, and resolves
|
||||||
|
# from the local store. See the nixos-anywhere input in flake.nix.
|
||||||
|
nix run ".#disko" -- \
|
||||||
--mode disko "./hosts/$config/disk-config.nix"
|
--mode disko "./hosts/$config/disk-config.nix"
|
||||||
|
|
||||||
echo ">> installing sops host key so it can decrypt on boot #1"
|
echo ">> installing sops host key so it can decrypt on boot #1"
|
||||||
@@ -524,11 +661,11 @@ case "$cmd" in
|
|||||||
echo ">> root ssh password from Proton Pass ($root_item)"
|
echo ">> root ssh password from Proton Pass ($root_item)"
|
||||||
export SSHPASS="$root_pw"
|
export SSHPASS="$root_pw"
|
||||||
unset root_pw
|
unset root_pw
|
||||||
nix run github:nix-community/nixos-anywhere -- \
|
nix run ".#nixos-anywhere" -- \
|
||||||
--env-password "${anywhere[@]}"
|
--env-password "${anywhere[@]}"
|
||||||
unset SSHPASS
|
unset SSHPASS
|
||||||
else
|
else
|
||||||
nix run github:nix-community/nixos-anywhere -- "${anywhere[@]}"
|
nix run ".#nixos-anywhere" -- "${anywhere[@]}"
|
||||||
fi
|
fi
|
||||||
fi
|
fi
|
||||||
;;
|
;;
|
||||||
@@ -600,7 +737,7 @@ case "$cmd" in
|
|||||||
# partition at /var/lib/sops-nix/age.txt so sops decrypts on first boot.
|
# partition at /var/lib/sops-nix/age.txt so sops decrypts on first boot.
|
||||||
# (The Pi's vfat partition isn't mounted at runtime, so the key can't live
|
# (The Pi's vfat partition isn't mounted at runtime, so the key can't live
|
||||||
# there.) Key stays off-repo, out of the nix store, and out of the image.
|
# there.) Key stays off-repo, out of the nix store, and out of the image.
|
||||||
keyfile="$HOME/.config/homelab/$config/age.txt"
|
keyfile="$KEYDIR/$config/age.txt"
|
||||||
if [ -f "$keyfile" ]; then
|
if [ -f "$keyfile" ]; then
|
||||||
echo ">> installing sops age key onto the root partition"
|
echo ">> installing sops age key onto the root partition"
|
||||||
sudo partprobe "$dev" 2>/dev/null || sudo blockdev --rereadpt "$dev" 2>/dev/null || true
|
sudo partprobe "$dev" 2>/dev/null || sudo blockdev --rereadpt "$dev" 2>/dev/null || true
|
||||||
@@ -613,10 +750,13 @@ case "$cmd" in
|
|||||||
| sort -rn | head -1 | cut -d' ' -f2)"
|
| sort -rn | head -1 | cut -d' ' -f2)"
|
||||||
[ -n "$rootpart" ] || die "no ext4 root partition found on $dev — place $keyfile at /var/lib/sops-nix/age.txt manually"
|
[ -n "$rootpart" ] || die "no ext4 root partition found on $dev — place $keyfile at /var/lib/sops-nix/age.txt manually"
|
||||||
mnt="$(mktemp -d)"
|
mnt="$(mktemp -d)"
|
||||||
|
# Unmount + remove even if the install fails, so a retry doesn't trip
|
||||||
|
# over the card still being mounted on a stale temp dir.
|
||||||
|
trap 'sudo umount "$mnt" 2>/dev/null || true; rmdir "$mnt" 2>/dev/null || true' EXIT
|
||||||
sudo mount "$rootpart" "$mnt"
|
sudo mount "$rootpart" "$mnt"
|
||||||
sudo install -Dm600 "$keyfile" "$mnt/var/lib/sops-nix/age.txt"
|
sudo install -Dm600 "$keyfile" "$mnt/var/lib/sops-nix/age.txt"
|
||||||
sudo sync
|
sudo sync
|
||||||
sudo umount "$mnt"; rmdir "$mnt"
|
sudo umount "$mnt"; rmdir "$mnt"; trap - EXIT
|
||||||
echo ">> age key installed (/var/lib/sops-nix/age.txt)"
|
echo ">> age key installed (/var/lib/sops-nix/age.txt)"
|
||||||
fi
|
fi
|
||||||
echo ">> done — insert the card into the Pi and boot."
|
echo ">> done — insert the card into the Pi and boot."
|
||||||
|
|||||||
Reference in New Issue
Block a user