deploy: harden kexec-local, key vault items by config, add VM test
kexec-local could never actually jump. nixos-images' kexec-run.sh ends with `nohup sh -c "sleep 6 && $SCRIPT_DIR/kexec -e" &` and returns immediately, so the EXIT trap's `rm -rf "$stage"` deleted the kexec binary out from under the sleeping shell. The box stayed on the old kernel and it looked like a slow boot. Clear the trap before jumping, verify /sys/kernel/kexec_loaded, then sleep past the timer. Preflight everything before the point of no return, since this jumps the machine you are typing at: CONFIG_KEXEC, kernel lockdown, exec-capable staging dir, free space, RAM vs image size, and that the tarball holds all five expected files. Stage on /var/tmp rather than /tmp because kexec-run.sh appends to initrd in place and execs from that directory. sync before jumping (kexec -e skips unmount). Confirmation prompt naming the host, since run in the wrong terminal this kexecs the laptop; --yes skips it. Drop the ssh-keygen -R added to the remote kexec path: kexec-run.sh copies /etc/ssh/ssh_host_* into the appended initrd and restore-remote-access.nix installs them back, so the host key survives the jump. Proton Pass items are now keyed by <config> instead of <host>, since the address is incidental and the config name is stable. kexec therefore takes <config> <host>. Resolve titles among --filter-state active items first: a trashed item with the same title shadowed the active one and returned an empty password, which is indistinguishable from "no entry" and silently fell back to prompting (hit on darman@neptun). Other fixes: replace `ls glob | head -1` (returns empty with exit 0 on no match) with a helper that dies; guard against untracked hosts/<config> since flakes ignore untracked files; feed the sudo password more than once under setsid; handle empty arrays under set -u; tolerate empty FSTYPE in the SD-card root partition lookup; preflight zstdcat/dd/lsblk before the destructive dd; list image and flash in the usage strings. Add checks.x86_64-linux.kexec-local, a VM test driving the real script. It is the only way to exercise kexec-local, which cannot be rehearsed on hardware. It asserts the box left the old kernel, returned as nixos-installer, lost its old /run, and kept its ssh host key. HOMELAB_KEXEC_TARBALL lets it reuse a prebuilt installer instead of building ~500MB inside the guest. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
Claude Opus 4.8
+324
-57
@@ -1,12 +1,27 @@
|
||||
#!/usr/bin/env bash
|
||||
# Deploy a NixOS host from this flake. ALL arguments are mandatory (no defaults).
|
||||
#
|
||||
# ./deploy kexec <host> headless kexec into a RAM installer, for a
|
||||
# read-only-root box (ZimaOS) where
|
||||
# nixos-anywhere can't ssh-copy-id. Ships our
|
||||
# SSH login key. Then run `install`.
|
||||
# ./deploy install <config> <host> first install (nixos-anywhere). Wipes the
|
||||
# OS disk. Ships the host's sops key.
|
||||
# ./deploy kexec <config> <host> headless kexec into a RAM installer, for a
|
||||
# read-only-root box (ZimaOS) where
|
||||
# nixos-anywhere can't ssh-copy-id. Ships our
|
||||
# SSH login key. Then run `install`. <config>
|
||||
# is only used to look up the vault item.
|
||||
# ./deploy kexec-local [--yes] kexec THIS machine into the RAM installer,
|
||||
# no ssh/second machine involved. Run as root,
|
||||
# locally, on the box you're installing onto.
|
||||
# Disks are untouched; console drops for
|
||||
# ~1-2 min then comes back as the installer.
|
||||
# Prompts for confirmation (--yes skips it),
|
||||
# because run on the wrong terminal this
|
||||
# kexecs your laptop. TMPDIR (default
|
||||
# /var/tmp) must be exec-capable and hold
|
||||
# ~3x the tarball.
|
||||
# Then run `install <config> localhost`.
|
||||
# ./deploy install <config> <host> first install. Wipes the OS disk. Ships the
|
||||
# host's sops key. <host>=localhost/127.0.0.1
|
||||
# skips nixos-anywhere/ssh and runs disko +
|
||||
# nixos-install directly against /mnt (use
|
||||
# after `kexec-local`, or on a live ISO).
|
||||
# ./deploy switch <config> <host> rebuild + activate on a running host.
|
||||
# ./deploy boot <config> <host> stage for next boot, don't activate now.
|
||||
# ./deploy test <config> <host> activate without adding a boot entry.
|
||||
@@ -22,11 +37,14 @@
|
||||
#
|
||||
# Password prompts are auto-filled from the "HomeLab" Proton Pass vault when
|
||||
# `pass-cli` is installed and logged in; otherwise every command prompts exactly
|
||||
# as before. Items:
|
||||
# as before. Both items are keyed by <config>, never by <host>: the address is
|
||||
# incidental (DHCP, a new box, localhost) while the config name is the stable
|
||||
# identity of the machine being built.
|
||||
# darman@<config> darman's sudo password (switch/boot/test)
|
||||
# root@<host> root's ssh password (kexec/install)
|
||||
# root@<config> root's ssh password (kexec/install)
|
||||
# Override with HOMELAB_PASS_ITEM / HOMELAB_PASS_ROOT_ITEM / HOMELAB_PASS_VAULT.
|
||||
set -euo pipefail
|
||||
shopt -s nullglob
|
||||
|
||||
# Locate the repo root (flake dir) regardless of where this script lives on disk.
|
||||
SCRIPT_DIR="$(cd "$(dirname "$(realpath "$0")")" && pwd)"
|
||||
@@ -36,16 +54,93 @@ export PATH="/nix/var/nix/profiles/default/bin:$PATH"
|
||||
|
||||
die() { echo "error: $*" >&2; exit 1; }
|
||||
|
||||
need() { command -v "$1" >/dev/null 2>&1 || die "missing required tool: $1"; }
|
||||
|
||||
# Exactly one path matching a glob, or die. `ls glob | head -1` silently yields
|
||||
# an empty string when nothing matches (head exits 0, so set -e never fires) and
|
||||
# the failure only surfaces later as a confusing tar/dd error.
|
||||
one_match() {
|
||||
local what="$1"; shift
|
||||
local f=("$@") # caller expands the glob (nullglob is on)
|
||||
[ "${#f[@]}" -gt 0 ] || die "no $what found — did the build actually produce one?"
|
||||
printf '%s\n' "${f[0]}"
|
||||
}
|
||||
|
||||
# Sets tb / cpio / bbox — the kexec tarball plus the static cpio+gzip that
|
||||
# kexec-run.sh needs on PATH to rebuild its initrd.
|
||||
#
|
||||
# HOMELAB_KEXEC_TARBALL (with _CPIO / _GZIP) short-circuits the build and uses a
|
||||
# prebuilt installer instead. That lets the VM test in flake.nix drive this
|
||||
# script offline, and lets you re-kexec a box without rebuilding ~500MB.
|
||||
kexec_artifacts() {
|
||||
if [ -n "${HOMELAB_KEXEC_TARBALL:-}" ]; then
|
||||
tb="$HOMELAB_KEXEC_TARBALL"
|
||||
[ -f "$tb" ] || die "HOMELAB_KEXEC_TARBALL=$tb is not a file"
|
||||
cpio="${HOMELAB_KEXEC_CPIO:-$(command -v cpio || true)}"
|
||||
bbox="${HOMELAB_KEXEC_GZIP:-$(command -v gzip || true)}"
|
||||
{ [ -n "$cpio" ] && [ -n "$bbox" ]; } \
|
||||
|| die "set HOMELAB_KEXEC_CPIO / HOMELAB_KEXEC_GZIP, or put cpio+gzip on PATH"
|
||||
echo ">> using prebuilt kexec installer: $tb"
|
||||
else
|
||||
need nix
|
||||
echo ">> building kexec installer + static tools"
|
||||
nix build .#nixosConfigurations.kexec.config.system.build.kexecInstallerTarball \
|
||||
-o result-kexec
|
||||
tb="$(one_match 'kexec tarball' result-kexec/*.tar.gz)"
|
||||
cpio="$(nix build --no-link --print-out-paths nixpkgs#pkgsStatic.cpio)/bin/cpio"
|
||||
bbox="$(nix build --no-link --print-out-paths nixpkgs#pkgsStatic.busybox)/bin/busybox"
|
||||
fi
|
||||
}
|
||||
|
||||
# Flakes only see git-tracked files: an untracked hosts/<config>/ is silently
|
||||
# invisible to `nix build`/`nixos-install`, which then fails obscurely or builds
|
||||
# a stale config. Check before doing anything destructive.
|
||||
require_tracked() {
|
||||
local config="$1" cfgfile="hosts/$1/configuration.nix"
|
||||
[ -e "$cfgfile" ] || die "no $cfgfile in the repo"
|
||||
git -C "$REPO" ls-files --error-unmatch "$cfgfile" >/dev/null 2>&1 \
|
||||
|| die "$cfgfile is untracked — 'git add hosts/$config' first (flakes ignore untracked files)"
|
||||
}
|
||||
|
||||
# The password field of a Proton Pass item ("--field password" prints the bare
|
||||
# value, one line), or empty if pass-cli is missing / logged out / has no such
|
||||
# item — every caller then falls back to the normal interactive prompt.
|
||||
#
|
||||
# Resolve the title to an item id among ACTIVE items first, because `item view
|
||||
# --item-title` has no state filter: Proton Pass keeps deleted items in the
|
||||
# trash, and if a trashed item shares the title, view can match THAT one and
|
||||
# return an empty password with exit 0. Empty is indistinguishable from "no such
|
||||
# item", so the only symptom is a silent fall back to the interactive prompt
|
||||
# even though the vault clearly holds the entry. (Hit for real on darman@neptun,
|
||||
# which had an Active and a Trashed copy.)
|
||||
proton_pass_password() {
|
||||
local title="$1"
|
||||
local title="$1" vault="${HOMELAB_PASS_VAULT:-HomeLab}" id pw
|
||||
command -v pass-cli >/dev/null 2>&1 || return 0
|
||||
pass-cli item view \
|
||||
--vault-name "${HOMELAB_PASS_VAULT:-HomeLab}" \
|
||||
--item-title "$title" \
|
||||
--field password --output human 2>/dev/null | head -1
|
||||
|
||||
# Lines look like: - [ITEM_ID]: the title (state=Active)
|
||||
id="$(pass-cli item list --vault-name "$vault" --filter-state active \
|
||||
--output human 2>/dev/null \
|
||||
| awk -v t="$title" '
|
||||
{ i = index($0, "]: "); if (i == 0) next
|
||||
id = substr($0, 4, i - 4)
|
||||
rest = substr($0, i + 3)
|
||||
sub(/ \(state=[^)]*\)$/, "", rest)
|
||||
if (rest == t) { print id; exit } }' || true)"
|
||||
|
||||
if [ -n "$id" ]; then
|
||||
pw="$(pass-cli item view --vault-name "$vault" --item-id "$id" \
|
||||
--field password --output human 2>/dev/null | head -1 || true)"
|
||||
# Resolved an active item but its password field is blank: worth saying so,
|
||||
# otherwise this looks identical to having no vault entry at all.
|
||||
[ -n "$pw" ] || echo ">> vault item '$title' resolved but its password field is empty" >&2
|
||||
else
|
||||
# No active match — fall back to the title lookup so an older pass-cli
|
||||
# without --filter-state still works exactly as it used to.
|
||||
pw="$(pass-cli item view --vault-name "$vault" --item-title "$title" \
|
||||
--field password --output human 2>/dev/null | head -1 || true)"
|
||||
fi
|
||||
|
||||
printf '%s' "$pw"
|
||||
}
|
||||
|
||||
# Path to an sshpass binary (system one, else built from nixpkgs). Empty if
|
||||
@@ -56,20 +151,18 @@ sshpass_bin() {
|
||||
| sed 's|$|/bin/sshpass|'
|
||||
}
|
||||
|
||||
cmd="${1:-}"; [ -n "$cmd" ] || die "usage: ./deploy <kexec|install|switch|boot|test> ..."
|
||||
cmd="${1:-}"; [ -n "$cmd" ] || die "usage: ./deploy <kexec|kexec-local|install|switch|boot|test|image|flash> ..."
|
||||
|
||||
case "$cmd" in
|
||||
kexec)
|
||||
host="${2:-}"; [ -n "$host" ] || die "usage: ./deploy kexec <host>"
|
||||
config="${2:-}"; host="${3:-}"
|
||||
{ [ -n "$config" ] && [ -n "$host" ]; } || die "usage: ./deploy kexec <config> <host>"
|
||||
|
||||
need ssh; need scp
|
||||
|
||||
echo ">> building kexec installer + static tools"
|
||||
nix build .#nixosConfigurations.kexec.config.system.build.kexecInstallerTarball \
|
||||
-o result-kexec
|
||||
tb="$(ls result-kexec/*.tar.gz | head -1)"
|
||||
# kexec/run rebuilds an initrd with `cpio` + `gzip` from PATH — ZimaOS lacks
|
||||
# both. Ship static ones: GNU cpio (reliable -o -H newc), busybox as gzip.
|
||||
cpio="$(nix build --no-link --print-out-paths nixpkgs#pkgsStatic.cpio)/bin/cpio"
|
||||
bbox="$(nix build --no-link --print-out-paths nixpkgs#pkgsStatic.busybox)/bin/busybox"
|
||||
kexec_artifacts
|
||||
|
||||
# One password prompt: multiplex scp + ssh over a shared control connection.
|
||||
cm="/tmp/homelab-cm-%r@%h:%p"
|
||||
@@ -78,13 +171,21 @@ case "$cmd" in
|
||||
|
||||
# Root's password from Proton Pass, fed to ssh/scp via sshpass -e. Only the
|
||||
# first (master) connection authenticates; the rest ride the control socket.
|
||||
# SSHPASS is exported into the sshpass child only, never onto a command line.
|
||||
#
|
||||
# SSHPASS is exported here rather than passed as `env SSHPASS=... sshpass`.
|
||||
# Both end up equally safe at rest: `env` execs its target immediately, so
|
||||
# the assignment is only in argv for the sub-millisecond before exec, after
|
||||
# which /proc/PID/cmdline reads plain `sshpass -e`. Exporting just closes
|
||||
# that race window and drops a process. Either way the secret lives in the
|
||||
# child's environ, which is readable by the owner and root only.
|
||||
sp=()
|
||||
root_pw="$(proton_pass_password "${HOMELAB_PASS_ROOT_ITEM:-root@$host}" || true)"
|
||||
root_item="${HOMELAB_PASS_ROOT_ITEM:-root@$config}"
|
||||
root_pw="$(proton_pass_password "$root_item" || true)"
|
||||
if [ -n "$root_pw" ]; then
|
||||
sshpass="$(sshpass_bin || true)"
|
||||
if [ -n "$sshpass" ]; then
|
||||
sp=(env "SSHPASS=$root_pw" "$sshpass" -e)
|
||||
export SSHPASS="$root_pw"
|
||||
sp=("$sshpass" -e)
|
||||
# sshpass drives the password prompt; don't let a key/agent short-circuit
|
||||
# into an interactive one for a host that only accepts passwords.
|
||||
o+=(-o PreferredAuthentications=password -o PubkeyAuthentication=no)
|
||||
@@ -92,26 +193,150 @@ case "$cmd" in
|
||||
echo ">> sshpass unavailable — falling back to the interactive prompt" >&2
|
||||
fi
|
||||
fi
|
||||
unset root_pw
|
||||
|
||||
if [ ${#sp[@]} -gt 0 ]; then
|
||||
echo ">> connecting to root@$host (password from Proton Pass)"
|
||||
if [ "${#sp[@]}" -gt 0 ]; then
|
||||
echo ">> connecting to root@$host (password from Proton Pass: $root_item)"
|
||||
else
|
||||
echo ">> connecting to root@$host (enter the root password once)"
|
||||
fi
|
||||
"${sp[@]}" ssh "${o[@]}" "root@$host" 'mkdir -p /tmp/bin'
|
||||
"${sp[@]+"${sp[@]}"}" ssh "${o[@]}" "root@$host" 'mkdir -p /tmp/bin'
|
||||
scp "${o[@]}" "$cpio" "root@$host:/tmp/bin/cpio"
|
||||
scp "${o[@]}" "$bbox" "root@$host:/tmp/bin/gzip" # busybox as gzip (argv0)
|
||||
unset root_pw
|
||||
|
||||
echo ">> streaming installer + kexec-ing. SSH drops as the box jumps into the"
|
||||
echo " RAM installer. Disks are untouched."
|
||||
rc=0
|
||||
ssh "${o[@]}" "root@$host" \
|
||||
'chmod +x /tmp/bin/*; mkdir -p /tmp/k && tar -C /tmp/k -xzf - && PATH=/tmp/bin:$PATH /tmp/k/kexec/run' \
|
||||
< "$tb" || true
|
||||
< "$tb" || rc=$?
|
||||
# A dropped connection IS the success case here, so this can't be fatal —
|
||||
# but a full /tmp, a tar error or a missing static tool looks identical from
|
||||
# the outside, so at least surface the code instead of swallowing it.
|
||||
[ "$rc" -eq 0 ] || echo ">> ssh exited $rc — expected if the box jumped; suspect this if install can't connect" >&2
|
||||
|
||||
ssh "${o[@]}" -O exit "root@$host" 2>/dev/null || true # close control socket
|
||||
unset SSHPASS
|
||||
|
||||
# NB: no ssh-keygen -R here on purpose. kexec-run.sh copies /etc/ssh/ssh_host_*
|
||||
# into the appended initrd and restore-remote-access.nix installs them back
|
||||
# into the installer's /etc/ssh, so the host key SURVIVES the jump. Clearing
|
||||
# known_hosts would just throw away the TOFU record for no reason.
|
||||
|
||||
echo ">> box is kexec-ing. Wait ~1-2 min for the installer + network, then:"
|
||||
echo " ./deploy install <config> $host"
|
||||
echo " ./deploy install $config $host"
|
||||
;;
|
||||
|
||||
kexec-local)
|
||||
# No ssh, no second machine: build the same RAM installer as `kexec`, but
|
||||
# run it directly on this box (you're sitting at it). The current shell
|
||||
# drops when the kernel switches, same as any reboot — that's expected,
|
||||
# not a failure. Disks are untouched; only the running kernel changes.
|
||||
#
|
||||
# This is a one-way trip on the machine you are typing at, so every check
|
||||
# that can fail is done BEFORE the point of no return, and nothing that the
|
||||
# jump depends on is cleaned up behind it (see the trap discussion below).
|
||||
[ "$(id -u)" = 0 ] || die "kexec-local must run as root (sudo ./deploy kexec-local)"
|
||||
|
||||
assume_yes=""
|
||||
[ "${2:-}" = "--yes" ] && assume_yes=1
|
||||
|
||||
need tar; need install; need mktemp; need find; need sync; need nohup
|
||||
|
||||
# --- preflight: reasons the jump would fail, checked while it's still safe --
|
||||
# CONFIG_KEXEC. Without it kexec --load fails and you've built ~1GB for
|
||||
# nothing; with lockdown it fails at load time too (Secure Boot blocks the
|
||||
# kexec_load syscall unless the image is signed).
|
||||
[ -e /sys/kernel/kexec_loaded ] \
|
||||
|| die "this kernel has no kexec support (CONFIG_KEXEC) — boot install media instead"
|
||||
if [ -r /sys/kernel/security/lockdown ] \
|
||||
&& ! grep -q '\[none\]' /sys/kernel/security/lockdown 2>/dev/null; then
|
||||
die "kernel lockdown is active — kexec_load is blocked. Disable Secure Boot, or boot install media."
|
||||
fi
|
||||
|
||||
kexec_artifacts
|
||||
|
||||
# Staging lives on /var/tmp, not /tmp: kexec-run.sh APPENDS a fresh cpio to
|
||||
# kexec/initrd in place and runs binaries out of this directory, so it needs
|
||||
# real space and exec permission. A tmpfs /tmp is both size-capped (ENOSPC
|
||||
# mid-append = a half-written initrd) and frequently noexec.
|
||||
stage="$(TMPDIR="${TMPDIR:-/var/tmp}" mktemp -d)"
|
||||
trap 'rm -rf "$stage"' EXIT
|
||||
|
||||
# noexec would only surface as a cryptic "Permission denied" on kexec/run.
|
||||
printf '#!/bin/sh\nexit 0\n' > "$stage/.execmove"
|
||||
chmod +x "$stage/.execmove"
|
||||
"$stage/.execmove" 2>/dev/null \
|
||||
|| die "$stage is mounted noexec — set TMPDIR to an exec-capable filesystem"
|
||||
rm -f "$stage/.execmove"
|
||||
|
||||
# Need room for the tarball, its extracted contents, and the appended cpio.
|
||||
tb_sz="$(stat -Lc %s "$tb")"
|
||||
avail="$(df -B1 --output=avail "$stage" | tail -1 | tr -d ' ')"
|
||||
[ "$avail" -ge $(( tb_sz * 3 )) ] \
|
||||
|| die "only $(( avail / 1024 / 1024 ))MB free on $stage, need ~$(( tb_sz * 3 / 1024 / 1024 ))MB — set TMPDIR elsewhere"
|
||||
|
||||
install -Dm755 "$cpio" "$stage/bin/cpio"
|
||||
install -Dm755 "$bbox" "$stage/bin/gzip" # busybox as gzip (argv0)
|
||||
mkdir -p "$stage/k" && tar -C "$stage/k" -xzf "$tb"
|
||||
|
||||
# Fail loudly here rather than with a bare "not found" from kexec/run.
|
||||
for f in run kexec bzImage initrd ip; do
|
||||
[ -f "$stage/k/kexec/$f" ] || die "kexec tarball is missing kexec/$f — bad build?"
|
||||
done
|
||||
|
||||
# The installer root is a RAM disk: the initrd has to fit in memory with
|
||||
# room to unpack. Refuse rather than OOM halfway into the new kernel, where
|
||||
# there is no way back except the power button.
|
||||
img_sz="$(( $(stat -Lc %s "$stage/k/kexec/initrd") + $(stat -Lc %s "$stage/k/kexec/bzImage") ))"
|
||||
mem_kb="$(awk '/^MemTotal:/{print $2}' /proc/meminfo)"
|
||||
[ "$(( mem_kb * 1024 ))" -ge "$(( img_sz * 3 ))" ] \
|
||||
|| die "only $(( mem_kb / 1024 ))MB RAM for a $(( img_sz / 1024 / 1024 ))MB RAM-disk installer — too tight, boot install media instead"
|
||||
|
||||
# --- confirmation: this script normally runs on the LAPTOP ------------------
|
||||
# A stray `kexec-local` here would jump the machine you develop on, not the
|
||||
# target. Name it explicitly so a wrong-terminal mistake is visible.
|
||||
echo ">> about to kexec THIS machine:"
|
||||
echo " hostname: $(uname -n)"
|
||||
echo " kernel: $(uname -r)"
|
||||
echo " root: $(findmnt -no SOURCE / 2>/dev/null || echo '?')"
|
||||
echo " Disks are untouched; only the running kernel changes. Console drops"
|
||||
echo " for ~1-2 min, then comes back as the installer."
|
||||
if [ -z "$assume_yes" ]; then
|
||||
read -rp ">> type 'yes' to kexec $(uname -n) into the RAM installer: " ok
|
||||
[ "$ok" = yes ] || die "aborted"
|
||||
fi
|
||||
|
||||
# kexec -e jumps straight to the new kernel: no unmount, no journal flush,
|
||||
# no systemd shutdown. Anything still in page cache is lost and the OS disk
|
||||
# is left dirty. Cheap insurance, since the box may yet be rebooted back.
|
||||
sync
|
||||
|
||||
echo ">> loading the new kernel"
|
||||
if ! PATH="$stage/bin:$PATH" "$stage/k/kexec/run"; then
|
||||
rm -rf "$stage"
|
||||
die "kexec/run failed — machine untouched, still on the old kernel"
|
||||
fi
|
||||
|
||||
# kexec --load succeeded iff the kernel now reports a loaded image. If this
|
||||
# is 0 the background `kexec -e` below will do nothing and we'd hang forever
|
||||
# waiting for a jump that cannot happen.
|
||||
[ "$(cat /sys/kernel/kexec_loaded 2>/dev/null || echo 0)" = 1 ] \
|
||||
|| { rm -rf "$stage"; die "kexec reported success but no image is loaded — aborting"; }
|
||||
|
||||
# THE trap MUST GO NOW. kexec-run.sh backgrounds `nohup sh -c "sleep 6 &&
|
||||
# $SCRIPT_DIR/kexec -e"` and returns immediately, so the binary that
|
||||
# performs the jump still has to exist ~6s after this script would normally
|
||||
# exit. Letting the EXIT trap rm -rf "$stage" deletes it out from under that
|
||||
# sleeping shell and the machine silently never jumps.
|
||||
trap - EXIT
|
||||
|
||||
sync
|
||||
echo ">> kernel loaded; jumping in ~6s. (staging left at $stage on purpose —"
|
||||
echo " the backgrounded kexec still needs it; it's gone after the jump.)"
|
||||
# Outlive the sleep 6 so the jump happens while this script is still alive.
|
||||
sleep 60
|
||||
die "still here after 60s — the jump did not happen. Check dmesg; the old system is intact."
|
||||
;;
|
||||
|
||||
install)
|
||||
@@ -120,35 +345,61 @@ case "$cmd" in
|
||||
hostkey="$HOME/.config/homelab/$config/ssh_host_ed25519_key"
|
||||
[ -f "$hostkey" ] || die "missing host key: $hostkey"
|
||||
[ -d "./hosts/$config" ] || die "no ./hosts/$config directory in the repo"
|
||||
require_tracked "$config"
|
||||
|
||||
# Stage the pre-generated SSH host key so sops can decrypt on boot #1.
|
||||
stage="$(mktemp -d)"
|
||||
trap 'rm -rf "$stage"' EXIT
|
||||
install -Dm600 "$hostkey" "$stage/etc/ssh/ssh_host_ed25519_key"
|
||||
install -Dm644 "$hostkey.pub" "$stage/etc/ssh/ssh_host_ed25519_key.pub"
|
||||
if [ "$host" = "localhost" ] || [ "$host" = "127.0.0.1" ]; then
|
||||
# Local install: no ssh, no nixos-anywhere. Run after `kexec-local` (or
|
||||
# from a live ISO) so /mnt is free to wipe — this IS the box, no second
|
||||
# machine in the loop, so skip straight to disko + nixos-install.
|
||||
[ "$(id -u)" = 0 ] || die "local install must run as root"
|
||||
[ -f "./hosts/$config/disk-config.nix" ] || die "no ./hosts/$config/disk-config.nix"
|
||||
|
||||
echo ">> nixos-anywhere .#$config onto root@$host (OS disk WILL be wiped)"
|
||||
anywhere=(--flake ".#$config"
|
||||
--extra-files "$stage"
|
||||
--generate-hardware-config nixos-generate-config "./hosts/$config/hardware-configuration.nix"
|
||||
--target-host "root@$host")
|
||||
echo ">> disko .#$config onto this box's OS disk (WILL be wiped)"
|
||||
nix run github:nix-community/disko -- \
|
||||
--mode disko "./hosts/$config/disk-config.nix"
|
||||
|
||||
# nixos-anywhere's --env-password reads root's ssh password from $SSHPASS
|
||||
# (it ships its own sshpass), so a vault hit skips the ssh-copy-id prompt.
|
||||
root_pw="$(proton_pass_password "${HOMELAB_PASS_ROOT_ITEM:-root@$host}" || true)"
|
||||
if [ -n "$root_pw" ]; then
|
||||
echo ">> root ssh password from Proton Pass"
|
||||
env "SSHPASS=$root_pw" nix run github:nix-community/nixos-anywhere -- \
|
||||
--env-password "${anywhere[@]}"
|
||||
echo ">> installing sops host key so it can decrypt on boot #1"
|
||||
install -Dm600 "$hostkey" /mnt/etc/ssh/ssh_host_ed25519_key
|
||||
install -Dm644 "$hostkey.pub" /mnt/etc/ssh/ssh_host_ed25519_key.pub
|
||||
|
||||
echo ">> nixos-install .#$config into /mnt"
|
||||
nixos-install --root /mnt --flake ".#$config"
|
||||
else
|
||||
nix run github:nix-community/nixos-anywhere -- "${anywhere[@]}"
|
||||
# Stage the pre-generated SSH host key so sops can decrypt on boot #1.
|
||||
stage="$(mktemp -d)"
|
||||
trap 'rm -rf "$stage"' EXIT
|
||||
install -Dm600 "$hostkey" "$stage/etc/ssh/ssh_host_ed25519_key"
|
||||
install -Dm644 "$hostkey.pub" "$stage/etc/ssh/ssh_host_ed25519_key.pub"
|
||||
|
||||
echo ">> nixos-anywhere .#$config onto root@$host (OS disk WILL be wiped)"
|
||||
anywhere=(--flake ".#$config"
|
||||
--extra-files "$stage"
|
||||
--generate-hardware-config nixos-generate-config "./hosts/$config/hardware-configuration.nix"
|
||||
--target-host "root@$host")
|
||||
|
||||
# nixos-anywhere's --env-password reads root's ssh password from $SSHPASS
|
||||
# (it ships its own sshpass), so a vault hit skips the ssh-copy-id prompt.
|
||||
# Exported rather than `env SSHPASS=...` for consistency with `kexec`;
|
||||
# see the note there — it's a marginal win, not a leak fix.
|
||||
root_item="${HOMELAB_PASS_ROOT_ITEM:-root@$config}"
|
||||
root_pw="$(proton_pass_password "$root_item" || true)"
|
||||
if [ -n "$root_pw" ]; then
|
||||
echo ">> root ssh password from Proton Pass ($root_item)"
|
||||
export SSHPASS="$root_pw"
|
||||
unset root_pw
|
||||
nix run github:nix-community/nixos-anywhere -- \
|
||||
--env-password "${anywhere[@]}"
|
||||
unset SSHPASS
|
||||
else
|
||||
nix run github:nix-community/nixos-anywhere -- "${anywhere[@]}"
|
||||
fi
|
||||
fi
|
||||
unset root_pw
|
||||
;;
|
||||
|
||||
switch|boot|test)
|
||||
config="${2:-}"; host="${3:-}"
|
||||
{ [ -n "$config" ] && [ -n "$host" ]; } || die "usage: ./deploy $cmd <config> <host>"
|
||||
require_tracked "$config"
|
||||
|
||||
echo ">> nixos-rebuild $cmd .#$config on darman@$host"
|
||||
# --ask-sudo-password, not the deprecated --use-remote-sudo: common.nix sets
|
||||
@@ -167,8 +418,14 @@ case "$cmd" in
|
||||
# piped stdin. setsid drops the controlling terminal, so getpass falls back
|
||||
# to stdin and takes the vault password (it warns about echo — harmless,
|
||||
# nothing is echoed since the password never reaches the terminal).
|
||||
#
|
||||
# Caveat of dropping the tty: EVERY prompt in the subtree now reads this
|
||||
# stdin, not just the sudo one. Feed the line a few times so a retry or a
|
||||
# second sudo ask doesn't hit EOF and hang. Anything else that prompts
|
||||
# (an ssh key passphrase, a host-key confirmation) will still fail — fix
|
||||
# those out of band rather than by feeding more lines here.
|
||||
echo ">> sudo password from Proton Pass ($item)"
|
||||
printf '%s\n' "$pw" | setsid -w "${rebuild[@]}"
|
||||
printf '%s\n%s\n%s\n' "$pw" "$pw" "$pw" | setsid -w "${rebuild[@]}"
|
||||
else
|
||||
"${rebuild[@]}"
|
||||
fi
|
||||
@@ -176,16 +433,22 @@ case "$cmd" in
|
||||
;;
|
||||
|
||||
image|flash)
|
||||
config="${2:-}"; [ -n "$config" ] || die "usage: ./deploy $cmd <config> [<dev>]"
|
||||
# Validate the device BEFORE building, so a bad `flash` fails fast.
|
||||
if [ "$cmd" = flash ]; then
|
||||
dev="${3:-}"; [ -n "$dev" ] || die "usage: ./deploy flash <config> <dev> (e.g. /dev/sdX)"
|
||||
config="${2:-}"; dev="${3:-}"
|
||||
{ [ -n "$config" ] && [ -n "$dev" ]; } \
|
||||
|| die "usage: ./deploy flash <config> <dev> (e.g. /dev/sdX)"
|
||||
# Validate the device and the tools BEFORE building, so a bad `flash`
|
||||
# fails fast instead of dying halfway through writing the card.
|
||||
[ -b "$dev" ] || die "$dev is not a block device"
|
||||
need zstdcat; need dd; need lsblk
|
||||
else
|
||||
config="${2:-}"; [ -n "$config" ] || die "usage: ./deploy image <config>"
|
||||
fi
|
||||
require_tracked "$config"
|
||||
|
||||
echo ">> building SD image for .#$config (aarch64 needs qemu binfmt)"
|
||||
nix build ".#nixosConfigurations.$config.config.system.build.sdImage" -o result-sd
|
||||
img="$(ls result-sd/sd-image/*.img.zst | head -1)"
|
||||
img="$(one_match 'SD image' result-sd/sd-image/*.img.zst)"
|
||||
echo ">> image: $img"
|
||||
[ "$cmd" = image ] && exit 0
|
||||
|
||||
@@ -205,8 +468,12 @@ case "$cmd" in
|
||||
echo ">> installing sops age key onto the root partition"
|
||||
sudo partprobe "$dev" 2>/dev/null || sudo blockdev --rereadpt "$dev" 2>/dev/null || true
|
||||
sudo udevadm settle 2>/dev/null || true
|
||||
# largest ext4 partition = the NixOS root.
|
||||
rootpart="$(lsblk -blno PATH,FSTYPE,SIZE "$dev" | awk '$2=="ext4"{print $3, $1}' | sort -rn | head -1 | awk '{print $2}')"
|
||||
# Largest ext4 partition = the NixOS root. -P (KEY="value" pairs) instead
|
||||
# of a columnar listing: an empty FSTYPE collapses under whitespace
|
||||
# splitting and shifts every later field.
|
||||
rootpart="$(lsblk -bPo PATH,FSTYPE,SIZE "$dev" \
|
||||
| sed -n 's/^PATH="\([^"]*\)" FSTYPE="ext4" SIZE="\([0-9]*\)"$/\2 \1/p' \
|
||||
| sort -rn | head -1 | cut -d' ' -f2)"
|
||||
[ -n "$rootpart" ] || die "no ext4 root partition found on $dev — place $keyfile at /var/lib/sops-nix/age.txt manually"
|
||||
mnt="$(mktemp -d)"
|
||||
sudo mount "$rootpart" "$mnt"
|
||||
@@ -219,6 +486,6 @@ case "$cmd" in
|
||||
;;
|
||||
|
||||
*)
|
||||
die "unknown command '$cmd' (kexec|install|switch|boot|test)"
|
||||
die "unknown command '$cmd' (kexec|kexec-local|install|switch|boot|test|image|flash)"
|
||||
;;
|
||||
esac
|
||||
|
||||
Reference in New Issue
Block a user