Files
homelab/CLAUDE.md
T
darmanandClaude Opus 5 d089743f75 feat(terra): talk to luna with push-to-talk
luna-voiced: mic -> whisper-cpp (local, vulkan) -> Hermes' API on mars ->
ElevenLabs streaming TTS -> speaker, bound to SUPER+V. State, amplitude
and a viseme timeline are published on a unix socket; nothing consumes
that yet, but it is the data source the quickshell avatar will read.

STT stays local on purpose. mars already runs faster-whisper for Telegram
voice notes, but routing the mic through it would put every spoken word
on the network for no latency win.

Shape notes, all of them things the first live runs decided:

  - Viseme offsets are relative to speech_start (emitted when the first
    PCM frame reaches the speaker), never wall clock, so the shell and
    the daemon never have to compare clocks. ElevenLabs sends character
    alignment, not visemes, so the mapping is ours; the glTF morph target
    names are the contract.
  - clean_for_speech is a whitelist. She writes "(¬‿¬)", whose characters
    live in Latin-1 and punctuation blocks rather than any emoji range,
    so enumerating what to drop never converges. Em dashes become commas
    because they are a pause — dropping one ran two clauses together and
    changed what a sentence meant.
  - whisper invents words out of silence ("you", "Thank you."), and a
    held key over a quiet room is long AND empty, so has_speech gates on
    the loudest 100ms window rather than on duration.
  - Beeps on key down/up and a distinct low tone for "heard nothing":
    push-to-talk with no acknowledgement is indistinguishable from broken,
    and a dead mic must not feel like a slow reply.
  - A warmup request at startup: her first reply after the container
    starts costs ~16s against ~5s warm, and the first one is always the
    one you are waiting for.
  - `luna-voice watch` prints the socket protocol and `luna-voice say`
    runs a turn from typed text, which is how a broken mic is told apart
    from a broken pipeline.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-20 22:45:17 +02:00

16 KiB
Raw Blame History

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Flake-based NixOS config for a homelab. Hosts: jupiter (ZimaBlade NAS, x86_64), neptun (netcup public reverse proxy + tailnet node, x86_64), mercury (Raspberry Pi 3B+ DNS/DHCP, aarch64). See README.md for the full install/deploy walkthrough.

Layout

flake.nix          # nixosConfigurations: real hosts + test/util targets
common.nix         # shared base: user darman (key-only ssh), nix settings, firewall :22, tz
services/<cat>/*.nix   # one reusable NixOS module per service, grouped by category
                       # (media, network, vpn, identity, dev, desktop); each opens
                       # ITS OWN firewall ports. services/containers.nix (podman
                       # backend) stays at the top level, shared across categories.
hosts/<h>/         # configuration.nix + disk-config.nix (disko) + hardware-configuration.nix + secrets.nix
secrets/<h>.yaml   # sops-nix, age-encrypted per host
scripts/deploy     # config-agnostic deploy wrapper (all args mandatory)
scripts/edit_secrets
.sops.yaml         # per-host encryption rules (admin key + each host's key)

A host = common.nix + the services/** modules it imports + its hosts/<h>/configuration.nix. services/ modules are engine-agnostic and shared across hosts (e.g. services/vpn/tailscale.nix, services/network/caddy.nix used by jupiter and neptun).

Commands

Eval/verify a config before deploying (eval only checks the module tree, not freeform config like pihole's TOML or a container's runtime):

nix eval --raw .#nixosConfigurations.<host>.config.system.build.toplevel.drvPath

Deploy (from a non-NixOS laptop too — runs nixos-rebuild/nixos-anywhere via nix run):

./scripts/deploy switch  <config> <host>     # daily rebuild + activate
./scripts/deploy install <config> <host>     # first install (nixos-anywhere, wipes OS disk)
./scripts/deploy kexec   <config> <host>     # RO-root box (ZimaOS): kexec into a RAM installer first
sudo ./scripts/deploy kexec-local [--yes]    # kexec THIS box (no ssh); confirm prompt unless --yes
./scripts/deploy image   mercury             # build the aarch64 SD image
./scripts/deploy flash   mercury /dev/sdX    # build + write SD + drop the sops age key

Both password prompts are auto-filled from the "HomeLab" Proton Pass vault, keyed by <config>, not <host> — items darman@<config> (sudo) and root@<config> (ssh). A trashed Proton Pass item with the same title shadows the active one and yields an empty password, so the script resolves the title among --filter-state active items first; a plain pass-cli item view --item-title silently returns the trashed copy and you get an interactive prompt with no explanation.

HOMELAB_KEXEC_TARBALL (+ _CPIO / _GZIP) makes kexec/kexec-local reuse a prebuilt installer instead of rebuilding ~500MB. The VM test below uses this.

Secrets (needs the admin age key at ~/.config/sops/age/keys.txt):

./scripts/edit_secrets secrets/<host>.yaml

Claude: never run sops --decrypt/edit_secrets --show and print the result — that puts every plaintext secret in the file into the conversation transcript, not just the one you wanted. To add or change a single value non-interactively, use sops --set '["key"] "value"' secrets/<host>.yaml (quote the value as JSON), which writes without ever displaying the file's existing contents.

Test a service config BEFORE touching hardware — always do this for nontrivial changes:

# x86 QEMU VM of mercury's DNS/DHCP stack (fast; validates pihole/unbound at runtime)
nix build .#nixosConfigurations.mercury-vm.config.system.build.vm -o result
./result/bin/run-mercury-vm-vm            # ssh -p 2223 darman@localhost (pw: test)
# jupiter services as a VirtualBox OVA
nix build .#nixosConfigurations.jupiter-vbox.config.system.build.virtualBoxOVA
# end-to-end VM test of `deploy kexec-local` (~45s once the tarball is built)
nix build .#checks.x86_64-linux.kexec-local -L
# VM test of luna's app hosting on mars (hosts/mars/luna-sites.nix): podman socket
# proxy, registry validation, caddy routes, reboot persistence
nix build .#checks.x86_64-linux.luna-sites -L

checks.kexec-local is the only way to exercise kexec-local at all: it jumps the machine you are typing at, so it cannot be rehearsed on real hardware and a failure looks exactly like a slow boot. It asserts the box actually left the old kernel (SSH drops then returns), came back as nixos-installer, lost its old /run, and kept its ssh host key. Run it after ANY change to the kexec paths.

Secrets (sops-nix)

  • Each secrets/<host>.yaml is encrypted to the admin key (edit) + that host's key (runtime decrypt); rules in .sops.yaml. Private keys live OFF-repo: ~/.config/sops/age/keys.txt (admin), ~/.config/homelab/<host>/ (host keys).
  • jupiter/neptun decrypt with their ssh host key (ssh-to-age recipient), shipped at install via nixos-anywhere --extra-files.
  • mercury (SD image, no --extra-files) uses a dedicated age key at /var/lib/sops-nix/age.txt — ./scripts/deploy flash writes it to the ext4 root partition.
  • A service password that must come from sops but whose module has no passwordFile hook (pihole, adguard) is injected via sops.templates → an env file → the service (FTLCONF_* for pihole). See hosts/mercury/configuration.nix.

Non-obvious gotchas (all learned the hard way)

  • aarch64 (mercury): the x86 laptop needs extra-platforms = aarch64-linux in /etc/nix/nix.custom.conf (NOT /etc/nix/nix.conf — Determinate Nix regenerates that)
    • qemu-user-static-binfmt, else emulated builds fail with "platform mismatch". Or build on the Pi with --build-host darman@<ip>.
  • pihole on mercury is a CONTAINER (services/network/pihole.nix, official image, host networking, caps NET_ADMIN/NET_RAW/SYS_NICE/CHOWN, FTLCONF_* env config). The native services.pihole-ftl module segfaults on the Pi 3B+ aarch64 — do not switch back.
  • services.unbound.resolveLocalQueries = false is required: unbound listens on :5335, so leaving it true points the host's resolv.conf at 127.0.0.1:53 with nothing there → boot-time DNS deadlock (starves image pulls / list downloads). Host resolves via upstream networking.nameservers; pihole forwards to unbound explicitly at 127.0.0.1#5335.
  • Remote deploy pushes unsigned closures: hosts set nix.settings.trusted-users = [ "root" "@wheel" ] (in common.nix) so a laptop-built closure is accepted by the target.
  • jupiter: boot.kernelParams = [ "reboot=pci" ] (warm reboot hangs on that board); eMMC initrd modules pinned in configuration.nix (generate-config misses them); the 16TB×2 RAID0 data lives on /mnt/data with nofail, kept OUT of disko (never wiped).
  • terra: ./scripts/deploy kexec-local hangs hard — do not use it there. Confirmed on real hardware: kexec's device_shutdown() pass runs (SCSI disks sync fine in the log), then the machine goes dark and never comes back — journalctl --list-boots showed a ~15min gap before the next boot, i.e. a genuine hang needing a manual power cycle, not a slow jump. Near-certainly amdgpu (RX 6800 XT): discrete AMD GPUs are known to hang during kexec's device-shutdown pass with no clean handoff before the jump — same class of issue as jupiter's reboot=pci workaround, just fatal here instead of slow. Use ./scripts/deploy install terra localhost instead (README's "First install on terra" section) — it detects it isn't inside a live installer yet and reboots via a real systemctl reboot + systemd-boot one-shot findiso= entry, not kexec.
  • ./scripts/deploy install <config> localhost's behavior depends on uname -n (is_live_installer()): on a real running OS it builds installer-iso, stages it locally, and reboots into it (local_install_prepare_and_reboot()); only inside nixos-installer (kexec) or homelab-installer (installer-iso) does it actually run disko + nixos-install. installer-iso's homelab-auto-install.service closes the loop: it reads homelab.install=<config> back off /proc/cmdline (set by the prepare step) and re-runs the identical command itself once homelab-checkout.service has cloned the repo — the whole reinstall is one command and unattended after the first reboot. It confirms (type yes) before rebooting, like flash/kexec-local; --yes skips that and is what the ISO passes itself. It always ASKS where to stage the iso file (never auto-picks — the wrong disk here is destroyed mid-install); HOMELAB_INSTALLER_STAGE_DIR skips the prompt for scripted use. Both this and kexec-local self-elevate via sudo (require_root()) rather than requiring you to prefix the command yourself.
  • terra runs Limine, not systemd-boot — bootctl set-oneshot is useless there (bootctl status lists ✗ One-shot entry control, and CachyOS's pacman hooks regenerate limine.conf anyway). local_install_prepare_and_reboot() therefore picks its one-shot mechanism at runtime: a systemd-boot loader entry when $BOOT/loader/entries exists, otherwise arm_efi_bootnext() — a temporary UEFI entry that EFI-stub-boots the kernel off the ESP, armed via the firmware's BootNext. Created with efibootmgr --create-only (NOT --create, which pushes it to the front of BootOrder and would make a wiped installer the permanent default if anything went wrong). BootNext is spent by that one boot, so a failed attempt still comes back on the normal bootloader. The EFI-stub path needs the kernel on the ESP itself, not on a separate XBOOTLDR — hence --print-esp-path rather than --print-boot-path in that mode. homelab-auto-install.service deletes the leftover NVRAM entry as soon as it boots; both it and the script match the label Homelab Installer EXACTLY (a prefix match would delete the Windows or Limine entry).
  • installer-iso must force boot.initrd.systemd.enable = false. findiso= is a SCRIPT-stage-1 feature (stage-1-init.sh loop-mounts the file it points at and symlinks it to /dev/root). The systemd initrd — default since 26.05 — has NO findiso path: it mounts /iso straight from /dev/disk/by-label/<volumeID> (iso-image.nix), a label that only exists when the ISO is the physical boot medium. EFI-stub-booted off the ESP with the iso as a plain file on another fs, that label never appears; stage 1 mounts /sysroot fine, then times out on /sysroot/nix/.ro-store waiting for /dev/disk/by-label/nixos-minimal-… and drops to an emergency shell. The whole install <config> localhost findiso path depends on script stage 1.
  • The staging-dir guard must fail CLOSED, and findmnt needs --nofsroot: on btrfs findmnt -no SOURCE prints /dev/sdb2[/@], which lsblk cannot open, so a naive parent-device lookup comes back empty. Treating empty as "different disk" silently allowed staging the iso on the very disk disko then wiped — terra's CachyOS root is exactly that layout, so it hit the live case. disks_backing() (lsblk -rnso NAME,TYPE) returns EVERY whole-disk ancestor because LVM/RAID can span several (/mnt/ssd_01 → sdd + sde), and an empty result is a hard error, not a pass. btrfs staging is refused outright: stage-1 mounts a btrfs volume's TOP level, so an iso inside a subvolume is unreachable via findiso=.
  • findiso= must keep its leading slash: stage-1 tests [ -e "/findiso$isoPath" ], so stripping the mountpoint prefix off a stagedir whose mountpoint is / yields var/tmp/x.iso → /findisovar/tmp/x.iso → emergency shell, after you have already rebooted out of the working OS.
  • The auto-install needs the host key shipped to it, and a $HOME: the ISO is built from a public repo with no credentials, so local_install_prepare_and_reboot() copies the key onto the boot partition and passes that partition's PARTUUID as homelab.keypart=; the service mounts it and drops the key in /root/.config/homelab/<config>/ before running the install. Also, systemd does NOT set $HOME for a system service without User= (SetLoginEnvironment= defaults to false), and scripts/deploy runs under set -u — hence $KEYDIR instead of a bare $HOME, plus Environment=HOME=/root on the unit.
  • installer-iso must enable experimental-features itself. installation-cd-minimal leaves them unset, so nix run and nixos-install --flake both die with "experimental Nix feature 'nix-command' is disabled". The nixos-images kexec installer sets them itself, which is why the same install <config> localhost branch worked after kexec-local but not from the ISO.
  • disko/nixos-anywhere run as nix run .#disko / .#nixos-anywhere, from this flake's locked inputs — not nix run github:.... They execute while a disk is being wiped, so the revision must be the reviewed one in flake.lock, and it has to resolve without network.
  • disko wipes only the OS disk named in hosts/<h>/disk-config.nix; data disks are plain fileSystems in configuration.nix.
  • nixos-anywhere/kexec needs a writable root; ZimaOS root is read-only, hence the ./scripts/deploy kexec step that streams a RAM installer (with static cpio/gzip since ZimaOS lacks them).
  • kexec/run jumps ~6s AFTER it returns: nixos-images' kexec-run.sh ends with nohup sh -c "sleep 6 && $SCRIPT_DIR/kexec -e" &. So the staging dir must OUTLIVE the script — an rm -rf in an EXIT trap deletes the binary that performs the jump and the box silently stays on the old kernel. kexec-local clears its trap before jumping and then sleeps 60s on purpose. Covered by checks.kexec-local.
  • The kexec installer KEEPS the box's ssh host key: kexec-run.sh copies /etc/ssh/ssh_host_* into the appended initrd and restore-remote-access.nix installs them back. So do NOT ssh-keygen -R after a kexec — the key does not change, and clearing it just throws away the known_hosts record.
  • kexec-local stages on /var/tmp, not /tmp: kexec-run.sh appends a fresh cpio to kexec/initrd in place and execs binaries from that dir, so a size-capped or noexec tmpfs gives a half-written initrd or a bare "Permission denied".
  • A sops secret a USER service reads needs owner, not a mode. luna-voiced (services/desktop/luna-voice.nix) runs as darman's user unit, and a user manager cannot read root:root 0400 — LoadCredential does not bridge that gap either. Hence sops.secrets.<x>.owner = "darman" in hosts/terra/secrets.nix.
  • elevenlabs_api_key is in BOTH secrets/mars.yaml and secrets/terra.yaml — same value. terra calls ElevenLabs directly (the client-direct path Hermes documents) rather than relaying audio through mars, so rotating that key means editing two files. hermes_api_server_key is duplicated for the same reason: mars serves the API, terra is its only client.
  • whisper invents words out of silence — "you", "Thank you.", "Bye." — and a held push-to-talk key over a quiet room is long AND empty, so duration alone does not catch it. stt::has_speech gates on the loudest 100ms window instead; without it every stray keypress sends a hallucinated sentence to luna.
  • terra's greeter is a throwaway Hyprland running dotfiles/quickshell/greeter.qml (services/desktop/quickshell-greeter.nix). It must exit after login or greetd never starts the session, and with a Lua config hyprctl dispatch exit is REJECTED — it needs hyprctl dispatch 'hl.dsp.exit()'. Test the flow without touching the real greetd by running greetd's fakegreet "qs -p …/greeter.qml" inside a nested Hyprland (user user, password password, then answer 9).