luna-voiced: mic -> whisper-cpp (local, vulkan) -> Hermes' API on mars ->
ElevenLabs streaming TTS -> speaker, bound to SUPER+V. State, amplitude
and a viseme timeline are published on a unix socket; nothing consumes
that yet, but it is the data source the quickshell avatar will read.
STT stays local on purpose. mars already runs faster-whisper for Telegram
voice notes, but routing the mic through it would put every spoken word
on the network for no latency win.
Shape notes, all of them things the first live runs decided:
- Viseme offsets are relative to speech_start (emitted when the first
PCM frame reaches the speaker), never wall clock, so the shell and
the daemon never have to compare clocks. ElevenLabs sends character
alignment, not visemes, so the mapping is ours; the glTF morph target
names are the contract.
- clean_for_speech is a whitelist. She writes "(¬‿¬)", whose characters
live in Latin-1 and punctuation blocks rather than any emoji range,
so enumerating what to drop never converges. Em dashes become commas
because they are a pause — dropping one ran two clauses together and
changed what a sentence meant.
- whisper invents words out of silence ("you", "Thank you."), and a
held key over a quiet room is long AND empty, so has_speech gates on
the loudest 100ms window rather than on duration.
- Beeps on key down/up and a distinct low tone for "heard nothing":
push-to-talk with no acknowledgement is indistinguishable from broken,
and a dead mic must not feel like a slow reply.
- A warmup request at startup: her first reply after the container
starts costs ~16s against ~5s warm, and the first one is always the
one you are waiting for.
- `luna-voice watch` prints the socket protocol and `luna-voice say`
runs a turn from typed text, which is how a broken mic is told apart
from a broken pipeline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
232 lines
16 KiB
Markdown
232 lines
16 KiB
Markdown
# CLAUDE.md
|
||
|
||
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||
|
||
Flake-based NixOS config for a homelab. Hosts: **jupiter** (ZimaBlade NAS, x86_64),
|
||
**neptun** (netcup public reverse proxy + tailnet node, x86_64), **mercury** (Raspberry
|
||
Pi 3B+ DNS/DHCP, aarch64). See `README.md` for the full install/deploy walkthrough.
|
||
|
||
## Layout
|
||
|
||
```
|
||
flake.nix # nixosConfigurations: real hosts + test/util targets
|
||
common.nix # shared base: user darman (key-only ssh), nix settings, firewall :22, tz
|
||
services/<cat>/*.nix # one reusable NixOS module per service, grouped by category
|
||
# (media, network, vpn, identity, dev, desktop); each opens
|
||
# ITS OWN firewall ports. services/containers.nix (podman
|
||
# backend) stays at the top level, shared across categories.
|
||
hosts/<h>/ # configuration.nix + disk-config.nix (disko) + hardware-configuration.nix + secrets.nix
|
||
secrets/<h>.yaml # sops-nix, age-encrypted per host
|
||
scripts/deploy # config-agnostic deploy wrapper (all args mandatory)
|
||
scripts/edit_secrets
|
||
.sops.yaml # per-host encryption rules (admin key + each host's key)
|
||
```
|
||
|
||
A host = `common.nix` + the `services/**` modules it imports + its `hosts/<h>/configuration.nix`.
|
||
`services/` modules are engine-agnostic and shared across hosts (e.g. `services/vpn/tailscale.nix`,
|
||
`services/network/caddy.nix` used by jupiter and neptun).
|
||
|
||
## Commands
|
||
|
||
Eval/verify a config before deploying (eval only checks the module tree, not
|
||
freeform config like pihole's TOML or a container's runtime):
|
||
```
|
||
nix eval --raw .#nixosConfigurations.<host>.config.system.build.toplevel.drvPath
|
||
```
|
||
|
||
Deploy (from a non-NixOS laptop too — runs nixos-rebuild/nixos-anywhere via `nix run`):
|
||
```
|
||
./scripts/deploy switch <config> <host> # daily rebuild + activate
|
||
./scripts/deploy install <config> <host> # first install (nixos-anywhere, wipes OS disk)
|
||
./scripts/deploy kexec <config> <host> # RO-root box (ZimaOS): kexec into a RAM installer first
|
||
sudo ./scripts/deploy kexec-local [--yes] # kexec THIS box (no ssh); confirm prompt unless --yes
|
||
./scripts/deploy image mercury # build the aarch64 SD image
|
||
./scripts/deploy flash mercury /dev/sdX # build + write SD + drop the sops age key
|
||
```
|
||
|
||
Both password prompts are auto-filled from the "HomeLab" Proton Pass vault, keyed by
|
||
**`<config>`, not `<host>`** — items `darman@<config>` (sudo) and `root@<config>` (ssh).
|
||
A *trashed* Proton Pass item with the same title shadows the active one and yields an
|
||
empty password, so the script resolves the title among `--filter-state active` items
|
||
first; a plain `pass-cli item view --item-title` silently returns the trashed copy and
|
||
you get an interactive prompt with no explanation.
|
||
|
||
`HOMELAB_KEXEC_TARBALL` (+ `_CPIO` / `_GZIP`) makes `kexec`/`kexec-local` reuse a
|
||
prebuilt installer instead of rebuilding ~500MB. The VM test below uses this.
|
||
|
||
Secrets (needs the admin age key at `~/.config/sops/age/keys.txt`):
|
||
```
|
||
./scripts/edit_secrets secrets/<host>.yaml
|
||
```
|
||
|
||
**Claude: never run `sops --decrypt`/`edit_secrets --show` and print the result — that
|
||
puts every plaintext secret in the file into the conversation transcript, not just the
|
||
one you wanted.** To add or change a single value non-interactively, use
|
||
`sops --set '["key"] "value"' secrets/<host>.yaml` (quote the value as JSON), which
|
||
writes without ever displaying the file's existing contents.
|
||
|
||
Test a service config BEFORE touching hardware — always do this for nontrivial changes:
|
||
```
|
||
# x86 QEMU VM of mercury's DNS/DHCP stack (fast; validates pihole/unbound at runtime)
|
||
nix build .#nixosConfigurations.mercury-vm.config.system.build.vm -o result
|
||
./result/bin/run-mercury-vm-vm # ssh -p 2223 darman@localhost (pw: test)
|
||
# jupiter services as a VirtualBox OVA
|
||
nix build .#nixosConfigurations.jupiter-vbox.config.system.build.virtualBoxOVA
|
||
# end-to-end VM test of `deploy kexec-local` (~45s once the tarball is built)
|
||
nix build .#checks.x86_64-linux.kexec-local -L
|
||
# VM test of luna's app hosting on mars (hosts/mars/luna-sites.nix): podman socket
|
||
# proxy, registry validation, caddy routes, reboot persistence
|
||
nix build .#checks.x86_64-linux.luna-sites -L
|
||
```
|
||
|
||
`checks.kexec-local` is the only way to exercise `kexec-local` at all: it jumps the
|
||
machine you are typing at, so it cannot be rehearsed on real hardware and a failure
|
||
looks exactly like a slow boot. It asserts the box actually left the old kernel
|
||
(SSH drops then returns), came back as `nixos-installer`, lost its old `/run`, and
|
||
kept its ssh host key. Run it after ANY change to the kexec paths.
|
||
|
||
## Secrets (sops-nix)
|
||
|
||
- Each `secrets/<host>.yaml` is encrypted to the **admin** key (edit) + that **host's**
|
||
key (runtime decrypt); rules in `.sops.yaml`. Private keys live OFF-repo:
|
||
`~/.config/sops/age/keys.txt` (admin), `~/.config/homelab/<host>/` (host keys).
|
||
- jupiter/neptun decrypt with their **ssh host key** (`ssh-to-age` recipient), shipped at
|
||
install via `nixos-anywhere --extra-files`.
|
||
- mercury (SD image, no `--extra-files`) uses a **dedicated age key** at
|
||
`/var/lib/sops-nix/age.txt` — `./scripts/deploy flash` writes it to the ext4 root partition.
|
||
- A service password that must come from sops but whose module has no `passwordFile`
|
||
hook (pihole, adguard) is injected via `sops.templates` → an env file → the service
|
||
(`FTLCONF_*` for pihole). See `hosts/mercury/configuration.nix`.
|
||
|
||
## Non-obvious gotchas (all learned the hard way)
|
||
|
||
- **aarch64 (mercury)**: the x86 laptop needs `extra-platforms = aarch64-linux` in
|
||
`/etc/nix/nix.custom.conf` (NOT `/etc/nix/nix.conf` — Determinate Nix regenerates that)
|
||
+ `qemu-user-static-binfmt`, else emulated builds fail with "platform mismatch". Or
|
||
build on the Pi with `--build-host darman@<ip>`.
|
||
- **pihole on mercury is a CONTAINER** (`services/network/pihole.nix`, official image, host
|
||
networking, caps NET_ADMIN/NET_RAW/SYS_NICE/CHOWN, `FTLCONF_*` env config). The native
|
||
`services.pihole-ftl` module **segfaults on the Pi 3B+ aarch64** — do not switch back.
|
||
- **`services.unbound.resolveLocalQueries = false`** is required: unbound listens on
|
||
:5335, so leaving it true points the host's resolv.conf at 127.0.0.1:**53** with nothing
|
||
there → boot-time DNS deadlock (starves image pulls / list downloads). Host resolves via
|
||
upstream `networking.nameservers`; pihole forwards to unbound explicitly at `127.0.0.1#5335`.
|
||
- **Remote deploy pushes unsigned closures**: hosts set `nix.settings.trusted-users =
|
||
[ "root" "@wheel" ]` (in common.nix) so a laptop-built closure is accepted by the target.
|
||
- **jupiter**: `boot.kernelParams = [ "reboot=pci" ]` (warm reboot hangs on that board);
|
||
eMMC initrd modules pinned in `configuration.nix` (generate-config misses them); the
|
||
16TB×2 **RAID0** data lives on `/mnt/data` with `nofail`, kept OUT of disko (never wiped).
|
||
- **terra: `./scripts/deploy kexec-local` hangs hard — do not use it there.** Confirmed
|
||
on real hardware: kexec's `device_shutdown()` pass runs (SCSI disks sync fine in the
|
||
log), then the machine goes dark and never comes back — `journalctl --list-boots`
|
||
showed a ~15min gap before the next boot, i.e. a genuine hang needing a manual power
|
||
cycle, not a slow jump. Near-certainly amdgpu (RX 6800 XT): discrete AMD GPUs are known
|
||
to hang during kexec's device-shutdown pass with no clean handoff before the jump —
|
||
same class of issue as jupiter's `reboot=pci` workaround, just fatal here instead of
|
||
slow. Use `./scripts/deploy install terra localhost` instead (README's "First install
|
||
on terra" section) — it detects it isn't inside a live installer yet and reboots via
|
||
a real `systemctl reboot` + systemd-boot one-shot `findiso=` entry, not kexec.
|
||
- **`./scripts/deploy install <config> localhost`'s behavior depends on `uname -n`**
|
||
(`is_live_installer()`): on a real running OS it builds `installer-iso`, stages it
|
||
locally, and reboots into it (`local_install_prepare_and_reboot()`); only inside
|
||
`nixos-installer` (kexec) or `homelab-installer` (installer-iso) does it actually run
|
||
disko + `nixos-install`. `installer-iso`'s `homelab-auto-install.service` closes the
|
||
loop: it reads `homelab.install=<config>` back off `/proc/cmdline` (set by the prepare
|
||
step) and re-runs the identical command itself once `homelab-checkout.service` has
|
||
cloned the repo — the whole reinstall is one command and unattended after the first
|
||
reboot. It confirms (type `yes`) before rebooting, like `flash`/`kexec-local`; `--yes`
|
||
skips that and is what the ISO passes itself. It always ASKS where to stage the iso
|
||
file (never auto-picks — the wrong disk here is destroyed mid-install);
|
||
`HOMELAB_INSTALLER_STAGE_DIR` skips the prompt for scripted use. Both this and
|
||
`kexec-local` self-elevate via `sudo` (`require_root()`) rather than requiring you to
|
||
prefix the command yourself.
|
||
- **terra runs Limine, not systemd-boot** — `bootctl set-oneshot` is useless there
|
||
(`bootctl status` lists `✗ One-shot entry control`, and CachyOS's pacman hooks
|
||
regenerate `limine.conf` anyway). `local_install_prepare_and_reboot()` therefore
|
||
picks its one-shot mechanism at runtime: a systemd-boot loader entry when
|
||
`$BOOT/loader/entries` exists, otherwise `arm_efi_bootnext()` — a temporary UEFI
|
||
entry that EFI-stub-boots the kernel off the ESP, armed via the firmware's
|
||
`BootNext`. Created with `efibootmgr --create-only` (NOT `--create`, which pushes
|
||
it to the front of `BootOrder` and would make a wiped installer the permanent
|
||
default if anything went wrong). BootNext is spent by that one boot, so a failed
|
||
attempt still comes back on the normal bootloader. The EFI-stub path needs the
|
||
kernel on the **ESP** itself, not on a separate XBOOTLDR — hence `--print-esp-path`
|
||
rather than `--print-boot-path` in that mode. `homelab-auto-install.service` deletes
|
||
the leftover NVRAM entry as soon as it boots; both it and the script match the label
|
||
`Homelab Installer` EXACTLY (a prefix match would delete the Windows or Limine entry).
|
||
- **`installer-iso` must force `boot.initrd.systemd.enable = false`.** `findiso=` is a
|
||
SCRIPT-stage-1 feature (`stage-1-init.sh` loop-mounts the file it points at and
|
||
symlinks it to `/dev/root`). The systemd initrd — default since 26.05 — has NO findiso
|
||
path: it mounts `/iso` straight from `/dev/disk/by-label/<volumeID>` (iso-image.nix),
|
||
a label that only exists when the ISO is the physical boot medium. EFI-stub-booted off
|
||
the ESP with the iso as a plain file on another fs, that label never appears; stage 1
|
||
mounts `/sysroot` fine, then times out on `/sysroot/nix/.ro-store` waiting for
|
||
`/dev/disk/by-label/nixos-minimal-…` and drops to an emergency shell. The whole
|
||
`install <config> localhost` findiso path depends on script stage 1.
|
||
- **The staging-dir guard must fail CLOSED, and `findmnt` needs `--nofsroot`**: on btrfs
|
||
`findmnt -no SOURCE` prints `/dev/sdb2[/@]`, which `lsblk` cannot open, so a naive
|
||
parent-device lookup comes back empty. Treating empty as "different disk" silently
|
||
allowed staging the iso on the very disk disko then wiped — terra's CachyOS root is
|
||
exactly that layout, so it hit the live case. `disks_backing()` (`lsblk -rnso
|
||
NAME,TYPE`) returns EVERY whole-disk ancestor because LVM/RAID can span several
|
||
(`/mnt/ssd_01` → sdd + sde), and an empty result is a hard error, not a pass. btrfs
|
||
staging is refused outright: stage-1 mounts a btrfs volume's TOP level, so an iso
|
||
inside a subvolume is unreachable via `findiso=`.
|
||
- **`findiso=` must keep its leading slash**: stage-1 tests `[ -e "/findiso$isoPath" ]`,
|
||
so stripping the mountpoint prefix off a stagedir whose mountpoint is `/` yields
|
||
`var/tmp/x.iso` → `/findisovar/tmp/x.iso` → emergency shell, after you have already
|
||
rebooted out of the working OS.
|
||
- **The auto-install needs the host key shipped to it, and a `$HOME`**: the ISO is built
|
||
from a public repo with no credentials, so `local_install_prepare_and_reboot()` copies
|
||
the key onto the boot partition and passes that partition's PARTUUID as
|
||
`homelab.keypart=`; the service mounts it and drops the key in
|
||
`/root/.config/homelab/<config>/` before running the install. Also, systemd does NOT
|
||
set `$HOME` for a system service without `User=` (`SetLoginEnvironment=` defaults to
|
||
false), and `scripts/deploy` runs under `set -u` — hence `$KEYDIR` instead of a bare
|
||
`$HOME`, plus `Environment=HOME=/root` on the unit.
|
||
- **`installer-iso` must enable `experimental-features` itself.** `installation-cd-minimal`
|
||
leaves them unset, so `nix run` and `nixos-install --flake` both die with "experimental
|
||
Nix feature 'nix-command' is disabled". The nixos-images `kexec` installer sets them
|
||
itself, which is why the same `install <config> localhost` branch worked after
|
||
`kexec-local` but not from the ISO.
|
||
- **disko/nixos-anywhere run as `nix run .#disko` / `.#nixos-anywhere`**, from this
|
||
flake's locked inputs — not `nix run github:...`. They execute while a disk is being
|
||
wiped, so the revision must be the reviewed one in `flake.lock`, and it has to resolve
|
||
without network.
|
||
- **disko wipes only the OS disk** named in `hosts/<h>/disk-config.nix`; data disks are
|
||
plain `fileSystems` in `configuration.nix`.
|
||
- `nixos-anywhere`/kexec needs a writable root; **ZimaOS root is read-only**, hence the
|
||
`./scripts/deploy kexec` step that streams a RAM installer (with static cpio/gzip since
|
||
ZimaOS lacks them).
|
||
- **`kexec/run` jumps ~6s AFTER it returns**: nixos-images' `kexec-run.sh` ends with
|
||
`nohup sh -c "sleep 6 && $SCRIPT_DIR/kexec -e" &`. So the staging dir must OUTLIVE the
|
||
script — an `rm -rf` in an EXIT trap deletes the binary that performs the jump and the
|
||
box silently stays on the old kernel. `kexec-local` clears its trap before jumping and
|
||
then sleeps 60s on purpose. Covered by `checks.kexec-local`.
|
||
- **The kexec installer KEEPS the box's ssh host key**: `kexec-run.sh` copies
|
||
`/etc/ssh/ssh_host_*` into the appended initrd and `restore-remote-access.nix` installs
|
||
them back. So do NOT `ssh-keygen -R` after a kexec — the key does not change, and
|
||
clearing it just throws away the known_hosts record.
|
||
- **`kexec-local` stages on `/var/tmp`, not `/tmp`**: `kexec-run.sh` appends a fresh cpio
|
||
to `kexec/initrd` in place and execs binaries from that dir, so a size-capped or
|
||
`noexec` tmpfs gives a half-written initrd or a bare "Permission denied".
|
||
- **A sops secret a USER service reads needs `owner`, not a mode.** `luna-voiced`
|
||
(`services/desktop/luna-voice.nix`) runs as darman's user unit, and a user manager
|
||
cannot read root:root 0400 — `LoadCredential` does not bridge that gap either.
|
||
Hence `sops.secrets.<x>.owner = "darman"` in `hosts/terra/secrets.nix`.
|
||
- **`elevenlabs_api_key` is in BOTH `secrets/mars.yaml` and `secrets/terra.yaml`** —
|
||
same value. terra calls ElevenLabs directly (the client-direct path Hermes
|
||
documents) rather than relaying audio through mars, so rotating that key means
|
||
editing two files. `hermes_api_server_key` is duplicated for the same reason:
|
||
mars serves the API, terra is its only client.
|
||
- **whisper invents words out of silence** — "you", "Thank you.", "Bye." — and a
|
||
held push-to-talk key over a quiet room is long AND empty, so duration alone
|
||
does not catch it. `stt::has_speech` gates on the loudest 100ms window instead;
|
||
without it every stray keypress sends a hallucinated sentence to luna.
|
||
- **terra's greeter is a throwaway Hyprland running `dotfiles/quickshell/greeter.qml`**
|
||
(`services/desktop/quickshell-greeter.nix`). It must exit after login or greetd never
|
||
starts the session, and with a Lua config `hyprctl dispatch exit` is REJECTED — it needs
|
||
`hyprctl dispatch 'hl.dsp.exit()'`. Test the flow without touching the real greetd by
|
||
running greetd's `fakegreet "qs -p …/greeter.qml"` inside a nested Hyprland
|
||
(user `user`, password `password`, then answer `9`).
|