feat(terra): talk to luna with push-to-talk
luna-voiced: mic -> whisper-cpp (local, vulkan) -> Hermes' API on mars ->
ElevenLabs streaming TTS -> speaker, bound to SUPER+V. State, amplitude
and a viseme timeline are published on a unix socket; nothing consumes
that yet, but it is the data source the quickshell avatar will read.
STT stays local on purpose. mars already runs faster-whisper for Telegram
voice notes, but routing the mic through it would put every spoken word
on the network for no latency win.
Shape notes, all of them things the first live runs decided:
- Viseme offsets are relative to speech_start (emitted when the first
PCM frame reaches the speaker), never wall clock, so the shell and
the daemon never have to compare clocks. ElevenLabs sends character
alignment, not visemes, so the mapping is ours; the glTF morph target
names are the contract.
- clean_for_speech is a whitelist. She writes "(¬‿¬)", whose characters
live in Latin-1 and punctuation blocks rather than any emoji range,
so enumerating what to drop never converges. Em dashes become commas
because they are a pause — dropping one ran two clauses together and
changed what a sentence meant.
- whisper invents words out of silence ("you", "Thank you."), and a
held key over a quiet room is long AND empty, so has_speech gates on
the loudest 100ms window rather than on duration.
- Beeps on key down/up and a distinct low tone for "heard nothing":
push-to-talk with no acknowledgement is indistinguishable from broken,
and a dead mic must not feel like a slow reply.
- A warmup request at startup: her first reply after the container
starts costs ~16s against ~5s warm, and the first one is always the
one you are waiting for.
- `luna-voice watch` prints the socket protocol and `luna-voice say`
runs a turn from typed text, which is how a broken mic is told apart
from a broken pipeline.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -19,8 +19,27 @@ in
|
||||
../../services/desktop/desktop-hyprland.nix
|
||||
../../services/desktop/desktop-apps.nix
|
||||
../../services/desktop/librechat.nix
|
||||
../../services/desktop/luna-voice.nix
|
||||
];
|
||||
|
||||
# Talk to luna (Hermes Agent on mars) with SUPER+V — see
|
||||
# hosts/terra/home/hyprland.nix for the push-to-talk binds.
|
||||
homelab.lunaVoice = {
|
||||
enable = true;
|
||||
hermesKeyFile = config.sops.secrets.hermes_api_server_key.path;
|
||||
elevenLabsKeyFile = config.sops.secrets.elevenlabs_api_key.path;
|
||||
# luna's own voice — must stay in sync with `tts.elevenlabs.voice_id` in
|
||||
# her config.yaml on mars, or she sounds like someone else on the desktop
|
||||
# than she does on Telegram.
|
||||
voiceId = "LZAcK8Cx5QjdQhfBsJQZ";
|
||||
systemPrompt = ''
|
||||
You are being spoken to out loud and your reply will be read aloud by a
|
||||
speech synthesiser. Answer in one or two sentences unless asked for
|
||||
detail. No markdown, no lists, no code blocks — they do not survive the
|
||||
trip to audio.
|
||||
'';
|
||||
};
|
||||
|
||||
networking.hostName = "terra";
|
||||
|
||||
homelab.greeter = {
|
||||
|
||||
@@ -195,6 +195,12 @@ in
|
||||
# expand / collapse the hyprchrome bar as a whole
|
||||
(bind "SUPER + A" (dsp.global "quickshell:chrome"))
|
||||
|
||||
# push-to-talk for luna (services/desktop/luna-voice.nix). Two binds,
|
||||
# not one: the daemon records between the press and the release, so
|
||||
# the release edge is what ends the utterance.
|
||||
(bind "SUPER + V" (dsp.exec "luna-voice ptt-down"))
|
||||
(bindo "SUPER + V" (dsp.exec "luna-voice ptt-up") { release = true; })
|
||||
|
||||
(bind "SUPER + B" (dsp.exec "vivaldi"))
|
||||
(bind "SUPER + E" (dsp.exec "cosmic-files"))
|
||||
(bind "SUPER + SHIFT + G" (dsp.workspaceRef "game"))
|
||||
@@ -237,7 +243,7 @@ in
|
||||
(bind "SUPER + T" (dsp.togglespecial "terminal"))
|
||||
(bind "SUPER + C" (dsp.togglespecial "communications"))
|
||||
(bind "SUPER + M" (dsp.togglespecial "music"))
|
||||
(bind "SUPER + V" (dsp.togglespecial "version_control"))
|
||||
# (bind "SUPER + V" (dsp.togglespecial "version_control"))
|
||||
|
||||
# cycle workspaces
|
||||
(bind "SUPER + ALT + up" (dsp.workspaceRef "e+1"))
|
||||
|
||||
@@ -27,6 +27,18 @@
|
||||
# tokens. All four are random, generated once with `sops --set` (see
|
||||
# CLAUDE.md) — losing/rotating them just invalidates existing sessions and
|
||||
# any saved per-user API keys, nothing else depends on their value.
|
||||
# luna-voiced (services/desktop/luna-voice.nix) runs as a USER service, so
|
||||
# these cannot stay root:root 0400 — a user manager cannot read them and
|
||||
# LoadCredential does not bridge that gap either. Hence owner, not mode.
|
||||
#
|
||||
# hermes_api_server_key mirrors secrets/mars.yaml (the serving side);
|
||||
# elevenlabs_api_key mirrors it too, because client-direct TTS is the
|
||||
# documented low-latency path — terra calls ElevenLabs itself rather than
|
||||
# relaying audio through mars. Rotating that key therefore touches BOTH
|
||||
# files; see CLAUDE.md.
|
||||
sops.secrets.hermes_api_server_key.owner = "darman";
|
||||
sops.secrets.elevenlabs_api_key.owner = "darman";
|
||||
|
||||
sops.secrets.librechat_creds_key = { };
|
||||
sops.secrets.librechat_creds_iv = { };
|
||||
sops.secrets.librechat_jwt_secret = { };
|
||||
|
||||
Reference in New Issue
Block a user