feat(terra): hands-free voice mode, toggled with SUPER+V
The key is a toggle rather than push-to-talk, because "voice mode is active" has to be a state something can display; a held key cannot sustain one. So the daemon grew a VAD loop: speech starts a capture, `silenceMs` of quiet ends it. A turn is awaited inline, which is also what stops her hearing herself — the tick cannot fire while she speaks. The keybind runs the CLI, not a quickshell GlobalShortcut, so the mic can still be switched off while the shell is reloading and a QML error can never leave it open with nothing on screen saying so. Every threshold had to be resized for terra's mic, which is QUIET: 0.001-0.002 ambient against 0.0045 speaking, about 6dB apart. The VAD floor of 0.006 and whisper's own 0.015 gate both sat ABOVE the user's voice, and a 3.5x multiplier cannot separate signals that close — three independent reasons nothing could ever trigger. Hence vadMult 1.8, vadFloor 0.0025, and stt::normalize lifting the clip (capped at 30x, so near-silence is not amplified into words) before whisper sees it. The floor tracker also had to stop feeding on everything: a rising sound dragged its own threshold up faster than it rose, and a tone peaking at 0.029 was outrun by a threshold that reached 0.018. Only samples below the current trigger update it now. Four diagnostics, because none of the above was visible while it was failing: `watch` prints the socket protocol, `say` runs a turn from typed text (a broken mic vs a broken pipeline), `level` is a live meter with the trigger marked, `devices` lists the inputs. Amplitude is now emitted on every tick, not only while she speaks — a meter that moves only during playback cannot answer "is it hearing me". Also: connect-time replay of mode/state, so restarting the shell while voice mode is on does not leave it blind until the next event, which in an idle conversation is never. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -71,6 +71,34 @@ in
|
||||
description = "File holding the ElevenLabs API key.";
|
||||
};
|
||||
|
||||
# Mic tuning. Defaults are set for terra's AIR 192|4, which is QUIET:
|
||||
# ~0.002 ambient against ~0.0045 for normal speech. Raising the gain on
|
||||
# the interface is worth more than any value here — everything downstream,
|
||||
# whisper included, works better with headroom.
|
||||
inputDevice = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
description = "Substring of the input device name; null uses the system default.";
|
||||
};
|
||||
|
||||
vadMult = lib.mkOption {
|
||||
type = lib.types.float;
|
||||
default = 1.8;
|
||||
description = "Speech triggers above noise_floor * this.";
|
||||
};
|
||||
|
||||
vadFloor = lib.mkOption {
|
||||
type = lib.types.float;
|
||||
default = 0.0025;
|
||||
description = "Absolute lower bound on the trigger, whatever the floor says.";
|
||||
};
|
||||
|
||||
silenceMs = lib.mkOption {
|
||||
type = lib.types.int;
|
||||
default = 2000;
|
||||
description = "Silence that ends an utterance.";
|
||||
};
|
||||
|
||||
systemPrompt = lib.mkOption {
|
||||
type = lib.types.nullOr lib.types.str;
|
||||
default = null;
|
||||
@@ -100,8 +128,13 @@ in
|
||||
LUNA_ELEVENLABS_KEY_FILE = toString cfg.elevenLabsKeyFile;
|
||||
LUNA_WHISPER_BIN = "${pkgs.whisper-cpp-vulkan}/bin/whisper-cli";
|
||||
LUNA_WHISPER_MODEL = whisperModel;
|
||||
LUNA_VAD_MULT = toString cfg.vadMult;
|
||||
LUNA_VAD_FLOOR = toString cfg.vadFloor;
|
||||
LUNA_SILENCE_MS = toString cfg.silenceMs;
|
||||
} // lib.optionalAttrs (cfg.systemPrompt != null) {
|
||||
LUNA_SYSTEM_PROMPT = cfg.systemPrompt;
|
||||
} // lib.optionalAttrs (cfg.inputDevice != null) {
|
||||
LUNA_INPUT_DEVICE = cfg.inputDevice;
|
||||
};
|
||||
|
||||
serviceConfig = {
|
||||
|
||||
Reference in New Issue
Block a user