feat(terra): hands-free voice mode, toggled with SUPER+V

The key is a toggle rather than push-to-talk, because "voice mode is
active" has to be a state something can display; a held key cannot
sustain one. So the daemon grew a VAD loop: speech starts a capture,
`silenceMs` of quiet ends it. A turn is awaited inline, which is also
what stops her hearing herself — the tick cannot fire while she speaks.

The keybind runs the CLI, not a quickshell GlobalShortcut, so the mic can
still be switched off while the shell is reloading and a QML error can
never leave it open with nothing on screen saying so.

Every threshold had to be resized for terra's mic, which is QUIET:
0.001-0.002 ambient against 0.0045 speaking, about 6dB apart. The VAD
floor of 0.006 and whisper's own 0.015 gate both sat ABOVE the user's
voice, and a 3.5x multiplier cannot separate signals that close — three
independent reasons nothing could ever trigger. Hence vadMult 1.8,
vadFloor 0.0025, and stt::normalize lifting the clip (capped at 30x, so
near-silence is not amplified into words) before whisper sees it.

The floor tracker also had to stop feeding on everything: a rising sound
dragged its own threshold up faster than it rose, and a tone peaking at
0.029 was outrun by a threshold that reached 0.018. Only samples below
the current trigger update it now.

Four diagnostics, because none of the above was visible while it was
failing: `watch` prints the socket protocol, `say` runs a turn from typed
text (a broken mic vs a broken pipeline), `level` is a live meter with
the trigger marked, `devices` lists the inputs. Amplitude is now emitted
on every tick, not only while she speaks — a meter that moves only during
playback cannot answer "is it hearing me".

Also: connect-time replay of mode/state, so restarting the shell while
voice mode is on does not leave it blind until the next event, which in
an idle conversation is never.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
2026-09-20 23:37:51 +02:00
co-authored by Claude Opus 5
parent d089743f75
commit 67dd30eb72
7 changed files with 476 additions and 26 deletions
+74 -5
View File
@@ -17,7 +17,7 @@ pub async fn transcribe(bin: &str, model: &Path, samples: &[f32]) -> Result<Stri
let dir = std::env::var("XDG_RUNTIME_DIR").unwrap_or_else(|_| "/tmp".into());
let wav = Path::new(&dir).join(format!("luna-stt-{}.wav", std::process::id()));
audio::write_wav(&wav, samples, audio::STT_RATE)?;
audio::write_wav(&wav, &normalize(samples), audio::STT_RATE)?;
let out = Command::new(bin)
.args(["-m"])
@@ -40,11 +40,38 @@ pub async fn transcribe(bin: &str, model: &Path, samples: &[f32]) -> Result<Stri
Ok(clean(&String::from_utf8_lossy(&out.stdout)))
}
/// Scales a quiet recording up before whisper sees it.
///
/// Whisper is trained on normalised audio and degrades badly on very quiet
/// input, which is precisely what a low-gain interface produces — speech at a
/// 0.03 peak is intelligible to a person and mush to the model. The gain is
/// capped so a recording of near-silence is not amplified into hiss the model
/// will confidently transcribe as words.
fn normalize(samples: &[f32]) -> Vec<f32> {
const TARGET_PEAK: f32 = 0.6;
const MAX_GAIN: f32 = 30.0;
let peak = samples.iter().fold(0.0f32, |m, s| m.max(s.abs()));
if peak <= f32::EPSILON {
return samples.to_vec();
}
let gain = (TARGET_PEAK / peak).min(MAX_GAIN);
// Never attenuate: a loud recording is already fine, and turning it down
// would only lose resolution in the 16-bit conversion.
if gain <= 1.0 {
return samples.to_vec();
}
samples.iter().map(|s| (s * gain).clamp(-1.0, 1.0)).collect()
}
/// True if any 100ms window is loud enough to be a voice. Peak rather than
/// overall RMS: a short sentence inside a long silent recording averages down
/// to nothing, and that is the common case for a held key.
fn has_speech(samples: &[f32]) -> bool {
const FLOOR: f32 = 0.015;
// Matches the VAD's lower clamp rather than a "typical" mic: a quiet
// interface puts normal speech at ~0.004, and gating at 0.015 threw away
// every word it captured.
const FLOOR: f32 = 0.0025;
let window = (audio::STT_RATE as usize) / 10;
samples.chunks(window).any(|w| {
let sum: f32 = w.iter().map(|s| s * s).sum();
@@ -70,14 +97,56 @@ fn clean(raw: &str) -> String {
#[cfg(test)]
mod tests {
use super::{clean, has_speech};
use super::{clean, has_speech, normalize};
use crate::audio::STT_RATE;
fn peak(v: &[f32]) -> f32 {
v.iter().fold(0.0f32, |m, s| m.max(s.abs()))
}
#[test]
fn normalize_lifts_a_quiet_recording() {
// 0.004 RMS speech peaks around 0.02, which is 30x under target —
// exactly at the gain cap, so it just reaches full scale.
let quiet: Vec<f32> = (0..1000).map(|i| 0.02 * ((i % 7) as f32 - 3.0) / 3.0).collect();
assert!((peak(&normalize(&quiet)) - 0.6).abs() < 0.01);
}
#[test]
fn normalize_leaves_a_loud_recording_alone() {
let loud: Vec<f32> = vec![0.9, -0.8, 0.7];
assert_eq!(normalize(&loud), loud);
}
#[test]
fn normalize_caps_the_gain_so_silence_stays_silent() {
let hiss: Vec<f32> = vec![0.0001, -0.0001];
// 0.6/0.0001 would be 6000x; the cap keeps it at 30x.
assert!(peak(&normalize(&hiss)) <= 0.0001 * 30.0 + 1e-9);
}
#[test]
fn normalize_handles_pure_silence() {
assert_eq!(normalize(&[0.0, 0.0]), vec![0.0, 0.0]);
}
#[test]
fn silence_is_not_speech() {
assert!(!has_speech(&vec![0.0f32; STT_RATE as usize]));
// Room tone: audible on a meter, not a voice.
assert!(!has_speech(&vec![0.004f32; STT_RATE as usize]));
// Terra's measured room tone. Speech on the same mic reaches
// 0.004–0.005, so the gate sits between the two.
assert!(!has_speech(&vec![0.001f32; STT_RATE as usize]));
}
#[test]
fn quiet_speech_still_counts() {
// 0.004 is what normal speech measures on terra's interface; gating
// it out discarded every word the VAD had just captured.
let mut s = vec![0.001f32; STT_RATE as usize];
for v in s.iter_mut().take(STT_RATE as usize / 5) {
*v = 0.004;
}
assert!(has_speech(&s));
}
#[test]