feat(terra): hands-free voice mode, toggled with SUPER+V
The key is a toggle rather than push-to-talk, because "voice mode is active" has to be a state something can display; a held key cannot sustain one. So the daemon grew a VAD loop: speech starts a capture, `silenceMs` of quiet ends it. A turn is awaited inline, which is also what stops her hearing herself — the tick cannot fire while she speaks. The keybind runs the CLI, not a quickshell GlobalShortcut, so the mic can still be switched off while the shell is reloading and a QML error can never leave it open with nothing on screen saying so. Every threshold had to be resized for terra's mic, which is QUIET: 0.001-0.002 ambient against 0.0045 speaking, about 6dB apart. The VAD floor of 0.006 and whisper's own 0.015 gate both sat ABOVE the user's voice, and a 3.5x multiplier cannot separate signals that close — three independent reasons nothing could ever trigger. Hence vadMult 1.8, vadFloor 0.0025, and stt::normalize lifting the clip (capped at 30x, so near-silence is not amplified into words) before whisper sees it. The floor tracker also had to stop feeding on everything: a rising sound dragged its own threshold up faster than it rose, and a tone peaking at 0.029 was outrun by a threshold that reached 0.018. Only samples below the current trigger update it now. Four diagnostics, because none of the above was visible while it was failing: `watch` prints the socket protocol, `say` runs a turn from typed text (a broken mic vs a broken pipeline), `level` is a live meter with the trigger marked, `devices` lists the inputs. Amplitude is now emitted on every tick, not only while she speaks — a meter that moves only during playback cannot answer "is it hearing me". Also: connect-time replay of mode/state, so restarting the shell while voice mode is on does not leave it blind until the next event, which in an idle conversation is never. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This commit is contained in:
@@ -32,9 +32,14 @@ impl Capture {
|
||||
|
||||
let (r, rec, buf) = (rms.clone(), recording.clone(), buffer.clone());
|
||||
let rate = spawn_stream(move || {
|
||||
let device = cpal::default_host()
|
||||
.default_input_device()
|
||||
.ok_or_else(|| anyhow!("no input device"))?;
|
||||
let device = input_device()?;
|
||||
// Named in the log because "the mic does not work" is usually
|
||||
// "the default source is the wrong device", and nothing else in
|
||||
// the pipeline can tell you that.
|
||||
eprintln!(
|
||||
"luna-voice: input device {:?}",
|
||||
device.name().unwrap_or_else(|_| "<unnamed>".into())
|
||||
);
|
||||
let config = device.default_input_config()?;
|
||||
let rate = config.sample_rate().0;
|
||||
let channels = config.channels() as usize;
|
||||
@@ -163,6 +168,47 @@ impl Playback {
|
||||
}
|
||||
}
|
||||
|
||||
/// The microphone, honouring LUNA_INPUT_DEVICE (a case-insensitive substring
|
||||
/// of the device name) before falling back to the system default.
|
||||
///
|
||||
/// Worth having because the default source is a system-wide setting that has
|
||||
/// nothing to do with which input you actually speak into: a box with an
|
||||
/// audio interface and onboard audio has several, and picking the silent one
|
||||
/// looks exactly like a broken pipeline.
|
||||
fn input_device() -> Result<cpal::Device> {
|
||||
let host = cpal::default_host();
|
||||
if let Ok(want) = std::env::var("LUNA_INPUT_DEVICE") {
|
||||
let want = want.to_lowercase();
|
||||
if let Ok(devices) = host.input_devices() {
|
||||
for device in devices {
|
||||
if device
|
||||
.name()
|
||||
.map(|n| n.to_lowercase().contains(&want))
|
||||
.unwrap_or(false)
|
||||
{
|
||||
return Ok(device);
|
||||
}
|
||||
}
|
||||
}
|
||||
eprintln!("luna-voice: no input device matching {want:?}; using the default");
|
||||
}
|
||||
host.default_input_device()
|
||||
.ok_or_else(|| anyhow!("no input device"))
|
||||
}
|
||||
|
||||
/// Every input cpal can see, for `luna-voice devices`.
|
||||
pub fn list_inputs() -> Result<Vec<String>> {
|
||||
let host = cpal::default_host();
|
||||
let default = host.default_input_device().and_then(|d| d.name().ok());
|
||||
let mut out = Vec::new();
|
||||
for device in host.input_devices()? {
|
||||
let name = device.name().unwrap_or_else(|_| "<unnamed>".into());
|
||||
let marker = if Some(&name) == default.as_ref() { " (default)" } else { "" };
|
||||
out.push(format!("{name}{marker}"));
|
||||
}
|
||||
Ok(out)
|
||||
}
|
||||
|
||||
/// Builds a cpal stream on a dedicated thread that then parks forever,
|
||||
/// keeping it alive. Returns the device's sample rate, or the build error —
|
||||
/// a stream that fails here must surface at startup, not as silence later.
|
||||
|
||||
Reference in New Issue
Block a user