Commit Graph
5 Commits
Author SHA1 Message Date
darmanandClaude Opus 5.5 4b366f952e feat(terra): multilingual whisper small for luna's voice
base.en heard "backup" as "Mac up" and turned German into English nonsense.
small with -l auto gets both right at ~1s per turn (vs ~0.4s); turbo was
~2.3s for no visible gain. Drops the subtitle credits multilingual whisper
invents out of silence.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_011XkDJkdDzneVVdnkZfrvTY
2026-10-03 01:48:29 +02:00
darmanandClaude Opus 5.5 53a19c6b54 feat(terra): luna's voice rules live in luna-voiced's system prompt
Audio-tag and speech-stripping rules moved here from her SOUL.md: only
luna-voiced sends [voice], so they no longer ride along in every
Telegram prompt.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-02 20:25:45 +02:00
darmanandClaude Opus 5.5 e4ee637ccd feat(terra): luna v3 voice, channel marker and rail panel
- TTS defaults to eleven_v3_conversational over the Text to Dialogue
  socket: performs audio tags and gets her German right. Flash still
  works via the TTS socket.
- Every spoken message is prefixed with a [voice] marker so luna's
  profile on mars can tell the channel apart.
- LunaIndicator/LunaCall replaced by a LunaPanel in the bar rail, with
  captions; headless test updated to match.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-09-25 00:52:03 +02:00
darmanandClaude Opus 5 67dd30eb72 feat(terra): hands-free voice mode, toggled with SUPER+V
The key is a toggle rather than push-to-talk, because "voice mode is
active" has to be a state something can display; a held key cannot
sustain one. So the daemon grew a VAD loop: speech starts a capture,
`silenceMs` of quiet ends it. A turn is awaited inline, which is also
what stops her hearing herself — the tick cannot fire while she speaks.

The keybind runs the CLI, not a quickshell GlobalShortcut, so the mic can
still be switched off while the shell is reloading and a QML error can
never leave it open with nothing on screen saying so.

Every threshold had to be resized for terra's mic, which is QUIET:
0.001-0.002 ambient against 0.0045 speaking, about 6dB apart. The VAD
floor of 0.006 and whisper's own 0.015 gate both sat ABOVE the user's
voice, and a 3.5x multiplier cannot separate signals that close — three
independent reasons nothing could ever trigger. Hence vadMult 1.8,
vadFloor 0.0025, and stt::normalize lifting the clip (capped at 30x, so
near-silence is not amplified into words) before whisper sees it.

The floor tracker also had to stop feeding on everything: a rising sound
dragged its own threshold up faster than it rose, and a tone peaking at
0.029 was outrun by a threshold that reached 0.018. Only samples below
the current trigger update it now.

Four diagnostics, because none of the above was visible while it was
failing: `watch` prints the socket protocol, `say` runs a turn from typed
text (a broken mic vs a broken pipeline), `level` is a live meter with
the trigger marked, `devices` lists the inputs. Amplitude is now emitted
on every tick, not only while she speaks — a meter that moves only during
playback cannot answer "is it hearing me".

Also: connect-time replay of mode/state, so restarting the shell while
voice mode is on does not leave it blind until the next event, which in
an idle conversation is never.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-20 23:37:51 +02:00
darmanandClaude Opus 5 d089743f75 feat(terra): talk to luna with push-to-talk
luna-voiced: mic -> whisper-cpp (local, vulkan) -> Hermes' API on mars ->
ElevenLabs streaming TTS -> speaker, bound to SUPER+V. State, amplitude
and a viseme timeline are published on a unix socket; nothing consumes
that yet, but it is the data source the quickshell avatar will read.

STT stays local on purpose. mars already runs faster-whisper for Telegram
voice notes, but routing the mic through it would put every spoken word
on the network for no latency win.

Shape notes, all of them things the first live runs decided:

  - Viseme offsets are relative to speech_start (emitted when the first
    PCM frame reaches the speaker), never wall clock, so the shell and
    the daemon never have to compare clocks. ElevenLabs sends character
    alignment, not visemes, so the mapping is ours; the glTF morph target
    names are the contract.
  - clean_for_speech is a whitelist. She writes "(¬‿¬)", whose characters
    live in Latin-1 and punctuation blocks rather than any emoji range,
    so enumerating what to drop never converges. Em dashes become commas
    because they are a pause — dropping one ran two clauses together and
    changed what a sentence meant.
  - whisper invents words out of silence ("you", "Thank you."), and a
    held key over a quiet room is long AND empty, so has_speech gates on
    the loudest 100ms window rather than on duration.
  - Beeps on key down/up and a distinct low tone for "heard nothing":
    push-to-talk with no acknowledgement is indistinguishable from broken,
    and a dead mic must not feel like a slow reply.
  - A warmup request at startup: her first reply after the container
    starts costs ~16s against ~5s warm, and the first one is always the
    one you are waiting for.
  - `luna-voice watch` prints the socket protocol and `luna-voice say`
    runs a turn from typed text, which is how a broken mic is told apart
    from a broken pipeline.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
2026-09-20 22:45:17 +02:00