Compare commits
31
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e4ee637ccd | ||
|
|
386a46c4fe | ||
|
|
f8a7b61280 | ||
|
|
4adb9347b8 | ||
|
|
565308a448 | ||
|
|
b3200d965d | ||
|
|
ef1be8a7d9 | ||
|
|
67dd30eb72 | ||
|
|
d089743f75 | ||
|
|
8dc3affdda | ||
|
|
9566624686 | ||
|
|
10046bf846 | ||
|
|
461d6cc441 | ||
|
|
1dd8edcbb0 | ||
|
|
b4c60b9f7b | ||
|
|
1bdd70fc4d | ||
|
|
a723222223 | ||
|
|
71900ddfcf | ||
|
|
a0519b5004 | ||
|
|
2b2f0994e9 | ||
|
|
22caa06904 | ||
|
|
812c1af1f2 | ||
|
|
e0f5c6b37b | ||
|
|
bef5215bea | ||
|
|
b2982c47a1 | ||
|
|
524315f495 | ||
|
|
625938922c | ||
|
|
fec8d7cb25 | ||
|
|
be98be0100 | ||
|
|
25e17f163e | ||
|
|
9b96d3c6f8 |
@@ -17,3 +17,6 @@ keys.txt
|
|||||||
|
|
||||||
# local visual-verification output
|
# local visual-verification output
|
||||||
.artifacts/
|
.artifacts/
|
||||||
|
|
||||||
|
# rust build output
|
||||||
|
target/
|
||||||
|
|||||||
@@ -210,6 +210,38 @@ kept its ssh host key. Run it after ANY change to the kexec paths.
|
|||||||
- **`kexec-local` stages on `/var/tmp`, not `/tmp`**: `kexec-run.sh` appends a fresh cpio
|
- **`kexec-local` stages on `/var/tmp`, not `/tmp`**: `kexec-run.sh` appends a fresh cpio
|
||||||
to `kexec/initrd` in place and execs binaries from that dir, so a size-capped or
|
to `kexec/initrd` in place and execs binaries from that dir, so a size-capped or
|
||||||
`noexec` tmpfs gives a half-written initrd or a bare "Permission denied".
|
`noexec` tmpfs gives a half-written initrd or a bare "Permission denied".
|
||||||
|
- **A sops secret a USER service reads needs `owner`, not a mode.** `luna-voiced`
|
||||||
|
(`services/desktop/luna-voice.nix`) runs as darman's user unit, and a user manager
|
||||||
|
cannot read root:root 0400 — `LoadCredential` does not bridge that gap either.
|
||||||
|
Hence `sops.secrets.<x>.owner = "darman"` in `hosts/terra/secrets.nix`.
|
||||||
|
- **`elevenlabs_api_key` is in BOTH `secrets/mars.yaml` and `secrets/terra.yaml`** —
|
||||||
|
same value. terra calls ElevenLabs directly (the client-direct path Hermes
|
||||||
|
documents) rather than relaying audio through mars, so rotating that key means
|
||||||
|
editing two files. `hermes_api_server_key` is duplicated for the same reason:
|
||||||
|
mars serves the API, terra is its only client.
|
||||||
|
- **terra's mic is QUIET, and every voice threshold is sized for it.** Measured on
|
||||||
|
the AIR 192|4 (`luna-voice level`): ~0.001–0.002 RMS ambient, only 0.0045–0.005
|
||||||
|
for normal speech — about 6dB of separation. Consequences, all of which looked
|
||||||
|
like "the daemon is broken": a VAD multiplier above ~2 can never fire, an
|
||||||
|
absolute trigger floor of 0.006 sits *above* the user's voice, and whisper's own
|
||||||
|
gate at 0.015 discarded every word the VAD did capture. Hence `vadMult = 1.8`,
|
||||||
|
`vadFloor = 0.0025`, and `stt::normalize` scaling the clip up (capped at 30x)
|
||||||
|
before whisper sees it. Raising the gain on the interface is worth more than any
|
||||||
|
of these; tune via `homelab.lunaVoice.vad*` rather than editing Rust.
|
||||||
|
- **`try_recv()` POPS — an `if let Ok(Cancel)` destroys every other command.**
|
||||||
|
luna-voiced polls for a cancel while she speaks; written as a single `if let`,
|
||||||
|
any `say`/`toggle`/`mode off` arriving mid-reply was taken off the channel and
|
||||||
|
dropped, so SUPER+V during her answer did nothing at all and the next turn never
|
||||||
|
ran. Non-cancel commands are collected and re-sent into the channel after the
|
||||||
|
turn instead (`requeue` in `turn()`).
|
||||||
|
- **A rising sound must not update the noise floor.** Feeding every sample into the
|
||||||
|
floor EMA lets speech drag its own threshold up faster than it rises — measured,
|
||||||
|
a tone peaking at 0.029 was chased by a threshold that reached 0.018 and never
|
||||||
|
triggered. Only samples *below* the current trigger update the floor.
|
||||||
|
- **whisper invents words out of silence** — "you", "Thank you.", "Bye." — and a
|
||||||
|
held push-to-talk key over a quiet room is long AND empty, so duration alone
|
||||||
|
does not catch it. `stt::has_speech` gates on the loudest 100ms window instead;
|
||||||
|
without it every stray keypress sends a hallucinated sentence to luna.
|
||||||
- **terra's greeter is a throwaway Hyprland running `dotfiles/quickshell/greeter.qml`**
|
- **terra's greeter is a throwaway Hyprland running `dotfiles/quickshell/greeter.qml`**
|
||||||
(`services/desktop/quickshell-greeter.nix`). It must exit after login or greetd never
|
(`services/desktop/quickshell-greeter.nix`). It must exit after login or greetd never
|
||||||
starts the session, and with a Lua config `hyprctl dispatch exit` is REJECTED — it needs
|
starts the session, and with a Lua config `hyprctl dispatch exit` is REJECTED — it needs
|
||||||
|
|||||||
@@ -70,8 +70,9 @@ configured via `.qmlls.ini` for editor diagnostics.
|
|||||||
## Architecture
|
## Architecture
|
||||||
|
|
||||||
`shell.qml` is a `Scope` instantiating the top-level pieces as siblings:
|
`shell.qml` is a `Scope` instantiating the top-level pieces as siblings:
|
||||||
`HyprChromeShell` (the status rail), the eleven launcher variants,
|
`HyprChromeShell` (the status rail, plus the surfaces it owns — launcher, polkit,
|
||||||
`Notifications`, `VolumeOsd` and `Vitals`. Each manages its own
|
askpass, notifications), the ten remaining launcher variants,
|
||||||
|
`VolumeOsd` and `Vitals`. Each manages its own
|
||||||
`PanelWindow`(s); there is no central layout manager.
|
`PanelWindow`(s); there is no central layout manager.
|
||||||
|
|
||||||
The one exception, and the pattern to follow for anything new that needs it, is
|
The one exception, and the pattern to follow for anything new that needs it, is
|
||||||
@@ -100,6 +101,8 @@ on it, hence the index loops in `HyprChromeShell`.
|
|||||||
and its panels, with `Bar/Panels/BarPanel.qml` the chamfered chrome they all
|
and its panels, with `Bar/Panels/BarPanel.qml` the chamfered chrome they all
|
||||||
extend; `Widgets/Polkit/` is the authentication agent and its dialog;
|
extend; `Widgets/Polkit/` is the authentication agent and its dialog;
|
||||||
`Widgets/Launcher/` is the primary application launcher (`SUPER_L`);
|
`Widgets/Launcher/` is the primary application launcher (`SUPER_L`);
|
||||||
|
`Widgets/Notifications/` is the notification daemon and its popup stack on the
|
||||||
|
rail's monitor (`NotificationCard` is the headless-renderable card);
|
||||||
`Widgets/Greeter/` is the greetd login screen, run standalone via `greeter.qml`
|
`Widgets/Greeter/` is the greetd login screen, run standalone via `greeter.qml`
|
||||||
(see `services/desktop/quickshell-greeter.nix`), NOT part of `shell.qml`;
|
(see `services/desktop/quickshell-greeter.nix`), NOT part of `shell.qml`;
|
||||||
`Theme/Theme.qml` is this tree's palette singleton. `DebugWindow.qml` stages a
|
`Theme/Theme.qml` is this tree's palette singleton. `DebugWindow.qml` stages a
|
||||||
|
|||||||
@@ -4,7 +4,7 @@ import QtQuick.Layouts
|
|||||||
import Quickshell
|
import Quickshell
|
||||||
import Quickshell.Wayland
|
import Quickshell.Wayland
|
||||||
import qs.HyprChrome.Theme
|
import qs.HyprChrome.Theme
|
||||||
import qs.HyprChrome.Widgets.Bar.Debug
|
import qs.HyprChrome.Widgets.Bar.Luna
|
||||||
import qs.HyprChrome.Widgets.Bar.Host
|
import qs.HyprChrome.Widgets.Bar.Host
|
||||||
import qs.HyprChrome.Widgets.Bar.Vitals
|
import qs.HyprChrome.Widgets.Bar.Vitals
|
||||||
import qs.HyprChrome.Widgets.Bar.Workspaces
|
import qs.HyprChrome.Widgets.Bar.Workspaces
|
||||||
@@ -87,8 +87,8 @@ PanelWindow {
|
|||||||
//
|
//
|
||||||
// The zone still follows the target height, so tiled windows reflow once
|
// The zone still follows the target height, so tiled windows reflow once
|
||||||
// per toggle, at the start, and slide while the panels animate.
|
// per toggle, at the start, and slide while the panels animate.
|
||||||
readonly property real expandedContent: Math.max(hostPanel.expandedHeight, workspacesPanel.expandedHeight, vitalsPanel.expandedHeight, testPanel.expandedHeight, trayPanel.expandedHeight) + window.margin * 2
|
readonly property real expandedContent: Math.max(hostPanel.expandedHeight, workspacesPanel.expandedHeight, vitalsPanel.expandedHeight, lunaPanel.expandedHeight, trayPanel.expandedHeight) + window.margin * 2
|
||||||
readonly property real contentHeight: Math.max(hostPanel.targetHeight, workspacesPanel.targetHeight, vitalsPanel.targetHeight, testPanel.targetHeight, trayPanel.targetHeight) + window.margin * 2
|
readonly property real contentHeight: Math.max(hostPanel.targetHeight, workspacesPanel.targetHeight, vitalsPanel.targetHeight, lunaPanel.targetHeight, trayPanel.targetHeight) + window.margin * 2
|
||||||
|
|
||||||
implicitHeight: Math.round(window.expandedContent)
|
implicitHeight: Math.round(window.expandedContent)
|
||||||
|
|
||||||
@@ -157,8 +157,8 @@ PanelWindow {
|
|||||||
// and a trace at each end.
|
// and a trace at each end.
|
||||||
Item { Layout.fillWidth: true }
|
Item { Layout.fillWidth: true }
|
||||||
|
|
||||||
TestPanel {
|
LunaPanel {
|
||||||
id: testPanel
|
id: lunaPanel
|
||||||
|
|
||||||
expanded: window.expanded
|
expanded: window.expanded
|
||||||
toggleOnClick: false
|
toggleOnClick: false
|
||||||
|
|||||||
@@ -0,0 +1,200 @@
|
|||||||
|
pragma ComponentBehavior: Bound
|
||||||
|
|
||||||
|
import QtQuick
|
||||||
|
import qs.HyprChrome.Theme
|
||||||
|
import qs.HyprChrome.Widgets.Bar.Panels
|
||||||
|
|
||||||
|
// luna's voice loop on the rail (services/desktop/luna-voice.nix).
|
||||||
|
//
|
||||||
|
// Always present, unlike the daemon it watches: the panel is where you look
|
||||||
|
// to find out whether she is listening, so it has to be there when the answer
|
||||||
|
// is "no" — including when luna-voiced is not running at all. SUPER+V toggles
|
||||||
|
// LISTENING, not this.
|
||||||
|
//
|
||||||
|
// Collapsed it is the state and nothing else, because that is the one fact
|
||||||
|
// worth a glance; expanded it carries the last exchange, which is the thing
|
||||||
|
// you actually want during the seconds her reply takes to arrive.
|
||||||
|
BarPanel {
|
||||||
|
id: luna
|
||||||
|
|
||||||
|
// Overridable so the headless render can stage every state without a
|
||||||
|
// daemon; in the rail they track the socket below.
|
||||||
|
property string daemonState: state.state
|
||||||
|
property bool active: state.active
|
||||||
|
property bool connected: state.connected
|
||||||
|
property real rms: state.rms
|
||||||
|
property string userText: state.userText
|
||||||
|
property string lunaText: state.lunaText
|
||||||
|
|
||||||
|
// Instantiated here rather than in the shell: the panel is the only
|
||||||
|
// reader, and it is a second reader that would make this shell state.
|
||||||
|
readonly property var state: LunaState {}
|
||||||
|
|
||||||
|
panelId: "LUN"
|
||||||
|
title: "LUNA"
|
||||||
|
meta: luna.active ? "VOICE" : "IDLE"
|
||||||
|
|
||||||
|
// Sized to content like the tray, and per DENSITY: collapsed it is one
|
||||||
|
// word, so reserving the expanded body's width would leave the rail
|
||||||
|
// carrying a mostly empty panel for the state it spends most of its time
|
||||||
|
// in. It floats between two spacers, so the width change costs no other
|
||||||
|
// panel anything.
|
||||||
|
implicitWidth: luna.expanded
|
||||||
|
? Math.max(luna.headerMinWidth, luna.padding * 2 + body.implicitWidth)
|
||||||
|
: Math.max(luna.headerMinWidth, luna.briefLeft + brief.implicitWidth + luna.padding)
|
||||||
|
|
||||||
|
readonly property color stateColor: {
|
||||||
|
if (!luna.connected)
|
||||||
|
return Theme.disabled;
|
||||||
|
switch (luna.daemonState) {
|
||||||
|
case "listening":
|
||||||
|
return Theme.accent;
|
||||||
|
case "thinking":
|
||||||
|
return Theme.muted;
|
||||||
|
case "speaking":
|
||||||
|
return Theme.accentSoft;
|
||||||
|
default:
|
||||||
|
return luna.active ? Theme.text : Theme.raised;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
readonly property string stateLabel: {
|
||||||
|
if (!luna.connected)
|
||||||
|
return "OFFLINE";
|
||||||
|
if (!luna.active)
|
||||||
|
return "OFF";
|
||||||
|
switch (luna.daemonState) {
|
||||||
|
case "listening":
|
||||||
|
return "LISTENING";
|
||||||
|
case "thinking":
|
||||||
|
return "THINKING";
|
||||||
|
case "speaking":
|
||||||
|
return "SPEAKING";
|
||||||
|
default:
|
||||||
|
return "READY";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
summary: Row {
|
||||||
|
id: brief
|
||||||
|
spacing: 6
|
||||||
|
|
||||||
|
// Breathes only while the mic is open, so "she is listening to me
|
||||||
|
// right now" is legible without reading the word next to it.
|
||||||
|
Rectangle {
|
||||||
|
width: 7
|
||||||
|
height: 7
|
||||||
|
radius: 4
|
||||||
|
anchors.verticalCenter: parent.verticalCenter
|
||||||
|
color: luna.stateColor
|
||||||
|
|
||||||
|
SequentialAnimation on opacity {
|
||||||
|
running: luna.daemonState === "listening"
|
||||||
|
loops: Animation.Infinite
|
||||||
|
alwaysRunToEnd: true
|
||||||
|
NumberAnimation { to: 0.25; duration: 700; easing.type: Easing.InOutQuad }
|
||||||
|
NumberAnimation { to: 1.0; duration: 700; easing.type: Easing.InOutQuad }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Text {
|
||||||
|
anchors.verticalCenter: parent.verticalCenter
|
||||||
|
text: luna.stateLabel
|
||||||
|
color: luna.stateColor
|
||||||
|
font.family: Theme.displayFont
|
||||||
|
font.pixelSize: 13
|
||||||
|
font.bold: true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Column {
|
||||||
|
id: body
|
||||||
|
spacing: 6
|
||||||
|
|
||||||
|
// Level meter: live whenever voice mode is on, not only once a
|
||||||
|
// trigger has fired, so a mic too quiet to reach the threshold still
|
||||||
|
// shows movement instead of looking broken.
|
||||||
|
Row {
|
||||||
|
id: meter
|
||||||
|
width: 260
|
||||||
|
height: 18
|
||||||
|
spacing: 2
|
||||||
|
|
||||||
|
readonly property int count: 28
|
||||||
|
readonly property bool sweeping: luna.daemonState === "thinking"
|
||||||
|
property real phase: 0
|
||||||
|
|
||||||
|
NumberAnimation on phase {
|
||||||
|
running: meter.sweeping
|
||||||
|
loops: Animation.Infinite
|
||||||
|
from: 0
|
||||||
|
to: 1
|
||||||
|
duration: 1200
|
||||||
|
}
|
||||||
|
|
||||||
|
Repeater {
|
||||||
|
model: meter.count
|
||||||
|
|
||||||
|
Rectangle {
|
||||||
|
required property int index
|
||||||
|
|
||||||
|
width: (meter.width - (meter.count - 1) * meter.spacing) / meter.count
|
||||||
|
height: {
|
||||||
|
if (meter.sweeping) {
|
||||||
|
const pos = index / (meter.count - 1);
|
||||||
|
return 2 + 14 * Math.max(0, 1 - Math.abs(pos - meter.phase) * 6);
|
||||||
|
}
|
||||||
|
if (!luna.active && luna.daemonState !== "speaking")
|
||||||
|
return 2;
|
||||||
|
// sqrt opens up the quiet end, which on this mic is
|
||||||
|
// where all of the speech actually sits.
|
||||||
|
const level = Math.min(1, Math.sqrt(luna.rms) * 6.0);
|
||||||
|
const centre = 1 - Math.abs(index / (meter.count - 1) - 0.5) * 1.6;
|
||||||
|
return 2 + 16 * level * Math.max(0.15, centre);
|
||||||
|
}
|
||||||
|
anchors.bottom: parent.bottom
|
||||||
|
color: luna.stateColor
|
||||||
|
opacity: meter.sweeping ? 0.9 : 0.5 + 0.5 * (height / 18)
|
||||||
|
|
||||||
|
Behavior on height {
|
||||||
|
enabled: !meter.sweeping
|
||||||
|
NumberAnimation { duration: 70; easing.type: Easing.OutQuad }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Text {
|
||||||
|
width: 260
|
||||||
|
visible: luna.userText !== ""
|
||||||
|
text: `> ${luna.userText}`
|
||||||
|
color: Theme.muted
|
||||||
|
font.family: Theme.microFont
|
||||||
|
font.pixelSize: 10
|
||||||
|
wrapMode: Text.WordWrap
|
||||||
|
maximumLineCount: 2
|
||||||
|
elide: Text.ElideRight
|
||||||
|
}
|
||||||
|
|
||||||
|
Text {
|
||||||
|
width: 260
|
||||||
|
visible: luna.lunaText !== ""
|
||||||
|
text: luna.lunaText
|
||||||
|
color: Theme.text
|
||||||
|
font.family: Theme.microFont
|
||||||
|
font.pixelSize: 11
|
||||||
|
wrapMode: Text.WordWrap
|
||||||
|
maximumLineCount: 4
|
||||||
|
elide: Text.ElideRight
|
||||||
|
}
|
||||||
|
|
||||||
|
Text {
|
||||||
|
visible: luna.userText === "" && luna.lunaText === ""
|
||||||
|
text: luna.connected ? "SUPER+V TO TALK" : "DAEMON NOT RUNNING"
|
||||||
|
color: Theme.muted
|
||||||
|
font.family: Theme.microFont
|
||||||
|
font.pixelSize: 9
|
||||||
|
font.letterSpacing: 1.4
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,106 @@
|
|||||||
|
pragma ComponentBehavior: Bound
|
||||||
|
|
||||||
|
import QtQuick
|
||||||
|
import Quickshell
|
||||||
|
import Quickshell.Io
|
||||||
|
|
||||||
|
// Everything the Luna surfaces know, read off luna-voiced's unix socket
|
||||||
|
// (services/desktop/luna-voice.nix).
|
||||||
|
//
|
||||||
|
// Read-only on purpose: the key that toggles voice mode runs `luna-voice
|
||||||
|
// toggle` from Hyprland rather than writing here, so the toggle keeps working
|
||||||
|
// while the shell is reloading — and a QML error can never leave the mic on
|
||||||
|
// with nothing showing it.
|
||||||
|
//
|
||||||
|
// The daemon is a USER service that may start after the shell, restart under
|
||||||
|
// it, or not be installed at all, so a missing socket is a normal state and
|
||||||
|
// not an error: `reconnect` retries quietly and `connected` is what the
|
||||||
|
// widget shows.
|
||||||
|
QtObject {
|
||||||
|
id: root
|
||||||
|
|
||||||
|
// "idle" | "listening" | "thinking" | "speaking"
|
||||||
|
property string state: "idle"
|
||||||
|
// Voice mode itself — the widget is only on screen while this is true.
|
||||||
|
property bool active: false
|
||||||
|
property bool connected: socket.connected
|
||||||
|
// Mic level while listening, her output level while speaking. Drives the
|
||||||
|
// only genuinely live part of the visuals.
|
||||||
|
property real rms: 0
|
||||||
|
property string userText: ""
|
||||||
|
property string lunaText: ""
|
||||||
|
property string error: ""
|
||||||
|
|
||||||
|
readonly property var socket: Socket {
|
||||||
|
id: socket
|
||||||
|
path: `${Quickshell.env("XDG_RUNTIME_DIR") ?? "/run/user/1000"}/luna-voiced.sock`
|
||||||
|
connected: true
|
||||||
|
|
||||||
|
onConnectionStateChanged: {
|
||||||
|
if (!socket.connected) {
|
||||||
|
// A daemon that went away is not a daemon that is idle: reset
|
||||||
|
// rather than leave the last frame of a conversation frozen
|
||||||
|
// on screen.
|
||||||
|
root.active = false;
|
||||||
|
root.state = "idle";
|
||||||
|
root.rms = 0;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
parser: SplitParser {
|
||||||
|
splitMarker: "\n"
|
||||||
|
onRead: line => root.ingest(line)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Retry forever, cheaply. Setting `connected` true on an already-open
|
||||||
|
// socket is a no-op, so this doubles as the initial connect.
|
||||||
|
readonly property var reconnect: Timer {
|
||||||
|
interval: 2000
|
||||||
|
running: !socket.connected
|
||||||
|
repeat: true
|
||||||
|
onTriggered: socket.connected = true
|
||||||
|
}
|
||||||
|
|
||||||
|
function ingest(line: string): void {
|
||||||
|
let msg;
|
||||||
|
try {
|
||||||
|
msg = JSON.parse(line);
|
||||||
|
} catch (e) {
|
||||||
|
return; // a half-written line is not worth a console error
|
||||||
|
}
|
||||||
|
|
||||||
|
switch (msg.type) {
|
||||||
|
case "mode":
|
||||||
|
root.active = msg.active;
|
||||||
|
if (!msg.active) {
|
||||||
|
root.state = "idle";
|
||||||
|
root.rms = 0;
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
case "state":
|
||||||
|
root.state = msg.value;
|
||||||
|
// Level is only meaningful during the phase that produces it.
|
||||||
|
if (msg.value !== "listening" && msg.value !== "speaking")
|
||||||
|
root.rms = 0;
|
||||||
|
break;
|
||||||
|
case "amplitude":
|
||||||
|
root.rms = msg.rms;
|
||||||
|
break;
|
||||||
|
case "caption":
|
||||||
|
if (msg.speaker === "user") {
|
||||||
|
// A new user caption starts a turn, so it is what clears her
|
||||||
|
// last reply — NOT speech_start, which arrives after her
|
||||||
|
// first caption, not before it.
|
||||||
|
root.userText = msg.text;
|
||||||
|
root.lunaText = "";
|
||||||
|
} else {
|
||||||
|
root.lunaText = msg.text;
|
||||||
|
}
|
||||||
|
break;
|
||||||
|
case "error":
|
||||||
|
root.error = msg.message;
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -8,6 +8,7 @@ import qs.HyprChrome.Widgets
|
|||||||
import qs.HyprChrome.Widgets.Polkit
|
import qs.HyprChrome.Widgets.Polkit
|
||||||
import qs.HyprChrome.Widgets.Launcher
|
import qs.HyprChrome.Widgets.Launcher
|
||||||
import qs.HyprChrome.Widgets.Askpass
|
import qs.HyprChrome.Widgets.Askpass
|
||||||
|
import qs.HyprChrome.Widgets.Notifications
|
||||||
|
|
||||||
// The hyprchrome shell: owns everything the rail's surfaces have to agree on,
|
// The hyprchrome shell: owns everything the rail's surfaces have to agree on,
|
||||||
// and instantiates them.
|
// and instantiates them.
|
||||||
@@ -210,6 +211,11 @@ Scope {
|
|||||||
onDismissed: shell.expanded = false
|
onDismissed: shell.expanded = false
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Popups on the rail's monitor, below its exclusive zone.
|
||||||
|
Notifications {
|
||||||
|
screen: shell.targetScreen
|
||||||
|
}
|
||||||
|
|
||||||
// Polkit authentication agent. It registers for this logind session on
|
// Polkit authentication agent. It registers for this logind session on
|
||||||
// creation, so it replaces hyprpolkitagent rather than coexisting with it —
|
// creation, so it replaces hyprpolkitagent rather than coexisting with it —
|
||||||
// only one agent may hold a session (see hosts/terra/home/hyprland.nix).
|
// only one agent may hold a session (see hosts/terra/home/hyprland.nix).
|
||||||
|
|||||||
@@ -0,0 +1,272 @@
|
|||||||
|
pragma ComponentBehavior: Bound
|
||||||
|
|
||||||
|
import QtQuick
|
||||||
|
import QtQuick.Layouts
|
||||||
|
import Quickshell
|
||||||
|
import Quickshell.Widgets
|
||||||
|
import qs.HyprChrome.Theme
|
||||||
|
|
||||||
|
// Headlessly renderable visual core of one notification. Every field arrives as
|
||||||
|
// a plain property and every action leaves as a signal; Notifications.qml maps
|
||||||
|
// the server's Notification onto it.
|
||||||
|
//
|
||||||
|
// Owns its own expiry: the drain meter IS the timer, so hovering pauses both.
|
||||||
|
Item {
|
||||||
|
id: root
|
||||||
|
|
||||||
|
property string appName: ""
|
||||||
|
property string summary: ""
|
||||||
|
property string body: ""
|
||||||
|
// Icon name, path or URL; `image` (avatar, album art) wins over it.
|
||||||
|
property string appIcon: ""
|
||||||
|
property string image: ""
|
||||||
|
// 0 low, 1 normal, 2 critical — NotificationUrgency's values.
|
||||||
|
property int urgency: 1
|
||||||
|
property string time: ""
|
||||||
|
|
||||||
|
// Read structurally ({ identifier, text }). The "default" action is the
|
||||||
|
// card's own click, not a chip.
|
||||||
|
property var actions: []
|
||||||
|
|
||||||
|
// Milliseconds; 0 never expires.
|
||||||
|
property int timeout: 0
|
||||||
|
// Off for headless renders, which set `remaining` directly.
|
||||||
|
property bool countdown: true
|
||||||
|
property real remaining: 1
|
||||||
|
|
||||||
|
signal dismissed
|
||||||
|
signal expired
|
||||||
|
signal actionInvoked(string identifier)
|
||||||
|
|
||||||
|
readonly property bool critical: root.urgency === 2
|
||||||
|
readonly property color tint: root.critical ? Theme.hot : root.urgency === 0 ? Theme.muted : Theme.accent
|
||||||
|
|
||||||
|
readonly property var chipActions: {
|
||||||
|
const out = [];
|
||||||
|
for (let i = 0; i < root.actions.length; i++) {
|
||||||
|
if (root.actions[i].identifier !== "default")
|
||||||
|
out.push(root.actions[i]);
|
||||||
|
}
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
readonly property bool hasDefault: root.chipActions.length !== root.actions.length
|
||||||
|
|
||||||
|
readonly property string iconSource: {
|
||||||
|
const s = root.image !== "" ? root.image : root.appIcon;
|
||||||
|
if (s === "")
|
||||||
|
return "";
|
||||||
|
if (s.startsWith("/"))
|
||||||
|
return "file://" + s;
|
||||||
|
if (s.includes(":/"))
|
||||||
|
return s;
|
||||||
|
return Quickshell.iconPath(s, true);
|
||||||
|
}
|
||||||
|
|
||||||
|
implicitWidth: 360
|
||||||
|
implicitHeight: panel.implicitHeight
|
||||||
|
|
||||||
|
NumberAnimation on remaining {
|
||||||
|
id: drain
|
||||||
|
|
||||||
|
from: 1
|
||||||
|
to: 0
|
||||||
|
duration: Math.max(1, root.timeout)
|
||||||
|
running: root.countdown && root.timeout > 0
|
||||||
|
paused: running && hover.hovered
|
||||||
|
}
|
||||||
|
|
||||||
|
// Not drain's finished(): a value-source animation need not emit it.
|
||||||
|
onRemainingChanged: {
|
||||||
|
if (root.remaining <= 0 && root.countdown && root.timeout > 0)
|
||||||
|
root.expired();
|
||||||
|
}
|
||||||
|
|
||||||
|
HoverHandler {
|
||||||
|
id: hover
|
||||||
|
}
|
||||||
|
|
||||||
|
// Under the panel, so the close box and chips take their own clicks.
|
||||||
|
MouseArea {
|
||||||
|
anchors.fill: parent
|
||||||
|
cursorShape: root.hasDefault ? Qt.PointingHandCursor : Qt.ArrowCursor
|
||||||
|
onClicked: {
|
||||||
|
if (root.hasDefault)
|
||||||
|
root.actionInvoked("default");
|
||||||
|
else
|
||||||
|
root.dismissed();
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
component MicroText: Text {
|
||||||
|
color: Theme.muted
|
||||||
|
font.family: Theme.microFont
|
||||||
|
font.pixelSize: 8
|
||||||
|
font.letterSpacing: 0.9
|
||||||
|
elide: Text.ElideRight
|
||||||
|
}
|
||||||
|
|
||||||
|
NotificationPanel {
|
||||||
|
id: panel
|
||||||
|
|
||||||
|
width: root.width
|
||||||
|
panelId: root.critical ? "CRT" : root.urgency === 0 ? "LOW" : "NTF"
|
||||||
|
title: (root.appName !== "" ? root.appName : "NOTIFICATION").toUpperCase()
|
||||||
|
meta: root.time
|
||||||
|
accentColor: root.tint
|
||||||
|
|
||||||
|
onCloseRequested: root.dismissed()
|
||||||
|
|
||||||
|
ColumnLayout {
|
||||||
|
width: parent.width
|
||||||
|
spacing: 8
|
||||||
|
|
||||||
|
RowLayout {
|
||||||
|
Layout.fillWidth: true
|
||||||
|
spacing: 10
|
||||||
|
|
||||||
|
// Framed, so a transparent icon still reads as a tile.
|
||||||
|
Rectangle {
|
||||||
|
Layout.alignment: Qt.AlignTop
|
||||||
|
visible: root.iconSource !== ""
|
||||||
|
implicitWidth: 40
|
||||||
|
implicitHeight: 40
|
||||||
|
color: Theme.selection
|
||||||
|
border.width: 1
|
||||||
|
border.color: Theme.hair
|
||||||
|
|
||||||
|
IconImage {
|
||||||
|
anchors.centerIn: parent
|
||||||
|
implicitSize: root.image !== "" ? 38 : 28
|
||||||
|
source: root.iconSource
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
ColumnLayout {
|
||||||
|
Layout.fillWidth: true
|
||||||
|
Layout.alignment: Qt.AlignTop
|
||||||
|
spacing: 3
|
||||||
|
|
||||||
|
Text {
|
||||||
|
Layout.fillWidth: true
|
||||||
|
visible: root.summary !== ""
|
||||||
|
text: root.summary
|
||||||
|
textFormat: Text.PlainText
|
||||||
|
wrapMode: Text.Wrap
|
||||||
|
maximumLineCount: 2
|
||||||
|
elide: Text.ElideRight
|
||||||
|
color: Theme.text
|
||||||
|
font.family: Theme.displayFont
|
||||||
|
font.pixelSize: 13
|
||||||
|
font.bold: true
|
||||||
|
font.letterSpacing: 0.4
|
||||||
|
}
|
||||||
|
|
||||||
|
// Body markup is advertised, so StyledText, never RichText.
|
||||||
|
Text {
|
||||||
|
Layout.fillWidth: true
|
||||||
|
visible: root.body !== ""
|
||||||
|
text: root.body
|
||||||
|
textFormat: Text.StyledText
|
||||||
|
wrapMode: Text.Wrap
|
||||||
|
maximumLineCount: 5
|
||||||
|
elide: Text.ElideRight
|
||||||
|
color: Theme.textAlpha(0.72)
|
||||||
|
linkColor: root.tint
|
||||||
|
font.family: Theme.displayFont
|
||||||
|
font.pixelSize: 11
|
||||||
|
font.letterSpacing: 0.3
|
||||||
|
onLinkActivated: link => Qt.openUrlExternally(link)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Flow {
|
||||||
|
Layout.fillWidth: true
|
||||||
|
visible: root.chipActions.length > 0
|
||||||
|
spacing: 6
|
||||||
|
|
||||||
|
Repeater {
|
||||||
|
model: root.chipActions
|
||||||
|
|
||||||
|
Rectangle {
|
||||||
|
id: chip
|
||||||
|
|
||||||
|
required property var modelData
|
||||||
|
|
||||||
|
width: chipLabel.implicitWidth + 16
|
||||||
|
height: chipLabel.implicitHeight + 10
|
||||||
|
color: chipArea.containsMouse ? root.tint : "transparent"
|
||||||
|
border.width: 1
|
||||||
|
border.color: chipArea.containsMouse ? root.tint : Theme.disabled
|
||||||
|
|
||||||
|
Text {
|
||||||
|
id: chipLabel
|
||||||
|
|
||||||
|
anchors.centerIn: parent
|
||||||
|
text: chip.modelData.text.toUpperCase()
|
||||||
|
color: chipArea.containsMouse ? Theme.surface : Theme.text
|
||||||
|
font.family: Theme.microFont
|
||||||
|
font.pixelSize: 9
|
||||||
|
font.letterSpacing: 0.8
|
||||||
|
}
|
||||||
|
|
||||||
|
MouseArea {
|
||||||
|
id: chipArea
|
||||||
|
|
||||||
|
anchors.fill: parent
|
||||||
|
hoverEnabled: true
|
||||||
|
cursorShape: Qt.PointingHandCursor
|
||||||
|
onClicked: root.actionInvoked(chip.modelData.identifier)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Expiry drain; a sticky card states it instead.
|
||||||
|
RowLayout {
|
||||||
|
Layout.fillWidth: true
|
||||||
|
spacing: 8
|
||||||
|
|
||||||
|
Row {
|
||||||
|
id: meter
|
||||||
|
|
||||||
|
readonly property int segments: 24
|
||||||
|
|
||||||
|
Layout.fillWidth: true
|
||||||
|
Layout.preferredHeight: 4
|
||||||
|
visible: root.timeout > 0
|
||||||
|
spacing: 2
|
||||||
|
|
||||||
|
Repeater {
|
||||||
|
model: meter.segments
|
||||||
|
|
||||||
|
Rectangle {
|
||||||
|
required property int index
|
||||||
|
|
||||||
|
readonly property bool lit: index < Math.ceil(root.remaining * meter.segments)
|
||||||
|
|
||||||
|
width: Math.max(2, (meter.width - (meter.segments - 1) * meter.spacing) / meter.segments)
|
||||||
|
height: meter.height
|
||||||
|
color: lit ? root.tint : Theme.textAlpha(0.06)
|
||||||
|
border.width: 1
|
||||||
|
border.color: lit ? root.tint : Theme.textAlpha(0.18)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
MicroText {
|
||||||
|
visible: root.timeout > 0
|
||||||
|
text: hover.hovered ? "HOLD" : Math.ceil(root.remaining * root.timeout / 1000) + "S"
|
||||||
|
color: hover.hovered ? root.tint : Theme.muted
|
||||||
|
}
|
||||||
|
|
||||||
|
MicroText {
|
||||||
|
Layout.fillWidth: true
|
||||||
|
visible: root.timeout <= 0
|
||||||
|
text: root.critical ? "CRITICAL · STICKY" : "STICKY"
|
||||||
|
color: root.critical ? Theme.hot : Theme.muted
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,210 @@
|
|||||||
|
pragma ComponentBehavior: Bound
|
||||||
|
|
||||||
|
import QtQuick
|
||||||
|
import QtQuick.Shapes
|
||||||
|
import qs.HyprChrome.Theme
|
||||||
|
|
||||||
|
// Chamfered chrome for one notification card: PolkitPanel's silhouette, tighter
|
||||||
|
// metrics, a tintable accent (urgency) and a close box in the header.
|
||||||
|
//
|
||||||
|
// Not PolkitPanel itself: that file is the prompt's alone to restyle.
|
||||||
|
Item {
|
||||||
|
id: panel
|
||||||
|
|
||||||
|
property string panelId: ""
|
||||||
|
property string title: ""
|
||||||
|
property string meta: ""
|
||||||
|
property color accentColor: Theme.accent
|
||||||
|
|
||||||
|
property int chamfer: 12
|
||||||
|
property int padding: 12
|
||||||
|
property int outlineWidth: 1
|
||||||
|
property int accentLineThickness: 3
|
||||||
|
|
||||||
|
readonly property int headerHeight: 26
|
||||||
|
property int headerGap: 10
|
||||||
|
readonly property int headerPadding: 8
|
||||||
|
|
||||||
|
property real capGap: 4
|
||||||
|
readonly property real capOffset: panel.capGap / Math.SQRT2
|
||||||
|
readonly property real activeChamfer: Math.max(2, Math.min(panel.chamfer, panel.height / 2 - 1))
|
||||||
|
readonly property real accentLineWidth: Math.min(panel.width, slugChip.width + panel.headerPadding * 2)
|
||||||
|
|
||||||
|
default property alias content: body.data
|
||||||
|
|
||||||
|
signal closeRequested
|
||||||
|
|
||||||
|
implicitHeight: Math.round(body.y + body.height + panel.padding)
|
||||||
|
|
||||||
|
Shape {
|
||||||
|
id: panelShape
|
||||||
|
|
||||||
|
anchors.fill: parent
|
||||||
|
preferredRendererType: Shape.CurveRenderer
|
||||||
|
|
||||||
|
ShapePath {
|
||||||
|
fillColor: Theme.surface
|
||||||
|
strokeColor: Theme.hair
|
||||||
|
strokeWidth: panel.outlineWidth
|
||||||
|
|
||||||
|
startX: 0; startY: 0
|
||||||
|
PathLine { x: panelShape.width - panel.activeChamfer; y: 0 }
|
||||||
|
PathLine { x: panelShape.width; y: panel.activeChamfer }
|
||||||
|
PathLine { x: panelShape.width; y: panelShape.height }
|
||||||
|
PathLine { x: panel.activeChamfer; y: panelShape.height }
|
||||||
|
PathLine { x: 0; y: panelShape.height - panel.activeChamfer }
|
||||||
|
PathLine { x: 0; y: 0 }
|
||||||
|
}
|
||||||
|
|
||||||
|
ShapePath {
|
||||||
|
fillColor: panel.accentColor
|
||||||
|
strokeWidth: 0
|
||||||
|
|
||||||
|
startX: panelShape.width - panel.activeChamfer + panel.capOffset
|
||||||
|
startY: -panel.capOffset
|
||||||
|
PathLine { x: panelShape.width + panel.capOffset; y: panel.activeChamfer - panel.capOffset }
|
||||||
|
PathLine { x: panelShape.width + panel.capOffset; y: -panel.capOffset }
|
||||||
|
PathLine { x: panelShape.width - panel.activeChamfer + panel.capOffset; y: -panel.capOffset }
|
||||||
|
}
|
||||||
|
|
||||||
|
ShapePath {
|
||||||
|
fillColor: panel.accentColor
|
||||||
|
strokeWidth: 0
|
||||||
|
|
||||||
|
startX: panel.activeChamfer - panel.capOffset
|
||||||
|
startY: panelShape.height + panel.capOffset
|
||||||
|
PathLine { x: -panel.capOffset; y: panelShape.height - panel.activeChamfer + panel.capOffset }
|
||||||
|
PathLine { x: -panel.capOffset; y: panelShape.height + panel.capOffset }
|
||||||
|
PathLine { x: panel.activeChamfer - panel.capOffset; y: panelShape.height + panel.capOffset }
|
||||||
|
}
|
||||||
|
|
||||||
|
ShapePath {
|
||||||
|
fillColor: panel.accentColor
|
||||||
|
strokeWidth: 0
|
||||||
|
|
||||||
|
startX: 0; startY: 0
|
||||||
|
PathLine { x: panel.accentLineWidth; y: 0 }
|
||||||
|
PathLine { x: panel.accentLineWidth; y: panel.accentLineThickness }
|
||||||
|
PathLine { x: 0; y: panel.accentLineThickness }
|
||||||
|
PathLine { x: 0; y: 0 }
|
||||||
|
}
|
||||||
|
|
||||||
|
ShapePath {
|
||||||
|
fillColor: panel.accentColor
|
||||||
|
strokeWidth: 0
|
||||||
|
|
||||||
|
startX: panelShape.width; startY: panelShape.height
|
||||||
|
PathLine { x: panelShape.width - panel.accentLineWidth; y: panelShape.height }
|
||||||
|
PathLine { x: panelShape.width - panel.accentLineWidth; y: panelShape.height - panel.accentLineThickness }
|
||||||
|
PathLine { x: panelShape.width; y: panelShape.height - panel.accentLineThickness }
|
||||||
|
PathLine { x: panelShape.width; y: panelShape.height }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Row {
|
||||||
|
id: headerRow
|
||||||
|
|
||||||
|
x: panel.headerPadding
|
||||||
|
y: Math.round((panel.headerHeight - height) / 2)
|
||||||
|
width: headerEnd.x - x - 8
|
||||||
|
spacing: panel.headerPadding + 6
|
||||||
|
|
||||||
|
Rectangle {
|
||||||
|
id: slugChip
|
||||||
|
|
||||||
|
width: slugText.implicitWidth + 8
|
||||||
|
height: slugText.implicitHeight + 4
|
||||||
|
color: panel.accentColor
|
||||||
|
|
||||||
|
Text {
|
||||||
|
id: slugText
|
||||||
|
|
||||||
|
anchors.centerIn: parent
|
||||||
|
text: panel.panelId
|
||||||
|
color: Theme.surface
|
||||||
|
font.family: Theme.microFont
|
||||||
|
font.pixelSize: 10
|
||||||
|
font.bold: true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Text {
|
||||||
|
anchors.verticalCenter: slugChip.verticalCenter
|
||||||
|
width: Math.max(0, headerRow.width - slugChip.width - headerRow.spacing)
|
||||||
|
text: panel.title
|
||||||
|
color: Theme.text
|
||||||
|
font.family: Theme.displayFont
|
||||||
|
font.pixelSize: 11
|
||||||
|
font.bold: true
|
||||||
|
font.letterSpacing: 1.1
|
||||||
|
elide: Text.ElideRight
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Row {
|
||||||
|
id: headerEnd
|
||||||
|
|
||||||
|
// Clear of the top-right chamfer.
|
||||||
|
anchors.right: parent.right
|
||||||
|
anchors.rightMargin: panel.activeChamfer + 2
|
||||||
|
y: Math.round((panel.headerHeight - height) / 2)
|
||||||
|
spacing: 8
|
||||||
|
|
||||||
|
Text {
|
||||||
|
anchors.verticalCenter: parent.verticalCenter
|
||||||
|
visible: panel.meta.length > 0
|
||||||
|
text: panel.meta
|
||||||
|
color: Theme.muted
|
||||||
|
font.family: Theme.microFont
|
||||||
|
font.pixelSize: 8
|
||||||
|
font.letterSpacing: 0.7
|
||||||
|
}
|
||||||
|
|
||||||
|
Rectangle {
|
||||||
|
id: closeBox
|
||||||
|
|
||||||
|
width: 16
|
||||||
|
height: 16
|
||||||
|
color: closeArea.containsMouse ? panel.accentColor : "transparent"
|
||||||
|
border.width: 1
|
||||||
|
border.color: closeArea.containsMouse ? panel.accentColor : Theme.disabled
|
||||||
|
|
||||||
|
Text {
|
||||||
|
anchors.centerIn: parent
|
||||||
|
text: "✕"
|
||||||
|
color: closeArea.containsMouse ? Theme.surface : Theme.muted
|
||||||
|
font.family: Theme.microFont
|
||||||
|
font.pixelSize: 9
|
||||||
|
font.bold: true
|
||||||
|
}
|
||||||
|
|
||||||
|
MouseArea {
|
||||||
|
id: closeArea
|
||||||
|
|
||||||
|
anchors.fill: parent
|
||||||
|
hoverEnabled: true
|
||||||
|
cursorShape: Qt.PointingHandCursor
|
||||||
|
onClicked: panel.closeRequested()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Rectangle {
|
||||||
|
x: 1
|
||||||
|
y: panel.headerHeight
|
||||||
|
width: parent.width - 2
|
||||||
|
height: 1
|
||||||
|
color: Theme.text
|
||||||
|
opacity: 0.12
|
||||||
|
}
|
||||||
|
|
||||||
|
// Measured by childrenRect: children carry their own size, no anchoring.
|
||||||
|
Item {
|
||||||
|
id: body
|
||||||
|
|
||||||
|
x: panel.padding
|
||||||
|
y: panel.headerHeight + panel.headerGap
|
||||||
|
width: Math.max(0, panel.width - panel.padding * 2)
|
||||||
|
height: childrenRect.height
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,131 @@
|
|||||||
|
pragma ComponentBehavior: Bound
|
||||||
|
|
||||||
|
import Quickshell
|
||||||
|
import Quickshell.Services.Notifications
|
||||||
|
import Quickshell.Wayland
|
||||||
|
import QtQuick
|
||||||
|
|
||||||
|
// Notification daemon and its popup stack, migrated from widgets/notifications.
|
||||||
|
// Owns the server and the surface; NotificationCard is the visual core.
|
||||||
|
Scope {
|
||||||
|
id: root
|
||||||
|
|
||||||
|
// Driven by the shell: the rail's monitor.
|
||||||
|
property var screen: null
|
||||||
|
|
||||||
|
// Arrival time per notification id; the server keeps none.
|
||||||
|
property var arrived: ({})
|
||||||
|
|
||||||
|
// expireTimeout is in ms (verified); -1 means "server default".
|
||||||
|
function timeoutFor(n) {
|
||||||
|
if (n.expireTimeout > 0)
|
||||||
|
return n.expireTimeout;
|
||||||
|
if (n.expireTimeout === 0 || n.urgency === NotificationUrgency.Critical)
|
||||||
|
return 0;
|
||||||
|
return 5000;
|
||||||
|
}
|
||||||
|
|
||||||
|
NotificationServer {
|
||||||
|
id: server
|
||||||
|
|
||||||
|
keepOnReload: true
|
||||||
|
bodySupported: true
|
||||||
|
bodyMarkupSupported: true
|
||||||
|
bodyHyperlinksSupported: true
|
||||||
|
imageSupported: true
|
||||||
|
actionsSupported: true
|
||||||
|
|
||||||
|
onNotification: notification => {
|
||||||
|
const id = notification.id;
|
||||||
|
root.arrived[id] = Qt.formatTime(new Date(), "hh:mm");
|
||||||
|
notification.closed.connect(() => delete root.arrived[id]);
|
||||||
|
notification.tracked = true;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
PanelWindow {
|
||||||
|
id: win
|
||||||
|
|
||||||
|
screen: root.screen
|
||||||
|
visible: server.trackedNotifications.values.length > 0
|
||||||
|
|
||||||
|
WlrLayershell.namespace: "hyprchrome-notifications"
|
||||||
|
WlrLayershell.layer: WlrLayer.Overlay
|
||||||
|
// Normal with no zone of its own: sits below the rail's exclusive zone.
|
||||||
|
exclusionMode: ExclusionMode.Normal
|
||||||
|
color: "transparent"
|
||||||
|
|
||||||
|
anchors {
|
||||||
|
top: true
|
||||||
|
right: true
|
||||||
|
}
|
||||||
|
|
||||||
|
margins.top: 12
|
||||||
|
margins.right: 16
|
||||||
|
|
||||||
|
// Slack on every side for the corner caps, which sit outside each card.
|
||||||
|
implicitWidth: 360 + 8
|
||||||
|
implicitHeight: Math.max(1, stack.implicitHeight + 8)
|
||||||
|
|
||||||
|
mask: Region {
|
||||||
|
item: stack
|
||||||
|
}
|
||||||
|
|
||||||
|
Column {
|
||||||
|
id: stack
|
||||||
|
|
||||||
|
x: 4
|
||||||
|
y: 4
|
||||||
|
width: 360
|
||||||
|
spacing: 14
|
||||||
|
|
||||||
|
Repeater {
|
||||||
|
model: server.trackedNotifications
|
||||||
|
|
||||||
|
NotificationCard {
|
||||||
|
id: card
|
||||||
|
|
||||||
|
required property Notification modelData
|
||||||
|
|
||||||
|
width: stack.width
|
||||||
|
|
||||||
|
appName: card.modelData.appName
|
||||||
|
summary: card.modelData.summary
|
||||||
|
body: card.modelData.body
|
||||||
|
appIcon: card.modelData.appIcon
|
||||||
|
image: card.modelData.image
|
||||||
|
urgency: card.modelData.urgency
|
||||||
|
time: root.arrived[card.modelData.id] ?? ""
|
||||||
|
actions: card.modelData.actions
|
||||||
|
timeout: root.timeoutFor(card.modelData)
|
||||||
|
|
||||||
|
onDismissed: card.modelData.dismiss()
|
||||||
|
onExpired: card.modelData.expire()
|
||||||
|
|
||||||
|
// invoke() closes a non-resident notification itself.
|
||||||
|
onActionInvoked: identifier => {
|
||||||
|
const actions = card.modelData.actions;
|
||||||
|
for (let i = 0; i < actions.length; i++) {
|
||||||
|
if (actions[i].identifier === identifier) {
|
||||||
|
actions[i].invoke();
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Slides in from the right edge.
|
||||||
|
opacity: 0
|
||||||
|
transform: Translate { id: slide; x: 40 }
|
||||||
|
Component.onCompleted: enter.start()
|
||||||
|
|
||||||
|
ParallelAnimation {
|
||||||
|
id: enter
|
||||||
|
|
||||||
|
NumberAnimation { target: card; property: "opacity"; to: 1; duration: 160; easing.type: Easing.OutCubic }
|
||||||
|
NumberAnimation { target: slide; property: "x"; to: 0; duration: 200; easing.type: Easing.OutCubic }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -5,7 +5,6 @@ import Quickshell
|
|||||||
import Quickshell.Widgets
|
import Quickshell.Widgets
|
||||||
import qs.widgets.bar
|
import qs.widgets.bar
|
||||||
import qs.widgets.launcher
|
import qs.widgets.launcher
|
||||||
import qs.widgets.notifications
|
|
||||||
import qs.widgets.osd
|
import qs.widgets.osd
|
||||||
import qs.widgets.systray
|
import qs.widgets.systray
|
||||||
import qs.widgets.theme
|
import qs.widgets.theme
|
||||||
@@ -36,8 +35,6 @@ Scope {
|
|||||||
OrbitLauncher {} // 10 — radial targeting arena
|
OrbitLauncher {} // 10 — radial targeting arena
|
||||||
CyberDock {} // 11 — cyberpunk bottom cartridge dock
|
CyberDock {} // 11 — cyberpunk bottom cartridge dock
|
||||||
|
|
||||||
Notifications {}
|
|
||||||
|
|
||||||
VolumeOsd {}
|
VolumeOsd {}
|
||||||
|
|
||||||
// Host vitals HUD — toggle with SUPER CTRL V.
|
// Host vitals HUD — toggle with SUPER CTRL V.
|
||||||
|
|||||||
@@ -0,0 +1,90 @@
|
|||||||
|
import QtQuick
|
||||||
|
import qs.HyprChrome.Widgets.Bar.Luna
|
||||||
|
|
||||||
|
// The rail's luna panel in both densities and every daemon state. Collapsed
|
||||||
|
// is the left column (state only), expanded the right (messages) — the two
|
||||||
|
// things the panel has to get right, side by side.
|
||||||
|
//
|
||||||
|
// ./tools/quickshell-preview/render.sh \
|
||||||
|
// tests/LunaPanelHeadless.qml \
|
||||||
|
// .artifacts/quickshell-preview/luna-panel.png 640 560
|
||||||
|
Rectangle {
|
||||||
|
width: 640
|
||||||
|
height: 560
|
||||||
|
color: "#0f1012"
|
||||||
|
|
||||||
|
Row {
|
||||||
|
x: 16
|
||||||
|
y: 16
|
||||||
|
spacing: 24
|
||||||
|
|
||||||
|
Column {
|
||||||
|
spacing: 12
|
||||||
|
|
||||||
|
LunaPanel {
|
||||||
|
expanded: false
|
||||||
|
connected: true
|
||||||
|
active: false
|
||||||
|
daemonState: "idle"
|
||||||
|
}
|
||||||
|
|
||||||
|
LunaPanel {
|
||||||
|
expanded: false
|
||||||
|
connected: true
|
||||||
|
active: true
|
||||||
|
daemonState: "listening"
|
||||||
|
rms: 0.02
|
||||||
|
}
|
||||||
|
|
||||||
|
LunaPanel {
|
||||||
|
expanded: false
|
||||||
|
connected: true
|
||||||
|
active: true
|
||||||
|
daemonState: "thinking"
|
||||||
|
}
|
||||||
|
|
||||||
|
LunaPanel {
|
||||||
|
expanded: false
|
||||||
|
connected: true
|
||||||
|
active: true
|
||||||
|
daemonState: "speaking"
|
||||||
|
rms: 0.05
|
||||||
|
}
|
||||||
|
|
||||||
|
LunaPanel {
|
||||||
|
expanded: false
|
||||||
|
connected: false
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Column {
|
||||||
|
spacing: 12
|
||||||
|
|
||||||
|
LunaPanel {
|
||||||
|
expanded: true
|
||||||
|
connected: true
|
||||||
|
active: false
|
||||||
|
daemonState: "idle"
|
||||||
|
}
|
||||||
|
|
||||||
|
LunaPanel {
|
||||||
|
expanded: true
|
||||||
|
connected: true
|
||||||
|
active: true
|
||||||
|
daemonState: "listening"
|
||||||
|
rms: 0.02
|
||||||
|
userText: "hey luna, what's the time?"
|
||||||
|
}
|
||||||
|
|
||||||
|
LunaPanel {
|
||||||
|
expanded: true
|
||||||
|
connected: true
|
||||||
|
active: true
|
||||||
|
daemonState: "speaking"
|
||||||
|
rms: 0.05
|
||||||
|
userText: "hey luna, what's the time?"
|
||||||
|
lunaText: "It's 23:40 on Sunday night, chief. Shouldn't you be asleep?"
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,58 @@
|
|||||||
|
import QtQuick
|
||||||
|
import qs.HyprChrome.Widgets.Notifications
|
||||||
|
|
||||||
|
// Offscreen render of the three urgencies, with actions, a link and a
|
||||||
|
// part-drained meter.
|
||||||
|
//
|
||||||
|
// ./tools/quickshell-preview/render.sh \
|
||||||
|
// tests/NotificationCardHeadless.qml \
|
||||||
|
// .artifacts/quickshell-preview/notifications.png 400 520
|
||||||
|
Rectangle {
|
||||||
|
width: 400
|
||||||
|
height: 520
|
||||||
|
color: "#1b1d20"
|
||||||
|
|
||||||
|
Column {
|
||||||
|
x: 20
|
||||||
|
y: 20
|
||||||
|
width: 360
|
||||||
|
spacing: 14
|
||||||
|
|
||||||
|
NotificationCard {
|
||||||
|
width: parent.width
|
||||||
|
appName: "Firefox"
|
||||||
|
summary: "Download complete"
|
||||||
|
body: "<b>nixos-26.05-x86_64.iso</b> finished — <a href=\"file:///tmp\">open folder</a>"
|
||||||
|
appIcon: "firefox"
|
||||||
|
time: "14:32"
|
||||||
|
actions: [
|
||||||
|
{ identifier: "default", text: "Open" },
|
||||||
|
{ identifier: "show", text: "Show in folder" },
|
||||||
|
{ identifier: "dismiss", text: "Dismiss" }
|
||||||
|
]
|
||||||
|
timeout: 5000
|
||||||
|
countdown: false
|
||||||
|
remaining: 0.6
|
||||||
|
}
|
||||||
|
|
||||||
|
NotificationCard {
|
||||||
|
width: parent.width
|
||||||
|
appName: "smartd"
|
||||||
|
summary: "Disk /dev/sdb reports 8 reallocated sectors"
|
||||||
|
body: "SMART attribute 5 crossed its threshold. Back up the RAID0 set on /mnt/data now."
|
||||||
|
urgency: 2
|
||||||
|
time: "14:30"
|
||||||
|
}
|
||||||
|
|
||||||
|
NotificationCard {
|
||||||
|
width: parent.width
|
||||||
|
appName: "nix"
|
||||||
|
summary: "Build finished"
|
||||||
|
urgency: 0
|
||||||
|
time: "14:28"
|
||||||
|
timeout: 5000
|
||||||
|
countdown: false
|
||||||
|
remaining: 0.2
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -1,120 +0,0 @@
|
|||||||
import Quickshell.Services.Notifications
|
|
||||||
import Quickshell.Widgets
|
|
||||||
import QtQuick
|
|
||||||
import QtQuick.Layouts
|
|
||||||
import qs.widgets.theme
|
|
||||||
|
|
||||||
ColumnLayout {
|
|
||||||
id: root
|
|
||||||
|
|
||||||
required property Notification modelData
|
|
||||||
|
|
||||||
spacing: 0
|
|
||||||
Layout.fillWidth: true
|
|
||||||
|
|
||||||
Rectangle {
|
|
||||||
color: Theme.accent
|
|
||||||
implicitHeight: 5
|
|
||||||
|
|
||||||
Layout.fillWidth: true
|
|
||||||
}
|
|
||||||
|
|
||||||
WrapperRectangle {
|
|
||||||
Layout.fillWidth: true
|
|
||||||
|
|
||||||
margin: 12
|
|
||||||
border.width: 1
|
|
||||||
border.color: Theme.accent
|
|
||||||
color: Theme.surface
|
|
||||||
opacity: .9
|
|
||||||
|
|
||||||
ColumnLayout {
|
|
||||||
spacing: 4
|
|
||||||
|
|
||||||
Layout.fillWidth: true
|
|
||||||
|
|
||||||
RowLayout {
|
|
||||||
spacing: 8
|
|
||||||
|
|
||||||
Layout.fillWidth: true
|
|
||||||
|
|
||||||
IconImage {
|
|
||||||
visible: root.modelData.appIcon !== ""
|
|
||||||
source: root.modelData.appIcon
|
|
||||||
implicitSize: 20
|
|
||||||
}
|
|
||||||
|
|
||||||
Text {
|
|
||||||
text: root.modelData.summary
|
|
||||||
color: Theme.text
|
|
||||||
font.pointSize: 14
|
|
||||||
font.bold: true
|
|
||||||
elide: Text.ElideRight
|
|
||||||
|
|
||||||
Layout.fillWidth: true
|
|
||||||
}
|
|
||||||
|
|
||||||
Text {
|
|
||||||
id: dismissButton
|
|
||||||
|
|
||||||
text: "×"
|
|
||||||
color: Theme.accent
|
|
||||||
font.pointSize: 18
|
|
||||||
|
|
||||||
MouseArea {
|
|
||||||
anchors.fill: parent
|
|
||||||
cursorShape: Qt.PointingHandCursor
|
|
||||||
onClicked: root.modelData.dismiss()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Text {
|
|
||||||
visible: root.modelData.body !== ""
|
|
||||||
text: root.modelData.body
|
|
||||||
color: Theme.text
|
|
||||||
font.pointSize: 12
|
|
||||||
wrapMode: Text.Wrap
|
|
||||||
|
|
||||||
Layout.fillWidth: true
|
|
||||||
}
|
|
||||||
|
|
||||||
RowLayout {
|
|
||||||
visible: root.modelData.actions.length > 0
|
|
||||||
spacing: 6
|
|
||||||
|
|
||||||
Layout.fillWidth: true
|
|
||||||
|
|
||||||
Repeater {
|
|
||||||
model: root.modelData.actions
|
|
||||||
|
|
||||||
Rectangle {
|
|
||||||
id: actionButton
|
|
||||||
|
|
||||||
required property NotificationAction modelData
|
|
||||||
|
|
||||||
color: Theme.raised
|
|
||||||
border.width: 1
|
|
||||||
border.color: Theme.accent
|
|
||||||
implicitHeight: 28
|
|
||||||
implicitWidth: actionText.implicitWidth + 16
|
|
||||||
|
|
||||||
Text {
|
|
||||||
id: actionText
|
|
||||||
anchors.centerIn: parent
|
|
||||||
text: actionButton.modelData.text
|
|
||||||
color: Theme.text
|
|
||||||
font.pointSize: 12
|
|
||||||
}
|
|
||||||
|
|
||||||
MouseArea {
|
|
||||||
anchors.fill: parent
|
|
||||||
cursorShape: Qt.PointingHandCursor
|
|
||||||
onClicked: actionButton.modelData.invoke()
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,79 +0,0 @@
|
|||||||
import Quickshell
|
|
||||||
import Quickshell.Services.Notifications
|
|
||||||
import QtQuick
|
|
||||||
import QtQuick.Layouts
|
|
||||||
|
|
||||||
Scope {
|
|
||||||
id: root
|
|
||||||
|
|
||||||
NotificationServer {
|
|
||||||
id: server
|
|
||||||
|
|
||||||
keepOnReload: true
|
|
||||||
bodySupported: true
|
|
||||||
bodyMarkupSupported: true
|
|
||||||
imageSupported: true
|
|
||||||
actionsSupported: true
|
|
||||||
actionIconsSupported: true
|
|
||||||
|
|
||||||
onNotification: notification => {
|
|
||||||
notification.tracked = true;
|
|
||||||
|
|
||||||
const timeout = notification.expireTimeout === 0 ? 0 : notification.expireTimeout > 0 ? notification.expireTimeout : (notification.urgency === NotificationUrgency.Critical ? 0 : 5000);
|
|
||||||
|
|
||||||
if (timeout <= 0)
|
|
||||||
return;
|
|
||||||
|
|
||||||
let closed = false;
|
|
||||||
notification.closed.connect(() => {
|
|
||||||
closed = true;
|
|
||||||
});
|
|
||||||
|
|
||||||
const timer = expiryTimerComponent.createObject(root, {
|
|
||||||
interval: timeout
|
|
||||||
});
|
|
||||||
timer.triggered.connect(() => {
|
|
||||||
if (!closed)
|
|
||||||
notification.expire();
|
|
||||||
timer.destroy();
|
|
||||||
});
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
Component {
|
|
||||||
id: expiryTimerComponent
|
|
||||||
|
|
||||||
Timer {
|
|
||||||
running: true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
PanelWindow {
|
|
||||||
screen: Quickshell.screens.find(s => s.name === "DP-2")
|
|
||||||
|
|
||||||
color: "transparent"
|
|
||||||
|
|
||||||
anchors {
|
|
||||||
top: true
|
|
||||||
right: true
|
|
||||||
}
|
|
||||||
|
|
||||||
margins.top: 12
|
|
||||||
margins.right: 12
|
|
||||||
|
|
||||||
implicitWidth: 340
|
|
||||||
implicitHeight: column.implicitHeight
|
|
||||||
|
|
||||||
ColumnLayout {
|
|
||||||
id: column
|
|
||||||
anchors.fill: parent
|
|
||||||
spacing: 8
|
|
||||||
|
|
||||||
Repeater {
|
|
||||||
model: server.trackedNotifications
|
|
||||||
|
|
||||||
NotificationPopup {}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
Generated
+33
-33
@@ -14,11 +14,11 @@
|
|||||||
"uv2nix": "uv2nix"
|
"uv2nix": "uv2nix"
|
||||||
},
|
},
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1789662015,
|
"lastModified": 1789888175,
|
||||||
"narHash": "sha256-CveW/4U/zNIRfLpH6vMv8qisfotft8XJC47KqAhViZ0=",
|
"narHash": "sha256-BWzpZ0/HChcL8F2+hl0GoPaIsWBprMrReeOfrhLSk5o=",
|
||||||
"owner": "nix-community",
|
"owner": "nix-community",
|
||||||
"repo": "authentik-nix",
|
"repo": "authentik-nix",
|
||||||
"rev": "9a176c0a1889921f3ed19d29a047bafaf20b9a24",
|
"rev": "477392e1a239228de9a8011786cf673dfbb78c75",
|
||||||
"type": "github"
|
"type": "github"
|
||||||
},
|
},
|
||||||
"original": {
|
"original": {
|
||||||
@@ -30,16 +30,16 @@
|
|||||||
"authentik-src": {
|
"authentik-src": {
|
||||||
"flake": false,
|
"flake": false,
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1788978241,
|
"lastModified": 1789678329,
|
||||||
"narHash": "sha256-tAdDYHIur6ewVZlRz6JPtLivqctYLLtD0S8uehuuxSM=",
|
"narHash": "sha256-wt6cjarJI5NXSHgpONjqE9JiltBqkCoPyLLiIS83kuI=",
|
||||||
"owner": "goauthentik",
|
"owner": "goauthentik",
|
||||||
"repo": "authentik",
|
"repo": "authentik",
|
||||||
"rev": "dce85a5b64a429206199e9ffddc602060643f17a",
|
"rev": "e5a0d2f7572cb776eee7a3e9355937ce38973761",
|
||||||
"type": "github"
|
"type": "github"
|
||||||
},
|
},
|
||||||
"original": {
|
"original": {
|
||||||
"owner": "goauthentik",
|
"owner": "goauthentik",
|
||||||
"ref": "version/2026.8.2",
|
"ref": "version/2026.8.3",
|
||||||
"repo": "authentik",
|
"repo": "authentik",
|
||||||
"type": "github"
|
"type": "github"
|
||||||
}
|
}
|
||||||
@@ -67,11 +67,11 @@
|
|||||||
]
|
]
|
||||||
},
|
},
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1781152676,
|
"lastModified": 1789770686,
|
||||||
"narHash": "sha256-RxWs5ND31KzTG7wvMM+PMfUjyNpmIEr999lqNARaM5o=",
|
"narHash": "sha256-uZkBR7yHdIKUFB5SZdfgh1qkGfI3XmYmI/lTiquxbck=",
|
||||||
"owner": "nix-community",
|
"owner": "nix-community",
|
||||||
"repo": "disko",
|
"repo": "disko",
|
||||||
"rev": "ff8702b4de27f72b4c78573dfb89ec74e36abdf1",
|
"rev": "725ea35e410ad83be4931d1bff7e090eacaf3563",
|
||||||
"type": "github"
|
"type": "github"
|
||||||
},
|
},
|
||||||
"original": {
|
"original": {
|
||||||
@@ -255,11 +255,11 @@
|
|||||||
]
|
]
|
||||||
},
|
},
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1786747096,
|
"lastModified": 1789357259,
|
||||||
"narHash": "sha256-9QqhmaLVsPhKdMSBaWKjDqeGRn8G4ov4cVuZ6JFwXbo=",
|
"narHash": "sha256-YF2EwlbEEmLgEH0fs7H5sgz9JwwMjoKM2rY5jnGNYwE=",
|
||||||
"owner": "numtide",
|
"owner": "numtide",
|
||||||
"repo": "nix-vm-test",
|
"repo": "nix-vm-test",
|
||||||
"rev": "c8781a0ea2d8417506fff7722eae5a6316461212",
|
"rev": "91825d66623ff030255d98082b5c8af2bbc549ec",
|
||||||
"type": "github"
|
"type": "github"
|
||||||
},
|
},
|
||||||
"original": {
|
"original": {
|
||||||
@@ -286,11 +286,11 @@
|
|||||||
"treefmt-nix": "treefmt-nix"
|
"treefmt-nix": "treefmt-nix"
|
||||||
},
|
},
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1789567195,
|
"lastModified": 1790148129,
|
||||||
"narHash": "sha256-ymvPUaBvMmGvxsPbmrocpxO4XK+H+K12YjgKIyG3Rwk=",
|
"narHash": "sha256-oOp+FNzqKzFfn9+AwcvzBedUyuUniv7eGxQ0jfzFpQ8=",
|
||||||
"owner": "nix-community",
|
"owner": "nix-community",
|
||||||
"repo": "nixos-anywhere",
|
"repo": "nixos-anywhere",
|
||||||
"rev": "1c2f124e970fed2a49bd14ce0a8b4e9bff74d3b4",
|
"rev": "da83557d8b0bce57a52371b888ffd6d58724e55c",
|
||||||
"type": "github"
|
"type": "github"
|
||||||
},
|
},
|
||||||
"original": {
|
"original": {
|
||||||
@@ -339,11 +339,11 @@
|
|||||||
},
|
},
|
||||||
"nixpkgs": {
|
"nixpkgs": {
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1788316716,
|
"lastModified": 1789546076,
|
||||||
"narHash": "sha256-bc7rSpXIdn9QWGNqfWcPZWOhEVF8NoeAZkWq0XWnf/k=",
|
"narHash": "sha256-zVxLZiSnmaaPLwnhj7pwmqe3axBg/C6nG5JZsJMh2g4=",
|
||||||
"owner": "NixOS",
|
"owner": "NixOS",
|
||||||
"repo": "nixpkgs",
|
"repo": "nixpkgs",
|
||||||
"rev": "3ed67ec0a4d3c7ab4ae1f04f8ee8df07bfa506a2",
|
"rev": "b1b875982b17dabde9b4a37f3e229e74913e6db3",
|
||||||
"type": "github"
|
"type": "github"
|
||||||
},
|
},
|
||||||
"original": {
|
"original": {
|
||||||
@@ -370,11 +370,11 @@
|
|||||||
},
|
},
|
||||||
"nixpkgs-unstable": {
|
"nixpkgs-unstable": {
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1789632929,
|
"lastModified": 1790149054,
|
||||||
"narHash": "sha256-RjR8AyvGlWuw16XRj7C1YDEw4E27ciuyPeB2nDMzgTU=",
|
"narHash": "sha256-FfjXg6mAzKQBCY+7V9keGWOXG4DV2Eym/qA987lFl8A=",
|
||||||
"owner": "NixOS",
|
"owner": "NixOS",
|
||||||
"repo": "nixpkgs",
|
"repo": "nixpkgs",
|
||||||
"rev": "a32edd7654519351e48e80372a928df336394670",
|
"rev": "8825bebf6324e0579d012936eff73379af284b6d",
|
||||||
"type": "github"
|
"type": "github"
|
||||||
},
|
},
|
||||||
"original": {
|
"original": {
|
||||||
@@ -386,11 +386,11 @@
|
|||||||
},
|
},
|
||||||
"nixpkgs_2": {
|
"nixpkgs_2": {
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1789654592,
|
"lastModified": 1790113641,
|
||||||
"narHash": "sha256-vrwAiXmz+0hs/IWXZalFG/Ws6xQRcW+gClUDlfvcfjQ=",
|
"narHash": "sha256-D2aaQetafdpHxbwA4ldAxs51UX0b11pXL+JL/5L8VrM=",
|
||||||
"owner": "NixOS",
|
"owner": "NixOS",
|
||||||
"repo": "nixpkgs",
|
"repo": "nixpkgs",
|
||||||
"rev": "ecc58f32d1069a3e3f8a991e2233d365689f748d",
|
"rev": "1bc55b9def8165e82073919945c3239903fe4dc2",
|
||||||
"type": "github"
|
"type": "github"
|
||||||
},
|
},
|
||||||
"original": {
|
"original": {
|
||||||
@@ -524,11 +524,11 @@
|
|||||||
]
|
]
|
||||||
},
|
},
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1789691124,
|
"lastModified": 1789890976,
|
||||||
"narHash": "sha256-k+I+R6uwHX3VcJ7326qLV6vCahZUgsVl+i8sSU/Stxk=",
|
"narHash": "sha256-GKwH3zpy7tartuJMG0Rv/xUsdetG1QLmTVv8UKgJLmA=",
|
||||||
"owner": "Mic92",
|
"owner": "Mic92",
|
||||||
"repo": "sops-nix",
|
"repo": "sops-nix",
|
||||||
"rev": "1e73e8f7176d65e1b55e324de099bbfff4b2c574",
|
"rev": "7214124c20c1542c90deb54af50e2f53ae02711f",
|
||||||
"type": "github"
|
"type": "github"
|
||||||
},
|
},
|
||||||
"original": {
|
"original": {
|
||||||
@@ -539,16 +539,16 @@
|
|||||||
},
|
},
|
||||||
"systems": {
|
"systems": {
|
||||||
"locked": {
|
"locked": {
|
||||||
"lastModified": 1689347949,
|
"lastModified": 1680978846,
|
||||||
"narHash": "sha256-12tWmuL2zgBgZkdoB6qXZsgJEH9LR3oUgpaQq2RbI80=",
|
"narHash": "sha256-Gtqg8b/v49BFDpDetjclCYXm8mAnTrUzR0JnE2nv5aw=",
|
||||||
"owner": "nix-systems",
|
"owner": "nix-systems",
|
||||||
"repo": "default-linux",
|
"repo": "x86_64-linux",
|
||||||
"rev": "31732fcf5e8fea42e59c2488ad31a0e651500f68",
|
"rev": "2ecfcac5e15790ba6ce360ceccddb15ad16d08a8",
|
||||||
"type": "github"
|
"type": "github"
|
||||||
},
|
},
|
||||||
"original": {
|
"original": {
|
||||||
"owner": "nix-systems",
|
"owner": "nix-systems",
|
||||||
"repo": "default-linux",
|
"repo": "x86_64-linux",
|
||||||
"type": "github"
|
"type": "github"
|
||||||
}
|
}
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -24,6 +24,7 @@
|
|||||||
../../services/media/seerr.nix
|
../../services/media/seerr.nix
|
||||||
../../services/media/immich.nix
|
../../services/media/immich.nix
|
||||||
../../services/dev/gitea.nix
|
../../services/dev/gitea.nix
|
||||||
|
../../services/dev/gitea-mcp.nix
|
||||||
../../services/dev/obsidian-livesync.nix
|
../../services/dev/obsidian-livesync.nix
|
||||||
];
|
];
|
||||||
|
|
||||||
|
|||||||
@@ -47,8 +47,30 @@
|
|||||||
# the container with it — confirmed the hard way (~60-70s per start).
|
# the container with it — confirmed the hard way (~60-70s per start).
|
||||||
"x-systemd.mount-timeout=10s"
|
"x-systemd.mount-timeout=10s"
|
||||||
"_netdev"
|
"_netdev"
|
||||||
|
# Hermes' RequiresMountsFor mounts this at boot, not lazily, and nothing
|
||||||
|
# retries a failed mount.
|
||||||
|
"x-systemd.requires=jupiter-reachable.service"
|
||||||
];
|
];
|
||||||
};
|
};
|
||||||
|
|
||||||
|
# jupiter resolves to its tailnet IP, and network-online.target fires on the
|
||||||
|
# LAN link before tailscaled has routed tailscale0. 5min covers jupiter
|
||||||
|
# cold-booting alongside mars.
|
||||||
|
systemd.services.jupiter-reachable = {
|
||||||
|
description = "Wait for jupiter's samba over the tailnet";
|
||||||
|
after = [ "network-online.target" "tailscaled.service" ];
|
||||||
|
wants = [ "network-online.target" ];
|
||||||
|
serviceConfig = {
|
||||||
|
Type = "oneshot";
|
||||||
|
RemainAfterExit = true;
|
||||||
|
TimeoutStartSec = 300;
|
||||||
|
};
|
||||||
|
script = ''
|
||||||
|
until timeout 2 ${pkgs.bash}/bin/bash -c 'exec 3<>/dev/tcp/jupiter/445' 2>/dev/null; do
|
||||||
|
sleep 2
|
||||||
|
done
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
|
||||||
system.stateVersion = "26.05"; # set at install time; do NOT bump on upgrades
|
system.stateVersion = "26.05"; # set at install time; do NOT bump on upgrades
|
||||||
}
|
}
|
||||||
|
|||||||
+252
-106
@@ -52,6 +52,10 @@ let
|
|||||||
# is hers to make, anywhere inside HERMES_WRITE_SAFE_ROOT=/opt/data.
|
# is hers to make, anywhere inside HERMES_WRITE_SAFE_ROOT=/opt/data.
|
||||||
giteaHost = "git.mgaction.town";
|
giteaHost = "git.mgaction.town";
|
||||||
|
|
||||||
|
# neptun's tailnet IP: its Caddy is the only proxy whose X-Forwarded-Proto
|
||||||
|
# the dashboard should trust.
|
||||||
|
neptunTailnetIp = "100.64.0.5";
|
||||||
|
|
||||||
# luna's webhook filters, mounted READ-ONLY from the nix store rather than
|
# luna's webhook filters, mounted READ-ONLY from the nix store rather than
|
||||||
# written into hermesHome: that IS her write-safe root, so a writable copy
|
# written into hermesHome: that IS her write-safe root, so a writable copy
|
||||||
# would let her edit her own loop guard back out. A missing script fails
|
# would let her edit her own loop guard back out. A missing script fails
|
||||||
@@ -74,11 +78,7 @@ let
|
|||||||
builtins.readFile ./gitea-pr-review-prompt.md
|
builtins.readFile ./gitea-pr-review-prompt.md
|
||||||
);
|
);
|
||||||
|
|
||||||
# Mnemosyne memory provider (local SQLite, third-party plugin — not bundled
|
# Mnemosyne memory provider: third-party plugin, not in the image.
|
||||||
# with the official image). Fully built as a Nix derivation — see
|
|
||||||
# pkgs/mnemosyne-env.nix — and mounted READ-ONLY into the container at a
|
|
||||||
# fixed path. The oneshot near the bottom of this file only writes the
|
|
||||||
# plugin symlink Docker needs at $HERMES_HOME/plugins/mnemosyne.
|
|
||||||
mnemosyneEnv = pkgs.callPackage ../../pkgs/mnemosyne-env.nix { };
|
mnemosyneEnv = pkgs.callPackage ../../pkgs/mnemosyne-env.nix { };
|
||||||
|
|
||||||
# KittenTTS voice provider inputs (CPU-only; model + wheel hash-pinned).
|
# KittenTTS voice provider inputs (CPU-only; model + wheel hash-pinned).
|
||||||
@@ -89,22 +89,33 @@ let
|
|||||||
url = "https://github.com/KittenML/KittenTTS/releases/download/0.8.1/kittentts-0.8.1-py3-none-any.whl";
|
url = "https://github.com/KittenML/KittenTTS/releases/download/0.8.1/kittentts-0.8.1-py3-none-any.whl";
|
||||||
sha256 = "sha256-SCpDbE8fMZIVNxA3bkWf82iVF+vNp8KwUeL9QYe0GFE=";
|
sha256 = "sha256-SCpDbE8fMZIVNxA3bkWf82iVF+vNp8KwUeL9QYe0GFE=";
|
||||||
};
|
};
|
||||||
|
# uv parses the version out of the filename; the store hash prefix breaks it.
|
||||||
|
kittenttsWheelFile = "${pkgs.linkFarm "kittentts-wheel" {
|
||||||
|
"kittentts-0.8.1-py3-none-any.whl" = kittenttsWheel;
|
||||||
|
}}/kittentts-0.8.1-py3-none-any.whl";
|
||||||
|
kittenttsRuntimeLibs = "${pkgs.stdenv.cc.cc.lib}/lib ${pkgs.zlib}/lib";
|
||||||
kittenttsReqs = pkgs.writeText "kittentts-requirements.txt" (
|
kittenttsReqs = pkgs.writeText "kittentts-requirements.txt" (
|
||||||
builtins.readFile ./kittentts/requirements.txt
|
builtins.readFile ./kittentts/requirements.txt
|
||||||
);
|
);
|
||||||
|
# STT add-on: faster-whisper (CPU) for Telegram voice-note transcription,
|
||||||
|
# same side-venv, appended by the provisioning unit. Keeps one venv + one
|
||||||
|
# stamp for the whole local-audio stack. See kittentts/stt-requirements.txt.
|
||||||
|
kittenttsSttReqs = pkgs.writeText "kittentts-stt-requirements.txt" (
|
||||||
|
builtins.readFile ./kittentts/stt-requirements.txt
|
||||||
|
);
|
||||||
kittenttsStub = pkgs.writeText "kitten-tts-stub.py" (
|
kittenttsStub = pkgs.writeText "kitten-tts-stub.py" (
|
||||||
builtins.readFile ./kittentts/kitten-misaki-stub.py
|
builtins.readFile ./kittentts/kitten-misaki-stub.py
|
||||||
);
|
);
|
||||||
kittenttsModelOnnx = pkgs.fetchurl {
|
kittenttsModelOnnx = pkgs.fetchurl {
|
||||||
url = "https://huggingface.co/KittenML/kitten-tts-mini-0.8/resolve/main/kitten_tts_mini_v0_8.onnx";
|
url = "https://huggingface.co/KittenML/kitten-tts-mini-0.8/resolve/c02725660cea441db4c383af69f1f26f5cd00947/kitten_tts_mini_v0_8.onnx";
|
||||||
sha256 = "sha256-D1u65PxIAMmNvFRKh+z6eVEN4vuCItsw0S5b/pF335E=";
|
sha256 = "sha256-D1u65PxIAMmNvFRKh+z6eVEN4vuCItsw0S5b/pF335E=";
|
||||||
};
|
};
|
||||||
kittenttsModelVoices = pkgs.fetchurl {
|
kittenttsModelVoices = pkgs.fetchurl {
|
||||||
url = "https://huggingface.co/KittenML/kitten-tts-mini-0.8/resolve/main/voices.npz";
|
url = "https://huggingface.co/KittenML/kitten-tts-mini-0.8/resolve/c02725660cea441db4c383af69f1f26f5cd00947/voices.npz";
|
||||||
sha256 = "sha256-QK0mOJUrd7ey8wEn4mCOFp/GndJWtTvYqqNAmjMZPEI=";
|
sha256 = "sha256-QK0mOJUrd7ey8wEn4mCOFp/GndJWtTvYqqNAmjMZPEI=";
|
||||||
};
|
};
|
||||||
kittenttsModelConfig = pkgs.fetchurl {
|
kittenttsModelConfig = pkgs.fetchurl {
|
||||||
url = "https://huggingface.co/KittenML/kitten-tts-mini-0.8/resolve/main/config.json";
|
url = "https://huggingface.co/KittenML/kitten-tts-mini-0.8/resolve/c02725660cea441db4c383af69f1f26f5cd00947/config.json";
|
||||||
sha256 = "sha256-axYLybGeJOyyHoS8FPin2iH99H7HLUJFC8XPUUthgEo=";
|
sha256 = "sha256-axYLybGeJOyyHoS8FPin2iH99H7HLUJFC8XPUUthgEo=";
|
||||||
};
|
};
|
||||||
|
|
||||||
@@ -170,7 +181,7 @@ in
|
|||||||
wants = [ "network-online.target" ];
|
wants = [ "network-online.target" ];
|
||||||
after = [ "network-online.target" ];
|
after = [ "network-online.target" ];
|
||||||
unitConfig.RequiresMountsFor = [ "/mnt/jupiter" ];
|
unitConfig.RequiresMountsFor = [ "/mnt/jupiter" ];
|
||||||
path = [ pkgs.git pkgs.tea pkgs.curl pkgs.coreutils ];
|
path = [ pkgs.git pkgs.tea pkgs.curl pkgs.coreutils pkgs.yq-go pkgs.gnugrep pkgs.util-linux ];
|
||||||
serviceConfig.Type = "oneshot";
|
serviceConfig.Type = "oneshot";
|
||||||
# Everything here is either local or bounded to ~30s by the probe loop, so
|
# Everything here is either local or bounded to ~30s by the probe loop, so
|
||||||
# anything past two minutes is a hang, not slowness.
|
# anything past two minutes is a hang, not slowness.
|
||||||
@@ -241,6 +252,49 @@ in
|
|||||||
# scripts as uid ${hermesUid}.
|
# scripts as uid ${hermesUid}.
|
||||||
chown ${hermesUid}:${hermesGid} ${hermesHome}/scripts
|
chown ${hermesUid}:${hermesGid} ${hermesHome}/scripts
|
||||||
|
|
||||||
|
# trusted_proxies has no env override and Hermes owns config.yaml, so
|
||||||
|
# only this key is merged in; without it the Secure cookie flag breaks.
|
||||||
|
conf=${hermesHome}/config.yaml
|
||||||
|
[ -e "$conf" ] || : > "$conf"
|
||||||
|
# tts.provider is seeded, not pinned: `=` would stomp luna's own choice on
|
||||||
|
# every restart, and the engine is hers to change (KittenTTS stays as the
|
||||||
|
# offline fallback). `//` only fills it in when unset.
|
||||||
|
yq '.dashboard.trusted_proxies = ["${neptunTailnetIp}"]
|
||||||
|
| .tts.provider = (.tts.provider // "elevenlabs")' "$conf" > "$conf.new"
|
||||||
|
chown ${hermesUid}:${hermesGid} "$conf.new"
|
||||||
|
chmod --reference="$conf" "$conf.new"
|
||||||
|
mv -f "$conf.new" "$conf"
|
||||||
|
|
||||||
|
# The image's s6 init hands the gateway a fixed set of container env vars,
|
||||||
|
# and ELEVENLABS_API_KEY isn't among them (OPENCODE_GO_API_KEY is) — so the
|
||||||
|
# key only reaches Hermes via .env, which load_env() reads directly.
|
||||||
|
# Re-asserted every start: a sops rotation must win over a stale line.
|
||||||
|
# setpriv because root must not write into hermesHome — luna could plant a
|
||||||
|
# symlink there. The key arrives on an fd opened while still root, so it
|
||||||
|
# never lands in argv or the journal and her uid needs no /run/secrets access.
|
||||||
|
#
|
||||||
|
# API_SERVER_KEY is in the same boat, confirmed the same way: with it set
|
||||||
|
# only in the container env the server came up (so API_SERVER_ENABLED/HOST
|
||||||
|
# DID pass through) but answered every bearer token with
|
||||||
|
# "Invalid gateway API key" — the passthrough list carries the plain
|
||||||
|
# API_SERVER_* vars and drops the secret one.
|
||||||
|
write_env_key() {
|
||||||
|
setpriv --reuid=${hermesUid} --regid=${hermesGid} --clear-groups \
|
||||||
|
${pkgs.runtimeShell} -c '
|
||||||
|
set -eu
|
||||||
|
envf=$1
|
||||||
|
name=$2
|
||||||
|
umask 077
|
||||||
|
key=$(cat)
|
||||||
|
[ -e "$envf" ] || : > "$envf"
|
||||||
|
{ grep -v "^$name=" "$envf" || true
|
||||||
|
printf "%s=%s\n" "$name" "$key"; } > "$envf.new"
|
||||||
|
mv -f "$envf.new" "$envf"
|
||||||
|
' _ ${hermesHome}/.env "$1"
|
||||||
|
}
|
||||||
|
write_env_key ELEVENLABS_API_KEY < ${config.sops.secrets.elevenlabs_api_key.path}
|
||||||
|
write_env_key API_SERVER_KEY < ${config.sops.secrets.hermes_api_server_key.path}
|
||||||
|
|
||||||
if [ -d ${hermesHome}/.config ]; then
|
if [ -d ${hermesHome}/.config ]; then
|
||||||
chown ${hermesUid}:${hermesGid} ${hermesHome}/.config
|
chown ${hermesUid}:${hermesGid} ${hermesHome}/.config
|
||||||
fi
|
fi
|
||||||
@@ -262,15 +316,6 @@ in
|
|||||||
"${hermesHome}:/opt/data"
|
"${hermesHome}:/opt/data"
|
||||||
"${dropboxDir}:/opt/data/dropbox"
|
"${dropboxDir}:/opt/data/dropbox"
|
||||||
|
|
||||||
# Mnemosyne memory provider — a Nix-built python env, mounted :ro.
|
|
||||||
# Nothing fetched at boot, nothing writable from inside the container.
|
|
||||||
# The plugin symlink the oneshot at the bottom of this file writes
|
|
||||||
# points at the canonical store path (site-packages passthru), which
|
|
||||||
# is visible inside thanks to the existing /nix/store ro mount, so
|
|
||||||
# this /opt/data restatement is a readability alias, not a load
|
|
||||||
# bearing path.
|
|
||||||
"${mnemosyneEnv}:/opt/data/mnemosyne-env:ro"
|
|
||||||
|
|
||||||
# luna's Obsidian vault, synced with CouchDB on jupiter by
|
# luna's Obsidian vault, synced with CouchDB on jupiter by
|
||||||
# livesync-bridge.nix. Under /opt/data so she can write notes, not just
|
# livesync-bridge.nix. Under /opt/data so she can write notes, not just
|
||||||
# read them; the bridge runs as this same uid/gid so no chown is needed.
|
# read them; the bridge runs as this same uid/gid so no chown is needed.
|
||||||
@@ -308,12 +353,21 @@ in
|
|||||||
HERMES_DASHBOARD_HOST = "0.0.0.0"; # must be tailscale0-reachable, not just loopback
|
HERMES_DASHBOARD_HOST = "0.0.0.0"; # must be tailscale0-reachable, not just loopback
|
||||||
HERMES_DASHBOARD_OIDC_ISSUER = "https://auth.mgaction.town/application/o/hermes/";
|
HERMES_DASHBOARD_OIDC_ISSUER = "https://auth.mgaction.town/application/o/hermes/";
|
||||||
HERMES_DASHBOARD_OIDC_CLIENT_ID = "4BqdJu3htnMtSZnyEu5zHnsSOvlEbw3Ie3mYVlh6";
|
HERMES_DASHBOARD_OIDC_CLIENT_ID = "4BqdJu3htnMtSZnyEu5zHnsSOvlEbw3Ie3mYVlh6";
|
||||||
# uvicorn only trusts X-Forwarded-Proto from forwarded_allow_ips
|
# Pins the OIDC redirect_uri to https; otherwise it is rebuilt from
|
||||||
# (default 127.0.0.1); neptun's Caddy reaches this over a real routed
|
# forwarded headers (see trusted_proxies in prepare-dirs).
|
||||||
# tailnet IP, so without this it builds an http:// redirect_uri that
|
HERMES_DASHBOARD_PUBLIC_URL = "https://hermes.mgaction.town";
|
||||||
# Authentik rejects. Safe to trust any peer: 9119 is already scoped to
|
|
||||||
# loopback + tailscale0 only.
|
# KittenTTS: point huggingface_hub at the pre-seeded offline cache and
|
||||||
FORWARDED_ALLOW_IPS = "*";
|
# forbid network — no model drift, no boot-time fetch (review #4).
|
||||||
|
HF_HOME = "/opt/data/kittentts-hf";
|
||||||
|
HF_HUB_OFFLINE = "1";
|
||||||
|
|
||||||
|
# STT (local whisper): expose the side-venv's site-packages to Hermes's
|
||||||
|
# own interpreter so faster-whisper imports in-process. PYTHONPATH dirs
|
||||||
|
# don't run .pth auto-imports, but faster-whisper has no dead imports —
|
||||||
|
# it imports cleanly from a plain path injection (unlike kittentts,
|
||||||
|
# which is driven through the command wrapper for exactly that reason).
|
||||||
|
PYTHONPATH = "/opt/data/kittentts-venv/lib/python3.13/site-packages";
|
||||||
};
|
};
|
||||||
environmentFiles = [ config.sops.templates."hermes-agent.env".path ];
|
environmentFiles = [ config.sops.templates."hermes-agent.env".path ];
|
||||||
cmd = [ "gateway" "run" ];
|
cmd = [ "gateway" "run" ];
|
||||||
@@ -326,6 +380,10 @@ in
|
|||||||
];
|
];
|
||||||
requires = [ "hermes-agent-prepare-dirs.service" ];
|
requires = [ "hermes-agent-prepare-dirs.service" ];
|
||||||
unitConfig.RequiresMountsFor = [ "/mnt/jupiter" ];
|
unitConfig.RequiresMountsFor = [ "/mnt/jupiter" ];
|
||||||
|
# prepare-dirs is an inactive oneshot, and switch-to-configuration only
|
||||||
|
# restarts ACTIVE units — so editing its script otherwise lands nowhere
|
||||||
|
# until something else happens to restart the container.
|
||||||
|
restartTriggers = [ config.systemd.services.hermes-agent-prepare-dirs.script ];
|
||||||
};
|
};
|
||||||
|
|
||||||
# The two Gitea webhook routes, written as config (not via `hermes webhook
|
# The two Gitea webhook routes, written as config (not via `hermes webhook
|
||||||
@@ -462,26 +520,9 @@ in
|
|||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
|
|
||||||
# ---- Mnemosyne memory provider ----------------------------------------
|
# Hermes discovers memory providers under $HERMES_HOME/plugins; the target
|
||||||
# The provider's Python closure (mnemosyneEnv, callPackage'd
|
# is a store path, readable in the container via the /nix/store ro mount.
|
||||||
# pkgs/mnemosyne-env.nix above) is a READ-ONLY nix store path mounted into
|
# wantedBy, not requiredBy: a failure leaves Hermes on built-in memory.
|
||||||
# the container — nothing fetched at boot, nothing inside the container can
|
|
||||||
# write to it, and the mono-repo reproducibility story applies. Hermes only
|
|
||||||
# needs one mutable pickup point: $HERMES_HOME/plugins/mnemosyne, the
|
|
||||||
# symlink its discovery scan looks for. Both the plugin wrapper and its
|
|
||||||
# sibling `mnemosyne` core package live in that env's single site-packages,
|
|
||||||
# so one link covers them.
|
|
||||||
#
|
|
||||||
# Path handling: the symlink target is spelled in the CONTAINER's path
|
|
||||||
# space (/opt/data/...), because Hermes resolves the plugin from inside the
|
|
||||||
# container — the same host/container mismatch the webhook prompts already
|
|
||||||
# navigate via containerHome. Host-side the identical literal resolves
|
|
||||||
# onto the same store path through hermesHome's bind mount.
|
|
||||||
#
|
|
||||||
# Failure posture: after=, not requires= — a failed link write leaves the
|
|
||||||
# container running with whatever memory.provider falls back to Hermes's
|
|
||||||
# built-in memory, not a dead bot. The ro mount itself is evaluated at
|
|
||||||
# build time, so there is nothing provisionable to drift at runtime.
|
|
||||||
systemd.services.hermes-agent-mnemosyne-plugin = {
|
systemd.services.hermes-agent-mnemosyne-plugin = {
|
||||||
description = "Link Mnemosyne provider into the Hermes plugin dir";
|
description = "Link Mnemosyne provider into the Hermes plugin dir";
|
||||||
before = [ "podman-hermes-agent.service" ];
|
before = [ "podman-hermes-agent.service" ];
|
||||||
@@ -495,27 +536,13 @@ in
|
|||||||
};
|
};
|
||||||
script = ''
|
script = ''
|
||||||
set -euo pipefail
|
set -euo pipefail
|
||||||
|
|
||||||
pluginsDir=${hermesHome}/plugins
|
pluginsDir=${hermesHome}/plugins
|
||||||
pluginDir=$pluginsDir/mnemosyne
|
pluginDir=$pluginsDir/mnemosyne
|
||||||
# Target is the STORE path itself, not a /opt/data restatement: the
|
target=${mnemosyneEnv}/${mnemosyneEnv.sitePackages}/hermes_memory_provider
|
||||||
# container already ro-mounts /nix/store for git/tea (see the volumes
|
[ -d "$target" ] || { echo "$target missing" >&2; exit 1; }
|
||||||
# list), so the identical literal resolves on both sides of the bind
|
|
||||||
# mount. Using the canonical store path directly — not the
|
|
||||||
# /opt/data/mnemosyne-env mount — keeps one truth and still works
|
|
||||||
# whether Hermes resolves the link inside the container or host-side
|
|
||||||
# during debugging.
|
|
||||||
target="${mnemosyneEnv.sitePackages}/hermes_memory_provider"
|
|
||||||
|
|
||||||
if [ ! -d "$target" ]; then
|
|
||||||
echo "hermes_memory_provider not found in the mnemosyne env — unit bug, not transient" >&2
|
|
||||||
exit 1
|
|
||||||
fi
|
|
||||||
|
|
||||||
mkdir -p "$pluginsDir"
|
mkdir -p "$pluginsDir"
|
||||||
chown ${hermesUid}:${hermesGid} "$pluginsDir"
|
chown ${hermesUid}:${hermesGid} "$pluginsDir"
|
||||||
# Atomic swap: write to a temp name, rename over the old link. `-T`
|
|
||||||
# errors loudly if the target turned into a directory for any reason.
|
|
||||||
ln -sfn "$target" "$pluginDir.new"
|
ln -sfn "$target" "$pluginDir.new"
|
||||||
mv -Tf "$pluginDir.new" "$pluginDir"
|
mv -Tf "$pluginDir.new" "$pluginDir"
|
||||||
chown -h ${hermesUid}:${hermesGid} "$pluginDir"
|
chown -h ${hermesUid}:${hermesGid} "$pluginDir"
|
||||||
@@ -530,27 +557,51 @@ in
|
|||||||
# .pth-registered stub (kitten-misaki-stub.py) that fails loudly if misaki
|
# .pth-registered stub (kitten-misaki-stub.py) that fails loudly if misaki
|
||||||
# is ever actually used.
|
# is ever actually used.
|
||||||
#
|
#
|
||||||
# Provisioner invariants (shaped by the Mnemosyne-round review):
|
# Provisioner invariants (shaped by the Mnemosyne + KittenTTS review rounds):
|
||||||
# - `uv venv --clear`: re-provision cannot wedge on an existing dir.
|
# - Build in a ROOT-OWNED staging dir (/var/lib/hermes-kittentts, 0755 so
|
||||||
# - Root runs NO python from this venv: the unit itself does only fs
|
# the uid-986 delivery step can read it; nothing in it is secret):
|
||||||
# writes; the venv is chowned to the container uid before Hermes ever
|
# stamp, staging venv, uv cache and staging model copies all live there.
|
||||||
# imports from it. (Hermes executes provider code as uid 986.)
|
# Root never reads or executes anything the container can write — the
|
||||||
# - HF model cache is PRE-SEEDED from hash-pinned store paths, so
|
# container cannot symlink-takeover the stamp (review #1) or plant a
|
||||||
# HF_HUB_OFFLINE=1 gives zero boot-time network and no drift.
|
# wheel in the uv cache (review #2); only FINISHED artifacts are copied
|
||||||
# - Idempotent via a stamp keyed on the requirements hash; checked
|
# into hermesHome, and the stamp compares the delivered copy against
|
||||||
# against BOTH the venv python and model files being intact.
|
# staging byte-for-byte.
|
||||||
|
# - Root runs python ONLY from the staging venv (never from the delivered
|
||||||
|
# uid-986-owned tree in hermesHome) — after a root-side offline import
|
||||||
|
# check; a failing build aborts before anything is delivered.
|
||||||
|
# - HF model cache delivered to hermesHome from staging (refs/main ->
|
||||||
|
# snapshots/<real commit sha>), and the container env pins
|
||||||
|
# HF_HOME=/opt/data/kittentts-hf + HF_HUB_OFFLINE=1: zero boot-time
|
||||||
|
# network, no drift.
|
||||||
|
# - Idempotency stamp keyed on the FULL input set (requirements + wheel +
|
||||||
|
# stub + model files), not just requirements (review #5); a deleted
|
||||||
|
# delivered file falls out of the fast path and re-provisions cheaply.
|
||||||
|
# - Order after prepare-dirs (review #6) — mirrors the mnemosyne unit.
|
||||||
#
|
#
|
||||||
# Trust boundary: the venv lives inside hermesHome (HERMES_WRITE_SAFE_ROOT),
|
# Trust boundary: the DELIVERED venv lives inside hermesHome
|
||||||
# so luna can technically alter her own TTS engine. Deliberate: it's her
|
# (HERMES_WRITE_SAFE_ROOT), so luna can alter her own TTS engine — and a
|
||||||
# voice, not her jail — the webhook filter scripts remain the only
|
# deleted/modified copy just triggers a fresh delivery from the root-owned
|
||||||
|
# staging area on next boot (self-heals instead of wedging). Deliberate: it's
|
||||||
|
# her voice, not her jail — the webhook filter scripts remain the only
|
||||||
# write-protected-but-load-bearing items.
|
# write-protected-but-load-bearing items.
|
||||||
systemd.services.hermes-agent-kittentts-provision = {
|
systemd.services.hermes-agent-kittentts-provision = {
|
||||||
description = "Provision KittenTTS voice provider (side venv + offline HF cache)";
|
description = "Provision KittenTTS voice provider (side venv + offline HF cache)";
|
||||||
before = [ "podman-hermes-agent.service" ];
|
before = [ "podman-hermes-agent.service" ];
|
||||||
wantedBy = [ "podman-hermes-agent.service" ];
|
wantedBy = [ "podman-hermes-agent.service" ];
|
||||||
wants = [ "network-online.target" ];
|
wants = [ "network-online.target" ];
|
||||||
after = [ "network-online.target" ];
|
# after prepare-dirs (review #6): on a fresh state dir this unit must not
|
||||||
path = [ pkgs.uv pkgs.coreutils ];
|
# create hermesHome root-owned before prepare-dirs sets the ownership
|
||||||
|
# layout — same ordering contract the mnemosyne unit has. PLUS
|
||||||
|
# network-online ordering (review2 #3): this unit CAN download at boot
|
||||||
|
# (unlike mnemosyne's store-path build), so uv must not run before the
|
||||||
|
# network is actually up.
|
||||||
|
after = [
|
||||||
|
"network-online.target"
|
||||||
|
"hermes-agent-prepare-dirs.service"
|
||||||
|
];
|
||||||
|
requires = [ "hermes-agent-prepare-dirs.service" ];
|
||||||
|
# diffutils: `cmp` in the fast path; missing, it silently re-delivers every boot.
|
||||||
|
path = [ pkgs.uv pkgs.coreutils pkgs.diffutils pkgs.util-linux pkgs.auto-patchelf pkgs.patchelf ];
|
||||||
serviceConfig = {
|
serviceConfig = {
|
||||||
Type = "oneshot";
|
Type = "oneshot";
|
||||||
TimeoutStartSec = 600;
|
TimeoutStartSec = 600;
|
||||||
@@ -560,51 +611,146 @@ in
|
|||||||
|
|
||||||
venv=${hermesHome}/kittentts-venv
|
venv=${hermesHome}/kittentts-venv
|
||||||
hubDir=${hermesHome}/kittentts-hf/hub/models--KittenML--kitten-tts-mini-0.8
|
hubDir=${hermesHome}/kittentts-hf/hub/models--KittenML--kitten-tts-mini-0.8
|
||||||
snap=$hubDir/snapshots/kitten
|
# Real upstream commit SHA as snapshot dir: hf_hub_download resolves
|
||||||
stampFile=${hermesHome}/kittentts-provision.stamp
|
# refs/main -> snapshots/<sha>; "kitten" (review #3) is never found
|
||||||
reqHash=$(sha256sum ${kittenttsReqs} | cut -d' ' -f1)
|
# offline and silently triggers a re-download.
|
||||||
|
modelSha=c02725660cea441db4c383af69f1f26f5cd00947
|
||||||
|
snap=$hubDir/snapshots/$modelSha
|
||||||
|
# REVIEW #1/#2: nothing root touches lives in hermesHome. Stamp, staging
|
||||||
|
# venv and uv cache live root-owned under /var/lib/hermes-kittentts (a
|
||||||
|
# path the container can not pathwrite or symlinks into its own tree);
|
||||||
|
# the FINISHED staging venv and model files are the only things copied
|
||||||
|
# into hermesHome, and only after being validated. Root doesn't follow
|
||||||
|
# any uid-986-writable path while running as root.
|
||||||
|
stageDir=/var/lib/hermes-kittentts
|
||||||
|
# Root-side staging of the DELIVERED TREE (venv + full HF hub layout
|
||||||
|
# including refs/main) — everything root writes lives here.
|
||||||
|
# review2 #4: root never writes into hermesHome; delivery happens as
|
||||||
|
# the container uid via setpriv, copying from these root-owned sources.
|
||||||
|
stageVenv=$stageDir/venv
|
||||||
|
stageHub=$stageDir/kittentts-hf
|
||||||
|
stageSnap=$stageDir/hf-model
|
||||||
|
# Input key: requirements + wheel + stub + model files + resolved script.
|
||||||
|
# Review #5 — a miss on the old requirements-only stamp let a changed
|
||||||
|
# wheel/stub/model URL keep running the stale install forever.
|
||||||
|
inputHash=$({ cat ${kittenttsReqs} ${kittenttsSttReqs} ${kittenttsWheel} ${kittenttsStub} \
|
||||||
|
${kittenttsModelOnnx} ${kittenttsModelVoices} ${kittenttsModelConfig}
|
||||||
|
echo ${kittenttsRuntimeLibs}; } | sha256sum | cut -d' ' -f1)
|
||||||
|
stampFile=$stageDir/provision.stamp
|
||||||
|
|
||||||
# Idempotent early exit: stamp + venv + all three model files intact.
|
# Idempotent early exit: stamp matches the full input hash, staging venv
|
||||||
if [ -f "$stampFile" ] && [ "$(cat "$stampFile")" = "$reqHash" ] \
|
# validated, delivered copy intact check runs below.
|
||||||
&& [ -x "$venv/bin/python" ] \
|
if [ -f "$stampFile" ] && [ "$(cat "$stampFile")" = "$inputHash" ] \
|
||||||
&& [ -f "$snap/kitten_tts_mini_v0_8.onnx" ] \
|
&& [ -x "$stageVenv/bin/python" ] \
|
||||||
&& [ -f "$snap/voices.npz" ] \
|
&& [ -f "$stageSnap/config.json" ]; then
|
||||||
&& [ -f "$venv/lib/python3.13/site-packages/kitten_tts_stub.py" ]; then
|
# Delivered artifacts in hermesHome must ALSO match the staging copy:
|
||||||
|
# luna can rewrite her copy, that's fine — but then the missing file
|
||||||
|
# forces a re-provision (cheap copy, not a rebuild) so deletions
|
||||||
|
# cannot wedge the gateway without a voice.
|
||||||
|
if [ -x "$venv/bin/python" ] \
|
||||||
|
&& cmp -s "$stageSnap/config.json" "$snap/config.json" 2>/dev/null \
|
||||||
|
&& cmp -s "$stageSnap/kitten_tts_mini_v0_8.onnx" "$snap/kitten_tts_mini_v0_8.onnx" 2>/dev/null \
|
||||||
|
&& cmp -s "$stageSnap/voices.npz" "$snap/voices.npz" 2>/dev/null; then
|
||||||
exit 0
|
exit 0
|
||||||
fi
|
fi
|
||||||
|
fi
|
||||||
|
|
||||||
# Venv (rebuilt rather than broken on --no-clear: uv exits 2 otherwise).
|
# ---- staging venv + staged hub tree: root-owned path, uv cache
|
||||||
uv venv "$venv" --python ${pkgs.python313}/bin/python3 --clear --quiet
|
# included. Root runs python from HERE (container can write nothing in
|
||||||
UV_CACHE_DIR=${hermesHome}/kittentts-uv-cache \
|
# this tree); the FINISHED result is copied into hermesHome AS THE
|
||||||
uv pip install --python "$venv/bin/python" --quiet \
|
# CONTAINER USER via setpriv (review2 #4) — root never writes into
|
||||||
|
# hermesHome, so no uid-986-controlled path is ever followed while
|
||||||
|
# running as root; symlink-takeover of stamps/refs/install targets is
|
||||||
|
# structurally impossible. ----
|
||||||
|
mkdir -p "$stageDir" "$stageSnap" "$stageVenv" "$stageHub"
|
||||||
|
# 0755: the setpriv'd uid-986 delivery below must be able to read it.
|
||||||
|
chmod 0755 "$stageDir"
|
||||||
|
install -m 0444 ${kittenttsModelConfig} "$stageSnap/config.json"
|
||||||
|
install -m 0444 ${kittenttsModelOnnx} "$stageSnap/kitten_tts_mini_v0_8.onnx"
|
||||||
|
install -m 0444 ${kittenttsModelVoices} "$stageSnap/voices.npz"
|
||||||
|
|
||||||
|
# Skip the venv REBUILD when staging is still valid (review2 minor:
|
||||||
|
# damaged delivery should be a copy, not a rebuild) — but only for the
|
||||||
|
# CURRENT inputs, else a bump delivers the old venv under a new stamp.
|
||||||
|
if [ "$(cat "$stampFile" 2>/dev/null)" != "$inputHash" ] \
|
||||||
|
|| ! [ -x "$stageVenv/bin/python" ] \
|
||||||
|
|| ! [ -f "$stageVenv/lib/python3.13/site-packages/kitten_tts_stub.py" ]; then
|
||||||
|
rm -rf "$stageVenv"
|
||||||
|
UV_CACHE_DIR=$stageDir/uv-cache \
|
||||||
|
uv venv "$stageVenv" --python ${pkgs.python313}/bin/python3 --quiet
|
||||||
|
UV_CACHE_DIR=$stageDir/uv-cache \
|
||||||
|
uv pip install --python "$stageVenv/bin/python" --quiet \
|
||||||
--requirement ${kittenttsReqs}
|
--requirement ${kittenttsReqs}
|
||||||
# kittentts --no-deps: its overlay of spacy/misaki[en] is what drags in
|
# kittentts --no-deps: its overlay of spacy/misaki[en] is what drags in
|
||||||
# the CUDA tree; the requirements freeze already covers its real needs.
|
# the CUDA tree; the requirements freeze already covers its real needs.
|
||||||
UV_CACHE_DIR=${hermesHome}/kittentts-uv-cache \
|
UV_CACHE_DIR=$stageDir/uv-cache \
|
||||||
uv pip install --python "$venv/bin/python" --quiet --no-deps \
|
uv pip install --python "$stageVenv/bin/python" --quiet --no-deps \
|
||||||
${kittenttsWheel}
|
${kittenttsWheelFile}
|
||||||
|
|
||||||
# Dead-import shim: .pth auto-loads kitten_tts_stub at interpreter start
|
# Dead-import shim: .pth auto-loads kitten_tts_stub at interpreter
|
||||||
# so `from misaki import en, espeak` resolves without the real misaki.en.
|
# start so `from misaki import en, espeak` resolves without the real
|
||||||
siteDir=$("$venv/bin/python" -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')
|
# misaki.en.
|
||||||
|
siteDir=$("$stageVenv/bin/python" -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')
|
||||||
cp ${kittenttsStub} "$siteDir/kitten_tts_stub.py"
|
cp ${kittenttsStub} "$siteDir/kitten_tts_stub.py"
|
||||||
printf 'import kitten_tts_stub\n' > "$siteDir/zz-kitten-stub.pth"
|
printf 'import kitten_tts_stub\n' > "$siteDir/zz-kitten-stub.pth"
|
||||||
|
|
||||||
# Seed the HF cache with the hashed model files (exact hub layout;
|
# PyPI manylinux .so's (numpy, onnxruntime) need libstdc++, which the
|
||||||
# hf_hub_download scans refs/snapshots on disk offline).
|
# Nix loader never finds, on the host or in the container.
|
||||||
mkdir -p "$hubDir/refs" "$snap"
|
auto-patchelf --paths "$stageVenv" --libs ${kittenttsRuntimeLibs}
|
||||||
install -m 0444 ${kittenttsModelOnnx} "$snap/kitten_tts_mini_v0_8.onnx"
|
|
||||||
install -m 0444 ${kittenttsModelVoices} "$snap/voices.npz"
|
|
||||||
install -m 0444 ${kittenttsModelConfig} "$snap/config.json"
|
|
||||||
|
|
||||||
# Hand ownership to the container uid BEFORE any python runs in this
|
# STT add-on: faster-whisper into the SAME venv (shares the
|
||||||
# tree (root never imports from it — that was Mnemosyne review #3).
|
# numpy/onnxruntime pins; see kittentts/stt-requirements.txt).
|
||||||
chown -R ${hermesUid}:${hermesGid} "$venv" "$hubDir"
|
UV_CACHE_DIR=$stageDir/uv-cache \
|
||||||
|
uv pip install --python "$stageVenv/bin/python" --quiet \
|
||||||
|
--requirement ${kittenttsSttReqs}
|
||||||
|
# av's bundled ffmpeg libs also need the stdc++/zlib link (already
|
||||||
|
# covered by the auto-patchelf pass above — rerun it so the newly
|
||||||
|
# installed ctranslate2/av binaries get RPATHs too).
|
||||||
|
auto-patchelf --paths "$stageVenv" --libs ${kittenttsRuntimeLibs}
|
||||||
|
fi
|
||||||
|
|
||||||
# Stamp LAST — a half-provisioned venv fails the integrity check and
|
# Stage the full delivered hub tree (exact hf_hub_download layout:
|
||||||
# re-provisions on next boot rather than being trusted.
|
# refs/main -> snapshots/<sha>; review1 #3) under root-owned staging.
|
||||||
printf '%s' "$reqHash" > "$stampFile"
|
stageModelDir=$stageHub/hub/models--KittenML--kitten-tts-mini-0.8
|
||||||
chown ${hermesUid}:${hermesGid} "$stampFile"
|
mkdir -p "$stageModelDir/refs" "$stageModelDir/snapshots/$modelSha"
|
||||||
|
printf '%s' "$modelSha" > "$stageModelDir/refs/main"
|
||||||
|
install -m 0444 ${kittenttsModelOnnx} "$stageModelDir/snapshots/$modelSha/kitten_tts_mini_v0_8.onnx"
|
||||||
|
install -m 0444 ${kittenttsModelVoices} "$stageModelDir/snapshots/$modelSha/voices.npz"
|
||||||
|
install -m 0444 ${kittenttsModelConfig} "$stageModelDir/snapshots/$modelSha/config.json"
|
||||||
|
# No blobs/ indirection: kittentts reads paths RETURNED by
|
||||||
|
# hf_hub_download, which serves the resolved snapshot file directly
|
||||||
|
# (prefer-dir layout works offline for fully-materialized files).
|
||||||
|
|
||||||
|
# Root-side sanity: the staged interpreter must construct the model
|
||||||
|
# END-TO-END offline (review2 minor — import alone doesn't exercise
|
||||||
|
# hf_hub_download; a broken cache layout must fail HERE, not in the
|
||||||
|
# gateway). Points HF_HOME at the staged hub tree itself.
|
||||||
|
HF_HOME=$stageHub \
|
||||||
|
HF_HUB_OFFLINE=1 \
|
||||||
|
PHONEMIZER_ESPEAK_LIBRARY="$("$stageVenv/bin/python" -c 'import espeakng_loader,pathlib;print(pathlib.Path(espeakng_loader.get_library_path()))')" \
|
||||||
|
PHONEMIZER_ESPEAK_DATA_PATH="$("$stageVenv/bin/python" -c 'import espeakng_loader,pathlib;print(pathlib.Path(espeakng_loader.get_data_path()))')" \
|
||||||
|
"$stageVenv/bin/python" -c 'from kittentts import KittenTTS; KittenTTS("KittenML/kitten-tts-mini-0.8"); print("kittentts offline build ok")' >/dev/null
|
||||||
|
|
||||||
|
# ---- deliver AS THE CONTAINER USER (review2 #4): root never writes
|
||||||
|
# into hermesHome, so no symlink race and no `chown` step. setpriv
|
||||||
|
# drops to uid 986, rm -rf's the old delivered copies and copies the
|
||||||
|
# fresh staging tree in. cp-as-986 also fixes review2 #2: rm+cp in one
|
||||||
|
# step, no mv -Tf rename-replace on a non-empty directory.
|
||||||
|
setpriv --reuid=${hermesUid} --regid=${hermesGid} --clear-groups \
|
||||||
|
${pkgs.runtimeShell} -c '
|
||||||
|
set -eu
|
||||||
|
rm -rf "$1" "$2"
|
||||||
|
cp -a "$3" "$1"
|
||||||
|
cp -a "$4" "$2"
|
||||||
|
' _ \
|
||||||
|
"${hermesHome}/kittentts-venv" \
|
||||||
|
"${hermesHome}/kittentts-hf" \
|
||||||
|
"$stageVenv" \
|
||||||
|
"$stageHub"
|
||||||
|
|
||||||
|
# Stamp LAST, root-owned outside hermesHome — luna can delete it (which
|
||||||
|
# forces a cheap re-delivery on next boot), not tamper via symlink.
|
||||||
|
printf '%s' "$inputHash" > "$stampFile"
|
||||||
'';
|
'';
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,17 @@
|
|||||||
|
# STT add-on pins for the KittenTTS side-venv on mars (faster-whisper CPU).
|
||||||
|
#
|
||||||
|
# APPENDED to hosts/mars/kittentts/requirements.txt by the provisioning unit
|
||||||
|
# (see hermes-agent.nix, KittenTTS section) so ONE venv covers the entire
|
||||||
|
# local-audio stack: TTS (KittenTTS) + STT (faster-whisper). Shares the
|
||||||
|
# numpy/onnxruntime pins already frozen there — no version conflicts.
|
||||||
|
#
|
||||||
|
# faster-whisper runs CTranslate2 on CPU; the tiny/base model (~150 MB)
|
||||||
|
# downloads once from the HF hub then works fully offline. Validated
|
||||||
|
# 2026-09-19 with a live Telegram voice note transcribed on i5-4460 CPU.
|
||||||
|
#
|
||||||
|
# NOTE on av: PyP AV 18.x needs libz/libstdc++ from the host. On NixOS those
|
||||||
|
# come via the nix store — the provisioning unit exports LD_LIBRARY_PATH with
|
||||||
|
# them at runtime (see hermes-agent.nix's kittenttsLDLibraryPath).
|
||||||
|
faster-whisper==1.2.1
|
||||||
|
ctranslate2==4.8.2
|
||||||
|
av==18.1.0
|
||||||
@@ -1,46 +0,0 @@
|
|||||||
# Pinned requirements for a Mnemosyne side-venv on mars.
|
|
||||||
#
|
|
||||||
# Hermes vendors its own Python (the official image's venv) and deliberately
|
|
||||||
# stays minimal: no pip module inside it, PEP 668 external-management on top.
|
|
||||||
# Installing provider packages straight into that interpreter would fight the
|
|
||||||
# image on every rebuild, so Mnemosyne (and its plugin wrapper) live in their
|
|
||||||
# own venv instead — see the provisioning unit in hosts/mars/hermes-agent.nix.
|
|
||||||
#
|
|
||||||
# Freeze captured 2026-09-19 from a verified container-side install of
|
|
||||||
# `mnemosyne-memory[embeddings]` + `mnemosyne-hermes` — side venv at
|
|
||||||
# $HERMES_HOME/mnemosyne-venv, activated via $HERMES_HOME/plugins/mnemosyne.
|
|
||||||
# Versions pinned exactly; transitive deps frozen for reproducibility
|
|
||||||
# (onnxruntime/numpy drift under a long-lived SQLite state dir is what a
|
|
||||||
# freeze is here to prevent).
|
|
||||||
#
|
|
||||||
anyio==4.15.0
|
|
||||||
certifi==2026.7.22
|
|
||||||
charset-normalizer==3.5.1
|
|
||||||
click==8.5.0
|
|
||||||
fastembed==0.8.0
|
|
||||||
filelock==3.32.5
|
|
||||||
flatbuffers==25.12.19
|
|
||||||
fsspec==2026.7.0
|
|
||||||
h11==0.16.0
|
|
||||||
hf-xet==1.6.0
|
|
||||||
httpcore==1.0.9
|
|
||||||
httpx==0.28.1
|
|
||||||
huggingface-hub==1.32.0
|
|
||||||
idna==3.19
|
|
||||||
loguru==0.7.3
|
|
||||||
mmh3==5.3.0
|
|
||||||
mnemosyne-hermes==0.5.0
|
|
||||||
mnemosyne-memory==3.15.1
|
|
||||||
numpy==2.5.3
|
|
||||||
onnxruntime==1.30.0
|
|
||||||
packaging==26.3
|
|
||||||
pillow==12.3.0
|
|
||||||
protobuf==7.36.1
|
|
||||||
py-rust-stemmers==0.1.8
|
|
||||||
pyyaml==6.0.3
|
|
||||||
requests==2.34.2
|
|
||||||
sqlite-vec==0.1.9
|
|
||||||
tokenizers==0.23.2
|
|
||||||
tqdm==4.70.0
|
|
||||||
typing-extensions==4.16.0
|
|
||||||
urllib3==2.7.0
|
|
||||||
@@ -28,6 +28,11 @@
|
|||||||
sops.secrets.opencode_go_api_key = { };
|
sops.secrets.opencode_go_api_key = { };
|
||||||
sops.secrets.telegram_bot_token = { };
|
sops.secrets.telegram_bot_token = { };
|
||||||
sops.secrets.hermes_dashboard_oidc_client_secret = { };
|
sops.secrets.hermes_dashboard_oidc_client_secret = { };
|
||||||
|
# ElevenLabs TTS/STT. Unlike the webhook secret this must live in the
|
||||||
|
# container env — luna calls the API herself. restartUnits because podman
|
||||||
|
# reads --env-file only at container creation: rotating the key without it
|
||||||
|
# re-renders the file under a container that never re-reads it.
|
||||||
|
sops.secrets.elevenlabs_api_key.restartUnits = [ "podman-hermes-agent.service" ];
|
||||||
# Same value as secrets/jupiter.yaml (the sending side), stored WITHOUT a
|
# Same value as secrets/jupiter.yaml (the sending side), stored WITHOUT a
|
||||||
# trailing newline — a stray newline would change the HMAC key and fail
|
# trailing newline — a stray newline would change the HMAC key and fail
|
||||||
# every delivery. Written host-side by hermes-agent-webhook-routes, so it
|
# every delivery. Written host-side by hermes-agent-webhook-routes, so it
|
||||||
@@ -35,6 +40,18 @@
|
|||||||
sops.secrets.gitea_hermes_webhook_secret = {
|
sops.secrets.gitea_hermes_webhook_secret = {
|
||||||
restartUnits = [ "hermes-agent-webhook-routes.service" ];
|
restartUnits = [ "hermes-agent-webhook-routes.service" ];
|
||||||
};
|
};
|
||||||
|
# Bearer token for the OpenAI-compatible API server (see the env template
|
||||||
|
# below). Same value in secrets/terra.yaml, where luna-voiced reads it.
|
||||||
|
# API_SERVER_HOST=0.0.0.0 for the same reason the dashboard binds it (see
|
||||||
|
# hermes-agent.nix): the container is --network=host and nothing opens 8642
|
||||||
|
# in the firewall, so it is reachable over tailscale0 only — LAN-closed.
|
||||||
|
# prepare-dirs writes it into luna's .env (the container env alone does not
|
||||||
|
# reach the gateway — see hermes-agent.nix); the container then has to come
|
||||||
|
# back up to read it.
|
||||||
|
sops.secrets.hermes_api_server_key.restartUnits = [
|
||||||
|
"hermes-agent-prepare-dirs.service"
|
||||||
|
"podman-hermes-agent.service"
|
||||||
|
];
|
||||||
sops.templates."hermes-agent.env".content = ''
|
sops.templates."hermes-agent.env".content = ''
|
||||||
OPENCODE_GO_API_KEY=${config.sops.placeholder.opencode_go_api_key}
|
OPENCODE_GO_API_KEY=${config.sops.placeholder.opencode_go_api_key}
|
||||||
TELEGRAM_BOT_TOKEN=${config.sops.placeholder.telegram_bot_token}
|
TELEGRAM_BOT_TOKEN=${config.sops.placeholder.telegram_bot_token}
|
||||||
@@ -43,6 +60,11 @@
|
|||||||
WEBHOOK_ENABLED=true
|
WEBHOOK_ENABLED=true
|
||||||
WEBHOOK_PORT=8644
|
WEBHOOK_PORT=8644
|
||||||
HERMES_DASHBOARD_OIDC_CLIENT_SECRET=${config.sops.placeholder.hermes_dashboard_oidc_client_secret}
|
HERMES_DASHBOARD_OIDC_CLIENT_SECRET=${config.sops.placeholder.hermes_dashboard_oidc_client_secret}
|
||||||
|
ELEVENLABS_API_KEY=${config.sops.placeholder.elevenlabs_api_key}
|
||||||
|
API_SERVER_ENABLED=true
|
||||||
|
API_SERVER_HOST=0.0.0.0
|
||||||
|
API_SERVER_PORT=8642
|
||||||
|
API_SERVER_KEY=${config.sops.placeholder.hermes_api_server_key}
|
||||||
'';
|
'';
|
||||||
|
|
||||||
# luna's gitea push token (services/dev/gitea.nix provisions the account +
|
# luna's gitea push token (services/dev/gitea.nix provisions the account +
|
||||||
|
|||||||
@@ -18,9 +18,30 @@ in
|
|||||||
../../services/monitoring/node-exporter.nix
|
../../services/monitoring/node-exporter.nix
|
||||||
../../services/desktop/desktop-hyprland.nix
|
../../services/desktop/desktop-hyprland.nix
|
||||||
../../services/desktop/desktop-apps.nix
|
../../services/desktop/desktop-apps.nix
|
||||||
|
../../services/desktop/claude-desktop.nix
|
||||||
../../services/desktop/librechat.nix
|
../../services/desktop/librechat.nix
|
||||||
|
../../services/desktop/luna-voice.nix
|
||||||
];
|
];
|
||||||
|
|
||||||
|
# Talk to luna (Hermes Agent on mars) with SUPER+V — see
|
||||||
|
# hosts/terra/home/hyprland.nix for the push-to-talk binds.
|
||||||
|
homelab.lunaVoice = {
|
||||||
|
enable = true;
|
||||||
|
hermesKeyFile = config.sops.secrets.hermes_api_server_key.path;
|
||||||
|
elevenLabsKeyFile = config.sops.secrets.elevenlabs_api_key.path;
|
||||||
|
# luna's own voice — must stay in sync with `tts.elevenlabs.voice_id` in
|
||||||
|
# her config.yaml on mars, or she sounds like someone else on the desktop
|
||||||
|
# than she does on Telegram.
|
||||||
|
voiceId = "LZAcK8Cx5QjdQhfBsJQZ";
|
||||||
|
systemPrompt = ''
|
||||||
|
You are being spoken to out loud and your reply will be read aloud by a
|
||||||
|
speech synthesiser. Answer in one or two sentences unless asked for
|
||||||
|
detail. No markdown, no lists, no code blocks — they do not survive the
|
||||||
|
trip to audio. You may use an audio tag now and then, such as [laughs],
|
||||||
|
[sighs] or [whispers]; the synthesiser performs them.
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
|
||||||
networking.hostName = "terra";
|
networking.hostName = "terra";
|
||||||
|
|
||||||
homelab.greeter = {
|
homelab.greeter = {
|
||||||
|
|||||||
@@ -195,6 +195,11 @@ in
|
|||||||
# expand / collapse the hyprchrome bar as a whole
|
# expand / collapse the hyprchrome bar as a whole
|
||||||
(bind "SUPER + A" (dsp.global "quickshell:chrome"))
|
(bind "SUPER + A" (dsp.global "quickshell:chrome"))
|
||||||
|
|
||||||
|
# toggle luna's hands-free voice mode (services/desktop/luna-voice.nix).
|
||||||
|
# Runs the CLI rather than a quickshell GlobalShortcut so the mic can
|
||||||
|
# still be switched off while the shell is reloading.
|
||||||
|
(bind "SUPER + V" (dsp.exec "luna-voice toggle"))
|
||||||
|
|
||||||
(bind "SUPER + B" (dsp.exec "vivaldi"))
|
(bind "SUPER + B" (dsp.exec "vivaldi"))
|
||||||
(bind "SUPER + E" (dsp.exec "cosmic-files"))
|
(bind "SUPER + E" (dsp.exec "cosmic-files"))
|
||||||
(bind "SUPER + SHIFT + G" (dsp.workspaceRef "game"))
|
(bind "SUPER + SHIFT + G" (dsp.workspaceRef "game"))
|
||||||
@@ -237,7 +242,7 @@ in
|
|||||||
(bind "SUPER + T" (dsp.togglespecial "terminal"))
|
(bind "SUPER + T" (dsp.togglespecial "terminal"))
|
||||||
(bind "SUPER + C" (dsp.togglespecial "communications"))
|
(bind "SUPER + C" (dsp.togglespecial "communications"))
|
||||||
(bind "SUPER + M" (dsp.togglespecial "music"))
|
(bind "SUPER + M" (dsp.togglespecial "music"))
|
||||||
(bind "SUPER + V" (dsp.togglespecial "version_control"))
|
# (bind "SUPER + V" (dsp.togglespecial "version_control"))
|
||||||
|
|
||||||
# cycle workspaces
|
# cycle workspaces
|
||||||
(bind "SUPER + ALT + up" (dsp.workspaceRef "e+1"))
|
(bind "SUPER + ALT + up" (dsp.workspaceRef "e+1"))
|
||||||
|
|||||||
@@ -27,6 +27,18 @@
|
|||||||
# tokens. All four are random, generated once with `sops --set` (see
|
# tokens. All four are random, generated once with `sops --set` (see
|
||||||
# CLAUDE.md) — losing/rotating them just invalidates existing sessions and
|
# CLAUDE.md) — losing/rotating them just invalidates existing sessions and
|
||||||
# any saved per-user API keys, nothing else depends on their value.
|
# any saved per-user API keys, nothing else depends on their value.
|
||||||
|
# luna-voiced (services/desktop/luna-voice.nix) runs as a USER service, so
|
||||||
|
# these cannot stay root:root 0400 — a user manager cannot read them and
|
||||||
|
# LoadCredential does not bridge that gap either. Hence owner, not mode.
|
||||||
|
#
|
||||||
|
# hermes_api_server_key mirrors secrets/mars.yaml (the serving side);
|
||||||
|
# elevenlabs_api_key mirrors it too, because client-direct TTS is the
|
||||||
|
# documented low-latency path — terra calls ElevenLabs itself rather than
|
||||||
|
# relaying audio through mars. Rotating that key therefore touches BOTH
|
||||||
|
# files; see CLAUDE.md.
|
||||||
|
sops.secrets.hermes_api_server_key.owner = "darman";
|
||||||
|
sops.secrets.elevenlabs_api_key.owner = "darman";
|
||||||
|
|
||||||
sops.secrets.librechat_creds_key = { };
|
sops.secrets.librechat_creds_key = { };
|
||||||
sops.secrets.librechat_creds_iv = { };
|
sops.secrets.librechat_creds_iv = { };
|
||||||
sops.secrets.librechat_jwt_secret = { };
|
sops.secrets.librechat_jwt_secret = { };
|
||||||
|
|||||||
@@ -0,0 +1,139 @@
|
|||||||
|
{ lib
|
||||||
|
, stdenv
|
||||||
|
, fetchurl
|
||||||
|
, dpkg
|
||||||
|
, autoPatchelfHook
|
||||||
|
, makeWrapper
|
||||||
|
, wrapGAppsHook3
|
||||||
|
, alsa-lib
|
||||||
|
, at-spi2-atk
|
||||||
|
, at-spi2-core
|
||||||
|
, atk
|
||||||
|
, cairo
|
||||||
|
, cups
|
||||||
|
, dbus
|
||||||
|
, expat
|
||||||
|
, glib
|
||||||
|
, gtk3
|
||||||
|
, libcap_ng
|
||||||
|
, libdrm
|
||||||
|
, libgbm
|
||||||
|
, libseccomp
|
||||||
|
, libxkbcommon
|
||||||
|
, nspr
|
||||||
|
, nss
|
||||||
|
, pango
|
||||||
|
, systemd
|
||||||
|
, libx11
|
||||||
|
, libxcb
|
||||||
|
, libxcomposite
|
||||||
|
, libxdamage
|
||||||
|
, libxext
|
||||||
|
, libxfixes
|
||||||
|
, libxrandr
|
||||||
|
, libxtst
|
||||||
|
, libGL
|
||||||
|
, libayatana-appindicator
|
||||||
|
, libnotify
|
||||||
|
, libpulseaudio
|
||||||
|
, libsecret
|
||||||
|
, pipewire
|
||||||
|
, wayland
|
||||||
|
, qemu_kvm
|
||||||
|
, xdg-utils
|
||||||
|
}:
|
||||||
|
|
||||||
|
# Anthropic's official Linux build (Debian .deb), patchelf'd in place. The
|
||||||
|
# bundled Electron binary is kept rather than swapped for nixpkgs' electron:
|
||||||
|
# app.asar ships a V8 code cache (compile-cache/*.jsc) tied to that exact build.
|
||||||
|
#
|
||||||
|
# Cowork looks for qemu on $PATH but for OVMF and virtiofsd only at fixed
|
||||||
|
# Debian paths; services/desktop/claude-desktop.nix symlinks those into /usr.
|
||||||
|
#
|
||||||
|
# Bump: take the highest `Version:` listed in
|
||||||
|
# https://downloads.claude.ai/claude-desktop/apt/stable/dists/stable/main/binary-amd64/Packages
|
||||||
|
stdenv.mkDerivation (finalAttrs: {
|
||||||
|
pname = "claude-desktop";
|
||||||
|
version = "2.7032.0";
|
||||||
|
|
||||||
|
src = fetchurl {
|
||||||
|
url = "https://downloads.claude.ai/claude-desktop/apt/stable/pool/main/c/claude-desktop/claude-desktop_${finalAttrs.version}_amd64.deb";
|
||||||
|
hash = "sha256-Hn9FBLylsvay08QSPRRdcnZH538u4tBGhQcR5h59exE=";
|
||||||
|
};
|
||||||
|
|
||||||
|
nativeBuildInputs = [ dpkg autoPatchelfHook makeWrapper wrapGAppsHook3 ];
|
||||||
|
|
||||||
|
buildInputs = [
|
||||||
|
alsa-lib
|
||||||
|
at-spi2-atk
|
||||||
|
at-spi2-core
|
||||||
|
atk
|
||||||
|
cairo
|
||||||
|
cups
|
||||||
|
dbus
|
||||||
|
expat
|
||||||
|
glib
|
||||||
|
gtk3
|
||||||
|
libcap_ng # bundled virtiofsd
|
||||||
|
libdrm
|
||||||
|
libgbm
|
||||||
|
libseccomp # bundled virtiofsd
|
||||||
|
libxkbcommon
|
||||||
|
nspr
|
||||||
|
nss
|
||||||
|
pango
|
||||||
|
stdenv.cc.cc.lib # node-pty
|
||||||
|
systemd
|
||||||
|
libx11
|
||||||
|
libxcb
|
||||||
|
libxcomposite
|
||||||
|
libxdamage
|
||||||
|
libxext
|
||||||
|
libxfixes
|
||||||
|
libxrandr
|
||||||
|
libxtst
|
||||||
|
];
|
||||||
|
|
||||||
|
# dlopen'd by Chromium, so not in any DT_NEEDED.
|
||||||
|
runtimeDependencies = [
|
||||||
|
libGL
|
||||||
|
libayatana-appindicator
|
||||||
|
libnotify
|
||||||
|
libpulseaudio
|
||||||
|
libsecret
|
||||||
|
pipewire
|
||||||
|
systemd
|
||||||
|
wayland
|
||||||
|
];
|
||||||
|
|
||||||
|
dontWrapGApps = true;
|
||||||
|
|
||||||
|
installPhase = ''
|
||||||
|
runHook preInstall
|
||||||
|
|
||||||
|
mkdir -p $out/lib $out/bin
|
||||||
|
cp -r usr/lib/claude-desktop $out/lib/
|
||||||
|
cp -r usr/share $out/share
|
||||||
|
rm -r $out/share/lintian
|
||||||
|
|
||||||
|
# Setuid is impossible in the store; Chromium falls back to its userns sandbox.
|
||||||
|
rm $out/lib/claude-desktop/chrome-sandbox
|
||||||
|
|
||||||
|
runHook postInstall
|
||||||
|
'';
|
||||||
|
|
||||||
|
postFixup = ''
|
||||||
|
makeWrapper $out/lib/claude-desktop/claude-desktop $out/bin/claude-desktop \
|
||||||
|
"''${gappsWrapperArgs[@]}" \
|
||||||
|
--prefix PATH : ${lib.makeBinPath [ qemu_kvm xdg-utils ]}
|
||||||
|
'';
|
||||||
|
|
||||||
|
meta = {
|
||||||
|
description = "Desktop application for Claude.ai";
|
||||||
|
homepage = "https://claude.ai";
|
||||||
|
license = lib.licenses.unfree;
|
||||||
|
sourceProvenance = [ lib.sourceTypes.binaryNativeCode ];
|
||||||
|
platforms = [ "x86_64-linux" ];
|
||||||
|
mainProgram = "claude-desktop";
|
||||||
|
};
|
||||||
|
})
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
{ lib
|
||||||
|
, rustPlatform
|
||||||
|
, pkg-config
|
||||||
|
, alsa-lib
|
||||||
|
}:
|
||||||
|
|
||||||
|
# luna-voiced — the push-to-talk voice loop for luna (Hermes Agent on mars).
|
||||||
|
# Wiring (secrets, whisper model, systemd user unit) lives in
|
||||||
|
# services/desktop/luna-voice.nix; this only builds the binary.
|
||||||
|
#
|
||||||
|
# cargoLock.lockFile rather than a cargoHash: the lockfile is committed, so a
|
||||||
|
# dependency bump is a visible diff instead of a hash nobody can read.
|
||||||
|
#
|
||||||
|
# alsa-lib is cpal's backend even on a pipewire box — pipewire's ALSA
|
||||||
|
# compatibility layer is what actually serves it, so there is no pipewire
|
||||||
|
# build dependency here.
|
||||||
|
rustPlatform.buildRustPackage {
|
||||||
|
pname = "luna-voice";
|
||||||
|
version = "0.1.0";
|
||||||
|
|
||||||
|
src = ./luna-voice;
|
||||||
|
cargoLock.lockFile = ./luna-voice/Cargo.lock;
|
||||||
|
|
||||||
|
nativeBuildInputs = [ pkg-config ];
|
||||||
|
buildInputs = [ alsa-lib ];
|
||||||
|
|
||||||
|
meta = {
|
||||||
|
description = "Push-to-talk voice loop for luna (Hermes Agent)";
|
||||||
|
mainProgram = "luna-voice";
|
||||||
|
platforms = lib.platforms.linux;
|
||||||
|
};
|
||||||
|
}
|
||||||
Generated
+2166
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,19 @@
|
|||||||
|
[package]
|
||||||
|
name = "luna-voice"
|
||||||
|
version = "0.1.0"
|
||||||
|
edition = "2021"
|
||||||
|
|
||||||
|
[dependencies]
|
||||||
|
anyhow = "1"
|
||||||
|
base64 = "0.22"
|
||||||
|
cpal = "0.15"
|
||||||
|
futures-util = { version = "0.3", default-features = false, features = ["sink", "std"] }
|
||||||
|
hound = "3"
|
||||||
|
reqwest = { version = "0.12", default-features = false, features = ["json", "stream", "rustls-tls"] }
|
||||||
|
serde = { version = "1", features = ["derive"] }
|
||||||
|
serde_json = "1"
|
||||||
|
tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "sync", "time", "process", "io-util", "signal"] }
|
||||||
|
tokio-tungstenite = { version = "0.24", default-features = false, features = ["connect", "rustls-tls-webpki-roots"] }
|
||||||
|
|
||||||
|
[profile.release]
|
||||||
|
opt-level = 2
|
||||||
@@ -0,0 +1,311 @@
|
|||||||
|
use anyhow::{anyhow, Result};
|
||||||
|
use cpal::traits::{DeviceTrait, HostTrait, StreamTrait};
|
||||||
|
use std::collections::VecDeque;
|
||||||
|
use std::sync::atomic::{AtomicBool, AtomicU32, Ordering};
|
||||||
|
use std::sync::{Arc, Mutex};
|
||||||
|
|
||||||
|
/// What whisper wants. Everything captured is resampled to this before it
|
||||||
|
/// reaches disk.
|
||||||
|
pub const STT_RATE: u32 = 16_000;
|
||||||
|
/// What we ask ElevenLabs for (pcm_24000), resampled to the device on the way out.
|
||||||
|
pub const TTS_RATE: u32 = 24_000;
|
||||||
|
|
||||||
|
/// Mic. The stream runs for the whole session, not just while recording:
|
||||||
|
/// barge-in needs a live RMS while she is speaking, and restarting a cpal
|
||||||
|
/// stream per utterance costs ~100ms of dropped samples on pipewire.
|
||||||
|
///
|
||||||
|
/// cpal::Stream is !Send, so it is owned by a parked thread rather than by
|
||||||
|
/// this handle — which then holds nothing but atomics and a mutex and can be
|
||||||
|
/// shared across tokio tasks freely.
|
||||||
|
pub struct Capture {
|
||||||
|
pub rms: Arc<AtomicU32>,
|
||||||
|
recording: Arc<AtomicBool>,
|
||||||
|
buffer: Arc<Mutex<Vec<f32>>>,
|
||||||
|
rate: u32,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Capture {
|
||||||
|
pub fn new() -> Result<Self> {
|
||||||
|
let rms = Arc::new(AtomicU32::new(0));
|
||||||
|
let recording = Arc::new(AtomicBool::new(false));
|
||||||
|
let buffer = Arc::new(Mutex::new(Vec::new()));
|
||||||
|
|
||||||
|
let (r, rec, buf) = (rms.clone(), recording.clone(), buffer.clone());
|
||||||
|
let rate = spawn_stream(move || {
|
||||||
|
let device = input_device()?;
|
||||||
|
// Named in the log because "the mic does not work" is usually
|
||||||
|
// "the default source is the wrong device", and nothing else in
|
||||||
|
// the pipeline can tell you that.
|
||||||
|
eprintln!(
|
||||||
|
"luna-voice: input device {:?}",
|
||||||
|
device.name().unwrap_or_else(|_| "<unnamed>".into())
|
||||||
|
);
|
||||||
|
let config = device.default_input_config()?;
|
||||||
|
let rate = config.sample_rate().0;
|
||||||
|
let channels = config.channels() as usize;
|
||||||
|
let stream = device.build_input_stream(
|
||||||
|
&config.into(),
|
||||||
|
move |data: &[f32], _: &cpal::InputCallbackInfo| {
|
||||||
|
// Downmix to mono up front; nothing downstream cares about stereo.
|
||||||
|
let mono: Vec<f32> = data
|
||||||
|
.chunks(channels)
|
||||||
|
.map(|f| f.iter().sum::<f32>() / channels as f32)
|
||||||
|
.collect();
|
||||||
|
let sum: f32 = mono.iter().map(|s| s * s).sum();
|
||||||
|
let level = (sum / mono.len().max(1) as f32).sqrt();
|
||||||
|
r.store(level.to_bits(), Ordering::Relaxed);
|
||||||
|
if rec.load(Ordering::Relaxed) {
|
||||||
|
if let Ok(mut b) = buf.lock() {
|
||||||
|
b.extend_from_slice(&mono);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|e| eprintln!("luna-voice: capture error: {e}"),
|
||||||
|
None,
|
||||||
|
)?;
|
||||||
|
stream.play()?;
|
||||||
|
Ok((stream, rate))
|
||||||
|
})?;
|
||||||
|
|
||||||
|
Ok(Capture { rms, recording, buffer, rate })
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn level(&self) -> f32 {
|
||||||
|
f32::from_bits(self.rms.load(Ordering::Relaxed))
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn start(&self) {
|
||||||
|
self.buffer.lock().map(|mut b| b.clear()).ok();
|
||||||
|
self.recording.store(true, Ordering::Relaxed);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Stops recording and returns what was captured, resampled to STT_RATE.
|
||||||
|
pub fn stop(&self) -> Vec<f32> {
|
||||||
|
self.recording.store(false, Ordering::Relaxed);
|
||||||
|
let raw = self
|
||||||
|
.buffer
|
||||||
|
.lock()
|
||||||
|
.map(|mut b| std::mem::take(&mut *b))
|
||||||
|
.unwrap_or_default();
|
||||||
|
resample(&raw, self.rate, STT_RATE)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Speaker. Fed from the TTS websocket; `clear` is what makes barge-in
|
||||||
|
/// instant rather than "after the current sentence".
|
||||||
|
pub struct Playback {
|
||||||
|
queue: Arc<Mutex<VecDeque<f32>>>,
|
||||||
|
rate: u32,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Playback {
|
||||||
|
pub fn new() -> Result<Self> {
|
||||||
|
let queue: Arc<Mutex<VecDeque<f32>>> = Arc::new(Mutex::new(VecDeque::new()));
|
||||||
|
let q = queue.clone();
|
||||||
|
let rate = spawn_stream(move || {
|
||||||
|
let device = cpal::default_host()
|
||||||
|
.default_output_device()
|
||||||
|
.ok_or_else(|| anyhow!("no output device"))?;
|
||||||
|
let config = device.default_output_config()?;
|
||||||
|
let rate = config.sample_rate().0;
|
||||||
|
let channels = config.channels() as usize;
|
||||||
|
let stream = device.build_output_stream(
|
||||||
|
&config.into(),
|
||||||
|
move |data: &mut [f32], _: &cpal::OutputCallbackInfo| {
|
||||||
|
let mut q = match q.lock() {
|
||||||
|
Ok(q) => q,
|
||||||
|
Err(_) => return,
|
||||||
|
};
|
||||||
|
for frame in data.chunks_mut(channels) {
|
||||||
|
let s = q.pop_front().unwrap_or(0.0);
|
||||||
|
for out in frame.iter_mut() {
|
||||||
|
*out = s;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
},
|
||||||
|
|e| eprintln!("luna-voice: playback error: {e}"),
|
||||||
|
None,
|
||||||
|
)?;
|
||||||
|
stream.play()?;
|
||||||
|
Ok((stream, rate))
|
||||||
|
})?;
|
||||||
|
Ok(Playback { queue, rate })
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn push(&self, pcm: &[f32]) {
|
||||||
|
let out = resample(pcm, TTS_RATE, self.rate);
|
||||||
|
if let Ok(mut q) = self.queue.lock() {
|
||||||
|
q.extend(out);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn clear(&self) {
|
||||||
|
self.queue.lock().map(|mut q| q.clear()).ok();
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn pending(&self) -> usize {
|
||||||
|
self.queue.lock().map(|q| q.len()).unwrap_or(0)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Short sine burst, so you know the key registered without looking at
|
||||||
|
/// anything. Hermes' voice mode has `beep_enabled` for the same reason:
|
||||||
|
/// push-to-talk with no acknowledgement is indistinguishable from broken.
|
||||||
|
///
|
||||||
|
/// Raised-cosine envelope — a bare sine starting at full amplitude clicks.
|
||||||
|
pub fn beep(&self, freq: f32, ms: u32) {
|
||||||
|
let n = (self.rate as f32 * ms as f32 / 1000.0) as usize;
|
||||||
|
let samples: Vec<f32> = (0..n)
|
||||||
|
.map(|i| {
|
||||||
|
let t = i as f32 / self.rate as f32;
|
||||||
|
let progress = i as f32 / n as f32;
|
||||||
|
let envelope = (std::f32::consts::PI * progress).sin();
|
||||||
|
(t * freq * std::f32::consts::TAU).sin() * envelope * 0.15
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
if let Ok(mut q) = self.queue.lock() {
|
||||||
|
q.extend(samples);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The microphone, honouring LUNA_INPUT_DEVICE (a case-insensitive substring
|
||||||
|
/// of the device name) before falling back to the system default.
|
||||||
|
///
|
||||||
|
/// Worth having because the default source is a system-wide setting that has
|
||||||
|
/// nothing to do with which input you actually speak into: a box with an
|
||||||
|
/// audio interface and onboard audio has several, and picking the silent one
|
||||||
|
/// looks exactly like a broken pipeline.
|
||||||
|
fn input_device() -> Result<cpal::Device> {
|
||||||
|
let host = cpal::default_host();
|
||||||
|
if let Ok(want) = std::env::var("LUNA_INPUT_DEVICE") {
|
||||||
|
let want = want.to_lowercase();
|
||||||
|
if let Ok(devices) = host.input_devices() {
|
||||||
|
for device in devices {
|
||||||
|
if device
|
||||||
|
.name()
|
||||||
|
.map(|n| n.to_lowercase().contains(&want))
|
||||||
|
.unwrap_or(false)
|
||||||
|
{
|
||||||
|
return Ok(device);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
eprintln!("luna-voice: no input device matching {want:?}; using the default");
|
||||||
|
}
|
||||||
|
host.default_input_device()
|
||||||
|
.ok_or_else(|| anyhow!("no input device"))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Every input cpal can see, for `luna-voice devices`.
|
||||||
|
pub fn list_inputs() -> Result<Vec<String>> {
|
||||||
|
let host = cpal::default_host();
|
||||||
|
let default = host.default_input_device().and_then(|d| d.name().ok());
|
||||||
|
let mut out = Vec::new();
|
||||||
|
for device in host.input_devices()? {
|
||||||
|
let name = device.name().unwrap_or_else(|_| "<unnamed>".into());
|
||||||
|
let marker = if Some(&name) == default.as_ref() { " (default)" } else { "" };
|
||||||
|
out.push(format!("{name}{marker}"));
|
||||||
|
}
|
||||||
|
Ok(out)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Builds a cpal stream on a dedicated thread that then parks forever,
|
||||||
|
/// keeping it alive. Returns the device's sample rate, or the build error —
|
||||||
|
/// a stream that fails here must surface at startup, not as silence later.
|
||||||
|
fn spawn_stream<F>(build: F) -> Result<u32>
|
||||||
|
where
|
||||||
|
F: FnOnce() -> Result<(cpal::Stream, u32)> + Send + 'static,
|
||||||
|
{
|
||||||
|
let (tx, rx) = std::sync::mpsc::channel::<Result<u32, String>>();
|
||||||
|
std::thread::spawn(move || match build() {
|
||||||
|
Ok((_stream, rate)) => {
|
||||||
|
let _ = tx.send(Ok(rate));
|
||||||
|
// Dropping `stream` would stop the device, so the thread holds it
|
||||||
|
// for the life of the process.
|
||||||
|
loop {
|
||||||
|
std::thread::park();
|
||||||
|
}
|
||||||
|
#[allow(unreachable_code)]
|
||||||
|
drop(_stream);
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
let _ = tx.send(Err(e.to_string()));
|
||||||
|
}
|
||||||
|
});
|
||||||
|
rx.recv()
|
||||||
|
.map_err(|_| anyhow!("audio thread died during setup"))?
|
||||||
|
.map_err(|e| anyhow!(e))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Linear resampling. Speech at these rates does not justify a windowed sinc,
|
||||||
|
/// and rubato would add a dependency for inaudible gain.
|
||||||
|
pub fn resample(input: &[f32], from: u32, to: u32) -> Vec<f32> {
|
||||||
|
if from == to || input.is_empty() {
|
||||||
|
return input.to_vec();
|
||||||
|
}
|
||||||
|
let ratio = to as f64 / from as f64;
|
||||||
|
let out_len = (input.len() as f64 * ratio).round() as usize;
|
||||||
|
(0..out_len)
|
||||||
|
.map(|i| {
|
||||||
|
let src = i as f64 / ratio;
|
||||||
|
let j = src.floor() as usize;
|
||||||
|
let frac = (src - j as f64) as f32;
|
||||||
|
let a = input.get(j).copied().unwrap_or(0.0);
|
||||||
|
let b = input.get(j + 1).copied().unwrap_or(a);
|
||||||
|
a + (b - a) * frac
|
||||||
|
})
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// 16-bit mono WAV, which is the only thing whisper-cli accepts without
|
||||||
|
/// converting first.
|
||||||
|
pub fn write_wav(path: &std::path::Path, samples: &[f32], rate: u32) -> Result<()> {
|
||||||
|
let spec = hound::WavSpec {
|
||||||
|
channels: 1,
|
||||||
|
sample_rate: rate,
|
||||||
|
bits_per_sample: 16,
|
||||||
|
sample_format: hound::SampleFormat::Int,
|
||||||
|
};
|
||||||
|
let mut w = hound::WavWriter::create(path, spec)?;
|
||||||
|
for s in samples {
|
||||||
|
w.write_sample((s.clamp(-1.0, 1.0) * i16::MAX as f32) as i16)?;
|
||||||
|
}
|
||||||
|
w.finalize()?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Signed 16-bit little-endian, as ElevenLabs sends pcm_*.
|
||||||
|
pub fn pcm16_to_f32(bytes: &[u8]) -> Vec<f32> {
|
||||||
|
bytes
|
||||||
|
.chunks_exact(2)
|
||||||
|
.map(|b| i16::from_le_bytes([b[0], b[1]]) as f32 / i16::MAX as f32)
|
||||||
|
.collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resample_keeps_length_ratio() {
|
||||||
|
let input = vec![0.0f32; 1000];
|
||||||
|
assert_eq!(resample(&input, 48_000, 16_000).len(), 333);
|
||||||
|
assert_eq!(resample(&input, 24_000, 48_000).len(), 2000);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn resample_is_identity_at_equal_rates() {
|
||||||
|
let input = vec![0.1, -0.4, 0.9];
|
||||||
|
assert_eq!(resample(&input, 16_000, 16_000), input);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn pcm16_roundtrips_through_f32() {
|
||||||
|
let bytes = [0x00, 0x00, 0xff, 0x7f, 0x01, 0x80];
|
||||||
|
let got = pcm16_to_f32(&bytes);
|
||||||
|
assert_eq!(got.len(), 3);
|
||||||
|
assert!((got[0]).abs() < 1e-6);
|
||||||
|
assert!((got[1] - 1.0).abs() < 1e-3);
|
||||||
|
assert!((got[2] + 1.0).abs() < 1e-3);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,123 @@
|
|||||||
|
use anyhow::{bail, Result};
|
||||||
|
use futures_util::StreamExt;
|
||||||
|
use serde::{Deserialize, Serialize};
|
||||||
|
use tokio::sync::mpsc;
|
||||||
|
|
||||||
|
#[derive(Clone, Serialize, Deserialize, Debug)]
|
||||||
|
pub struct Message {
|
||||||
|
pub role: String,
|
||||||
|
pub content: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Message {
|
||||||
|
pub fn user(content: impl Into<String>) -> Self {
|
||||||
|
Message { role: "user".into(), content: content.into() }
|
||||||
|
}
|
||||||
|
pub fn assistant(content: impl Into<String>) -> Self {
|
||||||
|
Message { role: "assistant".into(), content: content.into() }
|
||||||
|
}
|
||||||
|
pub fn system(content: impl Into<String>) -> Self {
|
||||||
|
Message { role: "system".into(), content: content.into() }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Streams a reply from Hermes' OpenAI-compatible API, pushing text deltas
|
||||||
|
/// into `tx` as they arrive.
|
||||||
|
///
|
||||||
|
/// /v1/chat/completions with client-side history, not /v1/responses with
|
||||||
|
/// previous_response_id: the Responses API keeps the transcript server-side,
|
||||||
|
/// which is nicer, but this shape is the one every OpenAI client exercises so
|
||||||
|
/// it is the one least likely to differ between Hermes versions. Her memory
|
||||||
|
/// and skills are unaffected either way — those are not conversation state.
|
||||||
|
pub async fn stream_reply(
|
||||||
|
client: &reqwest::Client,
|
||||||
|
base_url: &str,
|
||||||
|
key: &str,
|
||||||
|
model: &str,
|
||||||
|
history: &[Message],
|
||||||
|
tx: mpsc::Sender<String>,
|
||||||
|
) -> Result<String> {
|
||||||
|
let body = serde_json::json!({
|
||||||
|
"model": model,
|
||||||
|
"messages": history,
|
||||||
|
"stream": true,
|
||||||
|
});
|
||||||
|
|
||||||
|
let resp = client
|
||||||
|
.post(format!("{}/v1/chat/completions", base_url.trim_end_matches('/')))
|
||||||
|
.bearer_auth(key)
|
||||||
|
.json(&body)
|
||||||
|
.send()
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
if !resp.status().is_success() {
|
||||||
|
let status = resp.status();
|
||||||
|
let text = resp.text().await.unwrap_or_default();
|
||||||
|
bail!("hermes {status}: {}", text.trim());
|
||||||
|
}
|
||||||
|
|
||||||
|
let mut stream = resp.bytes_stream();
|
||||||
|
let mut pending = String::new();
|
||||||
|
let mut full = String::new();
|
||||||
|
|
||||||
|
while let Some(chunk) = stream.next().await {
|
||||||
|
pending.push_str(&String::from_utf8_lossy(&chunk?));
|
||||||
|
// SSE frames are separated by a blank line; a chunk boundary can land
|
||||||
|
// mid-frame, so only complete frames are consumed.
|
||||||
|
while let Some(idx) = pending.find("\n\n") {
|
||||||
|
let frame: String = pending.drain(..idx + 2).collect();
|
||||||
|
for line in frame.lines() {
|
||||||
|
let Some(data) = line.strip_prefix("data:") else { continue };
|
||||||
|
let data = data.trim();
|
||||||
|
if data == "[DONE]" {
|
||||||
|
return Ok(full);
|
||||||
|
}
|
||||||
|
if let Some(delta) = parse_delta(data) {
|
||||||
|
full.push_str(&delta);
|
||||||
|
// A closed receiver means the reply was cancelled (barge-in).
|
||||||
|
if tx.send(delta).await.is_err() {
|
||||||
|
return Ok(full);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(full)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Pulls choices[0].delta.content out of one SSE data payload. Tool-call
|
||||||
|
/// frames and role-only frames carry no content and are skipped.
|
||||||
|
fn parse_delta(data: &str) -> Option<String> {
|
||||||
|
let v: serde_json::Value = serde_json::from_str(data).ok()?;
|
||||||
|
let content = v
|
||||||
|
.get("choices")?
|
||||||
|
.get(0)?
|
||||||
|
.get("delta")?
|
||||||
|
.get("content")?
|
||||||
|
.as_str()?;
|
||||||
|
(!content.is_empty()).then(|| content.to_string())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::parse_delta;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn extracts_content_delta() {
|
||||||
|
let d = r#"{"choices":[{"delta":{"content":"hi"}}]}"#;
|
||||||
|
assert_eq!(parse_delta(d).as_deref(), Some("hi"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn ignores_role_only_and_tool_frames() {
|
||||||
|
assert!(parse_delta(r#"{"choices":[{"delta":{"role":"assistant"}}]}"#).is_none());
|
||||||
|
assert!(parse_delta(r#"{"choices":[{"delta":{"tool_calls":[]}}]}"#).is_none());
|
||||||
|
assert!(parse_delta(r#"{"choices":[{"delta":{"content":""}}]}"#).is_none());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn ignores_malformed_json_rather_than_failing_the_stream() {
|
||||||
|
assert!(parse_delta("not json").is_none());
|
||||||
|
assert!(parse_delta("{}").is_none());
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,101 @@
|
|||||||
|
use anyhow::{Context, Result};
|
||||||
|
use std::path::PathBuf;
|
||||||
|
|
||||||
|
/// Everything the daemon needs, all from the environment so the NixOS module
|
||||||
|
/// is the only place that knows paths. Secrets arrive as *_FILE (sops paths),
|
||||||
|
/// never as values — an env var is readable from /proc for anything of the
|
||||||
|
/// user's, a 0400 file is not.
|
||||||
|
pub struct Config {
|
||||||
|
pub hermes_url: String,
|
||||||
|
pub hermes_key: String,
|
||||||
|
pub model: String,
|
||||||
|
pub system_prompt: Option<String>,
|
||||||
|
/// Prefixed to every spoken message so luna can tell this channel apart
|
||||||
|
/// from Telegram or the dashboard. Her own profile on mars keys off it —
|
||||||
|
/// which is why it is a per-message marker and not a system prompt: a
|
||||||
|
/// system prompt is this client's opinion, while the marker survives into
|
||||||
|
/// whatever history Hermes keeps and still identifies the channel later.
|
||||||
|
pub voice_marker: Option<String>,
|
||||||
|
|
||||||
|
pub eleven_key: String,
|
||||||
|
pub voice_id: String,
|
||||||
|
pub tts_model: String,
|
||||||
|
|
||||||
|
pub whisper_bin: String,
|
||||||
|
pub whisper_model: PathBuf,
|
||||||
|
|
||||||
|
pub socket_path: PathBuf,
|
||||||
|
/// Silence that ends an utterance in hands-free mode. Hermes' own default.
|
||||||
|
pub silence_ms: u64,
|
||||||
|
/// Level above the running noise floor that starts one.
|
||||||
|
pub vad_mult: f32,
|
||||||
|
/// Speech during playback louder than noise_floor * this counts as barge-in.
|
||||||
|
pub barge_in_mult: f32,
|
||||||
|
pub barge_in_grace_ms: u64,
|
||||||
|
}
|
||||||
|
|
||||||
|
fn env(key: &str) -> Result<String> {
|
||||||
|
std::env::var(key).with_context(|| format!("{key} is not set"))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn env_or(key: &str, default: &str) -> String {
|
||||||
|
std::env::var(key).unwrap_or_else(|_| default.to_string())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Reads a secret out of a file and trims it. A trailing newline in a bearer
|
||||||
|
/// token produces a 401 whose message says nothing about whitespace.
|
||||||
|
fn secret(key: &str) -> Result<String> {
|
||||||
|
let path = env(key)?;
|
||||||
|
let raw = std::fs::read_to_string(&path).with_context(|| format!("reading {path}"))?;
|
||||||
|
Ok(raw.trim().to_string())
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Config {
|
||||||
|
pub fn from_env() -> Result<Self> {
|
||||||
|
let runtime = env_or(
|
||||||
|
"XDG_RUNTIME_DIR",
|
||||||
|
&format!("/run/user/{}", unsafe { libc_getuid() }),
|
||||||
|
);
|
||||||
|
Ok(Config {
|
||||||
|
hermes_url: env_or("LUNA_HERMES_URL", "http://mars.orbit.sol:8642"),
|
||||||
|
hermes_key: secret("LUNA_HERMES_KEY_FILE")?,
|
||||||
|
model: env_or("LUNA_MODEL", "hermes"),
|
||||||
|
system_prompt: std::env::var("LUNA_SYSTEM_PROMPT").ok(),
|
||||||
|
// Empty string disables it rather than sending a bare prefix.
|
||||||
|
voice_marker: match env_or("LUNA_VOICE_MARKER", "[voice]") {
|
||||||
|
m if m.is_empty() => None,
|
||||||
|
m => Some(m),
|
||||||
|
},
|
||||||
|
|
||||||
|
eleven_key: secret("LUNA_ELEVENLABS_KEY_FILE")?,
|
||||||
|
voice_id: env("LUNA_VOICE_ID")?,
|
||||||
|
tts_model: env_or("LUNA_TTS_MODEL", "eleven_v3_conversational"),
|
||||||
|
|
||||||
|
whisper_bin: env_or("LUNA_WHISPER_BIN", "whisper-cli"),
|
||||||
|
whisper_model: PathBuf::from(env("LUNA_WHISPER_MODEL")?),
|
||||||
|
|
||||||
|
socket_path: PathBuf::from(env_or(
|
||||||
|
"LUNA_SOCKET",
|
||||||
|
&format!("{runtime}/luna-voiced.sock"),
|
||||||
|
)),
|
||||||
|
silence_ms: env_or("LUNA_SILENCE_MS", "2000").parse().unwrap_or(2000),
|
||||||
|
// 1.8, not Hermes' 3.0-ish: terra's interface has speech only
|
||||||
|
// ~6dB over its own ambient (0.0045 against 0.0022 measured), and
|
||||||
|
// no multiplier above ~2 can separate those. The real fix is gain
|
||||||
|
// on the interface; this makes it work at the gain it has.
|
||||||
|
vad_mult: env_or("LUNA_VAD_MULT", "1.8").parse().unwrap_or(1.8),
|
||||||
|
barge_in_mult: env_or("LUNA_BARGE_IN_MULT", "3.0").parse().unwrap_or(3.0),
|
||||||
|
barge_in_grace_ms: env_or("LUNA_BARGE_IN_GRACE_MS", "500")
|
||||||
|
.parse()
|
||||||
|
.unwrap_or(500),
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Avoids pulling in the libc crate for one call.
|
||||||
|
unsafe fn libc_getuid() -> u32 {
|
||||||
|
extern "C" {
|
||||||
|
fn getuid() -> u32;
|
||||||
|
}
|
||||||
|
getuid()
|
||||||
|
}
|
||||||
@@ -0,0 +1,239 @@
|
|||||||
|
use anyhow::Result;
|
||||||
|
use serde::Serialize;
|
||||||
|
use std::path::{Path, PathBuf};
|
||||||
|
use std::sync::{Arc, Mutex};
|
||||||
|
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
|
||||||
|
use tokio::net::{UnixListener, UnixStream};
|
||||||
|
use tokio::sync::{broadcast, mpsc};
|
||||||
|
|
||||||
|
use crate::visemes::Viseme;
|
||||||
|
|
||||||
|
/// Daemon -> shell. Newline-delimited JSON; the shell reads it with a
|
||||||
|
/// SplitParser and never has to spawn anything.
|
||||||
|
#[derive(Serialize, Clone, Debug)]
|
||||||
|
#[serde(tag = "type", rename_all = "snake_case")]
|
||||||
|
pub enum Event {
|
||||||
|
/// Whether voice mode is on at all. Distinct from `state`: the widget is
|
||||||
|
/// only on screen while this is true, and every state inside it is a
|
||||||
|
/// phase of an active conversation.
|
||||||
|
Mode { active: bool },
|
||||||
|
State { value: &'static str },
|
||||||
|
/// Emitted when the first PCM frame reaches the speaker. Viseme offsets
|
||||||
|
/// are relative to THIS, so the two processes never compare clocks.
|
||||||
|
SpeechStart,
|
||||||
|
Visemes { seq: Vec<Viseme> },
|
||||||
|
Amplitude { rms: f32 },
|
||||||
|
/// What was said, and by whom. Her side is emitted sentence by sentence
|
||||||
|
/// as it goes to the synthesiser, so subtitles can lead the audio the
|
||||||
|
/// same way the visemes do.
|
||||||
|
Caption { speaker: &'static str, text: String },
|
||||||
|
SpeechStop { reason: &'static str },
|
||||||
|
Error { message: String },
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Shell -> daemon.
|
||||||
|
#[derive(Debug, Clone)]
|
||||||
|
pub enum Command {
|
||||||
|
Ptt { down: bool },
|
||||||
|
Cancel,
|
||||||
|
/// Run a turn from typed text, skipping the microphone entirely. This is
|
||||||
|
/// how the brain -> TTS -> speaker half gets tested without speaking, and
|
||||||
|
/// how you tell a broken mic from a broken pipeline.
|
||||||
|
Say { text: String },
|
||||||
|
/// Turn hands-free voice mode on, off, or (None) to the opposite of what
|
||||||
|
/// it is now — the keybind sends None so one key does both.
|
||||||
|
Mode { on: Option<bool> },
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct Ipc {
|
||||||
|
pub events: broadcast::Sender<Event>,
|
||||||
|
path: PathBuf,
|
||||||
|
/// Replayed to every client the moment it connects. The shell can be
|
||||||
|
/// restarted at any time — including while voice mode is on — and a
|
||||||
|
/// broadcast-only socket would leave it showing nothing until the next
|
||||||
|
/// event happened to fire, which in an idle conversation is never.
|
||||||
|
sticky: Arc<Mutex<Vec<Event>>>,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Ipc {
|
||||||
|
/// Binds the socket and serves it until dropped. A stale socket from a
|
||||||
|
/// crashed run is removed first — bind() would fail with EADDRINUSE on a
|
||||||
|
/// file nothing is listening to.
|
||||||
|
pub fn bind(path: &Path, commands: mpsc::Sender<Command>) -> Result<Self> {
|
||||||
|
if path.exists() {
|
||||||
|
let _ = std::fs::remove_file(path);
|
||||||
|
}
|
||||||
|
if let Some(parent) = path.parent() {
|
||||||
|
std::fs::create_dir_all(parent)?;
|
||||||
|
}
|
||||||
|
let listener = UnixListener::bind(path)?;
|
||||||
|
let (events, _) = broadcast::channel(256);
|
||||||
|
let sticky: Arc<Mutex<Vec<Event>>> = Arc::new(Mutex::new(Vec::new()));
|
||||||
|
|
||||||
|
let tx = events.clone();
|
||||||
|
let accept_sticky = sticky.clone();
|
||||||
|
tokio::spawn(async move {
|
||||||
|
loop {
|
||||||
|
match listener.accept().await {
|
||||||
|
Ok((stream, _)) => {
|
||||||
|
let rx = tx.subscribe();
|
||||||
|
let cmds = commands.clone();
|
||||||
|
let snapshot = accept_sticky
|
||||||
|
.lock()
|
||||||
|
.map(|s| s.clone())
|
||||||
|
.unwrap_or_default();
|
||||||
|
tokio::spawn(client(stream, rx, cmds, snapshot));
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
eprintln!("luna-voice: accept failed: {e}");
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
Ok(Ipc { events, path: path.to_path_buf(), sticky })
|
||||||
|
}
|
||||||
|
|
||||||
|
pub fn emit(&self, event: Event) {
|
||||||
|
// Only the two events that describe a lasting condition are kept;
|
||||||
|
// replaying a viseme timeline or an old caption to a fresh client
|
||||||
|
// would animate a conversation that already finished.
|
||||||
|
if matches!(event, Event::Mode { .. } | Event::State { .. }) {
|
||||||
|
if let Ok(mut sticky) = self.sticky.lock() {
|
||||||
|
sticky.retain(|e| std::mem::discriminant(e) != std::mem::discriminant(&event));
|
||||||
|
sticky.push(event.clone());
|
||||||
|
}
|
||||||
|
}
|
||||||
|
// An error here only means nobody is connected, which is normal.
|
||||||
|
let _ = self.events.send(event);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
impl Drop for Ipc {
|
||||||
|
fn drop(&mut self) {
|
||||||
|
let _ = std::fs::remove_file(&self.path);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn client(
|
||||||
|
stream: UnixStream,
|
||||||
|
mut rx: broadcast::Receiver<Event>,
|
||||||
|
commands: mpsc::Sender<Command>,
|
||||||
|
snapshot: Vec<Event>,
|
||||||
|
) {
|
||||||
|
let (read, mut write) = stream.into_split();
|
||||||
|
let mut lines = BufReader::new(read).lines();
|
||||||
|
|
||||||
|
// Failure here must NOT abandon the connection: the one-shot clients
|
||||||
|
// (`luna-voice on`, `toggle`, …) write a command and close immediately,
|
||||||
|
// so this write hits a closed socket and errors — and returning on that
|
||||||
|
// would drop the command unread, which is exactly what it did once.
|
||||||
|
for event in snapshot {
|
||||||
|
let Ok(mut line) = serde_json::to_string(&event) else { continue };
|
||||||
|
line.push('\n');
|
||||||
|
if write.write_all(line.as_bytes()).await.is_err() {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
loop {
|
||||||
|
tokio::select! {
|
||||||
|
event = rx.recv() => {
|
||||||
|
let Ok(event) = event else { return };
|
||||||
|
let Ok(mut line) = serde_json::to_string(&event) else { continue };
|
||||||
|
line.push('\n');
|
||||||
|
if write.write_all(line.as_bytes()).await.is_err() {
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
line = lines.next_line() => {
|
||||||
|
match line {
|
||||||
|
Ok(Some(line)) => {
|
||||||
|
if let Some(cmd) = parse_command(&line) {
|
||||||
|
let _ = commands.send(cmd).await;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
_ => return,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn parse_command(line: &str) -> Option<Command> {
|
||||||
|
let v: serde_json::Value = serde_json::from_str(line).ok()?;
|
||||||
|
match v.get("type")?.as_str()? {
|
||||||
|
"ptt" => Some(Command::Ptt {
|
||||||
|
down: v.get("down").and_then(|d| d.as_bool()).unwrap_or(false),
|
||||||
|
}),
|
||||||
|
"cancel" => Some(Command::Cancel),
|
||||||
|
"say" => Some(Command::Say {
|
||||||
|
text: v.get("text")?.as_str()?.to_string(),
|
||||||
|
}),
|
||||||
|
"mode" => Some(Command::Mode {
|
||||||
|
on: v.get("on").and_then(|o| o.as_bool()),
|
||||||
|
}),
|
||||||
|
_ => None,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Client side of the same socket, for the `luna-voice ptt` subcommand that
|
||||||
|
/// Hyprland binds a key to before the quickshell widget exists.
|
||||||
|
pub async fn send_command(path: &Path, line: &str) -> Result<()> {
|
||||||
|
let mut stream = UnixStream::connect(path).await?;
|
||||||
|
stream.write_all(format!("{line}\n").as_bytes()).await?;
|
||||||
|
stream.flush().await?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn parses_ptt_commands() {
|
||||||
|
assert!(matches!(
|
||||||
|
parse_command(r#"{"type":"ptt","down":true}"#),
|
||||||
|
Some(Command::Ptt { down: true })
|
||||||
|
));
|
||||||
|
assert!(matches!(
|
||||||
|
parse_command(r#"{"type":"ptt","down":false}"#),
|
||||||
|
Some(Command::Ptt { down: false })
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn ptt_without_down_defaults_to_release() {
|
||||||
|
assert!(matches!(
|
||||||
|
parse_command(r#"{"type":"ptt"}"#),
|
||||||
|
Some(Command::Ptt { down: false })
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn mode_without_a_value_means_toggle() {
|
||||||
|
assert!(matches!(
|
||||||
|
parse_command(r#"{"type":"mode"}"#),
|
||||||
|
Some(Command::Mode { on: None })
|
||||||
|
));
|
||||||
|
assert!(matches!(
|
||||||
|
parse_command(r#"{"type":"mode","on":true}"#),
|
||||||
|
Some(Command::Mode { on: Some(true) })
|
||||||
|
));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn ignores_unknown_and_malformed() {
|
||||||
|
assert!(parse_command(r#"{"type":"nope"}"#).is_none());
|
||||||
|
assert!(parse_command("garbage").is_none());
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn events_serialise_with_a_type_tag() {
|
||||||
|
let json = serde_json::to_string(&Event::State { value: "listening" }).unwrap();
|
||||||
|
assert_eq!(json, r#"{"type":"state","value":"listening"}"#);
|
||||||
|
let json = serde_json::to_string(&Event::SpeechStart).unwrap();
|
||||||
|
assert_eq!(json, r#"{"type":"speech_start"}"#);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,611 @@
|
|||||||
|
//! luna-voiced — push-to-talk voice loop for luna (the Hermes Agent on mars).
|
||||||
|
//!
|
||||||
|
//! mic -> whisper (local, vulkan) -> Hermes /v1/chat/completions -> ElevenLabs
|
||||||
|
//! streaming TTS -> speaker, with the speaking state, amplitude and viseme
|
||||||
|
//! timeline published on a unix socket for the quickshell avatar.
|
||||||
|
//!
|
||||||
|
//! Nothing here talks to the compositor: push-to-talk arrives over the same
|
||||||
|
//! socket, from either `luna-voice ptt` or the shell's GlobalShortcut.
|
||||||
|
|
||||||
|
mod audio;
|
||||||
|
mod brain;
|
||||||
|
mod config;
|
||||||
|
mod ipc;
|
||||||
|
mod stt;
|
||||||
|
mod tts;
|
||||||
|
mod visemes;
|
||||||
|
|
||||||
|
use anyhow::Result;
|
||||||
|
use serde_json;
|
||||||
|
use config::Config;
|
||||||
|
use ipc::{Command, Event};
|
||||||
|
use std::sync::Arc;
|
||||||
|
use std::time::Duration;
|
||||||
|
use tokio::sync::mpsc;
|
||||||
|
|
||||||
|
#[tokio::main]
|
||||||
|
async fn main() -> Result<()> {
|
||||||
|
let mut args = std::env::args().skip(1);
|
||||||
|
match args.next().as_deref() {
|
||||||
|
None | Some("daemon") => daemon().await,
|
||||||
|
// Fire-and-forget clients so a keybind costs one short-lived process
|
||||||
|
// and never blocks the compositor.
|
||||||
|
Some("ptt-down") => client(r#"{"type":"ptt","down":true}"#).await,
|
||||||
|
Some("ptt-up") => client(r#"{"type":"ptt","down":false}"#).await,
|
||||||
|
Some("cancel") => client(r#"{"type":"cancel"}"#).await,
|
||||||
|
Some("toggle") => client(r#"{"type":"mode"}"#).await,
|
||||||
|
Some("on") => client(r#"{"type":"mode","on":true}"#).await,
|
||||||
|
Some("off") => client(r#"{"type":"mode","on":false}"#).await,
|
||||||
|
// The debugging tool: everything the avatar will eventually see,
|
||||||
|
// printed. `journalctl -fu luna-voiced` shows only what the daemon
|
||||||
|
// chose to log; this shows the actual protocol.
|
||||||
|
Some("watch") => watch().await,
|
||||||
|
// Opens the mic directly, bypassing the daemon: answers "is the
|
||||||
|
// microphone working at all" without any of the rest being involved.
|
||||||
|
Some("level") => level().await,
|
||||||
|
Some("devices") => {
|
||||||
|
for name in audio::list_inputs()? {
|
||||||
|
println!("{name}");
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
Some("say") => {
|
||||||
|
let text: Vec<String> = args.collect();
|
||||||
|
if text.is_empty() {
|
||||||
|
eprintln!("luna-voice: say what?");
|
||||||
|
std::process::exit(2);
|
||||||
|
}
|
||||||
|
let payload = serde_json::json!({ "type": "say", "text": text.join(" ") });
|
||||||
|
client(&payload.to_string()).await
|
||||||
|
}
|
||||||
|
Some(other) => {
|
||||||
|
eprintln!("luna-voice: unknown command {other:?}");
|
||||||
|
eprintln!(
|
||||||
|
"usage: luna-voice [daemon|toggle|on|off|say <text>|watch|level|devices|cancel]"
|
||||||
|
);
|
||||||
|
std::process::exit(2);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Level that counts as speech.
|
||||||
|
///
|
||||||
|
/// The lower clamp is the number that matters, and it has to sit under the
|
||||||
|
/// QUIETEST mic worth supporting, not under a typical one: terra's AIR 192|4
|
||||||
|
/// reads ~0.001 idle and only 0.004–0.005 for normal speech, so a 0.006 floor
|
||||||
|
/// — which looked conservative — put the trigger permanently above the user's
|
||||||
|
/// voice. Overshooting downward only costs silent false triggers.
|
||||||
|
fn root_threshold(floor: f32, mult: f32) -> f32 {
|
||||||
|
let min = std::env::var("LUNA_VAD_FLOOR")
|
||||||
|
.ok()
|
||||||
|
.and_then(|v| v.parse().ok())
|
||||||
|
.unwrap_or(0.0025);
|
||||||
|
(floor * mult).clamp(min, 0.05)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Tags a spoken line so luna can recognise the channel it came from.
|
||||||
|
///
|
||||||
|
/// Prefix rather than suffix: a long utterance can be truncated by whatever
|
||||||
|
/// history window she is running under, and the front is the end that
|
||||||
|
/// survives.
|
||||||
|
fn mark(marker: &Option<String>, text: &str) -> String {
|
||||||
|
match marker {
|
||||||
|
Some(m) => format!("{m} {text}"),
|
||||||
|
None => text.to_string(),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn socket_path() -> std::path::PathBuf {
|
||||||
|
let runtime = std::env::var("XDG_RUNTIME_DIR").unwrap_or_else(|_| "/run/user/1000".into());
|
||||||
|
std::env::var("LUNA_SOCKET")
|
||||||
|
.map(std::path::PathBuf::from)
|
||||||
|
.unwrap_or_else(|_| std::path::Path::new(&runtime).join("luna-voiced.sock"))
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn watch() -> Result<()> {
|
||||||
|
use tokio::io::{AsyncBufReadExt, BufReader};
|
||||||
|
let path = socket_path();
|
||||||
|
let stream = tokio::net::UnixStream::connect(&path)
|
||||||
|
.await
|
||||||
|
.map_err(|e| anyhow::anyhow!("{}: {e} (is luna-voiced running?)", path.display()))?;
|
||||||
|
let mut lines = BufReader::new(stream).lines();
|
||||||
|
eprintln!("luna-voice: watching {}", path.display());
|
||||||
|
while let Some(line) = lines.next_line().await? {
|
||||||
|
println!("{line}");
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Live microphone meter with the trigger point marked, so a mic that reads
|
||||||
|
/// nothing and a threshold that is set too high look different.
|
||||||
|
async fn level() -> Result<()> {
|
||||||
|
let capture = audio::Capture::new()?;
|
||||||
|
let mult: f32 = std::env::var("LUNA_VAD_MULT")
|
||||||
|
.ok()
|
||||||
|
.and_then(|v| v.parse().ok())
|
||||||
|
.unwrap_or(1.8);
|
||||||
|
let mut floor: f32 = capture.level().max(0.003);
|
||||||
|
eprintln!("luna-voice: speak — ctrl-c to stop");
|
||||||
|
loop {
|
||||||
|
let level = capture.level();
|
||||||
|
floor = floor * 0.95 + level * 0.05;
|
||||||
|
let threshold = root_threshold(floor, mult);
|
||||||
|
// Full scale 0.02, not 0.05: a quiet interface tops out around
|
||||||
|
// 0.005 and was drawing a single bar on the wider scale.
|
||||||
|
const FULL: f32 = 0.02;
|
||||||
|
let bars = ((level / FULL).min(1.0) * 40.0) as usize;
|
||||||
|
let mark = ((threshold / FULL).min(1.0) * 40.0) as usize;
|
||||||
|
let meter: String = (0..40)
|
||||||
|
.map(|i| {
|
||||||
|
if i == mark {
|
||||||
|
'|'
|
||||||
|
} else if i < bars {
|
||||||
|
'#'
|
||||||
|
} else {
|
||||||
|
'.'
|
||||||
|
}
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
print!(
|
||||||
|
"\r{meter} rms {level:.4} floor {floor:.4} trigger {threshold:.4} {} ",
|
||||||
|
if level > threshold { "SPEECH" } else { " " }
|
||||||
|
);
|
||||||
|
use std::io::Write;
|
||||||
|
let _ = std::io::stdout().flush();
|
||||||
|
tokio::time::sleep(Duration::from_millis(50)).await;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn client(line: &str) -> Result<()> {
|
||||||
|
ipc::send_command(&socket_path(), line).await
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Errors go to the desktop, not just the journal — terra runs its own
|
||||||
|
/// notification daemon (HyprChrome/Widgets/Notifications), so this is the one
|
||||||
|
/// channel that is visible while you are talking rather than debugging.
|
||||||
|
/// Best-effort: a missing notify-send must never take the daemon down.
|
||||||
|
fn notify(body: &str) {
|
||||||
|
let _ = std::process::Command::new("notify-send")
|
||||||
|
.args(["-a", "luna", "-u", "critical", "luna voice", body])
|
||||||
|
.spawn();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Where the loop is. One utterance at a time by construction: the state
|
||||||
|
/// machine is a single task, so there is no way to start a second reply while
|
||||||
|
/// one is in flight.
|
||||||
|
async fn daemon() -> Result<()> {
|
||||||
|
let cfg = Arc::new(Config::from_env()?);
|
||||||
|
let capture = audio::Capture::new()?;
|
||||||
|
let playback = Arc::new(audio::Playback::new()?);
|
||||||
|
|
||||||
|
let (cmd_tx, mut cmd_rx) = mpsc::channel::<Command>(16);
|
||||||
|
// Kept so a turn can put back commands that arrived while she was
|
||||||
|
// speaking instead of destroying them.
|
||||||
|
let requeue = cmd_tx.clone();
|
||||||
|
let ipc = Arc::new(ipc::Ipc::bind(&cfg.socket_path, cmd_tx)?);
|
||||||
|
eprintln!("luna-voice: listening on {}", cfg.socket_path.display());
|
||||||
|
|
||||||
|
let http = reqwest::Client::builder()
|
||||||
|
// Her replies can involve tool calls; the default 30s would cut those
|
||||||
|
// off mid-thought.
|
||||||
|
.timeout(Duration::from_secs(300))
|
||||||
|
.build()?;
|
||||||
|
|
||||||
|
let mut history: Vec<brain::Message> = Vec::new();
|
||||||
|
if let Some(prompt) = &cfg.system_prompt {
|
||||||
|
history.push(brain::Message::system(prompt.clone()));
|
||||||
|
}
|
||||||
|
|
||||||
|
ipc.emit(Event::State { value: "idle" });
|
||||||
|
|
||||||
|
// Her first reply after the container starts costs ~16s against ~5s warm,
|
||||||
|
// and the first one is always the one you are sitting there waiting for.
|
||||||
|
// Spending one throwaway token at login moves that cost off the
|
||||||
|
// conversation. Failure is ignored on purpose: mars may simply be down,
|
||||||
|
// which must not stop the daemon from coming up.
|
||||||
|
tokio::spawn({
|
||||||
|
let (http, url, key, model) =
|
||||||
|
(http.clone(), cfg.hermes_url.clone(), cfg.hermes_key.clone(), cfg.model.clone());
|
||||||
|
async move {
|
||||||
|
let (tx, mut rx) = mpsc::channel(4);
|
||||||
|
tokio::spawn(async move { while rx.recv().await.is_some() {} });
|
||||||
|
let warm = [brain::Message::user("hi")];
|
||||||
|
match brain::stream_reply(&http, &url, &key, &model, &warm, tx).await {
|
||||||
|
Ok(_) => eprintln!("luna-voice: warmed up"),
|
||||||
|
Err(e) => eprintln!("luna-voice: warmup skipped: {e}"),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
// Hands-free state. The key toggles `mode`; everything inside it is
|
||||||
|
// driven by the microphone rather than by keys.
|
||||||
|
let mut mode = false;
|
||||||
|
// Running estimate of the room, updated only while not capturing, so a
|
||||||
|
// loud sentence never raises the bar for the next one.
|
||||||
|
let mut floor: f32 = 0.01;
|
||||||
|
let mut capturing = false;
|
||||||
|
let mut last_voice = tokio::time::Instant::now();
|
||||||
|
let mut ticker = tokio::time::interval(Duration::from_millis(50));
|
||||||
|
|
||||||
|
loop {
|
||||||
|
let cmd = tokio::select! {
|
||||||
|
cmd = cmd_rx.recv() => match cmd {
|
||||||
|
Some(cmd) => Some(cmd),
|
||||||
|
None => break,
|
||||||
|
},
|
||||||
|
// Only ticks while voice mode is on. A turn is awaited inline
|
||||||
|
// below, so this cannot fire while she is speaking — which is
|
||||||
|
// also what stops her from hearing herself.
|
||||||
|
_ = ticker.tick(), if mode => None,
|
||||||
|
};
|
||||||
|
|
||||||
|
let Some(cmd) = cmd else {
|
||||||
|
let level = capture.level();
|
||||||
|
// Every tick, not just while she speaks: a meter that only moves
|
||||||
|
// during playback cannot tell "the mic is dead" apart from "the
|
||||||
|
// threshold is too high", which is exactly the question you have
|
||||||
|
// when nothing happens.
|
||||||
|
ipc.emit(Event::Amplitude { rms: level });
|
||||||
|
let threshold = root_threshold(floor, cfg.vad_mult);
|
||||||
|
if !capturing {
|
||||||
|
// Only samples BELOW the trigger update the floor. Feeding
|
||||||
|
// every sample in lets a rising sound drag the floor up with
|
||||||
|
// it, so the threshold outruns the speech that caused it and
|
||||||
|
// nothing ever fires — measured: a tone peaking at 0.029 was
|
||||||
|
// chased by a threshold that reached 0.018 on the way up.
|
||||||
|
if level <= threshold {
|
||||||
|
floor = floor * 0.95 + level * 0.05;
|
||||||
|
}
|
||||||
|
if level > threshold {
|
||||||
|
capture.start();
|
||||||
|
capturing = true;
|
||||||
|
last_voice = tokio::time::Instant::now();
|
||||||
|
ipc.emit(Event::State { value: "listening" });
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
if level > threshold {
|
||||||
|
last_voice = tokio::time::Instant::now();
|
||||||
|
}
|
||||||
|
if last_voice.elapsed() > Duration::from_millis(cfg.silence_ms) {
|
||||||
|
capturing = false;
|
||||||
|
let samples = capture.stop();
|
||||||
|
ipc.emit(Event::State { value: "thinking" });
|
||||||
|
match stt::transcribe(&cfg.whisper_bin, &cfg.whisper_model, &samples).await {
|
||||||
|
Ok(heard) if !heard.is_empty() => {
|
||||||
|
run_turn(
|
||||||
|
&cfg, &http, &ipc, &capture, &playback, &mut cmd_rx,
|
||||||
|
&requeue, &mut history, heard,
|
||||||
|
)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
|
Ok(_) => {
|
||||||
|
// Silent on purpose: in hands-free mode a false
|
||||||
|
// trigger happens whenever a door closes, and a
|
||||||
|
// beep for each one would be maddening.
|
||||||
|
ipc.emit(Event::State { value: "idle" });
|
||||||
|
}
|
||||||
|
Err(e) => {
|
||||||
|
eprintln!("luna-voice: {e:#}");
|
||||||
|
ipc.emit(Event::Error { message: e.to_string() });
|
||||||
|
notify(&e.to_string());
|
||||||
|
ipc.emit(Event::State { value: "idle" });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
continue;
|
||||||
|
};
|
||||||
|
|
||||||
|
match cmd {
|
||||||
|
Command::Mode { on } => {
|
||||||
|
let want = on.unwrap_or(!mode);
|
||||||
|
if want != mode {
|
||||||
|
mode = want;
|
||||||
|
if mode {
|
||||||
|
// Seeded low: a floor seeded high takes seconds to
|
||||||
|
// decay, and those are the seconds you spend
|
||||||
|
// talking to something that is not listening yet.
|
||||||
|
floor = capture.level().max(0.003);
|
||||||
|
playback.beep(880.0, 70);
|
||||||
|
} else {
|
||||||
|
if capturing {
|
||||||
|
capture.stop();
|
||||||
|
capturing = false;
|
||||||
|
}
|
||||||
|
playback.clear();
|
||||||
|
playback.beep(440.0, 90);
|
||||||
|
}
|
||||||
|
ipc.emit(Event::Mode { active: mode });
|
||||||
|
ipc.emit(Event::State { value: "idle" });
|
||||||
|
eprintln!("luna-voice: voice mode {}", if mode { "on" } else { "off" });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Command::Ptt { down: true } => {
|
||||||
|
// Anything still playing is cut off: pressing the key while
|
||||||
|
// she talks means you want to talk instead.
|
||||||
|
playback.clear();
|
||||||
|
capture.start();
|
||||||
|
playback.beep(880.0, 70);
|
||||||
|
ipc.emit(Event::State { value: "listening" });
|
||||||
|
}
|
||||||
|
Command::Ptt { down: false } => {
|
||||||
|
let samples = capture.stop();
|
||||||
|
playback.beep(587.0, 70);
|
||||||
|
ipc.emit(Event::State { value: "thinking" });
|
||||||
|
|
||||||
|
let heard =
|
||||||
|
match stt::transcribe(&cfg.whisper_bin, &cfg.whisper_model, &samples).await {
|
||||||
|
Ok(h) => h,
|
||||||
|
Err(e) => {
|
||||||
|
eprintln!("luna-voice: {e:#}");
|
||||||
|
ipc.emit(Event::Error { message: e.to_string() });
|
||||||
|
notify(&e.to_string());
|
||||||
|
ipc.emit(Event::State { value: "idle" });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
};
|
||||||
|
|
||||||
|
if heard.is_empty() {
|
||||||
|
// Distinct low tone: "I heard nothing" and "I am
|
||||||
|
// thinking" must not feel the same, or a dead mic reads
|
||||||
|
// as a slow reply.
|
||||||
|
playback.beep(220.0, 120);
|
||||||
|
eprintln!("luna-voice: nothing heard ({} samples)", samples.len());
|
||||||
|
ipc.emit(Event::SpeechStop { reason: "nothing_heard" });
|
||||||
|
ipc.emit(Event::State { value: "idle" });
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
|
||||||
|
run_turn(&cfg, &http, &ipc, &capture, &playback, &mut cmd_rx, &requeue, &mut history, heard)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
|
Command::Say { text } => {
|
||||||
|
ipc.emit(Event::State { value: "thinking" });
|
||||||
|
run_turn(&cfg, &http, &ipc, &capture, &playback, &mut cmd_rx, &requeue, &mut history, text)
|
||||||
|
.await;
|
||||||
|
}
|
||||||
|
Command::Cancel => {
|
||||||
|
playback.clear();
|
||||||
|
ipc.emit(Event::SpeechStop { reason: "cancelled" });
|
||||||
|
ipc.emit(Event::State { value: "idle" });
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Runs one turn and reports failure through every channel that has a
|
||||||
|
/// listener: journal, socket, desktop notification.
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::{mark, root_threshold};
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn marker_prefixes_spoken_lines() {
|
||||||
|
assert_eq!(
|
||||||
|
mark(&Some("[voice]".into()), "what's the time"),
|
||||||
|
"[voice] what's the time"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn no_marker_leaves_the_line_alone() {
|
||||||
|
assert_eq!(mark(&None, "what's the time"), "what's the time");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn threshold_sits_between_terras_room_tone_and_terras_speech() {
|
||||||
|
// Measured on the AIR 192|4: ~0.001 idle, 0.004–0.005 speaking. The
|
||||||
|
// trigger has to land strictly between those two, which is the whole
|
||||||
|
// reason the lower clamp is 0.0025 and not something rounder.
|
||||||
|
let t = root_threshold(0.001, 3.5);
|
||||||
|
assert!(t > 0.001, "would trigger on room tone: {t}");
|
||||||
|
assert!(t < 0.004, "would never trigger on speech: {t}");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn threshold_never_climbs_out_of_reach_of_speech() {
|
||||||
|
// A floor seeded from a loud moment must not lock the mic out.
|
||||||
|
assert_eq!(root_threshold(0.5, 3.5), 0.05);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn threshold_tracks_the_room_in_between() {
|
||||||
|
assert!((root_threshold(0.01, 3.5) - 0.035).abs() < 1e-6);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
#[allow(clippy::too_many_arguments)]
|
||||||
|
async fn run_turn(
|
||||||
|
cfg: &Config,
|
||||||
|
http: &reqwest::Client,
|
||||||
|
ipc: &Arc<ipc::Ipc>,
|
||||||
|
capture: &audio::Capture,
|
||||||
|
playback: &Arc<audio::Playback>,
|
||||||
|
cmd_rx: &mut mpsc::Receiver<Command>,
|
||||||
|
requeue: &mpsc::Sender<Command>,
|
||||||
|
history: &mut Vec<brain::Message>,
|
||||||
|
heard: String,
|
||||||
|
) {
|
||||||
|
if let Err(e) = turn(cfg, http, ipc, capture, playback, cmd_rx, requeue, history, heard).await {
|
||||||
|
eprintln!("luna-voice: {e:#}");
|
||||||
|
ipc.emit(Event::Error { message: e.to_string() });
|
||||||
|
notify(&e.to_string());
|
||||||
|
}
|
||||||
|
ipc.emit(Event::State { value: "idle" });
|
||||||
|
}
|
||||||
|
|
||||||
|
#[allow(clippy::too_many_arguments)]
|
||||||
|
async fn turn(
|
||||||
|
cfg: &Config,
|
||||||
|
http: &reqwest::Client,
|
||||||
|
ipc: &Arc<ipc::Ipc>,
|
||||||
|
capture: &audio::Capture,
|
||||||
|
playback: &Arc<audio::Playback>,
|
||||||
|
cmd_rx: &mut mpsc::Receiver<Command>,
|
||||||
|
requeue: &mpsc::Sender<Command>,
|
||||||
|
history: &mut Vec<brain::Message>,
|
||||||
|
heard: String,
|
||||||
|
) -> Result<()> {
|
||||||
|
eprintln!("luna-voice: heard {heard:?}");
|
||||||
|
// The caption shows what you SAID; the marker is only for her.
|
||||||
|
ipc.emit(Event::Caption { speaker: "user", text: heard.clone() });
|
||||||
|
history.push(brain::Message::user(mark(&cfg.voice_marker, &heard)));
|
||||||
|
|
||||||
|
// Commands that arrived while she was speaking, replayed into the
|
||||||
|
// channel once the turn is over rather than dropped.
|
||||||
|
let mut deferred: Vec<Command> = Vec::new();
|
||||||
|
|
||||||
|
// Noise floor sampled before she starts, so barge-in adapts to the room
|
||||||
|
// rather than to a constant.
|
||||||
|
let floor = capture.level().max(0.005);
|
||||||
|
|
||||||
|
let (sentence_tx, sentence_rx) = mpsc::channel::<String>(16);
|
||||||
|
let (event_tx, mut event_rx) = mpsc::channel::<tts::SpeechEvent>(64);
|
||||||
|
|
||||||
|
let tts_cfg = tts::TtsConfig {
|
||||||
|
key: cfg.eleven_key.clone(),
|
||||||
|
voice_id: cfg.voice_id.clone(),
|
||||||
|
model: cfg.tts_model.clone(),
|
||||||
|
};
|
||||||
|
let keep_tags = tts_cfg.is_v3();
|
||||||
|
let speech = tokio::spawn({
|
||||||
|
async move { tts::speak(&tts_cfg, sentence_rx, event_tx).await }
|
||||||
|
});
|
||||||
|
|
||||||
|
// Audio and visemes flow to the speaker/shell on their own task so the
|
||||||
|
// reply stream is never blocked by playback.
|
||||||
|
let player = tokio::spawn({
|
||||||
|
let ipc = ipc.clone();
|
||||||
|
let playback = playback.clone();
|
||||||
|
async move {
|
||||||
|
let mut started = false;
|
||||||
|
while let Some(ev) = event_rx.recv().await {
|
||||||
|
match ev {
|
||||||
|
tts::SpeechEvent::Audio(pcm) => {
|
||||||
|
if !started {
|
||||||
|
started = true;
|
||||||
|
ipc.emit(Event::SpeechStart);
|
||||||
|
ipc.emit(Event::State { value: "speaking" });
|
||||||
|
}
|
||||||
|
playback.push(&pcm);
|
||||||
|
}
|
||||||
|
// A chunk whose alignment held nothing visible still
|
||||||
|
// arrives; forwarding it would make the shell reset the
|
||||||
|
// mouth mid-word.
|
||||||
|
tts::SpeechEvent::Visemes(seq) if !seq.is_empty() => {
|
||||||
|
ipc.emit(Event::Visemes { seq })
|
||||||
|
}
|
||||||
|
tts::SpeechEvent::Visemes(_) => {}
|
||||||
|
tts::SpeechEvent::Done => break,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
});
|
||||||
|
|
||||||
|
let (delta_tx, mut delta_rx) = mpsc::channel::<String>(64);
|
||||||
|
let reply = tokio::spawn({
|
||||||
|
let http = http.clone();
|
||||||
|
let (url, key, model) = (cfg.hermes_url.clone(), cfg.hermes_key.clone(), cfg.model.clone());
|
||||||
|
let history = history.clone();
|
||||||
|
async move { brain::stream_reply(&http, &url, &key, &model, &history, delta_tx).await }
|
||||||
|
});
|
||||||
|
|
||||||
|
let mut splitter = tts::SentenceSplitter::new(keep_tags);
|
||||||
|
// What ElevenLabs bills: every character sent, audio tags included.
|
||||||
|
let mut billed = 0usize;
|
||||||
|
let caption = |sentence: &str| {
|
||||||
|
let text = tts::strip_tags(sentence);
|
||||||
|
if !text.is_empty() {
|
||||||
|
ipc.emit(Event::Caption { speaker: "luna", text });
|
||||||
|
}
|
||||||
|
};
|
||||||
|
while let Some(delta) = delta_rx.recv().await {
|
||||||
|
for sentence in splitter.push(&delta) {
|
||||||
|
caption(&sentence);
|
||||||
|
billed += sentence.chars().count();
|
||||||
|
if sentence_tx.send(sentence).await.is_err() {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if let Some(tail) = splitter.finish() {
|
||||||
|
caption(&tail);
|
||||||
|
billed += tail.chars().count();
|
||||||
|
let _ = sentence_tx.send(tail).await;
|
||||||
|
}
|
||||||
|
drop(sentence_tx);
|
||||||
|
eprintln!("luna-voice: tts {billed} chars on {}", cfg.tts_model);
|
||||||
|
|
||||||
|
let full = reply.await??;
|
||||||
|
eprintln!("luna-voice: said {:?}", full.trim());
|
||||||
|
if !full.trim().is_empty() {
|
||||||
|
history.push(brain::Message::assistant(full));
|
||||||
|
}
|
||||||
|
|
||||||
|
// Wait for the speaker to drain, watching for barge-in. Polling the queue
|
||||||
|
// is enough: cpal gives no drain callback, and 50ms of latency on "she
|
||||||
|
// finished" is invisible.
|
||||||
|
let grace = tokio::time::Instant::now() + Duration::from_millis(cfg.barge_in_grace_ms);
|
||||||
|
loop {
|
||||||
|
if speech.is_finished() && playback.pending() == 0 {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
if tokio::time::Instant::now() > grace
|
||||||
|
&& capture.level() > floor * cfg.barge_in_mult
|
||||||
|
{
|
||||||
|
playback.clear();
|
||||||
|
ipc.emit(Event::SpeechStop { reason: "barge_in" });
|
||||||
|
// She is told she was cut off, so the next turn can react to it
|
||||||
|
// rather than carrying on obliviously.
|
||||||
|
history.push(brain::Message::system(
|
||||||
|
"[The user interrupted you mid-sentence.]",
|
||||||
|
));
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
// Commands arriving mid-playback. try_recv POPS, so anything not
|
||||||
|
// handled here is destroyed: an `if let Ok(Cancel)` silently ate
|
||||||
|
// every `say`, `toggle` and `mode off` sent while she was talking,
|
||||||
|
// which presents as a keypress that does nothing.
|
||||||
|
match cmd_rx.try_recv() {
|
||||||
|
Ok(Command::Cancel) => {
|
||||||
|
playback.clear();
|
||||||
|
ipc.emit(Event::SpeechStop { reason: "cancelled" });
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
// Leaving voice mode has to shut her up immediately; the mode
|
||||||
|
// change itself is replayed below.
|
||||||
|
Ok(cmd @ Command::Mode { .. }) => {
|
||||||
|
playback.clear();
|
||||||
|
ipc.emit(Event::SpeechStop { reason: "cancelled" });
|
||||||
|
deferred.push(cmd);
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
Ok(cmd) => deferred.push(cmd),
|
||||||
|
Err(_) => {}
|
||||||
|
}
|
||||||
|
ipc.emit(Event::Amplitude { rms: capture.level() });
|
||||||
|
tokio::time::sleep(Duration::from_millis(50)).await;
|
||||||
|
}
|
||||||
|
|
||||||
|
for cmd in deferred {
|
||||||
|
let _ = requeue.try_send(cmd);
|
||||||
|
}
|
||||||
|
|
||||||
|
// A TTS failure must NOT be silent. The captions come from the text
|
||||||
|
// stream, not from the synthesiser, so a dead websocket still paints her
|
||||||
|
// whole reply on screen — "she answered but said nothing" is then
|
||||||
|
// indistinguishable from a working reply with the volume down, which is
|
||||||
|
// exactly how it presented.
|
||||||
|
if speech.is_finished() {
|
||||||
|
if let Ok(Err(e)) = speech.await {
|
||||||
|
eprintln!("luna-voice: tts failed: {e:#}");
|
||||||
|
ipc.emit(Event::Error { message: format!("tts: {e}") });
|
||||||
|
notify(&format!("voice failed: {e}"));
|
||||||
|
}
|
||||||
|
} else {
|
||||||
|
speech.abort();
|
||||||
|
}
|
||||||
|
player.abort();
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
@@ -0,0 +1,181 @@
|
|||||||
|
use crate::audio;
|
||||||
|
use anyhow::{bail, Result};
|
||||||
|
use std::path::Path;
|
||||||
|
use tokio::process::Command;
|
||||||
|
|
||||||
|
/// Shells out to whisper-cli rather than linking whisper.cpp: nixpkgs already
|
||||||
|
/// ships whisper-cpp-vulkan built against this box's GPU stack, and a bundled
|
||||||
|
/// build would have to rediscover that at cargo time.
|
||||||
|
pub async fn transcribe(bin: &str, model: &Path, samples: &[f32]) -> Result<String> {
|
||||||
|
// Whisper hallucinates confident words into silence — "you", "Thank
|
||||||
|
// you.", "Bye." — which then reach luna as if spoken. Duration alone does
|
||||||
|
// not catch it (a held key over a quiet room is long AND empty), so gate
|
||||||
|
// on level too.
|
||||||
|
if samples.len() < (audio::STT_RATE as usize) / 3 || !has_speech(samples) {
|
||||||
|
return Ok(String::new());
|
||||||
|
}
|
||||||
|
|
||||||
|
let dir = std::env::var("XDG_RUNTIME_DIR").unwrap_or_else(|_| "/tmp".into());
|
||||||
|
let wav = Path::new(&dir).join(format!("luna-stt-{}.wav", std::process::id()));
|
||||||
|
audio::write_wav(&wav, &normalize(samples), audio::STT_RATE)?;
|
||||||
|
|
||||||
|
let out = Command::new(bin)
|
||||||
|
.args(["-m"])
|
||||||
|
.arg(model)
|
||||||
|
.args(["-f"])
|
||||||
|
.arg(&wav)
|
||||||
|
// -nt: transcript only, no [00:00:00] prefixes. -np: no progress
|
||||||
|
// banner. Together they make stdout exactly the text.
|
||||||
|
.args(["-nt", "-np"])
|
||||||
|
.output()
|
||||||
|
.await?;
|
||||||
|
let _ = std::fs::remove_file(&wav);
|
||||||
|
|
||||||
|
if !out.status.success() {
|
||||||
|
bail!(
|
||||||
|
"whisper-cli failed: {}",
|
||||||
|
String::from_utf8_lossy(&out.stderr).trim()
|
||||||
|
);
|
||||||
|
}
|
||||||
|
Ok(clean(&String::from_utf8_lossy(&out.stdout)))
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Scales a quiet recording up before whisper sees it.
|
||||||
|
///
|
||||||
|
/// Whisper is trained on normalised audio and degrades badly on very quiet
|
||||||
|
/// input, which is precisely what a low-gain interface produces — speech at a
|
||||||
|
/// 0.03 peak is intelligible to a person and mush to the model. The gain is
|
||||||
|
/// capped so a recording of near-silence is not amplified into hiss the model
|
||||||
|
/// will confidently transcribe as words.
|
||||||
|
fn normalize(samples: &[f32]) -> Vec<f32> {
|
||||||
|
const TARGET_PEAK: f32 = 0.6;
|
||||||
|
const MAX_GAIN: f32 = 30.0;
|
||||||
|
|
||||||
|
let peak = samples.iter().fold(0.0f32, |m, s| m.max(s.abs()));
|
||||||
|
if peak <= f32::EPSILON {
|
||||||
|
return samples.to_vec();
|
||||||
|
}
|
||||||
|
let gain = (TARGET_PEAK / peak).min(MAX_GAIN);
|
||||||
|
// Never attenuate: a loud recording is already fine, and turning it down
|
||||||
|
// would only lose resolution in the 16-bit conversion.
|
||||||
|
if gain <= 1.0 {
|
||||||
|
return samples.to_vec();
|
||||||
|
}
|
||||||
|
samples.iter().map(|s| (s * gain).clamp(-1.0, 1.0)).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// True if any 100ms window is loud enough to be a voice. Peak rather than
|
||||||
|
/// overall RMS: a short sentence inside a long silent recording averages down
|
||||||
|
/// to nothing, and that is the common case for a held key.
|
||||||
|
fn has_speech(samples: &[f32]) -> bool {
|
||||||
|
// Matches the VAD's lower clamp rather than a "typical" mic: a quiet
|
||||||
|
// interface puts normal speech at ~0.004, and gating at 0.015 threw away
|
||||||
|
// every word it captured.
|
||||||
|
const FLOOR: f32 = 0.0025;
|
||||||
|
let window = (audio::STT_RATE as usize) / 10;
|
||||||
|
samples.chunks(window).any(|w| {
|
||||||
|
let sum: f32 = w.iter().map(|s| s * s).sum();
|
||||||
|
(sum / w.len().max(1) as f32).sqrt() > FLOOR
|
||||||
|
})
|
||||||
|
}
|
||||||
|
|
||||||
|
/// whisper emits bracketed non-speech annotations ([BLANK_AUDIO], (wind
|
||||||
|
/// blowing)) that are not words and must not reach the agent.
|
||||||
|
fn clean(raw: &str) -> String {
|
||||||
|
let mut out = String::new();
|
||||||
|
let mut depth = 0i32;
|
||||||
|
for c in raw.chars() {
|
||||||
|
match c {
|
||||||
|
'[' | '(' => depth += 1,
|
||||||
|
']' | ')' => depth = (depth - 1).max(0),
|
||||||
|
c if depth == 0 => out.push(c),
|
||||||
|
_ => {}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out.split_whitespace().collect::<Vec<_>>().join(" ")
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::{clean, has_speech, normalize};
|
||||||
|
use crate::audio::STT_RATE;
|
||||||
|
|
||||||
|
fn peak(v: &[f32]) -> f32 {
|
||||||
|
v.iter().fold(0.0f32, |m, s| m.max(s.abs()))
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn normalize_lifts_a_quiet_recording() {
|
||||||
|
// 0.004 RMS speech peaks around 0.02, which is 30x under target —
|
||||||
|
// exactly at the gain cap, so it just reaches full scale.
|
||||||
|
let quiet: Vec<f32> = (0..1000).map(|i| 0.02 * ((i % 7) as f32 - 3.0) / 3.0).collect();
|
||||||
|
assert!((peak(&normalize(&quiet)) - 0.6).abs() < 0.01);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn normalize_leaves_a_loud_recording_alone() {
|
||||||
|
let loud: Vec<f32> = vec![0.9, -0.8, 0.7];
|
||||||
|
assert_eq!(normalize(&loud), loud);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn normalize_caps_the_gain_so_silence_stays_silent() {
|
||||||
|
let hiss: Vec<f32> = vec![0.0001, -0.0001];
|
||||||
|
// 0.6/0.0001 would be 6000x; the cap keeps it at 30x.
|
||||||
|
assert!(peak(&normalize(&hiss)) <= 0.0001 * 30.0 + 1e-9);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn normalize_handles_pure_silence() {
|
||||||
|
assert_eq!(normalize(&[0.0, 0.0]), vec![0.0, 0.0]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn silence_is_not_speech() {
|
||||||
|
assert!(!has_speech(&vec![0.0f32; STT_RATE as usize]));
|
||||||
|
// Terra's measured room tone. Speech on the same mic reaches
|
||||||
|
// 0.004–0.005, so the gate sits between the two.
|
||||||
|
assert!(!has_speech(&vec![0.001f32; STT_RATE as usize]));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn quiet_speech_still_counts() {
|
||||||
|
// 0.004 is what normal speech measures on terra's interface; gating
|
||||||
|
// it out discarded every word the VAD had just captured.
|
||||||
|
let mut s = vec![0.001f32; STT_RATE as usize];
|
||||||
|
for v in s.iter_mut().take(STT_RATE as usize / 5) {
|
||||||
|
*v = 0.004;
|
||||||
|
}
|
||||||
|
assert!(has_speech(&s));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_short_utterance_in_a_long_silence_counts() {
|
||||||
|
let mut s = vec![0.0f32; STT_RATE as usize];
|
||||||
|
for v in s.iter_mut().take(STT_RATE as usize / 5) {
|
||||||
|
*v = 0.2;
|
||||||
|
}
|
||||||
|
assert!(has_speech(&s));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn strips_bracketed_annotations() {
|
||||||
|
assert_eq!(clean("[BLANK_AUDIO]"), "");
|
||||||
|
assert_eq!(clean(" hello (wind blowing) there \n"), "hello there");
|
||||||
|
assert_eq!(clean("[ music ] real words"), "real words");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn collapses_whitespace() {
|
||||||
|
assert_eq!(clean(" a b \n c "), "a b c");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn unbalanced_brackets_do_not_eat_the_rest() {
|
||||||
|
// The stray closer is dropped (depth floors at 0) but the words
|
||||||
|
// around it survive — the failure mode that matters is a lone
|
||||||
|
// bracket swallowing the whole transcript.
|
||||||
|
assert_eq!(clean("ok] fine"), "ok fine");
|
||||||
|
assert_eq!(clean("[ oops real words"), "");
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,529 @@
|
|||||||
|
use crate::audio;
|
||||||
|
use crate::visemes::{self, Viseme};
|
||||||
|
use anyhow::{anyhow, Result};
|
||||||
|
use base64::Engine;
|
||||||
|
use futures_util::{SinkExt, StreamExt};
|
||||||
|
use std::time::Duration;
|
||||||
|
use tokio::sync::mpsc;
|
||||||
|
use tokio_tungstenite::tungstenite::client::IntoClientRequest;
|
||||||
|
use tokio_tungstenite::tungstenite::Message as Ws;
|
||||||
|
|
||||||
|
pub enum SpeechEvent {
|
||||||
|
/// Decoded PCM at audio::TTS_RATE.
|
||||||
|
Audio(Vec<f32>),
|
||||||
|
/// Mouth shapes, timed from the start of this reply's audio.
|
||||||
|
Visemes(Vec<Viseme>),
|
||||||
|
Done,
|
||||||
|
}
|
||||||
|
|
||||||
|
pub struct TtsConfig {
|
||||||
|
pub key: String,
|
||||||
|
pub voice_id: String,
|
||||||
|
pub model: String,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl TtsConfig {
|
||||||
|
/// v3 is not served on the TTS websocket at all; it streams only over
|
||||||
|
/// Text to Dialogue, a different protocol.
|
||||||
|
pub fn is_v3(&self) -> bool {
|
||||||
|
self.model.starts_with("eleven_v3")
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The two sockets send the same audio frames under different field names.
|
||||||
|
struct Wire {
|
||||||
|
alignment: &'static [&'static str],
|
||||||
|
final_key: &'static str,
|
||||||
|
}
|
||||||
|
|
||||||
|
const TTS_WIRE: Wire = Wire { alignment: &["normalizedAlignment", "alignment"], final_key: "isFinal" };
|
||||||
|
const DIALOGUE_WIRE: Wire = Wire { alignment: &["alignment"], final_key: "is_final" };
|
||||||
|
|
||||||
|
/// Dialogue drops the socket after 20s without a client message, and a tool
|
||||||
|
/// call on mars can stall a reply longer than that.
|
||||||
|
const KEEP_ALIVE: Duration = Duration::from_secs(10);
|
||||||
|
|
||||||
|
type Socket = tokio_tungstenite::WebSocketStream<
|
||||||
|
tokio_tungstenite::MaybeTlsStream<tokio::net::TcpStream>,
|
||||||
|
>;
|
||||||
|
type Sink = futures_util::stream::SplitSink<Socket, Ws>;
|
||||||
|
type Source = futures_util::stream::SplitStream<Socket>;
|
||||||
|
|
||||||
|
/// Speaks sentences as they arrive on `sentences`, over one websocket per
|
||||||
|
/// reply.
|
||||||
|
///
|
||||||
|
/// pcm_24000 rather than the default mp3: there is no decoder in this process
|
||||||
|
/// and no reason to add one. sync_alignment makes the character timings ride
|
||||||
|
/// along with the audio frames instead of needing a second request.
|
||||||
|
pub async fn speak(
|
||||||
|
cfg: &TtsConfig,
|
||||||
|
sentences: mpsc::Receiver<String>,
|
||||||
|
events: mpsc::Sender<SpeechEvent>,
|
||||||
|
) -> Result<()> {
|
||||||
|
if cfg.is_v3() {
|
||||||
|
speak_dialogue(cfg, sentences, events).await
|
||||||
|
} else {
|
||||||
|
speak_tts(cfg, sentences, events).await
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn connect(url: &str, key: &str) -> Result<(Sink, Source)> {
|
||||||
|
let mut req = url.into_client_request()?;
|
||||||
|
// Header, not the ?authorization= query param: a key in a URL ends up in
|
||||||
|
// every proxy and error log that touches it.
|
||||||
|
req.headers_mut().insert(
|
||||||
|
"xi-api-key",
|
||||||
|
tokio_tungstenite::tungstenite::http::HeaderValue::from_str(key)?,
|
||||||
|
);
|
||||||
|
let (stream, _) = tokio_tungstenite::connect_async(req).await?;
|
||||||
|
Ok(stream.split())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn send_json(sink: &mut Sink, v: serde_json::Value) -> Result<()> {
|
||||||
|
sink.send(Ws::Text(v.to_string())).await?;
|
||||||
|
Ok(())
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn speak_tts(
|
||||||
|
cfg: &TtsConfig,
|
||||||
|
mut sentences: mpsc::Receiver<String>,
|
||||||
|
events: mpsc::Sender<SpeechEvent>,
|
||||||
|
) -> Result<()> {
|
||||||
|
let url = format!(
|
||||||
|
"wss://api.elevenlabs.io/v1/text-to-speech/{}/stream-input\
|
||||||
|
?model_id={}&output_format=pcm_{}&sync_alignment=true&auto_mode=true",
|
||||||
|
cfg.voice_id,
|
||||||
|
cfg.model,
|
||||||
|
audio::TTS_RATE
|
||||||
|
);
|
||||||
|
let (mut sink, source) = connect(&url, &cfg.key).await?;
|
||||||
|
|
||||||
|
// The protocol requires an opening frame of a single space before any
|
||||||
|
// real text; it carries the voice settings for the whole connection.
|
||||||
|
send_json(
|
||||||
|
&mut sink,
|
||||||
|
serde_json::json!({
|
||||||
|
"text": " ",
|
||||||
|
"voice_settings": { "stability": 0.4, "similarity_boost": 0.8, "speed": 1.0 },
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
.await?;
|
||||||
|
|
||||||
|
let reader = tokio::spawn(read_audio(source, events, &TTS_WIRE));
|
||||||
|
|
||||||
|
let sent: Result<()> = async {
|
||||||
|
while let Some(text) = sentences.recv().await {
|
||||||
|
// Every text frame must end with a space or the model holds the
|
||||||
|
// last word back waiting for more input.
|
||||||
|
let text = format!("{} ", text.trim());
|
||||||
|
send_json(&mut sink, serde_json::json!({ "text": text })).await?;
|
||||||
|
}
|
||||||
|
// Empty string = end of input. Without it the socket sits until the
|
||||||
|
// inactivity timeout and the last sentence never gets flushed.
|
||||||
|
send_json(&mut sink, serde_json::json!({ "text": "" })).await
|
||||||
|
}
|
||||||
|
.await;
|
||||||
|
|
||||||
|
// The server's own error beats the "connection reset" a send hit after it.
|
||||||
|
reader.await??;
|
||||||
|
sent
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn speak_dialogue(
|
||||||
|
cfg: &TtsConfig,
|
||||||
|
mut sentences: mpsc::Receiver<String>,
|
||||||
|
events: mpsc::Sender<SpeechEvent>,
|
||||||
|
) -> Result<()> {
|
||||||
|
let url = format!(
|
||||||
|
"wss://api.elevenlabs.io/v1/text-to-dialogue/stream-input\
|
||||||
|
?model_id={}&output_format=pcm_{}&sync_alignment=true",
|
||||||
|
cfg.model,
|
||||||
|
audio::TTS_RATE
|
||||||
|
);
|
||||||
|
let (mut sink, source) = connect(&url, &cfg.key).await?;
|
||||||
|
|
||||||
|
// No voice_settings: v3 reads only stability, and only as one of three
|
||||||
|
// presets; the default is its "natural" one.
|
||||||
|
send_json(&mut sink, serde_json::json!({ "voices": [cfg.voice_id] })).await?;
|
||||||
|
|
||||||
|
let reader = tokio::spawn(read_audio(source, events, &DIALOGUE_WIRE));
|
||||||
|
|
||||||
|
let sent: Result<()> = async {
|
||||||
|
loop {
|
||||||
|
match tokio::time::timeout(KEEP_ALIVE, sentences.recv()).await {
|
||||||
|
Ok(Some(text)) => {
|
||||||
|
// Inputs are concatenated verbatim, so the space matters
|
||||||
|
// here too. flush: the server otherwise holds text until
|
||||||
|
// ~40 chars and 8 words, so a short first sentence would
|
||||||
|
// wait for the second.
|
||||||
|
let text = format!("{} ", text.trim());
|
||||||
|
send_json(
|
||||||
|
&mut sink,
|
||||||
|
serde_json::json!({
|
||||||
|
"inputs": [{ "text": text, "voice_id": cfg.voice_id }],
|
||||||
|
"flush": true,
|
||||||
|
}),
|
||||||
|
)
|
||||||
|
.await?
|
||||||
|
}
|
||||||
|
Ok(None) => break,
|
||||||
|
Err(_) => send_json(&mut sink, serde_json::json!({ "keep_alive": true })).await?,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
send_json(&mut sink, serde_json::json!({ "close_socket": true })).await
|
||||||
|
}
|
||||||
|
.await;
|
||||||
|
|
||||||
|
reader.await??;
|
||||||
|
sent
|
||||||
|
}
|
||||||
|
|
||||||
|
async fn read_audio(
|
||||||
|
mut source: Source,
|
||||||
|
events: mpsc::Sender<SpeechEvent>,
|
||||||
|
wire: &'static Wire,
|
||||||
|
) -> Result<()> {
|
||||||
|
// Alignment times may be per-chunk or cumulative depending on the
|
||||||
|
// model; `emitted_ms` lets us normalise either into one timeline.
|
||||||
|
let mut emitted_ms: u32 = 0;
|
||||||
|
// v3 audio tags come back as characters, and one can straddle chunks.
|
||||||
|
let mut in_tag = false;
|
||||||
|
let mut result = Ok(());
|
||||||
|
while let Some(Ok(msg)) = source.next().await {
|
||||||
|
let Ws::Text(txt) = msg else { continue };
|
||||||
|
let Ok(v) = serde_json::from_str::<serde_json::Value>(&txt) else { continue };
|
||||||
|
|
||||||
|
if let Some(err) = v.get("error").and_then(|e| e.as_str()) {
|
||||||
|
let detail = v.get("message").and_then(|m| m.as_str()).unwrap_or("");
|
||||||
|
result = Err(anyhow!("elevenlabs: {err} {detail}"));
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
|
||||||
|
if let Some(b64) = v.get("audio").and_then(|a| a.as_str()) {
|
||||||
|
if let Ok(bytes) = base64::engine::general_purpose::STANDARD.decode(b64) {
|
||||||
|
let pcm = audio::pcm16_to_f32(&bytes);
|
||||||
|
let chunk_ms = (pcm.len() as f64 / audio::TTS_RATE as f64 * 1000.0) as u32;
|
||||||
|
let al = wire.alignment.iter().find_map(|k| v.get(*k).filter(|a| !a.is_null()));
|
||||||
|
if let Some(t) = al.and_then(|al| alignment_to_timeline(al, emitted_ms, &mut in_tag)) {
|
||||||
|
let _ = events.send(SpeechEvent::Visemes(t)).await;
|
||||||
|
}
|
||||||
|
emitted_ms += chunk_ms;
|
||||||
|
if events.send(SpeechEvent::Audio(pcm)).await.is_err() {
|
||||||
|
return Ok(()); // receiver gone: barge-in cancelled this reply
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if v.get(wire.final_key).and_then(|f| f.as_bool()).unwrap_or(false) {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let _ = events.send(SpeechEvent::Done).await;
|
||||||
|
result
|
||||||
|
}
|
||||||
|
|
||||||
|
fn alignment_to_timeline(
|
||||||
|
al: &serde_json::Value,
|
||||||
|
emitted_ms: u32,
|
||||||
|
in_tag: &mut bool,
|
||||||
|
) -> Option<Vec<Viseme>> {
|
||||||
|
let field = |camel: &str, snake: &str| al.get(camel).or_else(|| al.get(snake));
|
||||||
|
let chars: Vec<String> = al
|
||||||
|
.get("chars")?
|
||||||
|
.as_array()?
|
||||||
|
.iter()
|
||||||
|
.map(|c| c.as_str().unwrap_or(""))
|
||||||
|
.map(|c| {
|
||||||
|
// A tag is a laugh or a sigh, not letters: the mouth rests.
|
||||||
|
let tagged = *in_tag || c == "[";
|
||||||
|
*in_tag = tagged && c != "]";
|
||||||
|
if tagged { " ".to_string() } else { c.to_string() }
|
||||||
|
})
|
||||||
|
.collect();
|
||||||
|
let starts: Vec<i64> = num_array(field("charStartTimesMs", "char_start_times_ms")?)?;
|
||||||
|
let durs: Vec<i64> = num_array(field("charDurationsMs", "char_durations_ms")?)?;
|
||||||
|
let starts = absolutize(&starts, emitted_ms);
|
||||||
|
Some(visemes::timeline(&chars, &starts, &durs))
|
||||||
|
}
|
||||||
|
|
||||||
|
fn num_array(v: &serde_json::Value) -> Option<Vec<i64>> {
|
||||||
|
Some(v.as_array()?.iter().map(|n| n.as_i64().unwrap_or(0)).collect())
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Normalises chunk-relative and stream-absolute alignments into one
|
||||||
|
/// timeline. If the first start time is behind audio we have already emitted,
|
||||||
|
/// the timings must be relative to this chunk, so they get shifted.
|
||||||
|
fn absolutize(starts: &[i64], emitted_ms: u32) -> Vec<i64> {
|
||||||
|
let offset = match starts.first() {
|
||||||
|
Some(&first) if first < emitted_ms as i64 => emitted_ms as i64,
|
||||||
|
_ => 0,
|
||||||
|
};
|
||||||
|
starts.iter().map(|s| s + offset).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Splits streamed text into speakable sentences, and strips what should
|
||||||
|
/// never be read aloud. Hermes' own voice mode does the same thing for the
|
||||||
|
/// same reason: without it she pronounces asterisks.
|
||||||
|
pub struct SentenceSplitter {
|
||||||
|
buf: String,
|
||||||
|
in_think: bool,
|
||||||
|
/// Whether v3 audio tags survive to the synthesiser or are dropped.
|
||||||
|
keep_tags: bool,
|
||||||
|
}
|
||||||
|
|
||||||
|
impl SentenceSplitter {
|
||||||
|
pub fn new(keep_tags: bool) -> Self {
|
||||||
|
SentenceSplitter { buf: String::new(), in_think: false, keep_tags }
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Feeds a delta, returning any sentences that completed.
|
||||||
|
pub fn push(&mut self, delta: &str) -> Vec<String> {
|
||||||
|
self.buf.push_str(delta);
|
||||||
|
let mut out = Vec::new();
|
||||||
|
|
||||||
|
// <think> blocks are reasoning, not speech, and can be long.
|
||||||
|
loop {
|
||||||
|
if self.in_think {
|
||||||
|
match self.buf.find("</think>") {
|
||||||
|
Some(i) => {
|
||||||
|
self.buf.drain(..i + "</think>".len());
|
||||||
|
self.in_think = false;
|
||||||
|
}
|
||||||
|
None => {
|
||||||
|
self.buf.clear();
|
||||||
|
return out;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
match self.buf.find("<think>") {
|
||||||
|
Some(i) => {
|
||||||
|
let head: String = self.buf.drain(..i).collect();
|
||||||
|
self.buf.drain(.."<think>".len());
|
||||||
|
self.in_think = true;
|
||||||
|
out.extend(drain_sentences(&mut { head.clone() }, true, self.keep_tags));
|
||||||
|
}
|
||||||
|
None => break,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
out.extend(drain_sentences(&mut self.buf, false, self.keep_tags));
|
||||||
|
out.into_iter().filter(|s| !s.is_empty()).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Whatever is left when the reply ends.
|
||||||
|
pub fn finish(&mut self) -> Option<String> {
|
||||||
|
let rest = clean_for_speech(&std::mem::take(&mut self.buf), self.keep_tags);
|
||||||
|
(!rest.is_empty()).then_some(rest)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
fn drain_sentences(buf: &mut String, force: bool, keep_tags: bool) -> Vec<String> {
|
||||||
|
let mut out = Vec::new();
|
||||||
|
// A sentence is only worth sending once it is long enough to synthesise
|
||||||
|
// well; ElevenLabs' own guidance is ~20 characters minimum.
|
||||||
|
while let Some(end) = buf.find(['.', '!', '?', '\n']) {
|
||||||
|
if end < 20 && !force {
|
||||||
|
// Abbreviation or a very short clause — keep accumulating.
|
||||||
|
if buf.len() < 80 {
|
||||||
|
break;
|
||||||
|
}
|
||||||
|
}
|
||||||
|
let s: String = buf.drain(..end + 1).collect();
|
||||||
|
let s = clean_for_speech(&s, keep_tags);
|
||||||
|
if !s.is_empty() {
|
||||||
|
out.push(s);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
if force && !buf.is_empty() {
|
||||||
|
let s = clean_for_speech(&std::mem::take(buf), keep_tags);
|
||||||
|
if !s.is_empty() {
|
||||||
|
out.push(s);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Byte index just past a v3 audio tag starting at `i` ("[laughs]",
|
||||||
|
/// "[clears throat]"), if there is one. Letters and spaces only, so
|
||||||
|
/// "[1]" or a markdown link stays ordinary noise.
|
||||||
|
fn tag_end(s: &str, i: usize) -> Option<usize> {
|
||||||
|
let rest = s[i..].strip_prefix('[')?;
|
||||||
|
let close = rest.find(']')?;
|
||||||
|
let body = &rest[..close];
|
||||||
|
let end = i + 1 + close + 1;
|
||||||
|
let ok = !body.trim().is_empty()
|
||||||
|
&& body.len() <= 40
|
||||||
|
&& body.chars().all(|c| c.is_alphabetic() || c == ' ' || c == '-')
|
||||||
|
&& !s[end..].starts_with('(');
|
||||||
|
ok.then_some(end)
|
||||||
|
}
|
||||||
|
|
||||||
|
/// The text as a person reads it: captions show "ha!", not "[laughs]".
|
||||||
|
pub fn strip_tags(s: &str) -> String {
|
||||||
|
let mut out = String::with_capacity(s.len());
|
||||||
|
let mut skip_to = 0;
|
||||||
|
for (i, c) in s.char_indices() {
|
||||||
|
if i < skip_to {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
match tag_end(s, i) {
|
||||||
|
Some(end) => {
|
||||||
|
skip_to = end;
|
||||||
|
out.push(' ');
|
||||||
|
}
|
||||||
|
None => out.push(c),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out.split_whitespace().collect::<Vec<_>>().join(" ")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Markdown, emoji and kaomoji are for eyes.
|
||||||
|
///
|
||||||
|
/// A whitelist, not a blacklist: she writes things like "(¬‿¬)", whose
|
||||||
|
/// characters live in ordinary Latin-1 and punctuation blocks rather than in
|
||||||
|
/// any emoji range, so enumerating what to drop never converges. Letters
|
||||||
|
/// stays unicode-aware on purpose — German is in scope, "ö" is a letter.
|
||||||
|
///
|
||||||
|
/// Audio tags pass through untouched when `keep_tags`; otherwise they are
|
||||||
|
/// dropped whole, since a model without them would read "laughs" aloud.
|
||||||
|
pub fn clean_for_speech(s: &str, keep_tags: bool) -> String {
|
||||||
|
let mut out = String::with_capacity(s.len());
|
||||||
|
let mut skip_to = 0;
|
||||||
|
for (i, c) in s.char_indices() {
|
||||||
|
if i < skip_to {
|
||||||
|
continue;
|
||||||
|
}
|
||||||
|
if let Some(end) = tag_end(s, i) {
|
||||||
|
skip_to = end;
|
||||||
|
out.push(' ');
|
||||||
|
if keep_tags {
|
||||||
|
out.push_str(&s[i..end]);
|
||||||
|
out.push(' ');
|
||||||
|
}
|
||||||
|
} else if c.is_alphanumeric() || c.is_whitespace() || is_speakable_punctuation(c) {
|
||||||
|
out.push(c);
|
||||||
|
} else if matches!(c, '—' | '–') {
|
||||||
|
// " — " must not leave " , ": the comma binds to the word before it.
|
||||||
|
while out.chars().last().is_some_and(char::is_whitespace) {
|
||||||
|
out.pop();
|
||||||
|
}
|
||||||
|
// An em dash is a pause, not a space. Dropping it runs the two
|
||||||
|
// clauses together and changes what the sentence means; a comma
|
||||||
|
// is the closest thing the synthesiser understands. ASCII '-'
|
||||||
|
// deliberately does NOT land here — it is usually intra-word
|
||||||
|
// ("push-to-talk"), where a comma would be worse than a space.
|
||||||
|
out.push(',');
|
||||||
|
} else {
|
||||||
|
// Not dropped silently: a symbol between two words is usually a
|
||||||
|
// dash or a bullet, and gluing the words together mispronounces
|
||||||
|
// both.
|
||||||
|
out.push(' ');
|
||||||
|
}
|
||||||
|
}
|
||||||
|
out.split_whitespace().collect::<Vec<_>>().join(" ")
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Punctuation a TTS model actually uses for prosody. Everything else is
|
||||||
|
/// noise once it reaches audio.
|
||||||
|
fn is_speakable_punctuation(c: char) -> bool {
|
||||||
|
matches!(c, '.' | ',' | '!' | '?' | ';' | ':' | '\'' | '"')
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn absolutize_shifts_chunk_relative_times() {
|
||||||
|
assert_eq!(absolutize(&[0, 50], 1000), vec![1000, 1050]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn absolutize_leaves_already_absolute_times() {
|
||||||
|
assert_eq!(absolutize(&[1000, 1050], 1000), vec![1000, 1050]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn splitter_emits_complete_sentences_only() {
|
||||||
|
let mut s = SentenceSplitter::new(false);
|
||||||
|
assert!(s.push("This is a long enough first sentence").is_empty());
|
||||||
|
let got = s.push(". And more");
|
||||||
|
assert_eq!(got, vec!["This is a long enough first sentence."]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn splitter_drops_think_blocks() {
|
||||||
|
let mut s = SentenceSplitter::new(false);
|
||||||
|
s.push("<think>internal monologue that is long</think>");
|
||||||
|
let got = s.push("Actually speakable sentence here. ");
|
||||||
|
assert_eq!(got, vec!["Actually speakable sentence here."]);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn splitter_returns_tail_on_finish() {
|
||||||
|
let mut s = SentenceSplitter::new(false);
|
||||||
|
s.push("no terminator here");
|
||||||
|
assert_eq!(s.finish().as_deref(), Some("no terminator here"));
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn markdown_and_emoji_never_reach_the_voice() {
|
||||||
|
assert_eq!(clean_for_speech("**bold** `code` 🎉 ok", false), "bold code ok");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn kaomoji_and_dashes_are_stripped_without_gluing_words() {
|
||||||
|
// Her actual first reply over the API.
|
||||||
|
assert_eq!(
|
||||||
|
clean_for_speech("Loud and clear, boss — hallo! (¬‿¬)", false),
|
||||||
|
"Loud and clear, boss, hallo!"
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn em_dash_becomes_a_pause_but_hyphens_do_not() {
|
||||||
|
assert_eq!(clean_for_speech("I can't confirm — there's nothing", false), "I can't confirm, there's nothing");
|
||||||
|
assert_eq!(clean_for_speech("push-to-talk", false), "push to talk");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn keeps_non_ascii_letters() {
|
||||||
|
assert_eq!(clean_for_speech("schön, größer", false), "schön, größer");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn audio_tags_survive_for_v3_only() {
|
||||||
|
let s = "[laughs]Fine, boss. [clears throat] Na gut.";
|
||||||
|
assert_eq!(clean_for_speech(s, true), "[laughs] Fine, boss. [clears throat] Na gut.");
|
||||||
|
assert_eq!(clean_for_speech(s, false), "Fine, boss. Na gut.");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn brackets_that_are_not_tags_stay_noise() {
|
||||||
|
assert_eq!(clean_for_speech("see [1] and [the docs](x)", true), "see 1 and the docs x");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn captions_hide_tags() {
|
||||||
|
assert_eq!(strip_tags("[sighs] Okay, boss."), "Okay, boss.");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn tag_characters_rest_the_mouth_across_chunks() {
|
||||||
|
let mut in_tag = false;
|
||||||
|
let chunk = |chars: &[&str], in_tag: &mut bool| {
|
||||||
|
let n = chars.len() as i64;
|
||||||
|
let al = serde_json::json!({
|
||||||
|
"chars": chars,
|
||||||
|
"char_start_times_ms": (0..n).map(|i| i * 100).collect::<Vec<_>>(),
|
||||||
|
"char_durations_ms": vec![100; chars.len()],
|
||||||
|
});
|
||||||
|
alignment_to_timeline(&al, 0, in_tag).unwrap()
|
||||||
|
};
|
||||||
|
let t = chunk(&["[", "l", "a"], &mut in_tag);
|
||||||
|
assert!(t.iter().all(|v| v.shape == visemes::SILENCE));
|
||||||
|
assert!(in_tag);
|
||||||
|
let t = chunk(&["u", "]", "a"], &mut in_tag);
|
||||||
|
assert!(!in_tag);
|
||||||
|
assert_eq!(t.last().unwrap().shape, "aa");
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,131 @@
|
|||||||
|
use serde::Serialize;
|
||||||
|
|
||||||
|
/// VRM 1.0's standard mouth expressions, which is what the avatar rig exposes
|
||||||
|
/// (`aa`/`ih`/`ou`/`ee`/`oh`). A rig with the full ARKit 52 can map these onto
|
||||||
|
/// jawOpen/mouthFunnel/etc — the shell side owns that, not this.
|
||||||
|
///
|
||||||
|
/// These names are a CONTRACT with the balsam-generated morph targets: the
|
||||||
|
/// glTF target names become the QML ids, so renaming a shape key in Blender
|
||||||
|
/// breaks the binding silently, with no eval error.
|
||||||
|
pub const SILENCE: &str = "sil";
|
||||||
|
|
||||||
|
#[derive(Debug, Clone, PartialEq, Serialize)]
|
||||||
|
pub struct Viseme {
|
||||||
|
pub shape: &'static str,
|
||||||
|
/// Milliseconds from speech_start, NOT wall clock — the shell starts its
|
||||||
|
/// own timer when the first PCM frame hits the speaker, so the two clocks
|
||||||
|
/// never have to agree.
|
||||||
|
pub at: u32,
|
||||||
|
pub ms: u32,
|
||||||
|
}
|
||||||
|
|
||||||
|
/// One character to a mouth shape. English orthography is a lie about
|
||||||
|
/// phonetics, but at 40ms of animation per character nobody can tell: what
|
||||||
|
/// reads as "talking" is the OPEN/CLOSE rhythm, not vowel accuracy.
|
||||||
|
fn shape_for(c: char) -> &'static str {
|
||||||
|
match c.to_ascii_lowercase() {
|
||||||
|
'a' => "aa",
|
||||||
|
'e' => "ee",
|
||||||
|
'i' | 'y' => "ih",
|
||||||
|
'o' => "oh",
|
||||||
|
'u' | 'w' => "ou",
|
||||||
|
// Bilabials close the mouth; giving them silence is what produces the
|
||||||
|
// visible "p/b/m" beat.
|
||||||
|
'p' | 'b' | 'm' => SILENCE,
|
||||||
|
// Everything else is a partially open consonant. "ih" is the least
|
||||||
|
// committal shape to land on.
|
||||||
|
c if c.is_alphabetic() => "ih",
|
||||||
|
_ => SILENCE,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
/// Builds a playable timeline from ElevenLabs' character alignment.
|
||||||
|
///
|
||||||
|
/// The three arrays are parallel; a short or missing one means the frame was
|
||||||
|
/// truncated, so we take the shortest common prefix rather than panicking on
|
||||||
|
/// an index.
|
||||||
|
pub fn timeline(chars: &[String], starts: &[i64], durations: &[i64]) -> Vec<Viseme> {
|
||||||
|
let n = chars.len().min(starts.len()).min(durations.len());
|
||||||
|
let mut out: Vec<Viseme> = Vec::with_capacity(n);
|
||||||
|
|
||||||
|
for i in 0..n {
|
||||||
|
let c = match chars[i].chars().next() {
|
||||||
|
Some(c) => c,
|
||||||
|
None => continue,
|
||||||
|
};
|
||||||
|
let shape = shape_for(c);
|
||||||
|
let at = starts[i].max(0) as u32;
|
||||||
|
let ms = durations[i].max(0) as u32;
|
||||||
|
|
||||||
|
// Merge runs of the same shape: "ll" is one mouth position held, not
|
||||||
|
// two identical keyframes the shell would re-trigger.
|
||||||
|
match out.last_mut() {
|
||||||
|
Some(last) if last.shape == shape && last.at + last.ms >= at => {
|
||||||
|
last.ms = (at + ms).saturating_sub(last.at);
|
||||||
|
}
|
||||||
|
_ => out.push(Viseme { shape, at, ms }),
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// Sub-frame keys cannot be seen and only cost socket bytes.
|
||||||
|
out.retain(|v| v.ms >= 20);
|
||||||
|
out
|
||||||
|
}
|
||||||
|
|
||||||
|
#[cfg(test)]
|
||||||
|
mod tests {
|
||||||
|
use super::*;
|
||||||
|
|
||||||
|
fn s(v: &[&str]) -> Vec<String> {
|
||||||
|
v.iter().map(|s| s.to_string()).collect()
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn maps_vowels_to_distinct_shapes() {
|
||||||
|
let t = timeline(
|
||||||
|
&s(&["a", "e", "o"]),
|
||||||
|
&[0, 100, 200],
|
||||||
|
&[100, 100, 100],
|
||||||
|
);
|
||||||
|
assert_eq!(
|
||||||
|
t.iter().map(|v| v.shape).collect::<Vec<_>>(),
|
||||||
|
vec!["aa", "ee", "oh"]
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn merges_repeated_shapes_into_one_held_key() {
|
||||||
|
let t = timeline(&s(&["l", "l", "l"]), &[0, 50, 100], &[50, 50, 50]);
|
||||||
|
assert_eq!(t.len(), 1);
|
||||||
|
assert_eq!(t[0].shape, "ih");
|
||||||
|
assert_eq!(t[0].at, 0);
|
||||||
|
assert_eq!(t[0].ms, 150);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn bilabials_close_the_mouth() {
|
||||||
|
let t = timeline(&s(&["a", "m", "a"]), &[0, 100, 200], &[100, 100, 100]);
|
||||||
|
assert_eq!(t[1].shape, SILENCE);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn drops_keys_too_short_to_see() {
|
||||||
|
let t = timeline(&s(&["a", "e"]), &[0, 5], &[5, 100]);
|
||||||
|
assert_eq!(t.len(), 1);
|
||||||
|
assert_eq!(t[0].shape, "ee");
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn tolerates_truncated_arrays() {
|
||||||
|
let t = timeline(&s(&["a", "e", "o"]), &[0, 100], &[100]);
|
||||||
|
assert_eq!(t.len(), 1);
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn punctuation_and_spaces_are_silence() {
|
||||||
|
let t = timeline(&s(&[" ", ",", "!"]), &[0, 100, 200], &[100, 100, 100]);
|
||||||
|
assert_eq!(t.len(), 1);
|
||||||
|
assert_eq!(t[0].shape, SILENCE);
|
||||||
|
assert_eq!(t[0].ms, 300);
|
||||||
|
}
|
||||||
|
}
|
||||||
+28
-58
@@ -1,86 +1,56 @@
|
|||||||
# Mnemosyne memory provider for Hermes on mars — packaged for real (Nix).
|
# Mnemosyne memory provider for Hermes. Built here rather than pip-installed:
|
||||||
|
# the image's Python has no pip and is PEP 668 managed.
|
||||||
#
|
#
|
||||||
# Why derivations instead of a runtime side-venv: the official Hermes image
|
# Core deps only: the embeddings extra (fastembed/onnxruntime) is optional at
|
||||||
# vendors its own Python and stays off-limits to pip (no pip module, PEP 668),
|
# runtime, and recall falls back to FTS5.
|
||||||
# and a runtime venv built host-side breaks twice over inside the container:
|
|
||||||
# the venv's pyvenv.cfg records a /nix/store python home the container never
|
|
||||||
# mounts, and a plugins symlink with an absolute host path points nowhere
|
|
||||||
# from /opt/data. Building here means nothing is fetched at boot, nothing
|
|
||||||
# under the provider's control is writable from inside the container, and
|
|
||||||
# the closure is as reproducible as the rest of the host.
|
|
||||||
#
|
|
||||||
# Package set (one shared site-packages — the plugin wrapper imports its
|
|
||||||
# sibling `mnemosyne` core package, so withPackages, not separate envs):
|
|
||||||
#
|
|
||||||
# mnemosyne-memory core engine: SQLite/FTS5 storage, recall, tools.
|
|
||||||
# Base deps only (PyYAML); the optional extras (llm,
|
|
||||||
# embeddings via fastembed/onnxruntime, mcp, sync) are
|
|
||||||
# deliberately NOT pulled — recall uses the bundled FTS5
|
|
||||||
# lexical path, and the heavyweight ML stack (~hundreds of
|
|
||||||
# MB, live network on first vector use) buys nothing for
|
|
||||||
# a first deployment. Adding the embeddings extra later
|
|
||||||
# is pinning fastembed + sqlite-vec here.
|
|
||||||
# mnemosyne-hermes the wrapper Hermes discovers under $HERMES_HOME/plugins
|
|
||||||
# (installs itself as package `hermes_memory_provider`).
|
|
||||||
# Hard dependency: mnemosyne-memory, PyYAML.
|
|
||||||
#
|
|
||||||
# Platform note: both sdists are pure Python (build no C extensions), so
|
|
||||||
# nothing here constrains the host arch beyond the interpreter itself.
|
|
||||||
{
|
{
|
||||||
python3,
|
python3,
|
||||||
fetchPypi,
|
fetchPypi,
|
||||||
}:
|
}:
|
||||||
|
|
||||||
let
|
let
|
||||||
python = python3;
|
mnemosyneMemory = python3.pkgs.buildPythonPackage rec {
|
||||||
|
|
||||||
mnemosyneMemory = python.pkgs.buildPythonPackage rec {
|
|
||||||
pname = "mnemosyne-memory";
|
pname = "mnemosyne-memory";
|
||||||
version = "3.15.1";
|
version = "3.15.1";
|
||||||
pyproject = true;
|
pyproject = true;
|
||||||
|
|
||||||
src = fetchPypi {
|
src = fetchPypi {
|
||||||
inherit pname version;
|
pname = "mnemosyne_memory";
|
||||||
sha256 = "sha256-lspUMxc0pUSkhSUrNdiiO5OJ1NMC/S853EYSanXtXKM=";
|
inherit version;
|
||||||
|
hash = "sha256-lspUMxc0pUSkhSUrNdiiO5OJ1NMC/S853EYSanXtXKM=";
|
||||||
};
|
};
|
||||||
|
|
||||||
build-system = with python.pkgs; [ setuptools ];
|
build-system = with python3.pkgs; [ setuptools ];
|
||||||
|
dependencies = with python3.pkgs; [ pyyaml ];
|
||||||
|
|
||||||
# Base dependency set — everything else in the upstream metadata is an
|
doCheck = false; # tests want a live Hermes + LLM key
|
||||||
# optional extra (llm / embeddings / mcp / sync / test / dev) and is not
|
# Ships the Hermes plugin package too, not just the core.
|
||||||
# installed; see the file-level comment.
|
pythonImportsCheck = [
|
||||||
dependencies = with python.pkgs; [ pyyaml ];
|
"mnemosyne"
|
||||||
|
"hermes_memory_provider"
|
||||||
doCheck = false; # upstream tests want a live Hermes + LLM key present
|
];
|
||||||
pythonImportsCheck = [ "mnemosyne" ];
|
|
||||||
};
|
};
|
||||||
|
|
||||||
mnemosyneHermes = python.pkgs.buildPythonPackage rec {
|
mnemosyneHermes = python3.pkgs.buildPythonPackage rec {
|
||||||
pname = "mnemosyne-hermes";
|
pname = "mnemosyne-hermes";
|
||||||
version = "0.5.0";
|
version = "0.5.0";
|
||||||
pyproject = true;
|
pyproject = true;
|
||||||
|
|
||||||
src = fetchPypi {
|
src = fetchPypi {
|
||||||
inherit pname version;
|
pname = "mnemosyne_hermes";
|
||||||
sha256 = "sha256-CzEvnUw5oPFtT5bHQQ/GBdy2C/E7qShQn32irIRYKqw=";
|
inherit version;
|
||||||
|
hash = "sha256-CzEvnUw5oPFtT5bHQQ/GBdy2C/E7qShQn32irIRYKqw=";
|
||||||
};
|
};
|
||||||
|
|
||||||
build-system = with python.pkgs; [ setuptools ];
|
build-system = with python3.pkgs; [ setuptools ];
|
||||||
|
# Upstream asks for mnemosyne-memory[embeddings]; see the header.
|
||||||
# The wrapper declares `mnemosyne-memory[embeddings]>=3.11.1` on PyPI, but
|
dependencies = [ mnemosyneMemory ] ++ (with python3.pkgs; [ pyyaml ]);
|
||||||
# the embeddings extra is only consulted when vector recall is enabled
|
|
||||||
# (see above) — pass the core dependency explicitly rather than dragging
|
|
||||||
# in onnxruntime for nothing.
|
|
||||||
dependencies = [ mnemosyneMemory ] ++ (with python.pkgs; [ pyyaml ]);
|
|
||||||
|
|
||||||
doCheck = false;
|
doCheck = false;
|
||||||
pythonImportsCheck = [ "hermes_memory_provider" ];
|
pythonImportsCheck = [ "mnemosyne_hermes" ];
|
||||||
};
|
};
|
||||||
|
|
||||||
# The exposed value is the python env itself (a store path mounted :ro).
|
|
||||||
# Hermes only needs the site-packages dir inside it; `sitePackages` is a
|
|
||||||
# passthru the python interpreter derivation (and hence withPackages envs)
|
|
||||||
# exposes, so the caller uses `${env.sitePackages}` instead of guessing
|
|
||||||
# the python version in a path literal.
|
|
||||||
in
|
in
|
||||||
python.withPackages (_: [ mnemosyneMemory mnemosyneHermes ])
|
python3.withPackages (_: [
|
||||||
|
mnemosyneMemory
|
||||||
|
mnemosyneHermes
|
||||||
|
])
|
||||||
|
|||||||
+4
-2
@@ -8,6 +8,8 @@ gitea_luna_token: ENC[AES256_GCM,data:0ypW9oVFs1mXYPhPareMFRdkSYcvHSCm+fQOd7/76l
|
|||||||
gitea_hermes_webhook_secret: ENC[AES256_GCM,data:lV78H0xAehPxusSO/QruOYkt7fkMJrW+ScZL4UWYvgnBGn/D+1XHYPyHCqe2sEEWSlIaAgWMMoZzoVJ1Z1NFVQ==,iv:GmTZxoH2iiL/vTVgPfziXIFYD+Rl3cbh9hqXvWps+iw=,tag:jtXEUOVFfKrpTRK7S9PZMA==,type:str]
|
gitea_hermes_webhook_secret: ENC[AES256_GCM,data:lV78H0xAehPxusSO/QruOYkt7fkMJrW+ScZL4UWYvgnBGn/D+1XHYPyHCqe2sEEWSlIaAgWMMoZzoVJ1Z1NFVQ==,iv:GmTZxoH2iiL/vTVgPfziXIFYD+Rl3cbh9hqXvWps+iw=,tag:jtXEUOVFfKrpTRK7S9PZMA==,type:str]
|
||||||
couchdb_luna_password: ENC[AES256_GCM,data:V91is2h7UskI1rtwMzQyduNXoDPTYNwa4sw9K9WU+wE=,iv:k976ImKR19+CvGOVsHsrqMaFFtSQxVi6zABCJuQ5AWE=,tag:W2SN4SET/+o4+Wt6BOJ1LA==,type:str]
|
couchdb_luna_password: ENC[AES256_GCM,data:V91is2h7UskI1rtwMzQyduNXoDPTYNwa4sw9K9WU+wE=,iv:k976ImKR19+CvGOVsHsrqMaFFtSQxVi6zABCJuQ5AWE=,tag:W2SN4SET/+o4+Wt6BOJ1LA==,type:str]
|
||||||
obsidian_luna_passphrase: ENC[AES256_GCM,data:fqHtP3g4J40ddYL9lzCixrisdC/DEJJermE=,iv:FU6BGNcBnyP8Rz3dNBk0+K0aAQTDW6/0aVaFm1rBFkk=,tag:Mg1aKqh++2rmU8ROaVIgDw==,type:str]
|
obsidian_luna_passphrase: ENC[AES256_GCM,data:fqHtP3g4J40ddYL9lzCixrisdC/DEJJermE=,iv:FU6BGNcBnyP8Rz3dNBk0+K0aAQTDW6/0aVaFm1rBFkk=,tag:Mg1aKqh++2rmU8ROaVIgDw==,type:str]
|
||||||
|
elevenlabs_api_key: ENC[AES256_GCM,data:2oJeClO4/SFsbLwRgMe3bFXrmuDX7YOc57VfSjElV5A3vaIQqfN9eHzo/cm0vISxYgUv,iv:eIeKQJ6odDnmGLjHSLV3HbErVnLr7tatOtV83TvB2QA=,tag:XPkXRfDJ2ZK98TYXKeK/CA==,type:str]
|
||||||
|
hermes_api_server_key: ENC[AES256_GCM,data:Au80iziwdryFTCLnggC9DdjjZPrGPKpmRFgkXe/l9RQ7on9nXb1dvQ==,iv:hWvv4fLkhRgVTT2btI+TvpNBHfMOg9y4BQKn+ht7XDo=,tag:2dEZEys8vsTwc7w0ZiBK1w==,type:str]
|
||||||
sops:
|
sops:
|
||||||
age:
|
age:
|
||||||
- enc: |
|
- enc: |
|
||||||
@@ -28,7 +30,7 @@ sops:
|
|||||||
oyJ7PS3lW+PxH5AZkeeU7gXO/pz2oDku0aDOds7kaD3n0+qSWicQ+Q==
|
oyJ7PS3lW+PxH5AZkeeU7gXO/pz2oDku0aDOds7kaD3n0+qSWicQ+Q==
|
||||||
-----END AGE ENCRYPTED FILE-----
|
-----END AGE ENCRYPTED FILE-----
|
||||||
recipient: age1eapjg6tdrr0fuvmgs3q3nlvnjkaxez298qynqqqxt0lpcv0lrsyq7ayxjk
|
recipient: age1eapjg6tdrr0fuvmgs3q3nlvnjkaxez298qynqqqxt0lpcv0lrsyq7ayxjk
|
||||||
lastmodified: "2026-08-25T21:51:24Z"
|
lastmodified: "2026-09-20T20:02:50Z"
|
||||||
mac: ENC[AES256_GCM,data:dz249hf3w8Tn0JStFOhhpdCZFMx2yxmNABx1CbeIQ/JlICAU82e4fg8AzJQY9EMOEs3Zx6L61yieljD4A/HLip5rDVlAXqqLeklW60eb7BHuSO79YfFgot+rS05g8WqFkRJYWzmkXhMbErCI133n72XEdeqMUU/m0djOUZlVRLs=,iv:d7G6TJRLfmXvQ2BUG9Hi83lOB9anhmb9qneLeVSCBhc=,tag:F6X/0z00PjiYum5kf8YApA==,type:str]
|
mac: ENC[AES256_GCM,data:xJVgl094qzW8YyjwDvRw4ymKjUhL3uWtbVN6J25S1Gi2I2siZ17MiUUis3dKXxb3OKq0NODP9/4pJDqWPv1qt88inu3l/6QUQMY6AybbGF7ooiiOSvGhR3jnCHpISptepNLzyHu0Iky6PkTLe5f5ETSolZ5R+Qyd66ZuFmUBjCM=,iv:u8XWCyjJo/ib9VSzi/e1FEbSjod98c1i14LykC7BEtw=,tag:ZYtrw0b3iEI+yaRtATgbkQ==,type:str]
|
||||||
unencrypted_suffix: _unencrypted
|
unencrypted_suffix: _unencrypted
|
||||||
version: 3.13.3
|
version: 3.13.3
|
||||||
|
|||||||
+4
-2
@@ -5,6 +5,8 @@ librechat_creds_key: ENC[AES256_GCM,data:e2Ptf41yHu0KxfzjW4DP04CSJfbtdsJ8bwrgJyv
|
|||||||
librechat_creds_iv: ENC[AES256_GCM,data:iuW1Rxfu7Ei8zHVG7wsBTYYznbtErj7DC9B71OiiWGA=,iv:Ngs88C44gcSzxQZU1lMiy5kn9mM/ckuWKRRGOFJeeoQ=,tag:Q9B8vM4YMkIP3RyYVWSQqg==,type:str]
|
librechat_creds_iv: ENC[AES256_GCM,data:iuW1Rxfu7Ei8zHVG7wsBTYYznbtErj7DC9B71OiiWGA=,iv:Ngs88C44gcSzxQZU1lMiy5kn9mM/ckuWKRRGOFJeeoQ=,tag:Q9B8vM4YMkIP3RyYVWSQqg==,type:str]
|
||||||
librechat_jwt_secret: ENC[AES256_GCM,data:f/gljrQPZIXeLHXtqKCCYGEu2pXgbZd69CjyEhSVJ+AMuaVj7DWVclD5IMsEVERBHw5a7OndBSfLvgXRaKN6Gg==,iv:DG5CwCbQLCTv1++APCdFzAiWGKiEJl+9MhKIM2ykJQM=,tag:wdpY6l+5h4UtsluIncsyhA==,type:str]
|
librechat_jwt_secret: ENC[AES256_GCM,data:f/gljrQPZIXeLHXtqKCCYGEu2pXgbZd69CjyEhSVJ+AMuaVj7DWVclD5IMsEVERBHw5a7OndBSfLvgXRaKN6Gg==,iv:DG5CwCbQLCTv1++APCdFzAiWGKiEJl+9MhKIM2ykJQM=,tag:wdpY6l+5h4UtsluIncsyhA==,type:str]
|
||||||
librechat_jwt_refresh_secret: ENC[AES256_GCM,data:N/yAPtaodJX4t2C2A0bZ/LQaVc8SBMgg3KYJrWRyXCM6HAmgHtSQ9nWJO2fl+7A966mMtYvpATYcBIPj/K5nig==,iv:k5ZYi2OKjL9Z+lhVgdNsmGVQm+iCqmF15H3TUTbofWk=,tag:NVTdvlL1lpwVhjo2c2phpg==,type:str]
|
librechat_jwt_refresh_secret: ENC[AES256_GCM,data:N/yAPtaodJX4t2C2A0bZ/LQaVc8SBMgg3KYJrWRyXCM6HAmgHtSQ9nWJO2fl+7A966mMtYvpATYcBIPj/K5nig==,iv:k5ZYi2OKjL9Z+lhVgdNsmGVQm+iCqmF15H3TUTbofWk=,tag:NVTdvlL1lpwVhjo2c2phpg==,type:str]
|
||||||
|
hermes_api_server_key: ENC[AES256_GCM,data:NU+6zVmUeM9+m5eZCPdCiMLo76GqbsRxNM7EmVM2TmKTOUL2af7Msg==,iv:EMd7JKhQ3ZkFPi2xMGZ+mHHK05qQVDR+ZCbiYcWdhHU=,tag:JaElU4CyvD5qQbehHbM5qQ==,type:str]
|
||||||
|
elevenlabs_api_key: ENC[AES256_GCM,data:4IBKKMxor1sJ1skh1pgLrQnNX31cgQFiy89O1TZn71QHaq+xY7p+7JLtmGkhbZ10l12m,iv:Ww5RLYuv09xD1deFgM89u0wZ3hZ/0Pndda0kreMx2aQ=,tag:opgIPsDwSc+rLqseqjP6Gg==,type:str]
|
||||||
sops:
|
sops:
|
||||||
age:
|
age:
|
||||||
- enc: |
|
- enc: |
|
||||||
@@ -25,7 +27,7 @@ sops:
|
|||||||
sHjKfw8VrrmAR4pQf1dsY+wcyh4FsZxhP3Q+QIVq3eCIXS9PeJkGAg==
|
sHjKfw8VrrmAR4pQf1dsY+wcyh4FsZxhP3Q+QIVq3eCIXS9PeJkGAg==
|
||||||
-----END AGE ENCRYPTED FILE-----
|
-----END AGE ENCRYPTED FILE-----
|
||||||
recipient: age1rfcmu6zh40v4260l9hnf8ajs9vly0s06rx3ey76eu78dp9t7getqyhmkut
|
recipient: age1rfcmu6zh40v4260l9hnf8ajs9vly0s06rx3ey76eu78dp9t7getqyhmkut
|
||||||
lastmodified: "2026-08-21T23:14:15Z"
|
lastmodified: "2026-09-20T20:02:50Z"
|
||||||
mac: ENC[AES256_GCM,data:gD5C3hYFvIUP7zr4GK40LAtM2sskhGErEzdTxVKaRaPkKNYj+pDYd5uSrBxDHv6W2SAY0TtyN3GyNpD0z5b4yaDxiV8ETUOokkj8f9FlTaZk56S6I/21HvaGXW+Cfi1ioFtNGyJeAhSnNOJ6OE0AA9JfcftSJ8BdTe2LienjHwQ=,iv:LtKFum+f1kaE6/XT4Xj0ZEuFrhSXXsnILMgdebGYNDU=,tag:6DxwNjjgze2ldU1g3SdzzA==,type:str]
|
mac: ENC[AES256_GCM,data:fEgOKCOYrpnojw9yl3b7ZgJFCcmAs+DmY7etHgD5KNBtoZ3QgQUiTY5+kXvO5HMoENve+bUfyHymUDNjLlb45lItl9ZkAMKvTxGOe+5t0MR2X8BkB/2tjl/fS7CRBTJ0D4S84mH1hB2V2cfKcnGoKNsQyRJddi6OOey534QeV44=,iv:nQuRLkGklMXwkh8LpMC/Tmz1ewRiv8iKbK6IA88r2fA=,tag:LnzQc1sjrIdZ26AtH1Laxw==,type:str]
|
||||||
unencrypted_suffix: _unencrypted
|
unencrypted_suffix: _unencrypted
|
||||||
version: 3.13.2
|
version: 3.13.2
|
||||||
|
|||||||
@@ -0,0 +1,20 @@
|
|||||||
|
{ pkgs, ... }:
|
||||||
|
|
||||||
|
let
|
||||||
|
claude-desktop = pkgs.callPackage ../../pkgs/claude-desktop.nix { };
|
||||||
|
in
|
||||||
|
|
||||||
|
# Claude Desktop, including Cowork (a QEMU/KVM VM the app runs agent tasks in).
|
||||||
|
{
|
||||||
|
environment.systemPackages = [ claude-desktop ];
|
||||||
|
|
||||||
|
# The app only probes these Debian paths, never $PATH or an env var.
|
||||||
|
systemd.tmpfiles.rules = [
|
||||||
|
"L+ /usr/share/OVMF/OVMF_CODE_4M.fd - - - - ${pkgs.OVMF.fd}/FV/OVMF_CODE.fd"
|
||||||
|
"L+ /usr/share/OVMF/OVMF_VARS_4M.fd - - - - ${pkgs.OVMF.fd}/FV/OVMF_VARS.fd"
|
||||||
|
"L+ /usr/libexec/virtiofsd - - - - ${claude-desktop}/lib/claude-desktop/resources/virtiofsd"
|
||||||
|
];
|
||||||
|
|
||||||
|
boot.kernelModules = [ "vhost_vsock" ];
|
||||||
|
users.users.darman.extraGroups = [ "kvm" ];
|
||||||
|
}
|
||||||
@@ -16,6 +16,7 @@ in
|
|||||||
"steam-unwrapped"
|
"steam-unwrapped"
|
||||||
"steam-run"
|
"steam-run"
|
||||||
"claude-code"
|
"claude-code"
|
||||||
|
"claude-desktop" # services/desktop/claude-desktop.nix
|
||||||
"proton-pass-cli"
|
"proton-pass-cli"
|
||||||
"vivaldi"
|
"vivaldi"
|
||||||
"mongodb" # librechat's local db (services/desktop/librechat.nix) — SSPL
|
"mongodb" # librechat's local db (services/desktop/librechat.nix) — SSPL
|
||||||
@@ -45,6 +46,8 @@ in
|
|||||||
dotnetCorePackages.sdk_10_0
|
dotnetCorePackages.sdk_10_0
|
||||||
nodejs
|
nodejs
|
||||||
yaak # desktop API client (REST/GraphQL/gRPC)
|
yaak # desktop API client (REST/GraphQL/gRPC)
|
||||||
|
# pkgsRocm, not plain blender: Cycles needs HIP to render on the RX 6800 XT.
|
||||||
|
pkgsRocm.blender
|
||||||
];
|
];
|
||||||
|
|
||||||
fonts.packages = [ pkgs.nerd-fonts.departure-mono ];
|
fonts.packages = [ pkgs.nerd-fonts.departure-mono ];
|
||||||
|
|||||||
@@ -0,0 +1,169 @@
|
|||||||
|
{ config, pkgs, lib, ... }:
|
||||||
|
|
||||||
|
# Voice loop for luna: SUPER+V opens the mic, talk, hear her answer.
|
||||||
|
#
|
||||||
|
# mic -> whisper-cpp (local, vulkan) -> Hermes' OpenAI-compatible API on mars
|
||||||
|
# -> ElevenLabs streaming TTS -> speaker. State, amplitude, captions and a
|
||||||
|
# viseme timeline are published on a unix socket; the rail's LunaPanel reads
|
||||||
|
# them (dotfiles/quickshell/HyprChrome/Widgets/Bar/Luna), and the viseme
|
||||||
|
# stream is there for the avatar that replaces its meter later.
|
||||||
|
#
|
||||||
|
# A USER service, not a system one: it needs darman's pipewire session and his
|
||||||
|
# XDG_RUNTIME_DIR for the socket. That is also why the two secrets it reads
|
||||||
|
# are owner-darman in hosts/terra/secrets.nix — a user manager cannot read a
|
||||||
|
# root-owned 0400 file, and LoadCredential does not bridge that gap.
|
||||||
|
#
|
||||||
|
# STT is LOCAL on purpose: mars already runs faster-whisper for Telegram voice
|
||||||
|
# notes, but routing the mic through it would put every spoken word on the
|
||||||
|
# network for no latency win. This is the client-direct path Hermes itself
|
||||||
|
# documents.
|
||||||
|
|
||||||
|
let
|
||||||
|
cfg = config.homelab.lunaVoice;
|
||||||
|
luna-voice = pkgs.callPackage ../../pkgs/luna-voice.nix { };
|
||||||
|
|
||||||
|
# Pinned like the KittenTTS models on mars: a model that re-downloads at
|
||||||
|
# first use is a service that fails on a fresh install with no network.
|
||||||
|
whisperModel = pkgs.fetchurl {
|
||||||
|
url = "https://huggingface.co/ggerganov/whisper.cpp/resolve/5359861c739e955e79d9a303bcbc70fb988958b1/ggml-base.en.bin";
|
||||||
|
sha256 = "sha256-oDd5yG3zMjB19eeWyyzlAp8A7Ihp7uP9+4l6/jbG0AI=";
|
||||||
|
};
|
||||||
|
in
|
||||||
|
{
|
||||||
|
options.homelab.lunaVoice = {
|
||||||
|
enable = lib.mkEnableOption "luna voice daemon";
|
||||||
|
|
||||||
|
user = lib.mkOption {
|
||||||
|
type = lib.types.str;
|
||||||
|
default = "darman";
|
||||||
|
description = "User whose session runs the daemon.";
|
||||||
|
};
|
||||||
|
|
||||||
|
hermesUrl = lib.mkOption {
|
||||||
|
type = lib.types.str;
|
||||||
|
default = "http://mars.orbit.sol:8642";
|
||||||
|
description = "Base URL of luna's Hermes API server (tailnet-only).";
|
||||||
|
};
|
||||||
|
|
||||||
|
model = lib.mkOption {
|
||||||
|
type = lib.types.str;
|
||||||
|
default = "hermes-agent";
|
||||||
|
description = "Model name to request; Hermes routes it to her configured provider.";
|
||||||
|
};
|
||||||
|
|
||||||
|
voiceId = lib.mkOption {
|
||||||
|
type = lib.types.str;
|
||||||
|
description = "ElevenLabs voice id — luna's voice.";
|
||||||
|
};
|
||||||
|
|
||||||
|
ttsModel = lib.mkOption {
|
||||||
|
type = lib.types.str;
|
||||||
|
default = "eleven_v3_conversational";
|
||||||
|
description = ''
|
||||||
|
eleven_v3* streams over the Text to Dialogue socket and performs audio
|
||||||
|
tags ([laughs], [sighs]); it also gets her German right, which Flash
|
||||||
|
reads with an English accent. Anything else uses the TTS socket, e.g.
|
||||||
|
"eleven_flash_v2_5" — half the price per character, faster, no tags.
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
|
||||||
|
hermesKeyFile = lib.mkOption {
|
||||||
|
type = lib.types.path;
|
||||||
|
description = "File holding the API_SERVER_KEY bearer token.";
|
||||||
|
};
|
||||||
|
|
||||||
|
elevenLabsKeyFile = lib.mkOption {
|
||||||
|
type = lib.types.path;
|
||||||
|
description = "File holding the ElevenLabs API key.";
|
||||||
|
};
|
||||||
|
|
||||||
|
# Mic tuning. Defaults are set for terra's AIR 192|4, which is QUIET:
|
||||||
|
# ~0.002 ambient against ~0.0045 for normal speech. Raising the gain on
|
||||||
|
# the interface is worth more than any value here — everything downstream,
|
||||||
|
# whisper included, works better with headroom.
|
||||||
|
inputDevice = lib.mkOption {
|
||||||
|
type = lib.types.nullOr lib.types.str;
|
||||||
|
default = null;
|
||||||
|
description = "Substring of the input device name; null uses the system default.";
|
||||||
|
};
|
||||||
|
|
||||||
|
vadMult = lib.mkOption {
|
||||||
|
type = lib.types.float;
|
||||||
|
default = 1.8;
|
||||||
|
description = "Speech triggers above noise_floor * this.";
|
||||||
|
};
|
||||||
|
|
||||||
|
vadFloor = lib.mkOption {
|
||||||
|
type = lib.types.float;
|
||||||
|
default = 0.0025;
|
||||||
|
description = "Absolute lower bound on the trigger, whatever the floor says.";
|
||||||
|
};
|
||||||
|
|
||||||
|
silenceMs = lib.mkOption {
|
||||||
|
type = lib.types.int;
|
||||||
|
default = 2000;
|
||||||
|
description = "Silence that ends an utterance.";
|
||||||
|
};
|
||||||
|
|
||||||
|
voiceMarker = lib.mkOption {
|
||||||
|
type = lib.types.str;
|
||||||
|
default = "[voice]";
|
||||||
|
description = ''
|
||||||
|
Prefixed to every spoken message so luna can tell this channel from
|
||||||
|
Telegram or the dashboard, and adjust — no kaomoji, no markdown, short
|
||||||
|
answers. Put the rule in HER profile on mars rather than only in
|
||||||
|
systemPrompt below: the marker travels with the message, so it still
|
||||||
|
identifies the channel in whatever history Hermes keeps, and one place
|
||||||
|
governs her voice manners across every client that sends it.
|
||||||
|
|
||||||
|
Empty string disables the marker.
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
|
||||||
|
systemPrompt = lib.mkOption {
|
||||||
|
type = lib.types.nullOr lib.types.str;
|
||||||
|
default = null;
|
||||||
|
description = ''
|
||||||
|
Prepended to the conversation. Worth setting to something like "You
|
||||||
|
are being spoken to out loud; keep replies short" — her Telegram
|
||||||
|
register is too long-form to listen to.
|
||||||
|
'';
|
||||||
|
};
|
||||||
|
};
|
||||||
|
|
||||||
|
config = lib.mkIf cfg.enable {
|
||||||
|
environment.systemPackages = [ luna-voice ];
|
||||||
|
|
||||||
|
systemd.user.services.luna-voiced = {
|
||||||
|
description = "luna voice loop";
|
||||||
|
wantedBy = [ "graphical-session.target" ];
|
||||||
|
after = [ "graphical-session.target" "pipewire.service" ];
|
||||||
|
partOf = [ "graphical-session.target" ];
|
||||||
|
|
||||||
|
environment = {
|
||||||
|
LUNA_HERMES_URL = cfg.hermesUrl;
|
||||||
|
LUNA_MODEL = cfg.model;
|
||||||
|
LUNA_VOICE_ID = cfg.voiceId;
|
||||||
|
LUNA_TTS_MODEL = cfg.ttsModel;
|
||||||
|
LUNA_HERMES_KEY_FILE = toString cfg.hermesKeyFile;
|
||||||
|
LUNA_ELEVENLABS_KEY_FILE = toString cfg.elevenLabsKeyFile;
|
||||||
|
LUNA_WHISPER_BIN = "${pkgs.whisper-cpp-vulkan}/bin/whisper-cli";
|
||||||
|
LUNA_WHISPER_MODEL = whisperModel;
|
||||||
|
LUNA_VAD_MULT = toString cfg.vadMult;
|
||||||
|
LUNA_VAD_FLOOR = toString cfg.vadFloor;
|
||||||
|
LUNA_SILENCE_MS = toString cfg.silenceMs;
|
||||||
|
LUNA_VOICE_MARKER = cfg.voiceMarker;
|
||||||
|
} // lib.optionalAttrs (cfg.systemPrompt != null) {
|
||||||
|
LUNA_SYSTEM_PROMPT = cfg.systemPrompt;
|
||||||
|
} // lib.optionalAttrs (cfg.inputDevice != null) {
|
||||||
|
LUNA_INPUT_DEVICE = cfg.inputDevice;
|
||||||
|
};
|
||||||
|
|
||||||
|
serviceConfig = {
|
||||||
|
ExecStart = "${luna-voice}/bin/luna-voice daemon";
|
||||||
|
Restart = "on-failure";
|
||||||
|
RestartSec = 3;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
};
|
||||||
|
}
|
||||||
@@ -0,0 +1,26 @@
|
|||||||
|
{ pkgs, ... }:
|
||||||
|
|
||||||
|
# gitea-mcp in HTTP mode, tailnet-only at http://jupiter.orbit.sol:8093/mcp.
|
||||||
|
# It holds no token: each client sends its own PAT as `Authorization: Bearer
|
||||||
|
# <token>`, so an unauthenticated request can only see what anonymous gitea
|
||||||
|
# users can.
|
||||||
|
{
|
||||||
|
systemd.services.gitea-mcp = {
|
||||||
|
description = "Gitea MCP server";
|
||||||
|
after = [ "gitea.service" ];
|
||||||
|
wants = [ "gitea.service" ];
|
||||||
|
wantedBy = [ "multi-user.target" ];
|
||||||
|
environment.GITEA_HOST = "http://localhost:3000";
|
||||||
|
serviceConfig = {
|
||||||
|
# Binds every interface (no listen-address flag); tailnet-only because
|
||||||
|
# tailscale0 is trusted and 8093 stays closed on the LAN.
|
||||||
|
ExecStart = "${pkgs.gitea-mcp-server}/bin/gitea-mcp -t http -p 8093";
|
||||||
|
DynamicUser = true;
|
||||||
|
Restart = "on-failure";
|
||||||
|
ProtectSystem = "strict";
|
||||||
|
ProtectHome = true;
|
||||||
|
PrivateTmp = true;
|
||||||
|
NoNewPrivileges = true;
|
||||||
|
};
|
||||||
|
};
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user