Compare commits

..
Author SHA1 Message Date
luna 76ebc8c822 mars: add KittenTTS voice provider (CPU-only, offline mini model)
tts.provider: kittentts, voice Luna, mini-0.8 model (80M params) per
darman. CPU-only onnxruntime inference — no GPU on mars, validated ~1x
realtime on a weaker dev box. Upstream's misaki[en]+spacy declaration is
deliberately not honored (pulls torch→CUDA, 5.6 GB); a .pth-registered
stub satisfies the dead misaki import, failing loudly if it is ever
actually used. HF model cache pre-seeded from hash-pinned store paths so
HF_HUB_OFFLINE=1 runs with zero boot-time network. Provisioner shaped by
the Mnemosyne review: uv venv --clear, root never executes python from
the venv, ownership to the container uid before anything imports, stamp
checked against venv+model files.
2026-09-19 12:09:03 +00:00
44 changed files with 463 additions and 6346 deletions
-3
View File
@@ -17,6 +17,3 @@ keys.txt
# local visual-verification output
.artifacts/
# rust build output
target/
-32
View File
@@ -210,38 +210,6 @@ kept its ssh host key. Run it after ANY change to the kexec paths.
- **`kexec-local` stages on `/var/tmp`, not `/tmp`**: `kexec-run.sh` appends a fresh cpio
to `kexec/initrd` in place and execs binaries from that dir, so a size-capped or
`noexec` tmpfs gives a half-written initrd or a bare "Permission denied".
- **A sops secret a USER service reads needs `owner`, not a mode.** `luna-voiced`
(`services/desktop/luna-voice.nix`) runs as darman's user unit, and a user manager
cannot read root:root 0400 — `LoadCredential` does not bridge that gap either.
Hence `sops.secrets.<x>.owner = "darman"` in `hosts/terra/secrets.nix`.
- **`elevenlabs_api_key` is in BOTH `secrets/mars.yaml` and `secrets/terra.yaml`** —
same value. terra calls ElevenLabs directly (the client-direct path Hermes
documents) rather than relaying audio through mars, so rotating that key means
editing two files. `hermes_api_server_key` is duplicated for the same reason:
mars serves the API, terra is its only client.
- **terra's mic is QUIET, and every voice threshold is sized for it.** Measured on
the AIR 192|4 (`luna-voice level`): ~0.001–0.002 RMS ambient, only 0.0045–0.005
for normal speech — about 6dB of separation. Consequences, all of which looked
like "the daemon is broken": a VAD multiplier above ~2 can never fire, an
absolute trigger floor of 0.006 sits *above* the user's voice, and whisper's own
gate at 0.015 discarded every word the VAD did capture. Hence `vadMult = 1.8`,
`vadFloor = 0.0025`, and `stt::normalize` scaling the clip up (capped at 30x)
before whisper sees it. Raising the gain on the interface is worth more than any
of these; tune via `homelab.lunaVoice.vad*` rather than editing Rust.
- **`try_recv()` POPS — an `if let Ok(Cancel)` destroys every other command.**
luna-voiced polls for a cancel while she speaks; written as a single `if let`,
any `say`/`toggle`/`mode off` arriving mid-reply was taken off the channel and
dropped, so SUPER+V during her answer did nothing at all and the next turn never
ran. Non-cancel commands are collected and re-sent into the channel after the
turn instead (`requeue` in `turn()`).
- **A rising sound must not update the noise floor.** Feeding every sample into the
floor EMA lets speech drag its own threshold up faster than it rises — measured,
a tone peaking at 0.029 was chased by a threshold that reached 0.018 and never
triggered. Only samples *below* the current trigger update the floor.
- **whisper invents words out of silence** — "you", "Thank you.", "Bye." — and a
held push-to-talk key over a quiet room is long AND empty, so duration alone
does not catch it. `stt::has_speech` gates on the loudest 100ms window instead;
without it every stray keypress sends a hallucinated sentence to luna.
- **terra's greeter is a throwaway Hyprland running `dotfiles/quickshell/greeter.qml`**
(`services/desktop/quickshell-greeter.nix`). It must exit after login or greetd never
starts the session, and with a Lua config `hyprctl dispatch exit` is REJECTED — it needs
+2 -5
View File
@@ -70,9 +70,8 @@ configured via `.qmlls.ini` for editor diagnostics.
## Architecture
`shell.qml` is a `Scope` instantiating the top-level pieces as siblings:
`HyprChromeShell` (the status rail, plus the surfaces it owns — launcher, polkit,
askpass, notifications), the ten remaining launcher variants,
`VolumeOsd` and `Vitals`. Each manages its own
`HyprChromeShell` (the status rail), the eleven launcher variants,
`Notifications`, `VolumeOsd` and `Vitals`. Each manages its own
`PanelWindow`(s); there is no central layout manager.
The one exception, and the pattern to follow for anything new that needs it, is
@@ -101,8 +100,6 @@ on it, hence the index loops in `HyprChromeShell`.
and its panels, with `Bar/Panels/BarPanel.qml` the chamfered chrome they all
extend; `Widgets/Polkit/` is the authentication agent and its dialog;
`Widgets/Launcher/` is the primary application launcher (`SUPER_L`);
`Widgets/Notifications/` is the notification daemon and its popup stack on the
rail's monitor (`NotificationCard` is the headless-renderable card);
`Widgets/Greeter/` is the greetd login screen, run standalone via `greeter.qml`
(see `services/desktop/quickshell-greeter.nix`), NOT part of `shell.qml`;
`Theme/Theme.qml` is this tree's palette singleton. `DebugWindow.qml` stages a
@@ -4,7 +4,7 @@ import QtQuick.Layouts
import Quickshell
import Quickshell.Wayland
import qs.HyprChrome.Theme
import qs.HyprChrome.Widgets.Bar.Luna
import qs.HyprChrome.Widgets.Bar.Debug
import qs.HyprChrome.Widgets.Bar.Host
import qs.HyprChrome.Widgets.Bar.Vitals
import qs.HyprChrome.Widgets.Bar.Workspaces
@@ -87,8 +87,8 @@ PanelWindow {
//
// The zone still follows the target height, so tiled windows reflow once
// per toggle, at the start, and slide while the panels animate.
readonly property real expandedContent: Math.max(hostPanel.expandedHeight, workspacesPanel.expandedHeight, vitalsPanel.expandedHeight, lunaPanel.expandedHeight, trayPanel.expandedHeight) + window.margin * 2
readonly property real contentHeight: Math.max(hostPanel.targetHeight, workspacesPanel.targetHeight, vitalsPanel.targetHeight, lunaPanel.targetHeight, trayPanel.targetHeight) + window.margin * 2
readonly property real expandedContent: Math.max(hostPanel.expandedHeight, workspacesPanel.expandedHeight, vitalsPanel.expandedHeight, testPanel.expandedHeight, trayPanel.expandedHeight) + window.margin * 2
readonly property real contentHeight: Math.max(hostPanel.targetHeight, workspacesPanel.targetHeight, vitalsPanel.targetHeight, testPanel.targetHeight, trayPanel.targetHeight) + window.margin * 2
implicitHeight: Math.round(window.expandedContent)
@@ -157,8 +157,8 @@ PanelWindow {
// and a trace at each end.
Item { Layout.fillWidth: true }
LunaPanel {
id: lunaPanel
TestPanel {
id: testPanel
expanded: window.expanded
toggleOnClick: false
@@ -1,200 +0,0 @@
pragma ComponentBehavior: Bound
import QtQuick
import qs.HyprChrome.Theme
import qs.HyprChrome.Widgets.Bar.Panels
// luna's voice loop on the rail (services/desktop/luna-voice.nix).
//
// Always present, unlike the daemon it watches: the panel is where you look
// to find out whether she is listening, so it has to be there when the answer
// is "no" — including when luna-voiced is not running at all. SUPER+V toggles
// LISTENING, not this.
//
// Collapsed it is the state and nothing else, because that is the one fact
// worth a glance; expanded it carries the last exchange, which is the thing
// you actually want during the seconds her reply takes to arrive.
BarPanel {
id: luna
// Overridable so the headless render can stage every state without a
// daemon; in the rail they track the socket below.
property string daemonState: state.state
property bool active: state.active
property bool connected: state.connected
property real rms: state.rms
property string userText: state.userText
property string lunaText: state.lunaText
// Instantiated here rather than in the shell: the panel is the only
// reader, and it is a second reader that would make this shell state.
readonly property var state: LunaState {}
panelId: "LUN"
title: "LUNA"
meta: luna.active ? "VOICE" : "IDLE"
// Sized to content like the tray, and per DENSITY: collapsed it is one
// word, so reserving the expanded body's width would leave the rail
// carrying a mostly empty panel for the state it spends most of its time
// in. It floats between two spacers, so the width change costs no other
// panel anything.
implicitWidth: luna.expanded
? Math.max(luna.headerMinWidth, luna.padding * 2 + body.implicitWidth)
: Math.max(luna.headerMinWidth, luna.briefLeft + brief.implicitWidth + luna.padding)
readonly property color stateColor: {
if (!luna.connected)
return Theme.disabled;
switch (luna.daemonState) {
case "listening":
return Theme.accent;
case "thinking":
return Theme.muted;
case "speaking":
return Theme.accentSoft;
default:
return luna.active ? Theme.text : Theme.raised;
}
}
readonly property string stateLabel: {
if (!luna.connected)
return "OFFLINE";
if (!luna.active)
return "OFF";
switch (luna.daemonState) {
case "listening":
return "LISTENING";
case "thinking":
return "THINKING";
case "speaking":
return "SPEAKING";
default:
return "READY";
}
}
summary: Row {
id: brief
spacing: 6
// Breathes only while the mic is open, so "she is listening to me
// right now" is legible without reading the word next to it.
Rectangle {
width: 7
height: 7
radius: 4
anchors.verticalCenter: parent.verticalCenter
color: luna.stateColor
SequentialAnimation on opacity {
running: luna.daemonState === "listening"
loops: Animation.Infinite
alwaysRunToEnd: true
NumberAnimation { to: 0.25; duration: 700; easing.type: Easing.InOutQuad }
NumberAnimation { to: 1.0; duration: 700; easing.type: Easing.InOutQuad }
}
}
Text {
anchors.verticalCenter: parent.verticalCenter
text: luna.stateLabel
color: luna.stateColor
font.family: Theme.displayFont
font.pixelSize: 13
font.bold: true
}
}
Column {
id: body
spacing: 6
// Level meter: live whenever voice mode is on, not only once a
// trigger has fired, so a mic too quiet to reach the threshold still
// shows movement instead of looking broken.
Row {
id: meter
width: 260
height: 18
spacing: 2
readonly property int count: 28
readonly property bool sweeping: luna.daemonState === "thinking"
property real phase: 0
NumberAnimation on phase {
running: meter.sweeping
loops: Animation.Infinite
from: 0
to: 1
duration: 1200
}
Repeater {
model: meter.count
Rectangle {
required property int index
width: (meter.width - (meter.count - 1) * meter.spacing) / meter.count
height: {
if (meter.sweeping) {
const pos = index / (meter.count - 1);
return 2 + 14 * Math.max(0, 1 - Math.abs(pos - meter.phase) * 6);
}
if (!luna.active && luna.daemonState !== "speaking")
return 2;
// sqrt opens up the quiet end, which on this mic is
// where all of the speech actually sits.
const level = Math.min(1, Math.sqrt(luna.rms) * 6.0);
const centre = 1 - Math.abs(index / (meter.count - 1) - 0.5) * 1.6;
return 2 + 16 * level * Math.max(0.15, centre);
}
anchors.bottom: parent.bottom
color: luna.stateColor
opacity: meter.sweeping ? 0.9 : 0.5 + 0.5 * (height / 18)
Behavior on height {
enabled: !meter.sweeping
NumberAnimation { duration: 70; easing.type: Easing.OutQuad }
}
}
}
}
Text {
width: 260
visible: luna.userText !== ""
text: `> ${luna.userText}`
color: Theme.muted
font.family: Theme.microFont
font.pixelSize: 10
wrapMode: Text.WordWrap
maximumLineCount: 2
elide: Text.ElideRight
}
Text {
width: 260
visible: luna.lunaText !== ""
text: luna.lunaText
color: Theme.text
font.family: Theme.microFont
font.pixelSize: 11
wrapMode: Text.WordWrap
maximumLineCount: 4
elide: Text.ElideRight
}
Text {
visible: luna.userText === "" && luna.lunaText === ""
text: luna.connected ? "SUPER+V TO TALK" : "DAEMON NOT RUNNING"
color: Theme.muted
font.family: Theme.microFont
font.pixelSize: 9
font.letterSpacing: 1.4
}
}
}
@@ -1,106 +0,0 @@
pragma ComponentBehavior: Bound
import QtQuick
import Quickshell
import Quickshell.Io
// Everything the Luna surfaces know, read off luna-voiced's unix socket
// (services/desktop/luna-voice.nix).
//
// Read-only on purpose: the key that toggles voice mode runs `luna-voice
// toggle` from Hyprland rather than writing here, so the toggle keeps working
// while the shell is reloading — and a QML error can never leave the mic on
// with nothing showing it.
//
// The daemon is a USER service that may start after the shell, restart under
// it, or not be installed at all, so a missing socket is a normal state and
// not an error: `reconnect` retries quietly and `connected` is what the
// widget shows.
QtObject {
id: root
// "idle" | "listening" | "thinking" | "speaking"
property string state: "idle"
// Voice mode itself — the widget is only on screen while this is true.
property bool active: false
property bool connected: socket.connected
// Mic level while listening, her output level while speaking. Drives the
// only genuinely live part of the visuals.
property real rms: 0
property string userText: ""
property string lunaText: ""
property string error: ""
readonly property var socket: Socket {
id: socket
path: `${Quickshell.env("XDG_RUNTIME_DIR") ?? "/run/user/1000"}/luna-voiced.sock`
connected: true
onConnectionStateChanged: {
if (!socket.connected) {
// A daemon that went away is not a daemon that is idle: reset
// rather than leave the last frame of a conversation frozen
// on screen.
root.active = false;
root.state = "idle";
root.rms = 0;
}
}
parser: SplitParser {
splitMarker: "\n"
onRead: line => root.ingest(line)
}
}
// Retry forever, cheaply. Setting `connected` true on an already-open
// socket is a no-op, so this doubles as the initial connect.
readonly property var reconnect: Timer {
interval: 2000
running: !socket.connected
repeat: true
onTriggered: socket.connected = true
}
function ingest(line: string): void {
let msg;
try {
msg = JSON.parse(line);
} catch (e) {
return; // a half-written line is not worth a console error
}
switch (msg.type) {
case "mode":
root.active = msg.active;
if (!msg.active) {
root.state = "idle";
root.rms = 0;
}
break;
case "state":
root.state = msg.value;
// Level is only meaningful during the phase that produces it.
if (msg.value !== "listening" && msg.value !== "speaking")
root.rms = 0;
break;
case "amplitude":
root.rms = msg.rms;
break;
case "caption":
if (msg.speaker === "user") {
// A new user caption starts a turn, so it is what clears her
// last reply — NOT speech_start, which arrives after her
// first caption, not before it.
root.userText = msg.text;
root.lunaText = "";
} else {
root.lunaText = msg.text;
}
break;
case "error":
root.error = msg.message;
break;
}
}
}
@@ -8,7 +8,6 @@ import qs.HyprChrome.Widgets
import qs.HyprChrome.Widgets.Polkit
import qs.HyprChrome.Widgets.Launcher
import qs.HyprChrome.Widgets.Askpass
import qs.HyprChrome.Widgets.Notifications
// The hyprchrome shell: owns everything the rail's surfaces have to agree on,
// and instantiates them.
@@ -211,11 +210,6 @@ Scope {
onDismissed: shell.expanded = false
}
// Popups on the rail's monitor, below its exclusive zone.
Notifications {
screen: shell.targetScreen
}
// Polkit authentication agent. It registers for this logind session on
// creation, so it replaces hyprpolkitagent rather than coexisting with it —
// only one agent may hold a session (see hosts/terra/home/hyprland.nix).
@@ -1,272 +0,0 @@
pragma ComponentBehavior: Bound
import QtQuick
import QtQuick.Layouts
import Quickshell
import Quickshell.Widgets
import qs.HyprChrome.Theme
// Headlessly renderable visual core of one notification. Every field arrives as
// a plain property and every action leaves as a signal; Notifications.qml maps
// the server's Notification onto it.
//
// Owns its own expiry: the drain meter IS the timer, so hovering pauses both.
Item {
id: root
property string appName: ""
property string summary: ""
property string body: ""
// Icon name, path or URL; `image` (avatar, album art) wins over it.
property string appIcon: ""
property string image: ""
// 0 low, 1 normal, 2 critical — NotificationUrgency's values.
property int urgency: 1
property string time: ""
// Read structurally ({ identifier, text }). The "default" action is the
// card's own click, not a chip.
property var actions: []
// Milliseconds; 0 never expires.
property int timeout: 0
// Off for headless renders, which set `remaining` directly.
property bool countdown: true
property real remaining: 1
signal dismissed
signal expired
signal actionInvoked(string identifier)
readonly property bool critical: root.urgency === 2
readonly property color tint: root.critical ? Theme.hot : root.urgency === 0 ? Theme.muted : Theme.accent
readonly property var chipActions: {
const out = [];
for (let i = 0; i < root.actions.length; i++) {
if (root.actions[i].identifier !== "default")
out.push(root.actions[i]);
}
return out;
}
readonly property bool hasDefault: root.chipActions.length !== root.actions.length
readonly property string iconSource: {
const s = root.image !== "" ? root.image : root.appIcon;
if (s === "")
return "";
if (s.startsWith("/"))
return "file://" + s;
if (s.includes(":/"))
return s;
return Quickshell.iconPath(s, true);
}
implicitWidth: 360
implicitHeight: panel.implicitHeight
NumberAnimation on remaining {
id: drain
from: 1
to: 0
duration: Math.max(1, root.timeout)
running: root.countdown && root.timeout > 0
paused: running && hover.hovered
}
// Not drain's finished(): a value-source animation need not emit it.
onRemainingChanged: {
if (root.remaining <= 0 && root.countdown && root.timeout > 0)
root.expired();
}
HoverHandler {
id: hover
}
// Under the panel, so the close box and chips take their own clicks.
MouseArea {
anchors.fill: parent
cursorShape: root.hasDefault ? Qt.PointingHandCursor : Qt.ArrowCursor
onClicked: {
if (root.hasDefault)
root.actionInvoked("default");
else
root.dismissed();
}
}
component MicroText: Text {
color: Theme.muted
font.family: Theme.microFont
font.pixelSize: 8
font.letterSpacing: 0.9
elide: Text.ElideRight
}
NotificationPanel {
id: panel
width: root.width
panelId: root.critical ? "CRT" : root.urgency === 0 ? "LOW" : "NTF"
title: (root.appName !== "" ? root.appName : "NOTIFICATION").toUpperCase()
meta: root.time
accentColor: root.tint
onCloseRequested: root.dismissed()
ColumnLayout {
width: parent.width
spacing: 8
RowLayout {
Layout.fillWidth: true
spacing: 10
// Framed, so a transparent icon still reads as a tile.
Rectangle {
Layout.alignment: Qt.AlignTop
visible: root.iconSource !== ""
implicitWidth: 40
implicitHeight: 40
color: Theme.selection
border.width: 1
border.color: Theme.hair
IconImage {
anchors.centerIn: parent
implicitSize: root.image !== "" ? 38 : 28
source: root.iconSource
}
}
ColumnLayout {
Layout.fillWidth: true
Layout.alignment: Qt.AlignTop
spacing: 3
Text {
Layout.fillWidth: true
visible: root.summary !== ""
text: root.summary
textFormat: Text.PlainText
wrapMode: Text.Wrap
maximumLineCount: 2
elide: Text.ElideRight
color: Theme.text
font.family: Theme.displayFont
font.pixelSize: 13
font.bold: true
font.letterSpacing: 0.4
}
// Body markup is advertised, so StyledText, never RichText.
Text {
Layout.fillWidth: true
visible: root.body !== ""
text: root.body
textFormat: Text.StyledText
wrapMode: Text.Wrap
maximumLineCount: 5
elide: Text.ElideRight
color: Theme.textAlpha(0.72)
linkColor: root.tint
font.family: Theme.displayFont
font.pixelSize: 11
font.letterSpacing: 0.3
onLinkActivated: link => Qt.openUrlExternally(link)
}
}
}
Flow {
Layout.fillWidth: true
visible: root.chipActions.length > 0
spacing: 6
Repeater {
model: root.chipActions
Rectangle {
id: chip
required property var modelData
width: chipLabel.implicitWidth + 16
height: chipLabel.implicitHeight + 10
color: chipArea.containsMouse ? root.tint : "transparent"
border.width: 1
border.color: chipArea.containsMouse ? root.tint : Theme.disabled
Text {
id: chipLabel
anchors.centerIn: parent
text: chip.modelData.text.toUpperCase()
color: chipArea.containsMouse ? Theme.surface : Theme.text
font.family: Theme.microFont
font.pixelSize: 9
font.letterSpacing: 0.8
}
MouseArea {
id: chipArea
anchors.fill: parent
hoverEnabled: true
cursorShape: Qt.PointingHandCursor
onClicked: root.actionInvoked(chip.modelData.identifier)
}
}
}
}
// Expiry drain; a sticky card states it instead.
RowLayout {
Layout.fillWidth: true
spacing: 8
Row {
id: meter
readonly property int segments: 24
Layout.fillWidth: true
Layout.preferredHeight: 4
visible: root.timeout > 0
spacing: 2
Repeater {
model: meter.segments
Rectangle {
required property int index
readonly property bool lit: index < Math.ceil(root.remaining * meter.segments)
width: Math.max(2, (meter.width - (meter.segments - 1) * meter.spacing) / meter.segments)
height: meter.height
color: lit ? root.tint : Theme.textAlpha(0.06)
border.width: 1
border.color: lit ? root.tint : Theme.textAlpha(0.18)
}
}
}
MicroText {
visible: root.timeout > 0
text: hover.hovered ? "HOLD" : Math.ceil(root.remaining * root.timeout / 1000) + "S"
color: hover.hovered ? root.tint : Theme.muted
}
MicroText {
Layout.fillWidth: true
visible: root.timeout <= 0
text: root.critical ? "CRITICAL · STICKY" : "STICKY"
color: root.critical ? Theme.hot : Theme.muted
}
}
}
}
}
@@ -1,210 +0,0 @@
pragma ComponentBehavior: Bound
import QtQuick
import QtQuick.Shapes
import qs.HyprChrome.Theme
// Chamfered chrome for one notification card: PolkitPanel's silhouette, tighter
// metrics, a tintable accent (urgency) and a close box in the header.
//
// Not PolkitPanel itself: that file is the prompt's alone to restyle.
Item {
id: panel
property string panelId: ""
property string title: ""
property string meta: ""
property color accentColor: Theme.accent
property int chamfer: 12
property int padding: 12
property int outlineWidth: 1
property int accentLineThickness: 3
readonly property int headerHeight: 26
property int headerGap: 10
readonly property int headerPadding: 8
property real capGap: 4
readonly property real capOffset: panel.capGap / Math.SQRT2
readonly property real activeChamfer: Math.max(2, Math.min(panel.chamfer, panel.height / 2 - 1))
readonly property real accentLineWidth: Math.min(panel.width, slugChip.width + panel.headerPadding * 2)
default property alias content: body.data
signal closeRequested
implicitHeight: Math.round(body.y + body.height + panel.padding)
Shape {
id: panelShape
anchors.fill: parent
preferredRendererType: Shape.CurveRenderer
ShapePath {
fillColor: Theme.surface
strokeColor: Theme.hair
strokeWidth: panel.outlineWidth
startX: 0; startY: 0
PathLine { x: panelShape.width - panel.activeChamfer; y: 0 }
PathLine { x: panelShape.width; y: panel.activeChamfer }
PathLine { x: panelShape.width; y: panelShape.height }
PathLine { x: panel.activeChamfer; y: panelShape.height }
PathLine { x: 0; y: panelShape.height - panel.activeChamfer }
PathLine { x: 0; y: 0 }
}
ShapePath {
fillColor: panel.accentColor
strokeWidth: 0
startX: panelShape.width - panel.activeChamfer + panel.capOffset
startY: -panel.capOffset
PathLine { x: panelShape.width + panel.capOffset; y: panel.activeChamfer - panel.capOffset }
PathLine { x: panelShape.width + panel.capOffset; y: -panel.capOffset }
PathLine { x: panelShape.width - panel.activeChamfer + panel.capOffset; y: -panel.capOffset }
}
ShapePath {
fillColor: panel.accentColor
strokeWidth: 0
startX: panel.activeChamfer - panel.capOffset
startY: panelShape.height + panel.capOffset
PathLine { x: -panel.capOffset; y: panelShape.height - panel.activeChamfer + panel.capOffset }
PathLine { x: -panel.capOffset; y: panelShape.height + panel.capOffset }
PathLine { x: panel.activeChamfer - panel.capOffset; y: panelShape.height + panel.capOffset }
}
ShapePath {
fillColor: panel.accentColor
strokeWidth: 0
startX: 0; startY: 0
PathLine { x: panel.accentLineWidth; y: 0 }
PathLine { x: panel.accentLineWidth; y: panel.accentLineThickness }
PathLine { x: 0; y: panel.accentLineThickness }
PathLine { x: 0; y: 0 }
}
ShapePath {
fillColor: panel.accentColor
strokeWidth: 0
startX: panelShape.width; startY: panelShape.height
PathLine { x: panelShape.width - panel.accentLineWidth; y: panelShape.height }
PathLine { x: panelShape.width - panel.accentLineWidth; y: panelShape.height - panel.accentLineThickness }
PathLine { x: panelShape.width; y: panelShape.height - panel.accentLineThickness }
PathLine { x: panelShape.width; y: panelShape.height }
}
}
Row {
id: headerRow
x: panel.headerPadding
y: Math.round((panel.headerHeight - height) / 2)
width: headerEnd.x - x - 8
spacing: panel.headerPadding + 6
Rectangle {
id: slugChip
width: slugText.implicitWidth + 8
height: slugText.implicitHeight + 4
color: panel.accentColor
Text {
id: slugText
anchors.centerIn: parent
text: panel.panelId
color: Theme.surface
font.family: Theme.microFont
font.pixelSize: 10
font.bold: true
}
}
Text {
anchors.verticalCenter: slugChip.verticalCenter
width: Math.max(0, headerRow.width - slugChip.width - headerRow.spacing)
text: panel.title
color: Theme.text
font.family: Theme.displayFont
font.pixelSize: 11
font.bold: true
font.letterSpacing: 1.1
elide: Text.ElideRight
}
}
Row {
id: headerEnd
// Clear of the top-right chamfer.
anchors.right: parent.right
anchors.rightMargin: panel.activeChamfer + 2
y: Math.round((panel.headerHeight - height) / 2)
spacing: 8
Text {
anchors.verticalCenter: parent.verticalCenter
visible: panel.meta.length > 0
text: panel.meta
color: Theme.muted
font.family: Theme.microFont
font.pixelSize: 8
font.letterSpacing: 0.7
}
Rectangle {
id: closeBox
width: 16
height: 16
color: closeArea.containsMouse ? panel.accentColor : "transparent"
border.width: 1
border.color: closeArea.containsMouse ? panel.accentColor : Theme.disabled
Text {
anchors.centerIn: parent
text: "✕"
color: closeArea.containsMouse ? Theme.surface : Theme.muted
font.family: Theme.microFont
font.pixelSize: 9
font.bold: true
}
MouseArea {
id: closeArea
anchors.fill: parent
hoverEnabled: true
cursorShape: Qt.PointingHandCursor
onClicked: panel.closeRequested()
}
}
}
Rectangle {
x: 1
y: panel.headerHeight
width: parent.width - 2
height: 1
color: Theme.text
opacity: 0.12
}
// Measured by childrenRect: children carry their own size, no anchoring.
Item {
id: body
x: panel.padding
y: panel.headerHeight + panel.headerGap
width: Math.max(0, panel.width - panel.padding * 2)
height: childrenRect.height
}
}
@@ -1,131 +0,0 @@
pragma ComponentBehavior: Bound
import Quickshell
import Quickshell.Services.Notifications
import Quickshell.Wayland
import QtQuick
// Notification daemon and its popup stack, migrated from widgets/notifications.
// Owns the server and the surface; NotificationCard is the visual core.
Scope {
id: root
// Driven by the shell: the rail's monitor.
property var screen: null
// Arrival time per notification id; the server keeps none.
property var arrived: ({})
// expireTimeout is in ms (verified); -1 means "server default".
function timeoutFor(n) {
if (n.expireTimeout > 0)
return n.expireTimeout;
if (n.expireTimeout === 0 || n.urgency === NotificationUrgency.Critical)
return 0;
return 5000;
}
NotificationServer {
id: server
keepOnReload: true
bodySupported: true
bodyMarkupSupported: true
bodyHyperlinksSupported: true
imageSupported: true
actionsSupported: true
onNotification: notification => {
const id = notification.id;
root.arrived[id] = Qt.formatTime(new Date(), "hh:mm");
notification.closed.connect(() => delete root.arrived[id]);
notification.tracked = true;
}
}
PanelWindow {
id: win
screen: root.screen
visible: server.trackedNotifications.values.length > 0
WlrLayershell.namespace: "hyprchrome-notifications"
WlrLayershell.layer: WlrLayer.Overlay
// Normal with no zone of its own: sits below the rail's exclusive zone.
exclusionMode: ExclusionMode.Normal
color: "transparent"
anchors {
top: true
right: true
}
margins.top: 12
margins.right: 16
// Slack on every side for the corner caps, which sit outside each card.
implicitWidth: 360 + 8
implicitHeight: Math.max(1, stack.implicitHeight + 8)
mask: Region {
item: stack
}
Column {
id: stack
x: 4
y: 4
width: 360
spacing: 14
Repeater {
model: server.trackedNotifications
NotificationCard {
id: card
required property Notification modelData
width: stack.width
appName: card.modelData.appName
summary: card.modelData.summary
body: card.modelData.body
appIcon: card.modelData.appIcon
image: card.modelData.image
urgency: card.modelData.urgency
time: root.arrived[card.modelData.id] ?? ""
actions: card.modelData.actions
timeout: root.timeoutFor(card.modelData)
onDismissed: card.modelData.dismiss()
onExpired: card.modelData.expire()
// invoke() closes a non-resident notification itself.
onActionInvoked: identifier => {
const actions = card.modelData.actions;
for (let i = 0; i < actions.length; i++) {
if (actions[i].identifier === identifier) {
actions[i].invoke();
return;
}
}
}
// Slides in from the right edge.
opacity: 0
transform: Translate { id: slide; x: 40 }
Component.onCompleted: enter.start()
ParallelAnimation {
id: enter
NumberAnimation { target: card; property: "opacity"; to: 1; duration: 160; easing.type: Easing.OutCubic }
NumberAnimation { target: slide; property: "x"; to: 0; duration: 200; easing.type: Easing.OutCubic }
}
}
}
}
}
}
+3
View File
@@ -5,6 +5,7 @@ import Quickshell
import Quickshell.Widgets
import qs.widgets.bar
import qs.widgets.launcher
import qs.widgets.notifications
import qs.widgets.osd
import qs.widgets.systray
import qs.widgets.theme
@@ -35,6 +36,8 @@ Scope {
OrbitLauncher {} // 10 — radial targeting arena
CyberDock {} // 11 — cyberpunk bottom cartridge dock
Notifications {}
VolumeOsd {}
// Host vitals HUD — toggle with SUPER CTRL V.
@@ -1,90 +0,0 @@
import QtQuick
import qs.HyprChrome.Widgets.Bar.Luna
// The rail's luna panel in both densities and every daemon state. Collapsed
// is the left column (state only), expanded the right (messages) — the two
// things the panel has to get right, side by side.
//
// ./tools/quickshell-preview/render.sh \
// tests/LunaPanelHeadless.qml \
// .artifacts/quickshell-preview/luna-panel.png 640 560
Rectangle {
width: 640
height: 560
color: "#0f1012"
Row {
x: 16
y: 16
spacing: 24
Column {
spacing: 12
LunaPanel {
expanded: false
connected: true
active: false
daemonState: "idle"
}
LunaPanel {
expanded: false
connected: true
active: true
daemonState: "listening"
rms: 0.02
}
LunaPanel {
expanded: false
connected: true
active: true
daemonState: "thinking"
}
LunaPanel {
expanded: false
connected: true
active: true
daemonState: "speaking"
rms: 0.05
}
LunaPanel {
expanded: false
connected: false
}
}
Column {
spacing: 12
LunaPanel {
expanded: true
connected: true
active: false
daemonState: "idle"
}
LunaPanel {
expanded: true
connected: true
active: true
daemonState: "listening"
rms: 0.02
userText: "hey luna, what's the time?"
}
LunaPanel {
expanded: true
connected: true
active: true
daemonState: "speaking"
rms: 0.05
userText: "hey luna, what's the time?"
lunaText: "It's 23:40 on Sunday night, chief. Shouldn't you be asleep?"
}
}
}
}
@@ -1,58 +0,0 @@
import QtQuick
import qs.HyprChrome.Widgets.Notifications
// Offscreen render of the three urgencies, with actions, a link and a
// part-drained meter.
//
// ./tools/quickshell-preview/render.sh \
// tests/NotificationCardHeadless.qml \
// .artifacts/quickshell-preview/notifications.png 400 520
Rectangle {
width: 400
height: 520
color: "#1b1d20"
Column {
x: 20
y: 20
width: 360
spacing: 14
NotificationCard {
width: parent.width
appName: "Firefox"
summary: "Download complete"
body: "<b>nixos-26.05-x86_64.iso</b> finished — <a href=\"file:///tmp\">open folder</a>"
appIcon: "firefox"
time: "14:32"
actions: [
{ identifier: "default", text: "Open" },
{ identifier: "show", text: "Show in folder" },
{ identifier: "dismiss", text: "Dismiss" }
]
timeout: 5000
countdown: false
remaining: 0.6
}
NotificationCard {
width: parent.width
appName: "smartd"
summary: "Disk /dev/sdb reports 8 reallocated sectors"
body: "SMART attribute 5 crossed its threshold. Back up the RAID0 set on /mnt/data now."
urgency: 2
time: "14:30"
}
NotificationCard {
width: parent.width
appName: "nix"
summary: "Build finished"
urgency: 0
time: "14:28"
timeout: 5000
countdown: false
remaining: 0.2
}
}
}
@@ -0,0 +1,120 @@
import Quickshell.Services.Notifications
import Quickshell.Widgets
import QtQuick
import QtQuick.Layouts
import qs.widgets.theme
ColumnLayout {
id: root
required property Notification modelData
spacing: 0
Layout.fillWidth: true
Rectangle {
color: Theme.accent
implicitHeight: 5
Layout.fillWidth: true
}
WrapperRectangle {
Layout.fillWidth: true
margin: 12
border.width: 1
border.color: Theme.accent
color: Theme.surface
opacity: .9
ColumnLayout {
spacing: 4
Layout.fillWidth: true
RowLayout {
spacing: 8
Layout.fillWidth: true
IconImage {
visible: root.modelData.appIcon !== ""
source: root.modelData.appIcon
implicitSize: 20
}
Text {
text: root.modelData.summary
color: Theme.text
font.pointSize: 14
font.bold: true
elide: Text.ElideRight
Layout.fillWidth: true
}
Text {
id: dismissButton
text: "×"
color: Theme.accent
font.pointSize: 18
MouseArea {
anchors.fill: parent
cursorShape: Qt.PointingHandCursor
onClicked: root.modelData.dismiss()
}
}
}
Text {
visible: root.modelData.body !== ""
text: root.modelData.body
color: Theme.text
font.pointSize: 12
wrapMode: Text.Wrap
Layout.fillWidth: true
}
RowLayout {
visible: root.modelData.actions.length > 0
spacing: 6
Layout.fillWidth: true
Repeater {
model: root.modelData.actions
Rectangle {
id: actionButton
required property NotificationAction modelData
color: Theme.raised
border.width: 1
border.color: Theme.accent
implicitHeight: 28
implicitWidth: actionText.implicitWidth + 16
Text {
id: actionText
anchors.centerIn: parent
text: actionButton.modelData.text
color: Theme.text
font.pointSize: 12
}
MouseArea {
anchors.fill: parent
cursorShape: Qt.PointingHandCursor
onClicked: actionButton.modelData.invoke()
}
}
}
}
}
}
}
@@ -0,0 +1,79 @@
import Quickshell
import Quickshell.Services.Notifications
import QtQuick
import QtQuick.Layouts
Scope {
id: root
NotificationServer {
id: server
keepOnReload: true
bodySupported: true
bodyMarkupSupported: true
imageSupported: true
actionsSupported: true
actionIconsSupported: true
onNotification: notification => {
notification.tracked = true;
const timeout = notification.expireTimeout === 0 ? 0 : notification.expireTimeout > 0 ? notification.expireTimeout : (notification.urgency === NotificationUrgency.Critical ? 0 : 5000);
if (timeout <= 0)
return;
let closed = false;
notification.closed.connect(() => {
closed = true;
});
const timer = expiryTimerComponent.createObject(root, {
interval: timeout
});
timer.triggered.connect(() => {
if (!closed)
notification.expire();
timer.destroy();
});
}
}
Component {
id: expiryTimerComponent
Timer {
running: true
}
}
PanelWindow {
screen: Quickshell.screens.find(s => s.name === "DP-2")
color: "transparent"
anchors {
top: true
right: true
}
margins.top: 12
margins.right: 12
implicitWidth: 340
implicitHeight: column.implicitHeight
ColumnLayout {
id: column
anchors.fill: parent
spacing: 8
Repeater {
model: server.trackedNotifications
NotificationPopup {}
}
}
}
}
Generated
+33 -33
View File
@@ -14,11 +14,11 @@
"uv2nix": "uv2nix"
},
"locked": {
"lastModified": 1789888175,
"narHash": "sha256-BWzpZ0/HChcL8F2+hl0GoPaIsWBprMrReeOfrhLSk5o=",
"lastModified": 1789662015,
"narHash": "sha256-CveW/4U/zNIRfLpH6vMv8qisfotft8XJC47KqAhViZ0=",
"owner": "nix-community",
"repo": "authentik-nix",
"rev": "477392e1a239228de9a8011786cf673dfbb78c75",
"rev": "9a176c0a1889921f3ed19d29a047bafaf20b9a24",
"type": "github"
},
"original": {
@@ -30,16 +30,16 @@
"authentik-src": {
"flake": false,
"locked": {
"lastModified": 1789678329,
"narHash": "sha256-wt6cjarJI5NXSHgpONjqE9JiltBqkCoPyLLiIS83kuI=",
"lastModified": 1788978241,
"narHash": "sha256-tAdDYHIur6ewVZlRz6JPtLivqctYLLtD0S8uehuuxSM=",
"owner": "goauthentik",
"repo": "authentik",
"rev": "e5a0d2f7572cb776eee7a3e9355937ce38973761",
"rev": "dce85a5b64a429206199e9ffddc602060643f17a",
"type": "github"
},
"original": {
"owner": "goauthentik",
"ref": "version/2026.8.3",
"ref": "version/2026.8.2",
"repo": "authentik",
"type": "github"
}
@@ -67,11 +67,11 @@
]
},
"locked": {
"lastModified": 1789770686,
"narHash": "sha256-uZkBR7yHdIKUFB5SZdfgh1qkGfI3XmYmI/lTiquxbck=",
"lastModified": 1781152676,
"narHash": "sha256-RxWs5ND31KzTG7wvMM+PMfUjyNpmIEr999lqNARaM5o=",
"owner": "nix-community",
"repo": "disko",
"rev": "725ea35e410ad83be4931d1bff7e090eacaf3563",
"rev": "ff8702b4de27f72b4c78573dfb89ec74e36abdf1",
"type": "github"
},
"original": {
@@ -255,11 +255,11 @@
]
},
"locked": {
"lastModified": 1789357259,
"narHash": "sha256-YF2EwlbEEmLgEH0fs7H5sgz9JwwMjoKM2rY5jnGNYwE=",
"lastModified": 1786747096,
"narHash": "sha256-9QqhmaLVsPhKdMSBaWKjDqeGRn8G4ov4cVuZ6JFwXbo=",
"owner": "numtide",
"repo": "nix-vm-test",
"rev": "91825d66623ff030255d98082b5c8af2bbc549ec",
"rev": "c8781a0ea2d8417506fff7722eae5a6316461212",
"type": "github"
},
"original": {
@@ -286,11 +286,11 @@
"treefmt-nix": "treefmt-nix"
},
"locked": {
"lastModified": 1790148129,
"narHash": "sha256-oOp+FNzqKzFfn9+AwcvzBedUyuUniv7eGxQ0jfzFpQ8=",
"lastModified": 1789567195,
"narHash": "sha256-ymvPUaBvMmGvxsPbmrocpxO4XK+H+K12YjgKIyG3Rwk=",
"owner": "nix-community",
"repo": "nixos-anywhere",
"rev": "da83557d8b0bce57a52371b888ffd6d58724e55c",
"rev": "1c2f124e970fed2a49bd14ce0a8b4e9bff74d3b4",
"type": "github"
},
"original": {
@@ -339,11 +339,11 @@
},
"nixpkgs": {
"locked": {
"lastModified": 1789546076,
"narHash": "sha256-zVxLZiSnmaaPLwnhj7pwmqe3axBg/C6nG5JZsJMh2g4=",
"lastModified": 1788316716,
"narHash": "sha256-bc7rSpXIdn9QWGNqfWcPZWOhEVF8NoeAZkWq0XWnf/k=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "b1b875982b17dabde9b4a37f3e229e74913e6db3",
"rev": "3ed67ec0a4d3c7ab4ae1f04f8ee8df07bfa506a2",
"type": "github"
},
"original": {
@@ -370,11 +370,11 @@
},
"nixpkgs-unstable": {
"locked": {
"lastModified": 1790149054,
"narHash": "sha256-FfjXg6mAzKQBCY+7V9keGWOXG4DV2Eym/qA987lFl8A=",
"lastModified": 1789632929,
"narHash": "sha256-RjR8AyvGlWuw16XRj7C1YDEw4E27ciuyPeB2nDMzgTU=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "8825bebf6324e0579d012936eff73379af284b6d",
"rev": "a32edd7654519351e48e80372a928df336394670",
"type": "github"
},
"original": {
@@ -386,11 +386,11 @@
},
"nixpkgs_2": {
"locked": {
"lastModified": 1790113641,
"narHash": "sha256-D2aaQetafdpHxbwA4ldAxs51UX0b11pXL+JL/5L8VrM=",
"lastModified": 1789654592,
"narHash": "sha256-vrwAiXmz+0hs/IWXZalFG/Ws6xQRcW+gClUDlfvcfjQ=",
"owner": "NixOS",
"repo": "nixpkgs",
"rev": "1bc55b9def8165e82073919945c3239903fe4dc2",
"rev": "ecc58f32d1069a3e3f8a991e2233d365689f748d",
"type": "github"
},
"original": {
@@ -524,11 +524,11 @@
]
},
"locked": {
"lastModified": 1789890976,
"narHash": "sha256-GKwH3zpy7tartuJMG0Rv/xUsdetG1QLmTVv8UKgJLmA=",
"lastModified": 1789691124,
"narHash": "sha256-k+I+R6uwHX3VcJ7326qLV6vCahZUgsVl+i8sSU/Stxk=",
"owner": "Mic92",
"repo": "sops-nix",
"rev": "7214124c20c1542c90deb54af50e2f53ae02711f",
"rev": "1e73e8f7176d65e1b55e324de099bbfff4b2c574",
"type": "github"
},
"original": {
@@ -539,16 +539,16 @@
},
"systems": {
"locked": {
"lastModified": 1680978846,
"narHash": "sha256-Gtqg8b/v49BFDpDetjclCYXm8mAnTrUzR0JnE2nv5aw=",
"lastModified": 1689347949,
"narHash": "sha256-12tWmuL2zgBgZkdoB6qXZsgJEH9LR3oUgpaQq2RbI80=",
"owner": "nix-systems",
"repo": "x86_64-linux",
"rev": "2ecfcac5e15790ba6ce360ceccddb15ad16d08a8",
"repo": "default-linux",
"rev": "31732fcf5e8fea42e59c2488ad31a0e651500f68",
"type": "github"
},
"original": {
"owner": "nix-systems",
"repo": "x86_64-linux",
"repo": "default-linux",
"type": "github"
}
},
-1
View File
@@ -24,7 +24,6 @@
../../services/media/seerr.nix
../../services/media/immich.nix
../../services/dev/gitea.nix
../../services/dev/gitea-mcp.nix
../../services/dev/obsidian-livesync.nix
];
-22
View File
@@ -47,30 +47,8 @@
# the container with it — confirmed the hard way (~60-70s per start).
"x-systemd.mount-timeout=10s"
"_netdev"
# Hermes' RequiresMountsFor mounts this at boot, not lazily, and nothing
# retries a failed mount.
"x-systemd.requires=jupiter-reachable.service"
];
};
# jupiter resolves to its tailnet IP, and network-online.target fires on the
# LAN link before tailscaled has routed tailscale0. 5min covers jupiter
# cold-booting alongside mars.
systemd.services.jupiter-reachable = {
description = "Wait for jupiter's samba over the tailnet";
after = [ "network-online.target" "tailscaled.service" ];
wants = [ "network-online.target" ];
serviceConfig = {
Type = "oneshot";
RemainAfterExit = true;
TimeoutStartSec = 300;
};
script = ''
until timeout 2 ${pkgs.bash}/bin/bash -c 'exec 3<>/dev/tcp/jupiter/445' 2>/dev/null; do
sleep 2
done
'';
};
system.stateVersion = "26.05"; # set at install time; do NOT bump on upgrades
}
+106 -252
View File
@@ -52,10 +52,6 @@ let
# is hers to make, anywhere inside HERMES_WRITE_SAFE_ROOT=/opt/data.
giteaHost = "git.mgaction.town";
# neptun's tailnet IP: its Caddy is the only proxy whose X-Forwarded-Proto
# the dashboard should trust.
neptunTailnetIp = "100.64.0.5";
# luna's webhook filters, mounted READ-ONLY from the nix store rather than
# written into hermesHome: that IS her write-safe root, so a writable copy
# would let her edit her own loop guard back out. A missing script fails
@@ -78,7 +74,11 @@ let
builtins.readFile ./gitea-pr-review-prompt.md
);
# Mnemosyne memory provider: third-party plugin, not in the image.
# Mnemosyne memory provider (local SQLite, third-party plugin — not bundled
# with the official image). Fully built as a Nix derivation — see
# pkgs/mnemosyne-env.nix — and mounted READ-ONLY into the container at a
# fixed path. The oneshot near the bottom of this file only writes the
# plugin symlink Docker needs at $HERMES_HOME/plugins/mnemosyne.
mnemosyneEnv = pkgs.callPackage ../../pkgs/mnemosyne-env.nix { };
# KittenTTS voice provider inputs (CPU-only; model + wheel hash-pinned).
@@ -89,33 +89,22 @@ let
url = "https://github.com/KittenML/KittenTTS/releases/download/0.8.1/kittentts-0.8.1-py3-none-any.whl";
sha256 = "sha256-SCpDbE8fMZIVNxA3bkWf82iVF+vNp8KwUeL9QYe0GFE=";
};
# uv parses the version out of the filename; the store hash prefix breaks it.
kittenttsWheelFile = "${pkgs.linkFarm "kittentts-wheel" {
"kittentts-0.8.1-py3-none-any.whl" = kittenttsWheel;
}}/kittentts-0.8.1-py3-none-any.whl";
kittenttsRuntimeLibs = "${pkgs.stdenv.cc.cc.lib}/lib ${pkgs.zlib}/lib";
kittenttsReqs = pkgs.writeText "kittentts-requirements.txt" (
builtins.readFile ./kittentts/requirements.txt
);
# STT add-on: faster-whisper (CPU) for Telegram voice-note transcription,
# same side-venv, appended by the provisioning unit. Keeps one venv + one
# stamp for the whole local-audio stack. See kittentts/stt-requirements.txt.
kittenttsSttReqs = pkgs.writeText "kittentts-stt-requirements.txt" (
builtins.readFile ./kittentts/stt-requirements.txt
);
kittenttsStub = pkgs.writeText "kitten-tts-stub.py" (
builtins.readFile ./kittentts/kitten-misaki-stub.py
);
kittenttsModelOnnx = pkgs.fetchurl {
url = "https://huggingface.co/KittenML/kitten-tts-mini-0.8/resolve/c02725660cea441db4c383af69f1f26f5cd00947/kitten_tts_mini_v0_8.onnx";
url = "https://huggingface.co/KittenML/kitten-tts-mini-0.8/resolve/main/kitten_tts_mini_v0_8.onnx";
sha256 = "sha256-D1u65PxIAMmNvFRKh+z6eVEN4vuCItsw0S5b/pF335E=";
};
kittenttsModelVoices = pkgs.fetchurl {
url = "https://huggingface.co/KittenML/kitten-tts-mini-0.8/resolve/c02725660cea441db4c383af69f1f26f5cd00947/voices.npz";
url = "https://huggingface.co/KittenML/kitten-tts-mini-0.8/resolve/main/voices.npz";
sha256 = "sha256-QK0mOJUrd7ey8wEn4mCOFp/GndJWtTvYqqNAmjMZPEI=";
};
kittenttsModelConfig = pkgs.fetchurl {
url = "https://huggingface.co/KittenML/kitten-tts-mini-0.8/resolve/c02725660cea441db4c383af69f1f26f5cd00947/config.json";
url = "https://huggingface.co/KittenML/kitten-tts-mini-0.8/resolve/main/config.json";
sha256 = "sha256-axYLybGeJOyyHoS8FPin2iH99H7HLUJFC8XPUUthgEo=";
};
@@ -181,7 +170,7 @@ in
wants = [ "network-online.target" ];
after = [ "network-online.target" ];
unitConfig.RequiresMountsFor = [ "/mnt/jupiter" ];
path = [ pkgs.git pkgs.tea pkgs.curl pkgs.coreutils pkgs.yq-go pkgs.gnugrep pkgs.util-linux ];
path = [ pkgs.git pkgs.tea pkgs.curl pkgs.coreutils ];
serviceConfig.Type = "oneshot";
# Everything here is either local or bounded to ~30s by the probe loop, so
# anything past two minutes is a hang, not slowness.
@@ -252,49 +241,6 @@ in
# scripts as uid ${hermesUid}.
chown ${hermesUid}:${hermesGid} ${hermesHome}/scripts
# trusted_proxies has no env override and Hermes owns config.yaml, so
# only this key is merged in; without it the Secure cookie flag breaks.
conf=${hermesHome}/config.yaml
[ -e "$conf" ] || : > "$conf"
# tts.provider is seeded, not pinned: `=` would stomp luna's own choice on
# every restart, and the engine is hers to change (KittenTTS stays as the
# offline fallback). `//` only fills it in when unset.
yq '.dashboard.trusted_proxies = ["${neptunTailnetIp}"]
| .tts.provider = (.tts.provider // "elevenlabs")' "$conf" > "$conf.new"
chown ${hermesUid}:${hermesGid} "$conf.new"
chmod --reference="$conf" "$conf.new"
mv -f "$conf.new" "$conf"
# The image's s6 init hands the gateway a fixed set of container env vars,
# and ELEVENLABS_API_KEY isn't among them (OPENCODE_GO_API_KEY is) — so the
# key only reaches Hermes via .env, which load_env() reads directly.
# Re-asserted every start: a sops rotation must win over a stale line.
# setpriv because root must not write into hermesHome — luna could plant a
# symlink there. The key arrives on an fd opened while still root, so it
# never lands in argv or the journal and her uid needs no /run/secrets access.
#
# API_SERVER_KEY is in the same boat, confirmed the same way: with it set
# only in the container env the server came up (so API_SERVER_ENABLED/HOST
# DID pass through) but answered every bearer token with
# "Invalid gateway API key" — the passthrough list carries the plain
# API_SERVER_* vars and drops the secret one.
write_env_key() {
setpriv --reuid=${hermesUid} --regid=${hermesGid} --clear-groups \
${pkgs.runtimeShell} -c '
set -eu
envf=$1
name=$2
umask 077
key=$(cat)
[ -e "$envf" ] || : > "$envf"
{ grep -v "^$name=" "$envf" || true
printf "%s=%s\n" "$name" "$key"; } > "$envf.new"
mv -f "$envf.new" "$envf"
' _ ${hermesHome}/.env "$1"
}
write_env_key ELEVENLABS_API_KEY < ${config.sops.secrets.elevenlabs_api_key.path}
write_env_key API_SERVER_KEY < ${config.sops.secrets.hermes_api_server_key.path}
if [ -d ${hermesHome}/.config ]; then
chown ${hermesUid}:${hermesGid} ${hermesHome}/.config
fi
@@ -316,6 +262,15 @@ in
"${hermesHome}:/opt/data"
"${dropboxDir}:/opt/data/dropbox"
# Mnemosyne memory provider — a Nix-built python env, mounted :ro.
# Nothing fetched at boot, nothing writable from inside the container.
# The plugin symlink the oneshot at the bottom of this file writes
# points at the canonical store path (site-packages passthru), which
# is visible inside thanks to the existing /nix/store ro mount, so
# this /opt/data restatement is a readability alias, not a load
# bearing path.
"${mnemosyneEnv}:/opt/data/mnemosyne-env:ro"
# luna's Obsidian vault, synced with CouchDB on jupiter by
# livesync-bridge.nix. Under /opt/data so she can write notes, not just
# read them; the bridge runs as this same uid/gid so no chown is needed.
@@ -353,21 +308,12 @@ in
HERMES_DASHBOARD_HOST = "0.0.0.0"; # must be tailscale0-reachable, not just loopback
HERMES_DASHBOARD_OIDC_ISSUER = "https://auth.mgaction.town/application/o/hermes/";
HERMES_DASHBOARD_OIDC_CLIENT_ID = "4BqdJu3htnMtSZnyEu5zHnsSOvlEbw3Ie3mYVlh6";
# Pins the OIDC redirect_uri to https; otherwise it is rebuilt from
# forwarded headers (see trusted_proxies in prepare-dirs).
HERMES_DASHBOARD_PUBLIC_URL = "https://hermes.mgaction.town";
# KittenTTS: point huggingface_hub at the pre-seeded offline cache and
# forbid network — no model drift, no boot-time fetch (review #4).
HF_HOME = "/opt/data/kittentts-hf";
HF_HUB_OFFLINE = "1";
# STT (local whisper): expose the side-venv's site-packages to Hermes's
# own interpreter so faster-whisper imports in-process. PYTHONPATH dirs
# don't run .pth auto-imports, but faster-whisper has no dead imports —
# it imports cleanly from a plain path injection (unlike kittentts,
# which is driven through the command wrapper for exactly that reason).
PYTHONPATH = "/opt/data/kittentts-venv/lib/python3.13/site-packages";
# uvicorn only trusts X-Forwarded-Proto from forwarded_allow_ips
# (default 127.0.0.1); neptun's Caddy reaches this over a real routed
# tailnet IP, so without this it builds an http:// redirect_uri that
# Authentik rejects. Safe to trust any peer: 9119 is already scoped to
# loopback + tailscale0 only.
FORWARDED_ALLOW_IPS = "*";
};
environmentFiles = [ config.sops.templates."hermes-agent.env".path ];
cmd = [ "gateway" "run" ];
@@ -380,10 +326,6 @@ in
];
requires = [ "hermes-agent-prepare-dirs.service" ];
unitConfig.RequiresMountsFor = [ "/mnt/jupiter" ];
# prepare-dirs is an inactive oneshot, and switch-to-configuration only
# restarts ACTIVE units — so editing its script otherwise lands nowhere
# until something else happens to restart the container.
restartTriggers = [ config.systemd.services.hermes-agent-prepare-dirs.script ];
};
# The two Gitea webhook routes, written as config (not via `hermes webhook
@@ -520,9 +462,26 @@ in
'';
};
# Hermes discovers memory providers under $HERMES_HOME/plugins; the target
# is a store path, readable in the container via the /nix/store ro mount.
# wantedBy, not requiredBy: a failure leaves Hermes on built-in memory.
# ---- Mnemosyne memory provider ----------------------------------------
# The provider's Python closure (mnemosyneEnv, callPackage'd
# pkgs/mnemosyne-env.nix above) is a READ-ONLY nix store path mounted into
# the container — nothing fetched at boot, nothing inside the container can
# write to it, and the mono-repo reproducibility story applies. Hermes only
# needs one mutable pickup point: $HERMES_HOME/plugins/mnemosyne, the
# symlink its discovery scan looks for. Both the plugin wrapper and its
# sibling `mnemosyne` core package live in that env's single site-packages,
# so one link covers them.
#
# Path handling: the symlink target is spelled in the CONTAINER's path
# space (/opt/data/...), because Hermes resolves the plugin from inside the
# container — the same host/container mismatch the webhook prompts already
# navigate via containerHome. Host-side the identical literal resolves
# onto the same store path through hermesHome's bind mount.
#
# Failure posture: after=, not requires= — a failed link write leaves the
# container running with whatever memory.provider falls back to Hermes's
# built-in memory, not a dead bot. The ro mount itself is evaluated at
# build time, so there is nothing provisionable to drift at runtime.
systemd.services.hermes-agent-mnemosyne-plugin = {
description = "Link Mnemosyne provider into the Hermes plugin dir";
before = [ "podman-hermes-agent.service" ];
@@ -536,13 +495,27 @@ in
};
script = ''
set -euo pipefail
pluginsDir=${hermesHome}/plugins
pluginDir=$pluginsDir/mnemosyne
target=${mnemosyneEnv}/${mnemosyneEnv.sitePackages}/hermes_memory_provider
[ -d "$target" ] || { echo "$target missing" >&2; exit 1; }
# Target is the STORE path itself, not a /opt/data restatement: the
# container already ro-mounts /nix/store for git/tea (see the volumes
# list), so the identical literal resolves on both sides of the bind
# mount. Using the canonical store path directly — not the
# /opt/data/mnemosyne-env mount — keeps one truth and still works
# whether Hermes resolves the link inside the container or host-side
# during debugging.
target="${mnemosyneEnv.sitePackages}/hermes_memory_provider"
if [ ! -d "$target" ]; then
echo "hermes_memory_provider not found in the mnemosyne env — unit bug, not transient" >&2
exit 1
fi
mkdir -p "$pluginsDir"
chown ${hermesUid}:${hermesGid} "$pluginsDir"
# Atomic swap: write to a temp name, rename over the old link. `-T`
# errors loudly if the target turned into a directory for any reason.
ln -sfn "$target" "$pluginDir.new"
mv -Tf "$pluginDir.new" "$pluginDir"
chown -h ${hermesUid}:${hermesGid} "$pluginDir"
@@ -557,51 +530,27 @@ in
# .pth-registered stub (kitten-misaki-stub.py) that fails loudly if misaki
# is ever actually used.
#
# Provisioner invariants (shaped by the Mnemosyne + KittenTTS review rounds):
# - Build in a ROOT-OWNED staging dir (/var/lib/hermes-kittentts, 0755 so
# the uid-986 delivery step can read it; nothing in it is secret):
# stamp, staging venv, uv cache and staging model copies all live there.
# Root never reads or executes anything the container can write — the
# container cannot symlink-takeover the stamp (review #1) or plant a
# wheel in the uv cache (review #2); only FINISHED artifacts are copied
# into hermesHome, and the stamp compares the delivered copy against
# staging byte-for-byte.
# - Root runs python ONLY from the staging venv (never from the delivered
# uid-986-owned tree in hermesHome) — after a root-side offline import
# check; a failing build aborts before anything is delivered.
# - HF model cache delivered to hermesHome from staging (refs/main ->
# snapshots/<real commit sha>), and the container env pins
# HF_HOME=/opt/data/kittentts-hf + HF_HUB_OFFLINE=1: zero boot-time
# network, no drift.
# - Idempotency stamp keyed on the FULL input set (requirements + wheel +
# stub + model files), not just requirements (review #5); a deleted
# delivered file falls out of the fast path and re-provisions cheaply.
# - Order after prepare-dirs (review #6) — mirrors the mnemosyne unit.
# Provisioner invariants (shaped by the Mnemosyne-round review):
# - `uv venv --clear`: re-provision cannot wedge on an existing dir.
# - Root runs NO python from this venv: the unit itself does only fs
# writes; the venv is chowned to the container uid before Hermes ever
# imports from it. (Hermes executes provider code as uid 986.)
# - HF model cache is PRE-SEEDED from hash-pinned store paths, so
# HF_HUB_OFFLINE=1 gives zero boot-time network and no drift.
# - Idempotent via a stamp keyed on the requirements hash; checked
# against BOTH the venv python and model files being intact.
#
# Trust boundary: the DELIVERED venv lives inside hermesHome
# (HERMES_WRITE_SAFE_ROOT), so luna can alter her own TTS engine — and a
# deleted/modified copy just triggers a fresh delivery from the root-owned
# staging area on next boot (self-heals instead of wedging). Deliberate: it's
# her voice, not her jail — the webhook filter scripts remain the only
# Trust boundary: the venv lives inside hermesHome (HERMES_WRITE_SAFE_ROOT),
# so luna can technically alter her own TTS engine. Deliberate: it's her
# voice, not her jail — the webhook filter scripts remain the only
# write-protected-but-load-bearing items.
systemd.services.hermes-agent-kittentts-provision = {
description = "Provision KittenTTS voice provider (side venv + offline HF cache)";
before = [ "podman-hermes-agent.service" ];
wantedBy = [ "podman-hermes-agent.service" ];
wants = [ "network-online.target" ];
# after prepare-dirs (review #6): on a fresh state dir this unit must not
# create hermesHome root-owned before prepare-dirs sets the ownership
# layout — same ordering contract the mnemosyne unit has. PLUS
# network-online ordering (review2 #3): this unit CAN download at boot
# (unlike mnemosyne's store-path build), so uv must not run before the
# network is actually up.
after = [
"network-online.target"
"hermes-agent-prepare-dirs.service"
];
requires = [ "hermes-agent-prepare-dirs.service" ];
# diffutils: `cmp` in the fast path; missing, it silently re-delivers every boot.
path = [ pkgs.uv pkgs.coreutils pkgs.diffutils pkgs.util-linux pkgs.auto-patchelf pkgs.patchelf ];
after = [ "network-online.target" ];
path = [ pkgs.uv pkgs.coreutils ];
serviceConfig = {
Type = "oneshot";
TimeoutStartSec = 600;
@@ -611,146 +560,51 @@ in
venv=${hermesHome}/kittentts-venv
hubDir=${hermesHome}/kittentts-hf/hub/models--KittenML--kitten-tts-mini-0.8
# Real upstream commit SHA as snapshot dir: hf_hub_download resolves
# refs/main -> snapshots/<sha>; "kitten" (review #3) is never found
# offline and silently triggers a re-download.
modelSha=c02725660cea441db4c383af69f1f26f5cd00947
snap=$hubDir/snapshots/$modelSha
# REVIEW #1/#2: nothing root touches lives in hermesHome. Stamp, staging
# venv and uv cache live root-owned under /var/lib/hermes-kittentts (a
# path the container can not pathwrite or symlinks into its own tree);
# the FINISHED staging venv and model files are the only things copied
# into hermesHome, and only after being validated. Root doesn't follow
# any uid-986-writable path while running as root.
stageDir=/var/lib/hermes-kittentts
# Root-side staging of the DELIVERED TREE (venv + full HF hub layout
# including refs/main) — everything root writes lives here.
# review2 #4: root never writes into hermesHome; delivery happens as
# the container uid via setpriv, copying from these root-owned sources.
stageVenv=$stageDir/venv
stageHub=$stageDir/kittentts-hf
stageSnap=$stageDir/hf-model
# Input key: requirements + wheel + stub + model files + resolved script.
# Review #5 — a miss on the old requirements-only stamp let a changed
# wheel/stub/model URL keep running the stale install forever.
inputHash=$({ cat ${kittenttsReqs} ${kittenttsSttReqs} ${kittenttsWheel} ${kittenttsStub} \
${kittenttsModelOnnx} ${kittenttsModelVoices} ${kittenttsModelConfig}
echo ${kittenttsRuntimeLibs}; } | sha256sum | cut -d' ' -f1)
stampFile=$stageDir/provision.stamp
snap=$hubDir/snapshots/kitten
stampFile=${hermesHome}/kittentts-provision.stamp
reqHash=$(sha256sum ${kittenttsReqs} | cut -d' ' -f1)
# Idempotent early exit: stamp matches the full input hash, staging venv
# validated, delivered copy intact check runs below.
if [ -f "$stampFile" ] && [ "$(cat "$stampFile")" = "$inputHash" ] \
&& [ -x "$stageVenv/bin/python" ] \
&& [ -f "$stageSnap/config.json" ]; then
# Delivered artifacts in hermesHome must ALSO match the staging copy:
# luna can rewrite her copy, that's fine — but then the missing file
# forces a re-provision (cheap copy, not a rebuild) so deletions
# cannot wedge the gateway without a voice.
if [ -x "$venv/bin/python" ] \
&& cmp -s "$stageSnap/config.json" "$snap/config.json" 2>/dev/null \
&& cmp -s "$stageSnap/kitten_tts_mini_v0_8.onnx" "$snap/kitten_tts_mini_v0_8.onnx" 2>/dev/null \
&& cmp -s "$stageSnap/voices.npz" "$snap/voices.npz" 2>/dev/null; then
# Idempotent early exit: stamp + venv + all three model files intact.
if [ -f "$stampFile" ] && [ "$(cat "$stampFile")" = "$reqHash" ] \
&& [ -x "$venv/bin/python" ] \
&& [ -f "$snap/kitten_tts_mini_v0_8.onnx" ] \
&& [ -f "$snap/voices.npz" ] \
&& [ -f "$venv/lib/python3.13/site-packages/kitten_tts_stub.py" ]; then
exit 0
fi
fi
# ---- staging venv + staged hub tree: root-owned path, uv cache
# included. Root runs python from HERE (container can write nothing in
# this tree); the FINISHED result is copied into hermesHome AS THE
# CONTAINER USER via setpriv (review2 #4) — root never writes into
# hermesHome, so no uid-986-controlled path is ever followed while
# running as root; symlink-takeover of stamps/refs/install targets is
# structurally impossible. ----
mkdir -p "$stageDir" "$stageSnap" "$stageVenv" "$stageHub"
# 0755: the setpriv'd uid-986 delivery below must be able to read it.
chmod 0755 "$stageDir"
install -m 0444 ${kittenttsModelConfig} "$stageSnap/config.json"
install -m 0444 ${kittenttsModelOnnx} "$stageSnap/kitten_tts_mini_v0_8.onnx"
install -m 0444 ${kittenttsModelVoices} "$stageSnap/voices.npz"
# Skip the venv REBUILD when staging is still valid (review2 minor:
# damaged delivery should be a copy, not a rebuild) — but only for the
# CURRENT inputs, else a bump delivers the old venv under a new stamp.
if [ "$(cat "$stampFile" 2>/dev/null)" != "$inputHash" ] \
|| ! [ -x "$stageVenv/bin/python" ] \
|| ! [ -f "$stageVenv/lib/python3.13/site-packages/kitten_tts_stub.py" ]; then
rm -rf "$stageVenv"
UV_CACHE_DIR=$stageDir/uv-cache \
uv venv "$stageVenv" --python ${pkgs.python313}/bin/python3 --quiet
UV_CACHE_DIR=$stageDir/uv-cache \
uv pip install --python "$stageVenv/bin/python" --quiet \
# Venv (rebuilt rather than broken on --no-clear: uv exits 2 otherwise).
uv venv "$venv" --python ${pkgs.python313}/bin/python3 --clear --quiet
UV_CACHE_DIR=${hermesHome}/kittentts-uv-cache \
uv pip install --python "$venv/bin/python" --quiet \
--requirement ${kittenttsReqs}
# kittentts --no-deps: its overlay of spacy/misaki[en] is what drags in
# the CUDA tree; the requirements freeze already covers its real needs.
UV_CACHE_DIR=$stageDir/uv-cache \
uv pip install --python "$stageVenv/bin/python" --quiet --no-deps \
${kittenttsWheelFile}
UV_CACHE_DIR=${hermesHome}/kittentts-uv-cache \
uv pip install --python "$venv/bin/python" --quiet --no-deps \
${kittenttsWheel}
# Dead-import shim: .pth auto-loads kitten_tts_stub at interpreter
# start so `from misaki import en, espeak` resolves without the real
# misaki.en.
siteDir=$("$stageVenv/bin/python" -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')
# Dead-import shim: .pth auto-loads kitten_tts_stub at interpreter start
# so `from misaki import en, espeak` resolves without the real misaki.en.
siteDir=$("$venv/bin/python" -c 'import sysconfig; print(sysconfig.get_paths()["purelib"])')
cp ${kittenttsStub} "$siteDir/kitten_tts_stub.py"
printf 'import kitten_tts_stub\n' > "$siteDir/zz-kitten-stub.pth"
# PyPI manylinux .so's (numpy, onnxruntime) need libstdc++, which the
# Nix loader never finds, on the host or in the container.
auto-patchelf --paths "$stageVenv" --libs ${kittenttsRuntimeLibs}
# Seed the HF cache with the hashed model files (exact hub layout;
# hf_hub_download scans refs/snapshots on disk offline).
mkdir -p "$hubDir/refs" "$snap"
install -m 0444 ${kittenttsModelOnnx} "$snap/kitten_tts_mini_v0_8.onnx"
install -m 0444 ${kittenttsModelVoices} "$snap/voices.npz"
install -m 0444 ${kittenttsModelConfig} "$snap/config.json"
# STT add-on: faster-whisper into the SAME venv (shares the
# numpy/onnxruntime pins; see kittentts/stt-requirements.txt).
UV_CACHE_DIR=$stageDir/uv-cache \
uv pip install --python "$stageVenv/bin/python" --quiet \
--requirement ${kittenttsSttReqs}
# av's bundled ffmpeg libs also need the stdc++/zlib link (already
# covered by the auto-patchelf pass above — rerun it so the newly
# installed ctranslate2/av binaries get RPATHs too).
auto-patchelf --paths "$stageVenv" --libs ${kittenttsRuntimeLibs}
fi
# Hand ownership to the container uid BEFORE any python runs in this
# tree (root never imports from it — that was Mnemosyne review #3).
chown -R ${hermesUid}:${hermesGid} "$venv" "$hubDir"
# Stage the full delivered hub tree (exact hf_hub_download layout:
# refs/main -> snapshots/<sha>; review1 #3) under root-owned staging.
stageModelDir=$stageHub/hub/models--KittenML--kitten-tts-mini-0.8
mkdir -p "$stageModelDir/refs" "$stageModelDir/snapshots/$modelSha"
printf '%s' "$modelSha" > "$stageModelDir/refs/main"
install -m 0444 ${kittenttsModelOnnx} "$stageModelDir/snapshots/$modelSha/kitten_tts_mini_v0_8.onnx"
install -m 0444 ${kittenttsModelVoices} "$stageModelDir/snapshots/$modelSha/voices.npz"
install -m 0444 ${kittenttsModelConfig} "$stageModelDir/snapshots/$modelSha/config.json"
# No blobs/ indirection: kittentts reads paths RETURNED by
# hf_hub_download, which serves the resolved snapshot file directly
# (prefer-dir layout works offline for fully-materialized files).
# Root-side sanity: the staged interpreter must construct the model
# END-TO-END offline (review2 minor — import alone doesn't exercise
# hf_hub_download; a broken cache layout must fail HERE, not in the
# gateway). Points HF_HOME at the staged hub tree itself.
HF_HOME=$stageHub \
HF_HUB_OFFLINE=1 \
PHONEMIZER_ESPEAK_LIBRARY="$("$stageVenv/bin/python" -c 'import espeakng_loader,pathlib;print(pathlib.Path(espeakng_loader.get_library_path()))')" \
PHONEMIZER_ESPEAK_DATA_PATH="$("$stageVenv/bin/python" -c 'import espeakng_loader,pathlib;print(pathlib.Path(espeakng_loader.get_data_path()))')" \
"$stageVenv/bin/python" -c 'from kittentts import KittenTTS; KittenTTS("KittenML/kitten-tts-mini-0.8"); print("kittentts offline build ok")' >/dev/null
# ---- deliver AS THE CONTAINER USER (review2 #4): root never writes
# into hermesHome, so no symlink race and no `chown` step. setpriv
# drops to uid 986, rm -rf's the old delivered copies and copies the
# fresh staging tree in. cp-as-986 also fixes review2 #2: rm+cp in one
# step, no mv -Tf rename-replace on a non-empty directory.
setpriv --reuid=${hermesUid} --regid=${hermesGid} --clear-groups \
${pkgs.runtimeShell} -c '
set -eu
rm -rf "$1" "$2"
cp -a "$3" "$1"
cp -a "$4" "$2"
' _ \
"${hermesHome}/kittentts-venv" \
"${hermesHome}/kittentts-hf" \
"$stageVenv" \
"$stageHub"
# Stamp LAST, root-owned outside hermesHome — luna can delete it (which
# forces a cheap re-delivery on next boot), not tamper via symlink.
printf '%s' "$inputHash" > "$stampFile"
# Stamp LAST — a half-provisioned venv fails the integrity check and
# re-provisions on next boot rather than being trusted.
printf '%s' "$reqHash" > "$stampFile"
chown ${hermesUid}:${hermesGid} "$stampFile"
'';
};
}
-17
View File
@@ -1,17 +0,0 @@
# STT add-on pins for the KittenTTS side-venv on mars (faster-whisper CPU).
#
# APPENDED to hosts/mars/kittentts/requirements.txt by the provisioning unit
# (see hermes-agent.nix, KittenTTS section) so ONE venv covers the entire
# local-audio stack: TTS (KittenTTS) + STT (faster-whisper). Shares the
# numpy/onnxruntime pins already frozen there — no version conflicts.
#
# faster-whisper runs CTranslate2 on CPU; the tiny/base model (~150 MB)
# downloads once from the HF hub then works fully offline. Validated
# 2026-09-19 with a live Telegram voice note transcribed on i5-4460 CPU.
#
# NOTE on av: PyP AV 18.x needs libz/libstdc++ from the host. On NixOS those
# come via the nix store — the provisioning unit exports LD_LIBRARY_PATH with
# them at runtime (see hermes-agent.nix's kittenttsLDLibraryPath).
faster-whisper==1.2.1
ctranslate2==4.8.2
av==18.1.0
+46
View File
@@ -0,0 +1,46 @@
# Pinned requirements for a Mnemosyne side-venv on mars.
#
# Hermes vendors its own Python (the official image's venv) and deliberately
# stays minimal: no pip module inside it, PEP 668 external-management on top.
# Installing provider packages straight into that interpreter would fight the
# image on every rebuild, so Mnemosyne (and its plugin wrapper) live in their
# own venv instead — see the provisioning unit in hosts/mars/hermes-agent.nix.
#
# Freeze captured 2026-09-19 from a verified container-side install of
# `mnemosyne-memory[embeddings]` + `mnemosyne-hermes` — side venv at
# $HERMES_HOME/mnemosyne-venv, activated via $HERMES_HOME/plugins/mnemosyne.
# Versions pinned exactly; transitive deps frozen for reproducibility
# (onnxruntime/numpy drift under a long-lived SQLite state dir is what a
# freeze is here to prevent).
#
anyio==4.15.0
certifi==2026.7.22
charset-normalizer==3.5.1
click==8.5.0
fastembed==0.8.0
filelock==3.32.5
flatbuffers==25.12.19
fsspec==2026.7.0
h11==0.16.0
hf-xet==1.6.0
httpcore==1.0.9
httpx==0.28.1
huggingface-hub==1.32.0
idna==3.19
loguru==0.7.3
mmh3==5.3.0
mnemosyne-hermes==0.5.0
mnemosyne-memory==3.15.1
numpy==2.5.3
onnxruntime==1.30.0
packaging==26.3
pillow==12.3.0
protobuf==7.36.1
py-rust-stemmers==0.1.8
pyyaml==6.0.3
requests==2.34.2
sqlite-vec==0.1.9
tokenizers==0.23.2
tqdm==4.70.0
typing-extensions==4.16.0
urllib3==2.7.0
-22
View File
@@ -28,11 +28,6 @@
sops.secrets.opencode_go_api_key = { };
sops.secrets.telegram_bot_token = { };
sops.secrets.hermes_dashboard_oidc_client_secret = { };
# ElevenLabs TTS/STT. Unlike the webhook secret this must live in the
# container env — luna calls the API herself. restartUnits because podman
# reads --env-file only at container creation: rotating the key without it
# re-renders the file under a container that never re-reads it.
sops.secrets.elevenlabs_api_key.restartUnits = [ "podman-hermes-agent.service" ];
# Same value as secrets/jupiter.yaml (the sending side), stored WITHOUT a
# trailing newline — a stray newline would change the HMAC key and fail
# every delivery. Written host-side by hermes-agent-webhook-routes, so it
@@ -40,18 +35,6 @@
sops.secrets.gitea_hermes_webhook_secret = {
restartUnits = [ "hermes-agent-webhook-routes.service" ];
};
# Bearer token for the OpenAI-compatible API server (see the env template
# below). Same value in secrets/terra.yaml, where luna-voiced reads it.
# API_SERVER_HOST=0.0.0.0 for the same reason the dashboard binds it (see
# hermes-agent.nix): the container is --network=host and nothing opens 8642
# in the firewall, so it is reachable over tailscale0 only — LAN-closed.
# prepare-dirs writes it into luna's .env (the container env alone does not
# reach the gateway — see hermes-agent.nix); the container then has to come
# back up to read it.
sops.secrets.hermes_api_server_key.restartUnits = [
"hermes-agent-prepare-dirs.service"
"podman-hermes-agent.service"
];
sops.templates."hermes-agent.env".content = ''
OPENCODE_GO_API_KEY=${config.sops.placeholder.opencode_go_api_key}
TELEGRAM_BOT_TOKEN=${config.sops.placeholder.telegram_bot_token}
@@ -60,11 +43,6 @@
WEBHOOK_ENABLED=true
WEBHOOK_PORT=8644
HERMES_DASHBOARD_OIDC_CLIENT_SECRET=${config.sops.placeholder.hermes_dashboard_oidc_client_secret}
ELEVENLABS_API_KEY=${config.sops.placeholder.elevenlabs_api_key}
API_SERVER_ENABLED=true
API_SERVER_HOST=0.0.0.0
API_SERVER_PORT=8642
API_SERVER_KEY=${config.sops.placeholder.hermes_api_server_key}
'';
# luna's gitea push token (services/dev/gitea.nix provisions the account +
-21
View File
@@ -18,30 +18,9 @@ in
../../services/monitoring/node-exporter.nix
../../services/desktop/desktop-hyprland.nix
../../services/desktop/desktop-apps.nix
../../services/desktop/claude-desktop.nix
../../services/desktop/librechat.nix
../../services/desktop/luna-voice.nix
];
# Talk to luna (Hermes Agent on mars) with SUPER+V — see
# hosts/terra/home/hyprland.nix for the push-to-talk binds.
homelab.lunaVoice = {
enable = true;
hermesKeyFile = config.sops.secrets.hermes_api_server_key.path;
elevenLabsKeyFile = config.sops.secrets.elevenlabs_api_key.path;
# luna's own voice — must stay in sync with `tts.elevenlabs.voice_id` in
# her config.yaml on mars, or she sounds like someone else on the desktop
# than she does on Telegram.
voiceId = "LZAcK8Cx5QjdQhfBsJQZ";
systemPrompt = ''
You are being spoken to out loud and your reply will be read aloud by a
speech synthesiser. Answer in one or two sentences unless asked for
detail. No markdown, no lists, no code blocks — they do not survive the
trip to audio. You may use an audio tag now and then, such as [laughs],
[sighs] or [whispers]; the synthesiser performs them.
'';
};
networking.hostName = "terra";
homelab.greeter = {
+1 -6
View File
@@ -195,11 +195,6 @@ in
# expand / collapse the hyprchrome bar as a whole
(bind "SUPER + A" (dsp.global "quickshell:chrome"))
# toggle luna's hands-free voice mode (services/desktop/luna-voice.nix).
# Runs the CLI rather than a quickshell GlobalShortcut so the mic can
# still be switched off while the shell is reloading.
(bind "SUPER + V" (dsp.exec "luna-voice toggle"))
(bind "SUPER + B" (dsp.exec "vivaldi"))
(bind "SUPER + E" (dsp.exec "cosmic-files"))
(bind "SUPER + SHIFT + G" (dsp.workspaceRef "game"))
@@ -242,7 +237,7 @@ in
(bind "SUPER + T" (dsp.togglespecial "terminal"))
(bind "SUPER + C" (dsp.togglespecial "communications"))
(bind "SUPER + M" (dsp.togglespecial "music"))
# (bind "SUPER + V" (dsp.togglespecial "version_control"))
(bind "SUPER + V" (dsp.togglespecial "version_control"))
# cycle workspaces
(bind "SUPER + ALT + up" (dsp.workspaceRef "e+1"))
-12
View File
@@ -27,18 +27,6 @@
# tokens. All four are random, generated once with `sops --set` (see
# CLAUDE.md) — losing/rotating them just invalidates existing sessions and
# any saved per-user API keys, nothing else depends on their value.
# luna-voiced (services/desktop/luna-voice.nix) runs as a USER service, so
# these cannot stay root:root 0400 — a user manager cannot read them and
# LoadCredential does not bridge that gap either. Hence owner, not mode.
#
# hermes_api_server_key mirrors secrets/mars.yaml (the serving side);
# elevenlabs_api_key mirrors it too, because client-direct TTS is the
# documented low-latency path — terra calls ElevenLabs itself rather than
# relaying audio through mars. Rotating that key therefore touches BOTH
# files; see CLAUDE.md.
sops.secrets.hermes_api_server_key.owner = "darman";
sops.secrets.elevenlabs_api_key.owner = "darman";
sops.secrets.librechat_creds_key = { };
sops.secrets.librechat_creds_iv = { };
sops.secrets.librechat_jwt_secret = { };
-139
View File
@@ -1,139 +0,0 @@
{ lib
, stdenv
, fetchurl
, dpkg
, autoPatchelfHook
, makeWrapper
, wrapGAppsHook3
, alsa-lib
, at-spi2-atk
, at-spi2-core
, atk
, cairo
, cups
, dbus
, expat
, glib
, gtk3
, libcap_ng
, libdrm
, libgbm
, libseccomp
, libxkbcommon
, nspr
, nss
, pango
, systemd
, libx11
, libxcb
, libxcomposite
, libxdamage
, libxext
, libxfixes
, libxrandr
, libxtst
, libGL
, libayatana-appindicator
, libnotify
, libpulseaudio
, libsecret
, pipewire
, wayland
, qemu_kvm
, xdg-utils
}:
# Anthropic's official Linux build (Debian .deb), patchelf'd in place. The
# bundled Electron binary is kept rather than swapped for nixpkgs' electron:
# app.asar ships a V8 code cache (compile-cache/*.jsc) tied to that exact build.
#
# Cowork looks for qemu on $PATH but for OVMF and virtiofsd only at fixed
# Debian paths; services/desktop/claude-desktop.nix symlinks those into /usr.
#
# Bump: take the highest `Version:` listed in
# https://downloads.claude.ai/claude-desktop/apt/stable/dists/stable/main/binary-amd64/Packages
stdenv.mkDerivation (finalAttrs: {
pname = "claude-desktop";
version = "2.7032.0";
src = fetchurl {
url = "https://downloads.claude.ai/claude-desktop/apt/stable/pool/main/c/claude-desktop/claude-desktop_${finalAttrs.version}_amd64.deb";
hash = "sha256-Hn9FBLylsvay08QSPRRdcnZH538u4tBGhQcR5h59exE=";
};
nativeBuildInputs = [ dpkg autoPatchelfHook makeWrapper wrapGAppsHook3 ];
buildInputs = [
alsa-lib
at-spi2-atk
at-spi2-core
atk
cairo
cups
dbus
expat
glib
gtk3
libcap_ng # bundled virtiofsd
libdrm
libgbm
libseccomp # bundled virtiofsd
libxkbcommon
nspr
nss
pango
stdenv.cc.cc.lib # node-pty
systemd
libx11
libxcb
libxcomposite
libxdamage
libxext
libxfixes
libxrandr
libxtst
];
# dlopen'd by Chromium, so not in any DT_NEEDED.
runtimeDependencies = [
libGL
libayatana-appindicator
libnotify
libpulseaudio
libsecret
pipewire
systemd
wayland
];
dontWrapGApps = true;
installPhase = ''
runHook preInstall
mkdir -p $out/lib $out/bin
cp -r usr/lib/claude-desktop $out/lib/
cp -r usr/share $out/share
rm -r $out/share/lintian
# Setuid is impossible in the store; Chromium falls back to its userns sandbox.
rm $out/lib/claude-desktop/chrome-sandbox
runHook postInstall
'';
postFixup = ''
makeWrapper $out/lib/claude-desktop/claude-desktop $out/bin/claude-desktop \
"''${gappsWrapperArgs[@]}" \
--prefix PATH : ${lib.makeBinPath [ qemu_kvm xdg-utils ]}
'';
meta = {
description = "Desktop application for Claude.ai";
homepage = "https://claude.ai";
license = lib.licenses.unfree;
sourceProvenance = [ lib.sourceTypes.binaryNativeCode ];
platforms = [ "x86_64-linux" ];
mainProgram = "claude-desktop";
};
})
-32
View File
@@ -1,32 +0,0 @@
{ lib
, rustPlatform
, pkg-config
, alsa-lib
}:
# luna-voiced — the push-to-talk voice loop for luna (Hermes Agent on mars).
# Wiring (secrets, whisper model, systemd user unit) lives in
# services/desktop/luna-voice.nix; this only builds the binary.
#
# cargoLock.lockFile rather than a cargoHash: the lockfile is committed, so a
# dependency bump is a visible diff instead of a hash nobody can read.
#
# alsa-lib is cpal's backend even on a pipewire box — pipewire's ALSA
# compatibility layer is what actually serves it, so there is no pipewire
# build dependency here.
rustPlatform.buildRustPackage {
pname = "luna-voice";
version = "0.1.0";
src = ./luna-voice;
cargoLock.lockFile = ./luna-voice/Cargo.lock;
nativeBuildInputs = [ pkg-config ];
buildInputs = [ alsa-lib ];
meta = {
description = "Push-to-talk voice loop for luna (Hermes Agent)";
mainProgram = "luna-voice";
platforms = lib.platforms.linux;
};
}
-2166
View File
File diff suppressed because it is too large Load Diff
-19
View File
@@ -1,19 +0,0 @@
[package]
name = "luna-voice"
version = "0.1.0"
edition = "2021"
[dependencies]
anyhow = "1"
base64 = "0.22"
cpal = "0.15"
futures-util = { version = "0.3", default-features = false, features = ["sink", "std"] }
hound = "3"
reqwest = { version = "0.12", default-features = false, features = ["json", "stream", "rustls-tls"] }
serde = { version = "1", features = ["derive"] }
serde_json = "1"
tokio = { version = "1", features = ["rt-multi-thread", "macros", "net", "sync", "time", "process", "io-util", "signal"] }
tokio-tungstenite = { version = "0.24", default-features = false, features = ["connect", "rustls-tls-webpki-roots"] }
[profile.release]
opt-level = 2
-311
View File
@@ -1,311 +0,0 @@
use anyhow::{anyhow, Result};
use cpal::traits::{DeviceTrait, HostTrait, StreamTrait};
use std::collections::VecDeque;
use std::sync::atomic::{AtomicBool, AtomicU32, Ordering};
use std::sync::{Arc, Mutex};
/// What whisper wants. Everything captured is resampled to this before it
/// reaches disk.
pub const STT_RATE: u32 = 16_000;
/// What we ask ElevenLabs for (pcm_24000), resampled to the device on the way out.
pub const TTS_RATE: u32 = 24_000;
/// Mic. The stream runs for the whole session, not just while recording:
/// barge-in needs a live RMS while she is speaking, and restarting a cpal
/// stream per utterance costs ~100ms of dropped samples on pipewire.
///
/// cpal::Stream is !Send, so it is owned by a parked thread rather than by
/// this handle — which then holds nothing but atomics and a mutex and can be
/// shared across tokio tasks freely.
pub struct Capture {
pub rms: Arc<AtomicU32>,
recording: Arc<AtomicBool>,
buffer: Arc<Mutex<Vec<f32>>>,
rate: u32,
}
impl Capture {
pub fn new() -> Result<Self> {
let rms = Arc::new(AtomicU32::new(0));
let recording = Arc::new(AtomicBool::new(false));
let buffer = Arc::new(Mutex::new(Vec::new()));
let (r, rec, buf) = (rms.clone(), recording.clone(), buffer.clone());
let rate = spawn_stream(move || {
let device = input_device()?;
// Named in the log because "the mic does not work" is usually
// "the default source is the wrong device", and nothing else in
// the pipeline can tell you that.
eprintln!(
"luna-voice: input device {:?}",
device.name().unwrap_or_else(|_| "<unnamed>".into())
);
let config = device.default_input_config()?;
let rate = config.sample_rate().0;
let channels = config.channels() as usize;
let stream = device.build_input_stream(
&config.into(),
move |data: &[f32], _: &cpal::InputCallbackInfo| {
// Downmix to mono up front; nothing downstream cares about stereo.
let mono: Vec<f32> = data
.chunks(channels)
.map(|f| f.iter().sum::<f32>() / channels as f32)
.collect();
let sum: f32 = mono.iter().map(|s| s * s).sum();
let level = (sum / mono.len().max(1) as f32).sqrt();
r.store(level.to_bits(), Ordering::Relaxed);
if rec.load(Ordering::Relaxed) {
if let Ok(mut b) = buf.lock() {
b.extend_from_slice(&mono);
}
}
},
|e| eprintln!("luna-voice: capture error: {e}"),
None,
)?;
stream.play()?;
Ok((stream, rate))
})?;
Ok(Capture { rms, recording, buffer, rate })
}
pub fn level(&self) -> f32 {
f32::from_bits(self.rms.load(Ordering::Relaxed))
}
pub fn start(&self) {
self.buffer.lock().map(|mut b| b.clear()).ok();
self.recording.store(true, Ordering::Relaxed);
}
/// Stops recording and returns what was captured, resampled to STT_RATE.
pub fn stop(&self) -> Vec<f32> {
self.recording.store(false, Ordering::Relaxed);
let raw = self
.buffer
.lock()
.map(|mut b| std::mem::take(&mut *b))
.unwrap_or_default();
resample(&raw, self.rate, STT_RATE)
}
}
/// Speaker. Fed from the TTS websocket; `clear` is what makes barge-in
/// instant rather than "after the current sentence".
pub struct Playback {
queue: Arc<Mutex<VecDeque<f32>>>,
rate: u32,
}
impl Playback {
pub fn new() -> Result<Self> {
let queue: Arc<Mutex<VecDeque<f32>>> = Arc::new(Mutex::new(VecDeque::new()));
let q = queue.clone();
let rate = spawn_stream(move || {
let device = cpal::default_host()
.default_output_device()
.ok_or_else(|| anyhow!("no output device"))?;
let config = device.default_output_config()?;
let rate = config.sample_rate().0;
let channels = config.channels() as usize;
let stream = device.build_output_stream(
&config.into(),
move |data: &mut [f32], _: &cpal::OutputCallbackInfo| {
let mut q = match q.lock() {
Ok(q) => q,
Err(_) => return,
};
for frame in data.chunks_mut(channels) {
let s = q.pop_front().unwrap_or(0.0);
for out in frame.iter_mut() {
*out = s;
}
}
},
|e| eprintln!("luna-voice: playback error: {e}"),
None,
)?;
stream.play()?;
Ok((stream, rate))
})?;
Ok(Playback { queue, rate })
}
pub fn push(&self, pcm: &[f32]) {
let out = resample(pcm, TTS_RATE, self.rate);
if let Ok(mut q) = self.queue.lock() {
q.extend(out);
}
}
pub fn clear(&self) {
self.queue.lock().map(|mut q| q.clear()).ok();
}
pub fn pending(&self) -> usize {
self.queue.lock().map(|q| q.len()).unwrap_or(0)
}
/// Short sine burst, so you know the key registered without looking at
/// anything. Hermes' voice mode has `beep_enabled` for the same reason:
/// push-to-talk with no acknowledgement is indistinguishable from broken.
///
/// Raised-cosine envelope — a bare sine starting at full amplitude clicks.
pub fn beep(&self, freq: f32, ms: u32) {
let n = (self.rate as f32 * ms as f32 / 1000.0) as usize;
let samples: Vec<f32> = (0..n)
.map(|i| {
let t = i as f32 / self.rate as f32;
let progress = i as f32 / n as f32;
let envelope = (std::f32::consts::PI * progress).sin();
(t * freq * std::f32::consts::TAU).sin() * envelope * 0.15
})
.collect();
if let Ok(mut q) = self.queue.lock() {
q.extend(samples);
}
}
}
/// The microphone, honouring LUNA_INPUT_DEVICE (a case-insensitive substring
/// of the device name) before falling back to the system default.
///
/// Worth having because the default source is a system-wide setting that has
/// nothing to do with which input you actually speak into: a box with an
/// audio interface and onboard audio has several, and picking the silent one
/// looks exactly like a broken pipeline.
fn input_device() -> Result<cpal::Device> {
let host = cpal::default_host();
if let Ok(want) = std::env::var("LUNA_INPUT_DEVICE") {
let want = want.to_lowercase();
if let Ok(devices) = host.input_devices() {
for device in devices {
if device
.name()
.map(|n| n.to_lowercase().contains(&want))
.unwrap_or(false)
{
return Ok(device);
}
}
}
eprintln!("luna-voice: no input device matching {want:?}; using the default");
}
host.default_input_device()
.ok_or_else(|| anyhow!("no input device"))
}
/// Every input cpal can see, for `luna-voice devices`.
pub fn list_inputs() -> Result<Vec<String>> {
let host = cpal::default_host();
let default = host.default_input_device().and_then(|d| d.name().ok());
let mut out = Vec::new();
for device in host.input_devices()? {
let name = device.name().unwrap_or_else(|_| "<unnamed>".into());
let marker = if Some(&name) == default.as_ref() { " (default)" } else { "" };
out.push(format!("{name}{marker}"));
}
Ok(out)
}
/// Builds a cpal stream on a dedicated thread that then parks forever,
/// keeping it alive. Returns the device's sample rate, or the build error —
/// a stream that fails here must surface at startup, not as silence later.
fn spawn_stream<F>(build: F) -> Result<u32>
where
F: FnOnce() -> Result<(cpal::Stream, u32)> + Send + 'static,
{
let (tx, rx) = std::sync::mpsc::channel::<Result<u32, String>>();
std::thread::spawn(move || match build() {
Ok((_stream, rate)) => {
let _ = tx.send(Ok(rate));
// Dropping `stream` would stop the device, so the thread holds it
// for the life of the process.
loop {
std::thread::park();
}
#[allow(unreachable_code)]
drop(_stream);
}
Err(e) => {
let _ = tx.send(Err(e.to_string()));
}
});
rx.recv()
.map_err(|_| anyhow!("audio thread died during setup"))?
.map_err(|e| anyhow!(e))
}
/// Linear resampling. Speech at these rates does not justify a windowed sinc,
/// and rubato would add a dependency for inaudible gain.
pub fn resample(input: &[f32], from: u32, to: u32) -> Vec<f32> {
if from == to || input.is_empty() {
return input.to_vec();
}
let ratio = to as f64 / from as f64;
let out_len = (input.len() as f64 * ratio).round() as usize;
(0..out_len)
.map(|i| {
let src = i as f64 / ratio;
let j = src.floor() as usize;
let frac = (src - j as f64) as f32;
let a = input.get(j).copied().unwrap_or(0.0);
let b = input.get(j + 1).copied().unwrap_or(a);
a + (b - a) * frac
})
.collect()
}
/// 16-bit mono WAV, which is the only thing whisper-cli accepts without
/// converting first.
pub fn write_wav(path: &std::path::Path, samples: &[f32], rate: u32) -> Result<()> {
let spec = hound::WavSpec {
channels: 1,
sample_rate: rate,
bits_per_sample: 16,
sample_format: hound::SampleFormat::Int,
};
let mut w = hound::WavWriter::create(path, spec)?;
for s in samples {
w.write_sample((s.clamp(-1.0, 1.0) * i16::MAX as f32) as i16)?;
}
w.finalize()?;
Ok(())
}
/// Signed 16-bit little-endian, as ElevenLabs sends pcm_*.
pub fn pcm16_to_f32(bytes: &[u8]) -> Vec<f32> {
bytes
.chunks_exact(2)
.map(|b| i16::from_le_bytes([b[0], b[1]]) as f32 / i16::MAX as f32)
.collect()
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn resample_keeps_length_ratio() {
let input = vec![0.0f32; 1000];
assert_eq!(resample(&input, 48_000, 16_000).len(), 333);
assert_eq!(resample(&input, 24_000, 48_000).len(), 2000);
}
#[test]
fn resample_is_identity_at_equal_rates() {
let input = vec![0.1, -0.4, 0.9];
assert_eq!(resample(&input, 16_000, 16_000), input);
}
#[test]
fn pcm16_roundtrips_through_f32() {
let bytes = [0x00, 0x00, 0xff, 0x7f, 0x01, 0x80];
let got = pcm16_to_f32(&bytes);
assert_eq!(got.len(), 3);
assert!((got[0]).abs() < 1e-6);
assert!((got[1] - 1.0).abs() < 1e-3);
assert!((got[2] + 1.0).abs() < 1e-3);
}
}
-123
View File
@@ -1,123 +0,0 @@
use anyhow::{bail, Result};
use futures_util::StreamExt;
use serde::{Deserialize, Serialize};
use tokio::sync::mpsc;
#[derive(Clone, Serialize, Deserialize, Debug)]
pub struct Message {
pub role: String,
pub content: String,
}
impl Message {
pub fn user(content: impl Into<String>) -> Self {
Message { role: "user".into(), content: content.into() }
}
pub fn assistant(content: impl Into<String>) -> Self {
Message { role: "assistant".into(), content: content.into() }
}
pub fn system(content: impl Into<String>) -> Self {
Message { role: "system".into(), content: content.into() }
}
}
/// Streams a reply from Hermes' OpenAI-compatible API, pushing text deltas
/// into `tx` as they arrive.
///
/// /v1/chat/completions with client-side history, not /v1/responses with
/// previous_response_id: the Responses API keeps the transcript server-side,
/// which is nicer, but this shape is the one every OpenAI client exercises so
/// it is the one least likely to differ between Hermes versions. Her memory
/// and skills are unaffected either way — those are not conversation state.
pub async fn stream_reply(
client: &reqwest::Client,
base_url: &str,
key: &str,
model: &str,
history: &[Message],
tx: mpsc::Sender<String>,
) -> Result<String> {
let body = serde_json::json!({
"model": model,
"messages": history,
"stream": true,
});
let resp = client
.post(format!("{}/v1/chat/completions", base_url.trim_end_matches('/')))
.bearer_auth(key)
.json(&body)
.send()
.await?;
if !resp.status().is_success() {
let status = resp.status();
let text = resp.text().await.unwrap_or_default();
bail!("hermes {status}: {}", text.trim());
}
let mut stream = resp.bytes_stream();
let mut pending = String::new();
let mut full = String::new();
while let Some(chunk) = stream.next().await {
pending.push_str(&String::from_utf8_lossy(&chunk?));
// SSE frames are separated by a blank line; a chunk boundary can land
// mid-frame, so only complete frames are consumed.
while let Some(idx) = pending.find("\n\n") {
let frame: String = pending.drain(..idx + 2).collect();
for line in frame.lines() {
let Some(data) = line.strip_prefix("data:") else { continue };
let data = data.trim();
if data == "[DONE]" {
return Ok(full);
}
if let Some(delta) = parse_delta(data) {
full.push_str(&delta);
// A closed receiver means the reply was cancelled (barge-in).
if tx.send(delta).await.is_err() {
return Ok(full);
}
}
}
}
}
Ok(full)
}
/// Pulls choices[0].delta.content out of one SSE data payload. Tool-call
/// frames and role-only frames carry no content and are skipped.
fn parse_delta(data: &str) -> Option<String> {
let v: serde_json::Value = serde_json::from_str(data).ok()?;
let content = v
.get("choices")?
.get(0)?
.get("delta")?
.get("content")?
.as_str()?;
(!content.is_empty()).then(|| content.to_string())
}
#[cfg(test)]
mod tests {
use super::parse_delta;
#[test]
fn extracts_content_delta() {
let d = r#"{"choices":[{"delta":{"content":"hi"}}]}"#;
assert_eq!(parse_delta(d).as_deref(), Some("hi"));
}
#[test]
fn ignores_role_only_and_tool_frames() {
assert!(parse_delta(r#"{"choices":[{"delta":{"role":"assistant"}}]}"#).is_none());
assert!(parse_delta(r#"{"choices":[{"delta":{"tool_calls":[]}}]}"#).is_none());
assert!(parse_delta(r#"{"choices":[{"delta":{"content":""}}]}"#).is_none());
}
#[test]
fn ignores_malformed_json_rather_than_failing_the_stream() {
assert!(parse_delta("not json").is_none());
assert!(parse_delta("{}").is_none());
}
}
-101
View File
@@ -1,101 +0,0 @@
use anyhow::{Context, Result};
use std::path::PathBuf;
/// Everything the daemon needs, all from the environment so the NixOS module
/// is the only place that knows paths. Secrets arrive as *_FILE (sops paths),
/// never as values — an env var is readable from /proc for anything of the
/// user's, a 0400 file is not.
pub struct Config {
pub hermes_url: String,
pub hermes_key: String,
pub model: String,
pub system_prompt: Option<String>,
/// Prefixed to every spoken message so luna can tell this channel apart
/// from Telegram or the dashboard. Her own profile on mars keys off it —
/// which is why it is a per-message marker and not a system prompt: a
/// system prompt is this client's opinion, while the marker survives into
/// whatever history Hermes keeps and still identifies the channel later.
pub voice_marker: Option<String>,
pub eleven_key: String,
pub voice_id: String,
pub tts_model: String,
pub whisper_bin: String,
pub whisper_model: PathBuf,
pub socket_path: PathBuf,
/// Silence that ends an utterance in hands-free mode. Hermes' own default.
pub silence_ms: u64,
/// Level above the running noise floor that starts one.
pub vad_mult: f32,
/// Speech during playback louder than noise_floor * this counts as barge-in.
pub barge_in_mult: f32,
pub barge_in_grace_ms: u64,
}
fn env(key: &str) -> Result<String> {
std::env::var(key).with_context(|| format!("{key} is not set"))
}
fn env_or(key: &str, default: &str) -> String {
std::env::var(key).unwrap_or_else(|_| default.to_string())
}
/// Reads a secret out of a file and trims it. A trailing newline in a bearer
/// token produces a 401 whose message says nothing about whitespace.
fn secret(key: &str) -> Result<String> {
let path = env(key)?;
let raw = std::fs::read_to_string(&path).with_context(|| format!("reading {path}"))?;
Ok(raw.trim().to_string())
}
impl Config {
pub fn from_env() -> Result<Self> {
let runtime = env_or(
"XDG_RUNTIME_DIR",
&format!("/run/user/{}", unsafe { libc_getuid() }),
);
Ok(Config {
hermes_url: env_or("LUNA_HERMES_URL", "http://mars.orbit.sol:8642"),
hermes_key: secret("LUNA_HERMES_KEY_FILE")?,
model: env_or("LUNA_MODEL", "hermes"),
system_prompt: std::env::var("LUNA_SYSTEM_PROMPT").ok(),
// Empty string disables it rather than sending a bare prefix.
voice_marker: match env_or("LUNA_VOICE_MARKER", "[voice]") {
m if m.is_empty() => None,
m => Some(m),
},
eleven_key: secret("LUNA_ELEVENLABS_KEY_FILE")?,
voice_id: env("LUNA_VOICE_ID")?,
tts_model: env_or("LUNA_TTS_MODEL", "eleven_v3_conversational"),
whisper_bin: env_or("LUNA_WHISPER_BIN", "whisper-cli"),
whisper_model: PathBuf::from(env("LUNA_WHISPER_MODEL")?),
socket_path: PathBuf::from(env_or(
"LUNA_SOCKET",
&format!("{runtime}/luna-voiced.sock"),
)),
silence_ms: env_or("LUNA_SILENCE_MS", "2000").parse().unwrap_or(2000),
// 1.8, not Hermes' 3.0-ish: terra's interface has speech only
// ~6dB over its own ambient (0.0045 against 0.0022 measured), and
// no multiplier above ~2 can separate those. The real fix is gain
// on the interface; this makes it work at the gain it has.
vad_mult: env_or("LUNA_VAD_MULT", "1.8").parse().unwrap_or(1.8),
barge_in_mult: env_or("LUNA_BARGE_IN_MULT", "3.0").parse().unwrap_or(3.0),
barge_in_grace_ms: env_or("LUNA_BARGE_IN_GRACE_MS", "500")
.parse()
.unwrap_or(500),
})
}
}
// Avoids pulling in the libc crate for one call.
unsafe fn libc_getuid() -> u32 {
extern "C" {
fn getuid() -> u32;
}
getuid()
}
-239
View File
@@ -1,239 +0,0 @@
use anyhow::Result;
use serde::Serialize;
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex};
use tokio::io::{AsyncBufReadExt, AsyncWriteExt, BufReader};
use tokio::net::{UnixListener, UnixStream};
use tokio::sync::{broadcast, mpsc};
use crate::visemes::Viseme;
/// Daemon -> shell. Newline-delimited JSON; the shell reads it with a
/// SplitParser and never has to spawn anything.
#[derive(Serialize, Clone, Debug)]
#[serde(tag = "type", rename_all = "snake_case")]
pub enum Event {
/// Whether voice mode is on at all. Distinct from `state`: the widget is
/// only on screen while this is true, and every state inside it is a
/// phase of an active conversation.
Mode { active: bool },
State { value: &'static str },
/// Emitted when the first PCM frame reaches the speaker. Viseme offsets
/// are relative to THIS, so the two processes never compare clocks.
SpeechStart,
Visemes { seq: Vec<Viseme> },
Amplitude { rms: f32 },
/// What was said, and by whom. Her side is emitted sentence by sentence
/// as it goes to the synthesiser, so subtitles can lead the audio the
/// same way the visemes do.
Caption { speaker: &'static str, text: String },
SpeechStop { reason: &'static str },
Error { message: String },
}
/// Shell -> daemon.
#[derive(Debug, Clone)]
pub enum Command {
Ptt { down: bool },
Cancel,
/// Run a turn from typed text, skipping the microphone entirely. This is
/// how the brain -> TTS -> speaker half gets tested without speaking, and
/// how you tell a broken mic from a broken pipeline.
Say { text: String },
/// Turn hands-free voice mode on, off, or (None) to the opposite of what
/// it is now — the keybind sends None so one key does both.
Mode { on: Option<bool> },
}
pub struct Ipc {
pub events: broadcast::Sender<Event>,
path: PathBuf,
/// Replayed to every client the moment it connects. The shell can be
/// restarted at any time — including while voice mode is on — and a
/// broadcast-only socket would leave it showing nothing until the next
/// event happened to fire, which in an idle conversation is never.
sticky: Arc<Mutex<Vec<Event>>>,
}
impl Ipc {
/// Binds the socket and serves it until dropped. A stale socket from a
/// crashed run is removed first — bind() would fail with EADDRINUSE on a
/// file nothing is listening to.
pub fn bind(path: &Path, commands: mpsc::Sender<Command>) -> Result<Self> {
if path.exists() {
let _ = std::fs::remove_file(path);
}
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
let listener = UnixListener::bind(path)?;
let (events, _) = broadcast::channel(256);
let sticky: Arc<Mutex<Vec<Event>>> = Arc::new(Mutex::new(Vec::new()));
let tx = events.clone();
let accept_sticky = sticky.clone();
tokio::spawn(async move {
loop {
match listener.accept().await {
Ok((stream, _)) => {
let rx = tx.subscribe();
let cmds = commands.clone();
let snapshot = accept_sticky
.lock()
.map(|s| s.clone())
.unwrap_or_default();
tokio::spawn(client(stream, rx, cmds, snapshot));
}
Err(e) => {
eprintln!("luna-voice: accept failed: {e}");
return;
}
}
}
});
Ok(Ipc { events, path: path.to_path_buf(), sticky })
}
pub fn emit(&self, event: Event) {
// Only the two events that describe a lasting condition are kept;
// replaying a viseme timeline or an old caption to a fresh client
// would animate a conversation that already finished.
if matches!(event, Event::Mode { .. } | Event::State { .. }) {
if let Ok(mut sticky) = self.sticky.lock() {
sticky.retain(|e| std::mem::discriminant(e) != std::mem::discriminant(&event));
sticky.push(event.clone());
}
}
// An error here only means nobody is connected, which is normal.
let _ = self.events.send(event);
}
}
impl Drop for Ipc {
fn drop(&mut self) {
let _ = std::fs::remove_file(&self.path);
}
}
async fn client(
stream: UnixStream,
mut rx: broadcast::Receiver<Event>,
commands: mpsc::Sender<Command>,
snapshot: Vec<Event>,
) {
let (read, mut write) = stream.into_split();
let mut lines = BufReader::new(read).lines();
// Failure here must NOT abandon the connection: the one-shot clients
// (`luna-voice on`, `toggle`, …) write a command and close immediately,
// so this write hits a closed socket and errors — and returning on that
// would drop the command unread, which is exactly what it did once.
for event in snapshot {
let Ok(mut line) = serde_json::to_string(&event) else { continue };
line.push('\n');
if write.write_all(line.as_bytes()).await.is_err() {
break;
}
}
loop {
tokio::select! {
event = rx.recv() => {
let Ok(event) = event else { return };
let Ok(mut line) = serde_json::to_string(&event) else { continue };
line.push('\n');
if write.write_all(line.as_bytes()).await.is_err() {
return;
}
}
line = lines.next_line() => {
match line {
Ok(Some(line)) => {
if let Some(cmd) = parse_command(&line) {
let _ = commands.send(cmd).await;
}
}
_ => return,
}
}
}
}
}
fn parse_command(line: &str) -> Option<Command> {
let v: serde_json::Value = serde_json::from_str(line).ok()?;
match v.get("type")?.as_str()? {
"ptt" => Some(Command::Ptt {
down: v.get("down").and_then(|d| d.as_bool()).unwrap_or(false),
}),
"cancel" => Some(Command::Cancel),
"say" => Some(Command::Say {
text: v.get("text")?.as_str()?.to_string(),
}),
"mode" => Some(Command::Mode {
on: v.get("on").and_then(|o| o.as_bool()),
}),
_ => None,
}
}
/// Client side of the same socket, for the `luna-voice ptt` subcommand that
/// Hyprland binds a key to before the quickshell widget exists.
pub async fn send_command(path: &Path, line: &str) -> Result<()> {
let mut stream = UnixStream::connect(path).await?;
stream.write_all(format!("{line}\n").as_bytes()).await?;
stream.flush().await?;
Ok(())
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn parses_ptt_commands() {
assert!(matches!(
parse_command(r#"{"type":"ptt","down":true}"#),
Some(Command::Ptt { down: true })
));
assert!(matches!(
parse_command(r#"{"type":"ptt","down":false}"#),
Some(Command::Ptt { down: false })
));
}
#[test]
fn ptt_without_down_defaults_to_release() {
assert!(matches!(
parse_command(r#"{"type":"ptt"}"#),
Some(Command::Ptt { down: false })
));
}
#[test]
fn mode_without_a_value_means_toggle() {
assert!(matches!(
parse_command(r#"{"type":"mode"}"#),
Some(Command::Mode { on: None })
));
assert!(matches!(
parse_command(r#"{"type":"mode","on":true}"#),
Some(Command::Mode { on: Some(true) })
));
}
#[test]
fn ignores_unknown_and_malformed() {
assert!(parse_command(r#"{"type":"nope"}"#).is_none());
assert!(parse_command("garbage").is_none());
}
#[test]
fn events_serialise_with_a_type_tag() {
let json = serde_json::to_string(&Event::State { value: "listening" }).unwrap();
assert_eq!(json, r#"{"type":"state","value":"listening"}"#);
let json = serde_json::to_string(&Event::SpeechStart).unwrap();
assert_eq!(json, r#"{"type":"speech_start"}"#);
}
}
-611
View File
@@ -1,611 +0,0 @@
//! luna-voiced — push-to-talk voice loop for luna (the Hermes Agent on mars).
//!
//! mic -> whisper (local, vulkan) -> Hermes /v1/chat/completions -> ElevenLabs
//! streaming TTS -> speaker, with the speaking state, amplitude and viseme
//! timeline published on a unix socket for the quickshell avatar.
//!
//! Nothing here talks to the compositor: push-to-talk arrives over the same
//! socket, from either `luna-voice ptt` or the shell's GlobalShortcut.
mod audio;
mod brain;
mod config;
mod ipc;
mod stt;
mod tts;
mod visemes;
use anyhow::Result;
use serde_json;
use config::Config;
use ipc::{Command, Event};
use std::sync::Arc;
use std::time::Duration;
use tokio::sync::mpsc;
#[tokio::main]
async fn main() -> Result<()> {
let mut args = std::env::args().skip(1);
match args.next().as_deref() {
None | Some("daemon") => daemon().await,
// Fire-and-forget clients so a keybind costs one short-lived process
// and never blocks the compositor.
Some("ptt-down") => client(r#"{"type":"ptt","down":true}"#).await,
Some("ptt-up") => client(r#"{"type":"ptt","down":false}"#).await,
Some("cancel") => client(r#"{"type":"cancel"}"#).await,
Some("toggle") => client(r#"{"type":"mode"}"#).await,
Some("on") => client(r#"{"type":"mode","on":true}"#).await,
Some("off") => client(r#"{"type":"mode","on":false}"#).await,
// The debugging tool: everything the avatar will eventually see,
// printed. `journalctl -fu luna-voiced` shows only what the daemon
// chose to log; this shows the actual protocol.
Some("watch") => watch().await,
// Opens the mic directly, bypassing the daemon: answers "is the
// microphone working at all" without any of the rest being involved.
Some("level") => level().await,
Some("devices") => {
for name in audio::list_inputs()? {
println!("{name}");
}
Ok(())
}
Some("say") => {
let text: Vec<String> = args.collect();
if text.is_empty() {
eprintln!("luna-voice: say what?");
std::process::exit(2);
}
let payload = serde_json::json!({ "type": "say", "text": text.join(" ") });
client(&payload.to_string()).await
}
Some(other) => {
eprintln!("luna-voice: unknown command {other:?}");
eprintln!(
"usage: luna-voice [daemon|toggle|on|off|say <text>|watch|level|devices|cancel]"
);
std::process::exit(2);
}
}
}
/// Level that counts as speech.
///
/// The lower clamp is the number that matters, and it has to sit under the
/// QUIETEST mic worth supporting, not under a typical one: terra's AIR 192|4
/// reads ~0.001 idle and only 0.004–0.005 for normal speech, so a 0.006 floor
/// — which looked conservative — put the trigger permanently above the user's
/// voice. Overshooting downward only costs silent false triggers.
fn root_threshold(floor: f32, mult: f32) -> f32 {
let min = std::env::var("LUNA_VAD_FLOOR")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(0.0025);
(floor * mult).clamp(min, 0.05)
}
/// Tags a spoken line so luna can recognise the channel it came from.
///
/// Prefix rather than suffix: a long utterance can be truncated by whatever
/// history window she is running under, and the front is the end that
/// survives.
fn mark(marker: &Option<String>, text: &str) -> String {
match marker {
Some(m) => format!("{m} {text}"),
None => text.to_string(),
}
}
fn socket_path() -> std::path::PathBuf {
let runtime = std::env::var("XDG_RUNTIME_DIR").unwrap_or_else(|_| "/run/user/1000".into());
std::env::var("LUNA_SOCKET")
.map(std::path::PathBuf::from)
.unwrap_or_else(|_| std::path::Path::new(&runtime).join("luna-voiced.sock"))
}
async fn watch() -> Result<()> {
use tokio::io::{AsyncBufReadExt, BufReader};
let path = socket_path();
let stream = tokio::net::UnixStream::connect(&path)
.await
.map_err(|e| anyhow::anyhow!("{}: {e} (is luna-voiced running?)", path.display()))?;
let mut lines = BufReader::new(stream).lines();
eprintln!("luna-voice: watching {}", path.display());
while let Some(line) = lines.next_line().await? {
println!("{line}");
}
Ok(())
}
/// Live microphone meter with the trigger point marked, so a mic that reads
/// nothing and a threshold that is set too high look different.
async fn level() -> Result<()> {
let capture = audio::Capture::new()?;
let mult: f32 = std::env::var("LUNA_VAD_MULT")
.ok()
.and_then(|v| v.parse().ok())
.unwrap_or(1.8);
let mut floor: f32 = capture.level().max(0.003);
eprintln!("luna-voice: speak — ctrl-c to stop");
loop {
let level = capture.level();
floor = floor * 0.95 + level * 0.05;
let threshold = root_threshold(floor, mult);
// Full scale 0.02, not 0.05: a quiet interface tops out around
// 0.005 and was drawing a single bar on the wider scale.
const FULL: f32 = 0.02;
let bars = ((level / FULL).min(1.0) * 40.0) as usize;
let mark = ((threshold / FULL).min(1.0) * 40.0) as usize;
let meter: String = (0..40)
.map(|i| {
if i == mark {
'|'
} else if i < bars {
'#'
} else {
'.'
}
})
.collect();
print!(
"\r{meter} rms {level:.4} floor {floor:.4} trigger {threshold:.4} {} ",
if level > threshold { "SPEECH" } else { " " }
);
use std::io::Write;
let _ = std::io::stdout().flush();
tokio::time::sleep(Duration::from_millis(50)).await;
}
}
async fn client(line: &str) -> Result<()> {
ipc::send_command(&socket_path(), line).await
}
/// Errors go to the desktop, not just the journal — terra runs its own
/// notification daemon (HyprChrome/Widgets/Notifications), so this is the one
/// channel that is visible while you are talking rather than debugging.
/// Best-effort: a missing notify-send must never take the daemon down.
fn notify(body: &str) {
let _ = std::process::Command::new("notify-send")
.args(["-a", "luna", "-u", "critical", "luna voice", body])
.spawn();
}
/// Where the loop is. One utterance at a time by construction: the state
/// machine is a single task, so there is no way to start a second reply while
/// one is in flight.
async fn daemon() -> Result<()> {
let cfg = Arc::new(Config::from_env()?);
let capture = audio::Capture::new()?;
let playback = Arc::new(audio::Playback::new()?);
let (cmd_tx, mut cmd_rx) = mpsc::channel::<Command>(16);
// Kept so a turn can put back commands that arrived while she was
// speaking instead of destroying them.
let requeue = cmd_tx.clone();
let ipc = Arc::new(ipc::Ipc::bind(&cfg.socket_path, cmd_tx)?);
eprintln!("luna-voice: listening on {}", cfg.socket_path.display());
let http = reqwest::Client::builder()
// Her replies can involve tool calls; the default 30s would cut those
// off mid-thought.
.timeout(Duration::from_secs(300))
.build()?;
let mut history: Vec<brain::Message> = Vec::new();
if let Some(prompt) = &cfg.system_prompt {
history.push(brain::Message::system(prompt.clone()));
}
ipc.emit(Event::State { value: "idle" });
// Her first reply after the container starts costs ~16s against ~5s warm,
// and the first one is always the one you are sitting there waiting for.
// Spending one throwaway token at login moves that cost off the
// conversation. Failure is ignored on purpose: mars may simply be down,
// which must not stop the daemon from coming up.
tokio::spawn({
let (http, url, key, model) =
(http.clone(), cfg.hermes_url.clone(), cfg.hermes_key.clone(), cfg.model.clone());
async move {
let (tx, mut rx) = mpsc::channel(4);
tokio::spawn(async move { while rx.recv().await.is_some() {} });
let warm = [brain::Message::user("hi")];
match brain::stream_reply(&http, &url, &key, &model, &warm, tx).await {
Ok(_) => eprintln!("luna-voice: warmed up"),
Err(e) => eprintln!("luna-voice: warmup skipped: {e}"),
}
}
});
// Hands-free state. The key toggles `mode`; everything inside it is
// driven by the microphone rather than by keys.
let mut mode = false;
// Running estimate of the room, updated only while not capturing, so a
// loud sentence never raises the bar for the next one.
let mut floor: f32 = 0.01;
let mut capturing = false;
let mut last_voice = tokio::time::Instant::now();
let mut ticker = tokio::time::interval(Duration::from_millis(50));
loop {
let cmd = tokio::select! {
cmd = cmd_rx.recv() => match cmd {
Some(cmd) => Some(cmd),
None => break,
},
// Only ticks while voice mode is on. A turn is awaited inline
// below, so this cannot fire while she is speaking — which is
// also what stops her from hearing herself.
_ = ticker.tick(), if mode => None,
};
let Some(cmd) = cmd else {
let level = capture.level();
// Every tick, not just while she speaks: a meter that only moves
// during playback cannot tell "the mic is dead" apart from "the
// threshold is too high", which is exactly the question you have
// when nothing happens.
ipc.emit(Event::Amplitude { rms: level });
let threshold = root_threshold(floor, cfg.vad_mult);
if !capturing {
// Only samples BELOW the trigger update the floor. Feeding
// every sample in lets a rising sound drag the floor up with
// it, so the threshold outruns the speech that caused it and
// nothing ever fires — measured: a tone peaking at 0.029 was
// chased by a threshold that reached 0.018 on the way up.
if level <= threshold {
floor = floor * 0.95 + level * 0.05;
}
if level > threshold {
capture.start();
capturing = true;
last_voice = tokio::time::Instant::now();
ipc.emit(Event::State { value: "listening" });
}
} else {
if level > threshold {
last_voice = tokio::time::Instant::now();
}
if last_voice.elapsed() > Duration::from_millis(cfg.silence_ms) {
capturing = false;
let samples = capture.stop();
ipc.emit(Event::State { value: "thinking" });
match stt::transcribe(&cfg.whisper_bin, &cfg.whisper_model, &samples).await {
Ok(heard) if !heard.is_empty() => {
run_turn(
&cfg, &http, &ipc, &capture, &playback, &mut cmd_rx,
&requeue, &mut history, heard,
)
.await;
}
Ok(_) => {
// Silent on purpose: in hands-free mode a false
// trigger happens whenever a door closes, and a
// beep for each one would be maddening.
ipc.emit(Event::State { value: "idle" });
}
Err(e) => {
eprintln!("luna-voice: {e:#}");
ipc.emit(Event::Error { message: e.to_string() });
notify(&e.to_string());
ipc.emit(Event::State { value: "idle" });
}
}
}
}
continue;
};
match cmd {
Command::Mode { on } => {
let want = on.unwrap_or(!mode);
if want != mode {
mode = want;
if mode {
// Seeded low: a floor seeded high takes seconds to
// decay, and those are the seconds you spend
// talking to something that is not listening yet.
floor = capture.level().max(0.003);
playback.beep(880.0, 70);
} else {
if capturing {
capture.stop();
capturing = false;
}
playback.clear();
playback.beep(440.0, 90);
}
ipc.emit(Event::Mode { active: mode });
ipc.emit(Event::State { value: "idle" });
eprintln!("luna-voice: voice mode {}", if mode { "on" } else { "off" });
}
}
Command::Ptt { down: true } => {
// Anything still playing is cut off: pressing the key while
// she talks means you want to talk instead.
playback.clear();
capture.start();
playback.beep(880.0, 70);
ipc.emit(Event::State { value: "listening" });
}
Command::Ptt { down: false } => {
let samples = capture.stop();
playback.beep(587.0, 70);
ipc.emit(Event::State { value: "thinking" });
let heard =
match stt::transcribe(&cfg.whisper_bin, &cfg.whisper_model, &samples).await {
Ok(h) => h,
Err(e) => {
eprintln!("luna-voice: {e:#}");
ipc.emit(Event::Error { message: e.to_string() });
notify(&e.to_string());
ipc.emit(Event::State { value: "idle" });
continue;
}
};
if heard.is_empty() {
// Distinct low tone: "I heard nothing" and "I am
// thinking" must not feel the same, or a dead mic reads
// as a slow reply.
playback.beep(220.0, 120);
eprintln!("luna-voice: nothing heard ({} samples)", samples.len());
ipc.emit(Event::SpeechStop { reason: "nothing_heard" });
ipc.emit(Event::State { value: "idle" });
continue;
}
run_turn(&cfg, &http, &ipc, &capture, &playback, &mut cmd_rx, &requeue, &mut history, heard)
.await;
}
Command::Say { text } => {
ipc.emit(Event::State { value: "thinking" });
run_turn(&cfg, &http, &ipc, &capture, &playback, &mut cmd_rx, &requeue, &mut history, text)
.await;
}
Command::Cancel => {
playback.clear();
ipc.emit(Event::SpeechStop { reason: "cancelled" });
ipc.emit(Event::State { value: "idle" });
}
}
}
Ok(())
}
/// Runs one turn and reports failure through every channel that has a
/// listener: journal, socket, desktop notification.
#[cfg(test)]
mod tests {
use super::{mark, root_threshold};
#[test]
fn marker_prefixes_spoken_lines() {
assert_eq!(
mark(&Some("[voice]".into()), "what's the time"),
"[voice] what's the time"
);
}
#[test]
fn no_marker_leaves_the_line_alone() {
assert_eq!(mark(&None, "what's the time"), "what's the time");
}
#[test]
fn threshold_sits_between_terras_room_tone_and_terras_speech() {
// Measured on the AIR 192|4: ~0.001 idle, 0.004–0.005 speaking. The
// trigger has to land strictly between those two, which is the whole
// reason the lower clamp is 0.0025 and not something rounder.
let t = root_threshold(0.001, 3.5);
assert!(t > 0.001, "would trigger on room tone: {t}");
assert!(t < 0.004, "would never trigger on speech: {t}");
}
#[test]
fn threshold_never_climbs_out_of_reach_of_speech() {
// A floor seeded from a loud moment must not lock the mic out.
assert_eq!(root_threshold(0.5, 3.5), 0.05);
}
#[test]
fn threshold_tracks_the_room_in_between() {
assert!((root_threshold(0.01, 3.5) - 0.035).abs() < 1e-6);
}
}
#[allow(clippy::too_many_arguments)]
async fn run_turn(
cfg: &Config,
http: &reqwest::Client,
ipc: &Arc<ipc::Ipc>,
capture: &audio::Capture,
playback: &Arc<audio::Playback>,
cmd_rx: &mut mpsc::Receiver<Command>,
requeue: &mpsc::Sender<Command>,
history: &mut Vec<brain::Message>,
heard: String,
) {
if let Err(e) = turn(cfg, http, ipc, capture, playback, cmd_rx, requeue, history, heard).await {
eprintln!("luna-voice: {e:#}");
ipc.emit(Event::Error { message: e.to_string() });
notify(&e.to_string());
}
ipc.emit(Event::State { value: "idle" });
}
#[allow(clippy::too_many_arguments)]
async fn turn(
cfg: &Config,
http: &reqwest::Client,
ipc: &Arc<ipc::Ipc>,
capture: &audio::Capture,
playback: &Arc<audio::Playback>,
cmd_rx: &mut mpsc::Receiver<Command>,
requeue: &mpsc::Sender<Command>,
history: &mut Vec<brain::Message>,
heard: String,
) -> Result<()> {
eprintln!("luna-voice: heard {heard:?}");
// The caption shows what you SAID; the marker is only for her.
ipc.emit(Event::Caption { speaker: "user", text: heard.clone() });
history.push(brain::Message::user(mark(&cfg.voice_marker, &heard)));
// Commands that arrived while she was speaking, replayed into the
// channel once the turn is over rather than dropped.
let mut deferred: Vec<Command> = Vec::new();
// Noise floor sampled before she starts, so barge-in adapts to the room
// rather than to a constant.
let floor = capture.level().max(0.005);
let (sentence_tx, sentence_rx) = mpsc::channel::<String>(16);
let (event_tx, mut event_rx) = mpsc::channel::<tts::SpeechEvent>(64);
let tts_cfg = tts::TtsConfig {
key: cfg.eleven_key.clone(),
voice_id: cfg.voice_id.clone(),
model: cfg.tts_model.clone(),
};
let keep_tags = tts_cfg.is_v3();
let speech = tokio::spawn({
async move { tts::speak(&tts_cfg, sentence_rx, event_tx).await }
});
// Audio and visemes flow to the speaker/shell on their own task so the
// reply stream is never blocked by playback.
let player = tokio::spawn({
let ipc = ipc.clone();
let playback = playback.clone();
async move {
let mut started = false;
while let Some(ev) = event_rx.recv().await {
match ev {
tts::SpeechEvent::Audio(pcm) => {
if !started {
started = true;
ipc.emit(Event::SpeechStart);
ipc.emit(Event::State { value: "speaking" });
}
playback.push(&pcm);
}
// A chunk whose alignment held nothing visible still
// arrives; forwarding it would make the shell reset the
// mouth mid-word.
tts::SpeechEvent::Visemes(seq) if !seq.is_empty() => {
ipc.emit(Event::Visemes { seq })
}
tts::SpeechEvent::Visemes(_) => {}
tts::SpeechEvent::Done => break,
}
}
}
});
let (delta_tx, mut delta_rx) = mpsc::channel::<String>(64);
let reply = tokio::spawn({
let http = http.clone();
let (url, key, model) = (cfg.hermes_url.clone(), cfg.hermes_key.clone(), cfg.model.clone());
let history = history.clone();
async move { brain::stream_reply(&http, &url, &key, &model, &history, delta_tx).await }
});
let mut splitter = tts::SentenceSplitter::new(keep_tags);
// What ElevenLabs bills: every character sent, audio tags included.
let mut billed = 0usize;
let caption = |sentence: &str| {
let text = tts::strip_tags(sentence);
if !text.is_empty() {
ipc.emit(Event::Caption { speaker: "luna", text });
}
};
while let Some(delta) = delta_rx.recv().await {
for sentence in splitter.push(&delta) {
caption(&sentence);
billed += sentence.chars().count();
if sentence_tx.send(sentence).await.is_err() {
break;
}
}
}
if let Some(tail) = splitter.finish() {
caption(&tail);
billed += tail.chars().count();
let _ = sentence_tx.send(tail).await;
}
drop(sentence_tx);
eprintln!("luna-voice: tts {billed} chars on {}", cfg.tts_model);
let full = reply.await??;
eprintln!("luna-voice: said {:?}", full.trim());
if !full.trim().is_empty() {
history.push(brain::Message::assistant(full));
}
// Wait for the speaker to drain, watching for barge-in. Polling the queue
// is enough: cpal gives no drain callback, and 50ms of latency on "she
// finished" is invisible.
let grace = tokio::time::Instant::now() + Duration::from_millis(cfg.barge_in_grace_ms);
loop {
if speech.is_finished() && playback.pending() == 0 {
break;
}
if tokio::time::Instant::now() > grace
&& capture.level() > floor * cfg.barge_in_mult
{
playback.clear();
ipc.emit(Event::SpeechStop { reason: "barge_in" });
// She is told she was cut off, so the next turn can react to it
// rather than carrying on obliviously.
history.push(brain::Message::system(
"[The user interrupted you mid-sentence.]",
));
break;
}
// Commands arriving mid-playback. try_recv POPS, so anything not
// handled here is destroyed: an `if let Ok(Cancel)` silently ate
// every `say`, `toggle` and `mode off` sent while she was talking,
// which presents as a keypress that does nothing.
match cmd_rx.try_recv() {
Ok(Command::Cancel) => {
playback.clear();
ipc.emit(Event::SpeechStop { reason: "cancelled" });
break;
}
// Leaving voice mode has to shut her up immediately; the mode
// change itself is replayed below.
Ok(cmd @ Command::Mode { .. }) => {
playback.clear();
ipc.emit(Event::SpeechStop { reason: "cancelled" });
deferred.push(cmd);
break;
}
Ok(cmd) => deferred.push(cmd),
Err(_) => {}
}
ipc.emit(Event::Amplitude { rms: capture.level() });
tokio::time::sleep(Duration::from_millis(50)).await;
}
for cmd in deferred {
let _ = requeue.try_send(cmd);
}
// A TTS failure must NOT be silent. The captions come from the text
// stream, not from the synthesiser, so a dead websocket still paints her
// whole reply on screen — "she answered but said nothing" is then
// indistinguishable from a working reply with the volume down, which is
// exactly how it presented.
if speech.is_finished() {
if let Ok(Err(e)) = speech.await {
eprintln!("luna-voice: tts failed: {e:#}");
ipc.emit(Event::Error { message: format!("tts: {e}") });
notify(&format!("voice failed: {e}"));
}
} else {
speech.abort();
}
player.abort();
Ok(())
}
-181
View File
@@ -1,181 +0,0 @@
use crate::audio;
use anyhow::{bail, Result};
use std::path::Path;
use tokio::process::Command;
/// Shells out to whisper-cli rather than linking whisper.cpp: nixpkgs already
/// ships whisper-cpp-vulkan built against this box's GPU stack, and a bundled
/// build would have to rediscover that at cargo time.
pub async fn transcribe(bin: &str, model: &Path, samples: &[f32]) -> Result<String> {
// Whisper hallucinates confident words into silence — "you", "Thank
// you.", "Bye." — which then reach luna as if spoken. Duration alone does
// not catch it (a held key over a quiet room is long AND empty), so gate
// on level too.
if samples.len() < (audio::STT_RATE as usize) / 3 || !has_speech(samples) {
return Ok(String::new());
}
let dir = std::env::var("XDG_RUNTIME_DIR").unwrap_or_else(|_| "/tmp".into());
let wav = Path::new(&dir).join(format!("luna-stt-{}.wav", std::process::id()));
audio::write_wav(&wav, &normalize(samples), audio::STT_RATE)?;
let out = Command::new(bin)
.args(["-m"])
.arg(model)
.args(["-f"])
.arg(&wav)
// -nt: transcript only, no [00:00:00] prefixes. -np: no progress
// banner. Together they make stdout exactly the text.
.args(["-nt", "-np"])
.output()
.await?;
let _ = std::fs::remove_file(&wav);
if !out.status.success() {
bail!(
"whisper-cli failed: {}",
String::from_utf8_lossy(&out.stderr).trim()
);
}
Ok(clean(&String::from_utf8_lossy(&out.stdout)))
}
/// Scales a quiet recording up before whisper sees it.
///
/// Whisper is trained on normalised audio and degrades badly on very quiet
/// input, which is precisely what a low-gain interface produces — speech at a
/// 0.03 peak is intelligible to a person and mush to the model. The gain is
/// capped so a recording of near-silence is not amplified into hiss the model
/// will confidently transcribe as words.
fn normalize(samples: &[f32]) -> Vec<f32> {
const TARGET_PEAK: f32 = 0.6;
const MAX_GAIN: f32 = 30.0;
let peak = samples.iter().fold(0.0f32, |m, s| m.max(s.abs()));
if peak <= f32::EPSILON {
return samples.to_vec();
}
let gain = (TARGET_PEAK / peak).min(MAX_GAIN);
// Never attenuate: a loud recording is already fine, and turning it down
// would only lose resolution in the 16-bit conversion.
if gain <= 1.0 {
return samples.to_vec();
}
samples.iter().map(|s| (s * gain).clamp(-1.0, 1.0)).collect()
}
/// True if any 100ms window is loud enough to be a voice. Peak rather than
/// overall RMS: a short sentence inside a long silent recording averages down
/// to nothing, and that is the common case for a held key.
fn has_speech(samples: &[f32]) -> bool {
// Matches the VAD's lower clamp rather than a "typical" mic: a quiet
// interface puts normal speech at ~0.004, and gating at 0.015 threw away
// every word it captured.
const FLOOR: f32 = 0.0025;
let window = (audio::STT_RATE as usize) / 10;
samples.chunks(window).any(|w| {
let sum: f32 = w.iter().map(|s| s * s).sum();
(sum / w.len().max(1) as f32).sqrt() > FLOOR
})
}
/// whisper emits bracketed non-speech annotations ([BLANK_AUDIO], (wind
/// blowing)) that are not words and must not reach the agent.
fn clean(raw: &str) -> String {
let mut out = String::new();
let mut depth = 0i32;
for c in raw.chars() {
match c {
'[' | '(' => depth += 1,
']' | ')' => depth = (depth - 1).max(0),
c if depth == 0 => out.push(c),
_ => {}
}
}
out.split_whitespace().collect::<Vec<_>>().join(" ")
}
#[cfg(test)]
mod tests {
use super::{clean, has_speech, normalize};
use crate::audio::STT_RATE;
fn peak(v: &[f32]) -> f32 {
v.iter().fold(0.0f32, |m, s| m.max(s.abs()))
}
#[test]
fn normalize_lifts_a_quiet_recording() {
// 0.004 RMS speech peaks around 0.02, which is 30x under target —
// exactly at the gain cap, so it just reaches full scale.
let quiet: Vec<f32> = (0..1000).map(|i| 0.02 * ((i % 7) as f32 - 3.0) / 3.0).collect();
assert!((peak(&normalize(&quiet)) - 0.6).abs() < 0.01);
}
#[test]
fn normalize_leaves_a_loud_recording_alone() {
let loud: Vec<f32> = vec![0.9, -0.8, 0.7];
assert_eq!(normalize(&loud), loud);
}
#[test]
fn normalize_caps_the_gain_so_silence_stays_silent() {
let hiss: Vec<f32> = vec![0.0001, -0.0001];
// 0.6/0.0001 would be 6000x; the cap keeps it at 30x.
assert!(peak(&normalize(&hiss)) <= 0.0001 * 30.0 + 1e-9);
}
#[test]
fn normalize_handles_pure_silence() {
assert_eq!(normalize(&[0.0, 0.0]), vec![0.0, 0.0]);
}
#[test]
fn silence_is_not_speech() {
assert!(!has_speech(&vec![0.0f32; STT_RATE as usize]));
// Terra's measured room tone. Speech on the same mic reaches
// 0.004–0.005, so the gate sits between the two.
assert!(!has_speech(&vec![0.001f32; STT_RATE as usize]));
}
#[test]
fn quiet_speech_still_counts() {
// 0.004 is what normal speech measures on terra's interface; gating
// it out discarded every word the VAD had just captured.
let mut s = vec![0.001f32; STT_RATE as usize];
for v in s.iter_mut().take(STT_RATE as usize / 5) {
*v = 0.004;
}
assert!(has_speech(&s));
}
#[test]
fn a_short_utterance_in_a_long_silence_counts() {
let mut s = vec![0.0f32; STT_RATE as usize];
for v in s.iter_mut().take(STT_RATE as usize / 5) {
*v = 0.2;
}
assert!(has_speech(&s));
}
#[test]
fn strips_bracketed_annotations() {
assert_eq!(clean("[BLANK_AUDIO]"), "");
assert_eq!(clean(" hello (wind blowing) there \n"), "hello there");
assert_eq!(clean("[ music ] real words"), "real words");
}
#[test]
fn collapses_whitespace() {
assert_eq!(clean(" a b \n c "), "a b c");
}
#[test]
fn unbalanced_brackets_do_not_eat_the_rest() {
// The stray closer is dropped (depth floors at 0) but the words
// around it survive — the failure mode that matters is a lone
// bracket swallowing the whole transcript.
assert_eq!(clean("ok] fine"), "ok fine");
assert_eq!(clean("[ oops real words"), "");
}
}
-529
View File
@@ -1,529 +0,0 @@
use crate::audio;
use crate::visemes::{self, Viseme};
use anyhow::{anyhow, Result};
use base64::Engine;
use futures_util::{SinkExt, StreamExt};
use std::time::Duration;
use tokio::sync::mpsc;
use tokio_tungstenite::tungstenite::client::IntoClientRequest;
use tokio_tungstenite::tungstenite::Message as Ws;
pub enum SpeechEvent {
/// Decoded PCM at audio::TTS_RATE.
Audio(Vec<f32>),
/// Mouth shapes, timed from the start of this reply's audio.
Visemes(Vec<Viseme>),
Done,
}
pub struct TtsConfig {
pub key: String,
pub voice_id: String,
pub model: String,
}
impl TtsConfig {
/// v3 is not served on the TTS websocket at all; it streams only over
/// Text to Dialogue, a different protocol.
pub fn is_v3(&self) -> bool {
self.model.starts_with("eleven_v3")
}
}
/// The two sockets send the same audio frames under different field names.
struct Wire {
alignment: &'static [&'static str],
final_key: &'static str,
}
const TTS_WIRE: Wire = Wire { alignment: &["normalizedAlignment", "alignment"], final_key: "isFinal" };
const DIALOGUE_WIRE: Wire = Wire { alignment: &["alignment"], final_key: "is_final" };
/// Dialogue drops the socket after 20s without a client message, and a tool
/// call on mars can stall a reply longer than that.
const KEEP_ALIVE: Duration = Duration::from_secs(10);
type Socket = tokio_tungstenite::WebSocketStream<
tokio_tungstenite::MaybeTlsStream<tokio::net::TcpStream>,
>;
type Sink = futures_util::stream::SplitSink<Socket, Ws>;
type Source = futures_util::stream::SplitStream<Socket>;
/// Speaks sentences as they arrive on `sentences`, over one websocket per
/// reply.
///
/// pcm_24000 rather than the default mp3: there is no decoder in this process
/// and no reason to add one. sync_alignment makes the character timings ride
/// along with the audio frames instead of needing a second request.
pub async fn speak(
cfg: &TtsConfig,
sentences: mpsc::Receiver<String>,
events: mpsc::Sender<SpeechEvent>,
) -> Result<()> {
if cfg.is_v3() {
speak_dialogue(cfg, sentences, events).await
} else {
speak_tts(cfg, sentences, events).await
}
}
async fn connect(url: &str, key: &str) -> Result<(Sink, Source)> {
let mut req = url.into_client_request()?;
// Header, not the ?authorization= query param: a key in a URL ends up in
// every proxy and error log that touches it.
req.headers_mut().insert(
"xi-api-key",
tokio_tungstenite::tungstenite::http::HeaderValue::from_str(key)?,
);
let (stream, _) = tokio_tungstenite::connect_async(req).await?;
Ok(stream.split())
}
async fn send_json(sink: &mut Sink, v: serde_json::Value) -> Result<()> {
sink.send(Ws::Text(v.to_string())).await?;
Ok(())
}
async fn speak_tts(
cfg: &TtsConfig,
mut sentences: mpsc::Receiver<String>,
events: mpsc::Sender<SpeechEvent>,
) -> Result<()> {
let url = format!(
"wss://api.elevenlabs.io/v1/text-to-speech/{}/stream-input\
?model_id={}&output_format=pcm_{}&sync_alignment=true&auto_mode=true",
cfg.voice_id,
cfg.model,
audio::TTS_RATE
);
let (mut sink, source) = connect(&url, &cfg.key).await?;
// The protocol requires an opening frame of a single space before any
// real text; it carries the voice settings for the whole connection.
send_json(
&mut sink,
serde_json::json!({
"text": " ",
"voice_settings": { "stability": 0.4, "similarity_boost": 0.8, "speed": 1.0 },
}),
)
.await?;
let reader = tokio::spawn(read_audio(source, events, &TTS_WIRE));
let sent: Result<()> = async {
while let Some(text) = sentences.recv().await {
// Every text frame must end with a space or the model holds the
// last word back waiting for more input.
let text = format!("{} ", text.trim());
send_json(&mut sink, serde_json::json!({ "text": text })).await?;
}
// Empty string = end of input. Without it the socket sits until the
// inactivity timeout and the last sentence never gets flushed.
send_json(&mut sink, serde_json::json!({ "text": "" })).await
}
.await;
// The server's own error beats the "connection reset" a send hit after it.
reader.await??;
sent
}
async fn speak_dialogue(
cfg: &TtsConfig,
mut sentences: mpsc::Receiver<String>,
events: mpsc::Sender<SpeechEvent>,
) -> Result<()> {
let url = format!(
"wss://api.elevenlabs.io/v1/text-to-dialogue/stream-input\
?model_id={}&output_format=pcm_{}&sync_alignment=true",
cfg.model,
audio::TTS_RATE
);
let (mut sink, source) = connect(&url, &cfg.key).await?;
// No voice_settings: v3 reads only stability, and only as one of three
// presets; the default is its "natural" one.
send_json(&mut sink, serde_json::json!({ "voices": [cfg.voice_id] })).await?;
let reader = tokio::spawn(read_audio(source, events, &DIALOGUE_WIRE));
let sent: Result<()> = async {
loop {
match tokio::time::timeout(KEEP_ALIVE, sentences.recv()).await {
Ok(Some(text)) => {
// Inputs are concatenated verbatim, so the space matters
// here too. flush: the server otherwise holds text until
// ~40 chars and 8 words, so a short first sentence would
// wait for the second.
let text = format!("{} ", text.trim());
send_json(
&mut sink,
serde_json::json!({
"inputs": [{ "text": text, "voice_id": cfg.voice_id }],
"flush": true,
}),
)
.await?
}
Ok(None) => break,
Err(_) => send_json(&mut sink, serde_json::json!({ "keep_alive": true })).await?,
}
}
send_json(&mut sink, serde_json::json!({ "close_socket": true })).await
}
.await;
reader.await??;
sent
}
async fn read_audio(
mut source: Source,
events: mpsc::Sender<SpeechEvent>,
wire: &'static Wire,
) -> Result<()> {
// Alignment times may be per-chunk or cumulative depending on the
// model; `emitted_ms` lets us normalise either into one timeline.
let mut emitted_ms: u32 = 0;
// v3 audio tags come back as characters, and one can straddle chunks.
let mut in_tag = false;
let mut result = Ok(());
while let Some(Ok(msg)) = source.next().await {
let Ws::Text(txt) = msg else { continue };
let Ok(v) = serde_json::from_str::<serde_json::Value>(&txt) else { continue };
if let Some(err) = v.get("error").and_then(|e| e.as_str()) {
let detail = v.get("message").and_then(|m| m.as_str()).unwrap_or("");
result = Err(anyhow!("elevenlabs: {err} {detail}"));
break;
}
if let Some(b64) = v.get("audio").and_then(|a| a.as_str()) {
if let Ok(bytes) = base64::engine::general_purpose::STANDARD.decode(b64) {
let pcm = audio::pcm16_to_f32(&bytes);
let chunk_ms = (pcm.len() as f64 / audio::TTS_RATE as f64 * 1000.0) as u32;
let al = wire.alignment.iter().find_map(|k| v.get(*k).filter(|a| !a.is_null()));
if let Some(t) = al.and_then(|al| alignment_to_timeline(al, emitted_ms, &mut in_tag)) {
let _ = events.send(SpeechEvent::Visemes(t)).await;
}
emitted_ms += chunk_ms;
if events.send(SpeechEvent::Audio(pcm)).await.is_err() {
return Ok(()); // receiver gone: barge-in cancelled this reply
}
}
}
if v.get(wire.final_key).and_then(|f| f.as_bool()).unwrap_or(false) {
break;
}
}
let _ = events.send(SpeechEvent::Done).await;
result
}
fn alignment_to_timeline(
al: &serde_json::Value,
emitted_ms: u32,
in_tag: &mut bool,
) -> Option<Vec<Viseme>> {
let field = |camel: &str, snake: &str| al.get(camel).or_else(|| al.get(snake));
let chars: Vec<String> = al
.get("chars")?
.as_array()?
.iter()
.map(|c| c.as_str().unwrap_or(""))
.map(|c| {
// A tag is a laugh or a sigh, not letters: the mouth rests.
let tagged = *in_tag || c == "[";
*in_tag = tagged && c != "]";
if tagged { " ".to_string() } else { c.to_string() }
})
.collect();
let starts: Vec<i64> = num_array(field("charStartTimesMs", "char_start_times_ms")?)?;
let durs: Vec<i64> = num_array(field("charDurationsMs", "char_durations_ms")?)?;
let starts = absolutize(&starts, emitted_ms);
Some(visemes::timeline(&chars, &starts, &durs))
}
fn num_array(v: &serde_json::Value) -> Option<Vec<i64>> {
Some(v.as_array()?.iter().map(|n| n.as_i64().unwrap_or(0)).collect())
}
/// Normalises chunk-relative and stream-absolute alignments into one
/// timeline. If the first start time is behind audio we have already emitted,
/// the timings must be relative to this chunk, so they get shifted.
fn absolutize(starts: &[i64], emitted_ms: u32) -> Vec<i64> {
let offset = match starts.first() {
Some(&first) if first < emitted_ms as i64 => emitted_ms as i64,
_ => 0,
};
starts.iter().map(|s| s + offset).collect()
}
/// Splits streamed text into speakable sentences, and strips what should
/// never be read aloud. Hermes' own voice mode does the same thing for the
/// same reason: without it she pronounces asterisks.
pub struct SentenceSplitter {
buf: String,
in_think: bool,
/// Whether v3 audio tags survive to the synthesiser or are dropped.
keep_tags: bool,
}
impl SentenceSplitter {
pub fn new(keep_tags: bool) -> Self {
SentenceSplitter { buf: String::new(), in_think: false, keep_tags }
}
/// Feeds a delta, returning any sentences that completed.
pub fn push(&mut self, delta: &str) -> Vec<String> {
self.buf.push_str(delta);
let mut out = Vec::new();
// <think> blocks are reasoning, not speech, and can be long.
loop {
if self.in_think {
match self.buf.find("</think>") {
Some(i) => {
self.buf.drain(..i + "</think>".len());
self.in_think = false;
}
None => {
self.buf.clear();
return out;
}
}
}
match self.buf.find("<think>") {
Some(i) => {
let head: String = self.buf.drain(..i).collect();
self.buf.drain(.."<think>".len());
self.in_think = true;
out.extend(drain_sentences(&mut { head.clone() }, true, self.keep_tags));
}
None => break,
}
}
out.extend(drain_sentences(&mut self.buf, false, self.keep_tags));
out.into_iter().filter(|s| !s.is_empty()).collect()
}
/// Whatever is left when the reply ends.
pub fn finish(&mut self) -> Option<String> {
let rest = clean_for_speech(&std::mem::take(&mut self.buf), self.keep_tags);
(!rest.is_empty()).then_some(rest)
}
}
fn drain_sentences(buf: &mut String, force: bool, keep_tags: bool) -> Vec<String> {
let mut out = Vec::new();
// A sentence is only worth sending once it is long enough to synthesise
// well; ElevenLabs' own guidance is ~20 characters minimum.
while let Some(end) = buf.find(['.', '!', '?', '\n']) {
if end < 20 && !force {
// Abbreviation or a very short clause — keep accumulating.
if buf.len() < 80 {
break;
}
}
let s: String = buf.drain(..end + 1).collect();
let s = clean_for_speech(&s, keep_tags);
if !s.is_empty() {
out.push(s);
}
}
if force && !buf.is_empty() {
let s = clean_for_speech(&std::mem::take(buf), keep_tags);
if !s.is_empty() {
out.push(s);
}
}
out
}
/// Byte index just past a v3 audio tag starting at `i` ("[laughs]",
/// "[clears throat]"), if there is one. Letters and spaces only, so
/// "[1]" or a markdown link stays ordinary noise.
fn tag_end(s: &str, i: usize) -> Option<usize> {
let rest = s[i..].strip_prefix('[')?;
let close = rest.find(']')?;
let body = &rest[..close];
let end = i + 1 + close + 1;
let ok = !body.trim().is_empty()
&& body.len() <= 40
&& body.chars().all(|c| c.is_alphabetic() || c == ' ' || c == '-')
&& !s[end..].starts_with('(');
ok.then_some(end)
}
/// The text as a person reads it: captions show "ha!", not "[laughs]".
pub fn strip_tags(s: &str) -> String {
let mut out = String::with_capacity(s.len());
let mut skip_to = 0;
for (i, c) in s.char_indices() {
if i < skip_to {
continue;
}
match tag_end(s, i) {
Some(end) => {
skip_to = end;
out.push(' ');
}
None => out.push(c),
}
}
out.split_whitespace().collect::<Vec<_>>().join(" ")
}
/// Markdown, emoji and kaomoji are for eyes.
///
/// A whitelist, not a blacklist: she writes things like "(¬‿¬)", whose
/// characters live in ordinary Latin-1 and punctuation blocks rather than in
/// any emoji range, so enumerating what to drop never converges. Letters
/// stays unicode-aware on purpose — German is in scope, "ö" is a letter.
///
/// Audio tags pass through untouched when `keep_tags`; otherwise they are
/// dropped whole, since a model without them would read "laughs" aloud.
pub fn clean_for_speech(s: &str, keep_tags: bool) -> String {
let mut out = String::with_capacity(s.len());
let mut skip_to = 0;
for (i, c) in s.char_indices() {
if i < skip_to {
continue;
}
if let Some(end) = tag_end(s, i) {
skip_to = end;
out.push(' ');
if keep_tags {
out.push_str(&s[i..end]);
out.push(' ');
}
} else if c.is_alphanumeric() || c.is_whitespace() || is_speakable_punctuation(c) {
out.push(c);
} else if matches!(c, '—' | '–') {
// " — " must not leave " , ": the comma binds to the word before it.
while out.chars().last().is_some_and(char::is_whitespace) {
out.pop();
}
// An em dash is a pause, not a space. Dropping it runs the two
// clauses together and changes what the sentence means; a comma
// is the closest thing the synthesiser understands. ASCII '-'
// deliberately does NOT land here — it is usually intra-word
// ("push-to-talk"), where a comma would be worse than a space.
out.push(',');
} else {
// Not dropped silently: a symbol between two words is usually a
// dash or a bullet, and gluing the words together mispronounces
// both.
out.push(' ');
}
}
out.split_whitespace().collect::<Vec<_>>().join(" ")
}
/// Punctuation a TTS model actually uses for prosody. Everything else is
/// noise once it reaches audio.
fn is_speakable_punctuation(c: char) -> bool {
matches!(c, '.' | ',' | '!' | '?' | ';' | ':' | '\'' | '"')
}
#[cfg(test)]
mod tests {
use super::*;
#[test]
fn absolutize_shifts_chunk_relative_times() {
assert_eq!(absolutize(&[0, 50], 1000), vec![1000, 1050]);
}
#[test]
fn absolutize_leaves_already_absolute_times() {
assert_eq!(absolutize(&[1000, 1050], 1000), vec![1000, 1050]);
}
#[test]
fn splitter_emits_complete_sentences_only() {
let mut s = SentenceSplitter::new(false);
assert!(s.push("This is a long enough first sentence").is_empty());
let got = s.push(". And more");
assert_eq!(got, vec!["This is a long enough first sentence."]);
}
#[test]
fn splitter_drops_think_blocks() {
let mut s = SentenceSplitter::new(false);
s.push("<think>internal monologue that is long</think>");
let got = s.push("Actually speakable sentence here. ");
assert_eq!(got, vec!["Actually speakable sentence here."]);
}
#[test]
fn splitter_returns_tail_on_finish() {
let mut s = SentenceSplitter::new(false);
s.push("no terminator here");
assert_eq!(s.finish().as_deref(), Some("no terminator here"));
}
#[test]
fn markdown_and_emoji_never_reach_the_voice() {
assert_eq!(clean_for_speech("**bold** `code` 🎉 ok", false), "bold code ok");
}
#[test]
fn kaomoji_and_dashes_are_stripped_without_gluing_words() {
// Her actual first reply over the API.
assert_eq!(
clean_for_speech("Loud and clear, boss — hallo! (¬‿¬)", false),
"Loud and clear, boss, hallo!"
);
}
#[test]
fn em_dash_becomes_a_pause_but_hyphens_do_not() {
assert_eq!(clean_for_speech("I can't confirm — there's nothing", false), "I can't confirm, there's nothing");
assert_eq!(clean_for_speech("push-to-talk", false), "push to talk");
}
#[test]
fn keeps_non_ascii_letters() {
assert_eq!(clean_for_speech("schön, größer", false), "schön, größer");
}
#[test]
fn audio_tags_survive_for_v3_only() {
let s = "[laughs]Fine, boss. [clears throat] Na gut.";
assert_eq!(clean_for_speech(s, true), "[laughs] Fine, boss. [clears throat] Na gut.");
assert_eq!(clean_for_speech(s, false), "Fine, boss. Na gut.");
}
#[test]
fn brackets_that_are_not_tags_stay_noise() {
assert_eq!(clean_for_speech("see [1] and [the docs](x)", true), "see 1 and the docs x");
}
#[test]
fn captions_hide_tags() {
assert_eq!(strip_tags("[sighs] Okay, boss."), "Okay, boss.");
}
#[test]
fn tag_characters_rest_the_mouth_across_chunks() {
let mut in_tag = false;
let chunk = |chars: &[&str], in_tag: &mut bool| {
let n = chars.len() as i64;
let al = serde_json::json!({
"chars": chars,
"char_start_times_ms": (0..n).map(|i| i * 100).collect::<Vec<_>>(),
"char_durations_ms": vec![100; chars.len()],
});
alignment_to_timeline(&al, 0, in_tag).unwrap()
};
let t = chunk(&["[", "l", "a"], &mut in_tag);
assert!(t.iter().all(|v| v.shape == visemes::SILENCE));
assert!(in_tag);
let t = chunk(&["u", "]", "a"], &mut in_tag);
assert!(!in_tag);
assert_eq!(t.last().unwrap().shape, "aa");
}
}
-131
View File
@@ -1,131 +0,0 @@
use serde::Serialize;
/// VRM 1.0's standard mouth expressions, which is what the avatar rig exposes
/// (`aa`/`ih`/`ou`/`ee`/`oh`). A rig with the full ARKit 52 can map these onto
/// jawOpen/mouthFunnel/etc — the shell side owns that, not this.
///
/// These names are a CONTRACT with the balsam-generated morph targets: the
/// glTF target names become the QML ids, so renaming a shape key in Blender
/// breaks the binding silently, with no eval error.
pub const SILENCE: &str = "sil";
#[derive(Debug, Clone, PartialEq, Serialize)]
pub struct Viseme {
pub shape: &'static str,
/// Milliseconds from speech_start, NOT wall clock — the shell starts its
/// own timer when the first PCM frame hits the speaker, so the two clocks
/// never have to agree.
pub at: u32,
pub ms: u32,
}
/// One character to a mouth shape. English orthography is a lie about
/// phonetics, but at 40ms of animation per character nobody can tell: what
/// reads as "talking" is the OPEN/CLOSE rhythm, not vowel accuracy.
fn shape_for(c: char) -> &'static str {
match c.to_ascii_lowercase() {
'a' => "aa",
'e' => "ee",
'i' | 'y' => "ih",
'o' => "oh",
'u' | 'w' => "ou",
// Bilabials close the mouth; giving them silence is what produces the
// visible "p/b/m" beat.
'p' | 'b' | 'm' => SILENCE,
// Everything else is a partially open consonant. "ih" is the least
// committal shape to land on.
c if c.is_alphabetic() => "ih",
_ => SILENCE,
}
}
/// Builds a playable timeline from ElevenLabs' character alignment.
///
/// The three arrays are parallel; a short or missing one means the frame was
/// truncated, so we take the shortest common prefix rather than panicking on
/// an index.
pub fn timeline(chars: &[String], starts: &[i64], durations: &[i64]) -> Vec<Viseme> {
let n = chars.len().min(starts.len()).min(durations.len());
let mut out: Vec<Viseme> = Vec::with_capacity(n);
for i in 0..n {
let c = match chars[i].chars().next() {
Some(c) => c,
None => continue,
};
let shape = shape_for(c);
let at = starts[i].max(0) as u32;
let ms = durations[i].max(0) as u32;
// Merge runs of the same shape: "ll" is one mouth position held, not
// two identical keyframes the shell would re-trigger.
match out.last_mut() {
Some(last) if last.shape == shape && last.at + last.ms >= at => {
last.ms = (at + ms).saturating_sub(last.at);
}
_ => out.push(Viseme { shape, at, ms }),
}
}
// Sub-frame keys cannot be seen and only cost socket bytes.
out.retain(|v| v.ms >= 20);
out
}
#[cfg(test)]
mod tests {
use super::*;
fn s(v: &[&str]) -> Vec<String> {
v.iter().map(|s| s.to_string()).collect()
}
#[test]
fn maps_vowels_to_distinct_shapes() {
let t = timeline(
&s(&["a", "e", "o"]),
&[0, 100, 200],
&[100, 100, 100],
);
assert_eq!(
t.iter().map(|v| v.shape).collect::<Vec<_>>(),
vec!["aa", "ee", "oh"]
);
}
#[test]
fn merges_repeated_shapes_into_one_held_key() {
let t = timeline(&s(&["l", "l", "l"]), &[0, 50, 100], &[50, 50, 50]);
assert_eq!(t.len(), 1);
assert_eq!(t[0].shape, "ih");
assert_eq!(t[0].at, 0);
assert_eq!(t[0].ms, 150);
}
#[test]
fn bilabials_close_the_mouth() {
let t = timeline(&s(&["a", "m", "a"]), &[0, 100, 200], &[100, 100, 100]);
assert_eq!(t[1].shape, SILENCE);
}
#[test]
fn drops_keys_too_short_to_see() {
let t = timeline(&s(&["a", "e"]), &[0, 5], &[5, 100]);
assert_eq!(t.len(), 1);
assert_eq!(t[0].shape, "ee");
}
#[test]
fn tolerates_truncated_arrays() {
let t = timeline(&s(&["a", "e", "o"]), &[0, 100], &[100]);
assert_eq!(t.len(), 1);
}
#[test]
fn punctuation_and_spaces_are_silence() {
let t = timeline(&s(&[" ", ",", "!"]), &[0, 100, 200], &[100, 100, 100]);
assert_eq!(t.len(), 1);
assert_eq!(t[0].shape, SILENCE);
assert_eq!(t[0].ms, 300);
}
}
+58 -28
View File
@@ -1,56 +1,86 @@
# Mnemosyne memory provider for Hermes. Built here rather than pip-installed:
# the image's Python has no pip and is PEP 668 managed.
# Mnemosyne memory provider for Hermes on mars — packaged for real (Nix).
#
# Core deps only: the embeddings extra (fastembed/onnxruntime) is optional at
# runtime, and recall falls back to FTS5.
# Why derivations instead of a runtime side-venv: the official Hermes image
# vendors its own Python and stays off-limits to pip (no pip module, PEP 668),
# and a runtime venv built host-side breaks twice over inside the container:
# the venv's pyvenv.cfg records a /nix/store python home the container never
# mounts, and a plugins symlink with an absolute host path points nowhere
# from /opt/data. Building here means nothing is fetched at boot, nothing
# under the provider's control is writable from inside the container, and
# the closure is as reproducible as the rest of the host.
#
# Package set (one shared site-packages — the plugin wrapper imports its
# sibling `mnemosyne` core package, so withPackages, not separate envs):
#
# mnemosyne-memory core engine: SQLite/FTS5 storage, recall, tools.
# Base deps only (PyYAML); the optional extras (llm,
# embeddings via fastembed/onnxruntime, mcp, sync) are
# deliberately NOT pulled — recall uses the bundled FTS5
# lexical path, and the heavyweight ML stack (~hundreds of
# MB, live network on first vector use) buys nothing for
# a first deployment. Adding the embeddings extra later
# is pinning fastembed + sqlite-vec here.
# mnemosyne-hermes the wrapper Hermes discovers under $HERMES_HOME/plugins
# (installs itself as package `hermes_memory_provider`).
# Hard dependency: mnemosyne-memory, PyYAML.
#
# Platform note: both sdists are pure Python (build no C extensions), so
# nothing here constrains the host arch beyond the interpreter itself.
{
python3,
fetchPypi,
}:
let
mnemosyneMemory = python3.pkgs.buildPythonPackage rec {
python = python3;
mnemosyneMemory = python.pkgs.buildPythonPackage rec {
pname = "mnemosyne-memory";
version = "3.15.1";
pyproject = true;
src = fetchPypi {
pname = "mnemosyne_memory";
inherit version;
hash = "sha256-lspUMxc0pUSkhSUrNdiiO5OJ1NMC/S853EYSanXtXKM=";
inherit pname version;
sha256 = "sha256-lspUMxc0pUSkhSUrNdiiO5OJ1NMC/S853EYSanXtXKM=";
};
build-system = with python3.pkgs; [ setuptools ];
dependencies = with python3.pkgs; [ pyyaml ];
build-system = with python.pkgs; [ setuptools ];
doCheck = false; # tests want a live Hermes + LLM key
# Ships the Hermes plugin package too, not just the core.
pythonImportsCheck = [
"mnemosyne"
"hermes_memory_provider"
];
# Base dependency set — everything else in the upstream metadata is an
# optional extra (llm / embeddings / mcp / sync / test / dev) and is not
# installed; see the file-level comment.
dependencies = with python.pkgs; [ pyyaml ];
doCheck = false; # upstream tests want a live Hermes + LLM key present
pythonImportsCheck = [ "mnemosyne" ];
};
mnemosyneHermes = python3.pkgs.buildPythonPackage rec {
mnemosyneHermes = python.pkgs.buildPythonPackage rec {
pname = "mnemosyne-hermes";
version = "0.5.0";
pyproject = true;
src = fetchPypi {
pname = "mnemosyne_hermes";
inherit version;
hash = "sha256-CzEvnUw5oPFtT5bHQQ/GBdy2C/E7qShQn32irIRYKqw=";
inherit pname version;
sha256 = "sha256-CzEvnUw5oPFtT5bHQQ/GBdy2C/E7qShQn32irIRYKqw=";
};
build-system = with python3.pkgs; [ setuptools ];
# Upstream asks for mnemosyne-memory[embeddings]; see the header.
dependencies = [ mnemosyneMemory ] ++ (with python3.pkgs; [ pyyaml ]);
build-system = with python.pkgs; [ setuptools ];
# The wrapper declares `mnemosyne-memory[embeddings]>=3.11.1` on PyPI, but
# the embeddings extra is only consulted when vector recall is enabled
# (see above) — pass the core dependency explicitly rather than dragging
# in onnxruntime for nothing.
dependencies = [ mnemosyneMemory ] ++ (with python.pkgs; [ pyyaml ]);
doCheck = false;
pythonImportsCheck = [ "mnemosyne_hermes" ];
pythonImportsCheck = [ "hermes_memory_provider" ];
};
# The exposed value is the python env itself (a store path mounted :ro).
# Hermes only needs the site-packages dir inside it; `sitePackages` is a
# passthru the python interpreter derivation (and hence withPackages envs)
# exposes, so the caller uses `${env.sitePackages}` instead of guessing
# the python version in a path literal.
in
python3.withPackages (_: [
mnemosyneMemory
mnemosyneHermes
])
python.withPackages (_: [ mnemosyneMemory mnemosyneHermes ])
+2 -4
View File
@@ -8,8 +8,6 @@ gitea_luna_token: ENC[AES256_GCM,data:0ypW9oVFs1mXYPhPareMFRdkSYcvHSCm+fQOd7/76l
gitea_hermes_webhook_secret: ENC[AES256_GCM,data:lV78H0xAehPxusSO/QruOYkt7fkMJrW+ScZL4UWYvgnBGn/D+1XHYPyHCqe2sEEWSlIaAgWMMoZzoVJ1Z1NFVQ==,iv:GmTZxoH2iiL/vTVgPfziXIFYD+Rl3cbh9hqXvWps+iw=,tag:jtXEUOVFfKrpTRK7S9PZMA==,type:str]
couchdb_luna_password: ENC[AES256_GCM,data:V91is2h7UskI1rtwMzQyduNXoDPTYNwa4sw9K9WU+wE=,iv:k976ImKR19+CvGOVsHsrqMaFFtSQxVi6zABCJuQ5AWE=,tag:W2SN4SET/+o4+Wt6BOJ1LA==,type:str]
obsidian_luna_passphrase: ENC[AES256_GCM,data:fqHtP3g4J40ddYL9lzCixrisdC/DEJJermE=,iv:FU6BGNcBnyP8Rz3dNBk0+K0aAQTDW6/0aVaFm1rBFkk=,tag:Mg1aKqh++2rmU8ROaVIgDw==,type:str]
elevenlabs_api_key: ENC[AES256_GCM,data:2oJeClO4/SFsbLwRgMe3bFXrmuDX7YOc57VfSjElV5A3vaIQqfN9eHzo/cm0vISxYgUv,iv:eIeKQJ6odDnmGLjHSLV3HbErVnLr7tatOtV83TvB2QA=,tag:XPkXRfDJ2ZK98TYXKeK/CA==,type:str]
hermes_api_server_key: ENC[AES256_GCM,data:Au80iziwdryFTCLnggC9DdjjZPrGPKpmRFgkXe/l9RQ7on9nXb1dvQ==,iv:hWvv4fLkhRgVTT2btI+TvpNBHfMOg9y4BQKn+ht7XDo=,tag:2dEZEys8vsTwc7w0ZiBK1w==,type:str]
sops:
age:
- enc: |
@@ -30,7 +28,7 @@ sops:
oyJ7PS3lW+PxH5AZkeeU7gXO/pz2oDku0aDOds7kaD3n0+qSWicQ+Q==
-----END AGE ENCRYPTED FILE-----
recipient: age1eapjg6tdrr0fuvmgs3q3nlvnjkaxez298qynqqqxt0lpcv0lrsyq7ayxjk
lastmodified: "2026-09-20T20:02:50Z"
mac: ENC[AES256_GCM,data:xJVgl094qzW8YyjwDvRw4ymKjUhL3uWtbVN6J25S1Gi2I2siZ17MiUUis3dKXxb3OKq0NODP9/4pJDqWPv1qt88inu3l/6QUQMY6AybbGF7ooiiOSvGhR3jnCHpISptepNLzyHu0Iky6PkTLe5f5ETSolZ5R+Qyd66ZuFmUBjCM=,iv:u8XWCyjJo/ib9VSzi/e1FEbSjod98c1i14LykC7BEtw=,tag:ZYtrw0b3iEI+yaRtATgbkQ==,type:str]
lastmodified: "2026-08-25T21:51:24Z"
mac: ENC[AES256_GCM,data:dz249hf3w8Tn0JStFOhhpdCZFMx2yxmNABx1CbeIQ/JlICAU82e4fg8AzJQY9EMOEs3Zx6L61yieljD4A/HLip5rDVlAXqqLeklW60eb7BHuSO79YfFgot+rS05g8WqFkRJYWzmkXhMbErCI133n72XEdeqMUU/m0djOUZlVRLs=,iv:d7G6TJRLfmXvQ2BUG9Hi83lOB9anhmb9qneLeVSCBhc=,tag:F6X/0z00PjiYum5kf8YApA==,type:str]
unencrypted_suffix: _unencrypted
version: 3.13.3
+2 -4
View File
@@ -5,8 +5,6 @@ librechat_creds_key: ENC[AES256_GCM,data:e2Ptf41yHu0KxfzjW4DP04CSJfbtdsJ8bwrgJyv
librechat_creds_iv: ENC[AES256_GCM,data:iuW1Rxfu7Ei8zHVG7wsBTYYznbtErj7DC9B71OiiWGA=,iv:Ngs88C44gcSzxQZU1lMiy5kn9mM/ckuWKRRGOFJeeoQ=,tag:Q9B8vM4YMkIP3RyYVWSQqg==,type:str]
librechat_jwt_secret: ENC[AES256_GCM,data:f/gljrQPZIXeLHXtqKCCYGEu2pXgbZd69CjyEhSVJ+AMuaVj7DWVclD5IMsEVERBHw5a7OndBSfLvgXRaKN6Gg==,iv:DG5CwCbQLCTv1++APCdFzAiWGKiEJl+9MhKIM2ykJQM=,tag:wdpY6l+5h4UtsluIncsyhA==,type:str]
librechat_jwt_refresh_secret: ENC[AES256_GCM,data:N/yAPtaodJX4t2C2A0bZ/LQaVc8SBMgg3KYJrWRyXCM6HAmgHtSQ9nWJO2fl+7A966mMtYvpATYcBIPj/K5nig==,iv:k5ZYi2OKjL9Z+lhVgdNsmGVQm+iCqmF15H3TUTbofWk=,tag:NVTdvlL1lpwVhjo2c2phpg==,type:str]
hermes_api_server_key: ENC[AES256_GCM,data:NU+6zVmUeM9+m5eZCPdCiMLo76GqbsRxNM7EmVM2TmKTOUL2af7Msg==,iv:EMd7JKhQ3ZkFPi2xMGZ+mHHK05qQVDR+ZCbiYcWdhHU=,tag:JaElU4CyvD5qQbehHbM5qQ==,type:str]
elevenlabs_api_key: ENC[AES256_GCM,data:4IBKKMxor1sJ1skh1pgLrQnNX31cgQFiy89O1TZn71QHaq+xY7p+7JLtmGkhbZ10l12m,iv:Ww5RLYuv09xD1deFgM89u0wZ3hZ/0Pndda0kreMx2aQ=,tag:opgIPsDwSc+rLqseqjP6Gg==,type:str]
sops:
age:
- enc: |
@@ -27,7 +25,7 @@ sops:
sHjKfw8VrrmAR4pQf1dsY+wcyh4FsZxhP3Q+QIVq3eCIXS9PeJkGAg==
-----END AGE ENCRYPTED FILE-----
recipient: age1rfcmu6zh40v4260l9hnf8ajs9vly0s06rx3ey76eu78dp9t7getqyhmkut
lastmodified: "2026-09-20T20:02:50Z"
mac: ENC[AES256_GCM,data:fEgOKCOYrpnojw9yl3b7ZgJFCcmAs+DmY7etHgD5KNBtoZ3QgQUiTY5+kXvO5HMoENve+bUfyHymUDNjLlb45lItl9ZkAMKvTxGOe+5t0MR2X8BkB/2tjl/fS7CRBTJ0D4S84mH1hB2V2cfKcnGoKNsQyRJddi6OOey534QeV44=,iv:nQuRLkGklMXwkh8LpMC/Tmz1ewRiv8iKbK6IA88r2fA=,tag:LnzQc1sjrIdZ26AtH1Laxw==,type:str]
lastmodified: "2026-08-21T23:14:15Z"
mac: ENC[AES256_GCM,data:gD5C3hYFvIUP7zr4GK40LAtM2sskhGErEzdTxVKaRaPkKNYj+pDYd5uSrBxDHv6W2SAY0TtyN3GyNpD0z5b4yaDxiV8ETUOokkj8f9FlTaZk56S6I/21HvaGXW+Cfi1ioFtNGyJeAhSnNOJ6OE0AA9JfcftSJ8BdTe2LienjHwQ=,iv:LtKFum+f1kaE6/XT4Xj0ZEuFrhSXXsnILMgdebGYNDU=,tag:6DxwNjjgze2ldU1g3SdzzA==,type:str]
unencrypted_suffix: _unencrypted
version: 3.13.2
-20
View File
@@ -1,20 +0,0 @@
{ pkgs, ... }:
let
claude-desktop = pkgs.callPackage ../../pkgs/claude-desktop.nix { };
in
# Claude Desktop, including Cowork (a QEMU/KVM VM the app runs agent tasks in).
{
environment.systemPackages = [ claude-desktop ];
# The app only probes these Debian paths, never $PATH or an env var.
systemd.tmpfiles.rules = [
"L+ /usr/share/OVMF/OVMF_CODE_4M.fd - - - - ${pkgs.OVMF.fd}/FV/OVMF_CODE.fd"
"L+ /usr/share/OVMF/OVMF_VARS_4M.fd - - - - ${pkgs.OVMF.fd}/FV/OVMF_VARS.fd"
"L+ /usr/libexec/virtiofsd - - - - ${claude-desktop}/lib/claude-desktop/resources/virtiofsd"
];
boot.kernelModules = [ "vhost_vsock" ];
users.users.darman.extraGroups = [ "kvm" ];
}
-3
View File
@@ -16,7 +16,6 @@ in
"steam-unwrapped"
"steam-run"
"claude-code"
"claude-desktop" # services/desktop/claude-desktop.nix
"proton-pass-cli"
"vivaldi"
"mongodb" # librechat's local db (services/desktop/librechat.nix) — SSPL
@@ -46,8 +45,6 @@ in
dotnetCorePackages.sdk_10_0
nodejs
yaak # desktop API client (REST/GraphQL/gRPC)
# pkgsRocm, not plain blender: Cycles needs HIP to render on the RX 6800 XT.
pkgsRocm.blender
];
fonts.packages = [ pkgs.nerd-fonts.departure-mono ];
-169
View File
@@ -1,169 +0,0 @@
{ config, pkgs, lib, ... }:
# Voice loop for luna: SUPER+V opens the mic, talk, hear her answer.
#
# mic -> whisper-cpp (local, vulkan) -> Hermes' OpenAI-compatible API on mars
# -> ElevenLabs streaming TTS -> speaker. State, amplitude, captions and a
# viseme timeline are published on a unix socket; the rail's LunaPanel reads
# them (dotfiles/quickshell/HyprChrome/Widgets/Bar/Luna), and the viseme
# stream is there for the avatar that replaces its meter later.
#
# A USER service, not a system one: it needs darman's pipewire session and his
# XDG_RUNTIME_DIR for the socket. That is also why the two secrets it reads
# are owner-darman in hosts/terra/secrets.nix — a user manager cannot read a
# root-owned 0400 file, and LoadCredential does not bridge that gap.
#
# STT is LOCAL on purpose: mars already runs faster-whisper for Telegram voice
# notes, but routing the mic through it would put every spoken word on the
# network for no latency win. This is the client-direct path Hermes itself
# documents.
let
cfg = config.homelab.lunaVoice;
luna-voice = pkgs.callPackage ../../pkgs/luna-voice.nix { };
# Pinned like the KittenTTS models on mars: a model that re-downloads at
# first use is a service that fails on a fresh install with no network.
whisperModel = pkgs.fetchurl {
url = "https://huggingface.co/ggerganov/whisper.cpp/resolve/5359861c739e955e79d9a303bcbc70fb988958b1/ggml-base.en.bin";
sha256 = "sha256-oDd5yG3zMjB19eeWyyzlAp8A7Ihp7uP9+4l6/jbG0AI=";
};
in
{
options.homelab.lunaVoice = {
enable = lib.mkEnableOption "luna voice daemon";
user = lib.mkOption {
type = lib.types.str;
default = "darman";
description = "User whose session runs the daemon.";
};
hermesUrl = lib.mkOption {
type = lib.types.str;
default = "http://mars.orbit.sol:8642";
description = "Base URL of luna's Hermes API server (tailnet-only).";
};
model = lib.mkOption {
type = lib.types.str;
default = "hermes-agent";
description = "Model name to request; Hermes routes it to her configured provider.";
};
voiceId = lib.mkOption {
type = lib.types.str;
description = "ElevenLabs voice id — luna's voice.";
};
ttsModel = lib.mkOption {
type = lib.types.str;
default = "eleven_v3_conversational";
description = ''
eleven_v3* streams over the Text to Dialogue socket and performs audio
tags ([laughs], [sighs]); it also gets her German right, which Flash
reads with an English accent. Anything else uses the TTS socket, e.g.
"eleven_flash_v2_5" — half the price per character, faster, no tags.
'';
};
hermesKeyFile = lib.mkOption {
type = lib.types.path;
description = "File holding the API_SERVER_KEY bearer token.";
};
elevenLabsKeyFile = lib.mkOption {
type = lib.types.path;
description = "File holding the ElevenLabs API key.";
};
# Mic tuning. Defaults are set for terra's AIR 192|4, which is QUIET:
# ~0.002 ambient against ~0.0045 for normal speech. Raising the gain on
# the interface is worth more than any value here — everything downstream,
# whisper included, works better with headroom.
inputDevice = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
description = "Substring of the input device name; null uses the system default.";
};
vadMult = lib.mkOption {
type = lib.types.float;
default = 1.8;
description = "Speech triggers above noise_floor * this.";
};
vadFloor = lib.mkOption {
type = lib.types.float;
default = 0.0025;
description = "Absolute lower bound on the trigger, whatever the floor says.";
};
silenceMs = lib.mkOption {
type = lib.types.int;
default = 2000;
description = "Silence that ends an utterance.";
};
voiceMarker = lib.mkOption {
type = lib.types.str;
default = "[voice]";
description = ''
Prefixed to every spoken message so luna can tell this channel from
Telegram or the dashboard, and adjust — no kaomoji, no markdown, short
answers. Put the rule in HER profile on mars rather than only in
systemPrompt below: the marker travels with the message, so it still
identifies the channel in whatever history Hermes keeps, and one place
governs her voice manners across every client that sends it.
Empty string disables the marker.
'';
};
systemPrompt = lib.mkOption {
type = lib.types.nullOr lib.types.str;
default = null;
description = ''
Prepended to the conversation. Worth setting to something like "You
are being spoken to out loud; keep replies short" — her Telegram
register is too long-form to listen to.
'';
};
};
config = lib.mkIf cfg.enable {
environment.systemPackages = [ luna-voice ];
systemd.user.services.luna-voiced = {
description = "luna voice loop";
wantedBy = [ "graphical-session.target" ];
after = [ "graphical-session.target" "pipewire.service" ];
partOf = [ "graphical-session.target" ];
environment = {
LUNA_HERMES_URL = cfg.hermesUrl;
LUNA_MODEL = cfg.model;
LUNA_VOICE_ID = cfg.voiceId;
LUNA_TTS_MODEL = cfg.ttsModel;
LUNA_HERMES_KEY_FILE = toString cfg.hermesKeyFile;
LUNA_ELEVENLABS_KEY_FILE = toString cfg.elevenLabsKeyFile;
LUNA_WHISPER_BIN = "${pkgs.whisper-cpp-vulkan}/bin/whisper-cli";
LUNA_WHISPER_MODEL = whisperModel;
LUNA_VAD_MULT = toString cfg.vadMult;
LUNA_VAD_FLOOR = toString cfg.vadFloor;
LUNA_SILENCE_MS = toString cfg.silenceMs;
LUNA_VOICE_MARKER = cfg.voiceMarker;
} // lib.optionalAttrs (cfg.systemPrompt != null) {
LUNA_SYSTEM_PROMPT = cfg.systemPrompt;
} // lib.optionalAttrs (cfg.inputDevice != null) {
LUNA_INPUT_DEVICE = cfg.inputDevice;
};
serviceConfig = {
ExecStart = "${luna-voice}/bin/luna-voice daemon";
Restart = "on-failure";
RestartSec = 3;
};
};
};
}
-26
View File
@@ -1,26 +0,0 @@
{ pkgs, ... }:
# gitea-mcp in HTTP mode, tailnet-only at http://jupiter.orbit.sol:8093/mcp.
# It holds no token: each client sends its own PAT as `Authorization: Bearer
# <token>`, so an unauthenticated request can only see what anonymous gitea
# users can.
{
systemd.services.gitea-mcp = {
description = "Gitea MCP server";
after = [ "gitea.service" ];
wants = [ "gitea.service" ];
wantedBy = [ "multi-user.target" ];
environment.GITEA_HOST = "http://localhost:3000";
serviceConfig = {
# Binds every interface (no listen-address flag); tailnet-only because
# tailscale0 is trusted and 8093 stays closed on the LAN.
ExecStart = "${pkgs.gitea-mcp-server}/bin/gitea-mcp -t http -p 8093";
DynamicUser = true;
Restart = "on-failure";
ProtectSystem = "strict";
ProtectHome = true;
PrivateTmp = true;
NoNewPrivileges = true;
};
};
}