darmanandClaude Sonnet 5
969bd69d8d
Points Nous Research's Hermes Agent at terra's own ROCm ollama server (gemma4:12b) as a custom OpenAI-compatible provider instead of a cloud key. Native systemd mode via the hermes-agent flake's own NixOS module — simpler than container mode, avoids the podman-rootful-sudo dance its docs call out. Also bumps OLLAMA_CONTEXT_LENGTH (and Hermes' mirrored model.context_length) from ollama's ~4k default to 131072, load-tested with real multi-ten- thousand-token prompts rather than just idle `ollama ps` checks — chosen as the practical ceiling where VRAM headroom and prefill throughput both start visibly degrading, not just the largest number that technically fit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
37 lines
1.6 KiB
Nix
37 lines
1.6 KiB
Nix
{ ... }:
|
|
|
|
# Hermes Agent (https://github.com/NousResearch/hermes-agent) — Nous
|
|
# Research's tool-calling CLI/gateway agent, pointed at the local ollama
|
|
# server (see hosts/terra/configuration.nix) over its OpenAI-compatible /v1
|
|
# route instead of a cloud provider. The nixosModules.default this pulls in
|
|
# comes from the hermes-agent flake input itself (flake.nix), not from
|
|
# nixpkgs — it builds the agent from source via uv2nix, so there's no
|
|
# curl-|-bash installer or runtime pip/npm involved.
|
|
#
|
|
# Native mode (container.enable stays false): a hardened systemd service is
|
|
# enough here since the agent only needs to reach loopback ollama, and it
|
|
# avoids the podman-rootful-needs-passwordless-sudo dance the module's docs
|
|
# call out for container mode. addToSystemPackages puts `hermes` on darman's
|
|
# PATH sharing state (sessions, memories, cron) with the service instead of
|
|
# starting a second, disconnected ~/.hermes.
|
|
{
|
|
services.hermes-agent = {
|
|
enable = true;
|
|
addToSystemPackages = true;
|
|
|
|
settings.model = {
|
|
provider = "custom";
|
|
base_url = "http://127.0.0.1:11434/v1";
|
|
# gemma4:12b, not qwen3.6:35b-a3b: same daily-driver model already
|
|
# loaded for librechat (services/desktop/librechat.nix) — reusing it
|
|
# means no second model has to swap into the 6800 XT's 16G VRAM
|
|
# alongside whatever's already resident.
|
|
default = "gemma4:12b";
|
|
# No api_key — ollama's OpenAI-compat endpoint doesn't check one.
|
|
# Mirrors OLLAMA_CONTEXT_LENGTH set on the ollama service itself;
|
|
# this is a client-side hint only, ollama enforces the real limit.
|
|
context_length = 131072;
|
|
};
|
|
};
|
|
}
|