terra: add Hermes Agent, wired to local ollama
Points Nous Research's Hermes Agent at terra's own ROCm ollama server (gemma4:12b) as a custom OpenAI-compatible provider instead of a cloud key. Native systemd mode via the hermes-agent flake's own NixOS module — simpler than container mode, avoids the podman-rootful-sudo dance its docs call out. Also bumps OLLAMA_CONTEXT_LENGTH (and Hermes' mirrored model.context_length) from ollama's ~4k default to 131072, load-tested with real multi-ten- thousand-token prompts rather than just idle `ollama ps` checks — chosen as the practical ceiling where VRAM headroom and prefill throughput both start visibly degrading, not just the largest number that technically fit. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,36 @@
|
||||
{ ... }:
|
||||
|
||||
# Hermes Agent (https://github.com/NousResearch/hermes-agent) — Nous
|
||||
# Research's tool-calling CLI/gateway agent, pointed at the local ollama
|
||||
# server (see hosts/terra/configuration.nix) over its OpenAI-compatible /v1
|
||||
# route instead of a cloud provider. The nixosModules.default this pulls in
|
||||
# comes from the hermes-agent flake input itself (flake.nix), not from
|
||||
# nixpkgs — it builds the agent from source via uv2nix, so there's no
|
||||
# curl-|-bash installer or runtime pip/npm involved.
|
||||
#
|
||||
# Native mode (container.enable stays false): a hardened systemd service is
|
||||
# enough here since the agent only needs to reach loopback ollama, and it
|
||||
# avoids the podman-rootful-needs-passwordless-sudo dance the module's docs
|
||||
# call out for container mode. addToSystemPackages puts `hermes` on darman's
|
||||
# PATH sharing state (sessions, memories, cron) with the service instead of
|
||||
# starting a second, disconnected ~/.hermes.
|
||||
{
|
||||
services.hermes-agent = {
|
||||
enable = true;
|
||||
addToSystemPackages = true;
|
||||
|
||||
settings.model = {
|
||||
provider = "custom";
|
||||
base_url = "http://127.0.0.1:11434/v1";
|
||||
# gemma4:12b, not qwen3.6:35b-a3b: same daily-driver model already
|
||||
# loaded for librechat (services/desktop/librechat.nix) — reusing it
|
||||
# means no second model has to swap into the 6800 XT's 16G VRAM
|
||||
# alongside whatever's already resident.
|
||||
default = "gemma4:12b";
|
||||
# No api_key — ollama's OpenAI-compat endpoint doesn't check one.
|
||||
# Mirrors OLLAMA_CONTEXT_LENGTH set on the ollama service itself;
|
||||
# this is a client-side hint only, ollama enforces the real limit.
|
||||
context_length = 131072;
|
||||
};
|
||||
};
|
||||
}
|
||||
Reference in New Issue
Block a user