terra: ollama (ROCm) + LibreChat with local persistent memory
Local LLM server on the 6800 XT (ollama-rocm, gfx1030 needs no HSA_OVERRIDE_GFX_VERSION) fronted by a LibreChat web UI, talking to it over the OpenAI-compatible /v1 route. Also wires up LibreChat's persistent-memory feature, which needed its own agent+model plus a custom extraction prompt: the default 3b model couldn't reliably tell the user's stated facts apart from its own boilerplate, and even a tuned prompt didn't fix that — so memory extraction now reuses gemma4:12b, the same model as the daily-driver chat endpoint. flake.lock bump pulls in the ollama and librechat NixOS modules. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
This commit is contained in:
@@ -21,4 +21,14 @@
|
||||
username=darman
|
||||
password=${config.sops.placeholder.samba_password}
|
||||
'';
|
||||
|
||||
# LibreChat's CREDS_KEY/IV encrypt stored user credentials (linked 3rd-party
|
||||
# API keys etc) at rest in mongo; JWT_SECRET/JWT_REFRESH_SECRET sign session
|
||||
# tokens. All four are random, generated once with `sops --set` (see
|
||||
# CLAUDE.md) — losing/rotating them just invalidates existing sessions and
|
||||
# any saved per-user API keys, nothing else depends on their value.
|
||||
sops.secrets.librechat_creds_key = { };
|
||||
sops.secrets.librechat_creds_iv = { };
|
||||
sops.secrets.librechat_jwt_secret = { };
|
||||
sops.secrets.librechat_jwt_refresh_secret = { };
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user