Add project documentation and reference materials
Include Luna AI Assistant design docs covering channels, configuration, core architecture, memory, scheduler, and skills. Add reference docs from OpenClaw and ZeroClaw projects, plus Mistral and OpenAI API specs.
This commit is contained in:
@@ -0,0 +1,152 @@
|
||||
# Agent Loop & Tools
|
||||
|
||||
The ZeroClaw agent architecture is built around a recursive execution loop that allows an LLM to invoke external capabilities, observe their results, and refine its response until a final answer is reached.
|
||||
|
||||
***
|
||||
|
||||
## Tool Trait
|
||||
|
||||
The `Tool` trait in `tools/traits.rs` defines the interface for any capability exposed to the agent. It provides both the metadata needed for LLM registration and the execution logic for the tool itself.
|
||||
|
||||
```rust
|
||||
/// Result of a tool execution
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct ToolResult {
|
||||
pub success: bool,
|
||||
pub output: String,
|
||||
pub error: Option<String>,
|
||||
}
|
||||
|
||||
/// Description of a tool for the LLM
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct ToolSpec {
|
||||
pub name: String,
|
||||
pub description: String,
|
||||
pub parameters: serde_json::Value,
|
||||
}
|
||||
|
||||
/// Core tool trait — implement for any capability
|
||||
#[async_trait]
|
||||
pub trait Tool: Send + Sync {
|
||||
/// Tool name (used in LLM function calling)
|
||||
fn name(&self) -> &str;
|
||||
|
||||
/// Human-readable description
|
||||
fn description(&self) -> &str;
|
||||
|
||||
/// JSON schema for parameters
|
||||
fn parameters_schema(&self) -> serde_json::Value;
|
||||
|
||||
/// Execute the tool with given arguments
|
||||
async fn execute(&self, args: serde_json::Value) -> anyhow::Result<ToolResult>;
|
||||
|
||||
/// Get the full spec for LLM registration
|
||||
fn spec(&self) -> ToolSpec {
|
||||
ToolSpec {
|
||||
name: self.name().to_string(),
|
||||
description: self.description().to_string(),
|
||||
parameters: self.parameters_schema(),
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Agent Loop
|
||||
|
||||
The core logic resides in `agent/loop_.rs` within `run_tool_call_loop`. This function orchestrates the interaction between the provider and the tool registry.
|
||||
|
||||
The process follows a repeating cycle:
|
||||
1. Send the current conversation history and tool definitions to the LLM.
|
||||
2. Receive a response.
|
||||
3. Parse the response for tool calls (native or prompt-guided).
|
||||
4. If tool calls exist, execute them and append the results to the history, then repeat.
|
||||
5. If no tool calls exist, return the final text response.
|
||||
|
||||
### Iteration Safety
|
||||
To prevent infinite loops or runaway execution, the system enforces a strict iteration cap:
|
||||
* `DEFAULT_MAX_TOOL_ITERATIONS` = 10
|
||||
|
||||
### Streaming and Progress
|
||||
Streaming responses are handled by accumulating chunks before relaying them to the draft channel to minimize noise:
|
||||
* `STREAM_CHUNK_MIN_CHARS` = 80
|
||||
* `PROGRESS_MIN_INTERVAL_MS` = 500 (minimum time between progress updates)
|
||||
* `DRAFT_CLEAR_SENTINEL` = `\x00CLEAR\x00` (used to clear progress lines before showing the final answer)
|
||||
|
||||
***
|
||||
|
||||
## Auto-Compaction
|
||||
|
||||
Conversation history is managed through an auto-compaction mechanism to stay within context window limits while preserving essential information.
|
||||
|
||||
* **Threshold**: Triggered when non-system message count exceeds 50 (`DEFAULT_MAX_HISTORY_MESSAGES`).
|
||||
* **Retention**: Keeps the 20 most recent messages (`COMPACTION_KEEP_RECENT_MESSAGES`).
|
||||
* **Summarization**: Older messages are summarized by the LLM.
|
||||
* **Caps**: Summarization source is capped at 12,000 characters; the resulting summary is capped at 2,000 characters.
|
||||
|
||||
***
|
||||
|
||||
## Security & Pattern Matching
|
||||
|
||||
### Credential Scrubbing
|
||||
ZeroClaw proactively redacts sensitive information from tool outputs before they are fed back into the LLM history.
|
||||
|
||||
```rust
|
||||
static SENSITIVE_KV_REGEX: LazyLock<Regex> = LazyLock::new(|| {
|
||||
Regex::new(r#"(?i)(token|api[_-]?key|password|secret|user[_-]?key|bearer|credential)["']?\s*[:=]\s*(?:"([^"]{8,})"|'([^']{8,})'|([a-zA-Z0-9_\-\.]{8,}))"#).unwrap()
|
||||
});
|
||||
```
|
||||
The scrubbing logic preserves the first 4 characters for context then appends `*[REDACTED]`.
|
||||
|
||||
### Autosave
|
||||
Meaningful exchanges are automatically persisted to memory based on message length:
|
||||
* `AUTOSAVE_MIN_MESSAGE_CHARS` = 20
|
||||
|
||||
***
|
||||
|
||||
## Tool Serialization & Formats
|
||||
|
||||
Tools are converted into the appropriate format for the specific LLM provider.
|
||||
|
||||
### tools_to_openai_format()
|
||||
This function serializes the tool registry into the standard OpenAI function-calling schema used by many providers.
|
||||
|
||||
```rust
|
||||
fn tools_to_openai_format(tools_registry: &[Box<dyn Tool>]) -> Vec<serde_json::Value> {
|
||||
tools_registry
|
||||
.iter()
|
||||
.map(|tool| {
|
||||
serde_json::json!({
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": tool.name(),
|
||||
"description": tool.description(),
|
||||
"parameters": tool.parameters_schema()
|
||||
}
|
||||
})
|
||||
})
|
||||
.collect()
|
||||
}
|
||||
```
|
||||
|
||||
### Parsing Priority
|
||||
The system handles multiple response formats to support different providers:
|
||||
1. Native structured JSON (OpenAI style)
|
||||
2. XML tags (`<tool_call>`, `<invoke>`)
|
||||
3. Markdown code blocks with `tool_call` identifiers
|
||||
4. Provider-specific shortened formats (GLM, etc.)
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
Luna currently operates as a pure chat relay, streaming LLM responses without any capability to interact with the system or external APIs. This lack of a tool execution layer is the primary bottleneck preventing Luna from becoming an autonomous assistant.
|
||||
|
||||
### Bridging the Gap
|
||||
Implementing the agent loop pattern in Luna would involve:
|
||||
* Defining an `ITool` interface in C# that mirrors the Rust `Tool` trait.
|
||||
* Transitioning from a single request-response model to a recursive loop in the [[Core]] engine.
|
||||
* Integrating the tool loop with [[Skills]] to allow dynamic capability discovery.
|
||||
|
||||
The implementation of this protocol will turn Luna from a passive interface into an active participant capable of executing commands and managing its own [[Memory]].
|
||||
@@ -0,0 +1,132 @@
|
||||
# Channel Messages
|
||||
|
||||
The ZeroClaw messaging protocol is built around the `Channel` trait and a unified message format. This allows the system to treat disparate platforms—from Discord to Slack to SMS—as a consistent stream of `ChannelMessage` objects.
|
||||
|
||||
***
|
||||
|
||||
## Channel Trait
|
||||
|
||||
The `Channel` trait is the core abstraction for all platform implementations. It defines how the system interacts with a specific messaging service.
|
||||
|
||||
```rust
|
||||
#[async_trait]
|
||||
pub trait Channel: Send + Sync {
|
||||
/// Human-readable channel name
|
||||
fn name(&self) -> &str;
|
||||
|
||||
/// Send a message through this channel
|
||||
async fn send(&self, message: &SendMessage) -> anyhow::Result<()>;
|
||||
|
||||
/// Start listening for incoming messages (long-running)
|
||||
async fn listen(&self, tx: tokio::sync::mpsc::Sender<ChannelMessage>) -> anyhow::Result<()>;
|
||||
|
||||
/// Check if channel is healthy
|
||||
async fn health_check(&self) -> bool {
|
||||
true
|
||||
}
|
||||
|
||||
/// Signal that the bot is processing a response (e.g. "typing" indicator).
|
||||
async fn start_typing(&self, _recipient: &str) -> anyhow::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
/// Stop any active typing indicator.
|
||||
async fn stop_typing(&self, _recipient: &str) -> anyhow::Result<()> {
|
||||
Ok(())
|
||||
}
|
||||
|
||||
// ... draft and reaction methods
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Draft Update Protocol
|
||||
|
||||
ZeroClaw implements a progressive "Draft" protocol designed for streaming LLM responses. Instead of sending multiple message fragments, platforms that support editing (Telegram, Discord, Slack) can update a single message in place as the response is generated.
|
||||
|
||||
### Protocol Methods
|
||||
|
||||
* `supports_draft_updates()`: Returns true if the platform allows message editing.
|
||||
* `send_draft(&SendMessage)`: Sends the initial message and returns a platform-specific `message_id`.
|
||||
* `update_draft(recipient, message_id, text)`: Appends or replaces the content of the existing message.
|
||||
* `finalize_draft(recipient, message_id, text)`: Performs a final update, often used to apply markdown formatting or remove "typing" statuses.
|
||||
* `cancel_draft(recipient, message_id)`: Deletes the draft if the generation is aborted.
|
||||
|
||||
This protocol significantly reduces notification noise on user devices and provides a much smoother "typing" experience during long generations.
|
||||
|
||||
***
|
||||
|
||||
## Reactions and Pinning
|
||||
|
||||
ZeroClaw supports standard interactive elements across most platforms.
|
||||
|
||||
### Reactions
|
||||
* `add_reaction(channel_id, message_id, emoji)`: Adds a Unicode emoji reaction.
|
||||
* `remove_reaction(channel_id, message_id, emoji)`: Removes a previously added reaction.
|
||||
|
||||
### Pinning
|
||||
* `pin_message(channel_id, message_id)`: Pins a message to the channel.
|
||||
* `unpin_message(channel_id, message_id)`: Unpins a message.
|
||||
|
||||
***
|
||||
|
||||
## ChannelMessage Struct
|
||||
|
||||
The `ChannelMessage` is the Data Transfer Object (DTO) for all incoming and outgoing communication.
|
||||
|
||||
```rust
|
||||
pub struct ChannelMessage {
|
||||
pub id: String,
|
||||
pub sender: String,
|
||||
pub reply_target: String,
|
||||
pub content: String,
|
||||
pub channel: String,
|
||||
pub timestamp: u64,
|
||||
/// Platform thread identifier (e.g. Slack `ts`, Discord thread ID).
|
||||
/// When set, replies should be posted as threaded responses.
|
||||
pub thread_ts: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## SendMessage Builder
|
||||
|
||||
Sending messages uses a builder pattern to handle optional fields like subjects and threading context.
|
||||
|
||||
```rust
|
||||
pub struct SendMessage {
|
||||
pub content: String,
|
||||
pub recipient: String,
|
||||
pub subject: Option<String>,
|
||||
pub thread_ts: Option<String>,
|
||||
}
|
||||
|
||||
impl SendMessage {
|
||||
pub fn new(content: impl Into<String>, recipient: impl Into<String>) -> Self;
|
||||
pub fn with_subject(content: impl Into<String>, recipient: impl Into<String>, subject: impl Into<String>) -> Self;
|
||||
pub fn in_thread(mut self, thread_ts: Option<String>) -> Self;
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Platform Implementations
|
||||
|
||||
ZeroClaw includes implementations for 26 platforms, including:
|
||||
* Slack, Discord, Telegram, Microsoft Teams
|
||||
* WhatsApp (Twilio/Meta), Signal, Matrix
|
||||
* Twilio SMS, SendGrid Email, Postmark
|
||||
* IRC, XMPP, Mattermost, Rocket.Chat
|
||||
* Custom Webhooks and WebSocket adapters
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
Luna should adopt the `ChannelMessage` structure as its baseline message DTO in [[Core]].
|
||||
|
||||
The **Draft Update Protocol** is particularly critical for Luna's SignalR implementation. Rather than streaming raw text chunks to the frontend and letting the client manage the state, the SignalR hub can follow the `send_draft` / `update_draft` / `finalize_draft` flow. This ensures consistency between web clients and external messaging channels documented in [[Channels]].
|
||||
|
||||
The `SendMessage` builder pattern provides a clean API for Luna services to dispatch notifications without manually constructing complex JSON payloads.
|
||||
@@ -0,0 +1,213 @@
|
||||
# Compaction Strategy
|
||||
|
||||
ZeroClaw employs a dual-path conversation compaction system to manage long-running sessions. This strategy ensures the LLM maintains relevant context while staying within performance and token limits. Unlike basic history trimming, ZeroClaw uses proactive summarization to preserve state and reactive truncation to recover from context overflows.
|
||||
|
||||
***
|
||||
|
||||
## Constants
|
||||
|
||||
The following constants are defined in `agent/loop_.rs` and govern the default behavior of the compaction engine:
|
||||
|
||||
```rust
|
||||
/// Default trigger for auto-compaction when non-system message count exceeds this threshold.
|
||||
const DEFAULT_MAX_HISTORY_MESSAGES: usize = 50;
|
||||
|
||||
/// Keep this many most-recent non-system messages after compaction.
|
||||
const COMPACTION_KEEP_RECENT_MESSAGES: usize = 20;
|
||||
|
||||
/// Safety cap for compaction source transcript passed to the summarizer.
|
||||
const COMPACTION_MAX_SOURCE_CHARS: usize = 12_000;
|
||||
|
||||
/// Max characters retained in stored compaction summary.
|
||||
const COMPACTION_MAX_SUMMARY_CHARS: usize = 2_000;
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Two Compaction Triggers
|
||||
|
||||
ZeroClaw implements both agent-side proactive compaction and channel-side reactive recovery.
|
||||
|
||||
### 1. Proactive (Agent-Side)
|
||||
|
||||
The `auto_compact_history()` function in `agent/loop_.rs` is called after each turn in the interactive agent loop. It operates based on message count thresholds rather than token counts.
|
||||
|
||||
**Process:**
|
||||
1. **Threshold Check**: It counts non-system messages. If the count is less than or equal to `max_history` (default 50), it exits.
|
||||
2. **Preservation**: It identifies the system prompt (the first message if the role is "system") to ensure it is never compacted.
|
||||
3. **Range Calculation**: It determines how many messages to keep (`COMPACTION_KEEP_RECENT_MESSAGES`) and how many to compact.
|
||||
4. **Transcript Generation**: Older messages are formatted into a plain-text transcript via `build_compaction_transcript()`.
|
||||
5. **LLM Summarization**: The transcript is sent to a summarizer model with a temperature of 0.2.
|
||||
6. **Injection**: The resulting summary is inserted as an assistant message prefixed with `[Compaction summary]`.
|
||||
7. **Safety Fallback**: If summarization fails, it falls back to deterministic truncation via `truncate_with_ellipsis()`.
|
||||
8. **Hard Trim**: `trim_history()` runs afterward to ensure the history length is strictly enforced.
|
||||
|
||||
### 2. Reactive (Channel-Side)
|
||||
|
||||
The `compact_sender_history()` function in `channels/mod.rs` provides a fallback mechanism when the LLM returns a context-window overflow error.
|
||||
|
||||
**Process:**
|
||||
1. **Error Detection**: `is_context_window_overflow_error()` checks for 8 keyword patterns in the error string:
|
||||
- "exceeds the context window"
|
||||
- "context window of this model"
|
||||
- "maximum context length"
|
||||
- "context length exceeded"
|
||||
- "too many tokens"
|
||||
- "token limit exceeded"
|
||||
- "prompt is too long"
|
||||
- "input is too long"
|
||||
2. **Aggressive Truncation**: It keeps only the last `CHANNEL_HISTORY_COMPACT_KEEP_MESSAGES` messages.
|
||||
3. **Content Cap**: Each retained message is truncated to `CHANNEL_HISTORY_COMPACT_CONTENT_CHARS`.
|
||||
4. **User Recovery**: The system typically asks the user to resend their last message after the history has been thinned.
|
||||
|
||||
***
|
||||
|
||||
## Summarization Prompt
|
||||
|
||||
The summarizer uses a specific system and user prompt pair to ensure the output is useful for future context.
|
||||
|
||||
**System Prompt:**
|
||||
> You are a conversation compaction engine. Summarize older chat history into concise context for future turns. Preserve: user preferences, commitments, decisions, unresolved tasks, key facts. Omit: filler, repeated chit-chat, verbose tool logs. Output plain text bullet points only.
|
||||
|
||||
**User Prompt:**
|
||||
> Summarize the following conversation history for context preservation. Keep it short (max 12 bullet points).
|
||||
>
|
||||
> {transcript}
|
||||
|
||||
**Temperature:** 0.2
|
||||
|
||||
***
|
||||
|
||||
## Helper Functions
|
||||
|
||||
### trim_history()
|
||||
|
||||
Located in `agent/loop_.rs`, this function performs a hard drain on the oldest non-system messages.
|
||||
|
||||
```rust
|
||||
fn trim_history(history: &mut Vec<ChatMessage>, max_history: usize) {
|
||||
// Nothing to trim if within limit
|
||||
let has_system = history.first().map_or(false, |m| m.role == "system");
|
||||
let non_system_count = if has_system {
|
||||
history.len() - 1
|
||||
} else {
|
||||
history.len()
|
||||
};
|
||||
|
||||
if non_system_count <= max_history {
|
||||
return;
|
||||
}
|
||||
|
||||
let start = if has_system { 1 } else { 0 };
|
||||
let to_remove = non_system_count - max_history;
|
||||
history.drain(start..start + to_remove);
|
||||
}
|
||||
```
|
||||
|
||||
### build_compaction_transcript()
|
||||
|
||||
Formats the range of messages targeted for compaction into a single string for the summarizer.
|
||||
|
||||
```rust
|
||||
fn build_compaction_transcript(messages: &[ChatMessage]) -> String {
|
||||
let mut transcript = String::new();
|
||||
for msg in messages {
|
||||
let role = msg.role.to_uppercase();
|
||||
let _ = writeln!(transcript, "{role}: {}", msg.content.trim());
|
||||
}
|
||||
|
||||
if transcript.chars().count() > COMPACTION_MAX_SOURCE_CHARS {
|
||||
truncate_with_ellipsis(&transcript, COMPACTION_MAX_SOURCE_CHARS)
|
||||
} else {
|
||||
transcript
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
### apply_compaction_summary()
|
||||
|
||||
Splices the generated summary into the conversation history, replacing the original messages.
|
||||
|
||||
```rust
|
||||
fn apply_compaction_summary(
|
||||
history: &mut Vec<ChatMessage>,
|
||||
start: usize,
|
||||
compact_end: usize,
|
||||
summary: &str,
|
||||
) {
|
||||
let summary_msg = ChatMessage::assistant(format!("[Compaction summary]\n{}", summary.trim()));
|
||||
history.splice(start..compact_end, std::iter::once(summary_msg));
|
||||
}
|
||||
```
|
||||
|
||||
### Agent::trim_history()
|
||||
|
||||
The high-level implementation in `agent/agent.rs` handles the separation and reassembly of system messages during a trim.
|
||||
|
||||
```rust
|
||||
fn trim_history(&mut self) {
|
||||
let max = self.config.max_history_messages;
|
||||
if self.history.len() <= max {
|
||||
return;
|
||||
}
|
||||
|
||||
let mut system_messages = Vec::new();
|
||||
let mut other_messages = Vec::new();
|
||||
|
||||
for msg in self.history.drain(..) {
|
||||
match &msg {
|
||||
ConversationMessage::Chat(chat) if chat.role == "system" => {
|
||||
system_messages.push(msg);
|
||||
}
|
||||
_ => other_messages.push(msg),
|
||||
}
|
||||
}
|
||||
|
||||
if other_messages.len() > max {
|
||||
let drop_count = other_messages.len() - max;
|
||||
other_messages.drain(0..drop_count);
|
||||
}
|
||||
|
||||
self.history = system_messages;
|
||||
self.history.extend(other_messages);
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Injection Format
|
||||
|
||||
When compaction occurs, the summary is injected back into the history as a single assistant message. This ensures that the model sees the previous context as its own earlier summary of events.
|
||||
|
||||
**Format:**
|
||||
```text
|
||||
assistant: [Compaction summary]
|
||||
- Bullet point 1
|
||||
- Bullet point 2
|
||||
...
|
||||
```
|
||||
|
||||
The summary replaces the exact range of messages that were sent to the summarizer, maintaining chronological integrity between the system prompt and the recent "live" messages.
|
||||
|
||||
***
|
||||
|
||||
## Configuration
|
||||
|
||||
The compaction behavior is influenced by the following configuration settings:
|
||||
|
||||
* `config.agent.max_history_messages`: The primary threshold (default 50).
|
||||
* `compact_context`: A boolean flag to enable or disable the summarization logic.
|
||||
* Note: Internal constants like `COMPACTION_KEEP_RECENT_MESSAGES` are currently hardcoded in `loop_.rs`.
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
The ZeroClaw compaction strategy provides a significant upgrade over the basic LibrarianAgent compaction currently used in Luna. Implementing this approach in Luna would address several limitations:
|
||||
|
||||
* **Dual-Path Robustness**: Combining proactive message-count triggers with reactive overflow recovery ensures the agent never hits an unrecoverable "context too long" state.
|
||||
* **Structured Context**: Using a specific summarization prompt preserves critical state (decisions, preferences) that is often lost in simple truncation.
|
||||
* **Identifiable Injection**: The `[Compaction summary]` prefix allows the model (and debugging tools) to distinguish between raw history and summarized context.
|
||||
* **Threshold Management**: Moving toward the ZeroClaw constants would provide more predictable behavior in high-volume tool-call loops.
|
||||
|
||||
The summarization prompt text and the `apply_compaction_summary` pattern are directly compatible with Luna's [[Core]] architecture.
|
||||
@@ -0,0 +1,137 @@
|
||||
# Configuration
|
||||
|
||||
ZeroClaw utilizes a TOML-based configuration system designed for modularity, security, and extensibility. The configuration is primarily managed through `config.toml`, which is loaded from the workspace directory or a fallback legacy location.
|
||||
|
||||
***
|
||||
|
||||
## Config Struct
|
||||
|
||||
The core of the configuration system is defined in `config/schema.rs` via the `Config` struct. It contains over 30 sections, each governing a specific subsystem.
|
||||
|
||||
### Global Settings
|
||||
* `workspace_dir`: The root directory for ZeroClaw data (computed at runtime).
|
||||
* `config_path`: The path to the active `config.toml`.
|
||||
* `api_key`: Global API key for the default provider.
|
||||
* `api_url`: Base URL override for provider APIs.
|
||||
* `default_provider`: The primary provider ID (e.g., "anthropic", "ollama").
|
||||
* `default_model`: The default model used for queries.
|
||||
* `default_temperature`: Model temperature (0.0 to 2.0, default 0.7).
|
||||
|
||||
### Subsystem Sections
|
||||
* `observability`: Tracing and metrics configuration.
|
||||
* `autonomy`: Security policies and autonomy levels.
|
||||
* `security`: Secret management and sandbox settings.
|
||||
* `runtime`: Native vs Docker execution modes.
|
||||
* `reliability`: Retry logic and fallback providers.
|
||||
* `scheduler`: Periodic task execution settings.
|
||||
* `agent`: Orchestration parameters (history size, tool iterations).
|
||||
* `skills`: Skill loading and prompt injection modes.
|
||||
* `model_routes`: Routing specific hints to provider/model pairs.
|
||||
* `embedding_routes`: Routing for vector embedding models.
|
||||
* `channels_config`: Configuration for Telegram, Discord, Slack, etc.
|
||||
* `memory`: Backend settings for SQLite or vector storage.
|
||||
* `storage`: Persistent file storage provider configuration.
|
||||
* `secrets`: Encryption settings for credentials.
|
||||
* `browser`: Browser automation and "computer-use" sidecar.
|
||||
* `identity`: AIEOS or OpenClaw format identity documents.
|
||||
* `cost`: Budget enforcement and price tracking.
|
||||
* `hardware`: Physical world interaction (serial/probe) settings.
|
||||
* `hooks`: Lifecycle and built-in hook toggles.
|
||||
|
||||
***
|
||||
|
||||
## ModelProviderConfig
|
||||
|
||||
ZeroClaw supports multiple provider profiles via the `model_providers` HashMap.
|
||||
|
||||
```rust
|
||||
pub struct ModelProviderConfig {
|
||||
pub name: Option<String>,
|
||||
pub base_url: Option<String>,
|
||||
pub wire_api: Option<String>,
|
||||
pub requires_openai_auth: bool,
|
||||
pub azure_openai_resource: Option<String>,
|
||||
pub azure_openai_deployment: Option<String>,
|
||||
pub azure_openai_api_version: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
This allows for configuring multiple instances of the same provider (e.g., local Ollama vs. remote Ollama) or complex Azure OpenAI deployments with distinct resource names and versions.
|
||||
|
||||
***
|
||||
|
||||
## DelegateAgentConfig
|
||||
|
||||
Sub-agents used by the `delegate` tool are configured separately to allow for specialized behavior.
|
||||
|
||||
```rust
|
||||
pub struct DelegateAgentConfig {
|
||||
pub provider: String,
|
||||
pub model: String,
|
||||
pub system_prompt: Option<String>,
|
||||
pub api_key: Option<String>,
|
||||
pub temperature: Option<f64>,
|
||||
pub max_depth: u32,
|
||||
pub agentic: bool,
|
||||
pub allowed_tools: Vec<String>,
|
||||
pub max_iterations: usize,
|
||||
}
|
||||
```
|
||||
|
||||
Sub-agents can be limited to specific toolsets and have a `max_depth` to prevent infinite delegation loops.
|
||||
|
||||
***
|
||||
|
||||
## TOML Structure
|
||||
|
||||
Example `config.toml` demonstrating provider setup and security routing:
|
||||
|
||||
```toml
|
||||
default_provider = "anthropic"
|
||||
default_model = "claude-3-5-sonnet"
|
||||
|
||||
[model_providers.ollama_local]
|
||||
name = "ollama"
|
||||
base_url = "http://localhost:11434"
|
||||
|
||||
[[model_routes]]
|
||||
hint = "fast"
|
||||
provider = "ollama_local"
|
||||
model = "llama3"
|
||||
|
||||
[security]
|
||||
encrypt = true
|
||||
|
||||
[autonomy]
|
||||
level = "high"
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Hot-Reload
|
||||
|
||||
ZeroClaw supports hot-reloading for specific runtime fields without requiring a full service restart.
|
||||
|
||||
### Supported Fields
|
||||
* `api_key` / `api_url`
|
||||
* `default_provider`
|
||||
* `default_model`
|
||||
* `default_temperature`
|
||||
* `reliability` settings
|
||||
|
||||
### Mechanism
|
||||
The system utilizes a file watcher (via `config_file_stamp`) that monitors the `config.toml` modification time. When a change is detected, the `maybe_apply_runtime_config_update` function reloads the file, decrypts any secrets using the `SecretStore`, and updates the runtime provider cache.
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
ZeroClaw's configuration architecture serves as the target state for [[Configuration]] in Luna.
|
||||
|
||||
* **Subsystem Isolation**: Luna plans to migrate from `appsettings.json` to a per-subsystem TOML structure to improve modularity, similar to ZeroClaw's section pattern.
|
||||
* **Security First**: The pattern of separating `autonomy` and `security` configs is a requirement for Luna's [[Core]].
|
||||
* **Hot-Reload**: ZeroClaw's implementation of lightweight file watching for key LLM parameters is a "nice-to-have" feature Luna aims to adopt during the migration.
|
||||
|
||||
***
|
||||
|
||||
[[Configuration]] | [[Core]]
|
||||
@@ -0,0 +1,187 @@
|
||||
# Memory
|
||||
|
||||
ZeroClaw implements a hybrid memory system that combines traditional keyword search with modern vector similarity. This approach ensures that exact matches (like function names or specific terminology) and semantic matches (concepts and related ideas) are both discoverable.
|
||||
|
||||
***
|
||||
|
||||
## Memory Trait
|
||||
|
||||
The foundation of the memory system is the `Memory` trait. Any backend implementation must satisfy this interface to be used by the system.
|
||||
|
||||
```rust
|
||||
pub struct MemoryEntry {
|
||||
pub id: String,
|
||||
pub key: String,
|
||||
pub content: String,
|
||||
pub category: MemoryCategory,
|
||||
pub timestamp: String,
|
||||
pub session_id: Option<String>,
|
||||
pub score: Option<f64>,
|
||||
}
|
||||
|
||||
pub enum MemoryCategory {
|
||||
Core,
|
||||
Daily,
|
||||
Conversation,
|
||||
Custom(String),
|
||||
}
|
||||
|
||||
#[async_trait]
|
||||
pub trait Memory: Send + Sync {
|
||||
fn name(&self) -> &str;
|
||||
|
||||
async fn store(
|
||||
&self,
|
||||
key: &str,
|
||||
content: &str,
|
||||
category: MemoryCategory,
|
||||
session_id: Option<&str>,
|
||||
) -> anyhow::Result<()>;
|
||||
|
||||
async fn recall(
|
||||
&self,
|
||||
query: &str,
|
||||
limit: usize,
|
||||
session_id: Option<&str>,
|
||||
) -> anyhow::Result<Vec<MemoryEntry>>;
|
||||
|
||||
async fn get(&self, key: &str) -> anyhow::Result<Option<MemoryEntry>>;
|
||||
|
||||
async fn list(
|
||||
&self,
|
||||
category: Option<&MemoryCategory>,
|
||||
session_id: Option<&str>,
|
||||
) -> anyhow::Result<Vec<MemoryEntry>>;
|
||||
|
||||
async fn forget(&self, key: &str) -> anyhow::Result<bool>;
|
||||
|
||||
async fn count(&self) -> anyhow::Result<usize>;
|
||||
|
||||
async fn health_check(&self) -> bool;
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## SQLite Backend
|
||||
|
||||
The `SqliteMemory` implementation serves as the primary persistent backend. It uses `rusqlite` and is tuned for high performance and reliability in a local environment.
|
||||
|
||||
### Performance Tuning
|
||||
|
||||
The backend initializes with specific PRAGMAs to optimize for concurrent access and speed:
|
||||
|
||||
```rust
|
||||
conn.execute_batch(
|
||||
"PRAGMA journal_mode = WAL;
|
||||
PRAGMA synchronous = NORMAL;
|
||||
PRAGMA mmap_size = 8388608;
|
||||
PRAGMA cache_size = -2000;
|
||||
PRAGMA temp_store = MEMORY;",
|
||||
)?;
|
||||
```
|
||||
|
||||
- **WAL mode**: Enables concurrent reads even during write operations.
|
||||
- **mmap (8MB)**: Allows the OS to handle hot reads through memory mapping.
|
||||
- **temp_store MEMORY**: Ensures temporary tables never touch the disk.
|
||||
|
||||
***
|
||||
|
||||
## Schema Design
|
||||
|
||||
The system maintains a relational table for core data and a virtual table for full-text search (FTS5). Triggers keep these in sync automatically.
|
||||
|
||||
```sql
|
||||
-- Core memories table
|
||||
CREATE TABLE IF NOT EXISTS memories (
|
||||
id TEXT PRIMARY KEY,
|
||||
key TEXT NOT NULL UNIQUE,
|
||||
content TEXT NOT NULL,
|
||||
category TEXT NOT NULL DEFAULT 'core',
|
||||
embedding BLOB,
|
||||
created_at TEXT NOT NULL,
|
||||
updated_at TEXT NOT NULL,
|
||||
session_id TEXT
|
||||
);
|
||||
|
||||
-- FTS5 virtual table for keyword search
|
||||
CREATE VIRTUAL TABLE IF NOT EXISTS memories_fts USING fts5(
|
||||
key, content, content=memories, content_rowid=rowid
|
||||
);
|
||||
|
||||
-- Sync triggers
|
||||
CREATE TRIGGER IF NOT EXISTS memories_ai AFTER INSERT ON memories BEGIN
|
||||
INSERT INTO memories_fts(rowid, key, content)
|
||||
VALUES (new.rowid, new.key, new.content);
|
||||
END;
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Hybrid Search
|
||||
|
||||
The `recall` method executes a hybrid search strategy. It merges results from two distinct search mechanisms:
|
||||
|
||||
1. **Keyword Search**: Uses FTS5 BM25 scoring for exact word matches.
|
||||
2. **Vector Search**: Computes cosine similarity between query embeddings and stored memory embeddings.
|
||||
|
||||
### Score Fusion
|
||||
|
||||
Results are combined using a weighted average. The defaults are typically:
|
||||
- **Vector Weight**: 0.7
|
||||
- **Keyword Weight**: 0.3
|
||||
|
||||
If vector results are unavailable (e.g., if embeddings are disabled), the system falls back to keyword-only search. If both high-level search mechanisms return no results, it uses a final `LIKE %query%` fallback to ensure maximum recall.
|
||||
|
||||
***
|
||||
|
||||
## Embedding Cache
|
||||
|
||||
To avoid redundant API calls to embedding providers, ZeroClaw uses an internal LRU (Least Recently Used) cache stored in SQLite.
|
||||
|
||||
```sql
|
||||
CREATE TABLE IF NOT EXISTS embedding_cache (
|
||||
content_hash TEXT PRIMARY KEY,
|
||||
embedding BLOB NOT NULL,
|
||||
created_at TEXT NOT NULL,
|
||||
accessed_at TEXT NOT NULL
|
||||
);
|
||||
```
|
||||
|
||||
Whenever a memory is stored or a query is processed, the system checks this cache first using a deterministic content hash. Eviction occurs once the cache reaches its configured limit (default 10,000 entries).
|
||||
|
||||
***
|
||||
|
||||
## Backend Implementations
|
||||
|
||||
ZeroClaw supports several backend types configured via [[Configuration]]:
|
||||
|
||||
1. **SQLite**: The standard persistent local backend using FTS5 and BLOB embeddings.
|
||||
2. **PostgreSQL**: Used for distributed or cloud-hosted deployments (requires `pgvector`).
|
||||
3. **Lucid**: A bridge backend that synchronizes local SQLite memory with remote services.
|
||||
4. **Markdown**: A simple file-based implementation that stores memories as `.md` files in the workspace.
|
||||
5. **Qdrant**: A dedicated vector database backend for high-scale semantic search.
|
||||
6. **None**: An explicit no-op backend that disables all memory persistence.
|
||||
|
||||
***
|
||||
|
||||
## Timeout Guards
|
||||
|
||||
Database operations are protected by timeout guards. Specifically, opening a SQLite connection is capped at 300 seconds to prevent the system from hanging on locked or slow filesystems.
|
||||
|
||||
```rust
|
||||
const SQLITE_OPEN_TIMEOUT_CAP_SECS: u64 = 300;
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
ZeroClaw's memory architecture serves as the blueprint for [[Memory]]. While Luna currently has an `IMemoryStore` interface, the existing file-based implementation lacks sophisticated recall.
|
||||
|
||||
Adopting the ZeroClaw pattern provides:
|
||||
- **Structured Categories**: Using `MemoryCategory` (Core, Daily, Conversation) for better context management.
|
||||
- **Hybrid Recall**: Moving beyond simple file reading to a ranked keyword+vector search.
|
||||
- **SQLite Reference**: Luna can adopt the `SqliteMemory` implementation directly to replace the current unstructured storage.
|
||||
|
||||
Cross-links: [[Core]], [[Configuration]]
|
||||
@@ -0,0 +1,193 @@
|
||||
# ZeroClaw
|
||||
|
||||
## Overview
|
||||
ZeroClaw is a lean, trait-driven Rust AI assistant runtime (~1.7k stars, ~1.6k commits). Single binary, <5MB RAM, <10ms startup. Everything is a swappable trait. Repo: https://github.com/openagen/zeroclaw
|
||||
|
||||
## Architecture
|
||||
- Single-binary Rust runtime — no microservices, no plugins to install
|
||||
- Trait-driven design: every subsystem (Provider, Channel, Tool, Memory, Runtime, Security) is a trait you swap
|
||||
- Minimal footprint: <5MB RAM, <10ms cold start
|
||||
- All features compiled in-process, zero external dependencies for core functionality
|
||||
|
||||
## Providers
|
||||
15+ provider implementations behind a `Provider` trait:
|
||||
OpenAI, Anthropic, Gemini, Ollama, OpenRouter, Azure OpenAI, AWS Bedrock, GitHub Copilot, GLM, Telnyx, llama.cpp, vLLM, custom endpoints.
|
||||
|
||||
Key pattern — `ReliableProvider` wrapper: wraps any provider with automatic fallback chains and retry logic. If primary fails, falls through to secondary, tertiary, etc.
|
||||
|
||||
```rust
|
||||
// Provider trait (src/providers/traits.rs)
|
||||
pub trait Provider: Send + Sync {
|
||||
fn name(&self) -> &str;
|
||||
async fn chat(&self, request: ChatRequest) -> Result<ChatResponse>;
|
||||
async fn stream(&self, request: ChatRequest) -> Result<Box<dyn Stream<Item = Result<ChatChunk>>>>;
|
||||
}
|
||||
```
|
||||
|
||||
## Agent Loop
|
||||
- Full tool-calling protocol: LLM requests tool call → agent executes tool → feeds result back → loops until LLM stops requesting tools
|
||||
- `DelegateTool`: spawns a sub-agent with a scoped task for multi-agent delegation
|
||||
- Memory injection into agent context
|
||||
- Identity/persona system supporting AIEOS v1.1 JSON format and OpenClaw markdown format
|
||||
|
||||
## Channels
|
||||
26 channel implementations behind a `Channel` trait:
|
||||
CLI, Telegram, Discord, Slack, Mattermost, iMessage, Matrix, Signal, WhatsApp (Web + Business API), Email, IRC, Lark, DingTalk, QQ, Nostr, MQTT, Webhook, and more.
|
||||
|
||||
```rust
|
||||
// Channel message types (src/channels/traits.rs)
|
||||
pub struct ChannelMessage {
|
||||
pub id: String,
|
||||
pub sender: String,
|
||||
pub reply_target: Option<String>,
|
||||
pub content: String,
|
||||
pub channel: String,
|
||||
pub timestamp: DateTime<Utc>,
|
||||
pub thread_ts: Option<String>,
|
||||
}
|
||||
```
|
||||
|
||||
## Memory
|
||||
6 backends behind a `Memory` trait:
|
||||
- SQLite: hybrid FTS5 full-text + cosine vector similarity in a single query, zero external deps
|
||||
- PostgreSQL
|
||||
- Lucid bridge
|
||||
- Markdown (file-based)
|
||||
- Qdrant (vector DB)
|
||||
- Explicit none
|
||||
|
||||
```rust (src/memory/traits.rs)
|
||||
pub trait Memory: Send + Sync {
|
||||
fn name(&self) -> &str;
|
||||
async fn store(&self, entry: MemoryEntry) -> Result<()>;
|
||||
async fn recall(&self, query: &str, limit: usize) -> Result<Vec<MemoryEntry>>;
|
||||
}
|
||||
|
||||
pub struct MemoryEntry {
|
||||
pub id: String,
|
||||
pub key: String,
|
||||
pub content: String,
|
||||
pub category: MemoryCategory,
|
||||
pub timestamp: DateTime<Utc>,
|
||||
pub session_id: Option<String>,
|
||||
pub score: Option<f64>,
|
||||
}
|
||||
|
||||
pub enum MemoryCategory {
|
||||
Core,
|
||||
Daily,
|
||||
Conversation,
|
||||
Custom(String),
|
||||
}
|
||||
```
|
||||
|
||||
Custom embedding provider routing — memory backends can use different embedding models.
|
||||
|
||||
## Tools
|
||||
42+ tool implementations behind a `Tool` trait:
|
||||
|
||||
```rust
|
||||
// Tool trait (src/tools/traits.rs)
|
||||
pub trait Tool: Send + Sync {
|
||||
fn name(&self) -> &str;
|
||||
fn description(&self) -> &str;
|
||||
fn parameters_schema(&self) -> serde_json::Value;
|
||||
async fn execute(&self, args: serde_json::Value) -> Result<ToolResult>;
|
||||
}
|
||||
```
|
||||
|
||||
Tool categories:
|
||||
- **Shell & Files**: shell execution, file read/write/edit, glob search, content search
|
||||
- **Browser & Web**: browser automation, HTTP requests, web fetch/search, screenshots
|
||||
- **Documents**: PDF read, image info
|
||||
- **Git**: git operations (status, diff, commit, log, etc.)
|
||||
- **Scheduling**: cron add/list/remove/update/run
|
||||
- **Memory**: store/recall/forget (tools that interact with the Memory trait)
|
||||
- **Delegation**: DelegateTool for sub-agent spawning
|
||||
- **Hardware**: USB discover/introspect, robot-kit integration
|
||||
- **Integration**: Composio (1000+ OAuth apps), SOP tools, Pushover notifications
|
||||
|
||||
## Sessions
|
||||
- Per-sender conversation history
|
||||
- Compaction (summarization of old messages)
|
||||
- Autosave thresholds (every N messages, not just on shutdown)
|
||||
- Per-sender provider/model overrides (each conversation can use a different model)
|
||||
|
||||
## Security
|
||||
Comprehensive security stack (16 modules):
|
||||
- **SecurityPolicy**: Autonomy levels — readonly, supervised, full
|
||||
- **Workspace Scoping**: Operations restricted to declared workspace paths
|
||||
- **Filesystem Guards**: 14 system directories blocked, symlink escape detection, null byte injection blocked
|
||||
- **Sandboxing**: Landlock, Bubblewrap, Docker, Firejail — multiple isolation backends
|
||||
- **Gateway Pairing**: 6-digit codes + bearer tokens for channel authentication
|
||||
- **Encrypted Secrets**: SecretStore for API keys and credentials
|
||||
- **Audit Logging**: All security-relevant actions logged
|
||||
- **Emergency Stop**: EstopManager for immediate agent shutdown
|
||||
- **Prompt Guard**: Detection of prompt injection attempts
|
||||
- **Leak Detector**: Prevents secrets from appearing in agent output
|
||||
- **OTP**: One-time password support
|
||||
|
||||
## Runtime
|
||||
`RuntimeAdapter` trait with two implementations:
|
||||
- `NativeRuntime`: Direct execution on host OS
|
||||
- `DockerRuntime`: Isolated execution in Docker containers
|
||||
Tool execution can be routed through either runtime for isolation.
|
||||
|
||||
## Skills
|
||||
- TOML manifests (`SKILL.toml`) + `SKILL.md` instructions
|
||||
- open-skills sync for community skill repositories
|
||||
- Security audit on install: blocks symlinks, scripts, unsafe patterns
|
||||
|
||||
## Configuration
|
||||
- TOML-based config at `~/.zeroclaw/config.toml`
|
||||
- Hot-reload on key fields without restart
|
||||
- Per-channel, per-provider, per-tool configuration sections
|
||||
|
||||
## Tunnel
|
||||
Trait-based tunneling for exposing local instance to the internet:
|
||||
- Cloudflare Tunnel
|
||||
- Tailscale
|
||||
- ngrok
|
||||
- Custom tunnel implementations
|
||||
|
||||
***
|
||||
|
||||
## In-Depth Reference Pages
|
||||
|
||||
- [[ReliableProvider]] — Three-level failover (model chain, provider chain, retry loop), error classification, API key rotation
|
||||
- [[Agent Loop & Tools]] — Tool trait, agent execution loop, tool serialization, credential scrubbing
|
||||
- [[Memory]] — Memory trait, SQLite hybrid FTS5+vector search, embedding cache, 6 backends
|
||||
- [[Channel Messages]] — Channel trait, draft update protocol, ChannelMessage struct, SendMessage builder
|
||||
- [[Security]] — SecurityPolicy, autonomy levels, command allowlist, path validation, sandboxing, secret store, e-stop
|
||||
- [[Sessions]] — Per-sender storage, compaction logic, autosave thresholds, per-sender overrides
|
||||
- [[Configuration]] — TOML config struct (30+ sections), ModelProviderConfig, DelegateAgentConfig, hot-reload
|
||||
- [[Runtime Adapters]] — RuntimeAdapter trait, NativeRuntime vs DockerRuntime, capability querying
|
||||
- [[Skills]] — SKILL.toml manifest, security audit on install, open-skills sync, prompt injection
|
||||
- [[Compaction Strategy]] — Proactive agent-side and reactive channel-side compaction, summarization prompts, helper functions
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
### Patterns Worth Adopting
|
||||
|
||||
ZeroClaw is architecturally closest to Luna (monolithic binary, trait/interface-driven design). These patterns map directly:
|
||||
|
||||
- **ReliableProvider wrapper**: Luna's `IProvider` returning `IChatClient` is clean. Add a decorator that wraps any `IProvider` with fallback chains and retry logic. See [[ReliableProvider]] for the three-level failover pattern. Link to [[Core]].
|
||||
- **Tool trait + Agent loop**: This is Luna's #1 gap. The `Tool` trait maps cleanly to a C# `ITool` interface. The agent loop is the missing piece that turns Luna from a chat relay into an assistant. See [[Agent Loop & Tools]]. Link to [[Skills]].
|
||||
- **Memory with hybrid search**: Luna's `IMemoryStore` exists but has no recall. ZeroClaw's SQLite memory does hybrid FTS5 + cosine vector similarity in one query. See [[Memory]]. Link to Luna's [[Memory]].
|
||||
- **Channel message normalization**: The `ChannelMessage` struct should be the baseline for Luna's DTO. The draft update protocol maps to SignalR. See [[Channel Messages]]. Link to [[Channels]].
|
||||
- **Session autosave + per-sender overrides**: Autosave every N messages for crash resilience. Per-sender model overrides are cheap to add. See [[Sessions]]. Link to [[Core]].
|
||||
- **Compaction**: Dual-path approach (proactive message-count + reactive overflow recovery) with structured summarization prompts. See [[Compaction Strategy]].
|
||||
- **Security fundamentals**: Before tool execution, Luna needs filesystem guards, workspace scoping, and autonomy levels. See [[Security]]. ZeroClaw's `SecurityPolicy` is the reference.
|
||||
- **TOML config with hot-reload**: Luna's [[Configuration]] module plans this. ZeroClaw's per-subsystem section pattern is the target. See [[Configuration]].
|
||||
- **Runtime adapters**: When tool execution is added, a similar `IRuntimeAdapter` allows switching between native and Docker. See [[Runtime Adapters]].
|
||||
- **Skills system**: SKILL.toml manifest with security audit on install. See [[Skills]].
|
||||
|
||||
### Key Differences from Luna
|
||||
- Rust vs .NET/C# — different ecosystem, but the trait-driven architecture maps well to C# interfaces
|
||||
- ZeroClaw has 42+ tools; Luna has 0 — tool system is the critical gap
|
||||
- ZeroClaw has 15+ providers; Luna has 2 (Mistral, Ollama)
|
||||
- ZeroClaw has 26 channels; Luna has 1 (CLI)
|
||||
- ZeroClaw has comprehensive security (16 modules); Luna has none
|
||||
- ZeroClaw runs as a single <5MB binary; Luna's .NET runtime is heavier but offers richer DI/middleware ecosystem
|
||||
@@ -0,0 +1,152 @@
|
||||
# ReliableProvider
|
||||
|
||||
The `ReliableProvider` is a resilient decorator in ZeroClaw that wraps one or more standard providers to implement a three-level failover strategy. It ensures high availability by managing model fallbacks, provider redundancy, and intelligent retry logic with backoff.
|
||||
|
||||
***
|
||||
|
||||
## Provider Trait
|
||||
|
||||
All LLM integrations in ZeroClaw implement the `Provider` trait. This trait defines a unified interface for chat interactions, tool calling, and capability discovery.
|
||||
|
||||
```rust
|
||||
#[async_trait]
|
||||
pub trait Provider: Send + Sync {
|
||||
fn capabilities(&self) -> ProviderCapabilities {
|
||||
ProviderCapabilities::default()
|
||||
}
|
||||
|
||||
async fn chat_with_system(
|
||||
&self,
|
||||
system_prompt: Option<&str>,
|
||||
message: &str,
|
||||
model: &str,
|
||||
temperature: f64,
|
||||
) -> anyhow::Result<String>;
|
||||
|
||||
async fn chat_with_history(
|
||||
&self,
|
||||
messages: &[ChatMessage],
|
||||
model: &str,
|
||||
temperature: f64,
|
||||
) -> anyhow::Result<String>;
|
||||
|
||||
async fn chat(
|
||||
&self,
|
||||
request: ChatRequest<'_>,
|
||||
model: &str,
|
||||
temperature: f64,
|
||||
) -> anyhow::Result<ChatResponse>;
|
||||
|
||||
async fn chat_with_tools(
|
||||
&self,
|
||||
messages: &[ChatMessage],
|
||||
_tools: &[serde_json::Value],
|
||||
model: &str,
|
||||
temperature: f64,
|
||||
) -> anyhow::Result<ChatResponse>;
|
||||
}
|
||||
```
|
||||
|
||||
### ProviderCapabilities
|
||||
The `ProviderCapabilities` struct allows providers to declare their feature set, which `ReliableProvider` uses to route requests correctly:
|
||||
- `native_tool_calling`: Support for API-native function calling (Gemini, Anthropic, OpenAI).
|
||||
- `vision`: Support for multimodal image inputs.
|
||||
|
||||
### ToolsPayload
|
||||
When tools are used, `Provider::convert_tools` returns a `ToolsPayload` enum to handle varying API requirements:
|
||||
- `Gemini { function_declarations }`
|
||||
- `Anthropic { tools }`
|
||||
- `OpenAI { tools }`
|
||||
- `PromptGuided { instructions }`: Textual fallback for providers without native support.
|
||||
|
||||
***
|
||||
|
||||
## Three-Level Failover
|
||||
|
||||
The `ReliableProvider` implements a nested loop structure to exhaust all possibilities before returning an error.
|
||||
|
||||
```rust
|
||||
pub struct ReliableProvider {
|
||||
providers: Vec<(String, Box<dyn Provider>)>,
|
||||
max_retries: u32,
|
||||
base_backoff_ms: u64,
|
||||
api_keys: Vec<String>,
|
||||
key_index: AtomicUsize,
|
||||
model_fallbacks: HashMap<String, Vec<String>>,
|
||||
}
|
||||
```
|
||||
|
||||
1. **Level 1: Model Chain**: Iterates through the primary model and its configured fallbacks (e.g., try `claude-3-5-sonnet`, fallback to `claude-3-haiku`).
|
||||
2. **Level 2: Provider Chain**: For each model, iterates through registered providers in priority order (e.g., try Anthropic direct, fallback to OpenRouter).
|
||||
3. **Level 3: Retry Loop**: For a specific (provider, model) pair, retries transient failures with exponential backoff.
|
||||
|
||||
***
|
||||
|
||||
## Error Classification
|
||||
|
||||
Intelligent failure handling depends on distinguishing transient issues from permanent ones. ZeroClaw uses heuristics and status codes for this classification.
|
||||
|
||||
### Non-Retryable Errors
|
||||
The `is_non_retryable()` function identifies errors that won't resolve with retries, such as:
|
||||
- **Client Errors**: HTTP 4xx (except 429/408).
|
||||
- **Authentication**: Key-word matching for "invalid api key", "unauthorized", or "permission denied".
|
||||
- **Model Availability**: Heuristics for "model not found", "unknown model", or "unsupported".
|
||||
|
||||
### Rate Limiting and Quota
|
||||
- `is_rate_limited()`: Specifically detects HTTP 429 errors.
|
||||
- `is_non_retryable_rate_limit()`: Detects business-level failures returned as 429s, such as "insufficient balance", "quota exhausted", or "plan does not include requested model". These trigger a provider fallback immediately.
|
||||
|
||||
### Context Window Exceeded
|
||||
The system short-circuits when the context window is exceeded to avoid useless retries:
|
||||
```rust
|
||||
fn is_context_window_exceeded(err: &anyhow::Error) -> bool {
|
||||
let hints = [
|
||||
"exceeds the context window",
|
||||
"maximum context length",
|
||||
"token limit exceeded",
|
||||
"prompt is too long",
|
||||
];
|
||||
// ...
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Exponential Backoff
|
||||
|
||||
Retries use an exponential backoff strategy starting from `base_backoff_ms`.
|
||||
- **Retry-After Parsing**: The system parses the `Retry-After` header or error body.
|
||||
- **Capping**: Backoff is capped at 30 seconds to prevent indefinite stalls.
|
||||
- **Jitter**: While the core logic doubles the wait time, it ensures the wait respects provider-suggested intervals.
|
||||
|
||||
***
|
||||
|
||||
## API Key Rotation
|
||||
|
||||
ZeroClaw supports round-robin API key rotation for the same provider to maximize throughput:
|
||||
```rust
|
||||
fn rotate_key(&self) -> Option<&str> {
|
||||
if self.api_keys.is_empty() { return None; }
|
||||
let idx = self.key_index.fetch_add(1, Ordering::Relaxed) % self.api_keys.len();
|
||||
Some(&self.api_keys[idx])
|
||||
}
|
||||
```
|
||||
When a 429 is encountered (and it's not a quota error), the system cycles to the next key for the subsequent retry attempt.
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
Currently, Luna's [[Core]] implementation of `IProvider` is simple and lacks resilience. If a provider is down, the request fails.
|
||||
|
||||
By implementing a `ReliableProvider` decorator for Luna's `IProvider`, we can:
|
||||
- Wrap multiple `IChatClient` instances.
|
||||
- Implement per-sender model overrides to allow specific users to access higher-tier models with fallback to cheaper ones.
|
||||
- Integrate the ZeroClaw error classification logic to handle common LLM API failures gracefully.
|
||||
|
||||
This pattern is essential for moving from a prototype to a production-grade system where provider stability cannot be guaranteed.
|
||||
|
||||
***
|
||||
|
||||
[[Core]]
|
||||
[[Configuration]]
|
||||
@@ -0,0 +1,103 @@
|
||||
# Runtime Adapters
|
||||
|
||||
ZeroClaw uses the `RuntimeAdapter` trait to abstract the execution environment from the core agent logic. This abstraction allows the same agent code to run natively on a host machine, inside a Docker container, or on restricted edge runtimes without modification to the core loop.
|
||||
|
||||
***
|
||||
|
||||
## RuntimeAdapter Trait
|
||||
|
||||
The `RuntimeAdapter` trait defines the capabilities and interface for any environment ZeroClaw operates within. It is designed to be `Send + Sync` to allow safe sharing across asynchronous tasks.
|
||||
|
||||
```rust
|
||||
pub trait RuntimeAdapter: Send + Sync {
|
||||
/// Return the human-readable name of this runtime environment.
|
||||
fn name(&self) -> &str;
|
||||
|
||||
/// Report whether this runtime supports shell command execution.
|
||||
fn has_shell_access(&self) -> bool;
|
||||
|
||||
/// Report whether this runtime supports filesystem read/write.
|
||||
fn has_filesystem_access(&self) -> bool;
|
||||
|
||||
/// Return the base directory for persistent storage on this runtime.
|
||||
fn storage_path(&self) -> PathBuf;
|
||||
|
||||
/// Report whether this runtime supports long-running background processes.
|
||||
fn supports_long_running(&self) -> bool;
|
||||
|
||||
/// Return the maximum memory budget in bytes for this runtime.
|
||||
fn memory_budget(&self) -> u64 {
|
||||
0
|
||||
}
|
||||
|
||||
/// Build a shell command process configured for this runtime.
|
||||
fn build_shell_command(
|
||||
&self,
|
||||
command: &str,
|
||||
workspace_dir: &Path,
|
||||
) -> anyhow::Result<tokio::process::Command>;
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## NativeRuntime
|
||||
|
||||
The `NativeRuntime` implementation provides direct execution on the host operating system. It represents the least restrictive environment and is typically used for local development or trusted server environments.
|
||||
|
||||
* **Shell Access**: Returns `true`. Commands are executed directly via the system shell.
|
||||
* **Filesystem Access**: Returns `true`. The agent can interact with any path permitted by the user's OS permissions.
|
||||
* **Memory Limits**: Typically returns `0` (unlimited), relying on the OS to manage process resources.
|
||||
* **Command Building**: Spawns `tokio::process::Command` directly with the requested command string.
|
||||
|
||||
***
|
||||
|
||||
## DockerRuntime
|
||||
|
||||
The `DockerRuntime` provides isolated execution by wrapping operations in Docker containers. This is the preferred runtime for untrusted code execution or when strict environment reproducibility is required.
|
||||
|
||||
* **Shell Access**: Configurable, but generally `true`. Commands are wrapped in `docker exec` calls targeting a specific container.
|
||||
* **Filesystem Access**: Restricted to the volumes and mounts defined in the container configuration.
|
||||
* **Memory Budgets**: Returns the memory limits defined for the container, allowing the agent to adapt its cache and buffer sizes.
|
||||
* **Command Building**: Instead of direct execution, it constructs a command that executes inside the container namespace, often involving complex argument escaping and environment variable injection.
|
||||
|
||||
***
|
||||
|
||||
## Capability Querying
|
||||
|
||||
The agent loop queries the `RuntimeAdapter` before attempting to execute tools or background tasks. This ensures that the agent fails gracefully or skips unavailable functionality based on its environment.
|
||||
|
||||
1. **Tool Pre-flight**: Before executing a shell tool, the orchestrator checks `has_shell_access()`.
|
||||
2. **Persistence Check**: Before initializing disk-based state, the agent checks `has_filesystem_access()`.
|
||||
3. **Background Services**: The heartbeat loop and gateway server only start if `supports_long_running()` returns `true`.
|
||||
|
||||
This pattern prevents runtime errors by verifying environmental support at the logic gate rather than deep within the execution stack.
|
||||
|
||||
***
|
||||
|
||||
## Command Building
|
||||
|
||||
The `build_shell_command` method is the primary bridge between the agent and the OS. It takes a raw command string and a workspace directory, returning a configured `tokio::process::Command`.
|
||||
|
||||
Runtimes use this to:
|
||||
* Prepend sandbox wrappers (e.g., `firejail` or `sudo -u limited`).
|
||||
* Set environment variables specific to the runtime (e.g., `PATH` or `HOME`).
|
||||
* Handle working directory mapping (especially important in Docker where host paths and container paths differ).
|
||||
|
||||
***
|
||||
|
||||
## Tool Routing
|
||||
|
||||
Tool execution can be routed through specific runtimes based on security requirements. A "Security Router" might send filesystem operations to a native runtime for speed while routing unknown shell scripts to a Docker runtime for isolation. This routing logic relies on the standardized interface provided by the `RuntimeAdapter`.
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
Luna currently operates with implicit native execution. As Luna evolves to support tool execution and autonomous tasks, implementing an `IRuntimeAdapter` interface will be critical for safety.
|
||||
|
||||
* **Isolation**: Adopting a pattern similar to ZeroClaw's `DockerRuntime` would allow Luna to run generated code in a sandbox without risking the host system.
|
||||
* **Cross-Platform**: A runtime abstraction simplifies porting Luna to different OSs or containerized environments.
|
||||
* **Security Integration**: This pairs with [[Security]] patterns to ensure that capabilities are not just checked by the runtime, but also verified against user-defined security policies.
|
||||
|
||||
Cross-links: [[Core]], [[Skills]], [[Configuration]].
|
||||
@@ -0,0 +1,214 @@
|
||||
# Security
|
||||
|
||||
ZeroClaw implements a multi-layered security stack designed to provide defense-in-depth for autonomous agent operations. This system ensures that agents operate within defined boundaries, protect sensitive credentials, and provide a verifiable audit trail of all actions.
|
||||
|
||||
***
|
||||
|
||||
## SecurityPolicy
|
||||
|
||||
The `SecurityPolicy` is the central enforcement mechanism for all tool and command executions. It defines the agent's level of independence and the specific constraints on its operating environment.
|
||||
|
||||
### Autonomy Levels
|
||||
|
||||
Autonomy is categorized into three distinct levels, controlling the baseline behavior of the agent:
|
||||
|
||||
```rust
|
||||
pub enum AutonomyLevel {
|
||||
/// Read-only: can observe but not act
|
||||
ReadOnly,
|
||||
/// Supervised: acts but requires approval for risky operations
|
||||
Supervised,
|
||||
/// Full: autonomous execution within policy bounds
|
||||
Full,
|
||||
}
|
||||
```
|
||||
|
||||
### Risk Classification
|
||||
|
||||
Commands and operations are classified by their potential impact:
|
||||
|
||||
```rust
|
||||
pub enum CommandRiskLevel {
|
||||
Low,
|
||||
Medium,
|
||||
High,
|
||||
}
|
||||
|
||||
pub enum ToolOperation {
|
||||
Read,
|
||||
Act,
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Action Tracker
|
||||
|
||||
ZeroClaw uses a sliding-window rate limiter to prevent automated abuse or runaway processes. It maintains a 1-hour window of all side-effecting actions.
|
||||
|
||||
```rust
|
||||
pub struct ActionTracker {
|
||||
/// Timestamps of recent actions (kept within the last hour).
|
||||
actions: Mutex<Vec<Instant>>,
|
||||
}
|
||||
|
||||
impl ActionTracker {
|
||||
pub fn record(&self) -> usize {
|
||||
let mut actions = self.actions.lock();
|
||||
let cutoff = Instant::now()
|
||||
.checked_sub(std::time::Duration::from_secs(3600))
|
||||
.unwrap_or_else(Instant::now);
|
||||
actions.retain(|t| *t > cutoff);
|
||||
actions.push(Instant::now());
|
||||
actions.len()
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Command Allowlist
|
||||
|
||||
ZeroClaw employs a deny-by-default strategy for shell execution. Only explicitly listed commands are permitted, and they are subjected to rigorous parsing to prevent bypasses.
|
||||
|
||||
### Default Allowed Commands
|
||||
`git`, `npm`, `cargo`, `ls`, `cat`, `grep`, `find`, `echo`, `pwd`, `wc`, `head`, `tail`, `date`.
|
||||
|
||||
### Default Forbidden Paths
|
||||
The system blocks access to 14 system directories (e.g., `/etc`, `/root`, `/usr`, `/bin`, `/proc`) and 4 sensitive dotfile locations (`~/.ssh`, `~/.gnupg`, `~/.aws`, `~/.config`) even if workspace confinement is disabled.
|
||||
|
||||
### Shell Parsing Logic
|
||||
To prevent command injection via chained operators or environment variables, ZeroClaw uses a quote-aware shell segment splitter and environment assignment stripper:
|
||||
|
||||
```rust
|
||||
fn skip_env_assignments(s: &str) -> &str {
|
||||
// Strips 'FOO=bar' from 'FOO=bar cmd args'
|
||||
}
|
||||
|
||||
fn split_unquoted_segments(command: &str) -> Vec<String> {
|
||||
// Splits on ';', '|', '&&', '||', and newlines
|
||||
// Respects quotes to allow literal separators in arguments
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Path Validation
|
||||
|
||||
The `is_path_allowed()` function implements multiple layers of protection to enforce workspace confinement and prevent directory traversal.
|
||||
|
||||
### Validation Logic
|
||||
|
||||
```rust
|
||||
pub fn is_path_allowed(&self, path: &str) -> bool {
|
||||
// 1. Null-byte injection guard
|
||||
if path.contains('\0') { return false; }
|
||||
|
||||
// 2. Directory traversal detection
|
||||
if Path::new(path).components().any(|c| matches!(c, Component::ParentDir)) {
|
||||
return false;
|
||||
}
|
||||
|
||||
// 3. URL-encoding detection (..%2f)
|
||||
let lower = path.to_lowercase();
|
||||
if lower.contains("..%2f") || lower.contains("%2f..") {
|
||||
return false;
|
||||
}
|
||||
|
||||
// 4. Tilde expansion guard (blocks ~user forms)
|
||||
if path.starts_with('~') && path != "~" && !path.starts_with("~/") {
|
||||
return false;
|
||||
}
|
||||
|
||||
// 5. Absolute path block and forbidden prefix match
|
||||
let expanded = expand_user_path(path);
|
||||
if self.workspace_only && expanded.is_absolute() { return false; }
|
||||
// ... (forbidden path checks)
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Sandboxing
|
||||
|
||||
ZeroClaw supports multiple isolation backends via a unified `Sandbox` trait, allowing for varied levels of process and filesystem isolation.
|
||||
|
||||
- **Landlock**: Linux-native LSM for fine-grained filesystem restriction.
|
||||
- **Bubblewrap**: Unprivileged sandboxing utility (used by Flatpak).
|
||||
- **Docker**: Containerized isolation for high-risk environments.
|
||||
- **Firejail**: SUID-based sandbox for easy desktop application isolation.
|
||||
|
||||
### Landlock Implementation snippet:
|
||||
|
||||
```rust
|
||||
fn apply_restrictions(&self) -> std::io::Result<()> {
|
||||
let mut ruleset = Ruleset::default()
|
||||
.handle_access(AccessFs::ReadFile | AccessFs::WriteFile | AccessFs::ReadDir | ...)
|
||||
.and_then(|ruleset| ruleset.create())?;
|
||||
|
||||
// Grant access only to workspace and necessary system paths (/usr, /bin)
|
||||
if let Some(ref workspace) = self.workspace_dir {
|
||||
ruleset = ruleset.add_rule(PathBeneath::new(workspace_fd, ...))?;
|
||||
}
|
||||
ruleset.restrict_self()
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Secret Management
|
||||
|
||||
The `SecretStore` protects API keys and credentials using authenticated encryption (ChaCha20-Poly1305).
|
||||
|
||||
```rust
|
||||
pub struct SecretStore {
|
||||
key_path: PathBuf, // ~/.zeroclaw/.secret_key (mode 0600)
|
||||
enabled: bool,
|
||||
}
|
||||
```
|
||||
|
||||
Encryption ensures that secrets are never stored in plaintext in configuration files, preventing accidental exposure via `grep`, `git log`, or file sharing.
|
||||
|
||||
***
|
||||
|
||||
## Audit Logging
|
||||
|
||||
Every security-relevant event is recorded in a structured JSONL format. This provides a forensic trail of what was executed, by whom, and whether it was permitted by policy.
|
||||
|
||||
```rust
|
||||
pub struct AuditEvent {
|
||||
pub timestamp: DateTime<Utc>,
|
||||
pub event_type: AuditEventType,
|
||||
pub actor: Option<Actor>,
|
||||
pub action: Option<Action>,
|
||||
pub result: Option<ExecutionResult>,
|
||||
pub security: SecurityContext,
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Emergency Stop (E-Stop)
|
||||
|
||||
The `EstopManager` provides a "big red button" capability to immediately halt or restrict agent activity.
|
||||
|
||||
- **Levels**: `KillAll`, `NetworkKill`, `DomainBlock`, `ToolFreeze`.
|
||||
- **Fail-Closed**: If the state file is corrupt or unreadable, the system defaults to `KillAll`.
|
||||
- **OTP Requirement**: Resuming operations after an E-Stop engagement can be configured to require a One-Time Password (TOTP) to ensure human authorization.
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
Luna currently lacks a security framework. Adopting ZeroClaw's security stack is critical before enabling tool execution in non-trusted environments.
|
||||
|
||||
### Prioritized Adoption Order
|
||||
|
||||
1. **Filesystem Guards + Workspace Scoping**: Implement `is_path_allowed()` to prevent Luna from escaping its designated project directory.
|
||||
2. **Command Allowlist**: Restrict shell tools to a known-safe subset with defined autonomy levels.
|
||||
3. **Path Validation**: Integrate component-aware path checking in all file-handling tools.
|
||||
4. **Secret Store**: Migrate Luna's configuration to use ChaCha20-Poly1305 for API keys.
|
||||
5. **Audit Logging**: Implement structured event logging for all tool invocations.
|
||||
6. **Sandboxing**: Introduce Landlock or Docker backends for executing untrusted code.
|
||||
|
||||
See also: [[Core]], [[Skills]], [[Configuration]].
|
||||
@@ -0,0 +1,65 @@
|
||||
# Sessions
|
||||
|
||||
ZeroClaw implements a robust session management system designed for multi-user environments and crash resilience. Unlike simple stateless request-response loops, ZeroClaw treats every interaction as part of a persistent, per-sender conversation that survives process restarts and handles context growth through automated compaction.
|
||||
|
||||
***
|
||||
|
||||
## Per-Sender Storage
|
||||
|
||||
Each conversation is isolated and keyed by a combination of the sender ID and the communication channel. This ensures that users on different platforms (e.g., Discord vs. Slack) or multiple users within the same platform maintain independent histories.
|
||||
|
||||
- **Session Keying**: Sessions are typically indexed by `(channel_id, sender_id)`.
|
||||
- **Isolation**: Memory and history from one sender never bleed into another's session unless explicitly shared via global memory.
|
||||
- **Persistence**: Sessions are serialized and stored as JSON files, allowing the agent to resume exactly where it left off after a restart.
|
||||
|
||||
***
|
||||
|
||||
## Compaction Logic
|
||||
|
||||
To prevent context window overflow and maintain performance over long conversations, ZeroClaw uses an automated compaction mechanism defined in `agent/loop_.rs`. When the history grows beyond a specific threshold, old messages are summarized and replaced by a concise context block.
|
||||
|
||||
- **Threshold**: Compaction triggers when the non-system message count exceeds `DEFAULT_MAX_HISTORY_MESSAGES = 50`.
|
||||
- **Retention**: ZeroClaw keeps the `COMPACTION_KEEP_RECENT_MESSAGES = 20` most recent messages in their original form.
|
||||
- **Source Cap**: The transcript passed to the summarizer is limited to `COMPACTION_MAX_SOURCE_CHARS = 12_000` to avoid overwhelming the summarization model.
|
||||
- **Summary Cap**: The resulting summary is truncated to `COMPACTION_MAX_SUMMARY_CHARS = 2_000`.
|
||||
|
||||
The summarization process is handled by a recursive call to the provider, instructing it to preserve key facts, user preferences, and unresolved tasks while omitting filler and verbose tool logs.
|
||||
|
||||
***
|
||||
|
||||
## Autosave Thresholds
|
||||
|
||||
ZeroClaw prioritizes data integrity through aggressive autosaving. Sessions are not just saved on graceful shutdown; they are updated after every significant interaction.
|
||||
|
||||
- **Trigger**: `AUTOSAVE_MIN_MESSAGE_CHARS = 20`.
|
||||
- **Behavior**: If a user message exceeds this length, it is considered "meaningful" enough to trigger an immediate save of the session state.
|
||||
- **Crash Resilience**: This granular saving ensures that even in the event of a sudden crash or hardware failure, the agent loses at most the very last exchange.
|
||||
|
||||
***
|
||||
|
||||
## Per-Sender Overrides
|
||||
|
||||
Each session can carry its own configuration, allowing for dynamic behavior based on the specific user or channel requirements.
|
||||
|
||||
- **Model Overrides**: A specific conversation can be configured to use a different provider or model than the global default.
|
||||
- **Config Storage**: These overrides are stored alongside the session JSON, ensuring consistency across restarts.
|
||||
|
||||
***
|
||||
|
||||
## Session Lifecycle
|
||||
|
||||
1. **Creation**: A new session is initialized when a message is received from a previously unknown `(channel, sender)` pair.
|
||||
2. **Loading**: Upon receiving a message, ZeroClaw checks the local storage for an existing session file matching the sender's key.
|
||||
3. **Processing**: Messages are appended to the history, and compaction is checked before sending the request to the LLM.
|
||||
4. **Saving**: The session is written to disk after the LLM responds or when the autosave threshold is met.
|
||||
5. **Cleanup**: Older sessions can be archived or deleted based on global retention policies, though ZeroClaw typically favors long-term persistence.
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
Luna's SessionManager currently implements basic compaction via the LibrarianAgent, but lacks several of ZeroClaw's more robust features. Integrating these would significantly improve reliability and flexibility:
|
||||
|
||||
- **Autosave**: Implementing the `AUTOSAVE_MIN_MESSAGE_CHARS` logic to move away from shutdown-only saves. This is critical for [[Core]] stability.
|
||||
- **Per-Sender Overrides**: Enabling users to select specific models for their sessions, a feature currently missing from Luna's [[Configuration]].
|
||||
- **Configurable Thresholds**: Exposing compaction constants (like 50/20 messages) as configurable parameters rather than hardcoded values.
|
||||
@@ -0,0 +1,167 @@
|
||||
# Skills
|
||||
|
||||
The ZeroClaw skills system enables the extension of agent capabilities through modular, audited packages. Each skill can define custom prompts, tools (shell, HTTP, or scripts), and metadata. This system serves as the reference architecture for [[Skills]] in Luna.
|
||||
|
||||
***
|
||||
|
||||
## Skill Struct
|
||||
|
||||
ZeroClaw defines skills and their associated tools using robust Rust structures. A `Skill` is the top-level container, while `SkillTool` defines specific executable capabilities.
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct Skill {
|
||||
pub name: String,
|
||||
pub description: String,
|
||||
pub version: String,
|
||||
#[serde(default)]
|
||||
pub author: Option<String>,
|
||||
#[serde(default)]
|
||||
pub tags: Vec<String>,
|
||||
#[serde(default)]
|
||||
pub tools: Vec<SkillTool>,
|
||||
#[serde(default)]
|
||||
pub prompts: Vec<String>,
|
||||
#[serde(skip)]
|
||||
pub location: Option<PathBuf>,
|
||||
}
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
pub struct SkillTool {
|
||||
pub name: String,
|
||||
pub description: String,
|
||||
/// "shell", "http", "script"
|
||||
pub kind: String,
|
||||
/// The command/URL/script to execute
|
||||
pub command: String,
|
||||
#[serde(default)]
|
||||
pub args: HashMap<String, String>,
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## SKILL.toml Manifest
|
||||
|
||||
The preferred way to define a skill is via a `SKILL.toml` manifest. ZeroClaw also supports a legacy `SKILL.md` fallback for prompt-only skills.
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
struct SkillManifest {
|
||||
skill: SkillMeta,
|
||||
#[serde(default)]
|
||||
tools: Vec<SkillTool>,
|
||||
#[serde(default)]
|
||||
prompts: Vec<String>,
|
||||
}
|
||||
```
|
||||
|
||||
### Example SKILL.toml
|
||||
```toml
|
||||
[skill]
|
||||
name = "weather"
|
||||
description = "Fetch weather forecasts"
|
||||
version = "0.1.0"
|
||||
author = "Luna-Team"
|
||||
tags = ["utility", "api"]
|
||||
|
||||
[[tools]]
|
||||
name = "get_weather"
|
||||
description = "Fetch forecast from wttr.in"
|
||||
kind = "shell"
|
||||
command = "curl -s wttr.in/$CITY"
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Security Audit on Install
|
||||
|
||||
Security is a first-class citizen in ZeroClaw. The `load_skills()` function enforces a security gate via `audit::audit_skill_directory()`.
|
||||
|
||||
What gets blocked:
|
||||
- Symlinks (both for the directory itself and files within)
|
||||
- Unsafe path patterns (traversal attempts)
|
||||
- Insecure script patterns
|
||||
|
||||
The `copy_dir_recursive_secure` function ensures that no symlinks are introduced during the installation process:
|
||||
|
||||
```rust
|
||||
fn copy_dir_recursive_secure(src: &Path, dest: &Path) -> Result<()> {
|
||||
let src_meta = std::fs::symlink_metadata(src)?;
|
||||
if src_meta.file_type().is_symlink() {
|
||||
anyhow::bail!("Refusing to copy symlinked skill source path: {}", src.display());
|
||||
}
|
||||
// ... recursive copy logic ...
|
||||
if metadata.file_type().is_symlink() {
|
||||
anyhow::bail!("Refusing to copy symlink within skill source: {}", src_path.display());
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Open-Skills Repository Sync
|
||||
|
||||
ZeroClaw supports a community-driven repository of skills. The system performs a weekly shallow clone to discover and update these capabilities.
|
||||
|
||||
- **URL**: `https://github.com/besoeasy/open-skills`
|
||||
- **Sync Interval**: 7 days (`60 * 60 * 24 * 7` seconds)
|
||||
- **Mechanism**: `git clone --depth 1` for initialization, followed by periodic `git pull --ff-only`.
|
||||
|
||||
The `ensure_open_skills_repo` function manages this lifecycle, checking a `.zeroclaw-open-skills-sync` marker file to determine if a sync is required.
|
||||
|
||||
***
|
||||
|
||||
## Skill Prompt Injection
|
||||
|
||||
Skills are surfaced to the LLM through XML-structured injection in the system prompt. This is handled by `skills_to_prompt_with_mode()`.
|
||||
|
||||
### Full vs Compact Mode
|
||||
- **Full**: Injects name, description, location, all instructions, and tool metadata.
|
||||
- **Compact**: Injects only name, description, and location. Instructions and tools are loaded on demand by the LLM reading the file.
|
||||
|
||||
```rust
|
||||
pub fn skills_to_prompt_with_mode(
|
||||
skills: &[Skill],
|
||||
workspace_dir: &Path,
|
||||
mode: crate::config::SkillsPromptInjectionMode,
|
||||
) -> String {
|
||||
// ...
|
||||
for skill in skills {
|
||||
let _ = writeln!(prompt, " <skill>");
|
||||
write_xml_text_element(&mut prompt, 4, "name", &skill.name);
|
||||
write_xml_text_element(&mut prompt, 4, "description", &skill.description);
|
||||
// ...
|
||||
if matches!(mode, crate::config::SkillsPromptInjectionMode::Full) {
|
||||
// Injects <instructions> and <tools> tags
|
||||
}
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## CLI Management
|
||||
|
||||
The `handle_command()` function provides a CLI interface for managing the skills lifecycle:
|
||||
- `list`: Displays installed skills, versions, tools, and tags.
|
||||
- `audit`: Manually runs the security audit on a local or installed skill.
|
||||
- `install`: Clones from git or copies from a local path, followed by a mandatory audit.
|
||||
- `remove`: Securely deletes a skill directory, preventing path traversal.
|
||||
|
||||
***
|
||||
|
||||
## Secure Install
|
||||
|
||||
When installing a skill, ZeroClaw uses `install_git_skill_source` or `install_local_skill_source`. Both paths terminate in a mandatory `enforce_skill_security_audit()` call. If the audit fails, the installed files are immediately rolled back (deleted).
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
Luna's skills system is a planned feature that will adopt the ZeroClaw architecture:
|
||||
- **Manifest**: Adopt the `SKILL.toml` format for interoperability.
|
||||
- **Security**: Implement the same mandatory audit gate and symlink rejection policy.
|
||||
- **Injection**: Use the XML-structured prompt injection pattern to give the LLM clear boundaries for skill usage.
|
||||
|
||||
Cross-links: [[Skills]], [[Core]], [[Configuration]].
|
||||
Reference in New Issue
Block a user