Add project documentation and reference materials
Include Luna AI Assistant design docs covering channels, configuration, core architecture, memory, scheduler, and skills. Add reference docs from OpenClaw and ZeroClaw projects, plus Mistral and OpenAI API specs.
This commit is contained in:
@@ -0,0 +1,70 @@
|
||||
# Channel System
|
||||
|
||||
The Channel System in OpenClaw is a highly decoupled messaging platform integration layer. It allows OpenClaw to interface with diverse services like WhatsApp, Telegram, Slack, and Discord through a unified set of interfaces while preserving platform-specific capabilities.
|
||||
|
||||
***
|
||||
|
||||
## Plugin-Per-Platform Pattern
|
||||
|
||||
OpenClaw employs a strict plugin architecture where every supported messaging platform is a self-contained module located under `src/channels/plugins/`. This directory contains over 100 files, with each platform adapter implementing a standard set of interfaces defined in `types.plugin.ts` and `types.adapters.ts`.
|
||||
|
||||
Key aspects of this pattern include:
|
||||
* **Isolation**: Each plugin maintains its own dependencies and platform-specific logic (e.g., the Telegram plugin handles its own bot API calls).
|
||||
* **Standardized Lifecycle**: Plugins implement `ChannelLifecycleAdapter` to manage startup, shutdown, and health checks.
|
||||
* **Feature-Based Opt-in**: Plugins advertise their capabilities (e.g., `threads`, `reactions`, `media`) via a `ChannelCapabilities` object, allowing the core to gracefully degrade or enhance features per-channel.
|
||||
|
||||
***
|
||||
|
||||
## Shared Registry & Helpers
|
||||
|
||||
A centralized `registry.ts` manages all active channel plugins. Instead of hardcoding platform logic into the core, the registry provides a discovery mechanism for the system to interact with whatever plugins are currently loaded.
|
||||
|
||||
### Message Normalization
|
||||
Incoming raw messages from various platforms are normalized into a standard `MsgContext` before reaching the agent or session logic. This normalization ensures consistent handling of:
|
||||
* **Sender Identification**: Mapping platform-specific IDs to a common structure containing `SenderId`, `SenderName`, and `SenderUsername`.
|
||||
* **Thread Tracking**: Normalizing `ThreadId` and `ReplyToId` so the core can track conversations across platforms that represent threads differently (e.g., Slack's thread timestamps vs. Telegram's reply-to message IDs).
|
||||
* **Channel Metadata**: Attaching `Channel` (platform name) and `ChatType` (direct, group, or channel) to every inbound payload.
|
||||
|
||||
### Shared Utilities
|
||||
OpenClaw provides several helper modules that plugins use to reduce boilerplate:
|
||||
* `sender-identity.ts`: Validates and sanitizes sender metadata.
|
||||
* `chat-meta.ts`: Manages channel-level metadata like labels, blurbs, and documentation links.
|
||||
* `session-envelope.ts`: Handles the wrapping of messages for persistent storage.
|
||||
|
||||
***
|
||||
|
||||
## Access Control & Routing
|
||||
|
||||
OpenClaw enforces security and session boundaries at the channel level.
|
||||
|
||||
### Allowlist-based Access
|
||||
The system uses an allowlist-based policy for each channel. The `allowlist-match.ts` and `allow-from.ts` utilities provide logic to verify if a specific sender or group is permitted to interact with the agent. This ties directly into [[Security]] policies, preventing unauthorized access before a session is even initialized.
|
||||
|
||||
### Session Routing
|
||||
Incoming messages are routed to specific sessions based on their origin:
|
||||
* **Direct Messages (DMs)**: Usually routed to a per-sender session.
|
||||
* **Group Chats**: Messages are routed to a session keyed by the group ID.
|
||||
* **Thread Binding**: `thread-bindings-policy.ts` determines if a message should stay within an existing session or trigger the creation of a child session.
|
||||
|
||||
***
|
||||
|
||||
## Channel-Specific Auth
|
||||
|
||||
Authentication is delegated to the individual plugins through the `ChannelAuthAdapter`. Each platform handles its own credential requirements:
|
||||
* **Bot Tokens**: Simple token-based auth (Telegram, Discord).
|
||||
* **OAuth**: Flow-based authentication for user-level access (Slack, Matrix).
|
||||
* **QR Code Pairing**: Used by platforms like WhatsApp or Signal to link an existing account.
|
||||
|
||||
The `ChannelSetupAdapter` provides a standardized `ChannelSetupInput` bag containing fields for `botToken`, `appToken`, `privateKey`, and more, which are then stored in the system's secure configuration.
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
The OpenClaw channel system provides a blueprint for expanding Luna beyond its current CLI-only interface.
|
||||
|
||||
Key lessons for [[Channels]]:
|
||||
* **Establish a Rich Normalized Type**: Before adding a second channel, Luna should define a robust internal message format. This prevents the core logic from becoming littered with "if platform is Discord" checks.
|
||||
* **Abstract Outbound Routing**: By defining a common `ChannelOutboundAdapter`, Luna can send messages to any platform using a unified API, even if the underlying delivery mechanism (WebSocket, HTTP POST, CLI print) varies wildly.
|
||||
|
||||
Cross-references: [[Core]], [[Security]]
|
||||
@@ -0,0 +1,142 @@
|
||||
# Compaction Strategy
|
||||
|
||||
OpenClaw employs a multi-layered conversation compaction system to manage long-running sessions within finite model context windows. Unlike basic truncation, this system uses LLM-driven summarization, token-budget gating, and quality safeguards to preserve critical state, identifiers, and recent context.
|
||||
|
||||
***
|
||||
|
||||
## Trigger & Token Gating
|
||||
|
||||
Compaction is primarily triggered by "preflight" checks before processing a new turn, ensuring the model has sufficient headroom for its next response. This process is managed in `auto-reply/reply/agent-runner-memory.ts` and `memory-flush.ts`.
|
||||
|
||||
### Gating Logic
|
||||
The decision to compact is based on a projected token count compared against a computed threshold:
|
||||
- **Threshold Formula**: `contextWindowTokens - reserveTokensFloor - softThresholdTokens`
|
||||
- **Reserve Floor**: Defaults to 20,000 tokens (`reserveTokensFloor`), providing a safety buffer for tool outputs and reasoning.
|
||||
- **Soft Threshold**: An additional 4,000 token buffer (`softThresholdTokens`) to prevent compaction oscillations at the exact limit.
|
||||
- **Token Estimation**: Performed by `estimateMessagesTokens()`, which strips `toolResult.details` for security and applies a `SAFETY_MARGIN` of 1.2 to account for estimation inaccuracies.
|
||||
|
||||
### Execution Hooks
|
||||
- **shouldRunPreflightCompaction**: Runs before a turn to ensure the input fits the budget.
|
||||
- **shouldRunMemoryFlush**: Evaluates if the session should be "flushed" to long-term memory based on token pressure.
|
||||
- **Deduplication**: `hasAlreadyFlushedForCurrentCompaction` prevents redundant flushes within a single compaction cycle.
|
||||
- **Manual Trigger**: The `/compact` command allows users to manually force a compaction cycle.
|
||||
|
||||
***
|
||||
|
||||
## Summarization Core
|
||||
|
||||
The core summarization engine in `agents/compaction.ts` orchestrates the transformation of raw history into structured summaries.
|
||||
|
||||
### Resource Allocation
|
||||
- **SUMMARIZATION_OVERHEAD_TOKENS**: 4,096 tokens are reserved for the summarization prompt, system instructions, and previous summaries.
|
||||
- **Adaptive Chunking**: `computeAdaptiveChunkRatio` shrinks chunk sizes as the average message size increases. If a single message exceeds 50% of the context window, it is flagged as `isOversizedForSummary` and handled via fallback mechanisms.
|
||||
|
||||
### Orchestration Pipeline
|
||||
1. **summarizeChunks**: Splits the history into chunks based on `maxChunkTokens`.
|
||||
2. **summarizeWithFallback**: Attempts a full summary. On failure, it separates "small" messages from "oversized" ones, summarizing the small messages and annotating the oversized ones (e.g., `[Large message (~15K tokens) omitted from summary]`).
|
||||
3. **summarizeInStages**: For very large histories, it generates partial summaries and then merges them using `MERGE_SUMMARIES_INSTRUCTIONS`.
|
||||
|
||||
### Preservation Priorities
|
||||
The system uses `MERGE_SUMMARIES_INSTRUCTIONS` to ensure the model retains:
|
||||
- Active tasks and current status (in-progress, blocked, pending).
|
||||
- Commitments, decisions, and their rationale.
|
||||
- Unresolved user asks and key factual identifiers.
|
||||
- Recent context over older history.
|
||||
|
||||
***
|
||||
|
||||
## Safeguard Extension
|
||||
|
||||
The `compaction-safeguard.ts` hook acts as a safety layer, registering on `session_before_compact` to manage context sharing and content preservation.
|
||||
|
||||
### Context Preservation
|
||||
- **Recent Turns**: Preserves `DEFAULT_RECENT_TURNS_PRESERVE = 3` turns verbatim to maintain immediate conversational flow.
|
||||
- **History Pruning**: `pruneHistoryForContextShare` drops older chunks if the new content consumes too much of the history budget. Dropped messages are summarized and prepended as a `previousSummary`.
|
||||
- **Suffix Protection**: Critical metadata is appended to a protected suffix that survives truncation:
|
||||
- Tool failures (capped at 8 failures).
|
||||
- File operations (read/modified lists).
|
||||
- Workspace rules (extracted from `AGENTS.md`).
|
||||
|
||||
### Length Constraints
|
||||
- **MAX_COMPACTION_SUMMARY_CHARS**: 16,000 characters cap for the total summary.
|
||||
- **MAX_FILE_OPS_SECTION_CHARS**: 2,000 characters for file operation logs.
|
||||
- **MAX_FILE_OPS_LIST_CHARS**: 900 characters for the list of files.
|
||||
|
||||
***
|
||||
|
||||
## Quality Guard
|
||||
|
||||
The Quality Guard (`compaction-safeguard-quality.ts`) ensures the LLM-generated summary meets strict structural and content requirements.
|
||||
|
||||
### Required Sections
|
||||
Every summary must contain these exact Markdown headings:
|
||||
- `## Decisions`
|
||||
- `## Open TODOs`
|
||||
- `## Constraints/Rules`
|
||||
- `## Pending user asks`
|
||||
- `## Exact identifiers`
|
||||
|
||||
### Identifier Preservation
|
||||
The system extracts opaque identifiers (URLs, file paths, hex IDs, ports) using regex and enforces their preservation.
|
||||
- **Strict Policy**: If `identifierPolicy` is set to `strict`, the guard validates that all extracted identifiers appear in the final summary.
|
||||
- **Audit Loop**: `auditSummaryQuality` checks for section presence and identifier integrity. If checks fail, the system triggers a regeneration with `qualityFeedbackInstruction`.
|
||||
|
||||
***
|
||||
|
||||
## Default Instructions
|
||||
|
||||
Default behavior is governed by `compaction-instructions.ts`, which merges user-defined, runtime, and system-level instructions.
|
||||
|
||||
```typescript
|
||||
export const DEFAULT_COMPACTION_INSTRUCTIONS =
|
||||
"Write the summary body in the primary language used in the conversation.\n" +
|
||||
"Focus on factual content: what was discussed, decisions made, and current state.\n" +
|
||||
"Keep the required summary structure and section headers unchanged.\n" +
|
||||
"Do not translate or alter code, file paths, identifiers, or error messages.";
|
||||
```
|
||||
|
||||
Instructions are capped at `MAX_INSTRUCTION_LENGTH = 800` characters to prevent prompt bloat.
|
||||
|
||||
***
|
||||
|
||||
## Runtime Execution & Truncation
|
||||
|
||||
The `compact.ts` runner provides the entry point for both automated and manual compaction.
|
||||
|
||||
### Execution Flow
|
||||
1. **Preparation**: Opens the session, sanitizes history, and runs `before_compaction` hooks.
|
||||
2. **Safety Timeout**: Wraps the LLM call in `compactWithSafetyTimeout` to prevent hanging processes.
|
||||
3. **Post-Processing**: Runs `after_compaction` hooks and estimates the resulting token count.
|
||||
|
||||
### Session Truncation
|
||||
If enabled via `config.agents.defaults.compaction.truncateAfterCompaction`, the system physically rewrites the session JSONL file using `session-truncation.ts`.
|
||||
- **Removal**: Deletes message entries that were summarized.
|
||||
- **Re-parenting**: Re-parents orphaned entries to the nearest kept ancestor to maintain the integrity of the session tree.
|
||||
- **Archiving**: Optionally creates an archive of the original session file before truncation.
|
||||
|
||||
***
|
||||
|
||||
## Configuration Knobs
|
||||
|
||||
Compaction behavior can be tuned via `OpenClawConfig`:
|
||||
- `config.agents.defaults.compaction.model`: Override the model used for summarization.
|
||||
- `config.agents.defaults.compaction.reserveTokensFloor`: Minimum buffer (default ~20,000).
|
||||
- `config.agents.defaults.compaction.timeoutSeconds`: Max time allowed for a summarization call.
|
||||
- `config.agents.defaults.compaction.truncateAfterCompaction`: Boolean to enable physical file cleanup.
|
||||
- `memoryFlush`: Configuration for soft thresholds and forced flush triggers.
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
Luna currently uses a basic `LibrarianAgent` for compaction with no token-budget gating, no quality guards, and no structured instruction sets. To achieve OpenClaw-level reliability, Luna should adopt:
|
||||
|
||||
- **Token-Budget Gating**: Triggering compaction based on projected context usage rather than arbitrary turn counts.
|
||||
- **Structured Sections**: Enforcing a specific Markdown schema in summaries to ensure critical state is never lost.
|
||||
- **Identifier Preservation**: Using regex extraction and quality audits to protect file paths and IDs.
|
||||
- **Quality Audit Loop**: Implementing a verification step that can re-trigger summarization if requirements are missed.
|
||||
- **Session Truncation**: Physically cleaning up on-disk history files to prevent unbounded growth.
|
||||
|
||||
Patterns like the `compaction-safeguard` provide a more resilient approach for long-term project management in Luna by ensuring the project state, goals, and critical constraints are always prioritized in the model's working memory.
|
||||
|
||||
See also: [[Core]], [[Configuration]], [[Session Management]]
|
||||
@@ -0,0 +1,161 @@
|
||||
# Memory SDK
|
||||
|
||||
OpenClaw's memory system lives in a standalone package (`packages/memory-host-sdk`) that exposes composable engines for storage, embeddings, query processing, and foundation utilities. The SDK defines a clear host-engine boundary: the host application consumes the SDK surface, while the SDK encapsulates all storage, indexing, and retrieval logic behind exported contracts.
|
||||
|
||||
***
|
||||
|
||||
## Package Structure
|
||||
|
||||
The SDK entry point (`engine.ts`) re-exports four focused engine modules:
|
||||
|
||||
```typescript
|
||||
export * from "./engine-foundation.js";
|
||||
export * from "./engine-storage.js";
|
||||
export * from "./engine-embeddings.js";
|
||||
export * from "./engine-qmd.js";
|
||||
```
|
||||
|
||||
Each engine is a self-contained barrel export covering one concern. New code is directed to the focused subpaths rather than the aggregate surface.
|
||||
|
||||
***
|
||||
|
||||
## Engine Storage
|
||||
|
||||
The storage engine (`engine-storage.ts`) provides the persistence and retrieval primitives built on SQLite with the `sqlite-vec` vector extension.
|
||||
|
||||
### Key Exports
|
||||
|
||||
- **`MemoryChunk`** and **`MemoryFileEntry`** — the core data types. A chunk is an indexed segment of content with metadata; a file entry represents a source document on disk.
|
||||
- **`buildFileEntry`** / **`buildMultimodalChunkForIndexing`** — constructors for creating index-ready records.
|
||||
- **`chunkMarkdown`** — splits markdown content into semantically coherent chunks for embedding.
|
||||
- **`cosineSimilarity`** — vector distance computation for retrieval ranking.
|
||||
- **`parseEmbedding`** — deserializes stored embedding vectors.
|
||||
- **`ensureMemoryIndexSchema`** — creates or migrates the SQLite schema (tables, FTS indexes, vector columns).
|
||||
- **`loadSqliteVecExtension`** — loads the `sqlite-vec` native extension into the SQLite connection.
|
||||
- **`requireNodeSqlite`** — resolves the Node.js SQLite driver.
|
||||
- **`readMemoryFile`** — reads a memory file from disk with safety checks.
|
||||
- **`resolveMemoryBackendConfig`** — resolves backend-specific configuration (SQLite paths, vector dimensions, etc.).
|
||||
|
||||
### Storage Architecture
|
||||
|
||||
The storage layer uses SQLite as a single-file database with the `sqlite-vec` extension providing vector column support. Memory chunks are stored alongside their embeddings in the same database, enabling hybrid queries that combine full-text search with cosine vector similarity in a single SQL statement. This mirrors the approach used by ZeroClaw's SQLite backend (see [[ZeroClaw/Memory]]).
|
||||
|
||||
```typescript
|
||||
// Types exported for consumers
|
||||
type MemoryChunk = { /* chunk content, metadata, embedding vector */ };
|
||||
type MemoryFileEntry = { /* source file path, hash, modification time */ };
|
||||
type MemorySearchResult = { /* ranked results with scores and source info */ };
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Engine Embeddings
|
||||
|
||||
The embeddings engine (`engine-embeddings.ts`) provides a pluggable embedding provider system with six built-in backends and batch processing utilities.
|
||||
|
||||
### Embedding Providers
|
||||
|
||||
| Provider | Default Model | Module |
|
||||
|----------|--------------|--------|
|
||||
| Local | `DEFAULT_LOCAL_MODEL` | `embeddings.js` |
|
||||
| Gemini | `DEFAULT_GEMINI_EMBEDDING_MODEL` | `embeddings-gemini.js` |
|
||||
| Mistral | `DEFAULT_MISTRAL_EMBEDDING_MODEL` | `embeddings-mistral.js` |
|
||||
| Ollama | `DEFAULT_OLLAMA_EMBEDDING_MODEL` | `embeddings-ollama.js` |
|
||||
| OpenAI | `DEFAULT_OPENAI_EMBEDDING_MODEL` | `embeddings-openai.js` |
|
||||
| Voyage | `DEFAULT_VOYAGE_EMBEDDING_MODEL` | `embeddings-voyage.js` |
|
||||
|
||||
### Provider Interface
|
||||
|
||||
```typescript
|
||||
// Core provider contract
|
||||
type MemoryEmbeddingProvider = {
|
||||
name: string;
|
||||
// ... embedding computation methods
|
||||
};
|
||||
|
||||
type MemoryEmbeddingProviderAdapter = {
|
||||
// Adapts raw provider to the SDK contract
|
||||
};
|
||||
|
||||
// Factory and registry
|
||||
function getMemoryEmbeddingProvider(name: string): MemoryEmbeddingProvider;
|
||||
function listMemoryEmbeddingProviders(): string[];
|
||||
```
|
||||
|
||||
### Batch Processing
|
||||
|
||||
Three batch helpers handle high-throughput embedding jobs for initial indexing or re-indexing:
|
||||
|
||||
- **`runGeminiEmbeddingBatches`** — Gemini-specific batching with rate limit handling.
|
||||
- **`runOpenAiEmbeddingBatches`** — OpenAI batch API integration (`OPENAI_BATCH_ENDPOINT`).
|
||||
- **`runVoyageEmbeddingBatches`** — Voyage AI batch processing.
|
||||
|
||||
### Safety Utilities
|
||||
|
||||
- **`enforceEmbeddingMaxInputTokens`** — truncates input to provider-specific token limits.
|
||||
- **`estimateStructuredEmbeddingInputBytes`** / **`estimateUtf8Bytes`** — byte estimation for payload sizing.
|
||||
- **`hasNonTextEmbeddingParts`** — detects multimodal content that requires special handling.
|
||||
|
||||
***
|
||||
|
||||
## Engine Foundation
|
||||
|
||||
The foundation engine (`engine-foundation.ts`) re-exports core utilities from the main OpenClaw application that the memory system depends on. Rather than duplicating logic, the SDK imports these through relative paths to the monorepo `src/` directory.
|
||||
|
||||
### Key Capabilities
|
||||
|
||||
- **Agent Scope Resolution**: `resolveAgentDir`, `resolveAgentWorkspaceDir`, `resolveDefaultAgentId`, `resolveSessionAgentId` — determines which agent's memory space to operate on.
|
||||
- **Memory Search Configuration**: `resolveMemorySearchConfig` with `ResolvedMemorySearchConfig` type — controls search behavior (result limits, relevance thresholds).
|
||||
- **Configuration Loading**: `loadConfig`, `resolveStateDir` — reads OpenClaw configuration and state directories.
|
||||
- **Session Integration**: `resolveSessionTranscriptsDirForAgent`, `onSessionTranscriptUpdate` — watches for new conversation transcripts to index into memory.
|
||||
- **File Safety**: `writeFileWithinRoot` — prevents path traversal when writing memory files.
|
||||
- **Secrets**: `hasConfiguredSecretInput`, `normalizeResolvedSecretInputString` — resolves API keys for embedding providers.
|
||||
|
||||
***
|
||||
|
||||
## Engine QMD (Query Processing)
|
||||
|
||||
The QMD engine (`engine-qmd.ts`) handles query decomposition, keyword extraction, and scope filtering before executing memory searches.
|
||||
|
||||
### Query Pipeline
|
||||
|
||||
- **`extractKeywords`** — pulls meaningful terms from a natural language query, filtering stop words via `isQueryStopWordToken`.
|
||||
- **`parseQmdQueryJson`** — parses structured query results (type `QmdQueryResult`) from the QMD binary.
|
||||
- **Scope Filtering**: `deriveQmdScopeChannel`, `deriveQmdScopeChatType`, `isQmdScopeAllowed` — restricts search results to the appropriate channel and conversation type.
|
||||
- **Session Files**: `buildSessionEntry`, `listSessionFilesForAgent`, `sessionPathForFile` — manages the session transcript files that feed into memory indexing.
|
||||
- **CLI Integration**: `checkQmdBinaryAvailability`, `resolveCliSpawnInvocation`, `runCliCommand` — interfaces with an external QMD binary for query processing.
|
||||
|
||||
***
|
||||
|
||||
## Integration Pattern
|
||||
|
||||
The SDK is consumed by the host application through the aggregate `engine.ts` export. The boundary is clear:
|
||||
|
||||
- **Host responsibility**: Decides when to index, when to search, which agent scope to use, and how to present results.
|
||||
- **SDK responsibility**: Handles all storage I/O, embedding computation, vector indexing, query processing, and schema management.
|
||||
|
||||
The typical flow:
|
||||
|
||||
1. **Initialization**: Host calls `resolveMemoryBackendConfig` to set up the storage backend.
|
||||
2. **Indexing**: Host uses `engine-embeddings` to generate vectors and `engine-storage` to persist chunks.
|
||||
3. **Retrieval**: Host uses `engine-qmd` to process the query, then performs a similarity search via `engine-storage`.
|
||||
|
||||
This separation means the memory engine can be tested independently, and the host can swap backends (e.g., different SQLite configurations, different embedding providers) without changing application logic.
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
Luna's [[Memory]] module defines an `IMemoryStore` interface but the current file-based implementation only supports writing — there is no recall or search capability.
|
||||
|
||||
### Patterns to Adopt
|
||||
|
||||
- **Composable Engine Architecture**: Separating storage, embeddings, and query processing into distinct layers is a clean pattern for Luna. Rather than building a monolithic memory service, Luna could define `IMemoryStorage`, `IEmbeddingProvider`, and `IQueryProcessor` interfaces that compose into a full memory system.
|
||||
- **SQLite + Vector Extension**: Both OpenClaw and ZeroClaw use SQLite with vector extensions for hybrid search. Luna already references SQLite packages — finishing `SqliteMemoryStore` with this architecture is the natural path.
|
||||
- **Pluggable Embedding Providers**: Luna should support multiple embedding backends from the start. The factory pattern (`getMemoryEmbeddingProvider(name)`) maps to C# service registration.
|
||||
- **Batch Processing**: For initial indexing or re-indexing of conversation history, batch embedding helpers prevent rate-limiting issues.
|
||||
- **Query Preprocessing**: Keyword extraction and scope filtering before search improves relevance. Luna should implement query expansion as a pipeline step before hitting the vector store.
|
||||
|
||||
### Key Difference
|
||||
|
||||
OpenClaw's SDK is a package boundary within a monorepo — it re-exports from the main `src/` tree through barrel files. Luna, as a single compiled application, would implement this as a set of interfaces within a `Memory` namespace rather than a separate package. The architectural principle (composable engines with clear contracts) translates directly to C# DI registration.
|
||||
@@ -0,0 +1,94 @@
|
||||
# OpenClaw
|
||||
|
||||
## Overview
|
||||
OpenClaw is a mature TypeScript/Node.js AI assistant platform (~342k stars, ~24k commits). Local-first gateway control plane with plugin-SDK architecture. Monorepo with packages. Repo: https://github.com/openclaw/openclaw
|
||||
|
||||
## Architecture
|
||||
- Local-first WebSocket gateway as central control plane
|
||||
- Plugin-SDK architecture — features are installable packages, not inline code
|
||||
- Monorepo: `src/` for core, `packages/` for SDKs
|
||||
|
||||
## Agent Runtime
|
||||
- Pi agent runtime with RPC mode
|
||||
- Tool streaming and block streaming
|
||||
- Multi-agent routing
|
||||
- Agent compaction for long conversations
|
||||
|
||||
## Providers
|
||||
- Multiple AI providers via plugin-sdk
|
||||
- OAuth subscription auth (OpenAI Codex, Claude Code)
|
||||
|
||||
## Channels
|
||||
24+ messaging platforms via plugin-per-platform pattern with shared registry/helpers:
|
||||
WhatsApp, Telegram, Slack, Discord, Signal, iMessage, Matrix, IRC, Teams, LINE, Nostr, WeChat, and more.
|
||||
Each channel plugin: channel-specific auth, message normalization, allowlist-based access control.
|
||||
|
||||
## Memory
|
||||
- memory-host-sdk package with three engines:
|
||||
- engine-embeddings (vector embedding computation/storage)
|
||||
- engine-foundation (core memory primitives)
|
||||
- engine-storage (pluggable storage backends)
|
||||
- Pluggable vector DB support for semantic search
|
||||
|
||||
## Tools
|
||||
Rich tool ecosystem:
|
||||
- Browser: Managed Chromium via CDP
|
||||
- Canvas: A2UI visual workspace
|
||||
- Cron: Scheduled tasks
|
||||
- Nodes: Device-local (camera, screen, notifications)
|
||||
- Sessions: Session management tools
|
||||
- Web Fetch: HTTP with SSRF guards and visibility rules
|
||||
- PDF, TTS, Image tools
|
||||
|
||||
## Sessions
|
||||
- JSON-persisted session store
|
||||
- Pruning, capping, rotation, archiving
|
||||
- Per-session sandbox policies
|
||||
- Activation modes, group routing
|
||||
- Session lifecycle events
|
||||
|
||||
## Security
|
||||
- DM Pairing: 6-digit codes + per-channel allowlists
|
||||
- Per-session Sandboxing: Docker/SSH backends
|
||||
- Tool Allow/Deny Lists
|
||||
- SSRF Guards on web fetch
|
||||
- Web Fetch Visibility Rules
|
||||
|
||||
## Skills & Plugins
|
||||
Full plugin lifecycle:
|
||||
- Manifest Registry with capability declarations
|
||||
- Install/enable/disable/uninstall lifecycle
|
||||
- Workspace Skills via AGENTS.md and SKILL.md conventions
|
||||
- Bundled plugins + external ClawHub package registry
|
||||
|
||||
## Voice & Apps
|
||||
- Wake word detection, Talk Mode, TTS (ElevenLabs + system fallback)
|
||||
- macOS menu bar app, iOS/Android nodes, WebChat browser client
|
||||
|
||||
***
|
||||
|
||||
## In-Depth Reference Pages
|
||||
|
||||
- [[Session Management]] — SessionEntry type, store maintenance (pruning/capping/rotation), atomic writes, lifecycle events
|
||||
- [[Plugin Architecture]] — Plugin type system, 40+ provider hooks, tool factories, config schemas, channel handlers
|
||||
- [[Channel System]] — Plugin-per-platform pattern, shared registry, message normalization, access control
|
||||
- [[Security]] — DM/group access decisions, pairing, allowlists, per-session sandboxing, SSRF guards
|
||||
- [[Memory SDK]] — Composable engine architecture (storage, embeddings, QMD), SQLite + sqlite-vec, 6 embedding providers
|
||||
- [[Compaction Strategy]] — Token-budget gating, staged summarization, quality guards, identifier preservation, session truncation
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
### Patterns Worth Adopting
|
||||
- **Session Management**: Pruning/capping/rotation is much more mature than Luna's basic SessionManager. Configurable max message counts, session lifecycle events, group routing. See [[Session Management]] for details. Link to [[Core]].
|
||||
- **Plugin Architecture**: Manifest-based registry with install/enable/disable lifecycle is a good reference for Luna's [[Skills]] module. See [[Plugin Architecture]] for the hook system.
|
||||
- **Channel Normalization**: Plugin-per-platform with shared registry validates Luna's [[Channels]] design. Key lesson: establish a rich normalized message type before adding the second channel. See [[Channel System]].
|
||||
- **Security Layers**: DM pairing and per-session sandboxing are practical patterns Luna should adopt before exposing tool execution. See [[Security]] for the access decision system.
|
||||
- **Compaction**: Token-budget driven summarization with quality guards and structured output sections. Far more sophisticated than Luna's basic LibrarianAgent compaction. See [[Compaction Strategy]].
|
||||
|
||||
### Key Differences from Luna
|
||||
- OpenClaw is a platform for third-party plugin development; Luna is a standalone assistant
|
||||
- TypeScript vs .NET/C#
|
||||
- Features are installable packages vs compiled modules
|
||||
- Mature multi-app distribution (macOS, iOS, Android, WebChat) vs CLI only
|
||||
@@ -0,0 +1,131 @@
|
||||
# Plugin Architecture
|
||||
|
||||
OpenClaw employs a multi-tiered plugin architecture designed for high extensibility across model providers, messaging channels, and agent capabilities. The system is built on a massive type system that allows plugins to hook into almost every stage of the AI lifecycle, from initial message receipt to final inference and tool execution.
|
||||
|
||||
***
|
||||
|
||||
## Plugin Type System
|
||||
|
||||
The core of the architecture is defined in `src/plugins/types.ts`. It provides a unified contract for different plugin flavors: `ProviderPlugin`, `WebSearchProviderPlugin`, `SpeechProviderPlugin`, and the general `OpenClawPluginDefinition`.
|
||||
|
||||
### Provider Plugin Hooks
|
||||
|
||||
The `ProviderPlugin` is the most complex type, featuring over 40 hooks for deep integration with LLM providers. These are categorized into distinct functional areas:
|
||||
|
||||
* **Catalog & Discovery**: Hooks like `catalog` and `augmentModelCatalog` allow plugins to publish model definitions dynamically.
|
||||
* **Auth & Credentials**: `prepareRuntimeAuth` and `resolveSyntheticAuth` handle the exchange of source credentials for runtime tokens.
|
||||
* **Model Resolution**: `resolveDynamicModel` and `normalizeResolvedModel` provide last-mile control over how model IDs are mapped to API endpoints.
|
||||
* **Streaming & Transport**: `createStreamFn` and `wrapStreamFn` allow plugins to replace or wrap the default transport layer with provider-specific logic.
|
||||
* **Usage & Billing**: `resolveUsageAuth` and `fetchUsageSnapshot` enable integrated quota tracking.
|
||||
|
||||
```typescript
|
||||
export type ProviderPlugin = {
|
||||
id: string;
|
||||
label: string;
|
||||
auth: ProviderAuthMethod[];
|
||||
catalog?: ProviderPluginCatalog;
|
||||
resolveDynamicModel?: (ctx: ProviderResolveDynamicModelContext) => ProviderRuntimeModel | null | undefined;
|
||||
prepareRuntimeAuth?: (ctx: ProviderPrepareRuntimeAuthContext) => Promise<ProviderPreparedRuntimeAuth | null | undefined>;
|
||||
createStreamFn?: (ctx: ProviderCreateStreamFnContext) => StreamFn | null | undefined;
|
||||
fetchUsageSnapshot?: (ctx: ProviderFetchUsageSnapshotContext) => Promise<ProviderUsageSnapshot | null | undefined>;
|
||||
// ... and 30+ more hooks
|
||||
};
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Tool Factories
|
||||
|
||||
Plugins expose agent-level capabilities through `OpenClawPluginToolFactory`. This pattern allows tools to be instantiated with a trusted execution context that includes workspace paths, session identifiers, and security boundaries.
|
||||
|
||||
```typescript
|
||||
export type OpenClawPluginToolContext = {
|
||||
config?: OpenClawConfig;
|
||||
workspaceDir?: string;
|
||||
agentId?: string;
|
||||
sessionId?: string;
|
||||
deliveryContext?: DeliveryContext;
|
||||
senderIsOwner?: boolean;
|
||||
};
|
||||
|
||||
export type OpenClawPluginToolFactory = (
|
||||
ctx: OpenClawPluginToolContext,
|
||||
) => AnyAgentTool | AnyAgentTool[] | null | undefined;
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Config Schema
|
||||
|
||||
Plugins declare their configuration requirements using a Zod-like validation system. The `OpenClawPluginConfigSchema` allows the host to validate plugin settings, generate UI forms, and provide help text without loading the plugin's full implementation.
|
||||
|
||||
```typescript
|
||||
export type OpenClawPluginConfigSchema = {
|
||||
safeParse?: (value: unknown) => {
|
||||
success: boolean;
|
||||
data?: unknown;
|
||||
error?: { issues?: Array<{ path: Array<string | number>; message: string }> };
|
||||
};
|
||||
uiHints?: Record<string, PluginConfigUiHint>;
|
||||
jsonSchema?: Record<string, unknown>;
|
||||
};
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Channel Handlers
|
||||
|
||||
Interactive platform support (Telegram, Discord, Slack) is handled via specialized interactive handlers. These allow plugins to respond to platform-specific events like button clicks or modal submissions through a unified context.
|
||||
|
||||
```typescript
|
||||
export type PluginInteractiveTelegramHandlerContext = {
|
||||
channel: "telegram";
|
||||
callback: { data: string; namespace: string; payload: string };
|
||||
respond: {
|
||||
reply: (params: { text: string; buttons?: PluginInteractiveButtons }) => Promise<void>;
|
||||
editMessage: (params: { text: string; buttons?: PluginInteractiveButtons }) => Promise<void>;
|
||||
};
|
||||
};
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Conversation Bindings
|
||||
|
||||
Plugins can request "Conversation Bindings" to take over the message flow for a specific thread or user. This is used for interactive wizards or stateful interactions that bypass the general LLM dispatcher.
|
||||
|
||||
* `requestConversationBinding`: Attaches a plugin to the current conversation.
|
||||
* `detachConversationBinding`: Releases the conversation back to the general agent.
|
||||
* `getCurrentConversationBinding`: Checks if the conversation is currently owned by a plugin.
|
||||
|
||||
***
|
||||
|
||||
## Plugin Lifecycle
|
||||
|
||||
The lifecycle is managed through several stages, primarily defined in the `OpenClawPluginApi` provided during registration:
|
||||
|
||||
1. **Discovery**: Plugins are found in `bundled`, `global`, or `workspace` directories.
|
||||
2. **Registration**: `register(api)` is called. The plugin registers its tools, hooks, and services.
|
||||
3. **Activation**: `activate(api)` is called when the plugin is enabled and ready for service.
|
||||
4. **Runtime**: The plugin responds to lifecycle hooks such as `before_agent_start`, `llm_input`, and `agent_end`.
|
||||
5. **Uninstallation**: Managed via the `ClawHub` registry or local file deletion.
|
||||
|
||||
***
|
||||
|
||||
## Specialized Plugin Types
|
||||
|
||||
### Web Search Providers
|
||||
`WebSearchProviderPlugin` defines how the agent interacts with search engines. It requires specific credential resolution logic and a `createTool` factory that returns a standardized search tool definition.
|
||||
|
||||
### Speech Providers
|
||||
`SpeechProviderPlugin` handles text-to-speech (TTS) capabilities. It includes hooks for voice listing, directive parsing (e.g., for custom SSML or tokens), and synthesis for both standard and telephony audio formats.
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
While Luna does not require the full overhead of OpenClaw's provider ecosystem, it can adopt several key patterns:
|
||||
|
||||
* **Manifest-Based Registration**: Luna could use a simplified version of `OpenClawPluginDefinition` to allow for [[Skills]] that declare their dependencies and config schemas upfront.
|
||||
* **Hook Interceptors**: Adopting the `before_prompt_build` and `llm_output` hook patterns would allow Luna's [[Core]] to support middleware for logging, safety filtering, or memory injection.
|
||||
* **Trusted Contexts**: Passing a `ToolContext` similar to OpenClaw's would ensure that Luna's tools have consistent access to session and workspace state without relying on global variables.
|
||||
@@ -0,0 +1,151 @@
|
||||
# Security
|
||||
|
||||
OpenClaw implements a multi-layered security model designed to isolate tool execution and strictly control access via Direct Messages (DMs) and Groups. This model ensures that only authorized users can trigger sensitive operations while protecting the host system from potentially malicious tool output or SSRF attacks.
|
||||
|
||||
***
|
||||
|
||||
## DM/Group Access Decision System
|
||||
|
||||
The core logic for determining if a message should be processed resides in `src/security/dm-policy-shared.ts`. The system evaluates the `dmPolicy` and `groupPolicy` against the sender's identity.
|
||||
|
||||
### Access Decision Types
|
||||
|
||||
The `DmGroupAccessDecision` type defines the three possible outcomes of an access check:
|
||||
|
||||
```typescript
|
||||
export type DmGroupAccessDecision = "allow" | "block" | "pairing";
|
||||
```
|
||||
|
||||
### Access Reason Codes
|
||||
|
||||
The system provides granular reason codes for every decision, enabling precise logging and user feedback:
|
||||
|
||||
```typescript
|
||||
export const DM_GROUP_ACCESS_REASON = {
|
||||
GROUP_POLICY_ALLOWED: "group_policy_allowed",
|
||||
GROUP_POLICY_DISABLED: "group_policy_disabled",
|
||||
GROUP_POLICY_EMPTY_ALLOWLIST: "group_policy_empty_allowlist",
|
||||
GROUP_POLICY_NOT_ALLOWLISTED: "group_policy_not_allowlisted",
|
||||
DM_POLICY_OPEN: "dm_policy_open",
|
||||
DM_POLICY_DISABLED: "dm_policy_disabled",
|
||||
DM_POLICY_ALLOWLISTED: "dm_policy_allowlisted",
|
||||
DM_POLICY_PAIRING_REQUIRED: "dm_policy_pairing_required",
|
||||
DM_POLICY_NOT_ALLOWLISTED: "dm_policy_not_allowlisted",
|
||||
} as const;
|
||||
```
|
||||
|
||||
### Resolution Logic
|
||||
|
||||
The `resolveDmGroupAccessDecision()` function implements the policy evaluation:
|
||||
|
||||
```typescript
|
||||
export function resolveDmGroupAccessDecision(params: {
|
||||
isGroup: boolean;
|
||||
dmPolicy?: string | null;
|
||||
groupPolicy?: string | null;
|
||||
effectiveAllowFrom: Array<string | number>;
|
||||
effectiveGroupAllowFrom: Array<string | number>;
|
||||
isSenderAllowed: (allowFrom: string[]) => boolean;
|
||||
}) {
|
||||
const dmPolicy = params.dmPolicy ?? "pairing";
|
||||
const groupPolicy = params.groupPolicy ?? "allowlist";
|
||||
|
||||
if (params.isGroup) {
|
||||
// Group logic: evaluate against groupPolicy (open/disabled/allowlist)
|
||||
// Returns 'allow' if policy is open or user is in allowlist
|
||||
// Returns 'block' if policy is disabled or user not in allowlist
|
||||
} else {
|
||||
// DM logic: evaluate against dmPolicy
|
||||
if (dmPolicy === "disabled") return { decision: "block", ... };
|
||||
if (dmPolicy === "open") return { decision: "allow", ... };
|
||||
|
||||
if (params.isSenderAllowed(effectiveAllowFrom)) {
|
||||
return { decision: "allow", reasonCode: "dm_policy_allowlisted" };
|
||||
}
|
||||
|
||||
if (dmPolicy === "pairing") {
|
||||
return { decision: "pairing", reasonCode: "dm_policy_pairing_required" };
|
||||
}
|
||||
|
||||
return { decision: "block", reasonCode: "dm_policy_not_allowlisted" };
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## DM Pairing
|
||||
|
||||
The Pairing system allows users to prove their identity when `dmPolicy` is set to `pairing`.
|
||||
|
||||
1. **Initiation**: When an unknown user messages the bot, the system returns a `pairing` decision.
|
||||
2. **Challenge**: The bot provides a 6-digit pairing code (often via console or a side-channel).
|
||||
3. **Verification**: Once the user provides the correct code, their unique account ID is added to the **Store-backed Allowlist**.
|
||||
4. **Persistence**: Subsequent messages from this account ID are automatically allowed as they now appear in the resolved allowlist.
|
||||
|
||||
***
|
||||
|
||||
## Access Policies
|
||||
|
||||
Policies are configured per-channel and define the default security posture.
|
||||
|
||||
### dmPolicy Options
|
||||
- `disabled`: The bot will not respond to any DMs.
|
||||
- `open`: Anyone can message the bot (high risk).
|
||||
- `pairing`: Users must undergo the 6-digit pairing flow to gain access.
|
||||
- `allowlist`: Only users explicitly listed in the configuration file can message the bot.
|
||||
|
||||
### groupPolicy Options
|
||||
- `open`: Any user in the group can trigger the bot.
|
||||
- `disabled`: The bot is effectively silent in group settings.
|
||||
- `allowlist`: Only specific users within the group can trigger commands.
|
||||
|
||||
***
|
||||
|
||||
## Allowlist Management
|
||||
|
||||
OpenClaw uses a hybrid allowlist system:
|
||||
- **Config Allowlist**: Static entries defined in the `config.yaml` or environment variables.
|
||||
- **Store Allowlist**: Dynamic entries stored in a local database (e.g., `pairing-store.js`), primarily populated by successful pairings.
|
||||
|
||||
The `resolveEffectiveAllowFromLists()` function merges these sources, ensuring that per-channel scoping is respected. Wildcards (`*`) can be used in the config to grant broad access, though this is discouraged for production environments.
|
||||
|
||||
***
|
||||
|
||||
## Per-Session Sandboxing
|
||||
|
||||
To prevent tool execution from compromising the host, OpenClaw supports isolated backends:
|
||||
- **Docker Backend**: Each session spawns a transient container. Tools execute inside this container with limited CPU, memory, and no access to the host filesystem.
|
||||
- **SSH Backend**: Tools are executed on a remote machine or VM, isolating the main bot process from the execution environment.
|
||||
|
||||
***
|
||||
|
||||
## Tool Allow/Deny Lists
|
||||
|
||||
Security is further tightened by restricting which tools are available to which users or channels.
|
||||
- **Global Denylist**: Prevents high-risk tools (like `shell_execute`) from being loaded.
|
||||
- **Per-User Permissions**: Can restrict specific tools to "Admin" users only.
|
||||
|
||||
***
|
||||
|
||||
## SSRF Guards
|
||||
|
||||
The `web_fetch` tool and other network-touching skills include SSRF (Server-Side Request Forgery) protection. This includes:
|
||||
- **IP Blocklists**: Preventing requests to `localhost`, `127.0.0.1`, and internal private IP ranges (10.0.0.0/8, etc.).
|
||||
- **Protocol Restriction**: Only allowing `http` and `https`.
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
Luna currently lacks a security layer, which is a critical blocker for exposing tool execution to external interfaces.
|
||||
|
||||
### Priority Patterns to Adopt:
|
||||
1. **DM Pairing**: Implement the 6-digit pairing flow before allowing any tool interaction in DMs.
|
||||
2. **Access Decision Logic**: Port the `resolveDmGroupAccessDecision` pattern to [[Channels]] to centralize authorization.
|
||||
3. **SSRF Guards**: Ensure any "web search" or "fetch" skills implemented in [[Skills]] cannot hit internal Luna metadata services.
|
||||
4. **Docker Isolation**: Before enabling `shell_execute`, Luna must implement the Docker-based sandboxing seen in OpenClaw.
|
||||
|
||||
***
|
||||
|
||||
Cross-refs: [[Core]], [[Skills]], [[Channels]]
|
||||
@@ -0,0 +1,144 @@
|
||||
# Session Management
|
||||
|
||||
OpenClaw employs a centralized, file-based session management system that handles persistence, lifecycle events, and automated store maintenance. Unlike simple memory-only stores, this system is designed for high-concurrency environments with atomic write guarantees and strict disk budget enforcement.
|
||||
|
||||
***
|
||||
|
||||
## SessionEntry type
|
||||
|
||||
The `SessionEntry` structure in `src/config/sessions/types.ts` is the core data model, containing nearly 80 fields that track everything from model overrides to granular token usage and skill snapshots.
|
||||
|
||||
```typescript
|
||||
export type SessionEntry = {
|
||||
sessionId: string;
|
||||
updatedAt: number;
|
||||
sessionFile?: string;
|
||||
|
||||
// Model and Provider Overrides
|
||||
modelProvider?: string;
|
||||
model?: string;
|
||||
providerOverride?: string;
|
||||
modelOverride?: string;
|
||||
thinkingLevel?: string;
|
||||
|
||||
// Token Tracking and Context
|
||||
inputTokens?: number;
|
||||
outputTokens?: number;
|
||||
totalTokens?: number;
|
||||
totalTokensFresh?: boolean;
|
||||
compactionCount?: number;
|
||||
|
||||
// Delivery and Origin
|
||||
deliveryContext?: DeliveryContext;
|
||||
lastChannel?: SessionChannelId;
|
||||
origin?: SessionOrigin;
|
||||
|
||||
// Meta and Snapshots
|
||||
skillsSnapshot?: SessionSkillSnapshot;
|
||||
systemPromptReport?: SessionSystemPromptReport;
|
||||
acp?: SessionAcpMeta; // Agent Control Plane metadata
|
||||
queueMode?: "steer" | "followup" | "collect" | "queue" | "interrupt";
|
||||
};
|
||||
```
|
||||
|
||||
Key features of the type system:
|
||||
* **Model Overrides**: Allows per-session binding to specific models or providers, bypassing global defaults.
|
||||
* **Token Intelligence**: Tracks `totalTokensFresh` to determine if usage displays need a refresh.
|
||||
* **ACP Meta**: Preserves state for the Agent Control Plane, ensuring persistent agent behaviors across turns.
|
||||
* **Skill Snapshots**: Captures the state of available [[Skills]] at the time of the session turn.
|
||||
|
||||
***
|
||||
|
||||
## Store Maintenance
|
||||
|
||||
Automated maintenance is handled in `src/config/sessions/store-maintenance.ts` to prevent unbounded growth of the session file and ensure performance.
|
||||
|
||||
### Pruning and Capping
|
||||
* **`pruneStaleEntries()`**: Removes sessions older than a configured threshold (default: 30 days). It iterates through the store and deletes entries where `updatedAt` is before the cutoff.
|
||||
* **`capEntryCount()`**: Enforces a maximum number of sessions (default: 500). It sorts sessions by `updatedAt` descending and keeps only the most recent $N$ entries.
|
||||
|
||||
### File Rotation
|
||||
The `rotateSessionFile()` function monitors the `sessions.json` file size. If it exceeds the limit (default: 10MB), the system:
|
||||
1. Renames the current file to `sessions.json.bak.{timestamp}`.
|
||||
2. Maintains a maximum of 3 rotation backups, unlinking older ones.
|
||||
|
||||
### Maintenance Configuration
|
||||
The `resolveMaintenanceConfig()` function aggregates parameters from [[Configuration]]:
|
||||
* `pruneAfterMs`: Max age before eviction.
|
||||
* `maxEntries`: Hard limit on session count.
|
||||
* `rotateBytes`: File size trigger for rotation.
|
||||
* `maxDiskBytes`: Total disk budget for session transcripts.
|
||||
|
||||
***
|
||||
|
||||
## Session Store
|
||||
|
||||
The `SessionStore` in `src/config/sessions/store.ts` manages the I/O layer with a focus on reliability and cross-platform compatibility.
|
||||
|
||||
### Key Features
|
||||
* **Atomic Writes**: Uses `writeTextAtomic` to prevent file corruption during crashes.
|
||||
* **Write Locks**: Implements a `withSessionStoreLock` mechanism using a file-based lock to prevent race conditions between concurrent agent turns.
|
||||
* **Normalization**: Session keys are case-insensitive and trimmed via `normalizeStoreSessionKey()`. Legacy keys are automatically migrated to the normalized format on load.
|
||||
* **Windows Retry Semantics**: Includes specialized retry logic for Windows to handle transient file locks during the "truncate and write" phase.
|
||||
* **ACP Metadata Preservation**: Specifically protects `acp` metadata during updates, ensuring agent state isn't lost if a partial patch is applied.
|
||||
|
||||
```typescript
|
||||
export async function updateSessionStore<T>(
|
||||
storePath: string,
|
||||
mutator: (store: Record<string, SessionEntry>) => Promise<T> | T,
|
||||
opts?: SaveSessionStoreOptions,
|
||||
): Promise<T> {
|
||||
return await withSessionStoreLock(storePath, async () => {
|
||||
const store = loadSessionStore(storePath, { skipCache: true });
|
||||
const previousAcpByKey = collectAcpMetadataSnapshot(store);
|
||||
const result = await mutator(store);
|
||||
preserveExistingAcpMetadata({
|
||||
previousAcpByKey,
|
||||
nextStore: store,
|
||||
});
|
||||
await saveSessionStoreUnlocked(storePath, store, opts);
|
||||
return result;
|
||||
});
|
||||
}
|
||||
```
|
||||
|
||||
***
|
||||
|
||||
## Session Lifecycle Events
|
||||
|
||||
A lightweight pub/sub system in `src/sessions/session-lifecycle-events.ts` allows other subsystems to react to session changes.
|
||||
|
||||
```typescript
|
||||
export type SessionLifecycleEvent = {
|
||||
sessionKey: string;
|
||||
reason: string;
|
||||
parentSessionKey?: string;
|
||||
label?: string;
|
||||
};
|
||||
|
||||
export function onSessionLifecycleEvent(listener: SessionLifecycleListener): () => void {
|
||||
SESSION_LIFECYCLE_LISTENERS.add(listener);
|
||||
return () => { SESSION_LIFECYCLE_LISTENERS.delete(listener); };
|
||||
}
|
||||
|
||||
export function emitSessionLifecycleEvent(event: SessionLifecycleEvent): void {
|
||||
for (const listener of SESSION_LIFECYCLE_LISTENERS) {
|
||||
listener(event);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
This system is used to trigger [[Memory]] indexing, update [[Channels]] status, or log audit trails when sessions are created or deleted.
|
||||
|
||||
***
|
||||
|
||||
## Relevance to Luna
|
||||
|
||||
Luna's current `SessionManager` is a basic dictionary-backed store that lacks persistence and maintenance. To reach parity with OpenClaw, Luna should adopt several patterns:
|
||||
|
||||
1. **Atomic Persistence**: Implement a background saver for [[Core]] sessions that uses atomic file swaps to prevent data loss.
|
||||
2. **Maintenance Tasks**: Add a background service to prune old sessions and cap the total count to prevent memory leaks in the .NET runtime.
|
||||
3. **Locking**: Use a `SemaphoreSlim` or file-system lock for session updates to support concurrent requests from different [[Channels]].
|
||||
4. **Detailed Metadata**: Expand Luna's session objects to include token tracking and model overrides, allowing for better cost management and user customization.
|
||||
|
||||
Refer to [[Core]] for the current implementation status of Luna's session handling.
|
||||
Reference in New Issue
Block a user