Luna/Documentation/Luna AI Assistant/Context Compaction Research.md

# Context Compaction Research

This document captures a research analysis comparing Claude Code's compaction engine patterns against Luna's current implementation, with recommendations for adoption. Reference source: https://barazany.dev/blog/claude-codes-compaction-engine

## Luna's Current Compaction Architecture

| Component | Current Approach |
|---|---|
| Trigger | Token threshold: `session.TokenAmount > MaxContextTokens * 0.75` |
| Token estimation | Naive `text.Length / 4` heuristic (`TokenEstimator`) |
| Compaction method | Single LLM call via `LibrarianAgent` (Mistral Small) — plain bullet-point summary |
| Retained context | Last N messages (default: 2) carried forward |
| Post-compaction reconstruction | Summary wrapped in `MEMORY BEGIN/END` markers as an Assistant message, then retained messages appended |
| Persistent memory | File-based `MemoryStore` — raw conversation logs to disk, last 10 read back as system message |
| Tiers | None — only full LLM summarization |
| Cache awareness | None |
| Tool result management | None — tool outputs accumulate until full compaction |

## Claude Code's Three-Tier Pattern

Claude Code employs a tiered strategy to manage context while maximizing cache efficiency:

- **Tier 1**: Lightweight deterministic cleanup before every API call. This process clears old tool results (retaining only the last 5) and replaces them with placeholders. No LLM is involved at this stage.
- **Tier 2**: API-level server-side strategies for token management using Anthropic-specific infrastructure.
- **Tier 3**: Full LLM summarization as a last resort. This involves a structured 9-section summary with a chain-of-thought scratchpad. Post-compaction reconstruction includes a boundary marker, the summary, the 5 most recently read files (capped at 50K tokens), re-injected skills, tool definitions, and session hooks.

The key architectural insight is that cache economics drive every decision. This includes using `cache_edits` for surgical server-side deletions and ensuring the summarization call reuses the same cache key.

## Pattern Evaluation

### 1. Tier 1 — Deterministic Tool Result Cleanup
**Gap**: Luna currently has zero tool result management.
**Recommendation**: ADOPT.
**Effort**: Low.
**Details**: Implement a pre-call sanitizer that trims old tool results before each API call to prevent context bloating from large tool outputs.

### 2. Structured Compaction Prompt
**Gap**: Luna uses a simple "max 12 bullet points" prompt.
**Recommendation**: ADOPT.
**Effort**: Low.
**Details**: Replace the current prompt with a structured template that forces categorized output, including user intent, key decisions, unresolved tasks, relevant facts, and technical context.

### 3. Tiered Compaction (Delay LLM Summarization)
**Gap**: Luna only supports full LLM summarization.
**Recommendation**: ADOPT.
**Effort**: Moderate.
**Details**: Implement a 2-tier system. Tier 1 performs deterministic cleanup on every call, while Tier 2 triggers LLM summarization only when Tier 1 is insufficient. The Anthropic-specific server-side tier will be skipped.

### 4. Post-Compaction Reconstruction
**Gap**: Luna's reconstruction logic is basic and uses Assistant messages for summaries.
**Recommendation**: PARTIALLY ADOPT.
**Effort**: Moderate.
**Details**: Place the summary as a System message instead of an Assistant message. Re-inject agent instructions and add a boundary marker with metadata (timestamp, pre-compaction message count). Include a continuation message so the agent does not treat the summary as something to respond to.

### 5. Improved Token Estimation
**Gap**: Luna relies on a `text.Length / 4` heuristic.
**Recommendation**: ADOPT.
**Effort**: Low.
**Details**: Replace the current heuristic with a proper tokenizer, such as `Microsoft.ML.Tokenizers`, or a significantly improved heuristic.

### 6. Autonomous Continuation After Compaction
**Gap**: Compaction can disrupt the conversation flow.
**Recommendation**: ADOPT LIGHTLY.
**Effort**: Trivial.
**Details**: Prepend a brief system message after compaction, such as "Context was compacted. Continue naturally."

### 7. Cache-Aware `cache_edits`
**Gap**: This is specific to Anthropic's API.
**Recommendation**: NOT APPLICABLE.
**Effort**: N/A.
**Details**: This could be revisited if an Anthropic provider is added to `IProvider` in the future.

### 8. Same-Cache-Key Summarization
**Gap**: Luna uses a cheaper model (Mistral Small) for compaction.
**Recommendation**: NOT APPLICABLE.
**Effort**: N/A.
**Details**: Luna's current approach is effective when prompt caching is not a primary factor.

## Priority Adoption Matrix

| Priority | Pattern | Effort | Impact |
|---|---|---|---|
| P0 | Tier 1 — Deterministic tool result cleanup | Low | High |
| P0 | Structured compaction prompt | Low | High |
| P1 | Tiered compaction (delay LLM summarization) | Moderate | High |
| P1 | Post-compaction reconstruction improvements | Moderate | Medium |
| P1 | Improved token estimation | Low | Medium |
| P2 | Continuation message after compaction | Trivial | Low-Medium |
| N/A | Cache-aware `cache_edits` | — | Not applicable (Mistral) |
| N/A | Same-cache-key summarization | — | Not applicable |

## Known Issues Found During Analysis

There is a bug in `SessionManager.SaveSessionLogAsync()` at line 97:

```csharp
.Where(m => m.Role != ChatRole.System || m.Role != ChatRole.Tool)
```

This condition is always true due to De Morgan's law. It should use `&&` instead of `||` to correctly filter out system and tool messages.

## Cross-References

- [[Core]] — SessionManager, compaction flow, token estimation
- [[Memory]] — MemoryStore, persistent conversation logs
- [[Configuration]] — SessionOptions (ContextTokenThreshold, RetainedMessagesAfterCompacting)