M0: scaffold Cargo workspace, core types, event bus, permission engine, storage actor
Seven-crate workspace per docs/01-architecture.md. harness-core gets the domain types (Session/Message/Part/ToolState), a broadcast EventBus, the last-match-wins wildcard permission evaluator, and a SQLite storage actor (dedicated thread + mpsc, JSON-blob rows) with roundtrip tests. All other crates are compiling stubs. CI runs fmt/clippy -D warnings/test.
This commit is contained in:
Erik Simon
@@ -0,0 +1,66 @@
|
||||
# 10 — Milestones, Testing, Risks
|
||||
|
||||
## Milestones
|
||||
|
||||
Each milestone ends with a verifiable outcome (✅).
|
||||
|
||||
### M0 — Scaffolding
|
||||
Workspace + all crates compile empty; CI (fmt, clippy `-D warnings`, test). `harness-core`: id types, event bus, permission `evaluate` with table-driven tests, storage actor with roundtrip tests.
|
||||
✅ `cargo test` green; a bin stub prints version.
|
||||
|
||||
### M1 — Headless core loop + Anthropic
|
||||
Anthropic codec + provider (against fixtures), config loading, `Tool` trait + read/bash/glob/grep/edit/write (edit replacers ported **with opencode's test table**), engine loop end-to-end with a `MockProvider`, permission service with an auto-approve stub frontend. Debug command `harness run -p "<prompt>"`.
|
||||
✅ Integration test: scripted MockProvider does text → tool call (read) → final text with correct parts persisted; `harness run` works against real Anthropic.
|
||||
|
||||
### M2 — TUI
|
||||
Event loop, chat viewport + markdown renderer + cache, input, streaming render, permission modal wired to the real oneshot path, session picker/resume, abort.
|
||||
✅ Manually drive a multi-turn session with permission-gated edits; TestBackend snapshot tests pass.
|
||||
|
||||
### M3 — All providers + auth
|
||||
OpenAI (responses + chat codecs), Copilot (device-flow modal, token exchange with direct-Bearer fallback, `/models` routing to the three codecs), auth.json + token refresh, models.dev metadata + cost display, retry policy + doom-loop guard.
|
||||
✅ The same session works on all three providers; device-flow login completes inside the TUI; costs shown.
|
||||
|
||||
### M4 — Multiagent
|
||||
Agent registry (bundled markdown + overrides), task tool foreground/background, JobBoard + persistence + injection, aliases/reuse, depth limits, permission intersection, jobs pane + subtask drill-in.
|
||||
✅ An orchestrator session spawns a background explorer; the board appears in next-turn context (verify via request logging); reuse by alias continues the same child session.
|
||||
|
||||
### M5 — MCP, LSP, commands/skills
|
||||
rmcp integration + namespaced tools + permissions; LSP pool + diagnostics in edit/write output; markdown commands + skill tool.
|
||||
✅ A config-declared MCP server's tool is callable with ask-permission; editing a broken `.rs` file surfaces rust-analyzer errors in the tool result; `/mycommand` works.
|
||||
|
||||
### M6 — Polish
|
||||
Auto-compaction (Compactor impl + overflow trigger), session title generation (small_model), tool-output spill dir, theming, syntax-highlighted code blocks, crash-resume hardening, streaming-render perf pass.
|
||||
✅ A session driven past the context limit compacts and continues correctly.
|
||||
|
||||
## Testing strategy
|
||||
|
||||
- **Provider decoders:** recorded SSE fixtures per codec (text-only, tool call, parallel tool calls, thinking, errors, mid-stream disconnect, `data:` split across chunks) → assert exact `Vec<LlmEvent>` via `insta` snapshots. Request builders snapshot outgoing JSON (cache_control placement, tool schemas, header sets) with `wiremock`.
|
||||
- **Auth:** device-flow poll loop against wiremock (`authorization_pending → slow_down → success`); token-exchange expiry + refresh single-flight.
|
||||
- **Tools:** tempdir-based unit tests; edit replacers get a direct port of opencode's `edit.test.ts` case table; bash timeout/kill-process-group; truncation boundaries.
|
||||
- **Engine integration (MockProvider):** multi-step tool loop; permission ask/deny/reject via the real oneshot path; retry backoff (mock returns RateLimited twice); doom-loop trigger; cancellation mid-tool; task tool foreground + background with job-board assertions; depth-limit rejection; permission intersection.
|
||||
- **Permission engine:** table-driven last-match-wins/wildcard tests mirroring opencode's semantics.
|
||||
- **TUI:** ratatui `TestBackend` snapshots (empty session, streaming message, tool card, permission modal, jobs pane); input-handling unit tests on `AppState`.
|
||||
- **Storage:** actor roundtrips; resume-after-simulated-crash (unfinished assistant → aborted).
|
||||
|
||||
## Top risks
|
||||
|
||||
1. **Copilot token exchange variance** — opencode ships direct-Bearer; classic Copilot API wants the `copilot_internal/v2/token` exchange. Mitigation: dual-path `token.rs`, verify live early in M3.
|
||||
2. **SSE edge cases** — events split across TCP chunks, keepalive comments, provider-specific mid-stream error events. Mitigation: `eventsource-stream` + adversarial fixtures.
|
||||
3. **Streaming markdown perf in ratatui** — no incremental layout. Mitigation: per-part render cache + tail-only re-render + plain-text-tail fallback.
|
||||
4. **Edit replacer fidelity** — port the test suite, not just the code.
|
||||
5. **Child-process lifecycle (MCP/LSP/bash)** — zombies. Mitigation: process-group kills + `Drop` guards + central `ChildRegistry`.
|
||||
|
||||
## Key reference files
|
||||
|
||||
| What | Where |
|
||||
|---|---|
|
||||
| Inner stream state machine to mirror | `~/repos/opencode/packages/opencode/src/session/processor.ts` |
|
||||
| Outer loop semantics | `~/repos/opencode/packages/opencode/src/session/prompt.ts` |
|
||||
| Permission evaluate/ask semantics | `~/repos/opencode/packages/opencode/src/permission/index.ts` |
|
||||
| Edit replacer chain + tests (port verbatim) | `~/repos/opencode/packages/opencode/src/tool/edit.ts` |
|
||||
| Copilot auth/headers/model routing | `~/repos/opencode/packages/opencode/src/plugin/github-copilot/{copilot,models}.ts` |
|
||||
| Anthropic caching/beta headers | `~/repos/opencode/packages/opencode/src/provider/transform.ts` |
|
||||
| Retry policy details | `~/repos/opencode/packages/opencode/src/session/retry.ts` |
|
||||
| JobRecord, aliases, formatForPrompt | `~/repos/oh-my-opencode-slim/src/utils/background-job-board.ts` |
|
||||
| Session-reuse mechanics | `~/repos/oh-my-opencode-slim/src/hooks/task-session-manager/index.ts` |
|
||||
| Agent prompt content to port into markdown | `~/repos/oh-my-opencode-slim/src/agents/*.ts` |
|
||||
Reference in New Issue
Block a user