ai-harness/docs/10-milestones.md

10 — Milestones, Testing, Risks

Milestones

Each milestone ends with a verifiable outcome (✅).

M0 — Scaffolding

Workspace + all crates compile empty; CI (fmt, clippy -D warnings, test). harness-core: id types, event bus, permission evaluate with table-driven tests, storage actor with roundtrip tests. ✅ cargo test green; a bin stub prints version.

M1 — Headless core loop + Anthropic

Anthropic codec + provider (against fixtures), config loading, Tool trait + read/bash/glob/grep/edit/write (edit replacers ported with opencode's test table), engine loop end-to-end with a MockProvider, permission service with an auto-approve stub frontend. Debug command harness run -p "<prompt>". ✅ Integration test: scripted MockProvider does text → tool call (read) → final text with correct parts persisted; harness run works against real Anthropic.

M2 — TUI

Event loop, chat viewport + markdown renderer + cache, input, streaming render, permission modal wired to the real oneshot path, session picker/resume, abort. ✅ Manually drive a multi-turn session with permission-gated edits; TestBackend snapshot tests pass.

M3 — All providers + auth

OpenAI (responses + chat codecs), Copilot (device-flow modal, token exchange with direct-Bearer fallback, /models routing to the three codecs), auth.json + token refresh, models.dev metadata + cost display, retry policy + doom-loop guard. ✅ The same session works on all three providers; device-flow login completes inside the TUI; costs shown.

M4 — Multiagent

Agent registry (bundled markdown + overrides), task tool foreground/background, JobBoard + persistence + injection, aliases/reuse, depth limits, permission intersection, jobs pane + subtask drill-in. ✅ An orchestrator session spawns a background explorer; the board appears in next-turn context (verify via request logging); reuse by alias continues the same child session.

M5 — MCP, LSP, commands/skills

rmcp integration + namespaced tools + permissions; LSP pool + diagnostics in edit/write output; markdown commands + skill tool. ✅ A config-declared MCP server's tool is callable with ask-permission; editing a broken .rs file surfaces rust-analyzer errors in the tool result; /mycommand works.

M6 — Polish

Auto-compaction (Compactor impl + overflow trigger), session title generation (small_model), tool-output spill dir, theming, syntax-highlighted code blocks, crash-resume hardening, streaming-render perf pass. ✅ A session driven past the context limit compacts and continues correctly.

Testing strategy

• Provider decoders: recorded SSE fixtures per codec (text-only, tool call, parallel tool calls, thinking, errors, mid-stream disconnect, data: split across chunks) → assert exact Vec<LlmEvent> via insta snapshots. Request builders snapshot outgoing JSON (cache_control placement, tool schemas, header sets) with wiremock.
• Auth: device-flow poll loop against wiremock (authorization_pending → slow_down → success); token-exchange expiry + refresh single-flight.
• Tools: tempdir-based unit tests; edit replacers get a direct port of opencode's edit.test.ts case table; bash timeout/kill-process-group; truncation boundaries.
• Engine integration (MockProvider): multi-step tool loop; permission ask/deny/reject via the real oneshot path; retry backoff (mock returns RateLimited twice); doom-loop trigger; cancellation mid-tool; task tool foreground + background with job-board assertions; depth-limit rejection; permission intersection.
• Permission engine: table-driven last-match-wins/wildcard tests mirroring opencode's semantics.
• TUI: ratatui TestBackend snapshots (empty session, streaming message, tool card, permission modal, jobs pane); input-handling unit tests on AppState.
• Storage: actor roundtrips; resume-after-simulated-crash (unfinished assistant → aborted).

Top risks

1. Copilot token exchange variance — opencode ships direct-Bearer; classic Copilot API wants the copilot_internal/v2/token exchange. Mitigation: dual-path token.rs, verify live early in M3.
2. SSE edge cases — events split across TCP chunks, keepalive comments, provider-specific mid-stream error events. Mitigation: eventsource-stream + adversarial fixtures.
3. Streaming markdown perf in ratatui — no incremental layout. Mitigation: per-part render cache + tail-only re-render + plain-text-tail fallback.
4. Edit replacer fidelity — port the test suite, not just the code.
5. Child-process lifecycle (MCP/LSP/bash) — zombies. Mitigation: process-group kills + Drop guards + central ChildRegistry.

Key reference files

What	Where
Inner stream state machine to mirror	~/repos/opencode/packages/opencode/src/session/processor.ts
Outer loop semantics	~/repos/opencode/packages/opencode/src/session/prompt.ts
Permission evaluate/ask semantics	~/repos/opencode/packages/opencode/src/permission/index.ts
Edit replacer chain + tests (port verbatim)	~/repos/opencode/packages/opencode/src/tool/edit.ts
Copilot auth/headers/model routing	~/repos/opencode/packages/opencode/src/plugin/github-copilot/{copilot,models}.ts
Anthropic caching/beta headers	~/repos/opencode/packages/opencode/src/provider/transform.ts
Retry policy details	~/repos/opencode/packages/opencode/src/session/retry.ts
JobRecord, aliases, formatForPrompt	~/repos/oh-my-opencode-slim/src/utils/background-job-board.ts
Session-reuse mechanics	~/repos/oh-my-opencode-slim/src/hooks/task-session-manager/index.ts
Agent prompt content to port into markdown	~/repos/oh-my-opencode-slim/src/agents/*.ts