darman
420f00494f
Sets up the Cargo workspace (harness-core, harness-tools, harness-providers, harness-mcp, harness-lsp, harness-app, harness-tui) and the ten architecture docs under docs/. harness-core gets its foundational types (session/message/part/model ids), an in-process event bus, a table-driven permission evaluate function, and a SQLite-backed storage actor with schema and roundtrip coverage. Every crate compiles empty and a bin stub prints its version.
113 lines
6.6 KiB
Markdown
113 lines
6.6 KiB
Markdown
# 03 — Provider Layer
|
||
|
||
`harness-providers`: three providers sharing wire codecs, because Copilot multiplexes all three wire formats (chat completions, responses, anthropic messages).
|
||
|
||
```
|
||
src/
|
||
registry.rs # ProviderRegistry: id → Arc<dyn Provider>; model resolution "provider/model"
|
||
auth.rs # AuthStorage (auth.json, 0600)
|
||
modelsdev.rs # models.dev metadata cache
|
||
codec/
|
||
openai_chat.rs # request builder + SSE decoder for /chat/completions
|
||
openai_responses.rs # request builder + SSE decoder for /responses
|
||
anthropic.rs # request builder + SSE decoder for /v1/messages
|
||
openai.rs
|
||
anthropic.rs
|
||
copilot/
|
||
device_flow.rs
|
||
token.rs
|
||
provider.rs
|
||
```
|
||
|
||
## Provider trait & request model
|
||
|
||
```rust
|
||
#[async_trait]
|
||
pub trait Provider: Send + Sync {
|
||
fn id(&self) -> &str;
|
||
async fn list_models(&self) -> Result<Vec<ModelInfo>, ProviderError>;
|
||
async fn stream(&self, req: LlmRequest, cancel: CancellationToken)
|
||
-> Result<LlmEventStream, ProviderError>;
|
||
}
|
||
|
||
pub struct LlmRequest {
|
||
pub model: String,
|
||
pub system: Vec<String>, // ordered blocks (cache breakpoints need structure)
|
||
pub messages: Vec<WireMessage>, // role + Vec<WireContent{Text, ToolCall, ToolResult, Image}>
|
||
pub tools: Vec<ToolSchema>, // { name, description, parameters: Value }
|
||
pub temperature: Option<f32>, pub max_tokens: Option<u32>,
|
||
pub reasoning: Option<ReasoningOpts>, // effort low/medium/high; anthropic thinking budget
|
||
pub initiator: Initiator, // User | Agent → copilot x-initiator header
|
||
}
|
||
```
|
||
|
||
Decoders are pure functions `fn decode(sse: impl Stream<Item = Event>) -> LlmEventStream` written with `async_stream::try_stream!` over `eventsource_stream::Eventsource` — unit-testable against fixture files.
|
||
|
||
## Anthropic
|
||
|
||
- Auth: `x-api-key` header (API key from config/auth.json/`ANTHROPIC_API_KEY`).
|
||
- Headers: `anthropic-beta: interleaved-thinking-2025-05-14,fine-grained-tool-streaming-2025-05-14` (from opencode).
|
||
- **Prompt caching:** `cache_control: {type: "ephemeral"}` breakpoints on the first 2 system blocks + last 2 non-system messages (opencode's `applyCaching` in `provider/transform.ts`).
|
||
- SSE mapping: `content_block_start(thinking)` → `ReasoningStart`; `thinking_delta`/`signature_delta` deltas; `content_block_start(tool_use)` → `ToolInputStart`; `input_json_delta` → `ToolInputDelta` (accumulate, emit `ToolCall` at `content_block_stop`); `message_delta.usage` → `Finish`. Cache tokens from `cache_creation_input_tokens` / `cache_read_input_tokens`.
|
||
|
||
## OpenAI
|
||
|
||
- Auth: `Authorization: Bearer <api key>`.
|
||
- **Responses API** for gpt-* / o-* models; chat completions as fallback for compatible endpoints.
|
||
- Responses SSE: `response.output_item.added/delta/done` events; `response.completed` for usage. Chat SSE: `choices[].delta.tool_calls[]` accumulated by index; `[DONE]` sentinel.
|
||
- Reasoning models: `reasoning: { effort }`, `reasoning_summary: auto`; reasoning tokens counted separately in usage.
|
||
|
||
## GitHub Copilot
|
||
|
||
### Auth — GitHub OAuth device flow (`copilot/device_flow.rs`)
|
||
|
||
1. `POST https://github.com/login/device/code` body `{client_id, scope: "read:user"}`, `Accept: application/json` → `{verification_uri, user_code, device_code, interval}`. Emit `AppEvent::AuthPrompt` → TUI modal shows the code + URL.
|
||
2. Poll `POST https://github.com/login/oauth/access_token` body `{client_id, device_code, grant_type: "urn:ietf:params:oauth:grant-type:device_code"}` at `interval`; handle `authorization_pending` and `slow_down` (+5 s per RFC 8628).
|
||
3. Store the OAuth token in auth.json.
|
||
|
||
Client ID: `Ov23li8tweQw6odWQebz` (opencode's; confirm or register our own).
|
||
|
||
### Token strategy (`copilot/token.rs`) — flagged risk
|
||
|
||
opencode currently sends the **GitHub OAuth token directly** as the Bearer for `api.githubcopilot.com` (no exchange, `expires: 0`). The classic Copilot API instead requires exchanging it at `GET https://api.github.com/copilot_internal/v2/token` for a short-lived `{token, expires_at}`. Implement **both behind one function**: exchange as primary (cached until `expires_at − 120s`), direct-Bearer fallback if the exchange endpoint rejects. Verify against the live API early in M3.
|
||
|
||
### Requests (`copilot/provider.rs`)
|
||
|
||
- Base: `https://api.githubcopilot.com` (enterprise: `https://copilot-api.<domain>`).
|
||
- Headers: `Authorization: Bearer`, `User-Agent: ai-harness/<version>`, `X-GitHub-Api-Version: 2026-06-01`, `Openai-Intent: conversation-edits`, `x-initiator: agent|user` (subagent/utility sessions → `agent`), `Copilot-Vision-Request: true` when messages contain images. Anthropic-backed Copilot models also get `anthropic-beta: interleaved-thinking-2025-05-14`.
|
||
- `GET /models`: capabilities, billing, and `supported_endpoints` per model → route each model to the right codec: `/chat/completions` → openai_chat, `/responses` → openai_responses, `/v1/messages` → anthropic codec against `<base>/v1`. Only `model_picker_enabled` models surface in the picker.
|
||
|
||
## Auth storage (`auth.rs`)
|
||
|
||
Single file `~/.local/share/ai-harness/auth.json`, written mode `0600`, keyed by provider id:
|
||
|
||
```rust
|
||
enum AuthRecord {
|
||
OAuth { access: String, refresh: String, expires: i64 }, // expires 0 = never
|
||
Api { key: String },
|
||
}
|
||
```
|
||
|
||
Token refresh lives inside each provider's HTTP path: on 401 or known expiry, refresh under a `tokio::Mutex` (single-flight), persist, retry once.
|
||
|
||
Login is exposed as `EngineHandle::login(provider_id) -> LoginFlow`:
|
||
|
||
```rust
|
||
enum LoginFlow {
|
||
DeviceCode { user_code: String, verification_uri: String, done: oneshot::Receiver<Result<()>> },
|
||
ApiKey { submit: /* fn(String) */ },
|
||
}
|
||
```
|
||
|
||
## Model metadata & cost (`modelsdev.rs`)
|
||
|
||
- Fetch `https://models.dev/api.json` → cache `~/.cache/ai-harness/models.json` (24 h TTL); fallback to baked `assets/models-snapshot.json`.
|
||
- Yields `ModelInfo { context_limit, output_limit, cost { input, output, cache_read, cache_write } (per-million), reasoning, tool_call, attachment }` keyed by `(provider, model)`.
|
||
- At `Finish`: `cost = Σ usage_component × rate`; accumulate on message and session. Copilot models cost 0 unless metadata says otherwise.
|
||
|
||
## Testing
|
||
|
||
- Recorded SSE fixtures per codec (`tests/fixtures/{anthropic,openai_chat,openai_responses,copilot}/*.sse`): text-only, tool call, parallel tool calls, thinking, mid-stream error, `data:` split across chunks → assert exact `Vec<LlmEvent>` via `insta`.
|
||
- Request builders: snapshot outgoing JSON (cache_control placement, tool schemas, header sets) via `wiremock`.
|
||
- Device flow: wiremock sequence `authorization_pending → slow_down → success`; refresh single-flight test.
|