ai-harness/docs/03-providers.md

03 — Provider Layer

harness-providers: three providers sharing wire codecs, because Copilot multiplexes all three wire formats (chat completions, responses, anthropic messages).

src/
  registry.rs        # ProviderRegistry: id → Arc<dyn Provider>; model resolution "provider/model"
  auth.rs            # AuthStorage (auth.json, 0600)
  modelsdev.rs       # models.dev metadata cache
  codec/
    openai_chat.rs        # request builder + SSE decoder for /chat/completions
    openai_responses.rs   # request builder + SSE decoder for /responses
    anthropic.rs          # request builder + SSE decoder for /v1/messages
  openai.rs
  anthropic.rs
  copilot/
    device_flow.rs
    token.rs
    provider.rs

Provider trait & request model

#[async_trait]
pub trait Provider: Send + Sync {
    fn id(&self) -> &str;
    async fn list_models(&self) -> Result<Vec<ModelInfo>, ProviderError>;
    async fn stream(&self, req: LlmRequest, cancel: CancellationToken)
        -> Result<LlmEventStream, ProviderError>;
}

pub struct LlmRequest {
    pub model: String,
    pub system: Vec<String>,               // ordered blocks (cache breakpoints need structure)
    pub messages: Vec<WireMessage>,        // role + Vec<WireContent{Text, ToolCall, ToolResult, Image}>
    pub tools: Vec<ToolSchema>,            // { name, description, parameters: Value }
    pub temperature: Option<f32>, pub max_tokens: Option<u32>,
    pub reasoning: Option<ReasoningOpts>,  // effort low/medium/high; anthropic thinking budget
    pub initiator: Initiator,              // User | Agent → copilot x-initiator header
}

Decoders are pure functions fn decode(sse: impl Stream<Item = Event>) -> LlmEventStream written with async_stream::try_stream! over eventsource_stream::Eventsource — unit-testable against fixture files.

Anthropic

• Auth: x-api-key header (API key from config/auth.json/ANTHROPIC_API_KEY).
• Headers: anthropic-beta: interleaved-thinking-2025-05-14,fine-grained-tool-streaming-2025-05-14 (from opencode).
• Prompt caching: cache_control: {type: "ephemeral"} breakpoints on the first 2 system blocks + last 2 non-system messages (opencode's applyCaching in provider/transform.ts).
• SSE mapping: content_block_start(thinking) → ReasoningStart; thinking_delta/signature_delta deltas; content_block_start(tool_use) → ToolInputStart; input_json_delta → ToolInputDelta (accumulate, emit ToolCall at content_block_stop); message_delta.usage → Finish. Cache tokens from cache_creation_input_tokens / cache_read_input_tokens.

OpenAI

• Auth: Authorization: Bearer <api key>.
• Responses API for gpt-* / o-* models; chat completions as fallback for compatible endpoints.
• Responses SSE: response.output_item.added/delta/done events; response.completed for usage. Chat SSE: choices[].delta.tool_calls[] accumulated by index; [DONE] sentinel.
• Reasoning models: reasoning: { effort }, reasoning_summary: auto; reasoning tokens counted separately in usage.

GitHub Copilot

Auth — GitHub OAuth device flow (copilot/device_flow.rs)

1. POST https://github.com/login/device/code body {client_id, scope: "read:user"}, Accept: application/json → {verification_uri, user_code, device_code, interval}. Emit AppEvent::AuthPrompt → TUI modal shows the code + URL.
2. Poll POST https://github.com/login/oauth/access_token body {client_id, device_code, grant_type: "urn:ietf:params:oauth:grant-type:device_code"} at interval; handle authorization_pending and slow_down (+5 s per RFC 8628).
3. Store the OAuth token in auth.json.

Client ID: Ov23li8tweQw6odWQebz (opencode's; confirm or register our own).

Token strategy (copilot/token.rs) — flagged risk

opencode currently sends the GitHub OAuth token directly as the Bearer for api.githubcopilot.com (no exchange, expires: 0). The classic Copilot API instead requires exchanging it at GET https://api.github.com/copilot_internal/v2/token for a short-lived {token, expires_at}. Implement both behind one function: exchange as primary (cached until expires_at − 120s), direct-Bearer fallback if the exchange endpoint rejects. Verify against the live API early in M3.

Requests (copilot/provider.rs)

• Base: https://api.githubcopilot.com (enterprise: https://copilot-api.<domain>).
• Headers: Authorization: Bearer, User-Agent: ai-harness/<version>, X-GitHub-Api-Version: 2026-06-01, Openai-Intent: conversation-edits, x-initiator: agent|user (subagent/utility sessions → agent), Copilot-Vision-Request: true when messages contain images. Anthropic-backed Copilot models also get anthropic-beta: interleaved-thinking-2025-05-14.
• GET /models: capabilities, billing, and supported_endpoints per model → route each model to the right codec: /chat/completions → openai_chat, /responses → openai_responses, /v1/messages → anthropic codec against <base>/v1. Only model_picker_enabled models surface in the picker.

Auth storage (auth.rs)

Single file ~/.local/share/ai-harness/auth.json, written mode 0600, keyed by provider id:

enum AuthRecord {
    OAuth { access: String, refresh: String, expires: i64 },   // expires 0 = never
    Api   { key: String },
}

Token refresh lives inside each provider's HTTP path: on 401 or known expiry, refresh under a tokio::Mutex (single-flight), persist, retry once.

Login is exposed as EngineHandle::login(provider_id) -> LoginFlow:

enum LoginFlow {
    DeviceCode { user_code: String, verification_uri: String, done: oneshot::Receiver<Result<()>> },
    ApiKey     { submit: /* fn(String) */ },
}

Model metadata & cost (modelsdev.rs)

• Fetch https://models.dev/api.json → cache ~/.cache/ai-harness/models.json (24 h TTL); fallback to baked assets/models-snapshot.json.
• Yields ModelInfo { context_limit, output_limit, cost { input, output, cache_read, cache_write } (per-million), reasoning, tool_call, attachment } keyed by (provider, model).
• At Finish: cost = Σ usage_component × rate; accumulate on message and session. Copilot models cost 0 unless metadata says otherwise.

Testing

• Recorded SSE fixtures per codec (tests/fixtures/{anthropic,openai_chat,openai_responses,copilot}/*.sse): text-only, tool call, parallel tool calls, thinking, mid-stream error, data: split across chunks → assert exact Vec<LlmEvent> via insta.
• Request builders: snapshot outgoing JSON (cache_control placement, tool schemas, header sets) via wiremock.
• Device flow: wiremock sequence authorization_pending → slow_down → success; refresh single-flight test.