Luna/Documentation/References/ZeroClaw/ReliableProvider.md

# ReliableProvider

The `ReliableProvider` is a resilient decorator in ZeroClaw that wraps one or more standard providers to implement a three-level failover strategy. It ensures high availability by managing model fallbacks, provider redundancy, and intelligent retry logic with backoff.

***

## Provider Trait

All LLM integrations in ZeroClaw implement the `Provider` trait. This trait defines a unified interface for chat interactions, tool calling, and capability discovery.

```rust
#[async_trait]
pub trait Provider: Send + Sync {
    fn capabilities(&self) -> ProviderCapabilities {
        ProviderCapabilities::default()
    }

    async fn chat_with_system(
        &self,
        system_prompt: Option<&str>,
        message: &str,
        model: &str,
        temperature: f64,
    ) -> anyhow::Result<String>;

    async fn chat_with_history(
        &self,
        messages: &[ChatMessage],
        model: &str,
        temperature: f64,
    ) -> anyhow::Result<String>;

    async fn chat(
        &self,
        request: ChatRequest<'_>,
        model: &str,
        temperature: f64,
    ) -> anyhow::Result<ChatResponse>;

    async fn chat_with_tools(
        &self,
        messages: &[ChatMessage],
        _tools: &[serde_json::Value],
        model: &str,
        temperature: f64,
    ) -> anyhow::Result<ChatResponse>;
}
```

### ProviderCapabilities
The `ProviderCapabilities` struct allows providers to declare their feature set, which `ReliableProvider` uses to route requests correctly:
- `native_tool_calling`: Support for API-native function calling (Gemini, Anthropic, OpenAI).
- `vision`: Support for multimodal image inputs.

### ToolsPayload
When tools are used, `Provider::convert_tools` returns a `ToolsPayload` enum to handle varying API requirements:
- `Gemini { function_declarations }`
- `Anthropic { tools }`
- `OpenAI { tools }`
- `PromptGuided { instructions }`: Textual fallback for providers without native support.

***

## Three-Level Failover

The `ReliableProvider` implements a nested loop structure to exhaust all possibilities before returning an error.

```rust
pub struct ReliableProvider {
    providers: Vec<(String, Box<dyn Provider>)>,
    max_retries: u32,
    base_backoff_ms: u64,
    api_keys: Vec<String>,
    key_index: AtomicUsize,
    model_fallbacks: HashMap<String, Vec<String>>,
}
```

1. **Level 1: Model Chain**: Iterates through the primary model and its configured fallbacks (e.g., try `claude-3-5-sonnet`, fallback to `claude-3-haiku`).
2. **Level 2: Provider Chain**: For each model, iterates through registered providers in priority order (e.g., try Anthropic direct, fallback to OpenRouter).
3. **Level 3: Retry Loop**: For a specific (provider, model) pair, retries transient failures with exponential backoff.

***

## Error Classification

Intelligent failure handling depends on distinguishing transient issues from permanent ones. ZeroClaw uses heuristics and status codes for this classification.

### Non-Retryable Errors
The `is_non_retryable()` function identifies errors that won't resolve with retries, such as:
- **Client Errors**: HTTP 4xx (except 429/408).
- **Authentication**: Key-word matching for "invalid api key", "unauthorized", or "permission denied".
- **Model Availability**: Heuristics for "model not found", "unknown model", or "unsupported".

### Rate Limiting and Quota
- `is_rate_limited()`: Specifically detects HTTP 429 errors.
- `is_non_retryable_rate_limit()`: Detects business-level failures returned as 429s, such as "insufficient balance", "quota exhausted", or "plan does not include requested model". These trigger a provider fallback immediately.

### Context Window Exceeded
The system short-circuits when the context window is exceeded to avoid useless retries:
```rust
fn is_context_window_exceeded(err: &anyhow::Error) -> bool {
    let hints = [
        "exceeds the context window",
        "maximum context length",
        "token limit exceeded",
        "prompt is too long",
    ];
    // ...
}
```

***

## Exponential Backoff

Retries use an exponential backoff strategy starting from `base_backoff_ms`.
- **Retry-After Parsing**: The system parses the `Retry-After` header or error body.
- **Capping**: Backoff is capped at 30 seconds to prevent indefinite stalls.
- **Jitter**: While the core logic doubles the wait time, it ensures the wait respects provider-suggested intervals.

***

## API Key Rotation

ZeroClaw supports round-robin API key rotation for the same provider to maximize throughput:
```rust
fn rotate_key(&self) -> Option<&str> {
    if self.api_keys.is_empty() { return None; }
    let idx = self.key_index.fetch_add(1, Ordering::Relaxed) % self.api_keys.len();
    Some(&self.api_keys[idx])
}
```
When a 429 is encountered (and it's not a quota error), the system cycles to the next key for the subsequent retry attempt.

***

## Relevance to Luna

Currently, Luna's [[Core]] implementation of `IProvider` is simple and lacks resilience. If a provider is down, the request fails.

By implementing a `ReliableProvider` decorator for Luna's `IProvider`, we can:
- Wrap multiple `IChatClient` instances.
- Implement per-sender model overrides to allow specific users to access higher-tier models with fallback to cheaper ones.
- Integrate the ZeroClaw error classification logic to handle common LLM API failures gracefully.

This pattern is essential for moving from a prototype to a production-grade system where provider stability cannot be guaranteed.

***

[[Core]]
[[Configuration]]