Skip to main content

Overview

LLM APIs fail constantly — rate limits (429), server errors (5xx), auth expiry, and network timeouts. The FallbackProvider wraps multiple model providers with per-provider circuit breakers, automatically cascading to backup providers when failures occur.

Quick Start

Circuit Breaker

Each provider in the fallback chain has its own circuit breaker with three states:

Configuration

Error Classification

Default classification:
  • 429, 5xx, network errors"retry"
  • 401, 403, 404"cascade"
  • Content policy violations"fatal"

FallbackProvider

The FallbackProvider implements ModelProvider, making it transparent to the rest of the framework:

Events

Best Practices

  1. Order providers by preference — cheapest/fastest first, most reliable last
  2. Mix providers — don’t put all eggs in one basket (OpenAI + Anthropic + Google)
  3. Include a cheap fallbackgpt-4o-mini as the last resort keeps things running
  4. Monitor circuit states — use onFallback callback or events to alert on degradation