Overview
LLM APIs fail constantly — rate limits (429), server errors (5xx), auth expiry, and network timeouts. The FallbackProvider wraps multiple model providers with per-provider circuit breakers, automatically cascading to backup providers when failures occur.Quick Start
Circuit Breaker
Each provider in the fallback chain has its own circuit breaker with three states:Configuration
Error Classification
Default classification:
- 429, 5xx, network errors →
"retry" - 401, 403, 404 →
"cascade" - Content policy violations →
"fatal"
FallbackProvider
TheFallbackProvider implements ModelProvider, making it transparent to the rest of the framework:
Events
Best Practices
- Order providers by preference — cheapest/fastest first, most reliable last
- Mix providers — don’t put all eggs in one basket (OpenAI + Anthropic + Google)
- Include a cheap fallback —
gpt-4o-minias the last resort keeps things running - Monitor circuit states — use
onFallbackcallback or events to alert on degradation