Handle model provider failures

Overview

FallbackProvider tries configured providers in order and tracks a circuit breaker for each. Use it when another provider can satisfy the same request after an eligible failure. Ensure every candidate supports the required tools, input media, and output format.

Quick Start

This host function takes configured models and returns an Agent. The caller supplies credentials, runs the Agent, and closes it afterward.

import { Agent, withFallback, type ModelProvider } from "@agentium/core";

export function createResilientAssistant(primary: ModelProvider, backup: ModelProvider) {
  return new Agent({
    name: "assistant", register: false,
    model: withFallback([primary, backup]),
    instructions: "Answer the user's request. Ask for missing information.",
  });
}

Choose the order deliberately. A backup can have different quality, cost, latency, or regional availability. Test those differences with representative evaluations.

Circuit Breaker

State	Behavior
`closed`	Requests can attempt the provider
`open`	The provider is skipped until its cooldown elapses
`half-open`	Attempts can probe recovery; enough successful probes close the circuit

Configuration

This example uses the default classifier and custom thresholds. The object is independent of any live provider.

import { CircuitBreaker } from "@agentium/core";

const breaker = new CircuitBreaker({
  failureThreshold: 3,
  cooldownMs: 60_000,
  halfOpenMaxAttempts: 2,
});
console.log(breaker.state); // closed

Defaults are five failures, a 30-second cooldown, and two half-open successes. halfOpenMaxAttempts controls successful recovery probes; it is not a concurrency semaphore. Use a separate host limiter to bound concurrent work.

Error Classification

Classification	Fallback behavior
`retry`	Record failure and try the next eligible provider
`cascade`	Record failure and move to the next eligible provider
`fatal`	Throw without continuing the chain

The default classifier recognizes common HTTP status codes, network codes, and error-message patterns: 429/5xx and network failures generally retry; 401/403/404 cascade; recognized content-policy errors are fatal. Unknown errors default to retry. Supply classifyError(error) when your provider’s error shape requires a different policy. “Retry” here selects another provider; it does not guarantee repeated attempts against the same provider or replay of the entire Agent run.

FallbackProvider

import { FallbackProvider, type ModelProvider } from "@agentium/core";

export function createFallback(primary: ModelProvider, backup: ModelProvider) {
  return new FallbackProvider({
    providers: [primary, backup],
    circuitBreaker: { failureThreshold: 3, cooldownMs: 60_000 },
    onFallback: (from, to, error) => console.warn("provider fallback", from, to, error),
  });
}

Callbacks

Fallback does not emit EventBus events. Use onFallback and getBreaker(providerKey)?.state. The provider key is ${provider.providerId}:${provider.modelId}; derive it from the configured instance so a model override does not leave the lookup stale.

Streaming boundary

A stream may switch providers only before any chunk has been emitted. After output begins, a provider failure propagates to the caller. Starting another provider mid-stream would mix partial answers and tool-call state. Handle the incomplete response at the transport boundary; do not display it as a completed result.

Best Practices

Verify every fallback model supports your schema, tools, and media.
Exercise a failing first provider and a successful backup with fixtures before live testing.
Track fallback frequency and answer quality separately; successful delivery alone does not prove equivalent behavior.
Keep external effects idempotent. A model fallback does not make retrying an entire application operation safe.

Continue with model patterns, operating limits, and recovery contracts.

Build

Agent applications

Harness runtime

Teams and workflows

Data and capabilities

Interfaces and jobs

Voice and telephony

Verify and manage

Handle model provider failures

Overview

Quick Start

Circuit Breaker

Configuration

Error Classification

FallbackProvider

Callbacks

Streaming boundary

Best Practices

​Overview

​Quick Start

​Circuit Breaker

​Configuration

​Error Classification

​FallbackProvider

​Callbacks

​Streaming boundary

​Best Practices

Overview

Quick Start

Circuit Breaker

Configuration

Error Classification

FallbackProvider

Callbacks

Streaming boundary

Best Practices