OpenAI-Compatible Models

Many LLM providers expose an OpenAI-compatible Chat Completions API. Agentium lets you connect to any of them by simply providing a baseURL and apiKey — no custom provider code required. This works for providers like Together AI, Groq, Fireworks, OpenRouter, NVIDIA, DeepInfra, Cerebras, SambaNova, LM Studio, vLLM, and any other service that implements the OpenAI format.

Two Ways to Connect

Option 1: `openai()` Factory with Custom `baseURL`

The simplest approach — use the existing openai() factory:

import { openai } from "@agentium/core";

const model = openai("mistralai/Mixtral-8x7B-Instruct-v0.1", {
  apiKey: process.env.TOGETHER_API_KEY,
  baseURL: "https://api.together.xyz/v1",
});

This works because these providers use the same API format as OpenAI.

Option 2: `OpenAICompatibleProvider` Class

For more control or when building reusable providers, use the class directly:

import { OpenAICompatibleProvider } from "@agentium/core";

const model = new OpenAICompatibleProvider(
  "together",                                    // providerId
  "mistralai/Mixtral-8x7B-Instruct-v0.1",       // modelId
  { baseURL: "https://api.together.xyz/v1", apiKeyEnvVar: "TOGETHER_API_KEY" },
);

Provider Examples

Together AI

Access 100+ open-source models including Llama, Mistral, Qwen, and more.

import { Agent, openai } from "@agentium/core";

const agent = new Agent({
  name: "together-agent",
  model: openai("meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo", {
    apiKey: process.env.TOGETHER_API_KEY,
    baseURL: "https://api.together.xyz/v1",
  }),
  instructions: "You are a helpful assistant.",
});

const result = await agent.run("Explain how transformers work in 3 sentences.");
console.log(result.text);

Environment:

export TOGETHER_API_KEY="..."

Groq

Ultra-fast inference for open models.

import { Agent, openai } from "@agentium/core";

const agent = new Agent({
  name: "groq-agent",
  model: openai("llama-3.3-70b-versatile", {
    apiKey: process.env.GROQ_API_KEY,
    baseURL: "https://api.groq.com/openai/v1",
  }),
  instructions: "You are a fast, helpful assistant.",
});

const result = await agent.run("What is the capital of France?");
console.log(result.text);

Environment:

export GROQ_API_KEY="..."

Fireworks AI

Optimized inference for open-source models.

import { Agent, openai } from "@agentium/core";

const agent = new Agent({
  name: "fireworks-agent",
  model: openai("accounts/fireworks/models/llama-v3p1-70b-instruct", {
    apiKey: process.env.FIREWORKS_API_KEY,
    baseURL: "https://api.fireworks.ai/inference/v1",
  }),
  instructions: "You are a helpful assistant.",
});

const result = await agent.run("Write a haiku about programming.");
console.log(result.text);

Environment:

export FIREWORKS_API_KEY="..."

OpenRouter

Access models from multiple providers through a single API.

import { Agent, openai } from "@agentium/core";

const agent = new Agent({
  name: "openrouter-agent",
  model: openai("anthropic/claude-sonnet-4-20250514", {
    apiKey: process.env.OPENROUTER_API_KEY,
    baseURL: "https://openrouter.ai/api/v1",
  }),
  instructions: "You are a helpful assistant.",
});

const result = await agent.run("What are the pros and cons of microservices?");
console.log(result.text);

Environment:

export OPENROUTER_API_KEY="..."

NVIDIA NIM

NVIDIA-hosted inference for optimized models.

import { Agent, openai } from "@agentium/core";

const agent = new Agent({
  name: "nvidia-agent",
  model: openai("meta/llama-3.1-70b-instruct", {
    apiKey: process.env.NVIDIA_API_KEY,
    baseURL: "https://integrate.api.nvidia.com/v1",
  }),
  instructions: "You are a helpful assistant.",
});

const result = await agent.run("Explain GPU parallelism.");
console.log(result.text);

Environment:

export NVIDIA_API_KEY="..."

DeepInfra

Serverless inference at low cost.

import { Agent, openai } from "@agentium/core";

const agent = new Agent({
  name: "deepinfra-agent",
  model: openai("meta-llama/Meta-Llama-3.1-70B-Instruct", {
    apiKey: process.env.DEEPINFRA_API_KEY,
    baseURL: "https://api.deepinfra.com/v1/openai",
  }),
  instructions: "You are a helpful assistant.",
});

const result = await agent.run("What is retrieval-augmented generation?");
console.log(result.text);

Environment:

export DEEPINFRA_API_KEY="..."

LM Studio (Local)

Connect to a locally-running LM Studio server.

import { Agent, openai } from "@agentium/core";

const agent = new Agent({
  name: "local-agent",
  model: openai("local-model", {
    apiKey: "lm-studio",  // LM Studio doesn't validate keys
    baseURL: "http://localhost:1234/v1",
  }),
  instructions: "You are a helpful assistant.",
});

const result = await agent.run("Hello! How are you?");
console.log(result.text);

vLLM (Self-Hosted)

Connect to a self-hosted vLLM server.

import { Agent, openai } from "@agentium/core";

const agent = new Agent({
  name: "vllm-agent",
  model: openai("meta-llama/Meta-Llama-3.1-8B-Instruct", {
    apiKey: "token-abc123",  // your vLLM auth token
    baseURL: "http://localhost:8000/v1",
  }),
  instructions: "You are a helpful assistant.",
});

const result = await agent.run("What is vLLM?");
console.log(result.text);

Tool Calling

Tool calling works with any OpenAI-compatible provider that supports function calling:

import { Agent, openai, defineTool } from "@agentium/core";
import { z } from "zod";

const agent = new Agent({
  name: "tool-agent",
  model: openai("meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo", {
    apiKey: process.env.TOGETHER_API_KEY,
    baseURL: "https://api.together.xyz/v1",
  }),
  tools: [
    defineTool({
      name: "getWeather",
      description: "Get current weather for a city",
      parameters: z.object({ city: z.string() }),
      execute: async ({ city }) => `${city}: 22°C, Sunny`,
    }),
  ],
});

const result = await agent.run("What's the weather in Paris?");
console.log(result.text);

Not all models or providers support tool calling. Check your provider’s documentation for function calling support. Larger models (70B+) generally have better tool calling reliability.

Registering a Custom Provider

If you use a specific endpoint frequently, register it as a named provider:

import { modelRegistry, OpenAICompatibleProvider } from "@agentium/core";

modelRegistry.register(
  "together",
  (modelId) =>
    new OpenAICompatibleProvider("together", modelId, {
      baseURL: "https://api.together.xyz/v1",
      apiKeyEnvVar: "TOGETHER_API_KEY",
    }),
);

// Now use it like a built-in provider
const model = modelRegistry.resolve("together", "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo");

Supported Provider Endpoints

Provider	Base URL	Env Var	Tool Calling
Together AI	`https://api.together.xyz/v1`	`TOGETHER_API_KEY`	Yes (most models)
Groq	`https://api.groq.com/openai/v1`	`GROQ_API_KEY`	Yes
Fireworks	`https://api.fireworks.ai/inference/v1`	`FIREWORKS_API_KEY`	Yes
OpenRouter	`https://openrouter.ai/api/v1`	`OPENROUTER_API_KEY`	Model-dependent
NVIDIA NIM	`https://integrate.api.nvidia.com/v1`	`NVIDIA_API_KEY`	Yes
DeepInfra	`https://api.deepinfra.com/v1/openai`	`DEEPINFRA_API_KEY`	Yes
Cerebras	`https://api.cerebras.ai/v1`	`CEREBRAS_API_KEY`	Yes
SambaNova	`https://api.sambanova.ai/v1`	`SAMBANOVA_API_KEY`	Model-dependent
LM Studio	`http://localhost:1234/v1`	N/A	Model-dependent
vLLM	`http://localhost:8000/v1`	Custom	Model-dependent

Full Example

import { Agent, openai, CostTracker, defineTool } from "@agentium/core";
import { z } from "zod";

const costTracker = new CostTracker({
  pricing: {
    "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo": { promptPer1k: 0.00088, completionPer1k: 0.00088 },
  },
});

const agent = new Agent({
  name: "together-assistant",
  model: openai("meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo", {
    apiKey: process.env.TOGETHER_API_KEY,
    baseURL: "https://api.together.xyz/v1",
  }),
  instructions: "You are a helpful coding assistant.",
  tools: [
    defineTool({
      name: "searchDocs",
      description: "Search documentation",
      parameters: z.object({ query: z.string() }),
      execute: async ({ query }) => `Results for "${query}": ...`,
    }),
  ],
  costTracker,
  maxToolRoundtrips: 3,
});

const result = await agent.run("How do I set up a Next.js project with TypeScript?");
console.log(result.text);
console.log(`Cost: $${costTracker.getSummary().totalCost.toFixed(4)}`);

Cross-References

OpenAI — Direct OpenAI API
Ollama — Local models with a different interface
Custom Provider — Build a provider from scratch
Models Overview — All built-in providers

​OpenAI-Compatible Models

​Two Ways to Connect

​Option 1: openai() Factory with Custom baseURL

​Option 2: OpenAICompatibleProvider Class

​Provider Examples

​Together AI

​Groq

​Fireworks AI

​OpenRouter

​NVIDIA NIM

​DeepInfra

​LM Studio (Local)

​vLLM (Self-Hosted)

​Tool Calling

​Registering a Custom Provider

​Supported Provider Endpoints

​Full Example

​Cross-References

OpenAI-Compatible Models

Two Ways to Connect

Option 1: `openai()` Factory with Custom `baseURL`

Option 2: `OpenAICompatibleProvider` Class

Provider Examples

Together AI

Groq

Fireworks AI

OpenRouter

NVIDIA NIM

DeepInfra

LM Studio (Local)

vLLM (Self-Hosted)

Tool Calling

Registering a Custom Provider

Supported Provider Endpoints

Full Example

Cross-References