Connect to any OpenAI-compatible API endpoint — Together, Groq, Fireworks, OpenRouter, NVIDIA, LM Studio, and more.
Many LLM providers expose an OpenAI-compatible Chat Completions API. Agentium lets you connect to any of them by simply providing a baseURL and apiKey — no custom provider code required.This works for providers like Together AI, Groq, Fireworks, OpenRouter, NVIDIA, DeepInfra, Cerebras, SambaNova, LM Studio, vLLM, and any other service that implements the OpenAI format.
Two Ways to Connect
Option 1: openai() Factory with Custom baseURL
The simplest approach — use the existing openai() factory:
import { openai } from "@agentium/core";const model = openai("mistralai/Mixtral-8x7B-Instruct-v0.1", { apiKey: process.env.TOGETHER_API_KEY, baseURL: "https://api.together.xyz/v1",});
This works because these providers use the same API format as OpenAI.
Option 2: OpenAICompatibleProvider Class
For more control or when building reusable providers, use the class directly:
import { OpenAICompatibleProvider } from "@agentium/core";const model = new OpenAICompatibleProvider( "together", // providerId "mistralai/Mixtral-8x7B-Instruct-v0.1", // modelId { baseURL: "https://api.together.xyz/v1", apiKeyEnvVar: "TOGETHER_API_KEY" },);
Provider Examples
Together AI
Access 100+ open-source models including Llama, Mistral, Qwen, and more.
import { Agent, openai } from "@agentium/core";const agent = new Agent({ name: "together-agent", model: openai("meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo", { apiKey: process.env.TOGETHER_API_KEY, baseURL: "https://api.together.xyz/v1", }), instructions: "You are a helpful assistant.",});const result = await agent.run("Explain how transformers work in 3 sentences.");console.log(result.text);
Environment:
export TOGETHER_API_KEY="..."
Groq
Ultra-fast inference for open models.
import { Agent, openai } from "@agentium/core";const agent = new Agent({ name: "groq-agent", model: openai("llama-3.3-70b-versatile", { apiKey: process.env.GROQ_API_KEY, baseURL: "https://api.groq.com/openai/v1", }), instructions: "You are a fast, helpful assistant.",});const result = await agent.run("What is the capital of France?");console.log(result.text);
Environment:
export GROQ_API_KEY="..."
Fireworks AI
Optimized inference for open-source models.
import { Agent, openai } from "@agentium/core";const agent = new Agent({ name: "fireworks-agent", model: openai("accounts/fireworks/models/llama-v3p1-70b-instruct", { apiKey: process.env.FIREWORKS_API_KEY, baseURL: "https://api.fireworks.ai/inference/v1", }), instructions: "You are a helpful assistant.",});const result = await agent.run("Write a haiku about programming.");console.log(result.text);
Environment:
export FIREWORKS_API_KEY="..."
OpenRouter
Access models from multiple providers through a single API.
import { Agent, openai } from "@agentium/core";const agent = new Agent({ name: "openrouter-agent", model: openai("anthropic/claude-sonnet-4-20250514", { apiKey: process.env.OPENROUTER_API_KEY, baseURL: "https://openrouter.ai/api/v1", }), instructions: "You are a helpful assistant.",});const result = await agent.run("What are the pros and cons of microservices?");console.log(result.text);
Environment:
export OPENROUTER_API_KEY="..."
NVIDIA NIM
NVIDIA-hosted inference for optimized models.
import { Agent, openai } from "@agentium/core";const agent = new Agent({ name: "nvidia-agent", model: openai("meta/llama-3.1-70b-instruct", { apiKey: process.env.NVIDIA_API_KEY, baseURL: "https://integrate.api.nvidia.com/v1", }), instructions: "You are a helpful assistant.",});const result = await agent.run("Explain GPU parallelism.");console.log(result.text);
Environment:
export NVIDIA_API_KEY="..."
DeepInfra
Serverless inference at low cost.
import { Agent, openai } from "@agentium/core";const agent = new Agent({ name: "deepinfra-agent", model: openai("meta-llama/Meta-Llama-3.1-70B-Instruct", { apiKey: process.env.DEEPINFRA_API_KEY, baseURL: "https://api.deepinfra.com/v1/openai", }), instructions: "You are a helpful assistant.",});const result = await agent.run("What is retrieval-augmented generation?");console.log(result.text);
Environment:
export DEEPINFRA_API_KEY="..."
LM Studio (Local)
Connect to a locally-running LM Studio server.
import { Agent, openai } from "@agentium/core";const agent = new Agent({ name: "local-agent", model: openai("local-model", { apiKey: "lm-studio", // LM Studio doesn't validate keys baseURL: "http://localhost:1234/v1", }), instructions: "You are a helpful assistant.",});const result = await agent.run("Hello! How are you?");console.log(result.text);
vLLM (Self-Hosted)
Connect to a self-hosted vLLM server.
import { Agent, openai } from "@agentium/core";const agent = new Agent({ name: "vllm-agent", model: openai("meta-llama/Meta-Llama-3.1-8B-Instruct", { apiKey: "token-abc123", // your vLLM auth token baseURL: "http://localhost:8000/v1", }), instructions: "You are a helpful assistant.",});const result = await agent.run("What is vLLM?");console.log(result.text);
Tool Calling
Tool calling works with any OpenAI-compatible provider that supports function calling:
import { Agent, openai, defineTool } from "@agentium/core";import { z } from "zod";const agent = new Agent({ name: "tool-agent", model: openai("meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo", { apiKey: process.env.TOGETHER_API_KEY, baseURL: "https://api.together.xyz/v1", }), tools: [ defineTool({ name: "getWeather", description: "Get current weather for a city", parameters: z.object({ city: z.string() }), execute: async ({ city }) => `${city}: 22°C, Sunny`, }), ],});const result = await agent.run("What's the weather in Paris?");console.log(result.text);
Not all models or providers support tool calling. Check your provider’s documentation for function calling support. Larger models (70B+) generally have better tool calling reliability.
Registering a Custom Provider
If you use a specific endpoint frequently, register it as a named provider:
import { modelRegistry, OpenAICompatibleProvider } from "@agentium/core";modelRegistry.register( "together", (modelId) => new OpenAICompatibleProvider("together", modelId, { baseURL: "https://api.together.xyz/v1", apiKeyEnvVar: "TOGETHER_API_KEY", }),);// Now use it like a built-in providerconst model = modelRegistry.resolve("together", "meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo");
If the gateway is proxying GPT-5.4+ / GPT-5.6 / GPT-6 with tools, Chat Completions 400s unless reasoning_effort is "none". Agentium tries /v1/responses first (LiteLLM and recent OpenRouter expose it) and otherwise forces reasoning_effort: "none" so function tools still work. See Reasoning.
Full Example
The tracker preserves reported usage. A missing tariff or billing field makes total null. Supply a catalog that matches your billing provider, API, model or deployment, tier, and region; see cost accounting.
import { Agent, openai, defineTool } from "@agentium/core";import { z } from "zod";const agent = new Agent({ name: "together-assistant", model: openai("meta-llama/Meta-Llama-3.1-70B-Instruct-Turbo", { apiKey: process.env.TOGETHER_API_KEY, baseURL: "https://api.together.xyz/v1", }), instructions: "You are a helpful coding assistant.", tools: [ defineTool({ name: "searchDocs", description: "Search documentation", parameters: z.object({ query: z.string() }), execute: async ({ query }) => `Results for "${query}": ...`, }), ], cost: true, maxToolRoundtrips: 3,});const result = await agent.run("How do I set up a Next.js project with TypeScript?");console.log(result.text);console.log(result.costs);await agent.close();