Ollama (Local Models)

Use Ollama to run open-source models locally with Agentium. No API key required—ideal for development, privacy-sensitive workloads, and cost-free experimentation.

Setup

Install Ollama

Download and install Ollama from ollama.ai. Start the Ollama service (it runs on http://localhost:11434 by default).

Pull a model

Pull the model you want to use:

ollama pull llama3.1
ollama pull codellama
ollama pull mistral

Use in Agentium

Use the ollama factory with the model name:

import { ollama } from "@agentium/core";

const model = ollama("llama3.1");

Factory

import { ollama } from "@agentium/core";

const model = ollama("llama3.1");

string

required

The Ollama model name (e.g., llama3.1, codellama, mistral).

object

Optional configuration. See Config below.

Config

string

default:"http://localhost:11434"

Ollama server URL. Defaults to http://localhost:11434. Use this for remote Ollama instances or custom ports.

No API key needed

Ollama runs locally and does not require an API key. Just ensure the Ollama service is running.

Example

const model = ollama("llama3.1", {
  host: "http://localhost:11434",
});

// Remote Ollama instance
const remoteModel = ollama("mistral", {
  host: "http://192.168.1.100:11434",
});

Popular Models

Model	Use Case
`llama3.1`	General purpose, strong all-around performance
`codellama`	Code generation and understanding
`mistral`	Fast, efficient, good for chat
`mixtral`	Mixture of experts, higher capability
`phi3`	Small, fast, good for edge devices

Discover more models

Run ollama list to see installed models. Browse ollama.com/library for the full catalog.

Ollama supports image input for vision-capable models like llava, bakllava, and llama3.2-vision.

Images

Pass images as base64 data in ContentPart[]:

import { Agent, ollama, type ContentPart } from "@agentium/core";
import { readFileSync } from "node:fs";

const agent = new Agent({
  name: "VisionBot",
  model: ollama("llava"),
  instructions: "Describe images in detail.",
});

const imageData = readFileSync("photo.jpg").toString("base64");

const result = await agent.run([
  { type: "text", text: "What's in this image?" },
  { type: "image", data: imageData, mimeType: "image/jpeg" },
] as ContentPart[]);

Unsupported: Audio & Files

Audio and file inputs are not supported by Ollama. If passed, the provider logs a warning and skips them.

Tool Calling

Ollama supports function calling with select models. Enable tools on your agent as usual — Agentium handles the tool call protocol automatically:

import { Agent, ollama, defineTool } from "@agentium/core";
import { z } from "zod";

const agent = new Agent({
  name: "local-assistant",
  model: ollama("llama3.1"), // Supports tool calling
  tools: [
    defineTool({
      name: "getWeather",
      description: "Get current weather for a city",
      parameters: z.object({ city: z.string() }),
      execute: async ({ city }) => `${city}: 22°C, Sunny`,
    }),
  ],
  instructions: "You are a helpful assistant. Use tools when needed.",
});

const result = await agent.run("What's the weather in Paris?");
console.log(result.text);

Models with Tool Support

Model	Tool Calling
`llama3.1`	Yes
`llama3.2`	Yes
`mistral`	Yes
`mixtral`	Yes
`codellama`	No
`phi3`	No
`llava`	No (vision only)

Tool calling quality varies by model. Larger models (70B+) are more reliable for complex tool use. For production tool-calling workflows, consider llama3.1:70b or a cloud provider.

Performance Tips

GPU Acceleration

Ollama automatically uses GPU when available. Check GPU usage with:

ollama ps  # Shows running models and their GPU/CPU split

For best performance, ensure your model fits entirely in GPU memory. A 7B model typically needs ~4GB VRAM, 13B needs ~8GB, and 70B needs ~40GB.

Context Size

By default, Ollama uses a 2048-token context window. For agents with long conversations or large tool results, increase it:

# In your Modelfile or via the API
ollama run llama3.1 --ctx-size 8192

Model Selection Guidelines

Scenario	Recommended Model
General chat + tools	`llama3.1` or `llama3.1:70b`
Code generation	`codellama:13b` or `codellama:34b`
Vision tasks	`llava:13b` or `llama3.2-vision`
Fast responses (edge)	`phi3:mini` or `llama3.2:1b`
Complex reasoning	`mixtral:8x7b` or `llama3.1:70b`

Full Example

import { Agent, ollama } from "@agentium/core";

const agent = new Agent({
  name: "Local Assistant",
  model: ollama("llama3.1"),
  instructions: "You are a helpful assistant running locally.",
});

const output = await agent.run("Explain recursion in one sentence.");
console.log(output.text);

​Ollama (Local Models)

​Setup

​Factory

​Config

​Example

​Popular Models

​Multi-Modal Support

​Images

​Unsupported: Audio & Files

​Tool Calling

​Models with Tool Support

​Performance Tips

​GPU Acceleration

​Context Size

​Model Selection Guidelines

​Full Example

Ollama (Local Models)

Setup

Factory

Config

Example

Popular Models

Multi-Modal Support

Images

Unsupported: Audio & Files

Tool Calling

Models with Tool Support

Performance Tips

GPU Acceleration

Context Size

Model Selection Guidelines

Full Example