> ## Documentation Index
> Fetch the complete documentation index at: https://docs.agentium.in/llms.txt
> Use this file to discover all available pages before exploring further.

# Ollama (Local Models)

> Run local LLMs with Ollama and Agentium. No API key needed. Llama, CodeLlama, Mistral, and more.

# Ollama (Local Models)

Use **Ollama** to run open-source models locally with Agentium. No API key required—ideal for development, privacy-sensitive workloads, and cost-free experimentation.

***

## Setup

<Steps>
  <Step title="Install Ollama">
    Download and install Ollama from [ollama.ai](https://ollama.ai). Start the Ollama service (it runs on `http://localhost:11434` by default).
  </Step>

  <Step title="Pull a model">
    Pull the model you want to use:

    ```bash theme={null}
    ollama pull llama3.1
    ollama pull codellama
    ollama pull mistral
    ```
  </Step>

  <Step title="Use in Agentium">
    Use the `ollama` factory with the model name:
  </Step>
</Steps>

```typescript theme={null}
import { ollama } from "@agentium/core";

const model = ollama("llama3.1");
```

***

## Factory

```typescript theme={null}
import { ollama } from "@agentium/core";

const model = ollama("llama3.1");
```

<ParamField path="modelId" type="string" required>
  The Ollama model name (e.g., `llama3.1`, `codellama`, `mistral`).
</ParamField>

<ParamField path="config" type="object" required={false}>
  Optional configuration. See Config below.
</ParamField>

***

## Config

<ParamField path="host" type="string" required={false} default="http://localhost:11434">
  Ollama server URL. Defaults to `http://localhost:11434`. Use this for remote Ollama instances or custom ports.
</ParamField>

<Accordion title="No API key needed">
  Ollama runs locally and does not require an API key. Just ensure the Ollama service is running.
</Accordion>

### Example

```typescript theme={null}
const model = ollama("llama3.1", {
  host: "http://localhost:11434",
});

// Remote Ollama instance
const remoteModel = ollama("mistral", {
  host: "http://192.168.1.100:11434",
});
```

***

## Popular Models

| Model       | Use Case                                       |
| ----------- | ---------------------------------------------- |
| `llama3.1`  | General purpose, strong all-around performance |
| `codellama` | Code generation and understanding              |
| `mistral`   | Fast, efficient, good for chat                 |
| `mixtral`   | Mixture of experts, higher capability          |
| `phi3`      | Small, fast, good for edge devices             |

<Accordion title="Discover more models">
  Run `ollama list` to see installed models. Browse [ollama.com/library](https://ollama.com/library) for the full catalog.
</Accordion>

***

## Multi-Modal Support

Ollama supports **image** input for vision-capable models like `llava`, `bakllava`, and `llama3.2-vision`.

### Images

Pass images as base64 data in `ContentPart[]`:

```typescript theme={null}
import { Agent, ollama, type ContentPart } from "@agentium/core";
import { readFileSync } from "node:fs";

const agent = new Agent({
  name: "VisionBot",
  model: ollama("llava"),
  instructions: "Describe images in detail.",
});

const imageData = readFileSync("photo.jpg").toString("base64");

const result = await agent.run([
  { type: "text", text: "What's in this image?" },
  { type: "image", data: imageData, mimeType: "image/jpeg" },
] as ContentPart[]);
```

### Unsupported: Audio & Files

Audio and file inputs are not supported by Ollama. If passed, the provider logs a warning and skips them.

***

## Tool Calling

Ollama supports function calling with select models. Enable tools on your agent as usual — Agentium handles the tool call protocol automatically:

```typescript theme={null}
import { Agent, ollama, defineTool } from "@agentium/core";
import { z } from "zod";

const agent = new Agent({
  name: "local-assistant",
  model: ollama("llama3.1"), // Supports tool calling
  tools: [
    defineTool({
      name: "getWeather",
      description: "Get current weather for a city",
      parameters: z.object({ city: z.string() }),
      execute: async ({ city }) => `${city}: 22°C, Sunny`,
    }),
  ],
  instructions: "You are a helpful assistant. Use tools when needed.",
});

const result = await agent.run("What's the weather in Paris?");
console.log(result.text);
```

### Models with Tool Support

| Model       | Tool Calling     |
| ----------- | ---------------- |
| `llama3.1`  | Yes              |
| `llama3.2`  | Yes              |
| `mistral`   | Yes              |
| `mixtral`   | Yes              |
| `codellama` | No               |
| `phi3`      | No               |
| `llava`     | No (vision only) |

<Warning>
  Tool calling quality varies by model. Larger models (70B+) are more reliable for complex tool use. For production tool-calling workflows, consider `llama3.1:70b` or a cloud provider.
</Warning>

***

## Performance Tips

### GPU Acceleration

Ollama automatically uses GPU when available. Check GPU usage with:

```bash theme={null}
ollama ps  # Shows running models and their GPU/CPU split
```

For best performance, ensure your model fits entirely in GPU memory. A 7B model typically needs \~4GB VRAM, 13B needs \~8GB, and 70B needs \~40GB.

### Context Size

By default, Ollama uses a 2048-token context window. For agents with long conversations or large tool results, increase it:

```bash theme={null}
# In your Modelfile or via the API
ollama run llama3.1 --ctx-size 8192
```

### Model Selection Guidelines

| Scenario              | Recommended Model                  |
| --------------------- | ---------------------------------- |
| General chat + tools  | `llama3.1` or `llama3.1:70b`       |
| Code generation       | `codellama:13b` or `codellama:34b` |
| Vision tasks          | `llava:13b` or `llama3.2-vision`   |
| Fast responses (edge) | `phi3:mini` or `llama3.2:1b`       |
| Complex reasoning     | `mixtral:8x7b` or `llama3.1:70b`   |

***

## Full Example

```typescript theme={null}
import { Agent, ollama } from "@agentium/core";

const agent = new Agent({
  name: "Local Assistant",
  model: ollama("llama3.1"),
  instructions: "You are a helpful assistant running locally.",
});

const output = await agent.run("Explain recursion in one sentence.");
console.log(output.text);
```