> ## Documentation Index
> Fetch the complete documentation index at: https://docs.agentium.in/llms.txt
> Use this file to discover all available pages before exploring further.

# Embeddings

> Generate vector embeddings with OpenAI and Google providers

Embedding providers convert text into high-dimensional vectors for semantic search. Agentium provides a unified `EmbeddingProvider` interface with OpenAI and Google implementations.

## EmbeddingProvider Interface

```typescript theme={null}
interface EmbeddingProvider {
  readonly dimensions: number;
  embed(text: string): Promise<number[]>;
  embedBatch(texts: string[]): Promise<number[][]>;
  /** Optional - true for models like gemini-embedding-2 */
  readonly supportsMultimodal?: boolean;
  /** Optional - returns ONE vector for the full multimodal input */
  embedMultimodal?(input: string | ContentPart | ContentPart[]): Promise<number[]>;
}
```

## OpenAI Embeddings

```bash theme={null}
npm install openai
```

```typescript theme={null}
import { OpenAIEmbedding } from "@agentium/core";

const embedder = new OpenAIEmbedding({
  apiKey: process.env.OPENAI_API_KEY,  // optional, uses env var by default
  model: "text-embedding-3-small",      // optional, this is the default
});

const vector = await embedder.embed("Hello world");
console.log(vector.length); // 1536

const vectors = await embedder.embedBatch(["Hello", "World"]);
console.log(vectors.length); // 2
```

### Available Models

| Model                    | Dimensions | Best For                    |
| ------------------------ | ---------- | --------------------------- |
| `text-embedding-3-small` | 1536       | General use, cost-effective |
| `text-embedding-3-large` | 3072       | Higher accuracy             |
| `text-embedding-ada-002` | 1536       | Legacy                      |

<ParamField body="apiKey" type="string">
  OpenAI API key. Falls back to `OPENAI_API_KEY` env var.
</ParamField>

<ParamField body="model" type="string" default="text-embedding-3-small">
  Embedding model name.
</ParamField>

## Google Embeddings

```bash theme={null}
npm install @google/genai
```

```typescript theme={null}
import { GoogleEmbedding } from "@agentium/core";

const embedder = new GoogleEmbedding({
  apiKey: process.env.GOOGLE_API_KEY,  // optional, uses env var by default
  model: "text-embedding-004",         // optional, this is the default
});

const vector = await embedder.embed("Hello world");
const vectors = await embedder.embedBatch(["Hello", "World"]);
```

### Available Models

| Model                  | Dimensions                   | Modalities                         | Best For                          |
| ---------------------- | ---------------------------- | ---------------------------------- | --------------------------------- |
| `text-embedding-004`   | 768                          | Text                               | General text, cost-effective      |
| `embedding-001`        | 768                          | Text                               | Legacy                            |
| `gemini-embedding-001` | 3072                         | Text                               | Higher accuracy text-only         |
| `gemini-embedding-2`   | 3072 (Matryoshka 128 - 3072) | Text + image + audio + video + PDF | Multimodal RAG and unified search |

<ParamField body="apiKey" type="string">
  Google API key. Falls back to `GOOGLE_API_KEY` env var.
</ParamField>

<ParamField body="model" type="string" default="text-embedding-004">
  Embedding model name. Switch to `gemini-embedding-2` for multimodal input.
</ParamField>

<ParamField body="dimensions" type="number">
  Override the output dimension. `gemini-embedding-2` supports any value from 128 to 3072 (recommended: 768, 1536, or 3072) thanks to Matryoshka Representation Learning.
</ParamField>

## Multimodal Embeddings (Gemini Embedding 2)

`gemini-embedding-2` maps text, images, audio, video, and PDFs into a single unified vector space. One call returns one aggregated 3072-dim vector for the whole input - useful when you want a single semantic key for a heterogeneous document.

### Index an image with a caption

```typescript theme={null}
import {
  GoogleEmbedding,
  InMemoryVectorStore,
  partsFromFile,
} from "@agentium/core";

const embedder = new GoogleEmbedding({ model: "gemini-embedding-2" });
const store = new InMemoryVectorStore(embedder);

const imagePart = await partsFromFile("./photo.jpg");

await store.upsert("photos", {
  id: "photo-1",
  content: "Sunset over the Sahara",
  parts: [
    { type: "text", text: "Sunset over the Sahara" },
    imagePart,
  ],
  metadata: { tags: ["nature", "africa"] },
});
```

### Search by image

```typescript theme={null}
const queryImage = await partsFromFile("./query.jpg");
const hits = await store.search("photos", [queryImage], { topK: 5 });
console.log(hits);
```

### Supported modalities

* **Text** -> `{ type: "text", text }`
* **Image** -> `{ type: "image", data, mimeType }` (base64 or HTTPS URL; supported MIME `image/png|jpeg|gif|webp`)
* **Audio** -> `{ type: "audio", data, mimeType }`
* **Video** -> `{ type: "file", data, mimeType: "video/mp4" }`
* **PDF** -> `{ type: "file", data, mimeType: "application/pdf" }`

Helpers:

* `partsFromFile(path, mimeType?)` reads a local file, infers MIME from extension, and returns the right `ContentPart`.
* `fetchAsBase64(url)` downloads an HTTPS URL and returns `{ data, mimeType }`.

### Output dimensions (Matryoshka)

`gemini-embedding-2` is trained with Matryoshka Representation Learning, so you can store smaller vectors without retraining:

```typescript theme={null}
const compact = new GoogleEmbedding({
  model: "gemini-embedding-2",
  dimensions: 768,  // also supported: 128-3072; recommended 768 / 1536 / 3072
});
```

### Important: v1 and v2 vectors are NOT interchangeable

Embeddings produced by `text-embedding-004` / `gemini-embedding-001` live in a different semantic space than those produced by `gemini-embedding-2`. If you upgrade, **re-index your collection** - cosine similarity across the two spaces is meaningless.

### Limitations

* **One aggregated vector per call.** If you need separate vectors per item, call `embedMultimodal` once per item (the Gemini Batch API for per-item vectors is on the roadmap).
* **Vector backends store text only.** The `content` field and the embedding are persisted; the original `parts` are not. Put any structured payload you need to keep (image URL, file path, page number) in `metadata`.
* **Per-call media limits** (enforced by Google): 6 images, 120s video, 180s audio, 6 PDF pages.

## Using with KnowledgeBase

Embedding providers are passed to `KnowledgeBase` via the vector store. Most vector stores accept an `EmbeddingProvider` in their configuration or the `KnowledgeBase` handles embedding internally.

```typescript theme={null}
import { KnowledgeBase, InMemoryVectorStore, OpenAIEmbedding } from "@agentium/core";

const embedder = new OpenAIEmbedding();
const vectorStore = new InMemoryVectorStore(1536);

const kb = new KnowledgeBase({
  name: "docs",
  vectorStore,
});
```

<Card title="RAG Example" icon="magnifying-glass" href="/knowledge/rag-example">
  See a complete end-to-end RAG implementation using embeddings.
</Card>