KnowledgeBase. The knowledge base delegates vector embedding to that store; it does not infer an embedding model from a dimension number.
Use HashEmbedding for deterministic local wiring checks and a measured semantic model for real retrieval. Model availability and media support remain provider contracts.
EmbeddingProvider interface
embedBatch() returns a vector per text input. embedMultimodal() returns one vector for its combined input.
Try a local embedder
Use an ESM TypeScript project ("type": "module" in package.json) on a supported Node version.
embedding.ts and run npx tsx embedding.ts:
OpenAI embeddings
Installopenai, set OPENAI_API_KEY, and run this with an account that can use the selected embedding model:
Available models
OpenAI documentstext-embedding-3-small and text-embedding-3-large, with default vector sizes 1,536 and 3,072 respectively. The adapter also retains a dimension mapping for legacy text-embedding-ada-002. Provider guidance was checked on 2026-10-04. OpenAI embedding guide.
The adapter accepts apiKey, model, and dimensions. Its defaults are OPENAI_API_KEY, text-embedding-3-small, and that model’s known dimension. If you override dimensions, make the vector backend match and re-index existing documents as necessary.
Google embeddings
Install@google/genai and set GOOGLE_API_KEY. This example explicitly selects gemini-embedding-2; the v4 adapter’s constructor default is still text-embedding-004 when model is omitted.
Available models
Google’s current guide describesgemini-embedding-2 for text and multimodal input and gemini-embedding-001 for text. Both default to 3,072 dimensions. Checked 2026-10-04. Google embedding guide.
The adapter accepts apiKey, model, and dimensions. Its dimension lookup also includes older text-embedding-004 and embedding-001 IDs at 768; that lookup is not a guarantee those models remain available to an account. The adapter’s supportsMultimodal flag is based on a model-name prefix, so also verify the selected model’s actual provider capabilities.
EmbeddingGemma 2
EmbeddingGemmaEmbedding is included in the 4.11.0 candidate. Text and code embeddings arrived in 4.7.0; images, audio, and OpenAI-compatible video arrived in 4.8.0; native Ollama video arrived in 4.8.3. Version 4.5.0 does not export the class. These guides prepare 4.11.0; use matching local packages until exact registry availability is verified. OpenAIEmbedding, GoogleEmbedding, and HashEmbedding do not gain embedQuery() or embedMultimodalQuery().openai here means any OpenAI-compatible embeddings server, such as vLLM, llama.cpp, or LM Studio. It does not use the openai npm package. A bearer token is sent only when apiKey is set.
The model uses different prefixes for stored text and search queries. embed() and embedBatch() apply the document prefix. embedQuery() applies the query prefix. Vector search calls embedQuery() when the provider defines it, so a KnowledgeBase query stays on the query side. The other embedders omit embedQuery(), and search keeps calling embed() for them.
documentPrompt and queryPrompt replace those prefixes. Include the trailing space. A titled passage uses a document prompt such as title: src/store.ts | text: .
The native vector length is 768. dimensions accepts any integer from 128 through 768. Google’s evaluated lengths are 768, 512, 256, and 128. If the server returns a longer vector, the adapter keeps the prefix and re-normalizes it. A vector that is already the requested length is kept as returned. Checked 2026-10-09. EmbeddingGemma 2 announcement and text embedding guide.
This example indexes one policy and searches it. Ollama must already be serving embeddinggemma-2. The index stays in memory.
supportsMultimodal is true. embedMultimodal() embeds one document and puts the document prefix on its text. embedMultimodalQuery() does the same for a search made of content parts, using the query prefix. A plain string search still uses embedQuery(). Prefixes are not applied to images, audio, or video. Text that already starts with task: or title: is left as written.
Ollama’s /api/embed accepts text, images, audio, and video as a native payload. One clip is raw base64; several are an array. A remote URL is downloaded before the request. Pass the video file — do not extract frames into image parts. Optional video processing (maxFrames, fps, overflowStrategy, addTimestamps) is sent as processing_kwargs. The OpenAI-compatible backend uses chat content: image_url, input_audio, and video_url. An image URL is passed through on that backend. application/pdf is rejected. PDF input still uses GoogleEmbedding with gemini-embedding-2.
The server process has to load the encoders you use. A text-only EmbeddingGemma load (vision and audio disabled) cannot embed images, audio, or video. EmbeddingGemma 2 and Gemini Embedding 2 are different models. A shared length of 768 does not make their vectors comparable. Re-index when you switch.
This example indexes one image and searches it with text. Ollama must already be serving EmbeddingGemma 2 with its vision encoder, and photo.jpg must be in the working directory. The index stays in memory.
backend: "openai" and, when the server checks auth, apiKey. Code search uses task: "code". Video is a file part whose MIME type starts with video/ on either backend.
Multimodal embeddings (Gemini Embedding 2)
This complete example indexes an image with a caption and searches with another image. It requiresphoto.jpg and query.jpg in the working directory, Google credentials, and the optional client above. It makes live embedding requests and keeps its index only in memory.
Index an image with a caption
Search by image
PassContentPart[] as the search query, as shown above. A plain numeric vector bypasses query embedding; a string uses text embedding. The stored document and query must use the same embedding space and dimensions.
Supported modalities
Core’s multimodal inputs usetext, image, audio, and file parts. For a supported Google embedding model, the adapter converts image/audio/video/PDF parts into the provider request. partsFromFile(path, mimeType?) reads a file and returns one ContentPart; fetchAsBase64(url) fetches a remote resource and returns data and MIME type. Select trusted source files/URLs and respect the provider’s current size and media limits.
Output dimensions (Matryoshka)
Google documents reduced dimensions for its embedding models. Configure the requested dimension explicitly and keep the store consistent:Important: v1 and v2 vectors are NOT interchangeable
Changing the model changes the embedding space, even when vector lengths match. Re-index documents when switching model/version, dimensions, or incompatible input preparation. Keep the prior index until the replacement has passed your retrieval checks.Limitations
- Multimodal embedding returns one vector for the combined input; call separately when each item needs its own vector.
- The vector adapters retain text, vectors, and metadata, not original media parts. Store useful source references in metadata.
- Provider acceptance, account permissions, media limits, and live latency are not established by TypeScript compilation.
- Batch behavior is adapter-specific; the Google wrapper issues per-input requests in bounded groups rather than a managed provider batch job.
Using with KnowledgeBase
This complete local program uses the same provider → vector store → knowledge-base composition as a live setup:KnowledgeBase.close() closes its store, so coordinate ownership if multiple consumers share it. Continue with the retrieval tutorial to pass evidence into an Agent, or vector stores to choose persistence.