Documentation Index

Fetch the complete documentation index at: https://docs.agentium.in/llms.txt

Use this file to discover all available pages before exploring further.

​

Semantic Tool Selection

​

The problem

1. Prompt bloat. 100 tools × ~80 tokens each = 8K tokens of pure tool definitions per turn, before the user’s message even gets to the model.
2. Tool confusion. Models pick worse tools when more options are available (a phenomenon Cohere and Anthropic have both documented).
3. Slower generation. More input tokens means more time-to-first-token.

​

How it works

                                  ┌──────────────────────────────────┐
   Tools at init  ────────────────│ embedder.embedBatch              │ ─▶ one vector per tool, cached
                                  └──────────────────────────────────┘

   User turn ────▶ embedder.embed(query) ─▶ cosine vs cached tool vectors ─▶ top-K (optionally reranked)

​

Quick start

import { Agent, openai, OpenAIEmbedding, SemanticToolSelector } from "@agentium/core";

const selector = new SemanticToolSelector({
  embedder: new OpenAIEmbedding({ model: "text-embedding-3-small" }),
  topK: 5,
});

await selector.indexTools(allMyTools); // 100+ tools

// Per turn:
const shortlist = await selector.select(userInput);
const agent = new Agent({ name: "scoped", model: openai("gpt-4o"), tools: shortlist });

​

Configuration

interface SemanticToolSelectorConfig {
  embedder: EmbeddingProvider;    // required — see Embeddings doc for choices
  reranker?: Reranker;            // optional — for tighter selection
  topK?: number;                  // default 10
  rerankMultiplier?: number;      // default 3 — only used when reranker is set
}

​

embedder

• Cheap and fast — picks happen every turn.
• Decent on short text — tool names + 1-2 sentence descriptions.

​

reranker

import { CohereReranker } from "@agentium/core";

const selector = new SemanticToolSelector({
  embedder: new OpenAIEmbedding(),
  reranker: new CohereReranker(),
  topK: 5,
  rerankMultiplier: 4,            // fetch top-20, rerank to top-5
});

1. Bi-encoder scores all indexed tools by cosine similarity to the query.
2. Top topK * rerankMultiplier tools are passed to the reranker.
3. Reranker scores each (query, tool description) pair and returns the top topK.

​

topK and rerankMultiplier

​

API

​

indexTools(tools: ToolDef[])

await selector.indexTools(allTools);
console.log(`Indexed ${selector.size} tools`); // 100

• At agent boot
• Whenever the tool set changes (e.g. an MCP server connects)
• Not on every turn

​

select(query: string, options?: { topK?: number }): Promise<ToolDef[]>

const tools = await selector.select("What's the weather in Tokyo?", { topK: 3 });

• Returns [] if indexTools hasn’t been called.
• Empty query — still returns the closest topK (the embedding of "" is rarely useful but doesn’t error).
• Same tool indexed twice — both copies returned independently; dedupe at construction time.

​

size

selector.size // number of indexed tools

​

Wire into a per-request agent

const selector = new SemanticToolSelector({ embedder, topK: 5 });
await selector.indexTools(allTools);

app.post("/chat", async (req, res) => {
  const shortlist = await selector.select(req.body.input);
  const agent = factory.create({
    tenantId: req.user.tenant,
    userId: req.user.id,
  });
  agent.setTools([...alwaysOn, ...shortlist]); // mix critical + selected
  const result = await agent.run(req.body.input);
  res.json(result);
});

​

Tips

• Always include critical tools unconditionally. handoff, approval, pollResult, getArtifact should always be in the agent’s toolset regardless of semantic match.
• Tool descriptions matter more than names. “fetch the temperature for a city” beats “weather_api_v3”.
• Reindex after dynamic tool registration. MCP connections happen async; call indexTools again after a successful connect.
• Cache the index. If your tool set is stable across processes, persist the embeddings to disk and re-hydrate on boot.

​

Performance characteristics

​

See also

• Reranking — the optional second pass
• Tool Polish — pair with inputExamples for better selection accuracy
• AgentFactory — for per-request agents