Documentation Index

Fetch the complete documentation index at: https://docs.xhipai.com/llms.txt

Use this file to discover all available pages before exploring further.

​

System Architecture

​

Monorepo Structure

​

Package Overview

​

Layered Architecture

1. SDK Layer — Agent, Team, Workflow, VoiceAgent, BrowserAgent. The primary API surface for defining behavior, orchestrating agents, and running workflows.
2. Engine Layer — LLM Loop, Tool Executor, MemoryManager (sessions, summaries, user facts, user profile, entities, decisions, learnings), SkillManager. Core execution logic with automatic retry, tool caching, token-based history trimming, reasoning, and cross-session personalization.
3. Safety Layer — Sandbox (isolated subprocess execution with timeout and memory limits), Approval Manager (human-in-the-loop gating before tool execution), Guardrails (input/output validation).
4. Model Abstraction — ModelProvider interface and adapters for text models. RealtimeProvider interface for voice/streaming models. Factory functions: openai(), anthropic(), google(), ollama(), vertex(), openaiRealtime(), googleLive().
5. Protocol Integration — MCP Client for consuming external tools, A2A Client for calling remote agents.
6. Infrastructure — Storage (in-memory, SQLite, PostgreSQL, MongoDB), Vector Stores, and Embeddings. All pluggable.
7. Registry & Auto-Discovery — Agents, Teams, and Workflows auto-register into a global Registry on construction. Transport layers read from the registry dynamically, so entities created at any time are immediately available over HTTP and WebSocket without restart or re-wiring.
8. Transport (Optional) — Express REST, Socket.IO WebSocket, Voice Gateway (real-time audio streaming), Browser Gateway (live browser observation), and A2A Server. Uses the Registry for live auto-discovery of agents, teams, and workflows.
9. Queue (Optional) — BullMQ workers for background job processing.

​

Data Flow — Text Agent

User Input
    │
Agent.run() / Agent.stream()
    │
buildMessages (history + system instructions + MemoryManager.buildContext() + skill instructions)
    │
LLM Loop (with retry)
    │
ModelProvider (OpenAI / Anthropic / Google / Ollama / Vertex)
    │
Response (text / tool calls)
    │
Tool Executor (if tool calls)
  ├── Approval check (if requiresApproval is set)
  ├── Sandbox execution (if sandbox is enabled)
  ├── Local tools (with optional caching)
  ├── MCP tools (external servers)
  └── A2A tools (remote agents)
    │
Loop until final response
    │
MemoryManager.appendMessages() → auto-summarize overflow
    │
MemoryManager.afterRun() → fire-and-forget extraction
  (user facts, user profile, entities, learnings)
    │
Output to caller

​

Detailed Flow

1. User Input — A string or multi-modal content (text, images, files).
2. Agent — Receives input, loads session history from MemoryManager, injects memory context and skill instructions into the system prompt.
3. buildMessages — Constructs the message array: system prompt (with summaries, user facts, user profile, entities, decisions, learnings, skill instructions), session history (auto-trimmed if maxTokens is set), current user message.
4. LLM Loop — Sends messages to the model with automatic retry on transient failures (429, 5xx, network errors).
5. ModelProvider — Translates to the provider API format.
6. Response — Either text or tool calls.
7. Tool Executor — If tool calls:
   • Checks human approval if requiresApproval is set on the tool or agent.
   • Runs the tool in a sandboxed subprocess if sandbox is enabled.
   • Executes the tool, appends results, and loops back to the model.
8. MemoryManager.appendMessages — Persists the new turn to session storage and auto-summarizes overflow.
9. MemoryManager.afterRun — Asynchronously extracts user facts, user profile, entities, and learnings from the conversation for future personalization.
10. Output — Returns or streams the final response to the caller.

​

Data Flow — Voice Agent

Audio Input (WebSocket / Socket.IO)
    │
VoiceAgent.connect()
    │
RealtimeProvider (OpenAI Realtime / Google Live)
    │
Bidirectional audio stream
    │
Tool calls (if any) → Tool Executor
    │
MemoryManager.appendMessages() (session persistence)
    │
MemoryManager.afterRun() (non-blocking extraction)
    │
Audio Output → Client

• VoiceSession — wraps the realtime provider connection, routes tool calls, emits events.
• Session persistence — conversation history saved via MemoryManager, restored on reconnect.
• Memory extraction — user facts, profile, entities, and learnings extracted from voice transcripts (non-blocking).

​

Data Flow — Browser Agent

Task (string)
    │
BrowserAgent.run()
    │
Launch Playwright (with stealth config + humanize settings)
    │
Screenshot → ModelProvider (vision)
    │
LLM decides action (click, type, scroll, navigate, done, fail)
    │
BrowserProvider executes action
  ├── CredentialVault resolves {{placeholders}} for type actions
  ├── DOM extraction (optional, for hybrid vision+DOM approach)
  └── Loop detection (maxRepeats threshold)
    │
Screenshot → next iteration
    │
Loop until "done" or "fail" or maxSteps reached
    │
Close browser (with optional cookie/auth persistence)
    │
Output result + action history

• Stealth mode — patches navigator.webdriver, WebGL, plugins to avoid bot detection.
• Humanize mode — random delays, mouse movement curves, typing variation.
• Credential vault — secrets never reach the LLM; only {{placeholders}} are used.
• Video recording — Playwright-native recording of browser sessions.
• Parallel browsing — multiple pages/tabs via BrowserProvider.
• Cookie persistence — save and restore storageState across runs.

​

Event System

​

Memory Architecture

​

Skills Architecture

​

Registry & Auto-Discovery

import { Agent, openai, registry } from "@agentium/core";

new Agent({ name: "bot", model: openai("gpt-4o") });

registry.list();
// { agents: ["bot"], teams: [], workflows: [] }

​

Performance Optimizations

​

Core Design Principles

• Zero Meta-Framework Dependency — No Next.js, Remix, or framework-specific runtime. Use Agentium with any Node.js server or headless.
• Optional Peer Dependencies — Providers (openai, anthropic, etc.) are peer dependencies. Lazy-loaded so you only bundle what you use.
• Event-Driven — EventBus emits lifecycle events. Subscribe for logging, analytics, or custom middleware.
• Pluggable Everything — Storage, models, vector stores, and transport are all swappable. Configure once, change later without rewriting logic.
• Safety by Default — Sandbox execution and human-in-the-loop approval are opt-in per tool or agent-wide. Guardrails validate input and output.
• Open Protocol Support — MCP for tool integration and A2A for agent interoperability. Connect to the broader AI ecosystem without vendor lock-in.
• Production Resilient — Automatic retry with exponential backoff, token-based context trimming, and non-blocking background operations ensure reliability at scale.