> ## Documentation Index
> Fetch the complete documentation index at: https://docs.agentium.in/llms.txt
> Use this file to discover all available pages before exploring further.
# Browser Agents
> Vision-based autonomous browser automation. An agent sees screenshots, decides actions, and operates web pages via Playwright.
# Browser Agents
Agentium supports **autonomous browser automation** through the `BrowserAgent` class in `@agentium/browser`. The agent uses a vision-capable LLM (GPT-4o, Gemini) to interpret screenshots of a browser and decide what actions to take — clicking, typing, scrolling, navigating — until the task is complete.
Browser agents use Playwright under the hood. After installing the package, run `npx playwright install chromium` to download the browser binary.
***
## Installation
```bash npm theme={null}
npm install @agentium/browser playwright
npx playwright install chromium
```
```bash pnpm theme={null}
pnpm add @agentium/browser playwright
npx playwright install chromium
```
***
## Quick Start
```typescript theme={null}
import { BrowserAgent } from "@agentium/browser";
import { openai } from "@agentium/core";
const browser = new BrowserAgent({
name: "web-navigator",
model: openai("gpt-4o"),
startUrl: "https://www.google.com",
maxSteps: 20,
});
const result = await browser.run(
"Search for 'TypeScript agent framework' and tell me the first 3 results"
);
console.log(result.success); // true
console.log(result.result); // "1. LangChain.js — ..."
console.log(result.steps.length); // number of actions taken
```
***
## How It Works
Playwright opens a Chromium browser (headless by default) and navigates to the start URL.
A PNG screenshot of the viewport is captured at CSS-pixel resolution
(same dimensions as the configured viewport). By default a hit-tested
accessibility tree is also extracted and sent alongside it
(`useDOM: true`).
The screenshot (and DOM tree if enabled) and task description are sent to a vision-capable LLM.
The model returns a structured JSON action: click at coordinates, type text, scroll, navigate, etc.
The action is executed via Playwright's browser API.
Steps 2-5 repeat until the model returns "done" (task complete) or "fail" (task impossible), or the max step limit is reached.
***
## BrowserAgentConfig
```typescript theme={null}
const agent = new BrowserAgent(config: BrowserAgentConfig);
```
Name of the browser agent.
Vision-capable model. Must support image inputs (e.g., `openai("gpt-4o")`, `google("gemini-2.5-flash")`).
Extra instructions appended to the system prompt. Use for task-specific guidance.
Maximum number of vision loop iterations before the agent gives up.
Run browser without a visible window. Set to `false` for debugging and demos.
Browser viewport size in pixels. The model sees screenshots at this resolution.
Initial URL to navigate to before starting the task.
Milliseconds to wait after each action for the page to settle.
Max consecutive identical actions before the agent auto-fails (loop detection).
Include a simplified DOM/accessibility tree alongside the screenshot. Each
interactive element is tagged with its exact center coordinate and
hit-tested so the listed point is guaranteed to land on the labeled
element (not on an overlay). This dramatically improves click accuracy and
is on by default. Set to `false` only if you want a pure-vision flow or
need to save tokens on very simple pages.
Path to a Playwright storageState JSON file. Restores cookies, localStorage, and sessionStorage from a previous session. Use this to maintain login state across runs.
Enable video recording of the browser session. Pass `true` for the default directory (`./browser-videos`) or `{ dir: "/path" }` for a custom location.
Enable anti-bot-detection mode. Patches `navigator.webdriver`, spoofs plugins, languages, WebGL renderer, and more. Pass `true` for sensible defaults or a `StealthConfig` object for fine control (custom user-agent, locale, timezone, geolocation, proxy, `deviceScaleFactor`). `deviceScaleFactor` defaults to `1` — set to `2` only if your host display is actually Retina and you want sharper screenshots.
Simulate human-like behavior — variable typing speed, jittered click coordinates, Bézier mouse movement curves, random micro-pauses. Pass `true` for defaults or a `HumanizeConfig` for fine control.
Secure credential store. The LLM only sees named placeholders — real values are injected at execution time and scrubbed from all output.
Track vision model token usage and enforce budgets across browser runs. Each vision loop step (screenshot → LLM → action) records its token usage. The same tracker can be shared with text and voice agents for unified cost monitoring.
Secondary (usually cheaper) model used for the `extract` action and any
text-only sub-task. Pair `model: openai("gpt-4o")` with
`pageExtractionLLM: openai("gpt-4o-mini")` to dramatically cut cost on
scraping-heavy workloads. Falls back to `model` if not set.
Maximum number of actions the model can return in a single step. Setting
to 1 reverts to the v2.0.x "one action per step" behavior. Higher values
speed up form filling significantly.
Vision mode. `true` always sends a screenshot, `false` never does
(DOM-only operation), `"auto"` (default) sends a screenshot on the first
step, when `useDOM` is false, and whenever the model just used the
`screenshot` action. Significant token savings on text-heavy workloads.
Actions to run before the LLM loop starts — saves vision tokens on
boilerplate (cookie banners, login flow, scrolling to section). Same
schema as actions the model emits.
Retry budget for transient failures (invalid JSON from the model, action
exceptions, locator timeouts). Separate from `maxRepeats` which detects
the model emitting the same action over and over.
If the task string contains a URL and no explicit `startUrl` is set, the
agent navigates there before the first LLM call.
Additional instructions appended to the default system prompt. Alias for
`instructions` — both are concatenated.
Completely replace the default system prompt. The credentials and
response-format sections are still appended automatically so the runtime
contract holds. Most users should prefer `instructions` /
`extendSystemMessage` instead.
Allow the `evaluate` action to run arbitrary JavaScript in the page. Off
by default for safety — only enable when you trust the source of task
strings.
Restrict navigation to specific domains. Wildcard patterns supported:
`"example.com"`, `"*.example.com"`, `"http*://example.com"`, `"*"`.
Any `navigate` action to a non-matching URL throws.
Block navigation to specific domains. Same pattern format as
`allowedDomains`. When both are set, a URL must be in `allowedDomains`
AND not in `prohibitedDomains`.
Connect to an existing browser via Chrome DevTools Protocol instead of
launching one (`"http://localhost:9222"`). When set, `headless`,
`stealth.args`, `recordVideo`, etc. are ignored — the existing browser's
configuration is used. Closing the agent detaches without killing the
remote browser.
Custom tools the BrowserAgent itself can invoke during a run. The agent
emits `{ "action": "tool", "name": "", "args": {...} }` and the
runtime dispatches to the tool's `execute(args, ctx)`. Use for 2FA
codes, API calls, file I/O — anything the browser can't do alone.
Logging level: `"debug"`, `"info"`, `"warn"`, `"error"`, `"silent"`.
***
## run()
```typescript theme={null}
const result = await agent.run(task: string, opts?: BrowserRunOpts);
```
Natural language description of what the agent should do in the browser.
Override the config's `startUrl` for this run.
Per-run API key override for the vision model.
Path to save cookies/auth state after the run completes. Load it back on the next run via `storageState` in config.
### BrowserRunOutput
| Field | Type | Description |
| ----------------- | --------------- | ---------------------------------------------- |
| `result` | `string` | Final text result or failure reason |
| `success` | `boolean` | Whether the task completed successfully |
| `steps` | `BrowserStep[]` | Full action history with screenshots |
| `finalUrl` | `string` | URL at completion |
| `finalScreenshot` | `Buffer` | Last screenshot (PNG) |
| `durationMs` | `number` | Total time taken |
| `videoPath` | `string?` | Video file path (if `recordVideo` was enabled) |
***
## Available Actions
The model can choose from these actions at each step. Most actions accept
either an `index` (preferred — resolves against the DOM snapshot we showed
the model) or `x`/`y` coordinates (fallback).
### Element-targeted
| Action | Parameters | Description |
| ------------------ | ----------------------------------------------- | -------------------------------------------------------------------------------------------------------- |
| `click` | `index` \| `x`/`y` + `description` | Click an element. Resolution order: `index` → quoted phrase in `description` → coordinates. |
| `type` | `index` \| `x`/`y`, `text`, `clear?`, `submit?` | Focus an input and type. Defaults to clearing the field; set `submit: true` to press Enter after typing. |
| `scroll` | `direction`, `amount?` \| `index` | Scroll the page, or scroll an indexed element into view. |
| `dropdown_options` | `index` | Read the option list of a native `