Performance Evaluation - Agentium

Overview

PerformanceEval measures runtime performance metrics against configurable limits.

Quick Start

Install @agentium/core@4.0.0, @agentium/eval@4.0.0, and the optional openai SDK. Set OPENAI_API_KEY; this example makes live provider calls. For a no-key runnable project, start with the quality-gate recipe.

import { PerformanceEval } from "@agentium/eval";
import { Agent, openai } from "@agentium/core";

const agent = new Agent({ name: "fast-bot", model: openai(process.env.OPENAI_FAST_MODEL ?? "gpt-6-luna") });

const evaluator = new PerformanceEval({
  name: "latency-test",
  agent,
  maxDurationMs: 5000,
  maxTimeToFirstTokenMs: 1000,
  maxTokens: 500,
  cases: [
    { name: "greeting", input: "Hello!" },
    { name: "complex", input: "Explain quantum computing" },
  ],
});

try {
  const result = await evaluator.run();
  console.log(`${result.passed}/${result.total} passed`);
  if (result.failed > 0) process.exitCode = 1;
} finally {
  await agent.close();
}

Metrics Tracked

Metric	Config	Description
Duration	`maxDurationMs`	Total wall-clock time
TTFT	`maxTimeToFirstTokenMs`	Time to first token
Tokens	`maxTokens`	Total token consumption
Memory	—	Heap memory delta (always tracked)