Reviewed corrections and regression checks

A correction is application evidence about a previous answer. Store what was wrong, the reviewed replacement, and the original input. Retrieval can supply that evidence to later runs; it does not guarantee the model will never repeat the mistake. These are focused host functions. Your application owns authentication, an Agent configured with memory.corrections, a scoped storage/vector backend, and the embedding dependency. See correction configuration before wiring a review UI.

Record a reviewed correction

The host passes verified ownership and the run being reviewed. This fictional invoice example records a field-level mapping with explicit user scope. Validate and authorize a real correction before calling this function; the values below are demonstration data.

import { Agent } from "@agentium/core";

type ReviewOwner = { userId: string; tenantId: string };

export async function recordChargeCorrection(
  agent: Agent,
  owner: ReviewOwner,
  runId: string,
  originalInput: string,
) {
  const memory = agent.memory;
  if (!memory?.getCorrectionStore()) throw new Error("Configure memory.corrections first");
  return memory.recordCorrection({
    agentName: agent.name, runId, originalInput,
    scope: "user", userId: owner.userId, tenantId: owner.tenantId,
    field: "chargeCode", originalValue: "THC", correctedValue: "DTHC",
    reason: "Reviewed convention: this fictional vendor uses THC for destination handling.",
    entityKey: "demo-vendor",
  });
}

Verify a correction ID is returned and the stored record has the expected scope. If your workflow intends tenant-wide or agent-wide knowledge, choose that scope deliberately. The default correction scope is agent scope; do not assume omission means private user memory. An HTTP review endpoint must verify the caller may review the referenced run and may write the selected knowledge scope. A request containing a runId or userId does not establish that authority. Follow hosted authentication, not a local router example, for user-facing access.

Build scoped regression cases

Host function: pass the Agent and an already authorized owner. The host must also have permission to read this correction store. This example selects that owner’s user-scoped records and evaluates each original input in a fresh session with the same identity.

import { Agent } from "@agentium/core";
import { custom, EvalSuite } from "@agentium/eval";

export async function evaluateChargeCorrections(
  agent: Agent,
  owner: { userId: string; tenantId: string },
) {
  const store = agent.memory?.getCorrectionStore();
  if (!store) throw new Error("Configure memory.corrections first");
  const records = await store.listCorrections({ agentName: agent.name });
  const cases = records.flatMap((record) => {
    if (record.scope !== "user" || record.userId !== owner.userId ||
        record.tenantId !== owner.tenantId || !record.originalInput || record.field !== "chargeCode") return [];
    return [{
      name: `correction-${record.id}`,
      input: `${record.originalInput}\nReply with only the corrected charge code.`,
      expected: record.correctedValue,
      runOpts: { ...owner },
    }];
  });
  if (cases.length === 0) throw new Error("No scoped charge-code corrections with original inputs");
  return new EvalSuite({
    name: "charge-correction-regression", agent, cases, concurrency: 1,
    scorers: [custom("corrected-code", (_input, output, expected) => {
      const pass = output.text.trim() === expected;
      return { score: pass ? 1 : 0, pass, reason: `Expected ${expected}` };
    })],
  }).run();
}

Inspect failed cases, then make your CLI fail when the suite’s failed count is nonzero. The quality-gate project shows reporting, process exit, and cleanup. This scorer fits a code-only answer; use schema/field validation when your real output is structured. A passing regression establishes behavior for those cases under the tested configuration. Retain held-out examples to check that a correction did not simply overfit one input. The caller closes the Agent after evaluation and other consumers settle.

Keep provenance and quality separate

Feature	What it contributes	What to verify
Learnings	Retrieved experience and provenance	Whether a supported source actually justifies the claim
Corrections	Reviewed replacement values and related evidence	Scope, reviewer authority, and retrieval relevance
Contradiction invalidation	Best-effort retirement of conflicting extracted learnings	Which records changed; similarity is not a proof of contradiction
Reflection	A model critique and bounded revision	Calibration against independent application assertions
Curator	Maintenance and reconciliation	Repair results and remaining store/index failures
Metrics	Counts of recorded corrections and critiques	Review coverage and denominator before interpreting accuracy

A correction rate is not automatically an accuracy rate: some mistakes are never reviewed, one answer may receive several corrections, and a review can arrive later. A model’s critique score is likewise not a calibrated confidence guarantee.

More patterns

Use memory patterns for preferences/history, evaluation patterns for assertions, and external Agent hosting for another execution framework. Use register: false and explicit cleanup for temporary Agents; registry names are not resource ownership.

Complete projects

Agents & memory

Models & Jev

Tools & knowledge

Teams & jobs

Deploy & devices

Quality & production

Reviewed corrections and regression checks

Record a reviewed correction

Build scoped regression cases

Keep provenance and quality separate

More patterns

​Record a reviewed correction

​Build scoped regression cases

​Keep provenance and quality separate

​More patterns

Record a reviewed correction

Build scoped regression cases

Keep provenance and quality separate

More patterns