ExecutionGraphRuntime when a graph must retain per-node progress and reconcile interrupted work. The host supplies a versioned catalog of executable tools, Agents, nested graphs, and bounded repeats. A static plan or optional model planner selects those operations through JSON; a proposal cannot supply code or grants.
This guide describes Agentium 4.9.0. Review release and upgrade guidance, then start with the provider-free graph project for a complete executable example.
Choose the recovery boundary
The graph adapter reuses Workflow, Agent, ToolExecutor and the durable task protocol. Ordinary
workflowDriver remains non-durable. Recovery requires stateless JSON-compatible bindings; unsupported mutable or opaque behavior fails preflight.
Bind executable behavior on the host
GraphBinding has four variants. Every binding requires id, version, input and output schemas.
Choose exactly one of
program.plan or program.planner. A plan node names a catalog binding and exact version, with a JSON args object. The entire candidate is validated before its work can dispatch: identities, schemas, references, cycles, operation versions, bytes, nodes, depth, concurrency and supported recovery policy.
dependsOn provides control ordering. inputsFrom declares data dependencies. Argument references also infer a data dependency:
referenceInputs and referenceOutputs paths on bindings; "" denotes the whole value. Supply a pure validateTemplate callback for future-valued inputs. The runtime resolves references and validates concrete arguments before dispatch. Static argument templates refer to producer outputs; root program input informs planning.
Persist compatible task identity
Each task pins its manifest/catalog digest, program and binding versions, original input, policy revision, grants, budgets and absolute deadline. Plain plans, nested templates, limits, budgets and grants are copied at construction. Opaque schemas, provider/Agent objects and callback implementations remain host-attested by version. Bump versions when instructions, schemas, callable code or captured dependencies change. Use a verified persistentDurableTaskStore, such as MongoDBDurableTaskStore, and set requireDurable: true. The host owns credentials, database lifecycle, authenticated tenant/actor admission, atomic CAS, fencing and receipt retention. Default InMemoryDurableTaskStore advertises capabilities.durable: false; it is a local fixture and cannot survive process loss. requireDurable: true rejects it.
create() admits the task. run(key, workerId) acquires fenced ownership and executes or recovers it. inspect(key) validates retained state without claiming ownership or writing. cancel(key, reason) requests persisted cancellation observed by active lease polling. A completed task returns retained output without another dispatch. Generated graph reference contains complete signatures.
Recover nodes and effects
Materialized revisions and node incarnations have stable IDs. Inputs, outputs and attempts carry digests and worker fences. Completion persists before successors start. Completed nodes reuse their output; interrupted running work starts under a newer fence; failed or canceled work remains terminal. A changed descriptor or stale fence fails before resumed effects. Restricted Agent nodes restart from node entry. Complete recorded model responses replay at request-bound positions with stable tool-call IDs. Approved tools are reauthorized. Confirmed raw outputs and receipts reuse stable budget admission IDs without duplicate logical charges. Interrupted model reservations remain conservatively charged as unknown usage. Recovery replenishes neither budgets nor the original deadline. A tool must declarerecovery: "pure" or provide a versioned DurableActionConnector, host-bound destination, and immutable resolveResult(resultRef). pure is the host’s assertion that the callback performs no external mutation. The SDK cannot prove that assertion.
For effects, action identity binds node incarnation, call, binding/version, actual parsed arguments, destination, policy/grants and caller identity. The declared connector receives those arguments; the arbitrary tool callback is not substituted for it. Append-only origin evidence selects the originating catalog operation even when two connectors share destination/version.
Root admission occurs before remote dispatch is recorded as executing. Local admission rejection remains known not dispatched. The permit covers dispatch and receipt settlement; confirmed replay and reconciliation need no fresh dispatch allowance.
If a destination accepts an effect and the process dies before the receipt persists, recovery needs confirmed reconciliation, proven absence or destination idempotency. Missing or transient lookup evidence cannot justify blind redispatch. An unresolved outcome raises DurableRecoveryRequiredError. Fencing prevents stale local settlement but cannot retract physical remote work. Exactly-once external execution and remote rollback are not guaranteed. See recovery guidance.
Result caches, toModelOutput, dynamic Agent tool resolvers, handoffs, mutable memory, hooks and unsupported opaque continuation fail preflight. Recovery uses stateless harness configuration; ordinary session/continuity policies remain available through HarnessRuntime.
Bound all controlled work at the root
One root slot covers an actual provider request/stream or controlled tool/function callback. Graph, repeat and Agent containers hold no slot while waiting for children, so capacity one can complete without a parent-child deadlock. Agent tools, nested graphs, repeats and planning share the controller. Nested controlled dispatch from inside a leaf is rejected. FIFO admission checks cancellation and deadline.
Set explicit ceilings for your application. Model/tool/token budgets apply in addition to root limits. Expansion, attempt and revision counters persist before dispatch; removed nodes and interrupted reservations do not refund allowance. Provider transport retries retain their existing owner: one controlled provider invocation is one root work attempt. Queue/admission/release/exhaustion events are available through
onAdmission and controller snapshots.
Already-owned stream cleanup may finish after cancellation through the lease’s cleanup path, without admitting more work. A permit releases only after the owned operation settles. Receipt lookup and standalone storage/reconciliation do not occupy dispatch slots. Providers/connectors must honor cooperative cancellation and timeout; an SDK lease cannot preempt accepted remote work.
Add bounded model planning
Replace the static plan withprogram.planner containing provider, hostInstructions, maxCorrections, maxOutputTokens, and stream bounds. mode is "complete" by default or "stream". Both paths are tool-free, use authorized model ports and share root admission and harness budgets.
Host instructions retain system authority. Catalog descriptions, inputs, retrieved/tool content and observations are source data. Model text can propose an operation; validation and grants determine whether it may execute. Accepted/rejected proposals and causal diagnostics remain in the journal. Correction and fresh-call ownership stay bounded across recovery. Permanent exhaustion fails with the originating validation cause.
Plan defaults are maxBytes: 128000, maxNodes: 100, maxDepth: 32, and maxConcurrency: 5; narrow them with limits.plan. maxDepth bounds cumulative dependency and nesting depth, including dynamically planned children. Repeat iterations share nesting depth and consume lifetime expansion/revision allowances.
Stream complete pending revisions
Streaming accepts newline-terminated, self-contained frames with{ baseRevision, plan }:
maxFrameBytes, cumulative maxTotalBytes/maxFrames, output-token and correction limits. maxLatencyMs defaults to 30 seconds per planning call. All apply alongside root limits and the original deadline.