Skip to main content
Start by naming the failure you need to survive. Different mechanisms preserve different things.

Consider a refund timeout

The tool sends a refund request. The payment service accepts it. Your connection closes before the receipt arrives. The local exception cannot tell you whether the refund happened. A second request may duplicate the effect. Instead, use a stable business idempotency key and query the provider for authoritative evidence. Retain that receipt with the application operation. A durable action ledger can coordinate this decision only when the connector supplies a valid reconciliation contract.

Choose what to persist

  • Input references: immutable inputs or retained references, with ownership and integrity checks.
  • Action identity: deterministic IDs and canonical arguments; the same ID must not mean a different action on replay.
  • Approval: the exact prepared action, reviewer identity, and decision.
  • Outcome evidence: a provider receipt or other authoritative result, not only a local log line.
  • Policy and budgets: what was admitted and what must be rechecked before resumed work.
An in-memory durable store is useful for local tests but advertises that it is not durable. See the store and supervisor guide for persisted execution.

Exercise the failure, not only the happy path

Test a crash before dispatch, a crash after dispatch but before acknowledgment, duplicate delivery, lease loss, approval timeout, and cancellation with an in-flight action. Document whether each case resumes, waits for reconciliation, or requires an operator. Cancellation is a request to stop work. It cannot retract an effect already accepted by another system. Queue migration also needs an explicit procedure; follow the BullMQ migration guide before changing persisted scheduler formats.

Rehearse an unknown outcome

Use a controllable local connector before testing a real payment or notification. The following is a failure exercise, not a promise that a standard harness driver can replay effects. Run the exercise once with the failure before dispatch and once after acceptance. Verify that the provider fixture records one effect for the operation ID. Then test lease loss and duplicate delivery. Durable driver contracts describe the store, driver, and connector responsibilities; queue durable delivery adds wakeups around them.