Skip to main content

Capacity Planner

Combines KV estimator + hardware config + workload mix into actionable capacity numbers.

planCapacity(arch, hardware, workload, kvPrecision?, weightPrecision?)

The all-in-one function. Returns a complete CapacityPlan with memory breakdown, session counts, latency estimates, and cost.

maxConcurrentSessions(arch, hardware, avgContextTokens, kvPrecision?, weightPrecision?)

Compute how many sessions fit in HBM and on NAND for a given average context size.

estimateGpuCount(arch, targetUsers, avgContextTokens, kvPrecision?, weightPrecision?, gpu?)

Solve for the minimum GPU count needed to serve N concurrent users (HBM-only).

compareConfigs(arch, configs, workload, kvPrecision?, weightPrecision?)

Compare multiple hardware configurations side-by-side.