Thaw – LLM Agent Branching & Forking

LLM agents waste compute by re-running prefill across multiple exploration branches (rollouts, parallel attempts, best-of-N). Thaw snapshots a live inference session and forks it without re-prefilling, dramatically reducing costs. Target: AI labs, enterprises running multi-branch agent workflows.

KV cache reuse and inference optimization are among the hottest cost-reduction levers in production AI right now, as enterprise inference bills scale faster than anyone budgeted for — making this technically well-timed. The closest substitute is vLLM's prefix caching, which handles overlapping prefixes statically but doesn't support dynamic session forking mid-inference; no commercial product owns this specific niche yet. The $5k–$50k/mo band is plausible but probably undersells the ceiling — a single AI lab running large-scale MCTS or best-of-N rollouts could justify five-figure monthly contracts on compute savings alone, so the real question is whether pricing gets structured as infrastructure licensing or usage-based. The biggest risk is that the major inference providers (Together, Fireworks, Anyscale, and eventually the hyperscalers) absorb this as a native feature before a standalone vendor can establish enough customer lock-in to survive.