AWS Open-Sources Infinity: Agents Suspend While Waiting, So Waiting Costs No Compute

What happened

In conventional agent frameworks, the loop usually blocks during a tool call, with the process and its memory held until the tool returns. Infinity treats a tool call as a suspension point, borrowing the stackless-coroutine idea from programming languages: the conversation history already holds everything needed to continue, so the runtime persists it, discards the rest of the in-memory state, and lets any process resume the agent when a message arrives.

In the serverless reference architecture, each run is a bounded slice: load, one model completion, persist, dispatch at most one tool call, end. Amazon Web Services gives the example of an agent waiting on a three-day continuous-integration pipeline: before the pipeline reports back, it needs no active Lambda invocation and consumes no compute while waiting.

Infinity comes from the Hydro project at Amazon Web Services and UC Berkeley, under the Apache 2.0 license, with code in the hydro-project/infinity repository on GitHub. The runtime ships with built-in spawn_thread and report_to_parent tools for agent-to-agent communication; child threads handle events by default so the parent agent's context window does not bloat with events; within model API constraints, synthetic tool calls deliver asynchronous events.

Key facts

1. Suspension mechanism: when an agent calls a tool, it saves its conversation history and frees memory, resuming when the tool result arrives; each run is a bounded slice of load, one model completion, persist, and at most one tool call.
2. Benchmark method: the test ran 80,000 agents on a single thread of a server-class AMD EPYC 9R14 against a simulated model, with each completion about 1.2KB and resident memory of about 100KB per idle agent. The headline figure of 75,000 agents in 8GB is arithmetic from that rate; the test itself did not run on a Raspberry Pi.
3. Open-source details: Infinity is under the Apache 2.0 license, part of the Hydro project, with code in the hydro-project/infinity repository on GitHub, announced on the AWS Open Source Blog on Oct 23, 2026.
4. Serverless reference architecture: combining AWS Lambda, SQS and Aurora DSQL, a waiting agent needs no active Lambda invocation; persisted state, queue and timer infrastructure, external tool execution and later model calls can still incur costs, and AWS gave no dollar-denominated cost comparison.
5. Open questions left by the authors: models are mostly trained on traces where tool calls return synchronously, so reasoning under interleaved events is untested; scheduling is first-come-first-served, giving user-facing threads and background event handlers equal priority; compaction for large parent-child agent trees is unstudied; and event authenticity still needs a verification method.