Posts by Haonan Wang

Keep the Agent Warm: How ATOM Serves AgentX Workloads on AMD Instinct MI355X GPUs

A coding agent makes many model calls in one session, and each one re-sends almost everything before it: one recorded session makes 198 calls, and call #48 sends 195.5K tokens, only 1.3K of them new. Agentic workloads combine long shared histories and short outputs, interrupted by tool calls and parallel sub-agents. Serving them well takes more than raw GPU compute: it means keeping the agent warm, with its history cached and its decode moving when new requests arrive.

Read more ...