Posts by Pengzhan Zhao
Attention Decode on AMD MI450 GPUs: A Gluon Kernel Optimization Guide
- 27 July 2026
Agentic AI applications are pushing LLM inference into a new regime. A request can now reach one million tokens from aggregated long prompts, tool calls, retrieval results, and multi-turn reasoning. During the text generation phase, each new token must attend to all previous tokens from the KV cache, so the kernel needs to repeatedly read past states from HBM. As a result, the performance bottleneck shifts from compute units to the memory system.