Posts by Lei Zhang antiagainst
Two Waves, One Stage Apart: Warp-Pipelining Triton GEMMs on AMD Instinct
- 27 October 2026
A tiled general matrix multiplication (GEMM) computes on one pair of operand tiles while preparing the next. On a compute-bound shape, keeping the matrix units busy depends on how well those activities overlap. A common failure mode is that resident waves reach synchronization points in phase: both prepare data or wait, and neither has matrix work ready to issue. Extra buffering creates room for future tiles, but does not by itself establish the execution timing needed to hide that preparation.
Attention Decode on AMD MI450 GPUs: A Gluon Kernel Optimization Guide
- 27 July 2026
Agentic AI applications are pushing LLM inference into a new regime. A request can now reach one million tokens from aggregated long prompts, tool calls, retrieval results, and multi-turn reasoning. During the text generation phase, each new token must attend to all previous tokens from the KV cache, so the kernel needs to repeatedly read past states from HBM. As a result, the performance bottleneck shifts from compute units to the memory system.
Unleash Full GPU Potential: Overlap Communication and Computation with Triton-Distributed
- 06 May 2025
In distributed computing, AI workloads demand both massive parallelism and efficient data movement. A primary challenge lies in effectively overlapping computation with communication to maximize performance. GPUs are excellent at crunching numbers. However, their full potential often remains untapped due to relatively long inter-GPU communication. This results in their computing units staying idle for large amounts of time while waiting for data transfer from other nodes. In this blog, we will show how you can use the Triton-Distributed framework to generate kernels that overlap communication and computation, resulting in performance that can rival highly optimized libraries.