Posts by Haichen Zhang
Training EAGLE3 Speculative Decoding Draft Models on AMD Instinct MI350X with AMD Quark
- 27 October 2026
In a previous blog, we looked at the serving side of EAGLE3 speculative decoding for large language models on AMD Instinct™ GPUs: ROCm and vLLM backend enablement, and AMD Quark FP8 draft-model quantization. Serving is only half of the picture, though. Speculative decoding also depends on having a draft model worth serving. A serving stack can only accelerate a target model as far as the draft model allows.
Accelerating Large-Scale LLM Inference on AMD Instinct MI350X/MI355X with Eagle3 and AMD Quark
- 03 July 2026
Large language model (LLM) inference is increasingly constrained by autoregressive decoding. Even when prefill is highly optimized, the decode phase still generates tokens one step at a time, and each step typically requires running the full target model. For large mixture-of-experts and attention-heavy models such as Kimi-K2.5 and MiniMax-M2.5, this sequential pattern limits serving throughput and increases latency for real-time applications.