Posts tagged Benchmarking
Training EAGLE3 Speculative Decoding Draft Models on AMD Instinct MI350X with AMD Quark
- 27 October 2026
In a previous blog, we looked at the serving side of EAGLE3 speculative decoding for large language models on AMD Instinct™ GPUs: ROCm and vLLM backend enablement, and AMD Quark FP8 draft-model quantization. Serving is only half of the picture, though. Speculative decoding also depends on having a draft model worth serving. A serving stack can only accelerate a target model as far as the draft model allows.
Accelerating Large-Scale LLM Inference on AMD Instinct MI350X/MI355X with Eagle3 and AMD Quark
- 03 July 2026
Large language model (LLM) inference is increasingly constrained by autoregressive decoding. Even when prefill is highly optimized, the decode phase still generates tokens one step at a time, and each step typically requires running the full target model. For large mixture-of-experts and attention-heavy models such as Kimi-K2.5 and MiniMax-M2.5, this sequential pattern limits serving throughput and increases latency for real-time applications.
Benchmarking Reasoning Models: From Tokens to Answers
- 24 July 2025
This blog shows you how to benchmark large language models’ reasoning tasks by distinguishing between mere token generation and genuine problem-solving. You will learn the importance of configuring models like Qwen3 with “thinking mode” enabled, how standard benchmarks can produce misleading results, why reasoning requires more than just generating tokens quickly, and how to build evaluations that reflect the model’s true problem-solving capabilities. Sounds interesting? Let’s dive right in!