Posts by David Limpus
UltraQuant on AMD Instinct: More Efficient Agentic Serving for Qwen3.8-MXFP4
- 25 September 2026
UltraQuant can raise decode throughput and cut latency under load without a measurable drop in reasoning accuracy. In this blog, we will show you how to bring that to long-context and agentic LLM serving on AMD Instinct GPUs with Qwen3.8-2.4T MXFP4. Once agent prompts reach hundreds of thousands of tokens, the key-value (KV) cache is the limiter and not compute. Multi-turn agents (long shared prefixes, short generations, high concurrency) hit this wall hardest.
4-bit KV Caching in LMCache: Offloading Quantized KV Beyond HBM for Context-Heavy Agents on AMD MI355X
- 28 August 2026
As agents carry ever-longer context from turn to turn, the KV cache becomes the resource that runs out first. Two techniques ease that pressure from different angles: KV quantization makes each cached token smaller, while a hierarchical cache like LMCache spills cold KV to CPU DRAM. The two have mostly been developed independently.
Productionizing TurboQuant on AMD GPUs for KV-Cache-Bound LLM Inference
- 11 June 2026
*The first three authors (Chakrabarti, Limpus, Rana) contributed equally to this work.