Posts by Hai Xiao
GLM-5.2 AgentX on AMD Instinct MI355X: Boosting Throughput and Interactivity
- 04 November 2026
Coding agents send long, growing prompts, reuse earlier context, and run parallel tasks, which puts heavy pressure on an inference server. In this blog you will learn how we used the InferenceX AgentX benchmark to improve GLM-5.2-MXFP4 serving on AMD Instinct MI355X GPUs. In this model, MXFP4 is a microscaling 4-bit floating-point weight format; SGLang is an open-source LLM serving framework. Cache efficiency, faster decode kernels, careful AgentX validation, and upstream collaboration raised peak throughput 1.72× and peak P90 interactivity 2.46×.
FlyDSL Mono Kernels for High-Interactivity LLM Serving in ATOM
- 03 November 2026
LLM applications with strict latency requirements need high interactivity. A coding agent needs to generate its next action quickly so it can call a tool and continue the task. By reducing decode latency, you can keep this loop moving.
Efficiently Serving NVFP4 Models on AMD Instinct™ MI350X/MI355X Accelerators via Online NVFP4 to Quark MXFP4 Requantization
- 08 September 2026
A growing share of frontier open-weight models are available in NVFP4 format, but on some AMD GPUs (MI350X / MI355X), FP4 compute support is limited to MXFP4. This means NVFP4 checkpoints couldn’t run on the native 4-bit acceleration path these GPUs provide.
Supercharge DeepSeek-R1 Inference on AMD Instinct MI300X
- 21 March 2025
Our previous blog post on this topic discussed how DeepSeek-R1 achieves competitive performance on AMD Instinct™ MI300X GPUs. We also included performance comparisons against Nvidia H200 GPUs and a short demo application illustrating real-world usage. In this blog we will delve into how using the SGLang framework, critical kernel optimizations like AI Tensor Engine for ROCm™, and hyperparameter tuning helps to achieve performance boosts.
SGLang: Fast Serving Framework for Large Language and Vision-Language Models on AMD Instinct GPUs
- 13 November 2024
In the rapidly evolving landscape of artificial intelligence, the ability to deploy large language models (LLMs) and vision-language models (VLMs) efficiently is crucial for real-time applications. SGLang is an open-source framework designed to meet these demands by delivering fast backend runtime, a flexible frontend language, and extensive model support for a variety of LLMs and VLMs.