Posted in 2026
Scaling 3D Gaussian Splatting with Distributed Compute: Why One GPU Is Never Enough
- 23 October 2026
gsplat is a GPU-optimized Python library for training and rendering 3D Gaussian Splatting (3DGS) models. In earlier posts we showed how to train gsplat scenes on AMD GPUs, took a closer look at the mathematics of 3DGS, and applied gsplat to autonomous driving.
FP8 GEMM Optimization on AMD RDNA™ 4 Architecture
- 23 October 2026
In this blog you will learn how to port an FP16 GEMM kernel to FP8 on AMD RDNA™ 4 GPUs. You will see what the FP8 WMMA instruction changes, how the smaller element size lets larger tiles and deeper K slices fit in LDS, and how four tile configurations compare against hipBLASLt on GEMM shapes from transformer workloads.
Optimizing World Action Models on AMD GPUs
- 21 October 2026
World Action Models (WAMs) offer a promising route to more physically grounded robot policies, but their video-model backbones make inference expensive. In this blog you will learn how to remove much of that cost on AMD GPUs without sacrificing task success, using two representative WAMs, FastWAM [1] and DiT4DiT [2].
MoE Training Optimization with Primus
- 21 October 2026
Mixture-of-Experts (MoE) has become a dominant architecture for frontier-scale language models, and our users increasingly train large MoE models on AMD Instinct™ MI355X[a] GPUs. Driven by those workload requirements, we have built a broad set of MoE training optimizations in Primus. This blog follows our earlier post MoE Training Best Practices on AMD GPUs. The earlier post established the foundational optimizations; this one describes what we built on top of them.
Micro-World 2: Streaming Real-time Interactive World Model
- 21 October 2026
Diffusion models have advanced rapidly in recent years, enabling high-quality video generation across a wide range of creative scenarios. As visual quality continues to improve, however, the field is increasingly shifting its focus from passive content generation toward controllable and interactive experiences. YUME 1.5 [1] explores controllable video generation through explicit interaction signals, while LingBot-World [2] and Sana-WM [3] investigate world models that generate future observations conditioned on the current environment and user actions. World Labs’ Marble [27] and Atlas [28] represent a complementary spatial-modeling direction: Marble creates editable, explicit 3D worlds from multimodal inputs, while Atlas organizes text, images, camera poses, and depth into a shared spatial context for generation, reconstruction, and simulation. Together, these efforts illustrate how generative models are evolving from passive content-synthesis tools toward controllable, persistent, and interactive world simulators.
hipCIM 26.06: GPU-Accelerated Whole-Slide Image I/O on AMD Instinct™ GPUs
- 20 October 2026
Every whole-slide image tells a story in pixels. A whole-slide image is a glass pathology slide scanned into a single very large digital image, stored as a pyramid of small tiles so a viewer can pan and zoom it. A single biopsy digitized at 20× magnification can span tens of thousands of pixels on each side and weigh in at several gigabytes. Before a model can learn anything from that slide, something has to read it off disk, decode its thousands of small JPEG-compressed tiles, and turn them into tensors the accelerator can work on. In digital pathology, that unglamorous first step is often where the wall-clock time quietly disappears.
Introducing rocOpt: GPU-Accelerated Decision Optimization on AMD ROCm
- 20 October 2026
rocOpt is a new open-source, GPU-accelerated engine for decision optimization on AMD Instinct GPUs. It solves large-scale linear programming (LP), mixed-integer linear programming (MILP), quadratic programming (QP), and vehicle routing problems on the ROCm software stack, and is ready for early users today on MI300X and MI355X with ROCm 7.2.3.
Bringing Ultra-Fast, Interactive LLM Inference to AMD Instinct™ MI355X GPUs
- 20 October 2026
AI inference is no longer just a throughput problem. As reasoning models, AI agents and interactive applications become mainstream, responsiveness has become a first-class metric. Users notice seconds, not tokens per second. Lowering time per output token (TPOT) is now critical for AI systems that feel conversational, responsive and useful.
FP16 GEMM Optimization on AMD RDNA™ 3 and RDNA™ 4 Architectures
- 16 October 2026
In this blog you will learn how to write a fast FP16 GEMM kernel for AMD RDNA™ 3 and RDNA™ 4 GPUs, one optimization at a time. You will see which techniques pay off on each architecture, why some help one and hurt the other, and how to measure the effect of every change in time, registers, and LDS, so you can apply the same approach to your own kernels. This is Part 1 of a two-part series. Part 2 will port the final kernel from this post to FP8 on RDNA 4.
Long Context, One Node: Serving GLM-5.3-FP8 on MI300X with DP8+EP8
- 14 October 2026
Serving a frontier Mixture-of-Experts model like GLM-5.3-FP8 at long context and high concurrency is a demanding workload. The model has 744 billion total parameters, with approximately 39 billion active per forward pass, and a native context limit of 1,048,576 tokens, as specified by max_position_embeddings in config.json. The most demanding serving regime combines long prompts, long generations, and many concurrent requests. That combination stresses both GPU memory and inter-GPU communication, and the layout you pick to spread the model across a node largely determines how much throughput you can sustain under that load.
Pre-Training Hybrid LLMs from Scratch in Primus on AMD Instinct™ GPUs
- 08 October 2026
In this blog you will learn how to pre-train hybrid LLMs from scratch with Primus on AMD Instinct™ MI355X GPUs. In particular, we will cover how to declare a hybrid layer stack with a single configuration string, how three linear-recurrent mixers (Mamba2, Gated DeltaNet, and Kimi Delta Attention) trade off throughput with training quality, which recipe settings determine whether your run fits in memory in the first place, and which environment variables can quietly change your results. Along the way you will receive the released configurations behind every number in this post, so that you can reproduce the runs yourself. Ready to train your own hybrid? Pull the Primus image and follow along — everything here runs on a single eight-GPU node.
DI Series: Serving Kimi-K3 MXFP4 Wide-EP Disaggregated on AMD Instinct MI300X and MI325X
- 08 October 2026
This post is part of the AMD Disaggregated Inference (DI) Blogs Series, which takes frontier open models from a single-node demo to production, multi-node serving on AMD Instinct™ GPUs
AMD Instinct MI455X vs MI355X: A Technical Look at the Advancing AI 2026 Inference Numbers
- 08 October 2026
At Advancing AI 2026, AMD shared a set of performance results comparing the new AMD Instinct™ MI455X (Helios) accelerator against the current-generation AMD Instinct™ MI355X. This post explains what went into those numbers: the workloads, the kernels, the test configuration, the metric definitions, and the caveats that sit behind each figure.
Completing the GPU Performance Picture: Understanding TAF Alongside Peak FLOPs and MAF
- 06 October 2026
Modern AI accelerators can execute an extraordinary number of floating-point operations every second. But a single FLOPs number rarely tells the full performance story. In the first two blogs in this series, we discussed Peak FLOPs and Max-Achievable FLOPs (MAF) for AMD Instinct™ GPUs., we discussed Peak FLOPs and Max-Achievable FLOPs (MAF) for AMD Instinct™ GPUs. Peak FLOPs describe the theoretical compute capability of the hardware. MAF goes a step further by measuring the highest sustained throughput the GPU can achieve after testing a range of GEMM sizes. This blog introduces a third metric: Typical Attained FLOPs (TAF). TAF measures FLOP-weighted sustained throughput across a fixed representative GEMM set selected before testing. The goal is not to replace Peak FLOPs or MAF. Each metric answers a different question, and together they provide a more complete view of GPU compute performance.
ROCm 10.1: Breaking the Data-Movement Bottleneck
- 05 October 2026
What limits your training run today: how fast the GPU computes, or how fast you can get data to it? For a growing number of workloads, the answer is the latter. ROCm 10.1 leads with that path, and carries a full slate of compiler, library, and tooling work behind it.
Running MolmoAct2 Robot Policy on an AMD Ryzen AI Max+ 395 Mini PC
- 30 September 2026
Teaching a robot to “put the bowl on the plate and place the cream cheese inside the bowl” used to require a rack of data-center GPUs sitting behind the robot. In this blog we show how a modern vision-language-action (VLA) model, MolmoAct2, can look at a scene, read a plain-English instruction, and drive a robot arm to complete the task, all running on a single AMD Ryzen AI Max+ 395 “Strix Halo” mini PC with the ROCm software stack. No cloud, no external accelerator, just a compact tabletop machine.
Automating Performance Bottleneck Identification with the TraceLens Agent
- 30 September 2026
GPUs are valuable resources, and it is imperative to make training and inference workloads on them as efficient as possible. Bottlenecks in these workloads can arise from multiple sources, and mitigations vary depending on the source. For example, a kernel may execute well below its achievable throughput, a collective may remain exposed on the critical path, or the device may sit idle while waiting on host-side dispatch. Identifying these bottlenecks is the first step to improving end-to-end performance.
AIM: Unified User Experience from Profile Discovery to Deployment
- 30 September 2026
This post is a continuation of the Multi-Accelerator Support for AIMs and AMD Solution Blueprints blog, which introduced (i) multi-accelerator coverage across AMD Instinct™ GPUs, Radeon™ PRO GPUs, and EPYC™ CPUs, and (ii) walked through the Document Summarization Solution Blueprint. Here we focus on AIMs specifically and show how they deliver a unified user experience from Radeon to Instinct, i.e., similar profile discovery commands, container workflow, and OpenAI-compatible API regardless of the accelerator.
Unlocking Sim2Real for a Robotic Arm with RL Accelerated by AMD Instinct GPUs
- 29 September 2026
Training a policy in simulation and transferring those weights to a real robot (sim2real) is a highly desirable goal in physical AI. Simulation scales data collection far beyond what a single robot can provide, but mismatches in physics and in the software stack make transfer difficult. In this blog post, we train a pick-and-lift policy for a UFactory XArm-6 with reinforcement learning on AMD Instinct GPUs, then deploy the same network on the real arm with a classical perception front-end in place of privileged simulator state.
Verl 0.9.0 on ROCm™ 10.0: Next-Generation RL Post-Training on AMD Instinct™ GPUs
- 28 September 2026
In Scaling RL with verl on AMD Instinct™ MI355X, we walked through verl’s Fully Async Policy trainer on AMD Instinct™ MI355X GPUs. That post used verl 0.7.1.amd0 on ROCm™ 7.0.2, with a 4+4 GPU split for GRPO on Qwen2.5-VL-7B (Megatron) and DAPO on Qwen2.5-Math-7B (FSDP2), plus a separate synchronous throughput comparison against NVIDIA B300.
UltraQuant on AMD Instinct: More Efficient Agentic Serving for Qwen3.8-MXFP4
- 25 September 2026
UltraQuant can raise decode throughput and cut latency under load without a measurable drop in reasoning accuracy. In this blog, we will show you how to bring that to long-context and agentic LLM serving on AMD Instinct GPUs with Qwen3.8-2.4T MXFP4. Once agent prompts reach hundreds of thousands of tokens, the key-value (KV) cache is the limiter and not compute. Multi-turn agents (long shared prefixes, short generations, high concurrency) hit this wall hardest.
Model Weight Profiles: Where Do the Parameters Go?
- 25 September 2026
This blog describes model weight profiles, which show how many model parameters are devoted to embeddings, attention, dense layers, and other components, and in which number formats. That split can be used to calculate the total model size, the savings from a quantization recipe, and the size of a runtime cache such as the KV cache.
Local Quantization and Multi-Backend Deployment with AMD Quark on Strix Halo
- 25 September 2026
For local AI, running a model on the target device is only part of the deployment workflow. Model preparation, including quantization and export, is another important step. AMD Quark provides memory-efficient quantization workflows that make it possible to optimize large models directly on platforms such as AMD Strix Halo.
Debugging Logprob Mismatches in LLM Reinforcement Learning
- 23 September 2026
In online reinforcement learning (RL), a language model generates responses, receives rewards, and updates its weights to improve future responses. Many systems split this loop between a rollout engine, which generates the responses, and a trainer, which learns from them. Both engines evaluate token probabilities: rollout records the log-probability, or logprob, of each token it generates, while the trainer evaluates those same tokens when computing its loss. After the update, the new weights are sent back to rollout, and the loop begins again.
Zebra-HyLo: Upcycling Transformers into Long-Context Hybrid LLMs on AMD Instinct™ GPUs
- 22 September 2026
Hybrid language models that interleave attention with linear sequence-modeling blocks have become the default answer to the cost of long context. Jamba, Samba, Qwen3-Next and Kimi-Linear all take this shape, and they all share one property that makes them expensive to adopt: they are pretrained from scratch. Every one of them pays the full cost of building a foundation model again, which means the enormous investment already sunk into existing Transformer checkpoints is thrown away.
Performance Profiling on AMD GPUs - Part 6: Advanced Thread Trace (ATT) - The Microscope for Your Application
- 22 September 2026
This post is the sixth part of the Performance Profiling on AMD GPUs series, which works through the
ROCm profiling tools one layer at a time. Part 1 laid the
foundations and introduced the profiling tools available for AMD GPUs,
Parts 2 and 3 walked through
basic and
advanced
profiling workflows with rocprofv3, rocprof-sys, and rocprof-compute, Part 4 applied those
workflows to a
Fortran OpenMP offload application,
and Part 5 used the resulting profiles to drive an
AI-assisted kernel optimization loop.
Those parts tell us which kernel to look at and how it uses the hardware. This part goes one level
below them: Advanced Thread Trace (ATT) records what the wavefronts of a single kernel do
instruction by instruction, so we can explain why that kernel performs the way it does.
Benchmarking Kimi-K3 Across vLLM, SGLang, and ATOM on MI350X
- 22 September 2026
Moonshot AI released the weights for Kimi-K3, a 2.8-trillion-parameter, 1M-context, natively-MXFP4 Mixture-of-Experts (MoE) model, and AMD Instinct™ support was available on day 0 across three independent serving frameworks: vLLM, SGLang, and ATOM. The recipes target the gfx950 generation (MI350X and MI355X). The measurements in this post were taken on an 8× MI350X node.
Serving GLM-5.2-MXFP4 on AMD Instinct™ MI355X: When Prefill Context Parallelism Pays
- 21 September 2026
Whether prefill context parallelism pays depends on prompt length. It divides the prompt across GPUs during prefill; on four AMD Instinct™ MI355X GPUs (gfx950) serving GLM-5.2-MXFP4 under SGLang at --tp-size 4, enabling it yields 43% to 54% more total throughput on 40,960- and 61,440-token prompts at concurrency 16 and above, and a first token about 1.5× to 2.3× faster at every concurrency. On 1024-token prompts it loses at every concurrency measured, by 8% to 18% of total throughput.
Hyperloom: A Multi-Agent Harness for Autonomous Inference Optimization on AMD GPUs
- 21 September 2026
ROCm™ Hyperloom delivered a median 1.73× inference speedup in extensive unattended evaluation on AMD Instinct™ GPUs, with gains ranging from 1.35× to 7.31×. It profiles each workload, searches framework and kernel optimizations, validates changes end to end, and carries proven results forward, without per-model human tuning.
Technical Dive into AMD MLPerf Inference v6.1 Submission
- 17 September 2026
MLPerf Inference v6.1 results were released on September 16, 2026. For AMD, this was a highly successful round in which we achieved several leadership scores, expanded benchmark coverage, introduced a new GPU with validated performance, and enabled record-setting results from our partners. In this round, AMD and its partners provided validated results across a diverse set of workloads, including dlrm-v3 (Deep Learning Recommendation Model v3), llama2-70b (Meta’s 70-billion-parameter language model), gpt-oss-120b (an open-weight 120-billion-parameter Mixture-of-Experts model), deepseek-r1 (DeepseekAI’s 671-billion-parameter Mixture-of-Experts model), and Wan 2.2-t2v (a 14-billion-parameter text-to-video generation model). Single-node results used 8 GPUs per node; gpt-oss-120b was additionally submitted at multi-node scale across 72 AMD Instinct™ MI355X GPUs. All results were validated through MLPerf peer review.
Reproducing AMD MLPerf Inference v6.1 Submission Results
- 17 September 2026
This blog shows you how to reproduce AMD submission results for MLPerf Inference v6.1 on AMD Instinct MI355X, MI350X, and MI350P GPUs using self-contained Docker images, publicly available quantized model weights, and a step-by-step benchmark recipe.
Implementing a High-Performance Custom Diffusion Attention Kernel with FlyDSL
- 17 September 2026
Readers may be familiar with traditional Transformer models and their attention mechanisms. The traditional autoregressive transformers generate tokens iteratively. Since this feature significantly limits the inference throughput, researchers have begun exploring approaches such as diffusion models that can generate multiple tokens in each iteration.
DFlash Speculative Decoding on AMD Instinct MI355X: Up to 5× Faster Qwen3.5 Inference
- 16 September 2026
Autoregressive decode generates one token per model forward pass, so single-request latency is bounded by how fast you can stream weights through the GPU — not by compute. Speculative decoding attacks that bottleneck by letting a small draft model propose several tokens that the large target model then verifies in a single parallel pass. In this post we bring DFlash [3] — a block-diffusion drafter — to AMD Instinct MI355X through vLLM [4] on ROCm [5], benchmark it head-to-head against Qwen3.5’s [6] built-in MTP drafter, and then quantize the target model to mxfp4 to show that speculation and quantization stack.
Thread Trace Part 1: ROCprof Compute Viewer
- 15 September 2026
Once you have identified a GPU kernel to optimize, you need to understand how its instructions execute and where waves spend time waiting. Thread trace gives you a detailed view of that behavior, helping you investigate stalls, gaps in execution, and competition for hardware resources.
Knowledge Graph Integration With Poro2 For Enriching Medical Text Processing
- 15 September 2026
Medical documentation contains specialized terminology, such as terms like “hepatomegalia” or “bilateraalinen pleuraeffuusio”, that healthcare professionals understand but patients struggle to comprehend. Making these documents accessible to patients improves healthcare outcomes, yet general-purpose LLMs often lack the specialized vocabulary needed for accurate simplifications.
An Educational GEMM Ladder for Helios GPUs
- 14 September 2026
AMD Helios will be an important platform for AI. Helios offers 432 GB of HBM4, 23 TB/s of HBM bandwidth per GPU, and 40 PFLOPs of FP4 compute AMD Instinct™ MI455X GPU. These capabilities will be especially valuable for large frontier models and long-context agentic workloads.
Iteratively Tuning hipBLASLt TensileLite Kernels: A Smaller Search, a Faster Kernel
- 09 September 2026
A TensileLite run can only return a kernel its configuration’s search space can express. That sounds like a technicality, but it is the one fact that decides whether a tuning campaign helps or hurts. The configuration is not a hint handed to an optimizer that is free to look elsewhere; it is the complete enumeration of what the run is allowed to consider, and anything outside it stays unreachable no matter how long the run is given. In a previous blog, Reverse-Engineering hipBLASLt TensileLite Kernels, we used that constraint defensively: decode a kernel you trust from its solution name, pin its roughly one hundred parameters as single-value ForkParameters, and the run cannot return anything slower, because the only kernel it can express is the one you started from. That gives tuning a floor.
veRL on AMD: Production-Ready RL Post-Training on ROCm
- 08 September 2026
Reinforcement learning post-training on AMD Instinct GPUs is here — with a turnkey container, AITER-accelerated vLLM and SGLang rollout, and accuracy validated on both MI300 and MI355.
Efficiently Serving NVFP4 Models on AMD Instinct™ MI350X/MI355X Accelerators via Online NVFP4 to Quark MXFP4 Requantization
- 08 September 2026
A growing share of frontier open-weight models are available in NVFP4 format, but on some AMD GPUs (MI350X / MI355X), FP4 compute support is limited to MXFP4. This means NVFP4 checkpoints couldn’t run on the native 4-bit acceleration path these GPUs provide.
Enabling DeepSeek-V4-Flash Training on AMD Instinct MI355X GPUs with Primus
- 03 September 2026
DeepSeek-AI released the DeepSeek-V4 series on April 24, 2026: a preview pair of MIT-licensed Mixture-of-Experts models, with DeepSeek-V4-Flash at 284B total parameters (13B activated) and a one-million-token context window. Flash pushes sparse attention further than any open-weight model before it: its 43 transformer layers interleave three different attention types, each layer sits inside a hyper-connection block rather than a plain residual, and every token is routed through 256 experts. Each of those choices breaks an assumption baked into stock Megatron-LM training code.
Optimizing ATOM and vLLM-ATOM for High-Interactivity Inference
- 01 September 2026
When you type a question into a chat assistant, ask a coding agent to make an edit, or watch tokens stream back from a model, what you feel is not throughput. You feel responsiveness. The pause before the first word appears, and the pace at which each following word arrives, decide whether an assistant feels instant or sluggish. As LLMs move into interactive assistants, coding agents, and multi-step tool-calling loops, this responsiveness — how quickly the system reacts to a single user — has become a first-class product requirement.
A Deep Dive into LDS Optimizations on AMD Instinct MI450 GPUs
- 28 August 2026
The first two authors (Plavsic, Zaghen) contributed equally to this work.
4-bit KV Caching in LMCache: Offloading Quantized KV Beyond HBM for Context-Heavy Agents on AMD MI355X
- 28 August 2026
As agents carry ever-longer context from turn to turn, the KV cache becomes the resource that runs out first. Two techniques ease that pressure from different angles: KV quantization makes each cached token smaller, while a hierarchical cache like LMCache spills cold KV to CPU DRAM. The two have mostly been developed independently.
ROCm 10.0: A Decade of Open Compute, Built for the Age of Agentic AI
- 27 August 2026
AMD shipped ROCm 1.0 in April 2016: an open-source GPU compute stack built around a C++ compiler and a GPU programming language called HIP, aimed at high-performance computing. A decade later, the same platform trains and serves frontier AI models across industries and enables nearly every compute application benefiting from GPUs.
Enabling Physical AI Agents with Lemonade
- 25 August 2026
In this blog we will demonstrate how to run local agents enabled by VLMs (Vision-Language Models) hosted by the Lemonade framework in the domain of robot control. These models allowed us to run an interactive robotic arm manipulation simulation entirely locally, showing the effectiveness of Lemonade for Physical AI.
Serving 64Mi-Token Contexts on One AMD Instinct™ MI355X Node
- 24 August 2026
Error parsing meta tag attribute “keywords”: No content.
DI Series: Scaling GLM-5.1-FP8 to 64 MI300X GPUs
- 21 August 2026
Serving a frontier Mixture-of-Experts (MoE) model well is a systems problem, and it gets harder the moment one node is not enough. GLM-5.1 is a good example: it is a large, sparse MoE that users want to run at long context, and it ships a new attention family that breaks assumptions older serving stacks quietly relied on. Fitting it on eight GPUs is only the start. The real question is how to keep it correct and fast as you spread it across several nodes.
Scaling RL with verl on AMD Instinct MI355X: Async Walkthrough and Sync Benchmark
- 18 August 2026
Reinforcement learning (RL) for large language models (LLMs) alternates between two phases: generation (rollout), where the current policy produces responses, and training, where those responses are used to update the policy. In verl, the key design choices are when these phases run relative to each other (synchronously or with overlap) and where they run (colocated on the same GPUs or on separate GPU pools). This blog first explains the differences between the two modes and when to use each.
Exploring XGBoost: A Deep Dive
- 18 August 2026
XGBoost (Extreme Gradient Boosting) is an open-source library that implements gradient-boosted decision trees, an ensemble method that builds an additive sequence of trees where each new tree is fit to the gradient of the loss left by the ones before it. It supports regression, classification, ranking, and survival objectives behind a single training loop, and is implemented as a high-performance C++ core with CPU and CUDA/HIP backends, exposed through Python, R, and JVM bindings. On large tabular datasets it is a standard production choice for both accuracy and training throughput. This blog opens the box on how it works, end to end.
Memory Instruction Scheduling for Lock-Stepped Kernels on AMD Instinct™ MI300X: Introducing the Series
- 17 August 2026
This post introduces a multi-part study of how instruction scheduling can influence the behavior of memory operations within loop iterations in GPU kernels where multiple waves execute in a steady-state lock-stepping manner. In this post, we establish the motivating tiled GEMM kernel, shared vocabulary, and ATT-based methodology; upcoming posts in the series will analyze specific VMEM and LDS scheduling bottlenecks in detail. GPUs can deliver very high memory throughput in general, but that potential is sometimes hard to realize in practice. Lock-stepping behavior, commonly seen in AI kernels that involve matrix multiplication, poses a particular challenge: when waves issue memory operations in sync, they can contend for shared resources and leave bandwidth on the table.
Bring Claude Code On‑Prem with AMD Instinct GPUs
- 17 August 2026
Agentic coding has become an indispensable part of modern software development. Tools like Claude Code don’t just autocomplete lines — they read entire codebases, plan and execute multi-file refactors, run tests, interpret failures, and iterate autonomously until a task is done. Developers who adopt these workflows report dramatic reductions in time spent on boilerplate, debugging, and context-switching. For engineering teams, agentic coding is fast becoming a competitive necessity rather than a convenience.
Production-Ready MXFP4 Online Rotation with Fused Kernels on AMD Instinct™ MI355X
- 13 August 2026
Serving large language models affordably increasingly depends on low-bit quantization, and MXFP4 is one of the most aggressive options — but the smaller models that need it most rely on online rotation to stay accurate, and that rotation has historically carried a steep latency tax. In this post you will learn how a single fused Gluon (Triton) kernel on AMD Instinct™ MI355X (CDNA4) removes that tax: we walk through the kernel-fusion design, the RS=64 optimization, the GEAK + Hyperloom tuning workflow, and end-to-end measurements showing online-rotation overhead falling from a prohibitive +5–10% to just +0.3–0.8% on Dense models — with no measurable accuracy loss. By the end, you will understand how to make accuracy-preserving MXFP4 online rotation cheap enough to ship by default.
Using ODC to Accelerate AMD SFT Training
- 12 August 2026
Large-scale training spends a surprising share of its wall-clock time waiting instead of computing. Under Fully Sharded Data Parallel (FSDP), every layer ends in a collective all-gather or reduce-scatter, and every collective is a barrier that the whole data-parallel group has to reach together. Feed that machinery variable-length supervised fine-tuning (SFT) data and the picture gets worse: some ranks draw long documents while others draw short ones, so the fast ranks sit idle waiting for the slow ones.
Quark Support for HuggingFace Diffusers and SVDQuant
- 06 August 2026
Diffusion models are heavy on memory and compute: a single text-to-image call runs a large transformer or UNet dozens of times. Quantization — storing weights (and sometimes activations) in low precision — is one of the most effective ways to cut both the memory footprint and the latency of these models.
AUP Learning Cloud: Streamlining AI Education on AMD
- 06 August 2026
AI education is becoming increasingly hands-on. Students are now expected to train models, build AI agents, experiment with large language models (LLMs), and develop applications accelerated by graphics processing units (GPUs), so courses depend as much on practical computing infrastructure as on learning materials. Yet standing up that infrastructure is usually the real bottleneck, and educators often spend more effort building it than teaching.
VSA: Accelerating Video Diffusion Inference with Sparse Attention on AMD GPUs
- 04 August 2026
Video generation powered by diffusion transformers has achieved remarkable quality, but the computational cost of attention mechanisms remains a critical bottleneck. With sequence lengths reaching tens of thousands of tokens in video generation tasks, the quadratic complexity of standard attention becomes prohibitively expensive.
Reverse-Engineering hipBLASLt TensileLite Kernels: From Solution Name to a Tuning Config
- 04 August 2026
In a previous blog, Customizing Kernels with hipBLASLt TensileLite GEMM Tuning, we showed how TensileLite Tuning generates brand-new GEMM kernels by searching a parameter space and selecting the fastest valid candidate for a given problem size. The search space is defined entirely by the tuning configuration YAML you provide. Its ForkParameters lists are expanded into a Cartesian product, and the tuning run can only ever return a kernel that is expressible as one combination within that product.
Introducing AMD CDNA™ 5 and the AMD Helios™ Rackscale Solution
- 04 August 2026
AI infrastructure is being redefined at the scale of the entire rack. This blog will walk you through how the AMD CDNA™ 5 architecture, the AMD Instinct™ MI455X GPU, and the AMD Helios rackscale solution come together as a single open platform for large-scale AI training and inference. You will see how AMD balances compute, memory, and networking across 72 GPUs, how open standards keep the platform flexible, and how the AMD ROCm™ software stack ties it all into a complete, deployable system, from silicon to rack. By the end, you will understand what changes when AI infrastructure is designed as a unified rack rather than a collection of discrete GPU servers, and where AMD is taking it next.
Closing the GPU Cluster Validation Gap: A Kubernetes-Native Approach with CVF
- 29 July 2026
As GPU clusters scale from dozens to thousands of accelerators, the assumption that every node is healthy at boot time becomes increasingly risky. Silent hardware degradation — a marginal XGMI link, a missing SR-IOV Virtual Function, or an under-performing RDMA rail — can turn a single node into a straggler that slows an entire distributed training job.
AMD GPU Operator v1.5.0: DRA Support, Automated GPU Node Recovery, and Expanded Kubernetes Infrastructure Control
- 28 July 2026
GPU Operator v1.5.0 introduces several major infrastructure capabilities for Kubernetes-based AMD GPU deployments, including support for Kubernetes Dynamic Resource Allocation (DRA), automated GPU node remediation workflows, and Node Problem Detector (NPD) integration.
Attention Decode on AMD MI450 GPUs: A Gluon Kernel Optimization Guide
- 27 July 2026
Agentic AI applications are pushing LLM inference into a new regime. A request can now reach one million tokens from aggregated long prompts, tool calls, retrieval results, and multi-turn reasoning. During the text generation phase, each new token must attend to all previous tokens from the KV cache, so the kernel needs to repeatedly read past states from HBM. As a result, the performance bottleneck shifts from compute units to the memory system.
Introducing Instella-MoE: A State-of-the-Art Fully Open Mixture-of-Experts Language Model
- 24 July 2026
AMD is excited to introduce Instella-MoE, a state-of-the-art fully open Mixture-of-Experts (MoE) language model with 16 billion total parameters and 2.8 billion active parameters. Trained from scratch on AMD Instinct™ MI300X and MI325X GPUs with the AMD ROCm™ software stack, Instella-MoE combines a sparsely activated MoE design with architectural innovations such as Gated Multi-head Latent Attention (Gated MLA) and FarSkip-Collective connectivity. Instella-MoE delivers competitive performance across a broad suite of benchmarks against both dense and MoE baselines (as shown in Figure 1) including models with comparable or larger active parameter counts, establishing it as one of the strongest fully open language models at its scale.
Enabling Language-specific Reasoning in Multilingual Models with Reinforcement Learning
- 24 July 2026
We introduce the Poro 2 Long family of models, a follow-up to the Poro 2 family that demonstrated exceptional performance in Finnish and English instruction-following and conversational tasks. This model family demonstrates our progress in developing models capable of reasoning in the same language as the user. A prerequisite for an effective reasoning model is the ability to process longer sequences beyond the sequence lengths used in pretraining; therefore, the Poro 2 Long models feature a context window of 128k tokens.
Serve Kimi-K2.5-MXFP4 on MI355X with ATOM
- 23 July 2026
In our previous Kimi-K2.5 posts, we moved from fused MoE optimization with FlyDSL [8] to W4A8 and W8A8 quantization with AMD Quark [9]. Those posts focused on kernel and quantization work: how to make the dominant MoE path faster, and how to trade off INT4, INT8, and FP8 formats on AMD Instinct™ MI300X and MI325X GPUs.
Onboard and Deploy Custom Models in AMD AI Workbench
- 23 July 2026
The AIM Catalog gives you a curated set of models that are ready to deploy on AMD hardware, but sometimes you might want to run a model that isn’t in the catalog, for example a model from the Hugging Face Hub, or one you or your team trained for a specific task. AMD AI Workbench lets you do that, directly through the graphical user interface (GUI). You point the AI Workbench at a Hugging Face repository or your own model registry, choose how the model should run, and the AMD AI Workbench onboards it into your project so you can deploy it like an AMD Inference Microservice (AIM).
Introducing AMD ROCm™ Infera: Scaling Goodput for Agentic AI with Distributed Inference Orchestration
- 23 July 2026
Today we are introducing AMD ROCm™ Infera, a distributed inference reference solution for large-scale deployments. Infera is a conductor for your inference GPU orchestra. Initial internal testing shows that Infera can improve goodput per GPU for realistic agentic workloads by up to 2.6×, as detailed below. Built for AMD Instinct™ GPUs, Infera is open source from day one, and the code is available at github.com/AMD-AGI/Infera.
Hyperloom - Autonomous Agentic Inference Optimization for AMD GPUs
- 23 July 2026
AMD is excited to introduce ROCm™ Hyperloom, a new open-source, agentic system aimed at automating the time-consuming task of optimizing end-to-end inference workloads. Using Hyperloom reduces the time to optimize an end-to-end workload from weeks to hours, saving time and enabling valuable resources to be allocated to other critical tasks. By combining various tools into an autonomous optimization loop, Hyperloom allows you to get the best performance out of your model and custom configuration on AMD Instinct GPUs.
Spur: Modern GPU Job Scheduling for HPC and AI Workloads
- 22 July 2026
The explosion of AI and large-scale model training has fundamentally changed how organizations think about GPU clusters. Traditional HPC schedulers were built in an era when CPUs dominated, and GPU support was retrofitted as an afterthought. Meanwhile, Kubernetes emerged from the cloud-native world with powerful orchestration primitives—but batch GPU scheduling and Slurm-style job workflows remain friction points unless you assemble a full ecosystem of operators, custom schedulers, and queueing layers on top of the default control plane.
Introducing ROCm™ AMD Infinity Context: A Purpose-Built KV Cache Tier for Distributed Inference
- 22 July 2026
In this blog, you will learn how ROCm™ AMD Infinity Context (ROCm AIC) addresses one of the fastest-growing bottlenecks in production AI systems: key-value (KV) cache management. You will explore the problem, the solution, the components that make up the stack, and how ROCm AIC works with existing AMD Instinct™ GPU deployments.
Deploy an Imaging AMD Solution Blueprint on AMD Radeon™ GPUs
- 22 July 2026
AMD introduces support for AMD Radeon™ AI PRO R9700 and AMD Radeon™ PRO W7900 GPUs in technical preview for the AMD enterprise AI reference stack, and the AMD GPU Operator:
Scaling MiniMax-M3 Inference with Distributed Serving and Operator Co-Design on AMD Instinct MI355X GPUs
- 21 July 2026
This blog walks you through a set of MiniMax-M3 inference optimizations on AMD Instinct™ MI355X GPUs using ATOM, AITER, and ATOMesh. For broader background on the inference engine and distributed serving layers used here, see the ROCm blogs on ATOM and ATOMesh.
Efficient MiniMax-M3 Inference on AMD Instinct GPUs with ATOM and ATOMesh
- 21 July 2026
MiniMax-M3 is a natively multimodal Mixture-of-Experts (MoE) foundation model with 428 billion total parameters and 22 billion activated per token. Released in June 2026, M3 is the first open-weight model to combine frontier coding and agentic capabilities, a 1-million-token context window, and native text, image, and video understanding within a single architecture. Its key innovation, MiniMax Sparse Attention (MSA), replaces standard quadratic attention with a block-level KV cache selection mechanism, reducing per-token compute to 1/20th of its predecessor at 1M context.
Building a High-Performance Video Inference Pipeline with ROCm Libraries Using C/C++
- 21 July 2026
AMD ROCm™ software is an open-source software stack designed for programming AMD GPUs, spanning everything from low-level kernel development to high-level applications. It provides developers with direct access to the full capabilities of AMD hardware, enabling high-performance and power-efficient computing.
Understanding Attention Algorithms and Their Backends for Image and Video Generation
- 20 July 2026
ComfyUI has become a widely adopted tool for running modern generative AI workloads, from text-to-image and video generation to more advanced pipelines like WAN2.2. Its node-based workflow makes it accessible without requiring deep coding knowledge, but under the hood it still relies heavily on one of the most important building blocks in modern AI models: attention.
SPIR-V on ROCm: A Portable IR for AMD GPUs
- 20 July 2026
ROCm has compiled GPU code ahead of time (AOT), once per target architecture, since its first release. CPU software shifted to runtime just-in-time (JIT) compilation in the 1990s — Java HotSpot, V8, LuaJIT, .NET RyuJIT, PyPy — once the matrix of target × workload × deployment became too large to enumerate at build time. With its adoption of SPIR-V, ROCm now brings the compile-once, specialize-on-device model to AMD GPUs.
GEAK V3: Agent-Driven, Repository-Level GPU Kernel Optimization across HIP, Triton, and FlyDSL on AMD GPUs
- 20 July 2026
In the ever-evolving world of GPU computing, optimizing kernels for performance and efficiency is a critical challenge. Hand-tuning kernels demands deep technical expertise and manual iteration. In this blog, you will read about how GEAK v3, the latest iteration of the agent-driven framework, tackles this problem using enhanced features such as task planning, test-harness discovery, patch-based handling of multi-file kernels, dynamic memory system and expert knowledge database. Our results show improvements across three kernel languages (HIP, Triton, and FlyDSL) and both CDNA and RDNA GPUs.
Performance Profiling on AMD GPUs – Part 5: Profiling-Driven Kernel Optimization with an AI Code-Assist Tool
- 16 July 2026
In this fifth installment of the Performance Profiling on AMD GPUs blog series, the focus shifts from tutorial format to a worked end-to-end case study: driving the iterative optimization of a double-precision HIP kernel on an AMD Instinct™ MI250 Graphics Compute Die (GCD), using the same profiling workflow that the earlier parts covered, combined with a modern AI code-assist tool to keep the per-experiment loop tight. Parts 1, 2, and 3 of the series covered the foundations of GPU performance profiling and basic and advanced usage of the ROCm profiling toolkit (see Additional Resources below); Part 4 applied that toolkit to a Fortran code with OpenMP offload.
Multi-Accelerator Support for AIMs and AMD Solution Blueprints
- 16 July 2026
With the latest release of AMD enterprise AI reference stack, Version 2.2 Release notes, we’re now introducing Multi-accelerator support. AMD Inference Microservices (AIMs) now run across AMD Instinct™ GPUs (MI300X, MI325X, MI350X, MI355X), AMD Radeon™ Pro GPUs (W7900 and R9700), and AMD EPYC™ CPUs (EPYC 9965). That same coverage also extends to the AMD Solution Blueprints.
When a Faster Kernel Doesn’t Speed Up Serving: Profiling FP8 KV Cache on AMD Instinct MI308X
- 15 July 2026
This case study starts with a result that didn’t add up. We enabled FP8 KV cache (--kv-cache-dtype fp8_e4m3) on a Kimi-K2.5-W4A8 (MoE + MLA) deployment on 8× AMD Instinct MI308X. At first glance, the trace had several encouraging signs: the MLA decode kernel ran 34% faster[1] than the BF16 baseline, going from 0.190 ms/call to 0.125 ms/call, and several existing categories such as GEMM, communication, and elementwise work moved slightly lower.
ROCm 7.14: TheRock Goes Production and Expands AMD’s AI Software Platform
- 15 July 2026
In this blog, you will explore the key updates and improvements that make the ROCm 7.14 release a major milestone in the evolution of AMD’s open source AI software ecosystem. At the heart of this release is the production debut of TheRock, AMD’s new automated, open-source build and release system that modernizes how ROCm is built, packaged, validated, and delivered.
From Vector Search to Agentic RAG: Building an Enterprise Research Analyst with hipVS
- 15 July 2026
hipVS and hipRAFT were introduced in this earlier blog post, which walked through the core vector search APIs. This post takes the next step: it shows what you can build with hipVS by walking through an Enterprise Research Analyst demo — an end-to-end working agentic RAG (Retrieval-Augmented Generation) system that uses hipVS as its retrieval backbone. This demo combines multi-document ingestion, LLM-powered query decomposition, parallel GPU-accelerated vector search, and cross-document synthesis with source citations, all running on AMD hardware. Along the way, it highlights the specific hipVS and hipRAFT APIs the demo exercises and shares code snippets you can adapt for your own applications.
LogsLop: A Tiny Summarization Tool for Enormous Log Files
- 14 July 2026
This blog introduces LogsLop, a summarization tool that can make huge repetitive log files tractable for human readers and artificial agents.
Local Image and Video Generation on AMD Ryzen™ AI Max+ Processor (Windows)
- 14 July 2026
Running local generative-AI image and video workflows on AMD hardware on Windows has typically meant going through the Windows Subsystem for Linux (WSL). That works, but it adds a virtualization layer, extra setup, and a mental model that many creators would rather avoid.
Triton-Based Optimization of Video Sparse Attention on ROCm
- 13 July 2026
Video generation has become a major frontier in generative modeling, driven by large-scale data and increasingly scalable architectures. Among modern architectures, Diffusion Transformers (DiTs) have emerged as a dominant paradigm [1,2,3,4] by representing videos as spatio-temporal token sequences, enabling long-range interactions across frames and spatial regions, as well as flexible multimodal conditioning with text or audio. However, full self-attention scales quadratically with token count, making it increasingly expensive as spatio-temporal resolution and model size grow. Video sparse attention (VSA) [5,6] mitigates this cost by approximating full attention with a subset of informative token interactions, but its practical efficiency in both training and inference depends heavily on hardware-aware Triton kernel implementations.
Serving NVFP4 Models on AMD Instinct™ MI355 Accelerators
- 13 July 2026
NVFP4 is an increasingly common deployment format: NVIDIA, AMD, and the open-source community have published NVFP4 quantized checkpoints of frontier models such as moonshotai/Kimi-K2.6, and many users want to deploy these checkpoints directly. AMD Instinct™ MI355 is built on the CDNA4 architecture, which has no native NVFP4 tensor execution path — meaning these checkpoints could not previously be served on MI355 without an expensive offline conversion to a different format.
QuickReduce INT3 Quantization and Benchmarking on MI355
- 13 July 2026
Large Language Models (LLMs) typically contain billions — or even tens of billions — of parameters. During inference, tensor parallelism (TP) is a widely used technique that distributes the compute across multiple GPUs. This approach, however, requires frequent, large-scale data synchronization between layers, introducing significant communication latency and placing enormous pressure on interconnect bandwidth.
GEAK Agent-Driven Optimization of the DeepSeekV4 MLA Kernel
- 13 July 2026
Optimizing LLM inference kernels requires more than a single kernel rewrite. Developers need to migrate reference implementations, analyze profiling results, validate correctness, and iterate quickly across different workload shapes. In this blog, we use DeepSeekV4 MLA as a case study to show how GEAK automates this workflow, from PyTorch-to-Triton migration to kernel-level optimization and SGLang end-to-end(E2E) validation on AMD GPUs.
Fast Image Generation and Editing with SGLang Diffusion on AMD GPUs
- 10 July 2026
Visual generative AI is advancing at an extraordinary pace. OpenAI’s GPT Image 2, released in April 2026, is a reasoning-enabled image model that performs an internal planning process before generating images. Its predecessor generated over 700 million images within just its first week after launch. Google’s Nano Banana (Gemini 3.1 Flash Image) delivers real-time, knowledge-grounded image generation and editing, while open-source video models such as HunyuanVideo can now produce fluid, high-fidelity clips from a single sentence.
Porting High-Performance HIP Kernels to FlyDSL
- 09 July 2026
In our series of posts we have explored how to utilize Matrix Core instructions and how to design high-performance GEMM kernels using HIP C++. In this post we’ll show how to port those kernels to FlyDSL, a new Python DSL developed at AMD to simplify kernel development and testing. By working through real-world examples, we’ll explore the key concepts of FlyDSL and how they map to low-level HIP C++ code. Finally, we’ll show how — despite being higher-level — FlyDSL kernels can match or even exceed the performance of hand-tuned C++ kernels with a fraction of the complexity.
AMD Instinct™ Network Traffic, Congestion Trends, and Harmonics in Scale-Out Networks for AI Training Clusters
- 09 July 2026
Error parsing meta tag attribute “keywords”: No content.
Towards Feature Complete Triton Support in JAX-Triton
- 08 July 2026
In this blog we’ll explore recent improvements to JAX-Triton project and learn about new features available at the AMD fork.
SGLang-ATOM: Bring ROCm-Native Acceleration to SGLang Serving
- 08 July 2026
Large language model serving teams often face two competing goals: keeping the flexibility and developer velocity of an ecosystem serving framework, while also reaching strong throughput, latency, and cost efficiency on production accelerators. In this blog, you will explore how SGLang-ATOM bridges these needs for AMD Instinct GPUs by connecting the SGLang serving experience with ATOM’s ROCm-native execution path.
Efficient Hyperparameter Optimization for Autonomous Driving Models with AMD Instinct GPU Partitioning
- 08 July 2026
For automotive OEMs and Tier-1 suppliers, developing production-grade perception models presents a significant bottleneck: achieving the accuracy required for safety-critical applications such as forward collision warning and autonomous emergency braking demands systematic hyperparameter optimization (HPO). However, HPO requires training hundreds of model variants, each taking hours on a single GPU, making thorough exploration of the parameter space prohibitively slow and expensive with conventional single-GPU setups.
RDC and RocProfiler Compared to DCGM for Commonly Used Metrics
- 07 July 2026
Modern GPU applications often need lightweight, repeatable performance checks that can run outside a full profiling session. A developer might want to confirm whether a workload is throttling, whether data is moving across GPU interconnects as expected, whether ECC counters are increasing, or whether the GPU is spending enough time on useful compute work. These checks are especially useful during application tuning, cluster validation, and regression testing.
Occupancy Math on the AMD MI355X GPU (CDNA4): A From-First-Principles Guide
- 07 July 2026
Ask a GPU kernel engineer how their kernel is doing and occupancy comes up within a sentence or two. It’s the number everyone quotes and the dial everyone reaches for — and, in our experience, the metric people understand least. Most treat it as an opaque percentage the profiler hands back. It isn’t. Occupancy is fully derivable by hand from a kernel’s resource usage and a handful of fixed hardware limits, and being able to do that derivation changes how you tune. In short: on MI355X GPU occupancy is set by whichever of four resource limiters runs out first (VGPRs and SGPRs — vector and scalar registers; LDS — the on-chip shared scratchpad; or workgroup/barrier slots), the VGPR file is 512 per lane, shared by regular and accumulator registers (not a separate AccVGPR pool), and maximizing occupancy is usually the wrong goal — in a measured MXFP8 MFMA sweep below, the matrix core holds ~97% of peak even as occupancy falls to a fraction of full.
Primus Tuning Agent: Closing the Configuration-Search Loop
- 06 July 2026
Error parsing meta tag attribute “keywords”: No content.
Accelerating Diffusers and xDiT Image Generation with MXFP4 using AMD Quark on AMD Instinct™ MI350 GPUs
- 06 July 2026
Diffusion models such as Black Forest Labs’ FLUX.1-dev [1] deliver stunning image quality but demand significant compute and memory bandwidth at inference time. To reduce inference cost without sacrificing image quality, precision-aware quantization techniques have become a critical optimization strategy.
Building a GPU-Resident YOLO26 Object Detection Pipeline on the AMD Radeon™ AI PRO R9700 GPU
- 03 July 2026
Modern AMD GPUs include a dedicated hardware block for video processing called the Video Core Next (VCN) engine. By chaining VCN directly into machine learning frameworks, you can build an object detection pipeline where a video frame stays in VRAM from decode to the final bounding boxes. The host sees only the surviving detections.
AgentKernelArena: Benchmarking AI Coding Agents for GPU Kernel Optimization on AMD Instinct GPUs
- 03 July 2026
AI coding agents such as Cursor Agent, Claude Code, and OpenAI Codex are improving fast, and people increasingly trust them with specialized, high-stakes work, including GPU kernel optimization. But most of the public evidence is still a cherry-picked demo on a single kernel, not a controlled, head-to-head comparison on the same tasks, the same hardware, and the same scoring rules. On AMD Instinct™ GPUs, where every percentage point of kernel performance translates directly into training and inference cost, that gap matters.
Accelerating Large-Scale LLM Inference on AMD Instinct MI350X/MI355X with Eagle3 and AMD Quark
- 03 July 2026
Large language model (LLM) inference is increasingly constrained by autoregressive decoding. Even when prefill is highly optimized, the decode phase still generates tokens one step at a time, and each step typically requires running the full target model. For large mixture-of-experts and attention-heavy models such as Kimi-K2.5 and MiniMax-M2.5, this sequential pattern limits serving throughput and increases latency for real-time applications.
Optimizing MI300X Inter-Chiplet Communication via the RCCL Tuner API
- 30 June 2026
The AMD Instinct MI300X’s chiplet architecture introduces non-uniform communication paths when running in CPX/NPS4 mode — and RCCL default algorithms don’t account for this topology.
OpenXLA and JAX - ROCm Support and the State of CI
- 29 June 2026
The OpenXLA compiler stack — XLA at the foundation, JAX as the front end — now runs upstream on AMD ROCm. XLA gates every pull request on real AMD Instinct silicon through its GitHub Actions workflow, side by side with the CUDA path; JAX runs the same hardware on every ROCm PR through its own workflows, with the merge gate rolling out next. pip install "jax[rocm7-local]" is a first-class entry point. This post documents how that backend is structured, what landed in the last twelve months, and how the CI pipeline that keeps it healthy is wired together. Part 1 covers OpenXLA on AMD — the XLA backend, what landed this year, and CI. Part 2 covers JAX on AMD — the plugin architecture, JAX-side changes, and the four-workflow test matrix.
Accelerating LLM Inference on AMD GPUs with Low-Latency GEMMs
- 29 June 2026
Large language model inference is becoming increasingly interactive. Users expect chatbots, coding assistants, agents, and real-time copilots to respond quickly, stream tokens smoothly, and stay responsive under concurrent load. In that setting, decode-time latency is not just a backend metric. It directly affects perceived quality.
MXFP6 and MXFP4 Mixed Precision for Accelerating Dense LLMs on AMD Instinct MI355X
- 26 June 2026
In this blog, you will learn how pairing MXFP6-E2M3 activations with MXFP4 weights can meaningfully recover accuracy lost to pure 4-bit MXFP4 quantization in specific workloads and configurations, while staying within 2–3% of MXFP4 throughput. You will see measured offline throughput, serving latency, and benchmark accuracy results comparing BF16, FP8, MXFP4, and W_MXFP4_A_MXFP6 on Llama-3.1-8B and Qwen3.6-27B on AMD Instinct MI355X.
Efficient GPU Utilization With Workload Pre-Emption in AMD Resource Manager
- 26 June 2026
GPU capacity is sought after and in high demand. Production inference services, fine-tuning jobs, and developer workspaces like VS Code or JupyterLab all compete for the same resources. The challenge is not just about provisioning enough GPUs, it is keeping them utilized and making sure prioritized work can access capacity when it needs it. Training jobs can drop to near-zero utilization between compute phases; inference services can go quiet between traffic bursts; R&D or experimentation models and development workspaces might be left running unutilized or after hours. This would mean that workloads hold on to GPUs they are no longer using, while other work sits queued.
DP Attention and TBO for DeepSeek-V4 on MI355X
- 24 June 2026
Running DeepSeek-V4 efficiently requires solving two intertwined problems: how to parallelize MoE communication across GPUs, and how to hide that communication behind useful compute. The dominant approach is Expert Parallel with all2all backends like DeepEP. This solves both problems, but it also requires specialized kernels, topology assumptions, and careful expert placement.
Faster Kimi-K2.5-W4A8 Decoding with EAGLE3 on AMD Instinct™ MI325X
- 23 June 2026
In our previous blog [7], we deployed Kimi-K2.5 [1] in W4A8 (INT4 weights + INT8 activations) on AMD Instinct™ MI325X, replacing the BF16 MFMA path in the fused MoE kernel with FlyDSL [2]’s INT8 MFMA implementation. The remaining bottleneck is the autoregressive nature of decoding itself: even with INT8 MFMA and INT4 weights, the framework still runs one full forward pass per generated token.
A Practical Guide to Running LLMs on AMD Radeon™ GPUs
- 19 June 2026
Running large language models on AMD Radeon™ GPUs has never been more accessible or more exciting. Thanks to rapid advancements in open‑source tooling and GPU acceleration, both Radeon™ integrated GPUs (iGPU) and discrete GPUs (dGPU) have become powerful, cost‑effective platforms for local AI. Whether you prefer a polished desktop application, a lightweight command‑line workflow, or a fully customizable runtime, a rich ecosystem of tools now makes it easy to deploy cutting‑edge models on your system. With today’s software stack, you can run state‑of‑the‑art language models directly on your Radeon™‑powered PC, whether you’re using integrated graphics or a high‑performance discrete card.
Efficient and Portable 3D Explorable World Generation on AMD GPUs
- 18 June 2026
Explorable 3D world generation is becoming a foundational capability for spatial and embodied intelligence. Training agents that can navigate, reason, and interact with environments requires not just static datasets, but rich, immersive worlds that support free-view exploration and consistent geometry. Recent works like Matrix3D1 have pushed this frontier forward by combining panoramic generation with explicit 3D reconstruction, enabling higher-quality and more coherent environments than prior video-only methods.
Comparative Analysis of Scale-Out RoCE Network Traffic Patterns and Loads in Training Large Language Models
- 18 June 2026
As large-scale AI workloads continue to grow, understanding network behavior becomes critical. This blog analyzes scale-out RoCE traffic patterns and loads in large language model training, helping you uncover bottlenecks, improve performance, and design more scalable ROCm-based systems.
Building and Deploying Custom hipBLASLt Libraries on AMD Instinct GPUs
- 18 June 2026
General Matrix Multiply (GEMM) operations are a core component of many generative AI workloads. Whether you are running attention mechanisms in the prefill phase of a Large Language Model (LLM) or generating tokens sequentially during the decode phase, matrix multiplication performance has a direct impact on end-to-end latency and throughput.
Utilizing AMD Schola and UnrealRoboticsLab with AMD ROCm™ Software to Train a Robotic Arm
- 17 June 2026
A great reinforcement learning (RL) training environment excels along many axes. Unreal® Engine brings a powerful combination of capabilities, including physically based rendering, high-fidelity visual environments, and a mature toolset for building rich interactive scenes. These strengths make it an excellent fit for training tasks that involve complex lighting or rich vision-based observations.
Technical Dive into AMD’s MLPerf Training v6.0 Submission
- 16 June 2026
AMD is proud to share its MLPerf Training v6.0 results, marking another step forward in our commitment to delivering competitive AI training performance using the latest AMD Instinct GPUs. This round covers three benchmarks — Llama 2 70B LoRA fine-tuning, Llama 3.1 8B pretraining, and Flux.1-schnell text-to-image pretraining — with AMD’s own submissions spanning the MI325X, MI350X, and MI355X Instinct GPUs, alongside a large-scale 512-GPU MI300X submission from OCI in partnership with AMD.
Reproducing AMD MLPerf Training v6.0 Submission Result
- 16 June 2026
This blog provides a step-by-step guide for reproducing AMD’s MLPerf Training 6.0 submission results. AMD submitted results on three benchmarks this round:
ATOMesh: Unlocking AMD Hardware for Scalable LLM Serving
- 16 June 2026
Large language model serving is moving from single-engine optimization to full-stack distributed inference. Production deployments must handle high concurrency, long-context prefill, latency-sensitive decode, KV cache store pressure, and multi-node GPU utilization at the same time. On AMD Instinct GPUs, the key opportunity is to connect ROCm-native kernels, communication libraries, inference engines, and distributed orchestration into one scalable serving stack.
ATOM: Unlocking Extreme AMD Instinct Inference with Software-Hardware Co-Optimization
- 15 June 2026
As LLM serving enters a phase defined by high concurrency, long-context workloads, sparse MoE activation, and multi-GPU deployment, the challenge is no longer basic functionality but sustaining peak efficiency on AMD GPUs under production-scale load. ATOM (AiTer Optimized Model) is built for that goal, following four core principles: system-level optimization for LLM inference on AMD Instinct™ GPUs, kernel-level acceleration through AITER, distributed inference scaling with MORI, and a rollout-engine path for RL workloads. It builds on earlier ROCm blog coverage of AITER and vLLM-ATOM, moving from kernel and plugin acceleration into the standalone ATOM inference engine. Rather than being a generic framework adapted to the ROCm™ software, ATOM is an execution engine designed with ROCm-first priorities, AITER-native operators, and deep optimization on the inference-critical path. Aligned with the AMD Instinct roadmap from single-node optimization to multi-node scale-out, ATOM evolves its architecture, kernel strategy, and distributed execution model in lockstep with each hardware generation.
Productionizing TurboQuant on AMD GPUs for KV-Cache-Bound LLM Inference
- 11 June 2026
*The first three authors (Chakrabarti, Limpus, Rana) contributed equally to this work.
Low Kruskal-Rank Adaptation
- 11 June 2026
In this blog, you will explore how to enhance Low-Rank Adaptation (LoRA) which uses matrix rank, and replace it with Kruskal rank for efficient training. LoRA is one of the most widely used parameter-efficient fine-tuning (PEFT) methods for adapting pre-trained large language models (LLMs) to downstream tasks. Although LoRA significantly reduces the number of trainable parameters and lowers fine-tuning costs, its performance is often limited by the inherent low-rank assumption. We revisit the notion of rank for LoRA update matrices and show that the standard matrix rank fails to capture duplicated directions and redundancy in the update subspace. Motivated by this analysis, we argue that the Kruskal rank offers a more informative criterion for characterizing update diversity. We therefore propose Low Kruskal Rank Adaptation (LoKRA), a new PEFT algorithm with provable theoretical guarantees that mitigates the limitations of LoRA. We further introduce LoKRA+, an enhanced variant that provides a tighter theoretical lower bound on the Kruskal rank and yields stronger empirical performance. Experiments on multiple LLMs show that our approach consistently outperforms LoRA and other baselines, establishing state-of-the-art performance across a range of benchmarks. The paper is accepted by ICML 2026 (paper link), and the code is publicly available on GitHub.
Dropless MoE Training in JAX with Primus-Turbo
- 10 June 2026
Mixture-of-Experts (MoE) models have become a standard way to scale a transformer’s parameter count without paying the full compute bill — but training them efficiently on GPUs forces an uncomfortable trade-off. The default path in JAX/MaxText keeps every expert’s tensors at a fixed shape and simply drops the tokens that overflow each expert’s capacity, trading model quality for speed. The fully dropless alternative keeps every token, but in pure JAX it hits a memory wall that makes it impractical at production scale.
ORBIT-2 based Weather and Climate Downscaling and Downscaled Global Forecasts on AMD Instinct
- 08 June 2026
Advances in complex modeling, collection of data, and computational capacity over the past several decades have resulted in accurate numerical weather prediction (NWP) models that are run every day as part of operations of large weather agencies [1]. In the past few years, data-driven AI models have emerged as an alternative to classes of NWP, making predictions at similar (or even slightly better) skill levels [2] [3] [4], with drastically lower computational costs at inference-time, effectively democratizing access to weather forecasting. The AI models have been most successful at a class of synoptic models: global weather prediction models at resolutions of \(10-30~\rm{km}\) with lead times starting from \(6-12~\rm{hours}\), because such models could be trained on \(\sim 40\) years of well-curated data such as the ERA5 reanalysis archives [8] of the European Centre for Medium-Range Weather Forecasts (ECMWF). In previous blog posts, we have discussed inference using SOTA synoptic AI models and training such models.
Adapting AIM LLMs For Specific Use Cases Through Fine-Tuning in AMD AI Workbench
- 03 June 2026
In this blog, you will learn how to fine-tune a pre-trained Large Language Model (LLM) with AMD AI Workbench without writing a single line of code and then deploy it using AMD Inference Microservices (AIMs). Rather than training a model from scratch, fine-tuning allows you to adapt a pre-trained model to your specific use case. In addition, AIMs provide standardized, portable inference microservices for serving AI models. AIMs abstract away the complexities involved in model serving by providing an intelligent orchestration layer that automatically configures runtime environments, detects available accelerators, and selects an optimized performance profile (configuration parameters for the inference engine).
Performance Profiling on AMD GPUs - Part 4: Fortran OpenMP Offload Edition
- 01 June 2026
This blog, like the previous articles in the profiling guide series (Part 1, Part 2, and Part 3), is designed to help you systematically analyze and improve the performance of your Fortran OpenMP offload applications running on AMD GPUs. This guide builds upon the foundational skills from the previous articles and introduces profiling techniques specifically tailored for Fortran applications that use OpenMP target offloading.
Out-of-the-Box ROLL Support on AMD GPUs: Accelerating Reinforcement Learning at Scale
- 01 June 2026
Reinforcement learning (RL) is rapidly becoming a foundational technology for Large Language Models (LLMs)—powering key abilities such as reasoning and agentic behaviors. As RL workloads grow more complex and computationally intensive, the ecosystem increasingly depends on scalable, high-performance frameworks that can fully utilize modern GPU clusters.
Running Variational Quantum Eigensolver with Qiskit Aer on AMD Instinct
- 29 May 2026
Quantum computing offers a fundamentally different approach to computational problems by leveraging quantum mechanical properties such as superposition and entanglement. Unlike a classical bit, which is always 0 or 1, a qubit can exist in a superposition of both, and in principle this gives a significant resource advantage: \(n\) qubits represent a state that would otherwise require \(2^n\) complex numbers on a classical computer. However, current quantum hardware is still in its early stages - noise and limited qubit counts constrain the scale of problems it can handle reliably. GPU-accelerated simulators efficiently emulate quantum circuits on classical hardware, though they inherit the same exponential memory cost and become impractical past a few dozen qubits. Of course, any problem whose quantum circuit can be fully simulated on classical hardware can also be handled with other methods that avoid the simulation overhead, but the real value of circuit simulation is the opportunity to develop, validate, and benchmark quantum algorithms in a controlled setting where exact solutions are known, so that the same algorithms can be trusted on future hardware tackling problems that remain intractable at scale today.
Enabling Speculative Speculative Decoding on MI300X
- 29 May 2026
Speculative speculative decoding (SSD) [1] is a recently proposed speculative decoding (SD) algorithm that further accelerates large language model (LLM) inference beyond conventional SD. In standard SD, a small draft model proposes several future tokens, and a large target model verifies them in parallel. SD already reduces the cost of purely autoregressive decoding, but it still contains a sequential dependency: the next draft step cannot start until the current verification step finishes.
Deep Dive Into 4-Wave Interleave FP8 GEMM
- 27 May 2026
Our previous two posts in this GEMM optimization series covered Matrix Core instructions and 8-wave ping-pong FP8 GEMM design. Here we discuss another algorithm design introduced by HipKittens - 4-wave interleave, which further improves the performance of the 8-wave ping-pong implementation. For the most complete understanding, we recommend reading this post alongside the source code.
AI Inference on AMD Ryzen™ AI Max Processor
- 25 May 2026
Local large language model (LLM) inference has rapidly evolved, but a persistent limitation remains: model size is constrained by available GPU memory. Discrete GPUs typically offer 8–24 GB of dedicated VRAM, which can limit the size of models that can run without incurring significant quality loss from aggressive quantization. As frontier open-weight models grow past 70B and 100B parameters, this gap is forcing more developers toward multi-GPU rigs or paid cloud endpoints just to evaluate a single checkpoint.
From Naive to Near-Peak: Building High-Performance GEMM Kernels with Gluon
- 22 May 2026
On a single MI355, our most-optimized FP16 GEMM kernel runs at 99% MFMA efficiency — the matrix engine sits idle for a handful of cycles per loop. Getting there took ten versions, a regression along the way, and a profiler open for the whole time. This post is a tour of that path: from a 520 TFLOPS naive baseline to a 1489 TFLOPS near-peak kernel (~3× speedup), then the same design carried forward to BF8 (3257 TFLOPS, 99.72%) and MXFP4 (5255 TFLOPS, 92.41%) for low-precision AI workloads.
From Build to Benchmark: ONNX Model Serving with Triton Inference Server on AMD GPUs
- 22 May 2026
Triton Inference Server is an open-source platform designed to streamline AI inferencing. It supports the deployment, scaling, and inference of trained models from multiple frameworks, including ONNX Runtime, TensorFlow, PyTorch, and others. It runs across cloud, data center, and edge environments, making it adaptable for diverse AI workloads.
ROCm 7.13: Expanding Hardware, Tools, and Reach
- 20 May 2026
AMD released ROCm Core 7.13, the AMD GPU Driver 31.30, and AMD GPU Virtualization 9.0. With these releases, ROCm software expands hardware support across enterprise datacenters. The platform introduces AMD’s latest Instinct accelerators, enables GPU virtualization on VMware ESXi and KVM, and delivers optimized performance for generative AI and large language models. The developer experience has been refined with streamlined profiling tools and open-source visibility into low-level performance analysis. As a result, AI development has become more practical across a broader range of hardware, spanning the latest AMD Instinct hardware and newly supported virtualized GPU partitioning modes.
QuickReduce FP4 Quantization and Benchmarking on MI355
- 20 May 2026
Large Language Models (LLMs) typically contain billions — or even tens of billions — of parameters. During inference, tensor parallelism is commonly employed to distribute the workload across multiple GPUs. This approach demands frequent, large-scale data synchronization between layers, introducing significant communication latency and placing enormous pressure on interconnect bandwidth.
Diffusion-based Atmospheric Downscaling on AMD Instinct GPUs
- 20 May 2026
In addition to forecasting, reconstruction is also a commonplace procedure in atmospheric and climate sciences. Reconstruction in this context generally means estimating and infilling missing data, most often due to lack of sensor coverage. The data could be missing spatially or temporally, that is from some regions or at certain times respectively. The data could also be present at some scales and missing at some other scales – for example satellite ice coverage data would have larger scales present but be missing smaller scales due to resolution limitations. Reconstructing these missing smaller scales is called downscaling in climate sciences.
Semantic Fencing of Video Streams Using Embedding Splits from Vision Foundation Models
- 15 May 2026
In this blog, we present a novel approach for semantically splitting vision datasets into training, validation, and test sets. Instead of relying on ad hoc metadata rules or random shuffles, we use embeddings to reason directly about similarity in latent space and construct splits that better reflect true generalization.
Further Accelerating Kimi-K2.5 on AMD Instinct™ MI325X: W4A8 & W8A8 Quantization with AMD Quark
- 14 May 2026
In our previous blog [7], we demonstrated how to accelerate Kimi-K2.5 [1] inference on AMD Instinct™ GPUs by profiling the model, identifying fused_moe as the dominant bottleneck (consuming 88–90% of GPU time), and replacing the default Triton-based kernel with a FlyDSL [2]-powered mixed-precision (BF16 + W4A16) fused MoE implementation.
Accelerating ComfyUI Workflows on AMD Instinct™ MI355X GPUs with ROCm
- 11 May 2026
ComfyUI is an open-source, graphical node-based interface for building generative AI workflows using diffusion models. With over 100,000 stars on GitHub, it has become one of the most widely adopted tools for text-to-image, text-to-video, and image-to-3D generation. You can build workflows by connecting nodes in a drag-and-drop visual interface (no coding required). A large community contributes custom nodes and workflow templates, making ComfyUI a versatile front-end for models ranging from 12B to 27B parameters. For background and setup across AMD platforms, see the earlier ROCm blogs Running ComfyUI on AMD Instinct, Getting Started with ComfyUI on AMD Radeon™ RX 9000 Series GPUs, and Running ComfyUI in Windows with ROCm on WSL.
vLLM-ATOM: Unlocking Native AMD Performance in the vLLM Ecosystem
- 07 May 2026
This blog walks you through vLLM-ATOM, the AMD-optimized plugin that supercharges vLLM on Instinct GPUs.
AMD-Powered 3D Gaussian Splatting for Autonomous Driving Scenes
- 07 May 2026
3D Gaussian Splatting (3DGS) is an innovative, explicit scene representation and rendering technique. It reconstructs photorealistic 3D environments from a set of images of the scene from a variety of angles. It represents a scene as a vast, learnable collection of 3D Gaussians, which are optimized with backpropagation using a differentiable rasterizer. This pipeline enables real-time novel view synthesis of the scene – generating images of the scene from previously unseen angles. It also permits easy scene editing by moving, copying, recolouring, etc. parts of the reconstructed 3D structure.
Accelerating Mixture-of-Experts Execution with FarSkip-Collective Models
- 05 May 2026
Whether you are running training or inference, the largest Mixture-of-Experts (MoE) based LLMs cannot fit on a single GPU; instead you must run collective-communication operations to integrate the work of multiple GPUs to work together on a single model.
TraceLens: Democratizing AI Performance Analysis
- 27 April 2026
Profiling modern AI workloads produces huge traces that are hard to interpret. Framework profilers record thousands of operations, kernels, and communication events, and engineers often end up staring at tools like Perfetto UI doing manual calculations. TraceLens speeds this up: it consumes existing framework traces and turns them into structured summaries and comparisons, allowing you to move on to the actual diagnosis and optimization.
Styled Text Image Generation with Eruku on AMD
- 24 April 2026
Producing text images where text is both readable and controllable while faithfully matching a target visual style is a challenging problem. It has broad applications ranging from synthetic handwritten text generation to graphic design. In these settings, you need more than plausible images; you need precise control over both text content and visual fidelity. This is where Eruku[1] stands out.
Primus Projection: Estimate Memory and Performance Before You Train
- 24 April 2026
Error parsing meta tag attribute “keywords”: No content.
Getting Started with FlyDSL Nightly Wheels on ROCm
- 20 April 2026
In the previous post on FlyDSL, we introduced the motivation behind FlyDSL and how it enables Python-native GPU kernel development using the AMD ROCm™ software stack. FlyDSL combines the flexibility of Python with the performance of MLIR and LLVM-based compilation, allowing developers to write GPU kernels in Python while targeting modern AMD hardware.
FLy: A New Paradigm for Speculative Decoding — Accepting Semantically Correct Drafts Beyond Exact Match
- 20 April 2026
Speculative decoding has emerged as a highly effective approach to accelerate large language model (LLM) inference, yet existing methods are severely bottlenecked by a rigid exact-match verification rule that discards many semantically valid continuations. Furthermore, existing training-based loose decoding methods often suffer from significant performance degradation on out-of-distribution (OOD) tasks.
Introduction to profiling tools for AMD hardware
- 10 April 2026
Getting a code to be functionally correct is not always enough. In many industries, it is also required that applications and their complex software stack run as efficiently as possible to meet operational demands. This is particularly challenging as hardware continues to evolve over time, and as a result codes may require further tuning. In practice, many application developers construct benchmarks, which are carefully designed to measure the performance, such as execution time, of a particular code within an operational-like setting. In other words: a good benchmark should be representative of the real work that needs to be done. These benchmarks are useful in that they provide insight into the characteristics of the application, and enable one to discover potential bottlenecks that could result in performance degradation during operational settings.
Serving CTR Recommendation Models with Triton Inference Server using the ONNX Runtime Backend
- 07 April 2026
In a previous ROCm blog post, “Triton Inference Server with vLLM on AMD GPUs”, deploying large language models using Triton Inference Server with the vLLM backend on ROCm-enabled AMD GPUs was introduced. In this blog, you will explore the ONNX Runtime and Python backends in the ROCm build of Triton Inference Server, along with an upgrade that aligns the build with the latest upstream Triton Inference Server release. You will also see how these enhancements expand AI model deployment capabilities and highlight the performance advantages of AMD Instinct GPUs using a representative recommendation model.
FlashInfer on ROCm: High‑Throughput Prefill Attention via AITER
- 06 April 2026
The explosive growth of large language models (LLMs) like DeepSeek-R1, Llama 3, and Qwen 3 has created an urgent need for efficient inference solutions. As these models scale to billions of parameters and context lengths extend to hundreds of thousands of tokens, the attention mechanism becomes a critical bottleneck, consuming substantial memory for key-value (KV) caches and requiring significant compute for each token generated.
Customizing Kernels with hipBLASLt TensileLite GEMM Tuning - Advanced User Guide
- 06 April 2026
Optimizing General Matrix Multiply (GEMM) operations is critical for maximizing the efficiency of AI models on AMD hardware. In our previous blog posts, we explored Offline Tuning, a method for selecting the best-performing kernel from an existing solution pool. For detailed instructions on using hipBLASLt-bench, please refer to hipBLASLt offline tuning part 1 and part 2. Additionally, for a streamlined experience, check out the Day 0 Developer Guide: hipBLASLt Offline GEMM Tuning Script which covers one-click offline tuning. Furthermore, for scenarios requiring dynamic runtime adaptation, developers can explore our recently published blog on hipBLASLt Online GEMM Tuning.
Deploy and Customize AMD Solution Blueprints
- 02 April 2026
AMD Solution Blueprints are ready-to-deploy, customizable reference applications built with AMD Inference Microservices (AIMs). They offer a microservice solution for a range of use cases, from standard chat interfaces to agentic frameworks, serving as both starting points for development and example implementations.
Reproducing the AMD MLPerf Inference v6.0 Submission Result
- 01 April 2026
MLPerf Inference v6.0 marked AMD’s fourth round of submissions to MLPerf Inference. This blog provides a step-by-step guide to reproducing AMD’s results on different vendor systems.
AMD Instinct™ GPUs MLPerf Inference v6.0 Submission
- 01 April 2026
The results for the MLPerf Inference v6.0 benchmark were released on April 1st 2026. In this round, AMD showcased the performance of the MI355X system, as well as the capability and versatility of the ROCm software stack.
Training a Robotic Arm Using MuJoCo and JAX on AMD Hardware with ROCm™
- 31 March 2026
Training a robotic arm to pick up an object and place it somewhere else may sound straightforward, but teaching a robot to do this reliably in the real world is one of the harder problems in robotics. Traditional approaches rely on hand-tuned motion planning and carefully scripted control logic, which is brittle and time-consuming to maintain as environments change.
Leveraging AMD AI Workbench and Autoscaling to Scale LLM Inference for Optimal Resource Utilization
- 31 March 2026
Explore how autoscaling with AMD Inference Microservices (AIMs) and AMD AI Workbench can automatically scale your resources in response to shifting AI workload demand. AI inference can be computationally intensive, with resource requirements that vary depending on traffic e.g., the number of inference requests your workload receives at any given time. Autoscaling addresses this by scaling resources up during peak traffic to maintain performance, and scaling them back down during quieter periods to reduce cost and resource consumption.
Programming Tensor Descriptors in Composable Kernel (CK)
- 25 March 2026
Writing efficient GPU kernels requires more than knowing the API—it demands a deep understanding of the underlying concepts, from GPU architecture to low-level programming patterns. This blog series demystifies GPU kernel programming on AMD GPUs by breaking down common kernels into their fundamental building blocks. Rather than treating GPU programming as a black box, each blog focuses on a specific concept, starting from first principles and building up to complete implementations with simple, insightful example code. In this blog, you will learn one of the most fundamental concepts in Composable Kernel (CK): the TensorDescriptor—a powerful abstraction for managing multi-dimensional data layouts and transformations. By the end of this series, you will be able to not only understand existing GPU kernels but also design and optimize your own.
GROMACS on AMD Instinct GPUs: A Complete Build Guide
- 24 March 2026
Molecular dynamics simulations power breakthroughs in drug discovery, materials science, and computational biology. GROMACS stands as one of the most widely used molecular dynamics engines, and pairing it with AMD’s latest GPU accelerators unlocks exceptional simulation throughput. This guide walks you through installing a complete GROMACS stack with OpenMPI support on AMD MI300X and MI355X systems — whether you’re deploying on bare metal or in containers.
Engineering Qwen-VL for Production: Vision Module Architecture and Optimization Practices
- 24 March 2026
Vision–language models (VLMs) have rapidly evolved from research prototypes into foundational components of modern AI systems, enabling unified reasoning over images, videos, and text. As model scale and application complexity increase, the focus of VLM development has shifted from isolated benchmark performance toward architectural efficiency, multimodal alignment, and production readiness. Within this landscape, Qwen-VL stands out as a practical and extensible vision–language model that emphasizes modular visual encoding, flexible multimodal integration, and scalability in real-world deployments. Rather than treating vision as a peripheral add-on, Qwen-VL adopts a tightly integrated design that allows visual representations to participate deeply in language reasoning, making it particularly well suited for both large-scale inference and domain-specific customization.
Accelerating Kimi-K2.5 on AMD Instinct™ MI300X: Optimizing Fused MoE with FlyDSL
- 24 March 2026
With the recent surge in popularity of OpenClaw [1], its officially recommended model, Kimi-K2.5 [2], has taken the AI community by storm. As developers and researchers flock to this powerful Mixture-of-Experts (MoE) LLM, the need for high-performance inference on cutting-edge hardware has never been more critical.
Edge-to-Cloud Robotics with AMD ROCm: From Data Collection to Real-Time Inference
- 23 March 2026
This blog walks through a full edge-to-cloud robotics AI solution, built entirely on the AMD ecosystem and the Hugging Face LeRobot framework. In case you are not familiar, LeRobot is an open source platform from Hugging Face that provides pre-trained models, datasets, and tools for real-world robotics using PyTorch.
AMD Device Metrics Exporter v1.4.2: Enhanced Observability, Deeper RAS Insights, and Smarter GPU Telemetry for Modern HPC & AI Clusters
- 23 March 2026
Modern GPU‑accelerated systems—whether powering massive AI training workloads or tightly scheduled HPC environments—depend heavily on high‑quality telemetry. Understanding how each GPU behaves under load, how often it hits power or thermal boundaries, and how reliably the hardware performs is central to maintaining performance, diagnosing failures, and tuning systems at scale.
hipBLASLt Online GEMM Tuning
- 19 March 2026
This blog post introduces the integration of hipBLASLt Online GEMM Tuning into LLM frameworks, illustrated through an example implementation of RTP-LLM. Developed by the AMD Quark Team, hipBLASLt Online Tuning provides a user-friendly approach to improving GEMM performance by enabling runtime tuning without requiring additional offline tuning steps.
Utilizing AMD Instinct GPU Accelerators for Weather and Precipitation Forecasting with NeuralGCM
- 19 March 2026
In recent years, the landscape of weather forecasting has evolved tremendously, employing cutting-edge AI technologies to enhance prediction accuracy and speed. In previous blogs, we have demonstrated how to run several state-of-the-art AI weather forecasting models, such as Pangu-Weather, GenCast, and Aurora. Following that, this blog focuses on emerging trends in weather forecasting models, particularly the innovative NeuralGCM, which melds the strengths of General Circulation Models (GCMs) and Machine Learning (ML). We will briefly outline the design of NeuralGCM and its hybrid approach for weather and precipitation forecasting. We will then go through the required environments, installation steps, and the inference process for generating forecasts and creating plots to compare the outputs to the ground truth provided by ERA5 data.
Multi-Node Distributed Inference for Diffusion Models with xDiT
- 18 March 2026
The first two authors (Lehtiranta, Kemppi) contributed equally to this work.
GROMACS Performance on AMD Instinct MI355X
- 13 March 2026
Are you planning a hardware upgrade for your molecular dynamics workflows? In this blog, we benchmark GROMACS on AMD’s latest Instinct MI355X GPU and compare it head-to-head with the MI300X, demonstrating significant throughput improvements that accelerate time-to-results for life-science research. You will see exactly how much faster MI355X runs the standard ADH dodec benchmark across 1 to 8 GPUs. Use these results to make informed decisions about your next HPC deployment.
FP8 GEMM Optimization on AMD CDNA™4 Architecture
- 10 March 2026
This blog post continues our previous blog Matrix Core Programming on AMD CDNA™3 and CDNA™4 Architecture, which introduced Matrix Cores and demonstrated how to use them in HIP kernels.
Getting Started with ComfyUI on AMD Radeon™ RX 9000 Series GPUs
- 09 March 2026
ComfyUI has become a widely adopted and versatile node-based interface for Stable Diffusion and other generative AI models, gaining significant traction within the AI content creation community. Unlike traditional web-based interfaces, ComfyUI provides a node-based workflow system that gives users complete control over their image and video generation pipelines. Its modular architecture allows for complex workflows involving multiple models, LoRAs, ControlNets, and custom processing steps.
Agentic Diagnosis for LLM Training at Scale
- 09 March 2026
In MaxText-Slurm: Production-Grade LLM Training with Built-In Observability, we introduced MaxText-Slurm — an open-source launch system and observability stack for running MaxText LLM training on AMD Instinct GPU clusters. We showed how a unified Prometheus time-series database (TSDB) collects GPU, host, network, and training metrics into a single queryable store, persisted to disk so that no data is lost even if the job crashes.
HPC Coding Agent - Part 3: MCP Tool for Profiling
- 06 March 2026
In this blog, we build an AI agent specialized in profiling and optimizing GPU-accelerated applications within High-Performance Computing (HPC) environments. Using open-source tools, we create a state-of-the-art agent and enhance its profiling capabilities through a custom Model Context Protocol (MCP) server. This server provides the agent with tools to leverage AMD’s profiling utilities for analyzing application performance on AMD GPUs.
Fine-Tuning AI Surrogate Models for Physics Simulations with Walrus on AMD Instinct GPU Accelerators
- 06 March 2026
Physics simulations are used for studying complex systems and are essential where experiments are difficult, expensive, or impossible. In our context, a simulation means numerically solving mathematical equations that are believed to describe a physical system and evolving them forward in time on a computer. They enable controlled exploration of physical behavior for science and engineering, but at a high computational cost, which in most cases increases rapidly with scale. Our focus is on continuum dynamics, where the system is represented by fields such as density, velocity, or temperature, defined on a grid and evolving over time. High-resolution physics simulations are slow to run, sensitive to numerical error and impractical for large parameter spaces. Surrogate models address these limitations by learning to approximate simulation dynamics directly from data. Once trained, they can produce fast predictions at a fraction of the cost, giving researchers the ability to rapidly explore parameter space and generate long rollouts.
Ensemble High-Resolution Weather Forecasting on AMD Instinct GPU Accelerators
- 06 March 2026
Weather prediction is fraught with uncertainty, as is the inference of any real-world phenomena dependent on physical observations. The consequence is that any estimated current state of the atmosphere as well as any forecast both carry a level of uncertainty. As such, any weather forecasting model, whether AI or traditional, needs to produce reasonable outputs despite the inherent uncertainty of inputs, and, if possible, quantify the uncertainty of the outputs for the user in some practical fashion.
HPC Coding Agent - Part 2: An MCP Tool for Code Optimization with OpenEvolve
- 04 March 2026
Large language models (LLMs) and LLM-driven agents (AI agents) are already trained on a massive amount of data where a considerable portion consists of code, and both models and agentic coding services are developed specifically for the purpose of coding. For users who want to optimize their code for certain purposes, for example runtime or memory efficiency, LLMs may produce plausible solutions, but these are often not optimal.
Streamlining Recommendation Model Training on AMD Instinct™ GPUs
- 02 March 2026
Recommendation model training and inference workloads represent a significant portion of computational requirements across industries including e-commerce, social media and content streaming platforms. Unlike LLMs, recommendation models result in to complex and often imbalanced communication across GPUs, along with a higher load on the CPU-GPU interconnect. The ROCm training docker [1] now includes essential libraries for recommendation model training. This blog demonstrates the functionality and ease of training recommendation models using ROCm, along with suggestions for improved configuration of these workloads. We also highlight the inherent benefits of the large HBM size on AMD Instinct™ GPUs for recommendation workloads.
MaxText-Slurm: Production-Grade LLM Training with Built-In Observability
- 02 March 2026
Training large language models (LLMs) at scale on GPU clusters is not just a compute problem — it is an operations problem. Launching multi-node distributed training, keeping it running reliably, and diagnosing failures when they happen all require tooling that most training frameworks do not provide. MaxText-Slurm is an open-source launch system and observability stack that bridges this gap for MaxText on AMD Instinct GPU clusters managed by Slurm.
Exploring Use Cases for Scalable AI: Implementing Ray with ROCm 7 Support for Efficient ML Workflows
- 27 February 2026
This blog builds on insights from our previous blog post, which introduced Ray 2.48.0.post0 running on ROCm 6.2 and demonstrated Reinforcement Learning from Human Feedback (RLHF) with verl 0.3.0.post0 and vLLM 0.6.4 on AMD GPUs. In this follow‑up, we introduce Ray 2.51.1 with ROCm 7.0.0, verl 0.6.0, and vLLM 0.11.0.dev, highlighting the new performance benefits and capabilities for large‑scale RLHF workloads.
PyTorch Offline Tuning with TunableOp
- 24 February 2026
In an earlier blog post, we explored how PyTorch TunableOp can potentially accelerate models through online tuning - where during model execution, PyTorch benchmarks and selects optimal BLAS kernels. While online tuning is effective, it introduces overhead due to the time needed to execute the ML model from end-to-end. If this is done once, the overhead may be acceptable, but for repeated tuning it may be cost-prohibitive to keep re-running the model.
LuminaSFT: Generating Synthetic Fine-Tuning Data for Small Language Models
- 24 February 2026
Small language models (SLMs) are emerging as a lightweight and cost-efficient alternative to large language models (LLMs). They significantly reduce inference costs and latency, and when carefully optimized for specific tasks, can approach—or even match—the performance of larger models. However, due to their limited parameter capacity, SLMs typically require stronger supervision to reach their full potential. Supervised fine-tuning (SFT) therefore plays a crucial role in enhancing their performance.
JAX-AITER: Bringing AMD’s Optimized AI Kernels to JAX on ROCm™
- 24 February 2026
If you’re building large models in JAX on AMD GPUs, you want fast, reliable kernels without spending weeks tuning them yourself. That’s exactly the need that led us to create JAX-AITER.
Getting Started with AMD Resource Manager: Efficient Sharing of AMD Instinct™ GPUs for R&D Teams and AI Practitioners
- 24 February 2026
In this blog, you will learn how to use AMD Resource Manager and its components for centralized AI infrastructure governance. It’s part of the AMD Enterprise AI Suite, a full-stack solution for developing, deploying and running AI workloads on a Kubernetes platform designed to support AMD compute. The AMD Resource Manager provides a user-friendly graphical user interface (GUI) and Command Line Interface (CLI) with a unified control plane that simplifies tasks such as managing compute clusters, user access, monitoring resource utilization, and allocating the right compute quotas to the right projects.
Primus-Pipeline: A More Flexible and Scalable Pipeline Parallelism Implementation
- 23 February 2026
Error parsing meta tag attribute “keywords”: No content.
FlyDSL: Expert GPU Kernel Development with the Ease of MLIR Python Native DSL on AMD GPUs
- 20 February 2026
The AMD ROCm™ software ecosystem continues to grow rapidly as developers build new kernels, compilers, and AI frameworks optimized for AMD GPUs. As workloads become more complex and the demand for both performance and agility increases, a clear need has emerged for a modern, flexible, and open GPU kernel authoring framework.
Introducing hipThreads: A C++ - Style Concurrency Library for AMD GPUs
- 19 February 2026
In this blog, you will learn how to accelerate C++ code developed for the CPU to run on AMD GPUs using hipThreads by incrementally porting familiar
std::thread patterns to GPU-resident hip::thread code.
We walk through a step-by-step SAXPY example, explain key concepts like persistent threads and fibers, and share real performance
results to help you evaluate when this model fits your workload.
Unlocking Sparse Acceleration on AMD GPUs with hipSPARSELt
- 17 February 2026
Sparse computation is a cornerstone of modern AI acceleration. As models like LLaMA and DINOv2 ViT-L scale in size and complexity, the demand for efficient matrix operations becomes increasingly critical. To address this, semi-structured sparsity, also known as the 2:4 structured sparsity pattern, has emerged as a powerful optimization technique.
Advanced MXFP4 Quantization: Combining Fine-Tuned Rotations with SmoothQuant for Near-Lossless Compression
- 17 February 2026
As language models continue to grow in popularity, reducing the cost of inference and accelerating model serving have become key challenges. Quantization offers a powerful solution by reducing the model size and leveraging inexpensive math operations, for example, using low-bitwidth formats like OCP MXFP4 (4.25 bits) available in AMD Instinct MI350X and MI355X accelerators.
Adaptive Top-K Selection: Eliminating Performance Cliffs Across All K Values on AMD GPUs
- 17 February 2026
Top-K selection is critical for LLMs and RAG workloads, yet standard Radix Sort implementations often suffer from performance cliffs at small K values due to fixed initialization overheads. In our AITER library (introduced in our previous blog [1]), we originally utilized an 11-bit radix sort for Top-K selection. While this approach excels at scale, we identified a critical efficiency gap for the lightweight filtering often required during modern inference.
Elevate Your LLM Inference: Autoscaling with Ray, ROCm 7.0.0, and SkyPilot
- 13 February 2026
This blog explores autoscaling of inference workloads in Ray Serve with a vLLM backend on AMD Instinct™ GPUs for large language models (LLMs). Furthermore, you will learn how to scale beyond a single cluster using SkyPilot, which enables multicloud scaling for Ray Serve. Combined with the AMD ROCm™ software platform, this creates a unified, cloud-agnostic platform that scales distributed LLM inference from single-GPU to multi-cluster deployments.
Reinforcement Learning from Human Feedback on AMD GPUs with verl and ROCm 7.0.0
- 12 February 2026
In our previous blog post, we introduced Volcano Engine Reinforcement Learning for LLMs (verl) 0.3.0.post0 with ROCm 6.2 and vLLM 0.6.4. In this blog post, we will provide you with an overview of verl 0.6.0 with ROCm 7.0.0 and vLLM 0.11.0.dev and its benefits for large-scale reinforcement learning from human feedback (RLHF). You will also learn about the modifications made to optimize verl performance on AMD Instinct™ MI300X GPUs. Next, you will walk through building the Docker image on your system, along with training scripts for single-node and multi-node setups. Lastly, we provide you with verl performance results, focusing on throughput and convergence accuracy achieved on AMD Instinct MI300X GPUs. Follow this guide to get started with verl on AMD Instinct GPUs and accelerate your RLHF training with ROCm-optimized performance.
Solution Blueprints: Accelerating AI Deployment with AMD Enterprise AI
- 11 February 2026
AMD Enterprise AI Suite standardizes the inference layer with AMD Inference Microservices (AIMs), a set of containers for optimized model serving on AMD Instinct™ GPUs with validated profiles and OpenAI-compatible APIs. However, production grade agentic and generative AI applications need more than inference endpoints. You need document loaders, embedding pipelines, vector databases, RAG logic, agent orchestration, and user interfaces. These components need to be wired together with proper Kubernetes resource definitions, GPU allocation, service discovery, and configuration management. This blog walks through the technical implementation of Solution Blueprints: how they’re structured, how they use Helm application charts for code reuse, and the patterns they demonstrate for multi-container orchestration. While the Enterprise AI Suite Overview covers the platform and the AIMs blog covers inference, this post focuses on application architecture and deployment patterns.
Digital Twins on AMD: Building Robotic Simulations Using Edge AI PCs
- 09 February 2026
Digital twins are becoming a core tool in robotics, automation, and intelligent systems. They provide a virtual representation of a physical system, allowing developers to validate robot behaviors, test motion strategies, and generate datasets before deploying anything in the real world.
Building Robotics Applications with Ryzen AI and ROS 2
- 09 February 2026
This blog showcases how to deploy power-efficient Ryzen AI perception models with ROS 2 - the Robot Operating System. We utilize the Ryzen AI Max+ 395 (Strix-Halo) platform, which is equipped with an efficient Ryzen AI NPU and iGPU. The Ryzen AI CVML Library is used to deploy supported models efficiently on the Ryzen AI platform. All of the code is available on GitHub in the AMD Ryzers repository and was originally presented at ROSCon’25.
Resilient Large-Scale Training: Integrating TorchFT with TorchTitan on AMD GPUs
- 08 February 2026
Training large AI models on AMD GPUs demands unwavering stability and robust fault-tolerance capabilities at cluster scale. Yet today’s ROCm-based multi-node GPU deployments often rely on brittle checkpoint-and-restart mechanisms to recover from failures. This approach wastes precious compute cycles and slows down training as model sizes and cluster scales grow. To address these challenges, we integrated PyTorch’s native fault-tolerance framework—TorchFT—with the TorchTitan training framework on AMD’s Primus-SaFE Kubernetes platform, achieving resilient, checkpoint-less training at hundred-GPU scale. This blog builds upon our previous work on the Primus ecosystem—for background on the platform architecture, see our earlier posts on Primus-SaFE, the Primus training framework, and training large models with Primus.
Accelerating Graph Layout with AI and ROCm on AMD GPUs
- 06 February 2026
Learn how easy it is to implement established graph algorithms, and deploy them on AMD GPUs with immediate performance improvements, using AI as a coding partner!
Micro-World: First AMD Open-Source World Models for Interactive Video Generation
- 05 February 2026
World models aim to simulate aspects of the real world, enabling more effective training and exploration of AI agents and ultimately paving the way toward richer forms of digital life. Games can be viewed as another form of world simulation, and their data is relatively easy to collect and annotate, making them a natural playground for building and studying world models. GameNGen [1] has demonstrated the potential of this direction, while works such as GameFactory [2], Matrix-Game [3], and Hunyuan-GameCraft [4] further showcase strong performance in game-oriented world modeling. However, these projects are either fully closed-sourced or release only partial components (typically inference-only), which limits reproducibility and community-driven progress.
Foundations of Molecular Generation with GP-MoLFormer on AMD Instinct MI300X Accelerators
- 03 February 2026
Nearly every technological breakthrough we celebrate begins with a material that did not exist before someone imagined it. Modern computing rests on engineered semiconductors, energy storage depends on carefully designed electrolytes, and sustainable technologies increasingly rely on alternatives to scarce or environmentally costly rare earth elements. Designing such materials with specific properties at scale is one of the most challenging and consequential problems in science.
Debugging NaN Results in CK Tile GEMM: A rocgdb Detective Story
- 30 January 2026
When developing high-performance GPU kernels, subtle bugs can lead to catastrophic failures like NaN (Not-a-Number) outputs. This post chronicles our journey of debugging a tricky NaN issue in AMD’s Composable Kernel (CK) Tile GEMM implementation using rocgdb. What started as mysterious NaN outputs ended with discovering a single-character typo that corrupted the data distribution.
ROCm 7.2: Smarter, Faster, and More Scalable for Modern AI Workloads
- 22 January 2026
Modern AI workloads demand more than raw compute—they require a tightly integrated software stack that can extract maximum performance, scale efficiently across systems, and operate reliably in production environments. With the latest ROCm 7.2 release, we’re delivering a broad set of optimizations and software enhancements designed to improve developer productivity, runtime performance, and enterprise readiness.
Nitro-AR: A Compact AR Transformer for High-Quality Image Generation
- 22 January 2026
Recent years have witnessed remarkable progress in image generation, driven by two major modeling paradigms: diffusion-based models and autoregressive (AR) models. Building upon our previously released Nitro-E, a light-weight diffusion model for fast image synthesis, this blog explores a complementary direction, applying the architecture in an AR framework.
LLM Inference Optimization Using AMD GPU Partitioning
- 22 January 2026
As AI and HPC workloads grow in complexity and scale, there’s a rising need for precise GPU resource management, robust memory isolation, and efficient multi-tenant scheduling. AMD’s Instinct™ MI300 series addresses this by offering dynamic partitioning capabilities. These allow a single physical device to be segmented into multiple isolated partitions, each tailored to the needs of specific workloads. This flexibility is particularly beneficial for AI inference tasks, where different models or instances may require distinct resource allocations. Maximizing the utilization of GPU resources while ensuring that each workload operates within its own isolated environment is crucial for performance and reliability.
ROCm Becomes a First-Class Platform in the vLLM Ecosystem
- 21 January 2026
As the generative AI ecosystem matures, vLLM embraces a multivendor ecosystem. The quality of support across hardware platforms becomes a defining priority: developers expect consistent, high-performance behavior no matter which GPU they choose. Today, we are proud to announce a major realization of that vision: AMD ROCm™ is now a first-class platform in the vLLM ecosystem.
Quickly Developing Powerful Flash Attention Using TileLang on AMD Instinct MI300X GPU
- 20 January 2026
Against the backdrop of the rapid development of the AMD ROCm™ software ecosystem, the high barrier to operator development has long been a bottleneck. The emergence of TileLang provides developers with an efficient solution. As an emerging AI operator development framework, tilelang encapsulates low-level GPU details with concise syntax, enabling developers to fully tap into the computing potential of AMD GPUs without requiring in-depth knowledge of low-level languages such as HIP. The AMD Instinct™ MI300X GPU, as a flagship GPU for AI workloads, boasts ultra-high bandwidth memory and powerful compute units, but it requires adaptive high-performance operators to unleash its capabilities. In this blog, we will take Flash Attention, a key kernel in both LLM training and inference, as an example to fully demonstrate the development process based on TileLang on the MI300X, highlighting the dual benefits of efficiency and performance that TileLang brings to AMD operator development.
Deep Dive into Primus: High-Performance Training for Large Language Models
- 15 January 2026
Primus is the AMD unified training framework designed to deliver high-performance, scalable large language models (LLMs) training across multiple backends – including TorchTitan and Megatron-LM. It provides a consistent CLI interface, while each backend ships with carefully optimized configurations for popular open-source models. These backend-specific presets ensure the best out-of-the-box performance on AMD Instinct™ GPUs. In this deep dive, we walk through the best practices for achieving peak performance when training dense LLMs on Primus.
Applying Compute Partitioning for Workloads on MI300X GPUs
- 14 January 2026
This blog explains how to use AMD GPU compute partitioning to increase throughput, utilization and reduce time-to-results for two different types of workloads:
Reimagining GPU Allocation in Kubernetes: Introducing the AMD GPU DRA Driver
- 13 January 2026
In this blog, you’ll learn how Kubernetes’ new Dynamic Resource Allocation (DRA) framework and the AMD GPU DRA Driver turn GPUs into first-class, attribute-aware resources. We’ll walk through how to publish AMD Instinct GPUs via ResourceSlices, request specific models and partition profiles with declarative ResourceClaims, and observe allocations through Kubernetes-native lifecycle objects, so you can simplify cluster operations compared to traditional Device Plugin–based setups.
Installing AMD HIP-Enabled GROMACS on HPC Systems: A LUMI Supercomputer Case Study
- 12 January 2026
Running molecular dynamics (MD) simulations efficiently is critical for accelerating scientific discovery in many life science use cases, e.g., drug discovery. GROMACS is a widely used, GPU-accelerated molecular dynamics engine powering many life science workflows, but its performance can vary significantly depending on the installation method and hardware configuration. For broader context on GROMACS applications in drug design, see recent research on GROMACS in cloud environments for alchemical drug design.
Athena-PRM: Enhancing Multimodal Reasoning with Data-Efficient Process Reward Models
- 12 January 2026
This blog introduces Athena-PRM, a multimodal Process Reward Model (PRM) designed to evaluate the reward score for each step in solving complex reasoning problems. To efficiently generate high-quality process-labeled data, we leverage prediction consistency between weak and strong completers as a criterion for identifying reliable process labels. We also develop two effective strategies to improve the performance of PRMs: ORM initialization and up-sampling for negative data.
Using Gradient Boosting Libraries on MI300X for Financial Risk Prediction
- 08 January 2026
In the world of machine learning, the choice of hardware can significantly impact the performance and efficiency of model training and prediction. Gradient Boosting Machines (GBMs) benefit greatly from GPU parallelization in several key algorithmic steps involving independent, repetitive computations. The most substantial speedup comes from histogram construction and best split searching, as these can be executed in parallel across features and candidate splits using thousands of GPU cores, vastly accelerating tree building. Additionally, the calculation of gradients and Hessians for each data point is naturally parallelizable and well suited to GPU architectures. Other operations—such as leaf value updates, data preprocessing (like quantization and normalization), and batch predictions—can also be distributed efficiently across GPU threads. By exploiting parallelism in these stages, GPUs dramatically reduce training and prediction time for GBMs, making them ideal for large datasets or scenarios where quick model iteration is crucial.
Introducing the AMD Network Operator v1.0.0: Simplifying High-Performance Networking for AMD Platforms
- 08 January 2026
In this blog, you will learn how the AMD Network Operator simplifies high-performance networking for AMD GPU clusters, automates NIC discovery and configuration, supports RDMA/RoCE workloads, and provides real-time monitoring to keep your AI/ML and HPC jobs running efficiently.
Bridging the Last Mile: Deploying Hummingbird-XT for Efficient Video Generation on AMD Consumer-Grade Platforms
- 08 January 2026
*The first three authors (Isobe, Cui, and Ge) contributed equally to this work.
High-Resolution Weather Forecasting with StormCast on AMD Instinct GPU Accelerators
- 07 January 2026
The traditional approach to numerical weather prediction is based on propagating a known atmospheric state forward in time in short steps using systems of partial differential equations directly obtained from physical considerations. A new approach is to use machine learning methods to directly proceed to a later state in one large step, typically moving forward multiple hours in a single jump.
Breaking the Accuracy-Speed Barrier: How MXFP4/6 Quantization Revolutionizes Image and Video Generation
- 07 January 2026
This blog introduces MXFP4 and MXFP6, the newly supported data types on AMD Instinct™ MI350 Series GPUs, and demonstrates their remarkable quality in image and video generation tasks. By reading this blog, you will discover how these low-bit formats can break the accuracy-speed tradeoff, boosting both efficiency and performance in generative AI workflows.
ROCm MaxText Testing — Decoupled (Offline) and Cloud-Integrated Modes
- 06 January 2026
In this blog, you will learn how to run MaxText unit tests on AMD ROCm GPUs in two complementary modes: offline (decoupled) and fully cloud-integrated. By the end, you will know when to use each mode, how to interpret the results, and how to fold them into your CI and debugging workflows.
ROCm Fork of MaxText: Structure and Strategy
- 06 January 2026
In this blog you will explore how the ROCm fork of MaxText is structured and how that structure supports ROCm and fully offline, decoupled workflows across platforms.
SparK: Query-Aware Unstructured Sparsity with Recoverable KV Cache Channel Pruning
- 02 January 2026
In this blog we will discuss SparK, a training-free, plug-and-play method for KV cache compression in large language models (LLMs). By addressing the overlooked redundancy in feature channels and employing a “prune-and-recover” strategy, SparK reduces KV cache storage by over 30% compared to traditional methods while maintaining model accuracy. It offers a robust solution for long-context inference, establishing a new perspective on unstructured sparsity.
Accelerating Multimodal Inference in vLLM: The One-Line Optimization for Large Multimodal Models
- 02 January 2026
Deploying multimodal models like Qwen3-VL or InternVL at scale reveals a hidden bottleneck. While Tensor Parallelism (TP) is essential for massive language decoders, it is often overkill for vision encoders. These encoders are typically small, often just 1-5% of total model size, so there is limited compute benefit from sharding them. However, they still incur expensive all-reduce communication costs after every single layer.