AMD ROCm Blogs

Using ODC to Accelerate AMD SFT Training

2026-08-12T00:00:00+00:00

Large-scale training spends a surprising share of its wall-clock time waiting instead of computing. Under Fully Sharded Data Parallel (FSDP), every layer ends in a collective all-gather or reduce-scatter, and every collective is a barrier that the whole data-parallel group has to reach together. Feed that machinery variable-length supervised fine-tuning (SFT) data and the picture gets worse: some ranks draw long documents while others draw short ones, so the fast ranks sit idle waiting for the slow ones.

Quark Support for HuggingFace Diffusers and SVDQuant

2026-08-06T00:00:00+00:00

Diffusion models are heavy on memory and compute: a single text-to-image call runs a large transformer or UNet dozens of times. Quantization — storing weights (and sometimes activations) in low precision — is one of the most effective ways to cut both the memory footprint and the latency of these models.

AUP Learning Cloud: Streamlining AI Education on AMD

2026-08-06T00:00:00+00:00

AI education is becoming increasingly hands-on. Students are now expected to train models, build AI agents, experiment with large language models (LLMs), and develop applications accelerated by graphics processing units (GPUs), so courses depend as much on practical computing infrastructure as on learning materials. Yet standing up that infrastructure is usually the real bottleneck, and educators often spend more effort building it than teaching.

VSA: Accelerating Video Diffusion Inference with Sparse Attention on AMD GPUs

2026-08-04T00:00:00+00:00

Video generation powered by diffusion transformers has achieved remarkable quality, but the computational cost of attention mechanisms remains a critical bottleneck. With sequence lengths reaching tens of thousands of tokens in video generation tasks, the quadratic complexity of standard attention becomes prohibitively expensive.

Reverse-Engineering hipBLASLt TensileLite Kernels: From Solution Name to a Tuning Config

2026-08-04T00:00:00+00:00

In a previous blog, Customizing Kernels with hipBLASLt TensileLite GEMM Tuning, we showed how TensileLite Tuning generates brand-new GEMM kernels by searching a parameter space and selecting the fastest valid candidate for a given problem size. The search space is defined entirely by the tuning configuration YAML you provide. Its ForkParameters lists are expanded into a Cartesian product, and the tuning run can only ever return a kernel that is expressible as one combination within that product.

Introducing AMD CDNA™ 5 and the AMD Helios™ Rackscale Solution

2026-08-04T00:00:00+00:00

AI infrastructure is being redefined at the scale of the entire rack. This blog will walk you through how the AMD CDNA™ 5 architecture, the AMD Instinct™ MI455X GPU, and the AMD Helios rackscale solution come together as a single open platform for large-scale AI training and inference. You will see how AMD balances compute, memory, and networking across 72 GPUs, how open standards keep the platform flexible, and how the AMD ROCm™ software stack ties it all into a complete, deployable system, from silicon to rack. By the end, you will understand what changes when AI infrastructure is designed as a unified rack rather than a collection of discrete GPU servers, and where AMD is taking it next.

Closing the GPU Cluster Validation Gap: A Kubernetes-Native Approach with CVF

2026-07-29T00:00:00+00:00

As GPU clusters scale from dozens to thousands of accelerators, the assumption that every node is healthy at boot time becomes increasingly risky. Silent hardware degradation — a marginal XGMI link, a missing SR-IOV Virtual Function, or an under-performing RDMA rail — can turn a single node into a straggler that slows an entire distributed training job.

AMD GPU Operator v1.5.0: DRA Support, Automated GPU Node Recovery, and Expanded Kubernetes Infrastructure Control

2026-07-28T00:00:00+00:00

GPU Operator v1.5.0 introduces several major infrastructure capabilities for Kubernetes-based AMD GPU deployments, including support for Kubernetes Dynamic Resource Allocation (DRA), automated GPU node remediation workflows, and Node Problem Detector (NPD) integration.

Attention Decode on AMD MI450 GPUs: A Gluon Kernel Optimization Guide

2026-07-27T00:00:00+00:00

Agentic AI applications are pushing LLM inference into a new regime. A request can now reach one million tokens from aggregated long prompts, tool calls, retrieval results, and multi-turn reasoning. During the text generation phase, each new token must attend to all previous tokens from the KV cache, so the kernel needs to repeatedly read past states from HBM. As a result, the performance bottleneck shifts from compute units to the memory system.

Introducing Instella-MoE: A State-of-the-Art Fully Open Mixture-of-Experts Language Model

2026-07-24T00:00:00+00:00

AMD is excited to introduce Instella-MoE, a state-of-the-art fully open Mixture-of-Experts (MoE) language model with 16 billion total parameters and 2.8 billion active parameters. Trained from scratch on AMD Instinct™ MI300X and MI325X GPUs with the AMD ROCm™ software stack, Instella-MoE combines a sparsely activated MoE design with architectural innovations such as Gated Multi-head Latent Attention (Gated MLA) and FarSkip-Collective connectivity. Instella-MoE delivers competitive performance across a broad suite of benchmarks against both dense and MoE baselines (as shown in Figure 1) including models with comparable or larger active parameter counts, establishing it as one of the strongest fully open language models at its scale.