Recent Posts#
Hands-On with CK-Tile: Develop and Run Optimized GEMM on AMD GPUs
Build high-performance GEMM kernels using CK-Tile on AMD Instinct GPUs with vendor-optimized pipelines and policies for AI and HPC workloads
Installing ROCm from source with Spack
Install ROCm and PyTorch from source using Spack. Learn how to optimize builds, manage dependencies, and streamline your GPU software stacks.
ROCm 6.4: Breaking Barriers in AI, HPC, and Modular GPU Software
Explore ROCm 6.4's key advancements: AI/HPC performance boosts, enhanced profiling tools, better Kubernetes support and modular drivers, accelerating AI and HPC workloads on AMD GPUs.
ROCm Gets Modular: Meet the Instinct Datacenter GPU Driver
We introduce the new Instinct driver-a modular GPU driver with independent releases simplifying workflows, system setup, and enhancing compatibility across toolkit versions.
Unlock Peak Performance on AMD GPUs with Triton Kernel Optimizations
Learn how Triton compiles and optimizes AI kernels on AMD GPUs, with deep dives into IR flows, hardware-specific passes, and performance tuning tips
Shrink LLMs, Boost Inference: INT4 Quantization on AMD GPUs with GPTQModel
Learn how to compress LLMs with GPTQModel and run them efficiently on AMD GPUs using INT4 quantization, reducing memory use, shrinking model size, and enabling fast inference
Power Up Llama 4 with AMD Instinct: A Developer’s Day 0 Quickstart
Explore the power of Meta’s Llama 4 multimodal models on AMD Instinct™ MI300X and MI325X GPUs - available from Day 0 with seamless vLLM integration
AMD Instinct™ MI325X GPUs Produce Strong Performance in MLPerf Inference v5.0
We showcase MI325X GPU optimizations that power our MLPerf v5.0 results on Llama 2 70B, highlighting performance tuning, quantization, and vLLM advancements.
Reproducing the AMD InstinctTM GPUs MLPerf Inference v5.0 Submission
A step-by-step guide to reproducing AMD’s MLPerf v5.0 results for Llama 2 70B & SDXL using ROCm on MI325X
Bring FLUX to Life on MI300X: Run and Optimize with Hugging Face Diffusers
The blog will walk you through the FLUX text-to-image diffusion model architecture and show you how to run and optimize it on MI300x.
What's New in the AMD GPU Operator v1.2.0 Release
This blog highlights the new feature enhancements that were released as part of the AMD GPU Operator v1.2.0 release. New features that enhance the use of AMD Instinct GPUs on Kubernetes including Automated Upgrades, Health Checks and Open-sourcing the codebase.
Accelerating LLM Inference: Up to 3x Speedup on MI300X with Speculative Decoding
This blog demonstrates out-of-the-box performance improvement in LLM inference using speculative decoding on MI300X.
Introducing ROCprofiler SDK - The Latest Toolkit for Performance Profiling
Discover ROCprofiler SDK – ROCm’s next-generation, unified, scalable, and high-performance profiling toolkit for AI and HPC workloads on AMD GPUs.
Speculative Decoding - Deep Dive
This blog shows the performance improvement achieved by applying speculative decoding with Llama models on AMD MI300X GPUs, tested across models, input sizes, and datasets.
Efficient MoE training on AMD ROCm: How-to use Megablocks on AMD GPUs
Learn how to use Megablocks to pre-train GPT2 Mixture of Experts (MoE) model, helping you scale your deep learning models effectiveness on AMD GPUs using ROCm
Supercharge DeepSeek-R1 Inference on AMD Instinct MI300X
Learn how to optimize DeepSeek-R1 on AMD MI300X with SGLang, AITER kernels and hyperparameter tuning for up to 5× throughput and 60% lower latency over Nvidia H200
AITER: AI Tensor Engine For ROCm
We introduce AMD's AI Tensor Engine for ROCm (AITER), our centralized high performance AI operators repository, designed to significantly accelerate AI workloads on AMD GPUs
Deploying Google’s Gemma 3 Model with vLLM on AMD Instinct™ MI300X GPUs: A Step-by-Step Guide
AMD is excited to announce the integration of Google’s Gemma 3 models with AMD Instinct™ MI300X GPUs
Analyzing the Impact of Tensor Parallelism Configurations on LLM Inference Performance
This blog analyzes how tensor parallelism impacts TCO and Scale for LLM deployments in production.
AI Inference Orchestration with Kubernetes on Instinct MI300X, Part 3
This blog is part 3 of a series aimed at providing a comprehensive, step-by-step guide for deploying and scaling AI inference workloads with Kubernetes and the AMD GPU Operator on the AMD Instinct platform
Optimized ROCm Docker for Distributed AI Training
AMD updated Docker images incorporate torchtune finetuning, FP8 support, single node performance boost, bug fixes & updated benchmarking for stable, efficient distributed training
AMD Advances Enterprise AI Through OPEA Integration
We announce AMD’s support of Open Platform for Enterprise AI (OPEA), integrating OPEA’s enterprise GenAI framework with AMD’s computing hardware and ROCm software
Instella-VL-1B: First AMD Vision Language Model
We introduce Instella-VL-1B, the first AMD vision language model for image understanding trained on MI300X GPUs, outperforming fully open-source models and matching or exceeding many open-weight counterparts in general multimodal benchmarks and OCR-related tasks.
Introducing Instella: New State-of-the-art Fully Open 3B Language Models
AMD is excited to announce Instella, a family of fully open state-of-the-art 3-billion-parameter language models (LMs). , In this blog we explain how the Instella models were trained, and how to access them.
Understanding RCCL Bandwidth and xGMI Performance on AMD Instinct™ MI300X
The blog explains the reasons behind RCCL bandwidth limitations and xGMI performance constraints, and provides actionable steps to maximize link efficiency on AMD MI300X
Measuring Max-Achievable FLOPs – Part 2
AMD measures Max-Achievable FLOPS through controlled benchmarking: real-world data patterns, thermally stable devices, and cold cache testing—revealing how actual performance differs from theoretical peaks.
Deploying Serverless AI Inference on AMD GPU Clusters
This blog helps targeted audience in setting up AI inference serverless deployment in a kubernetes cluster with AMD accelerators. Blog aims to provide a comprehensive guide for deploying and scaling AI inference workloads on serverless infrastructre.
Unlock DeepSeek-R1 Inference Performance on AMD Instinct™ MI300X GPU
This blog introduces the key performance optimizations made to enable DeepSeek-R1 Inference
How to Build a vLLM Container for Inference and Benchmarking
This post, the second in a series, provides a walkthrough for building a vLLM container that can be used for both inference and benchmarking.
Fine-tuning Phi-3.5-mini LLM at scale: Harnessing Accelerate and Slurm for multinode training
Fine-tuning Phi-3.5-mini-instruct LLM using multinode distributed training with Hugging Face Accelerate, Slurm, and Docker for scalable efficiency.
AI Inference Orchestration with Kubernetes on Instinct MI300X, Part 2
This blog is part 2 of a series aimed at providing a comprehensive, step-by-step guide for deploying and scaling AI inference workloads with Kubernetes and the AMD GPU Operator on the AMD Instinct platform
Understanding Peak, Max-Achievable & Delivered FLOPs, Part 1
Understanding Peak, Max-Achievable & Delivered FLOPs
Navigating vLLM Inference with ROCm and Kubernetes
Quick introduction to Kubernetes (K8s) and a step-by-step guide on how to use K8s to deploy vLLM using ROCm.
PyTorch Fully Sharded Data Parallel (FSDP) on AMD GPUs with ROCm
This blog guides you through the process of using PyTorch FSDP to fine-tune LLMs efficiently on AMD GPUs.
Deep dive into the MI300 compute and memory partition modes
This blog explains how to use the MI300 compute and memory partitioning modes to optimize your performance-critical applications.
MI300A - Exploring the APU advantage
This blog post introduces the MI300 APU hardware, how it differs from other discrete systems, and how to leverage its GPU programming
AI Inference Orchestration with Kubernetes on Instinct MI300X, Part 1
This blog is part 1 of a series aimed at providing a comprehensive, step-by-step guide for deploying and scaling AI inference workloads with Kubernetes and the AMD GPU Operator on the AMD Instinct platform
GEMM Kernel Optimization For AMD GPUs
Guide to how GEMMs can be tuned for optimal performance of AI models on AMD GPUs
Enhancing AI Training with AMD ROCm Software
AMD's GPU training optimizations deliver peak performance for advanced AI models through ROCm software stack.
Best practices for competitive inference optimization on AMD Instinct™ MI300X GPUs
Learn how to optimize large language model inference using vLLM on AMD's MI300X GPUs for enhanced performance and efficiency.
Announcing the AMD GPU Operator and Metrics Exporter
This post announces the AMD GPU Operator for Kubernetes and and the Device Metrics Exporter, including instructions for getting started with these new releases.
Distributed fine-tuning of MPT-30B using Composer on AMD GPUs
This blog uses Composer, a distributed framework, on AMD GPUs to fine-tune MPT-30B in single node as well as multinode
Vision Mamba on AMD GPU with ROCm
This blog explores Vision Mamba (Vim), an innovative and efficient backbone for vision tasks and evaluate its performance on AMD GPUs with ROCm.
Getting started with AMD ROCm containers: from base images to custom solutions
This post, the second in a series, provides a walkthrough for building a vLLM container that can be used for both inference and benchmarking.
Boosting Computational Fluid Dynamics Performance with AMD Instinct™ MI300X
The blog introduces CFD Ansys Fluent benchmarks and provides hands-on guide on installing and running four different Fluent models on AMD GPUs using ROCm.
Triton Inference Server with vLLM on AMD GPUs
This blog provides a how-to guide on setting up a Triton Inference Server with vLLM backend powered by AMD GPUs, showcasing robust performance with several LLMs
Training Transformers and Hybrid models on AMD Instinct MI300X Accelerators
This blog shows Zyphra's new training kernels for transformers and hybrid models on AMD Instinct MI300X accelerators, surpassing the H100s performance
Transformer based Encoder-Decoder models for image-captioning on AMD GPUs
The blog introduces image captioning and provides hands-on tutorials on three different Transformer-based encoder-decoder image captioning models: ViT-GPT2, BLIP, and Alpha- CLIP, deployed on AMD GPUs using ROCm.
SGLang: Fast Serving Framework for Large Language and Vision-Language Models on AMD Instinct GPUs
Discover SGLang, a fast serving framework designed for large language and vision-language models on AMD GPUs, supporting efficient runtime and a flexible programming interface.
Quantized 8-bit LLM training and inference using bitsandbytes on AMD GPUs
Learn how to use bitsandbytes’ 8-bit representations techniques, 8-bit optimizer and LLM.int8, to optimize your LLMs training and inference using ROCm on AMD GPUs
Introducing AMD's Next-Gen Fortran Compiler
In this post we present a brief preview of AMD's Next-Gen Fortran Compiler, our new open source Fortran complier optimized for AMD GPUs using OpenMP offloading, offering direct interface to ROCm and HIP.
Distributed Data Parallel Training on AMD GPU with ROCm
This blog demonstrates how to speed up the training of a ResNet model on the CIFAR-100 classification task using PyTorch DDP on AMD GPUs with ROCm.
Torchtune on AMD GPUs How-To Guide: Fine-tuning and Scaling LLMs with Multi-GPU Power
Torchtune is a PyTorch library that enables efficient fine-tuning of LLMs. In this blog we use Torchtune to fine-tune the Llama-3.1-8B model for summarization tasks using LoRA and showcasing scalable training across multiple GPUs.
CTranslate2: Efficient Inference with Transformer Models on AMD GPUs
Optimizing Transformer models with CTranslate2 for efficient inference on AMD GPUs
Inference with Llama 3.2 Vision LLMs on AMD GPUs Using ROCm
Meta's Llama 3.2 Vision models bring multimodal capabilities for vision-text tasks. This blog explores leveraging them on AMD GPUs with ROCm for efficient AI workflows.
Speed Up Text Generation with Speculative Sampling on AMD GPUs
This blog will introduce you to assisted text generation using Speculative Sampling. We briefly explain the principles underlying Speculative Sampling and demonstrate its implementation on AMD GPUs using ROCm.
Multinode Fine-Tuning of Stable Diffusion XL on AMD GPUs with Hugging Face Accelerate and OCI's Kubernetes Engine (OKE)
This blog demonstrates how to set-up and fine-tune a Stable Diffusion XL (SDXL) model in a multinode Oracle Cloud Infrastructure’s (OCI) Kubernetes Engine (OKE) on a cluster of AMD GPUs using ROCm
Enhancing vLLM Inference on AMD GPUs
In this blog, we’ll demonstrate the latest performance enhancements in vLLM inference on AMD Instinct accelerators using ROCm. In a nutshell, vLLM optimizes GPU memory utilization, allowing more efficient handling of large language models (LLMs) within existing hardware constraints, maximizing throughput and minimizing latency.
Supercharging JAX with Triton Kernels on AMD GPUs
In this blog post we guide you through developing a fused dropout activation kernel for matrices in Triton, calling the kernel from JAX, and benchmarking its performance.
Leaner LLM Inference with INT8 Quantization on AMD GPUs using PyTorch
This blog demonstrates how to use AMD GPUs to implement and evaluate INT8 quantization, and the derived inference speed-up of Llama family and Mistral LLM models.
Fine-tuning Llama 3 with Axolotl using ROCm on AMD GPUs
This blog demonstrates how to fine-tune Llama 3 with Axolotl using ROCm on AMD GPUs, and how to evaluate the performance of your LLM before and after fine-tuning.
Inferencing and serving with vLLM on AMD GPUs
Inferencing and Serving with vLLM on AMD GPUs
Getting to Know Your GPU: A Deep Dive into AMD SMI
This post introduces AMD System Management Interface (amd-smi), explaining how you can use it to access your GPU’s performance and status data
Presenting and demonstrating the use of the ROCm Offline Installer Creator, a tool enabling simple deployment of ROCm in disconnected environments in high-security environments and air-gapped networks.
Presenting and demonstrating the use of the ROCm Offline Installer Creator, a tool enabling simple deployment of ROCm in disconnected environments in high-security environments and air-gapped networks.
Optimize GPT Training: Enabling Mixed Precision Training in JAX using ROCm on AMD GPUs
Guide to modify our JAX-based nanoGPT model for mixed-precision training, optimizing speed and efficiency on AMD GPUs with ROCm.
Image Classification with BEiT, MobileNet, and EfficientNet using ROCm on AMD GPUs
Image Classification with BEiT, MobileNet, and EfficientNet on AMD GPU
Seismic stencil codes - part 1
Seismic Stencil Codes - Part 1: Seismic workloads in the HPC space have a long history of being powered by high-order finite difference methods on structured grids. This trend continues to this day.
Seismic stencil codes - part 2
Seismic Stencil Codes - Part 2: In the previous post, recall that the kernel with stencil computation in the z-direction suffered from low effective bandwidth. This low performance comes from generating substantial amounts of data to movement to global memory.
Seismic stencil codes - part 3
Seismic Stencil Codes - Part 3: In the last two blog posts, we developed a HIP kernel capable of computing high order finite differences commonly needed in seismic wave propagation.
Benchmarking Machine Learning using ROCm and AMD GPUs: Reproducing Our MLPerf Inference Submission
Benchmarking Machine Learning using ROCm and AMD GPUs: Reproducing Our MLPerf Inference Submission
Performing natural language processing tasks with LLMs on ROCm running on AMD GPUs
Performing natural language processing tasks with LLMs on ROCm running on AMD GPUs
Using AMD GPUs for Enhanced Time Series Forecasting with Transformers
Time series forecasting (TSF) predicts future behavior using past data. This guide focuses on implementing Transformers for TSF, covering preprocessing to evaluation using AMD hardware.
Inferencing with Grok-1 on AMD GPUs
We demonstrate that the massive Grok-1 Model from xAI can run seamlessly on the AMD MI300X GPU accelerator by leveraging the ROCm software platform.
Optimizing RoBERTa: Fine-Tuning with Mixed Precision on AMD
In this blog we explore how to fine-tune the Robustly Optimized BERT Pretraining Approach RoBERTa large language model, with emphasis on PyTorch's mixed precision capabilities. Specifically, we explore using AMD GPUs for mixed precision fine-tuning to achieve faster model training without any major impacts on accuracy.
Graph analytics on AMD GPUs using Gunrock
Graph analytics on AMD GPUs using Gunrock
Using statistical methods to reliably compare algorithm performance in large generative AI models with JAX Profiler on AMD GPUs
Using Statistical Methods to Reliably Compare Algorithm Performance in Large Generative AI Models with JAX Profiler on AMD GPUs
Accelerate PyTorch Models using torch.compile on AMD GPUs with ROCm
Accelerate PyTorch Models using torch.compile on AMD GPUs with ROCm
Accelerating models on ROCm using PyTorch TunableOp
Accelerating models on ROCm using PyTorch TunableOp
A Guide to Implementing and Training Generative Pre-trained Transformers (GPT) in JAX on AMD GPUs
A Guide to Implementing and Training Generative Pre-trained Transformers (GPT) in JAX on AMD GPUs
Deep Learning Recommendation Models on AMD GPUs
Deep Learning Recommendation Model on AMD GPU
Fine-tuning and Testing Cutting-Edge Speech Models using ROCm on AMD GPUs
This blog post demonstrates how to fine-tune and test three state-of-the-art machine learning Automatic Speech Recognition (ASR) models, running on AMD GPUs using ROCm.
TensorFlow Profiler in practice: Optimizing TensorFlow models on AMD GPUs
TensorFlow Profiler measures resource use and performance of models, helping identify bottlenecks for optimization. This blog demonstrates the use of the TensorFlow Profiler tool on AMD hardware.
Stone Ridge Expands Reservoir Simulation Options with AMD Instinct™ Accelerators
Stone Ridge Technology (SRT) pioneered the use of GPUs for high performance reservoir simulation (HPC) nearly a decade ago with ECHELON, its flagship software product. ECHELON, the first of its kind, engineered from the outset to harness the full potential of massively parallel GPUs, stands apart in the industry for its power, efficiency, and accuracy. Now, ECHELON has added support for AMDInstinct accelerators into its simulation engine, offering new flexibility and optionality to its clients.
SmoothQuant model inference on AMD Instinct MI300X using Composable Kernel
SmoothQuant model inference on AMD Instinct MI300X using Composable Kernel
Unveiling performance insights with PyTorch Profiler on an AMD GPU
Unveiling Performance Insights with PyTorch Profiler on an AMD GPU
Panoptic segmentation and instance segmentation with Detectron2 on AMD GPUs
Object Detection and Image Segmentation with Detectron2 on AMD GPU
AMD Collaboration with the University of Michigan offers High Performance Open-Source Solutions to the Bioinformatics Community
We are thrilled to share the success story of a 1.5-year collaboration between AMD and the University of Michigan, Ann Arbor where we used the AMD Instinct™ GPUs and ROCm™ software stack to optimize the sequence alignment bottleneck in long read processing workflows.
Siemens taps AMD Instinct™ GPUs to expand high-performance hardware options for Simcenter STAR-CCM+
Siemens recently announced that its Simcenter STAR-CCM+ multi-physics computational fluid dynamics (CFD) software now supports AMD Instinct™ GPUs for GPU-native computation. This move addresses its users' needs for computational efficiency, reduced simulation costs and energy usage, and greater hardware choice.
Accelerating Large Language Models with Flash Attention on AMD GPUs
Accelerating Large Language Models with Flash Attention on AMD GPUs
AMD in Action: Unveiling the Power of Application Tracing and Profiling
AMD in Action: Unveiling the Power of Application Tracing and Profiling
Step-by-Step Guide to Use OpenLLM on AMD GPUs
OpenLLM is an open-source platform for deploying large language models, enabling cloud or on-premises use. In this blog we focus on using OpenLLM to start an LLM server leveraging the capabilities of AMD GPUs
Inferencing with Mixtral 8x22B on AMD GPUs
Inferencing with Mixtral 8x22B on AMD GPUs
Training a Neural Collaborative Filtering (NCF) Recommender on an AMD GPU
Training a Neural Collaborative Filtering (NCF) Recommender on an AMD GPU
Multimodal (Visual and Language) understanding with LLaVA-NeXT
Multimodal instruction-following data with LLaVA-NeXT on AMD GPU
Table Question-Answering with TaPas
Table Question-Answering with TaPas
Unlocking Vision-Text Dual-Encoding: Multi-GPU Training of a CLIP-Like Model
Unlocking Vision-Text Dual-Encoding: Multi-GPU Training of a CLIP-Like Model
Transforming Words into Motion: A Guide to Video Generation with AMD GPU
Transforming Words into Motion: A Guide to Video Generation with AMD GPU
C++17 parallel algorithms and HIPSTDPAR #
C++17 parallel algorithms and HIPSTDPAR
Inferencing with AI2's OLMo model on AMD GPU
Inferencing with AI2's OLMo model on AMD GPU
Speech-to-Text on an AMD GPU with Whisper
Speech to Text on AMD with Whisper
Interacting with Contrastive Language-Image Pre-Training (CLIP) model on AMD GPU
Interacting with Contrastive Language-Image Pre-Training (CLIP) model on AMD GPU
Instruction fine-tuning of StarCoder with PEFT on multiple AMD GPUs
Instruction fine tuning of StarCoder with PEFT on multiple AMD GPUs
Text Summarization with FLAN-T5
Text Summarization with FLAN-T5 on AMD GPU
Affinity part 1 - Affinity, placement, and order
Affinity Part 1
Affinity part 2 - System topology and controlling affinity
Affinity Part 2
Enhancing LLM Accessibility: A Deep Dive into QLoRA Through Fine-tuning Llama 2 on a single AMD GPU
Enhancing LLM Accessibility: A Deep Dive into QLoRA Through Fine-tuning Llama 2 on a single AMD GPU
Developing Triton Kernels on AMD GPUs
This blog shows users how to develop and benchmark a custom Triton kernel
Enhancing LLM Accessibility: A Deep Dive into QLoRA Through Fine-tuning Llama Model on a single AMD GPU
This blog demonstrate how to use QLora to efficiently fine-tune Llama model on a single AMD GPU with ROCm.
GPU Unleashed: Training Reinforcement Learning Agents with Stable Baselines3 on an AMD GPU in Gymnasium Environment
GPU Unleashed: Training Reinforcement Learning Agents with Stable Baselines3 on an AMD GPU in Gymnasium Environment
ResNet for image classification using AMD GPUs
ResNet for image classification using AMD GPUs
Building semantic search with SentenceTransformers on AMD
Building semantic search with SentenceTransformers on AMD
Using the ChatGLM-6B bilingual language model with AMD GPUs
Bilingual language model ChatGLM-6B