Posts by Mou Li
Introducing AMD ROCm™ Infera: Scaling Goodput for Agentic AI with Distributed Inference Orchestration
- 23 July 2026
Today we are introducing AMD ROCm™ Infera, a distributed inference reference solution for large-scale deployments. Infera is a conductor for your inference GPU orchestra. Initial internal testing shows that Infera can improve goodput per GPU for realistic agentic workloads by up to 2.6×, as detailed below. Built for AMD Instinct™ GPUs, Infera is open source from day one, and the code is available at github.com/AMD-AGI/Infera.
Triton-Based Optimization of Video Sparse Attention on ROCm
- 13 July 2026
Video generation has become a major frontier in generative modeling, driven by large-scale data and increasingly scalable architectures. Among modern architectures, Diffusion Transformers (DiTs) have emerged as a dominant paradigm [1,2,3,4] by representing videos as spatio-temporal token sequences, enabling long-range interactions across frames and spatial regions, as well as flexible multimodal conditioning with text or audio. However, full self-attention scales quadratically with token count, making it increasingly expensive as spatio-temporal resolution and model size grow. Video sparse attention (VSA) [5,6] mitigates this cost by approximating full attention with a subset of informative token interactions, but its practical efficiency in both training and inference depends heavily on hardware-aware Triton kernel implementations.
MoE Training Best Practices on AMD GPUs
- 16 December 2025
This blog covers best practices for training Mixture-of-Experts (MoE) models on AMD Instinct™ MI300/MI355-series[a] GPUs with the ROCm ecosystem. Whether you’re new to MoE distributed architectures or optimizing trillion-parameter models, this guide will help you identify bottlenecks and maximize efficiency on AMD hardware.