Posts by Mohit Deopujari
DI Series: Scaling GLM-5.1-FP8 to 64 MI300X GPUs
- 21 August 2026
Serving a frontier Mixture-of-Experts (MoE) model well is a systems problem, and it gets harder the moment one node is not enough. GLM-5.1 is a good example: it is a large, sparse MoE that users want to run at long context, and it ships a new attention family that breaks assumptions older serving stacks quietly relied on. Fitting it on eight GPUs is only the start. The real question is how to keep it correct and fast as you spread it across several nodes.
Elevate Your LLM Inference: Autoscaling with Ray, ROCm 7.0.0, and SkyPilot
- 13 February 2026
This blog explores autoscaling of inference workloads in Ray Serve with a vLLM backend on AMD Instinct™ GPUs for large language models (LLMs). Furthermore, you will learn how to scale beyond a single cluster using SkyPilot, which enables multicloud scaling for Ray Serve. Combined with the AMD ROCm™ software platform, this creates a unified, cloud-agnostic platform that scales distributed LLM inference from single-GPU to multi-cluster deployments.
LLM Inference Optimization Using AMD GPU Partitioning
- 22 January 2026
As AI and HPC workloads grow in complexity and scale, there’s a rising need for precise GPU resource management, robust memory isolation, and efficient multi-tenant scheduling. AMD’s Instinct™ MI300 series addresses this by offering dynamic partitioning capabilities. These allow a single physical device to be segmented into multiple isolated partitions, each tailored to the needs of specific workloads. This flexibility is particularly beneficial for AI inference tasks, where different models or instances may require distinct resource allocations. Maximizing the utilization of GPU resources while ensuring that each workload operates within its own isolated environment is crucial for performance and reliability.