Posts by Pradeep Sakhamoori
Long Context, One Node: Serving GLM-5.3-FP8 on MI300X with DP8+EP8
- 14 October 2026
Serving a frontier Mixture-of-Experts model like GLM-5.3-FP8 at long context and high concurrency is a demanding workload. The model has 744 billion total parameters, with approximately 39 billion active per forward pass, and a native context limit of 1,048,576 tokens, as specified by max_position_embeddings in config.json. The most demanding serving regime combines long prompts, long generations, and many concurrent requests. That combination stresses both GPU memory and inter-GPU communication, and the layout you pick to spread the model across a node largely determines how much throughput you can sustain under that load.