Posts by He Cui
Micro-World 2: Streaming Real-time Interactive World Model
- 21 October 2026
Diffusion models have advanced rapidly in recent years, enabling high-quality video generation across a wide range of creative scenarios. As visual quality continues to improve, however, the field is increasingly shifting its focus from passive content generation toward controllable and interactive experiences. YUME 1.5 [1] explores controllable video generation through explicit interaction signals, while LingBot-World [2] and Sana-WM [3] investigate world models that generate future observations conditioned on the current environment and user actions. World Labs’ Marble [27] and Atlas [28] represent a complementary spatial-modeling direction: Marble creates editable, explicit 3D worlds from multimodal inputs, while Atlas organizes text, images, camera poses, and depth into a shared spatial context for generation, reconstruction, and simulation. Together, these efforts illustrate how generative models are evolving from passive content-synthesis tools toward controllable, persistent, and interactive world simulators.
Triton-Based Optimization of Video Sparse Attention on ROCm
- 13 July 2026
Video generation has become a major frontier in generative modeling, driven by large-scale data and increasingly scalable architectures. Among modern architectures, Diffusion Transformers (DiTs) have emerged as a dominant paradigm [1,2,3,4] by representing videos as spatio-temporal token sequences, enabling long-range interactions across frames and spatial regions, as well as flexible multimodal conditioning with text or audio. However, full self-attention scales quadratically with token count, making it increasingly expensive as spatio-temporal resolution and model size grow. Video sparse attention (VSA) [5,6] mitigates this cost by approximating full attention with a subset of informative token interactions, but its practical efficiency in both training and inference depends heavily on hardware-aware Triton kernel implementations.
Bridging the Last Mile: Deploying Hummingbird-XT for Efficient Video Generation on AMD Consumer-Grade Platforms
- 08 January 2026
*The first three authors (Isobe, Cui, and Ge) contributed equally to this work.
AMD Hummingbird Image to Video: A Lightweight Feedback-Driven Model for Efficient Image-to-Video Generation
- 03 August 2025
In this blog, we present AMD Hummingbird-I2V, a lightweight and feedback-driven image-to-video generation model designed to deliver high-quality results efficiently on resource-constrained hardware. Image-to-video (I2V) generation has become a significant challenge in computer vision, driven by the increasing demand for automated content creation in areas such as digital media production, animation, and advertising. While recent advancements have improved video quality, deploying I2V models in practical scenarios remains challenging due to their large model sizes and high inference costs. For example, DynamiCrafter [1] employs a 1.4B-parameter U-Net and typically requires 50 denoising steps to synthesize a single video. Step-Video [2], a DiT-based model with 30B parameters, takes approximately 30 minutes to generate one video on an AMD Instinct ™ MI250 GPU, making it impractical for latency-sensitive or resource-constrained environments, such as gaming-oriented desktop GPUs. In this work, we present AMD Hummingbird-I2V, a compact and efficient diffusion-based I2V model designed for high-quality video synthesis under limited computational budgets. Hummingbird-I2V adopts a lightweight U-Net architecture with 0.9B parameters and a novel two-stage training strategy guided by reward-based feedback, resulting in substantial improvements in inference speed, model efficiency, and visual quality. To further improve output resolution with minimal overhead, we introduce a super-resolution module at the end of the pipeline. Additionally, we leverage ReNeg [3], an AMD proposed reward-guided framework for learning negative embeddings via gradient descent, to further boost visual quality. As a result, Hummingbird-I2V can generate high-quality 4K video in just 11 seconds with 16 inference steps on an AMD Radeon™ RX 7900 XTX GPU. Quantitative results on the VBench-I2V [4] benchmark show that Hummingbird-I2V achieves state-of-the-art performance among U-Net-based diffusion models and competitive results compared to significantly larger DiT-based models. We provide a detailed analysis of the model architecture, training methodology, and benchmark performance.