Nitro-P: Pixel-Space Text-to-Image via Hierarchical Semantic-to-Pixel Generation#

Nitro-P: Pixel-Space Text-to-Image via Hierarchical Semantic-to-Pixel Generation
November 20, 2026 by Jingai Yu, Tong Shen, Dong Zhou, Dong Li, Emad Barsoum.
9 min read. | 2113 total words.

We present Nitro-P, an end-to-end pixel-space text-to-image (T2I) diffusion framework. Built upon our previously released Nitro-E, Nitro-P extends the Nitro family from latent-space T2I generation to a VAE-free pixel-space design. To achieve both high generation quality and computational efficiency, Nitro-P adopts a hierarchical semantic-to-pixel design, consisting of a patch-level semantic modeling component based on E-MMDiT and a pixel-level modulation-guided reconstruction component implemented as a lightweight U-Net [1] decoder. By operating directly in pixel space, Nitro-P removes the VAE decoder from the inference pipeline. This benefit is most visible in few-step sampling, where latent-space models still incur a non-negligible final decoding pass. Nitro-P achieves a throughput of 10.26 samples/s on a single AMD Instinct™ MI300X GPU for 512px image generation with a batch size of 32. Its distilled one-step variant further boosts throughput to 90.02 samples/s.

Figure 1 summarizes a detailed comparison against representative pixel-space diffusion models, showing that Nitro-P achieves promising GenEval performance with competitive throughput, while its distilled one-step variant substantially improves efficiency with only limited degradation in generation quality.

As part of AMD’s ongoing commitment to open-source research, we release both the model weights and the complete training code to support reproducibility and broader adoption. We hope this work can serve as a practical baseline for future studies and help advance progress in pixel-space generation. In this blog, we provide an overview of the model design and the key technical details behind it.

Scaling performance

Figure 1. Comparison with other models on GenEval and throughput.

Nitro Family and Nitro-P#

Building on AMD’s earlier Nitro series, Nitro-P extends Nitro-E from efficient latent-space T2I generation to end-to-end, VAE-free pixel-space generation. Nitro-1 and Nitro-T, together with Nitro-E, represent AMD’s earlier explorations in latent-space T2I. The main highlights of Nitro-P are as follows:

  • VAE-free pixel-space T2I with a hierarchical semantic-to-pixel design (E-MMDiT + modulation-guided reconstruction).

  • Distilled one-step checkpoints achieve throughputs of 90.02 samples/s at 512x512 and 26 samples/s at 1024x1024, both with a batch size of 32 on a single AMD Instinct™ MI300X GPU.

  • The full-step model achieves competitive GenEval scores of 0.88 at 512×512 and 0.87 at 1024×1024 (see Results).

  • Fully supported by AMD ROCm™ Software; trained from scratch on public data, with code and weights released.

Background and Architecture#

Text-to-Image and Latent-Space Diffusion#

In T2I synthesis, latent-space diffusion models have been particularly effective in practice, typically using a two-stage pipeline in which a Variational Autoencoder (VAE) [2] first compresses images into a lower-dimensional representation and diffusion is then performed in the latent space, substantially reducing the cost of training and sampling.

This design improves computational efficiency but makes generation dependent on the representation learned by the VAE. Visual information that is not faithfully retained during compression cannot be modeled directly by the diffusion process, and the generated latent representation must still be converted back to RGB by the VAE decoder. Moreover, the autoencoder and diffusion model are trained with separate objectives rather than optimized jointly. These trade-offs have motivated growing interest in performing generative modeling directly in pixel space [5–7].

From Latent-space to Pixel-Space Diffusion#

Pixel-space diffusion, however, is not free. The main challenge is computational cost. Treating every pixel as an individual Transformer [3] token would produce extremely long sequences, making self-attention impractical at high resolutions. Pixel-space Transformer models therefore commonly group neighboring pixels into patches.

These operations work as follows:

  • Patchification: before feeding the input into the Transformer, each neighboring \(p \times p\) pixel block is packed into a single token with channel dimension \(p \times p \times 3\).

  • Unpatchification: at the output side, the Transformer predicts tokens with channel dimension \(p \times p \times 3\), which are then reshaped back into the pixel image.

This design greatly improves computational efficiency, but it shifts the difficulty from attention cost to pixel decoding.

While patchification makes pixel-space Transformers computationally tractable, it also increases the dimensionality of the pixel values predicted from each token. As observed in JiT [4], it is difficult for the model to directly predict such high-dimensional \(p \times p \times 3\) outputs, especially when the prediction target is noise or velocity. A large patch size reduces token count and attention cost but blurs boundaries and textures; a small patch size preserves details but increases sequence length and quadratic attention cost. A single DiT backbone struggles to excel at both global semantics and local fidelity. DeCo [6] and PixelDiT [7] use local decoders to predict pixel values within each patch, and then assemble the decoded outputs through unpatchification.

More broadly, DiP [5], DeCo [6], and PixelDiT [7] show that the key is not to apply a full-resolution DiT [8] over every pixel, but to decouple global structure modeling from local detail restoration with a lightweight pixel decoder. In this paradigm, a diffusion Transformer handles text-conditioned semantics at coarse resolution, while a lightweight local module recovers full-resolution RGB. The specific decoder architectures differ across methods: DiP uses a shallow U-Net, DeCo employs attention-free linear decoder blocks and PixelDiT uses a pixel-level DiT with self-attention. This has become a mainstream design pattern for efficient pixel-space T2I.

Nitro-P: A Hierarchical Semantic-to-Pixel Framework#

Nitro-P architecture overview

Figure 2. Overview of Nitro-P.

Building on this paradigm, Nitro-P introduces a hierarchical semantic-to-pixel architecture for end-to-end pixel-space T2I generation: it avoids the VAE bottleneck of latent diffusion and replaces direct output prediction from patch tokens followed by unpatchification with a dedicated full-image convolutional decoder. An overview of the Nitro-P architecture is shown in Figure 2.

At the patch-level semantic modeling stage, Nitro-P builds on our Nitro-E, reusing the efficiency-oriented designs from E-MMDiT, including token compression, positional reinforcement, alternating subregion attention, and AdaLN-affine. Operating on 16×16 RGB patches, E-MMDiT models text-conditioned structure and outputs a coarse semantic feature map at 1/16 resolution, rather than directly predicting RGB values via unpatchification. Text conditioning uses Qwen3-1.7B [9] features projected by a caption embedder with a learnable positional embedding; the resulting tokens are further refined by four timestep-conditioned TextRefineBlocks before entering joint attention.

Consistent with recent observations in MiniT2I [10], we find that direct RGB prediction through unpatchification can produce visible patch artifacts in practice. We further observe that this issue becomes more pronounced under one-step distillation, where the model must produce the final image in a single pass and local reconstruction errors are no longer averaged away over multiple denoising steps. These findings reinforce the need for an explicit pixel reconstruction module, rather than predicting RGB values directly from patch tokens.

At the pixel-level reconstruction stage, a natural baseline is independent decoding: each patch reconstructs its own pixels in isolation, after which the patches are assembled by unpatchification. Because neighboring patches never interact at this stage, this design tends to produce visible seams at patch boundaries. Nitro-P instead adopts joint decoding with a lightweight U-Net decoder (similar in spirit to DiP): the full-resolution RGB velocity is reconstructed through multi-scale convolutions and skip connections over the entire image. This convolutional backbone models spatial neighborhoods and preserves local continuity, so patch artifacts are largely suppressed. As shown in Figure 3, joint decoding yields substantially fewer visible patch artifacts than independent decoding. To further strengthen semantic control, Nitro-P introduces a modulation-guided interface between E-MMDiT and the U-Net decoder, as detailed in the next section.

Scaling performance

Figure 3. Visual comparison of patch artifacts.

Modulation-Guided Semantic Injection#

An important design question is how to transfer the patch-level E-MMDiT feature map into the pixel-level U-Net. We investigate two alternatives: direct feature concatenation and modulation-guided injection. The concatenation baseline directly merges the E-MMDiT features with the U-Net features. In our modulation-guided design, the E-MMDiT feature map is projected into spatially varying scale and shift parameters, which modulate the normalized U-Net activations at the bottleneck. This design is related to the semantic modulation used in DeCo [6], but applies it to a different reconstruction architecture. In DeCo, the low-resolution semantic features produced by the DiT are upsampled and transformed into adaptive parameters that modulate the dense decoder queries in each pixel-decoder block. In Nitro-P the two-dimensional E-MMDiT feature map instead modulates the bottleneck of a full-image U-Net, enabling adaptive semantic control during multi-scale reconstruction across patch boundaries.

To isolate the contribution of modulation-guided semantic injection, we train a concat-only baseline that uses the same E-MMDiT backbone and U-Net decoder. All other training and inference settings are kept identical. As shown in Table 2, modulation-guided decoding consistently improves semantic fidelity over concat-only injection, producing more accurate object colors, more accurate object counts, and more coherent global layout and structural relationships. By comparison, the concat-only baseline often weakens the text-conditioned signal, leading to color mismatches, missing or duplicated objects, and inconsistent local details.

Scaling performance

Table 2. Side-by-side visual comparison under identical prompts. The left column shows results from the concat-only baseline, while the right column shows results from Nitro-P with AdaIN-guided decoding.

Experiments and Results#

Data#

We curated a dataset of 25M images containing both real and synthetic images. Compared with Nitro-E, we additionally include high-quality images from MiniT2I [10]. The details are described in Table 3. All the data are publicly available without any internal private data.

Source

Number of Images

Type of Images

Generation Method

Segment Anything 1B (SA1B)

11.1M

Real

Real images annotated via LLaVA

JourneyDB

4.4M

Synthetic

Midjourney generated images

FLUXDB

9.5M

Synthetic

Generated using FLUX.1-dev [16] with prompts from DataComp

High-quality data from MiniT2I

120K

Synthetic

A curated alignment fine-tuning mixture of BLIP3o-60K, LAION DALL-E 3 Discord set, and ShareGPT-4o-Image

Table 3. Details of our training data. It consists of a mix of real and synthetic images and captions totaling 25M text-image pairs.

Training Strategies#

Nitro-P uses Qwen3-1.7B as the text encoder. Compared with Llama 3.2 1B [11], Qwen3-1.7B demonstrates a better understanding of color and spatial relationships.

Beyond the standard flow-matching [12] objective, we adopt a frequency-aware flow-matching loss inspired by DeCo, emphasizing perceptually important frequencies in the Discrete Cosine Transform (DCT) domain. Following Nitro-E, we also use REPA [13] to align early transformer representations with frozen DINOv2 [14] features for faster and more stable convergence.

Our model is trained in a multi-stage curriculum, progressively increasing the image resolution from 256×256 to 512×512, and finally to 1024×1024. The early stages use a large-scale mixed dataset for broad visual coverage, while the later stages transition to a smaller high-quality data mixture for alignment-oriented fine-tuning. This design allows the model to first learn general text-to-image generation capabilities and then improve prompt fidelity and visual quality through high-quality supervision. For one-step generation, we further distill the model with a combination of adversarial and diffusion losses. The discriminator uses a frozen E-MMDiT feature extractor with a randomly initialized prediction head.

Implementation Details#

Our experiments are conducted using the PyTorch for ROCm training Docker image [15], which provides an optimized PyTorch environment for training models on AMD Instinct™ MI325X and MI300X accelerators. Our implementation is built on the training engine Accelerate with mixed precision training using the Bfloat16 datatype. Leveraging the large memory capacity of MI300X, we train the model with a batch size of 2048 using the native distributed data parallel.

For metrics, we adopt four widely used benchmarks:

Results#

We compare Nitro-P with recent open pixel-space text-to-image models in Table 4, and summarize the quality–throughput trade-off in Figure 1. At 512×512, Nitro-P reaches a GenEval score of 0.88 with 10.26 samples/s, offering a strong balance against methods such as DeCo and MiniT2I-L, while using fewer main-network TFLOPs. The one-step distilled model further increases throughput to 90.02 samples/s with a moderate GenEval drop (0.85). At 1024×1024, Nitro-P surpasses PixelDiT in quality (0.87 vs. 0.74). All models are trained only on public data. Qualitative examples are shown in Figure 4.

Methods Throughput
(samples/s)
Parameters
Main network
(M)
TFLOPs
Main network
Dataset Size
(M)
GenEval↑ IR↑ HPS↑ DPG↑
512x512
PixelGen 7.6 1100 1.42 36 0.79 0.33 28.08 80.03
PixNerd 6.9 1200 1.54 45 0.73 0.48 27.83 80.90
DeCo 6.13 1100 1.45 42 0.86 0.48 27.75 81.40
PixelDiT 5.54 1300 1.43 26 0.78 - - 83.70
MiniT2I-L 2.4 912 1.49 12 0.88 0.69 27.73 85.90
MiniT2I-4step - 258 - 12 0.84 - - 83.23
Nitro-P 10.26 679 0.74 25 0.88 0.98 30.04 86.33
Nitro-P-1step 90.02 679 0.74 25 0.85 0.99 29.79 83.54
1024x1024
PixelDiT 2.05 1300 6.18 26 0.74 0.95 30.98 83.50
Nitro-P 2.44 679 3.35 25 0.87 1.01 30.48 85.49
Nitro-P-1step 26.00 679 3.35 25 0.83 0.98 29.93 83.08

Table 4. Comparison of Nitro-P with other pixel-space diffusion models.

To isolate the effect of removing the VAE decoder from the inference path, we further compare end-to-end throughput of Nitro-P against Nitro-E on the same AMD Instinct™ MI300X GPU (batch size 32). Nitro-E still requires a final VAE decoding pass after denoising, whereas Nitro-P is VAE-free and operates directly in pixel space. As shown in Table 5, this difference is largest in the few-step regime: at 512×512, the one-step distilled Nitro-P reaches 90.02 samples/s, versus 44.13 samples/s for Nitro-E. The gap narrows as the number of sampling steps increases, and at the full 20-step setting Nitro-E is faster, because the denoising backbone dominates runtime and the latent-space path remains cheaper per step. The same trend holds at 1024×1024. These results indicate that Nitro-P is particularly well suited to low-NFE, high-throughput serving.

Models 1-step TP
2-step TP
5-step TP
20-step TP
512×512
Nitro-E 44.13 41.39 34.35 18.83
Nitro-P 90.02 64.83 33.76 10.26
1024×1024
Nitro-E 11.00 10.28 8.59 4.66
Nitro-P 26.00 16.86 8.42 2.44

Table 5. End-to-end throughput (samples/s) of Nitro-E and Nitro-P on a single AMD Instinct™ MI300X GPU with batch size 32. TP denotes throughput.

More Visual Examples#

Scaling performance

Figure 4. Images generated by Nitro-P

Summary#

We present Nitro-P, a VAE-free pixel-space extension of Nitro-E for text-to-image generation. Built on a hierarchical semantic-to-pixel design, Nitro-P reuses the efficiency-oriented E-MMDiT recipe for patch-level semantic modeling and pairs it with a lightweight U-Net for pixel-level reconstruction under modulation-guided semantic injection. Trained entirely on public data on AMD Instinct™ MI300X GPUs and released with open code and checkpoints—including one-step distilled variants—Nitro-P offers a practical baseline for efficient pixel-space generation.

Additional Resources#

Model: amd/Nitro-P · HuggingFace

Code: AMD-AIG-AIMA/Nitro-P

Related Work from AMD team:

Other useful Links

References#

  1. Olaf Ronneberger, Philipp Fischer, and Thomas Brox. U-Net: Convolutional networks for biomedical image segmentation. In International Conference on Medical Image Computing and Computer-Assisted Intervention, pages 234–241. Springer, 2015.

  2. Kingma, D. P., & Welling, M. Auto-Encoding Variational Bayes. arXiv, 2013.

  3. Dosovitskiy, A., et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale. ICLR, 2021.

  4. Li, T., & He, K. Back to Basics: Let Denoising Generative Models Denoise. arXiv, 2025.

  5. Chen, Z., et al. DiP: Taming Diffusion Models in Pixel Space. CVPR, 2026.

  6. Ma, Z., et al. DeCo: Frequency-Decoupled Pixel Diffusion for End-to-End Image Generation. CVPR, 2026.

  7. Yu, Y., et al. PixelDiT: Pixel Diffusion Transformers for Image Generation. CVPR, 2026.

  8. Peebles, W., & Xie, S. Scalable Diffusion Models with Transformers. ICCV, 2023.

  9. Qwen Team. Qwen3 Technical Report. arXiv, 2025. https://arxiv.org/abs/2505.09388

  10. Wang, X., et al. MiniT2I: A Minimalist Baseline for Text-to-Image Generation. Technical blog, 2026.

  11. Meta. Llama 3.2 Model Family. 2024. https://huggingface.co/meta-llama/Llama-3.2-1B

  12. Esser, P., et al. Scaling Rectified Flow Transformers for High-Resolution Image Synthesis. arXiv, 2024.

  13. Yu, S., et al. Representation Alignment for Generation: Training Diffusion Transformers Is Easier Than You Think. ICLR, 2025.

  14. Oquab, M., et al. DINOv2: Learning Robust Visual Features without Supervision. arXiv, 2024.

  15. PyTorch for ROCm Training Docker Image. Docker Hub, 2025.

  16. Batifol, S., et al. Flux. 1 kontext: Flow matching for in-context image generation and editing in latent space. arXiv, 2025

System Configuration#

Our testbed system was configured with an AMD Instinct™ MI300X GPU running tests done by AMD on 05/17/2025, results may vary based on configuration, usage, software version, and optimizations.

AMD Instinct™ MI300X platform System Model: Supermicro AS-8125GS-TNMR2 CPU: 2x AMD EPYC 9575F 64-Core Processor NUMA: 1 NUMA node per socket. NUMA auto-balancing disabled Memory: 2304 GiB (24 DIMMs x 96 GiB Micron Technology MTC40F204WS1RC48BB1 DDR5 4800 MT/s) Disk: 16,092 GiB (4x SAMSUNG MZQL23T8HCLS-00A07 3576 GiB, 2x SAMSUNG MZ1L2960HCJR-00A07 894 GiB) GPU: 8x AMD Instinct™ MI300X 192GB HBM3 750W Host OS: Ubuntu 22.04.4 System BIOS: 3.5 System Bios Vendor: American Megatrends International, LLC. Host GPU Driver: (amdgpu version): ROCm 6.3.4 Docker image: rocm/pytorch-training:v25.5

Disclaimers#

The information presented in this document is for informational purposes only and may contain technical inaccuracies, omissions, and typographical errors. The information contained herein is subject to change and may be rendered inaccurate for many reasons, including but not limited to product and roadmap changes, component and motherboard version changes, new model and/or product releases, product differences between differing manufacturers, software changes, BIOS flashes, firmware upgrades, or the like. Any computer system has risks of security vulnerabilities that cannot be completely prevented or mitigated. AMD assumes no obligation to update or otherwise correct or revise this information. However, AMD reserves the right to revise this information and to make changes from time to time to the content hereof without obligation of AMD to notify any person of such revisions or changes. THIS INFORMATION IS PROVIDED “AS IS.” AMD MAKES NO REPRESENTATIONS OR WARRANTIES WITH RESPECT TO THE CONTENTS HEREOF AND ASSUMES NO RESPONSIBILITY FOR ANY INACCURACIES, ERRORS, OR OMISSIONS THAT MAY APPEAR IN THIS INFORMATION. AMD SPECIFICALLY DISCLAIMS ANY IMPLIED WARRANTIES OF NON-INFRINGEMENT, MERCHANTABILITY, OR FITNESS FOR ANY PARTICULAR PURPOSE. IN NO EVENT WILL AMD BE LIABLE TO ANY PERSON FOR ANY RELIANCE, DIRECT, INDIRECT, SPECIAL, OR OTHER CONSEQUENTIAL DAMAGES ARISING FROM THE USE OF ANY INFORMATION CONTAINED HEREIN, EVEN IF AMD IS EXPRESSLY ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.

Third-party content is licensed to you directly by the third party that owns the content and is not licensed to you by AMD. ALL LINKED THIRD-PARTY CONTENT IS PROVIDED “AS IS” WITHOUT A WARRANTY OF ANY KIND. USE OF SUCH THIRD-PARTY CONTENT IS DONE AT YOUR SOLE DISCRETION AND UNDER NO CIRCUMSTANCES WILL AMD BE LIABLE TO YOU FOR ANY THIRD-PARTY CONTENT. YOU ASSUME ALL RISK AND ARE SOLELY RESPONSIBLE FOR ANY DAMAGES THAT MAY ARISE FROM YOUR USE OF SUCH THIRD-PARTY CONTENT.

AMD, the AMD Arrow logo, AMD Instinct, ROCm, and combinations thereof are trademarks of Advanced Micro Devices, Inc. Other product names used in this publication are for identification purposes only and may be trademarks of their respective companies. © 2026 Advanced Micro Devices, Inc. All rights reserved.