Running Flux.1 on Low-VRAM GPUs with ComfyUI & NF4 Quantization: Complete 8GB/12GB Setup Guide

1. The Flux.1 Revolution: 12B MMDiT Architecture

Released by Black Forest Labs (the original creators of Stable Diffusion), the Flux.1 model family established a new high-water mark for open-weights generative imagery. Featuring a 12-billion parameter Multimodal Diffusion Transformer (MMDiT) backbone, Flux excels in anatomical accuracy (hands, limbs, facial micro-expressions), intricate multi-subject spatial compositions, and sharp, legible typography embedded directly within scene environments.

However, this unprecedented quality comes with steep computational requirements:

• Uncompressed FP16 Weights: The base MMDiT diffusion model consumes approximately 23.8 GB of VRAM.
• T5-XXL Text Encoder: The language comprehension backbone requires an additional 9.6 GB of VRAM.
• VAE & Intermediate Latents: The variational autoencoder and latent noise buffers add another 2.5 GB.

In total, unquantized Flux requires over 36 GB of VRAM — pricing out developers and digital artists on consumer graphics hardware. Fortunately, modern quantization techniques allow us to compress this pipeline down to fit within an 8GB or 12GB GPU without discernible visual degradation. For spatial rendering context, explore our paper on Generative 3D, NeRFs & Spatial Computing.

2. The Memory Bottleneck: MMDiT + T5-XXL Footprint

When executing a diffusion step, memory pressure peaks during cross-attention between the image latents and text embeddings. If total memory exceeds physical VRAM, PyTorch triggers CUDA Out-Of-Memory (OOM) crashes, or the Windows NVIDIA driver swaps memory into slow system RAM across the PCIe bus, degrading generation times from 15 seconds to over 15 minutes.

To eliminate this bottleneck, we apply a two-pronged strategy:

1. Weight Quantization (NF4 or GGUF): Quantize the 12B MMDiT weights to 4-bit precision, shrinking model weights from 23.8 GB down to ~6.5 GB.
2. T5 Text Encoder Offloading: Load the T5-XXL encoder either as a quantized 4-bit/8-bit checkpoint or offload it entirely into CPU system RAM, executing text encoding on CPU before moving only the final embedding tensors to the GPU.

3. NF4 vs GGUF: Selecting the Right Quantization

Two primary quantization formats dominate the Flux ecosystem:

4. ComfyUI Installation & ComfyUI-GGUF Node Configuration

ComfyUI provides the most modular and memory-efficient execution environment for diffusion pipelines.

Follow these steps to set up the low-VRAM environment:

# 1. Clone ComfyUI
git clone https://github.com/comfyanonymous/ComfyUI.git
cd ComfyUI

# 2. Install PyTorch with CUDA 12.4
pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu124

# 3. Install ComfyUI-GGUF custom node
cd custom_nodes
git clone https://github.com/city96/ComfyUI-GGUF.git
pip install -r ComfyUI-GGUF/requirements.txt
cd ..

# 4. Launch ComfyUI with Low-VRAM flags
python main.py --lowvram --preview-method auto

5. Building the Low-VRAM Node Graph in ComfyUI

In the ComfyUI workspace, construct the pipeline using GGUF loader nodes:

1. Unet Loader (GGUF): Load flux1-schnell-Q4_K_S.gguf or flux1-dev-Q4_K_S.gguf.
2. DualCLIPLoader (GGUF): For clip_name1, select standard clip_l.safetensors (300MB). For clip_name2, select t5xxl_fp8_e4m3fn.safetensors (4.8GB) or t5-v1_1-xxl-encoder-Q4_K_M.gguf (2.5GB). Set type to flux.
3. VAELoader: Load standard ae.safetensors.
4. SamplerCustomAdvanced:
   • For Flux Schnell: Set steps to 4, CFG to 1.0, sampler to euler, scheduler to simple.
   • For Flux Dev: Set steps to 20–25, CFG to 3.5, sampler to euler, scheduler to beta.

6. Hardware Benchmarks & Generation Speeds (8GB to 24GB)

The table below demonstrates empirical 1024x1024 image generation times for Flux.1 Schnell (4 steps) and Flux.1 Dev (20 steps) across consumer graphics hardware:

To calculate hardware memory limits for other generative models, use our LLM GPU VRAM Calculator.

7. Frequently Asked Questions