ComfyUI Complete Guide (2026): Install, Workflows, ControlNet, Flux, SDXL

ComfyUI is the most powerful frontend for local image and video generation in 2026 — a node-based interface that exposes every internal step of diffusion models, supports new architectures on day one, and ships with the fastest sampling implementations. This guide covers everything: installation across NVIDIA / AMD / Mac, the node graph mental model, ControlNet, IPAdapter, regional prompting, LoRA stacks, Flux / SDXL / SD 3.5 workflows, video generation (Wan, HunyuanVideo, Mochi), and serious tuning for 24 GB and below.

Whether you are coming from Automatic1111, Forge, or fresh to local image gen, this is the reference.

Table of Contents

1. What ComfyUI Is and Why It Wins
2. Hardware Requirements
3. Installation: Windows, Linux, macOS
4. Folder Layout & Where Models Go
5. ComfyUI Manager — Mandatory First Install
6. The Node Graph Mental Model
7. Your First Workflow: SDXL Text-to-Image
8. Models in 2026: Flux, SDXL, SD 3.5, Pony, Illustrious
9. LoRAs and Embeddings
10. ControlNet — Composition, Pose, Depth, Canny
11. IPAdapter — Image Prompts and Style Transfer
12. Inpainting, Outpainting, and Upscaling
13. Regional Prompting and Conditioning Combine
14. Flux: Dev, Schnell, GGUF, Quantized
15. Video Generation: Wan 2.2, HunyuanVideo, Mochi
16. API Mode and Programmatic Use
17. VRAM Optimization Tricks
18. Speed Tuning: Sage Attention, Triton, Compile
19. Common Custom Node Packs Worth Installing
20. Troubleshooting
21. FAQ

---

What ComfyUI Is and Why It Wins {#what-is}

Diffusion image generation is a pipeline:

Prompt → Text Encoder → Conditioning
                        ↓
Empty Latent → Sampler ← Model (UNet/DiT)
                        ↓
                       VAE Decode → Image

A1111 / Forge / Fooocus hide this pipeline behind a tabbed UI. ComfyUI exposes it as a graph: nodes for "Load Checkpoint," "CLIP Text Encode," "KSampler," "VAE Decode," "Save Image," with explicit data flow between them. You can branch, merge, swap, and chain nodes arbitrarily.

This is why ComfyUI is the fastest to support new models: when Flux launched, supporting it was a matter of writing a new "Load Diffusion Model" node and a new sampler — no UI surgery required. Same for SD 3.5, Wan 2.2, HunyuanVideo, and every model since 2023.

The trade-off: more upfront learning. But every workflow is a JSON file you can load with one drag-and-drop. The community ships pre-built workflows for almost every common task.

---

Hardware Requirements {#requirements}

RAM: at least 2x your largest model file in system RAM for offload buffers. 32 GB is the realistic minimum, 64 GB recommended for video.

Disk: Flux Dev = 24 GB, SDXL = 7 GB, SD 1.5 = 4 GB, plus VAEs (300 MB each), CLIP (1-5 GB), LoRAs (10-500 MB each), ControlNet (1.5 GB each). Plan for 200-500 GB on NVMe.

AMD: RX 7900 XTX works via ROCm at 70-85% of equivalent NVIDIA speed. See AMD ROCm Setup.

Apple Silicon: M2 or newer via MPS, but 3-5x slower than NVIDIA. Use MLX-based alternatives like Draw Things for better performance.

---

Installation: Windows, Linux, macOS {#installation}

Windows (portable — recommended)

1. Download the latest ComfyUI_windows_portable.7z from github.com/comfyanonymous/ComfyUI/releases.
2. Extract with 7-Zip to e.g. D:\ComfyUI\.
3. Run run_nvidia_gpu.bat (NVIDIA) or run_cpu.bat (no GPU).
4. Browser opens to http://127.0.0.1:8188.

Linux / Mac (git)

git clone https://github.com/comfyanonymous/ComfyUI
cd ComfyUI
python3.11 -m venv venv
source venv/bin/activate

# NVIDIA CUDA 12.4
pip install torch torchvision --index-url https://download.pytorch.org/whl/cu124

# AMD ROCm 6.2
pip install torch torchvision --index-url https://download.pytorch.org/whl/rocm6.2

# Mac MPS
pip install torch torchvision

# ComfyUI dependencies
pip install -r requirements.txt

# Run
python main.py --listen 0.0.0.0

Docker

docker run --gpus all \
    -p 8188:8188 \
    -v $(pwd)/models:/app/models \
    -v $(pwd)/output:/app/output \
    -v $(pwd)/workflows:/app/workflows \
    yanwk/comfyui-boot:latest

yanwk/comfyui-boot includes Manager, common custom nodes, and starter models.

Useful launch flags

---

Folder Layout & Where Models Go {#folder-layout}

ComfyUI/
├── models/
│   ├── checkpoints/      # SD 1.5, SDXL, SD 3.5, Pony, Illustrious .safetensors
│   ├── unet/             # Flux UNet/DiT files
│   ├── diffusion_models/ # Newer alias for unet/
│   ├── clip/             # T5 / CLIP-L / CLIP-G text encoders
│   ├── vae/              # VAE decoders
│   ├── loras/            # LoRA files
│   ├── controlnet/       # ControlNet models
│   ├── embeddings/       # Textual inversion embeddings
│   ├── ipadapter/        # IPAdapter models
│   ├── upscale_models/   # 4x-UltraSharp, RealESRGAN, etc.
│   └── animatediff_models/
├── custom_nodes/         # Third-party node packs
├── workflows/            # Saved JSON workflows
├── input/                # Images for img2img, ControlNet
└── output/               # Generated images

To share models between A1111/Forge and ComfyUI without duplicating files:

# Edit ComfyUI/extra_model_paths.yaml
a111:
    base_path: /path/to/stable-diffusion-webui/
    checkpoints: models/Stable-diffusion
    loras: models/Lora
    controlnet: models/ControlNet
    upscale_models: models/ESRGAN
    embeddings: embeddings
    vae: models/VAE

---

ComfyUI Manager — Mandatory First Install {#manager}

cd ComfyUI/custom_nodes
git clone https://github.com/ltdrdata/ComfyUI-Manager

Restart ComfyUI. The "Manager" button appears on the right sidebar.

What it does:

• Install Missing Custom Nodes — when you load someone else's workflow.
• Install Models — auto-downloads missing checkpoints/LoRAs/ControlNets.
• Update All — keeps ComfyUI and all custom nodes current.
• Snapshot / Restore — versioned backups before risky updates.
• Disable / Uninstall — surgically remove a custom node.

Without Manager, every missing node is a manual git clone. Install it first; do not skip.

---

The Node Graph Mental Model {#node-graph}

A workflow is a directed graph. Each node:

• Has inputs (connectors on the left)
• Has outputs (connectors on the right)
• Performs a function (load model, encode text, sample, decode VAE)

Connections carry typed values:

• MODEL — diffusion model
• CLIP — text encoder
• VAE — image encoder/decoder
• CONDITIONING — encoded prompt
• LATENT — latent image (compressed representation)
• IMAGE — pixel image
• MASK — binary mask
• CONTROL_NET — ControlNet model
• STRING, INT, FLOAT — primitives

The default workflow looks like:

[Load Checkpoint] → MODEL ─┐
                  → CLIP   │
                  → VAE ───┼──┐
                          │  │
[CLIP Text Encode pos] ←──┘  │
        ↓ CONDITIONING        │
[KSampler] ←──────────────────┘
   ↑ LATENT (from Empty Latent Image)
   ↓ LATENT (after sampling)
[VAE Decode] → IMAGE
   ↓
[Save Image]

Master this and you can build anything. Common patterns:

• Two samplers in series — base + refiner (SDXL).
• Conditioning combine — merge two prompts with weights.
• ControlNet apply — modulate conditioning with control image.
• Latent composite — blend two latents before decoding.

---

Your First Workflow: SDXL Text-to-Image {#first-workflow}

1. Download sd_xl_base_1.0.safetensors from Hugging Face → put in models/checkpoints/.
2. Open ComfyUI → Load Default workflow (right sidebar).
3. In Load Checkpoint node, select sd_xl_base_1.0.safetensors.
4. In Empty Latent Image, set width/height to 1024×1024.
5. In CLIP Text Encode (Positive), write your prompt: e.g. "cinematic photo of a samurai standing in a misty forest, ultra-detailed, 35mm film".
6. In CLIP Text Encode (Negative), write: "blurry, deformed, extra fingers, low quality, bad anatomy".
7. In KSampler, set steps=25, cfg=7.0, sampler=dpmpp_2m, scheduler=karras.
8. Click Queue Prompt.

Expected time on RTX 4090: ~3-5 seconds per 1024² image.

---

Models in 2026: Flux, SDXL, SD 3.5, Pony, Illustrious {#models}

Flux GGUF / FP8 quantized

Flux Dev in BF16 needs ~24 GB. To run on 12-16 GB:

• Flux Dev FP8 (flux1-dev-fp8.safetensors) — ~12 GB, near-identical quality. Use --fast flag.
• Flux Dev GGUF Q8_0 — ~13 GB.
• Flux Dev GGUF Q4_K_S — ~7 GB. Use the ComfyUI-GGUF custom node by city96.

[UnetLoader (GGUF)] → MODEL
[DualCLIPLoader (GGUF)] → CLIP    # T5-XXL + CLIP-L
[Load VAE] → VAE

---

LoRAs and Embeddings {#loras}

LoRA chains

[Load Checkpoint] → MODEL ──────┐
                  → CLIP ───────┤
                                ↓
                       [Load LoRA #1]
                            ↓
                       [Load LoRA #2]
                            ↓
                       [Load LoRA #3]
                            ↓
                          MODEL → KSampler
                          CLIP → CLIP Text Encode

Each LoRA node has strength_model and strength_clip (0.0-1.5 typical, 1.0 default). Stack as many as you want — but >3 strong LoRAs usually conflict.

Embeddings (textual inversion)

Reference in your prompt with embedding:filename (without extension):

masterpiece, beautiful landscape, embedding:negative_easy in negative

Place .pt / .safetensors files in models/embeddings/.

Best LoRA sources in 2026

• Civitai — largest collection, NSFW filter optional.
• Hugging Face — official model authors.
• Tensor.Art — curated workflows.

Always check the LoRA's recommended trigger words and base model — an SDXL LoRA does not work on Flux.

---

ControlNet — Composition, Pose, Depth, Canny {#controlnet}

ControlNet conditions generation on a structural input (pose, depth map, edges, etc.).

Pattern

[Load Image] → IMAGE → [Canny Preprocessor] → IMAGE
                                                ↓
[Load ControlNet (canny)] → CONTROL_NET ─────┐ │
                                              ↓ ↓
[CLIP Text Encode pos] → CONDITIONING → [Apply ControlNet] → CONDITIONING
                                                      ↓
                                                [KSampler]

Common preprocessors

SDXL vs Flux ControlNet

SDXL has the most mature ControlNet ecosystem (xinsir, kohya, lllyasviel). Flux ControlNets are catching up — InstantX, Shakker-Labs, and Black Forest Labs ship Flux ControlNets but coverage is narrower.

---

IPAdapter — Image Prompts and Style Transfer {#ipadapter}

IPAdapter conditions generation on a reference image (instead of a text prompt). Best for style transfer, character consistency, and "make it look like this" workflows.

[Load Image (reference)] → IMAGE
                            ↓
[IPAdapter Unified Loader] → MODEL, IPADAPTER
                                   ↓
[IPAdapter Advanced] ← MODEL ←─────┘
        ↓ MODEL
[KSampler]

Use the ComfyUI_IPAdapter_plus custom node by cubiq.

IPAdapter strength

IPAdapter FaceID

For consistent character across generations: ip-adapter-faceid-portrait_sdxl.bin + face embedding extracted with InsightFace. One reference image → consistent character in any pose / scene / outfit.

---

Inpainting, Outpainting, and Upscaling {#inpainting}

Inpainting

[Load Image] → IMAGE
[Load Image (mask)] → IMAGE
[Image to Mask] → MASK
[VAE Encode (Inpaint)] → LATENT (with masked region noised)
[KSampler] (with denoise=0.8) → LATENT
[VAE Decode] → IMAGE

Use the dedicated inpainting checkpoint (e.g. sd_xl_base_1.0_inpainting_0.1.safetensors) for best results. Set sampler denoise to 0.7-0.95.

Outpainting

Use Pad Image for Outpainting node → mask the new edges → inpaint.

Upscaling

Two stages:

1. Latent upscale (cheap, blurry) — Latent Upscale by node, factor 2.0.
2. Image upscale model (sharp) — Upscale Image (using Model) with 4x-UltraSharp or RealESRGAN_x4plus_anime_6B for anime.

Or iterative upscale: small image → upscale 1.5x → low-denoise sampler pass → upscale 1.5x again. Best quality, slowest.

---

Regional Prompting and Conditioning Combine {#regional}

To prompt different regions of the image differently (left side: knight, right side: wizard):

Use ComfyUI_Cutoff or ComfyUI-RegionalPrompter custom nodes. Pattern:

[CLIP Text Encode "knight"] → COND_LEFT
[CLIP Text Encode "wizard"] → COND_RIGHT
[Conditioning (Set Area)] (left half) → COND_LEFT_AREA
[Conditioning (Set Area)] (right half) → COND_RIGHT_AREA
[Conditioning Combine] → COND_FINAL → KSampler

Areas are specified in pixel coordinates. Resolution must match Empty Latent Image size.

---

Flux: Dev, Schnell, GGUF, Quantized {#flux}

Flux is a 12B-parameter Diffusion Transformer (DiT) — different architecture from SD's UNet, with better prompt adherence and text rendering.

Files needed

models/unet/flux1-dev.safetensors           # 24 GB BF16
models/clip/t5xxl_fp16.safetensors          # 9.8 GB
models/clip/clip_l.safetensors              # 246 MB
models/vae/ae.safetensors                   # 335 MB

For 16 GB VRAM use flux1-dev-fp8.safetensors (12 GB) and t5xxl_fp8_e4m3fn.safetensors (5 GB).

Workflow

[Load Diffusion Model] → MODEL (flux1-dev)
[DualCLIPLoader] → CLIP (clip_l + t5xxl)
[Load VAE] → VAE (ae.safetensors)
[CLIP Text Encode] → CONDITIONING
[Empty Latent Image] (1024×1024) → LATENT
[KSamplerAdvanced] (20 steps, cfg=1.0, euler, simple scheduler)
[VAE Decode] → IMAGE

Flux uses cfg=1.0 (no classifier-free guidance) — set CFG to 1.0 always. Different from SD which uses CFG 5-10.

Flux Schnell

Same workflow, but use flux1-schnell.safetensors and 4 steps. ~5x faster, slightly lower quality.

Flux LoRA

[Load LoRA] (after Load Diffusion Model)

Most Flux LoRAs work at 0.7-1.0 strength. Civitai now has 3,000+ Flux LoRAs.

---

Video Generation: Wan 2.2, HunyuanVideo, Mochi {#video}

Wan 2.2 (recommended starting point)

Alibaba's open-source video model. 5-10 second 720p clips, ~24 GB VRAM at Q8 GGUF.

[UnetLoader (GGUF)] → MODEL (wan2.2-i2v-q8_0.gguf)
[DualCLIPLoader] → CLIP (umt5-xxl encoder)
[Load VAE] → VAE
[Load Image] → IMAGE (first frame for image-to-video)
[CLIP Text Encode] → CONDITIONING
[WanImageToVideo] → LATENT (sequence)
[KSampler] (30 steps)
[VAE Decode (sequence)] → IMAGES
[Video Combine (FFmpeg)] → MP4

Render time on RTX 4090: ~6-10 minutes for 5 seconds at 720p.

HunyuanVideo

Tencent's 13B video DiT. Highest quality, ~40 GB BF16 (fits in 24 GB at Q4 GGUF). Use kijai/ComfyUI-HunyuanVideoWrapper.

Mochi

Genmo's 10B model. Lower VRAM (~16 GB), faster, slightly lower quality than Hunyuan.

Frame interpolation and upscaling

After generating a 24fps video:

• RIFE for interpolation to 60fps (ComfyUI-Frame-Interpolation).
• Real-ESRGAN x4 for upscaling to 1440p / 4K.

---

API Mode and Programmatic Use {#api}

ComfyUI exposes POST /prompt for queuing workflows programmatically.

import json
import requests
import uuid

# Load workflow JSON saved from UI (Save (API Format))
with open("workflow_api.json") as f:
    workflow = json.load(f)

# Modify any node — e.g., set positive prompt
workflow["6"]["inputs"]["text"] = "a cyberpunk samurai at dawn"
workflow["3"]["inputs"]["seed"] = 42

# Submit
client_id = str(uuid.uuid4())
resp = requests.post("http://127.0.0.1:8188/prompt", json={
    "prompt": workflow,
    "client_id": client_id,
})
prompt_id = resp.json()["prompt_id"]

# Poll history
history = requests.get(f"http://127.0.0.1:8188/history/{prompt_id}").json()

WebSocket (/ws) streams progress events. Output images are at /view?filename=...&subfolder=....

---

VRAM Optimization Tricks {#vram-optimization}

For 12 GB VRAM running Flux Dev: --fast + Q8 T5 + Tile VAE + --lowvram if needed.

---

Speed Tuning: Sage Attention, Triton, Compile {#speed-tuning}

Sage Attention

pip install sageattention
python main.py --use-sage-attention

Sage Attention is a faster attention implementation than xformers / SDPA on Ada and Blackwell. 15-30% throughput improvement on Flux and SDXL.

Triton (Linux/WSL2)

pip install triton

Triton enables more efficient kernels. Required by Sage Attention and several custom node packs. Windows native does not officially support Triton; use WSL2.

torch.compile

Some custom node packs (kijai's wrappers, ComfyUI-MultiGPU) expose torch.compile modes. Compilation takes 1-3 minutes on first run but generation is 10-25% faster afterward. Mode max-autotune is fastest but adds ~5 min compile time.

TeaCache / FBCache

Caches diffusion model attention/MLP outputs across consecutive timesteps. 1.5-2.0x speedup with 1-3% quality loss. Custom nodes: ComfyUI-TeaCache, ComfyUI-FBCache.

---

Common Custom Node Packs Worth Installing {#custom-nodes}

---

Troubleshooting {#troubleshooting}

---

FAQ {#faq}

See answers to common ComfyUI questions below.

---

Sources: ComfyUI GitHub | ComfyUI Manager | Black Forest Labs Flux | Stability AI SD 3.5 | kijai's video nodes | city96 GGUF nodes | Internal benchmarks on RTX 3090, 4090, 5090, RX 7900 XTX, M4 Max.

Related guides on Local AI Master:

• Flux Local Image Generation
• Local AI Photographers
• Best GPUs for AI 2025
• AMD ROCm Setup for Local LLMs
• Local AI Video Generation
• vLLM Complete Setup Guide
• Multimodal AI Standard