Wiki source code of Flux.1 Bild KI

Version 2.1 by René Schmidt on 2025/03/25 22:08
Hide last authors
René Schmidt 1.1 1 [[Tutorial>>url:https://sandner.art/articles/tutorial/]]
2
3 = Fast Hyperrealism with FLUX.1: Local Generation Guide =
4
5 [[Daniel Sandner >>url:https://sandner.art/authors/daniel-sandner/]]August 16, 2024
6
7 FLUX.1 is suite of text-to-image models using novel functional approach to image synthesis. FLUX is creation of AI researchers from The Black Forest Team (VQGAN, Latent Diffusion, Stable Diffusion XL, SVD). FLUX.1  offers better text render, better details and compositions, and improvements in rendering anatomy. In this article we will install FLUX.1 (schnell) for local image generation (you may use the tutorial also to install the //dev //version).
8
9 * [[Introduction and System Requirements>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#introduction-andandnbspsystem-requirements]]
10 * [[Portable Setup of ComfyUI>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#portable-setup-of-comfyui]]
11 ** [[Download and Installation>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#download-and-installation]]
12 ** [[Adjusting System Paths>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#adjusting-system-paths]]
13 ** [[Running ComfyUI and Updating>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#running-comfyui-and-updating]]
14 ** [[Installing ComfyUI-Manager>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#installing-comfyui-manager]]
15 ** [[GGUF: Installing Support for GGUF Format Models and Encoders for Portable ComfyUI>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#gguf-installing-support-for-gguf-format-models-and-encoders-for-portable-comfyui]]
16 * [[FLUX.1>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#flux1]]
17 ** [[Needed Files/Models>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#needed-filesmodels]]
18 ** [[Workflows>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#workflows]]
19 ** [[FLUX.1 LoRAs>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#flux1-loras]]
20 ** [[FLUX.1 ControlNet>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#flux1-controlnet]]
21 ** [[Using X-LABS (flux-dev,) Controlnets in ComfyUI>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#using-x-labs-flux-dev-controlnets-in-comfyui]]
22 * [[Forge UI Supports FLUX.1 >>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#forge-ui-supports-flux1andnbsp]]
23 * [[Conclusion>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#conclusion]]
24 * [[References>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#references]]
25
26 == Introduction and System Requirements ==
27
28 You'll need an SDXL-capable machine. All workflows in this article were tested on an NVIDIA RTX A4000 with 16GB VRAM (system has 64GB RAM). I recommend at least 16GB VRAM for optimal performance of FLUX.1 (schnell), the more memory the better. FLUX.1 (Pro) version is more resource hungry and is available as an API at the time being.
29
30 If you are already using ComfyUI or SWARM, you may jump directly to FLUX.1 files download.[[image:https://sandner.art/media/posts/74/flux-sd-sandner-article.jpg||alt="FLUX generative AI model photorealism and stylized hyperrealistic portraits" height="683" width="2048"]]
31
32 FLUX excels at hyperrealistic details and produces notably improved hand poses out of the box.
33
34 == Portable Setup of ComfyUI ==
35
36 === Download and Installation ===
37
38 * Download Portable ComfyUI from here: [[https:~~/~~/github.com/comfyanonymous/ComfyUI/releases>>url:https://github.com/comfyanonymous/ComfyUI/releases]]
39 * Unzip in your target folder. Download 7zip from here: [[www.7-zip.org/>>url:https://www.7-zip.org/]]
40
41 === Adjusting System Paths ===
42
43 If you are just starting your AI generation journey, just put your downloaded models into proper ComfyUI folders. If you have models already downloaded for A1111 or SWARM installations, you do not need to download or duplicate those files.
44
45 * Find the file //**extra_model_paths.yaml.example**// in your ComfyUI, rename it to **extra_model_paths.yaml. Then open the .yaml file and add:**
46 * In "a111: base_path:" section add the path to your models, f.i. C:/stable-diffusion-webui
47 * At the end of the section (after controlnet: models/ControlNetentry), add two lines (needed for FLUX workflows):
48 * unet: models/unet
49 clip: models/clip
50
51 === Running ComfyUI and Updating ===
52
53 You will run Portable ComfyUI with run_nvidia_gpu.bat. You may update to the latest version in the future with //**update/update_comfyui.bat**//.
54
55 === Installing ComfyUI-Manager ===
56
57 For an effective work with various nodes, install ComfyUI-Manager (it will appear as the **Manager** button in ComfyUI main menu)
58
59 * Install git (you probably have it already)
60 * Go to **ComfyUI/custom_nodes directory.**
61 * **From there, run cmd command to open the terminal**
62 * **Enter **git clone https:~/~/github.com/ltdrdata/ComfyUI-Manager.git
63 * If you restart ComfyUI after the installation finishes (usually couple of seconds), ComfyUI-Manager should be ready.[[image:https://sandner.art/media/posts/74/FLUX-realism.jpg||alt="FLUX.1 schnell: Fast realism in details" height="700" width="2100"]]
64
65 FLUX.1: Realism in details
66
67 === GGUF: Installing Support for GGUF Format Models and Encoders for Portable ComfyUI ===
68
69 Currently ComfyUI does not support GGUF format yet. You can install custom nodes from here [[https:~~/~~/github.com/city96/ComfyUI-GGUF>>url:https://github.com/city96/ComfyUI-GGUF]]  (this way you can use GGUF dev and schnell quantized models for speed and lower filesize. Simplest installation for ComfyUI Portable:
70
71 * go to your Comfy folder where run_nvidia_gpu.bat resides
72 * run terminal cmd from this folder with the command:
73
74 {{{git clone https://github.com/city96/ComfyUI-GGUF ComfyUI/custom_nodes/ComfyUI-GGUF
75 .\python_embeded\python.exe -s -m pip install -r .\ComfyUI\custom_nodes\ComfyUI-GGUF\requirements.txt}}}
76
77 Now you can use custom GGUF nodes Unet Loader(GGUF) for models and DualCLIPLoader(GGUF) for encoders. Check examples in my test workflows [[https:~~/~~/github.com/sandner-art/ai-research/tree/main/FLUX>>url:https://github.com/sandner-art/ai-research/tree/main/FLUX]]
78
79 == FLUX.1 ==
80
81 === Needed Files/Models ===
82
83 1. Download text encoders [[https:~~/~~/huggingface.co/comfyanonymous/flux_text_encoders/tree/main>>url:https://huggingface.co/comfyanonymous/flux_text_encoders/tree/main]] . Files **clip_l.safetensors** and **t5xxl_fp8_e4m3fn.safetensors** will work with the provided workflows.The files will go to folder **models/clip**.
84 Alternative text encoders for better text rendering in the image are here: [[https:~~/~~/huggingface.co/zer0int/CLIP-GmP-ViT-L-14/tree/main >>url:https://huggingface.co/zer0int/CLIP-GmP-ViT-L-14/tree/main]] (try ViT-L-14-TEXT-detail-improved-hiT-GmP-TE-only-HF or ViT-L-14-TEXT-detail-improved-hiT-GmP-HF.
85 1. Download VAE [[https:~~/~~/huggingface.co/black-forest-labs/FLUX.1-schnell/tree/main/vae>>url:https://huggingface.co/black-forest-labs/FLUX.1-schnell/tree/main/vae]] The file **diffusion_pytorch_model.safetensors** (rename it as you want) will go to folder **models/VAE**.
86 1. Download the FLUX.1 model [[https:~~/~~/huggingface.co/Kijai/flux-fp8/tree/main>>url:https://huggingface.co/Kijai/flux-fp8/tree/main]] The file **flux1-schnell-fp8.safetensors** will go to folder **models/unet**. You may, of course, experiment with the development (dev) version as well.
87
88 Note that the **main model goes to models/unet folder** (not the usual Stable Diffusion folder). If there is no unet folder in your SD models path, make unet folder manually).
89
90 === Workflows ===
91
92 Run ComfyUI. Download the images or workflow from the test folder  [[https:~~/~~/github.com/sandner-art/ai-research/tree/main/FLUX>>url:https://github.com/sandner-art/ai-research/tree/main/FLUX]] and drag+drop into ComfyUI workplace in the browser. This will load the node workflow into ComfyUI.
93
94 If you see some red nodes indicating an error, install missing nodes with ComfyUI-Manager. Open the **MANAGER** button from the menu and use "Install Missing Custom Nodes".
95
96 === FLUX.1 LoRAs ===
97
98 FLUX.1 test LoRAs [[https:~~/~~/huggingface.co/XLabs-AI/flux-lora-collection/tree/main>>url:https://huggingface.co/XLabs-AI/flux-lora-collection/tree/main]] Workflows in references.
99
100 === FLUX.1 ControlNet ===
101
102 ControlNet  Collections [[https:~~/~~/huggingface.co/XLabs-AI/flux-controlnet-collections/tree/main>>url:https://huggingface.co/XLabs-AI/flux-controlnet-collections/tree/main]] . Workflows in references.
103
104 === Using X-LABS (flux-dev,) Controlnets in ComfyUI ===
105
106 For the XLABS test workflows to work, you need to put Controlnets models into the folder \ComfyUI\models\xlabs\controlnets, otherwise the workflow will not find the models!
107
René Schmidt 2.1 108 == Forge UI Supports FLUX.1 ==
René Schmidt 1.1 109
110 You may run FLUX models also in A1111 Forge now, check [[this article for more details>>url:https://sandner.art/flux1-in-forge-ui-setup-guide-with-sdsdxl-tips/]].[[image:https://sandner.art/media/posts/74/FLUX-sandner-art.jpg||alt="FLUX.1 (schnell) image generation for free on local machine" height="683" width="2048"]]
111
112 FLUX.1 (schnell)
113
René Schmidt 2.1 114
115
116 = ComfyUI Manager =
117
118
119 **ComfyUI-Manager** is an extension designed to enhance the usability of [[ComfyUI>>url:https://github.com/comfyanonymous/ComfyUI]]. It offers management functions to **install, remove, disable, and enable** various custom nodes of ComfyUI. Furthermore, this extension provides a hub feature and convenience functions to access a wide range of information within ComfyUI.
120
121 [[~[~[image:https://raw.githubusercontent.com/ltdrdata/ComfyUI-extension-tutorials/refs/heads/Main/ComfyUI-Manager/images/dialog.jpg~|~|alt="menu"~]~]>>url:https://raw.githubusercontent.com/ltdrdata/ComfyUI-extension-tutorials/refs/heads/Main/ComfyUI-Manager/images/dialog.jpg]]
122
123 == NOTICE ==
124
125
126 * V3.16: Support for uv has been added. Set use_uv in config.ini.
127 * V3.10: double-click feature is removed
128 ** This feature has been moved to [[https:~~/~~/github.com/ltdrdata/comfyui-connection-helper>>url:https://github.com/ltdrdata/comfyui-connection-helper]]
129 * V3.3.2: Overhauled. Officially supports [[https:~~/~~/comfyregistry.org/>>url:https://comfyregistry.org/]].
130 * You can see whole nodes info on [[ComfyUI Nodes Info>>url:https://ltdrdata.github.io/]] page.
131
132 == Installation ==
133
134
135 === Installation[method1] (General installation method: ComfyUI-Manager only) ===
136
137
138 To install ComfyUI-Manager in addition to an existing installation of ComfyUI, you can follow the following steps:
139
140 1. goto ComfyUI/custom_nodes dir in terminal(cmd)
141 1. git clone https:~/~/github.com/ltdrdata/ComfyUI-Manager comfyui-manager
142 1. Restart ComfyUI
143
René Schmidt 1.1 144 == Conclusion ==
145
146 FLUX.1's potential is truly impressive. Even as a base model, it generates astonishingly detailed images that often surpass expectations. While prompt engineering requires a bit more finesse due to the model's tendency towards hyperrealism, the quality is undeniable. The combination of specific details and careful style guidance is key.
147
148 Also the chronic pain of SD models—generating hands and limbs—is considerably better (even at quite crazy angles and poses).  However, character interaction with objects is not so great yet.
149
150 As we will surely see a fast evolution with custom LoRAs and finetuned versions of the base model, it seems most text-to-image models have found its contender.
151
152 You can test FLUX.1 (Schnell, Dev, Pro) on Civitai, and also train LoRAs for FLUX.1 there.
153
154 == References ==
155
156 * Link to ComfyUI GitHub repository [[https:~~/~~/github.com/comfyanonymous/ComfyUI>>url:https://github.com/comfyanonymous/ComfyUI]]
157 * Link to FLUX models [[https:~~/~~/huggingface.co/black-forest-labs>>url:https://huggingface.co/black-forest-labs]]
158 * Blackforest.ai, creator of FLUX [[https:~~/~~/blackforestlabs.ai/>>url:https://blackforestlabs.ai/]]
159 * Announcement  [[https:~~/~~/blackforestlabs.ai/announcing-black-forest-labs/>>url:https://blackforestlabs.ai/announcing-black-forest-labs/]]
160 * My test FLUX workflows [[https:~~/~~/github.com/sandner-art/ai-research/tree/main/FLUX>>url:https://github.com/sandner-art/ai-research/tree/main/FLUX]]
161 * FLUX workflows (various) [[https:~~/~~/openart.ai/workflows/all?keyword=flux>>url:https://openart.ai/workflows/all?keyword=flux]]
162 * FLUX Weights [[https:~~/~~/huggingface.co/silveroxides/flux1-nf4-weights/tree/main>>url:https://huggingface.co/silveroxides/flux1-nf4-weights/tree/main]]
163 * GGUF T5 encoders [[https:~~/~~/huggingface.co/city96/t5-v1_1-xxl-encoder-gguf/tree/main>>url:https://huggingface.co/city96/t5-v1_1-xxl-encoder-gguf/tree/main]]
164 * GGUF DEV + SCHNELL version [[https:~~/~~/huggingface.co/city96>>url:https://huggingface.co/city96]]
165 * ComfyUI GGUF [[https:~~/~~/github.com/city96/ComfyUI-GGUF>>url:https://github.com/city96/ComfyUI-GGUF]]

Applications

Navigation

Need help?

If you need help with XWiki you can contact: