Flux.1 Bild KI

1

[[Tutorial>>url:https://sandner.art/articles/tutorial/]]

2

3

= Fast Hyperrealism with FLUX.1: Local Generation Guide =

4

5

[[Daniel Sandner >>url:https://sandner.art/authors/daniel-sandner/]]August 16, 2024

6

7

FLUX.1 is suite of text-to-image models using novel functional approach to image synthesis. FLUX is creation of AI researchers from The Black Forest Team (VQGAN, Latent Diffusion, Stable Diffusion XL, SVD). FLUX.1 offers better text render, better details and compositions, and improvements in rendering anatomy. In this article we will install FLUX.1 (schnell) for local image generation (you may use the tutorial also to install the //dev //version).

8

9

* [[Introduction and System Requirements>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#introduction-andandnbspsystem-requirements]]

10

* [[Portable Setup of ComfyUI>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#portable-setup-of-comfyui]]

11

** [[Download and Installation>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#download-and-installation]]

12

** [[Adjusting System Paths>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#adjusting-system-paths]]

13

** [[Running ComfyUI and Updating>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#running-comfyui-and-updating]]

14

** [[Installing ComfyUI-Manager>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#installing-comfyui-manager]]

15

** [[GGUF: Installing Support for GGUF Format Models and Encoders for Portable ComfyUI>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#gguf-installing-support-for-gguf-format-models-and-encoders-for-portable-comfyui]]

16

* [[FLUX.1>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#flux1]]

17

** [[Needed Files/Models>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#needed-filesmodels]]

18

** [[Workflows>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#workflows]]

19

** [[FLUX.1 LoRAs>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#flux1-loras]]

20

** [[FLUX.1 ControlNet>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#flux1-controlnet]]

21

** [[Using X-LABS (flux-dev,) Controlnets in ComfyUI>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#using-x-labs-flux-dev-controlnets-in-comfyui]]

22

* [[Forge UI Supports FLUX.1 >>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#forge-ui-supports-flux1andnbsp]]

23

* [[Conclusion>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#conclusion]]

24

* [[References>>url:https://sandner.art/fast-hyperrealism-with-flux1-local-generation-guide/#references]]

25

26

== Introduction and System Requirements ==

27

28

You'll need an SDXL-capable machine. All workflows in this article were tested on an NVIDIA RTX A4000 with 16GB VRAM (system has 64GB RAM). I recommend at least 16GB VRAM for optimal performance of FLUX.1 (schnell), the more memory the better. FLUX.1 (Pro) version is more resource hungry and is available as an API at the time being.

29

30

If you are already using ComfyUI or SWARM, you may jump directly to FLUX.1 files download.[[image:https://sandner.art/media/posts/74/flux-sd-sandner-article.jpg||alt="FLUX generative AI model photorealism and stylized hyperrealistic portraits" height="683" width="2048"]]

31

32

FLUX excels at hyperrealistic details and produces notably improved hand poses out of the box.

33

34

== Portable Setup of ComfyUI ==

35

36

=== Download and Installation ===

37

38

* Download Portable ComfyUI from here: [[https:~~/~~/github.com/comfyanonymous/ComfyUI/releases>>url:https://github.com/comfyanonymous/ComfyUI/releases]]

39

* Unzip in your target folder. Download 7zip from here: [[www.7-zip.org/>>url:https://www.7-zip.org/]]

40

41

=== Adjusting System Paths ===

42

43

If you are just starting your AI generation journey, just put your downloaded models into proper ComfyUI folders. If you have models already downloaded for A1111 or SWARM installations, you do not need to download or duplicate those files.

44

45

* Find the file //**extra_model_paths.yaml.example**// in your ComfyUI, rename it to **extra_model_paths.yaml. Then open the .yaml file and add:**

46

* In "a111: base_path:" section add the path to your models, f.i. C:/stable-diffusion-webui

47

* At the end of the section (after controlnet: models/ControlNetentry), add two lines (needed for FLUX workflows):

* unet: models/unet

clip: models/clip

=== Running ComfyUI and Updating ===

52

53

You will run Portable ComfyUI with run_nvidia_gpu.bat. You may update to the latest version in the future with //**update/update_comfyui.bat**//.

54

55

=== Installing ComfyUI-Manager ===

56

57

For an effective work with various nodes, install ComfyUI-Manager (it will appear as the **Manager** button in ComfyUI main menu)

58

59

* Install git (you probably have it already)

60

* Go to **ComfyUI/custom_nodes directory.**

61

* **From there, run cmd command to open the terminal**

62

* **Enter **git clone https:~/~/github.com/ltdrdata/ComfyUI-Manager.git

63

* If you restart ComfyUI after the installation finishes (usually couple of seconds), ComfyUI-Manager should be ready.[[image:https://sandner.art/media/posts/74/FLUX-realism.jpg||alt="FLUX.1 schnell: Fast realism in details" height="700" width="2100"]]

64

65

FLUX.1: Realism in details

66

67

=== GGUF: Installing Support for GGUF Format Models and Encoders for Portable ComfyUI ===

68

69

Currently ComfyUI does not support GGUF format yet. You can install custom nodes from here [[https:~~/~~/github.com/city96/ComfyUI-GGUF>>url:https://github.com/city96/ComfyUI-GGUF]] (this way you can use GGUF dev and schnell quantized models for speed and lower filesize. Simplest installation for ComfyUI Portable:

70

71

* go to your Comfy folder where run_nvidia_gpu.bat resides

72

* run terminal cmd from this folder with the command:

73

74

{{{git clone https://github.com/city96/ComfyUI-GGUF ComfyUI/custom_nodes/ComfyUI-GGUF

75

.\python_embeded\python.exe -s -m pip install -r .\ComfyUI\custom_nodes\ComfyUI-GGUF\requirements.txt}}}

76

77

Now you can use custom GGUF nodes Unet Loader(GGUF) for models and DualCLIPLoader(GGUF) for encoders. Check examples in my test workflows [[https:~~/~~/github.com/sandner-art/ai-research/tree/main/FLUX>>url:https://github.com/sandner-art/ai-research/tree/main/FLUX]]

== FLUX.1 ==

=== Needed Files/Models ===

82

83

1. Download text encoders [[https:~~/~~/huggingface.co/comfyanonymous/flux_text_encoders/tree/main>>url:https://huggingface.co/comfyanonymous/flux_text_encoders/tree/main]] . Files **clip_l.safetensors** and **t5xxl_fp8_e4m3fn.safetensors** will work with the provided workflows.The files will go to folder **models/clip**.

84

Alternative text encoders for better text rendering in the image are here: [[https:~~/~~/huggingface.co/zer0int/CLIP-GmP-ViT-L-14/tree/main >>url:https://huggingface.co/zer0int/CLIP-GmP-ViT-L-14/tree/main]] (try ViT-L-14-TEXT-detail-improved-hiT-GmP-TE-only-HF or ViT-L-14-TEXT-detail-improved-hiT-GmP-HF.

85

1. Download VAE [[https:~~/~~/huggingface.co/black-forest-labs/FLUX.1-schnell/tree/main/vae>>url:https://huggingface.co/black-forest-labs/FLUX.1-schnell/tree/main/vae]] The file **diffusion_pytorch_model.safetensors** (rename it as you want) will go to folder **models/VAE**.

86

1. Download the FLUX.1 model [[https:~~/~~/huggingface.co/Kijai/flux-fp8/tree/main>>url:https://huggingface.co/Kijai/flux-fp8/tree/main]] The file **flux1-schnell-fp8.safetensors** will go to folder **models/unet**. You may, of course, experiment with the development (dev) version as well.

87

88

Note that the **main model goes to models/unet folder** (not the usual Stable Diffusion folder). If there is no unet folder in your SD models path, make unet folder manually).

=== Workflows ===

Run ComfyUI. Download the images or workflow from the test folder [[https:~~/~~/github.com/sandner-art/ai-research/tree/main/FLUX>>url:https://github.com/sandner-art/ai-research/tree/main/FLUX]] and drag+drop into ComfyUI workplace in the browser. This will load the node workflow into ComfyUI.

93

94

If you see some red nodes indicating an error, install missing nodes with ComfyUI-Manager. Open the **MANAGER** button from the menu and use "Install Missing Custom Nodes".

=== FLUX.1 LoRAs ===

FLUX.1 test LoRAs [[https:~~/~~/huggingface.co/XLabs-AI/flux-lora-collection/tree/main>>url:https://huggingface.co/XLabs-AI/flux-lora-collection/tree/main]] Workflows in references.

99

100

=== FLUX.1 ControlNet ===

101

102

ControlNet Collections [[https:~~/~~/huggingface.co/XLabs-AI/flux-controlnet-collections/tree/main>>url:https://huggingface.co/XLabs-AI/flux-controlnet-collections/tree/main]] . Workflows in references.

103

104

=== Using X-LABS (flux-dev,) Controlnets in ComfyUI ===

105

106

For the XLABS test workflows to work, you need to put Controlnets models into the folder \ComfyUI\models\xlabs\controlnets, otherwise the workflow will not find the models!

107

108

== Forge UI Supports FLUX.1 ==

109

110

You may run FLUX models also in A1111 Forge now, check [[this article for more details>>url:https://sandner.art/flux1-in-forge-ui-setup-guide-with-sdsdxl-tips/]].[[image:https://sandner.art/media/posts/74/FLUX-sandner-art.jpg||alt="FLUX.1 (schnell) image generation for free on local machine" height="683" width="2048"]]

FLUX.1 (schnell)

== Conclusion ==

FLUX.1's potential is truly impressive. Even as a base model, it generates astonishingly detailed images that often surpass expectations. While prompt engineering requires a bit more finesse due to the model's tendency towards hyperrealism, the quality is undeniable. The combination of specific details and careful style guidance is key.

117

118

Also the chronic pain of SD models—generating hands and limbs—is considerably better (even at quite crazy angles and poses). However, character interaction with objects is not so great yet.

119

120

As we will surely see a fast evolution with custom LoRAs and finetuned versions of the base model, it seems most text-to-image models have found its contender.

121

122

You can test FLUX.1 (Schnell, Dev, Pro) on Civitai, and also train LoRAs for FLUX.1 there.

== References ==

* Link to ComfyUI GitHub repository [[https:~~/~~/github.com/comfyanonymous/ComfyUI>>url:https://github.com/comfyanonymous/ComfyUI]]

127

* Link to FLUX models [[https:~~/~~/huggingface.co/black-forest-labs>>url:https://huggingface.co/black-forest-labs]]

128

* Blackforest.ai, creator of FLUX [[https:~~/~~/blackforestlabs.ai/>>url:https://blackforestlabs.ai/]]

129

* Announcement [[https:~~/~~/blackforestlabs.ai/announcing-black-forest-labs/>>url:https://blackforestlabs.ai/announcing-black-forest-labs/]]

130

* My test FLUX workflows [[https:~~/~~/github.com/sandner-art/ai-research/tree/main/FLUX>>url:https://github.com/sandner-art/ai-research/tree/main/FLUX]]

131

* FLUX workflows (various) [[https:~~/~~/openart.ai/workflows/all?keyword=flux>>url:https://openart.ai/workflows/all?keyword=flux]]

132

* FLUX Weights [[https:~~/~~/huggingface.co/silveroxides/flux1-nf4-weights/tree/main>>url:https://huggingface.co/silveroxides/flux1-nf4-weights/tree/main]]

133

* GGUF T5 encoders [[https:~~/~~/huggingface.co/city96/t5-v1_1-xxl-encoder-gguf/tree/main>>url:https://huggingface.co/city96/t5-v1_1-xxl-encoder-gguf/tree/main]]

134

* GGUF DEV + SCHNELL version [[https:~~/~~/huggingface.co/city96>>url:https://huggingface.co/city96]]

135

* ComfyUI GGUF [[https:~~/~~/github.com/city96/ComfyUI-GGUF>>url:https://github.com/city96/ComfyUI-GGUF]]

Wiki-Quellcode von Flux.1 Bild KI

Anwendungen

Navigation

Benötigen Sie Hilfe?