Try ICEdit Demo

Key Features of ICEdit

• Efficient Image Editing

  Achieves state-of-the-art results with only 0.5% of the training data and 1% of the parameters required by prior methods.

• Instruction-Based Editing

  Enables robust image modification via natural language prompts, ensuring high precision and efficiency.

• In-Context Generation

  Leverages large-scale Diffusion Transformer for enhanced generation capacity and native contextual awareness.

• Hybrid Tuning Strategy

  Combines LoRA and MoE for efficient adaptation and dynamic expert routing, ensuring flexibility without extensive retraining.

• Early Filter Inference-Time Scaling

  Uses vision-language models to select better initial noise early, improving edit quality.

• Open-Source and Cost-Effective

  Offers a more open-source approach with lower costs, faster processing speed (9 seconds per image), and powerful performance.

How ICEdit Works

ICEdit leverages large-scale Diffusion Transformer (DiT) technology with three key innovations:

• In-Context Editing Framework

  Uses in-context prompting to understand and apply edits without requiring structural changes to the model.

• LoRA-MoE Hybrid Tuning

  Combines LoRA (Low-Rank Adaptation) with Mixture-of-Experts routing to enable flexible and efficient adaptation.

Source: river-zhang.github.io/ICEdit-gh-pages/

• Inference-Time Scaling

  Employs vision-language models to select better initial noise patterns, significantly improving edit quality.

ICEdit represents a significant advancement in making sophisticated image editing accessible to everyone, regardless of their technical expertise in graphic design or image manipulation.

ICEdit Demo

ICEdit AI Installation in Local

Conda Environment Setup

conda create -n icedit python=3.10
conda activate icedit
pip install -r requirements.txt
pip install -U huggingface_hub

Download Pretrained Weights

If you can connect to Huggingface, you don't need to download the weights manually. Otherwise, download the following weights to your local machine:

• Flux.1-fill-dev
• ICEdit-MoE-LoRA

Inference in Bash (w/o VLM Inference-time Scaling)

Now you can have a try!
Note: Our model can only edit images with a width of 512 pixels (no restriction on height). If you pass in an image with a different width, the model will automatically resize it to 512 pixels.
If the model fails to generate the expected results, try changing the --seed parameter. Inference-time Scaling with VLM can also help improve results.

python scripts/inference.py --image assets/girl.png \
                            --instruction "Make her hair dark green and her clothes checked." \
                            --seed 42

Editing a 512×768 image requires 35 GB of GPU memory. If you need to run on a system with 24 GB of GPU memory (e.g., NVIDIA RTX3090), add the --enable-model-cpu-offload parameter:

python scripts/inference.py --image assets/girl.png \
                            --instruction "Make her hair dark green and her clothes checked." \
                            --enable-model-cpu-offload

If you have downloaded the pretrained weights locally, please pass the parameters during inference, as in:

python scripts/inference.py --image assets/girl.png \
                            --instruction "Make her hair dark green and her clothes checked." \
                            --flux-path /path/to/flux.1-fill-dev \
                            --lora-path /path/to/ICEdit-MoE-LoRA

Inference in Gradio Demo

We provide a Gradio demo for a more user-friendly image editing experience. Run the following command to start the demo:

python scripts/gradio_demo.py --port 7860

Like the inference script, if you want to run the demo on a system with 24 GB of GPU memory, add the --enable-model-cpu-offload parameter. If you have downloaded the pretrained weights locally, pass the parameters as in:

python scripts/gradio_demo.py --port 7860 \
                              --flux-path /path/to/flux.1-fill-dev (optional) \
                              --lora-path /path/to/ICEdit-MoE-LoRA (optional) \
                              --enable-model-cpu-offload (optional)

Then you can open the link in your browser to edit images!