X2Edit: Revisiting Arbitrary-Instruction Image Editing through Self-Constructed Data and Task-Aware Representation Learning
By: Jian Ma , Xujie Zhu , Zihao Pan and more
Potential Business Impact:
Makes editing pictures much easier and better.
Existing open-source datasets for arbitrary-instruction image editing remain suboptimal, while a plug-and-play editing module compatible with community-prevalent generative models is notably absent. In this paper, we first introduce the X2Edit Dataset, a comprehensive dataset covering 14 diverse editing tasks, including subject-driven generation. We utilize the industry-leading unified image generation models and expert models to construct the data. Meanwhile, we design reasonable editing instructions with the VLM and implement various scoring mechanisms to filter the data. As a result, we construct 3.7 million high-quality data with balanced categories. Second, to better integrate seamlessly with community image generation models, we design task-aware MoE-LoRA training based on FLUX.1, with only 8\% of the parameters of the full model. To further improve the final performance, we utilize the internal representations of the diffusion model and define positive/negative samples based on image editing types to introduce contrastive learning. Extensive experiments demonstrate that the model's editing performance is competitive among many excellent models. Additionally, the constructed dataset exhibits substantial advantages over existing open-source datasets. The open-source code, checkpoints, and datasets for X2Edit can be found at the following link: https://github.com/OPPO-Mente-Lab/X2Edit.
Similar Papers
MultiEdit: Advancing Instruction-based Image Editing on Diverse and Challenging Tasks
CV and Pattern Recognition
Teaches computers to edit pictures better with words.
Step1X-Edit: A Practical Framework for General Image Editing
CV and Pattern Recognition
Makes computer pictures change like magic.
SuperEdit: Rectifying and Facilitating Supervision for Instruction-Based Image Editing
CV and Pattern Recognition
Teaches computers to edit pictures better with clearer instructions.