NEP: Autoregressive Image Editing via Next Editing Token Prediction
By: Huimin Wu , Xiaojian Ma , Haozhe Zhao and more
Potential Business Impact:
Changes pictures exactly where you tell it to.
Text-guided image editing involves modifying a source image based on a language instruction and, typically, requires changes to only small local regions. However, existing approaches generate the entire target image rather than selectively regenerate only the intended editing areas. This results in (1) unnecessary computational costs and (2) a bias toward reconstructing non-editing regions, which compromises the quality of the intended edits. To resolve these limitations, we propose to formulate image editing as Next Editing-token Prediction (NEP) based on autoregressive image generation, where only regions that need to be edited are regenerated, thus avoiding unintended modification to the non-editing areas. To enable any-region editing, we propose to pre-train an any-order autoregressive text-to-image (T2I) model. Once trained, it is capable of zero-shot image editing and can be easily adapted to NEP for image editing, which achieves a new state-of-the-art on widely used image editing benchmarks. Moreover, our model naturally supports test-time scaling (TTS) through iteratively refining its generation in a zero-shot manner. The project page is: https://nep-bigai.github.io/
Similar Papers
Next-Embedding Prediction Makes Strong Vision Learners
CV and Pattern Recognition
Teaches computers to see by predicting picture parts.
Visual Autoregressive Modeling for Instruction-Guided Image Editing
CV and Pattern Recognition
Edits pictures perfectly, following your exact words.
The Promise of RL for Autoregressive Image Editing
CV and Pattern Recognition
Edits images accurately with less training