FoundIR-v2: Optimizing Pre-Training Data Mixtures for Image Restoration Foundation Model
By: Xiang Chen , Jinshan Pan , Jiangxin Dong and more
Potential Business Impact:
Fixes blurry pictures better by balancing training.
Recent studies have witnessed significant advances in image restoration foundation models driven by improvements in the scale and quality of pre-training data. In this work, we find that the data mixture proportions from different restoration tasks are also a critical factor directly determining the overall performance of all-in-one image restoration models. To this end, we propose a high-capacity diffusion-based image restoration foundation model, FoundIR-v2, which adopts a data equilibrium scheduling paradigm to dynamically optimize the proportions of mixed training datasets from different tasks. By leveraging the data mixing law, our method ensures a balanced dataset composition, enabling the model to achieve consistent generalization and comprehensive performance across diverse tasks. Furthermore, we introduce an effective Mixture-of-Experts (MoE)-driven scheduler into generative pre-training to flexibly allocate task-adaptive diffusion priors for each restoration task, accounting for the distinct degradation forms and levels exhibited by different tasks. Extensive experiments demonstrate that our method can address over 50 sub-tasks across a broader scope of real-world scenarios and achieves favorable performance against state-of-the-art approaches.
Similar Papers
Equipping Vision Foundation Model with Mixture of Experts for Out-of-Distribution Detection
CV and Pattern Recognition
Helps computers spot weird, new things.
FAPE-IR: Frequency-Aware Planning and Execution Framework for All-in-One Image Restoration
CV and Pattern Recognition
Fixes blurry pictures by understanding what's wrong.
An Efficient and Mixed Heterogeneous Model for Image Restoration
CV and Pattern Recognition
Fixes blurry pictures by combining different computer "brains."