MLLM-CL: Continual Learning for Multimodal Large Language Models
By: Hongbo Zhao , Fei Zhu , Rundong Wang and more
Potential Business Impact:
Lets AI learn new things without forgetting old ones.
Recent Multimodal Large Language Models (MLLMs) excel in vision-language understanding but face challenges in adapting to dynamic real-world scenarios that require continuous integration of new knowledge and skills. While continual learning (CL) offers a potential solution, existing benchmarks and methods suffer from critical limitations. In this paper, we introduce MLLM-CL, a novel benchmark encompassing domain and ability continual learning, where the former focuses on independently and identically distributed (IID) evaluation across evolving mainstream domains, whereas the latter evaluates on non-IID scenarios with emerging model ability. Methodologically, we propose preventing catastrophic interference through parameter isolation, along with an MLLM-based routing mechanism. Extensive experiments demonstrate that our approach can integrate domain-specific knowledge and functional abilities with minimal forgetting, significantly outperforming existing methods.
Similar Papers
When Continue Learning Meets Multimodal Large Language Model: A Survey
Machine Learning (CS)
Helps AI learn new things without forgetting old ones.
Continual Learning for VLMs: A Survey and Taxonomy Beyond Forgetting
CV and Pattern Recognition
Helps AI learn new things without forgetting old ones.
Self-Evolving LLMs via Continual Instruction Tuning
Machine Learning (CS)
Keeps AI smart on new tasks, not forgetting old ones.