Zero-Shot Adaptation of Parameter-Efficient Fine-Tuning in Diffusion Models
By: Farzad Farhadzadeh , Debasmit Das , Shubhankar Borse and more
Potential Business Impact:
Lets AI art tools learn new styles instantly.
We introduce ProLoRA, enabling zero-shot adaptation of parameter-efficient fine-tuning in text-to-image diffusion models. ProLoRA transfers pre-trained low-rank adjustments (e.g., LoRA) from a source to a target model without additional training data. This overcomes the limitations of traditional methods that require retraining when switching base models, often challenging due to data constraints. ProLoRA achieves this via projection of source adjustments into the target model's weight space, leveraging subspace and null space similarities and selectively targeting aligned layers. Evaluations on established text-to-image models demonstrate successful knowledge transfer and comparable performance without retraining.
Similar Papers
MetaLoRA: Tensor-Enhanced Adaptive Low-Rank Fine-tuning
Machine Learning (CS)
Teaches computers to learn new things faster.
ElaLoRA: Elastic & Learnable Low-Rank Adaptation for Efficient Model Fine-Tuning
Machine Learning (CS)
Makes AI learn faster with less effort.
Cross-LoRA: A Data-Free LoRA Transfer Framework across Heterogeneous LLMs
Machine Learning (CS)
Moves AI skills between different computer brains.