Lacuna Inc. at SemEval-2025 Task 4: LoRA-Enhanced Influence-Based Unlearning for LLMs
By: Aleksey Kudelya, Alexander Shirnin
Potential Business Impact:
Removes bad info from AI without breaking it.
This paper describes LIBU (LoRA enhanced influence-based unlearning), an algorithm to solve the task of unlearning - removing specific knowledge from a large language model without retraining from scratch and compromising its overall utility (SemEval-2025 Task 4: Unlearning sensitive content from Large Language Models). The algorithm combines classical \textit{influence functions} to remove the influence of the data from the model and \textit{second-order optimization} to stabilize the overall utility. Our experiments show that this lightweight approach is well applicable for unlearning LLMs in different kinds of task.
Similar Papers
LUNE: Efficient LLM Unlearning via LoRA Fine-Tuning with Negative Examples
Machine Learning (CS)
Lets computers forget unwanted information easily.
UnGuide: Learning to Forget with LoRA-Guided Diffusion Models
CV and Pattern Recognition
Removes bad ideas from AI art generators.
BLUR: A Bi-Level Optimization Approach for LLM Unlearning
Machine Learning (CS)
Teaches AI to forget bad or wrong information.