LLM Collaboration With Multi-Agent Reinforcement Learning
By: Shuo Liu , Zeyu Liang , Xueguang Lyu and more
Potential Business Impact:
Helps AI agents work together to write and code.
A large amount of work has been done in Multi-Agent Systems (MAS) for modeling and solving problems with multiple interacting agents. However, most LLMs are pretrained independently and not specifically optimized for coordination. Existing LLM fine-tuning frameworks rely on individual rewards, which require complex reward designs for each agent to encourage collaboration. To address these challenges, we model LLM collaboration as a cooperative Multi-Agent Reinforcement Learning (MARL) problem. We develop a multi-agent, multi-turn algorithm, Multi-Agent Group Relative Policy Optimization (MAGRPO), to solve it, building on current RL approaches for LLMs as well as MARL techniques. Our experiments on LLM writing and coding collaboration demonstrate that fine-tuning MAS with MAGRPO enables agents to generate high-quality responses efficiently through effective cooperation. Our approach opens the door to using other MARL methods for LLMs and highlights the associated challenges.
Similar Papers
Heterogeneous Group-Based Reinforcement Learning for LLM-based Multi-Agent Systems
Machine Learning (CS)
Teaches AI groups to work better, faster.
LAMARL: LLM-Aided Multi-Agent Reinforcement Learning for Cooperative Policy Generation
Robotics
Robots learn tasks faster with AI help.
Stronger Together: On-Policy Reinforcement Learning for Collaborative LLMs
Machine Learning (CS)
Teaches AI to work together better for harder tasks.