A Self-Improving Coding Agent
By: Maxime Robeyns, Martin Szummer, Laurence Aitchison
Potential Business Impact:
Computers fix themselves to do tasks better.
Recent advancements in Large Language Models (LLMs) have spurred interest in deploying LLM agents to undertake tasks in the world. LLMs are often deployed in agent systems: code that orchestrates LLM calls and provides them with tools. We demonstrate that an agent system, equipped with basic coding tools, can autonomously edit itself, and thereby improve its performance on benchmark tasks. We find performance gains from 17% to 53% on a random subset of SWE Bench Verified, with additional performance gains on LiveCodeBench, as well as synthetically generated agent benchmarks. Our work represents an advancement in the automated and open-ended design of agentic systems, and demonstrates a data-efficient, non gradient-based learning mechanism driven by LLM reflection and code updates.
Similar Papers
ResearchCodeAgent: An LLM Multi-Agent System for Automated Codification of Research Methodologies
Software Engineering
Writes computer code from research papers.
A Survey on Code Generation with LLM-based Agents
Software Engineering
Computers write and fix computer programs themselves.
From Code Foundation Models to Agents and Applications: A Practical Guide to Code Intelligence
Software Engineering
Helps computers write computer programs from words.