Improving Merge Pipeline Throughput in Continuous Integration via Pull Request Prioritization
By: Maximilian Jungwirth, Martin Gruber, Gordon Fraser
Potential Business Impact:
Speeds up software updates by predicting good changes.
Integrating changes into large monolithic software repositories is a critical step in modern software development that substantially impacts the speed of feature delivery, the stability of the codebase, and the overall productivity of development teams. To ensure the stability of the main branch, many organizations use merge pipelines that test software versions before the changes are permanently integrated. However, the load on merge pipelines is often so high that they become bottlenecks, despite the use of parallelization. Existing optimizations frequently rely on specific build systems, limiting their generalizability and applicability. In this paper we propose to optimize the order of PRs in merge pipelines using practical build predictions utilizing only historical build data, PR metadata, and contextual information to estimate the likelihood of successful builds in the merge pipeline. By dynamically prioritizing likely passing PRs during peak hours, this approach maximizes throughput when it matters most. Experiments conducted on a real-world, large-scale project demonstrate that predictive ordering significantly outperforms traditional first-in-first-out (FIFO), as well as non-learning-based ordering strategies. Unlike alternative optimizations, this approach is agnostic to the underlying build system and thus easily integrable into existing automated merge pipelines.
Similar Papers
On Queueing Theory for Large-Scale CI/CD Pipelines Optimization
Software Engineering
Makes computer code updates faster and cheaper.
Analyzing DevOps Practices Through Merge Request Data: A Case Study in Networking Software Company
Software Engineering
Helps teams make software faster and better.
Towards an Optimized Benchmarking Platform for CI/CD Pipelines
Distributed, Parallel, and Cluster Computing
Find software problems faster, saving computer power.