Gradient Projection onto Historical Descent Directions for Communication-Efficient Federated Learning
By: Arnaud Descours, Léonard Deroose, Jan Ramon
Potential Business Impact:
Makes AI learn faster with less data sent.
Federated Learning (FL) enables decentralized model training across multiple clients while optionally preserving data privacy. However, communication efficiency remains a critical bottleneck, particularly for large-scale models. In this work, we introduce two complementary algorithms: ProjFL, designed for unbiased compressors, and ProjFL+EF, tailored for biased compressors through an Error Feedback mechanism. Both methods rely on projecting local gradients onto a shared client-server subspace spanned by historical descent directions, enabling efficient information exchange with minimal communication overhead. We establish convergence guarantees for both algorithms under strongly convex, convex, and non-convex settings. Empirical evaluations on standard FL classification benchmarks with deep neural networks show that ProjFL and ProjFL+EF achieve accuracy comparable to existing baselines while substantially reducing communication costs.
Similar Papers
Federated Learning on Riemannian Manifolds: A Gradient-Free Projection-Based Approach
Optimization and Control
Trains AI models privately without seeing your data.
Convergence Analysis of Asynchronous Federated Learning with Gradient Compression for Non-Convex Optimization
Machine Learning (CS)
Makes phones learn together faster, without sharing private data.
Gradient Compression and Correlation Driven Federated Learning for Wireless Traffic Prediction
Distributed, Parallel, and Cluster Computing
Makes phone networks send less data.