Adaptive Spatial Goodness Encoding: Advancing and Scaling Forward-Forward Learning Without Backpropagation
By: Qingchun Gong, Robert Bogdan Staszewski, Kai Xu
Potential Business Impact:
Teaches computers to learn without errors.
The Forward-Forward (FF) algorithm offers a promising alternative to backpropagation (BP). Despite advancements in recent FF-based extensions, which have enhanced the original algorithm and adapted it to convolutional neural networks (CNNs), they often suffer from limited representational capacity and poor scalability to large-scale datasets, primarily due to exploding channel dimensionality. In this work, we propose adaptive spatial goodness encoding (ASGE), a new FF-based training framework tailored for CNNs. ASGE leverages feature maps to compute spatially-aware goodness representations at each layer, enabling layer-wise supervision. Crucially, this approach decouples classification complexity from channel dimensionality, thereby addressing the issue of channel explosion and achieving competitive performance compared to other BP-free methods. ASGE outperforms all other FF-based approaches across multiple benchmarks, delivering test accuracies of 99.65% on MNIST, 93.41% on FashionMNIST, 90.62% on CIFAR-10, and 65.42% on CIFAR-100. Moreover, we present the first successful application of FF-based training to ImageNet, with Top-1 and Top-5 accuracies of 26.21% and 47.49%. By entirely eliminating BP and significantly narrowing the performance gap with BP-trained models, the ASGE framework establishes a viable foundation toward scalable BP-free CNN training.
Similar Papers
Forward-Forward Autoencoder Architectures for Energy-Efficient Wireless Communications
Information Theory
Makes computer learning faster and use less power.
Leveraging generative adversarial networks with spatially adaptive denormalization for multivariate stochastic seismic data inversion
Geophysics
Maps underground rock types and properties from seismic data.
Scalable Forward-Forward Algorithm
Machine Learning (CS)
Trains computer brains without needing to go backward.