InstantFT: An FPGA-Based Runtime Subsecond Fine-tuning of CNN Models
By: Keisuke Sugiura, Hiroki Matsutani
Potential Business Impact:
Makes smart devices learn new things super fast.
Training deep neural networks (DNNs) requires significantly more computation and memory than inference, making runtime adaptation of DNNs challenging on resource-limited IoT platforms. We propose InstantFT, an FPGA-based method for ultra-fast CNN fine-tuning on IoT devices, by optimizing the forward and backward computations in parameter-efficient fine-tuning (PEFT). Experiments on datasets with concept drift demonstrate that InstantFT fine-tunes a pre-trained CNN 17.4x faster than existing Low-Rank Adaptation (LoRA)-based approaches, while achieving comparable accuracy. Our FPGA-based InstantFT reduces the fine-tuning time to just 0.36s and improves energy-efficiency by 16.3x, enabling on-the-fly adaptation of CNNs to non-stationary data distributions.
Similar Papers
Peregrine: One-Shot Fine-Tuning for FHE Inference of General Deep CNNs
CV and Pattern Recognition
Lets computers see images privately and accurately.
A Resource-Driven Approach for Implementing CNNs on FPGAs Using Adaptive IPs
Hardware Architecture
Makes AI run faster on small chips.
Energy-Efficient FPGA Framework for Non-Quantized Convolutional Neural Networks
Hardware Architecture
Makes AI faster and smarter on small devices.