Matching High-Dimensional Geometric Quantiles for Test-Time Adaptation of Transformers and Convolutional Networks Alike
By: Sravan Danda , Aditya Challa , Shlok Mehendale and more
Potential Business Impact:
Makes AI work better on new, slightly different pictures.
Test-time adaptation (TTA) refers to adapting a classifier for the test data when the probability distribution of the test data slightly differs from that of the training data of the model. To the best of our knowledge, most of the existing TTA approaches modify the weights of the classifier relying heavily on the architecture. It is unclear as to how these approaches are extendable to generic architectures. In this article, we propose an architecture-agnostic approach to TTA by adding an adapter network pre-processing the input images suitable to the classifier. This adapter is trained using the proposed quantile loss. Unlike existing approaches, we correct for the distribution shift by matching high-dimensional geometric quantiles. We prove theoretically that under suitable conditions minimizing quantile loss can learn the optimal adapter. We validate our approach on CIFAR10-C, CIFAR100-C and TinyImageNet-C by training both classic convolutional and transformer networks on CIFAR10, CIFAR100 and TinyImageNet datasets.
Similar Papers
Test-Time Model Adaptation for Quantized Neural Networks
CV and Pattern Recognition
Helps self-driving cars work better in changing weather.
Open-World Test-Time Adaptation with Hierarchical Feature Aggregation and Attention Affine
CV and Pattern Recognition
Helps AI tell real from fake, even when surprised.
Neural Collapse in Test-Time Adaptation
CV and Pattern Recognition
Fixes AI mistakes when data changes.