TRACE: A Generalizable Drift Detector for Streaming Data-Driven Optimization
By: Yuan-Ting Zhong , Ting Huang , Xiaolin Xiao and more
Potential Business Impact:
Finds hidden changes in data to improve computer decisions.
Many optimization tasks involve streaming data with unknown concept drifts, posing a significant challenge as Streaming Data-Driven Optimization (SDDO). Existing methods, while leveraging surrogate model approximation and historical knowledge transfer, are often under restrictive assumptions such as fixed drift intervals and fully environmental observability, limiting their adaptability to diverse dynamic environments. We propose TRACE, a TRAnsferable C}oncept-drift Estimator that effectively detects distributional changes in streaming data with varying time scales. TRACE leverages a principled tokenization strategy to extract statistical features from data streams and models drift patterns using attention-based sequence learning, enabling accurate detection on unseen datasets and highlighting the transferability of learned drift patterns. Further, we showcase TRACE's plug-and-play nature by integrating it into a streaming optimizer, facilitating adaptive optimization under unknown drifts. Comprehensive experimental results on diverse benchmarks demonstrate the superior generalization, robustness, and effectiveness of our approach in SDDO scenarios.
Similar Papers
Binary Anomaly Detection in Streaming IoT Traffic under Concept Drift
Machine Learning (CS)
Finds weird internet traffic faster and cheaper.
Tackling Time-Series Forecasting Generalization via Mitigating Concept Drift
Machine Learning (CS)
Predicts future events better by spotting changing patterns.
Improving Real-Time Concept Drift Detection using a Hybrid Transformer-Autoencoder Framework
Machine Learning (CS)
Spots changes in computer learning early.