Calibrating Geophysical Predictions under Constrained Probabilistic Distributions
By: Zhewen Hou , Jiajin Sun , Subashree Venkatasubramanian and more
Potential Business Impact:
Improves weather forecasts by learning from past patterns.
Machine learning (ML) has shown significant promise in studying complex geophysical dynamical systems, including turbulence and climate processes. Such systems often display sensitive dependence on initial conditions, reflected in positive Lyapunov exponents, where even small perturbations in short-term forecasts can lead to large deviations in long-term outcomes. Thus, meaningful inference requires not only accurate short-term predictions, but also consistency with the system's long-term attractor that is captured by the marginal distribution of state variables. Existing approaches attempt to address this challenge by incorporating spatial and temporal dependence, but these strategies become impractical when data are extremely sparse. In this work, we show that prior knowledge of marginal distributions offers valuable complementary information to short-term observations, motivating a distribution-informed learning framework. We introduce a calibration algorithm based on normalization and the Kernelized Stein Discrepancy (KSD) to enhance ML predictions. The method here employs KSD within a reproducing kernel Hilbert space to calibrate model outputs, improving their fidelity to known physical distributions. This not only sharpens pointwise predictions but also enforces consistency with non-local statistical structures rooted in physical principles. Through synthetic experiments-spanning offline climatological CO2 fluxes and online quasi-geostrophic flow simulations-we demonstrate the robustness and broad utility of the proposed framework.
Similar Papers
Crowdsourcing the Frontier: Advancing Hybrid Physics-ML Climate Simulation via $50,000 Kaggle Competition
Atmospheric and Oceanic Physics
AI helps climate models predict weather more accurately.
Monitoring the calibration of probability forecasts with an application to concept drift detection involving image classification
Machine Learning (Stat)
Keeps computer vision accurate over time.
The Ensemble Kalman Inversion Race
Data Analysis, Statistics and Probability
Finds best settings for climate prediction models.