CloudBreaker: Breaking the Cloud Covers of Sentinel-2 Images using Multi-Stage Trained Conditional Flow Matching on Sentinel-1
By: Saleh Sakib Ahmed, Sara Nowreen, M. Sohel Rahman
Potential Business Impact:
Makes cloudy satellite pictures clear for Earth study.
Cloud cover and nighttime conditions remain significant limitations in satellite-based remote sensing, often restricting the availability and usability of multi-spectral imagery. In contrast, Sentinel-1 radar images are unaffected by cloud cover and can provide consistent data regardless of weather or lighting conditions. To address the challenges of limited satellite imagery, we propose CloudBreaker, a novel framework that generates high-quality multi-spectral Sentinel-2 signals from Sentinel-1 data. This includes the reconstruction of optical (RGB) images as well as critical vegetation and water indices such as NDVI and NDWI.We employed a novel multi-stage training approach based on conditional latent flow matching and, to the best of our knowledge, are the first to integrate cosine scheduling with flow matching. CloudBreaker demonstrates strong performance, achieving a Frechet Inception Distance (FID) score of 0.7432, indicating high fidelity and realism in the generated optical imagery. The model also achieved Structural Similarity Index Measure (SSIM) of 0.6156 for NDWI and 0.6874 for NDVI, indicating a high degree of structural similarity. This establishes CloudBreaker as a promising solution for a wide range of remote sensing applications where multi-spectral data is typically unavailable or unreliable
Similar Papers
Not every day is a sunny day: Synthetic cloud injection for deep land cover segmentation robustness evaluation across data sources
CV and Pattern Recognition
Radar sees land through clouds for better maps.
SAR-to-RGB Translation with Latent Diffusion for Earth Observation
CV and Pattern Recognition
Makes cloudy satellite pictures clear for Earth study.
High-Quality Cloud-Free Optical Image Synthesis Using Multi-Temporal SAR and Contaminated Optical Data
CV and Pattern Recognition
Fixes cloudy satellite pictures for better maps.