PhysioME: A Robust Multimodal Self-Supervised Framework for Physiological Signals with Missing Modalities
By: Cheol-Hui Lee , Hwa-Yeon Lee , Min-Kyung Jung and more
Potential Business Impact:
Fixes medical readings when data is missing.
Missing or corrupted modalities are common in physiological signal-based medical applications owing to hardware constraints or motion artifacts. However, most existing methods assume the availability of all modalities, resulting in substantial performance degradation in the absence of any modality. To overcome this limitation, this study proposes PhysioME, a robust framework designed to ensure reliable performance under missing modality conditions. PhysioME adopts: (1) a multimodal self-supervised learning approach that combines contrastive learning with masked prediction; (2) a Dual-PathNeuroNet backbone tailored to capture the temporal dynamics of each physiological signal modality; and (3) a restoration decoder that reconstructs missing modality tokens, enabling flexible processing of incomplete inputs. The experimental results show that PhysioME achieves high consistency and generalization performance across various missing modality scenarios. These findings highlight the potential of PhysioME as a reliable tool for supporting clinical decision-making in real-world settings with imperfect data availability.
Similar Papers
Towards Robust Multimodal Physiological Foundation Models: Handling Arbitrary Missing Modalities
Signal Processing
Helps computers understand body signals, even if some are missing.
MMME: A Spontaneous Multi-Modal Micro-Expression Dataset Enabling Visual-Physiological Fusion
CV and Pattern Recognition
Reads hidden emotions using faces and body signals.
MoE-Health: A Mixture of Experts Framework for Robust Multimodal Healthcare Prediction
Machine Learning (CS)
Helps doctors predict sickness with mixed patient data.