Multimodal, Multi-Disease Medical Imaging Foundation Model (MerMED-FM)
By: Yang Zhou , Chrystie Wan Ning Quek , Jun Zhou and more
Potential Business Impact:
Helps doctors see diseases in many kinds of scans.
Current artificial intelligence models for medical imaging are predominantly single modality and single disease. Attempts to create multimodal and multi-disease models have resulted in inconsistent clinical accuracy. Furthermore, training these models typically requires large, labour-intensive, well-labelled datasets. We developed MerMED-FM, a state-of-the-art multimodal, multi-specialty foundation model trained using self-supervised learning and a memory module. MerMED-FM was trained on 3.3 million medical images from over ten specialties and seven modalities, including computed tomography (CT), chest X-rays (CXR), ultrasound (US), pathology patches, color fundus photography (CFP), optical coherence tomography (OCT) and dermatology images. MerMED-FM was evaluated across multiple diseases and compared against existing foundational models. Strong performance was achieved across all modalities, with AUROCs of 0.988 (OCT); 0.982 (pathology); 0.951 (US); 0.943 (CT); 0.931 (skin); 0.894 (CFP); 0.858 (CXR). MerMED-FM has the potential to be a highly adaptable, versatile, cross-specialty foundation model that enables robust medical imaging interpretation across diverse medical disciplines.
Similar Papers
Vision Foundation Models for Computed Tomography
Image and Video Processing
Helps doctors find sickness in body scans.
MEDFORM: A Foundation Model for Contrastive Learning of CT Imaging and Clinical Numeric Data in Multi-Cancer Analysis
CV and Pattern Recognition
Helps doctors find cancer better with scans and notes.
Foundation Models in Medical Imaging -- A Review and Outlook
Image and Video Processing
Helps doctors see diseases in medical pictures.