LLM-Fusion: A Novel Multimodal Fusion Model for Accelerated Material Discovery
By: Onur Boyar , Indra Priyadarsini , Seiji Takeda and more
Potential Business Impact:
Finds new materials faster and better.
Discovering materials with desirable properties in an efficient way remains a significant problem in materials science. Many studies have tackled this problem by using different sets of information available about the materials. Among them, multimodal approaches have been found to be promising because of their ability to combine different sources of information. However, fusion algorithms to date remain simple, lacking a mechanism to provide a rich representation of multiple modalities. This paper presents LLM-Fusion, a novel multimodal fusion model that leverages large language models (LLMs) to integrate diverse representations, such as SMILES, SELFIES, text descriptions, and molecular fingerprints, for accurate property prediction. Our approach introduces a flexible LLM-based architecture that supports multimodal input processing and enables material property prediction with higher accuracy than traditional methods. We validate our model on two datasets across five prediction tasks and demonstrate its effectiveness compared to unimodal and naive concatenation baselines.
Similar Papers
Towards LLM-Centric Multimodal Fusion: A Survey on Integration Strategies and Techniques
Computation and Language
AI learns from pictures, sounds, and words together.
$\text{M}^{2}$LLM: Multi-view Molecular Representation Learning with Large Language Models
Machine Learning (CS)
Helps find new medicines by understanding molecules better.
MatMMFuse: Multi-Modal Fusion model for Material Property Prediction
Machine Learning (CS)
Finds new materials faster by combining two computer brains.