Automated Radiology Report Generation Based on Topic-Keyword Semantic Guidance
By: Jing Xiao , Hongfei Liu , Ruiqi Dong and more
Potential Business Impact:
Helps doctors write X-ray reports faster.
Automated radiology report generation is essential in clinical practice. However, diagnosing radiological images typically requires physicians 5-10 minutes, resulting in a waste of valuable healthcare resources. Existing studies have not fully leveraged knowledge from historical radiology reports, lacking sufficient and accurate prior information. To address this, we propose a Topic-Keyword Semantic Guidance (TKSG) framework. This framework uses BiomedCLIP to accurately retrieve historical similar cases. Supported by multimodal, TKSG accurately detects topic words (disease classifications) and keywords (common symptoms) in diagnoses. The probabilities of topic terms are aggregated into a topic vector, serving as global information to guide the entire decoding process. Additionally, a semantic-guided attention module is designed to refine local decoding with keyword content, ensuring report accuracy and relevance. Experimental results show that our model achieves excellent performance on both IU X-Ray and MIMIC-CXR datasets. The code is available at https://github.com/SCNU203/TKSG.
Similar Papers
R2GenKG: Hierarchical Multi-modal Knowledge Graph for LLM-based Radiology Report Generation
CV and Pattern Recognition
Helps computers write better X-ray reports.
Ontology-Based Concept Distillation for Radiology Report Retrieval and Labeling
Machine Learning (CS)
Helps doctors find rare diseases in X-rays.
Any-to-Any Vision-Language Model for Multimodal X-ray Imaging and Radiological Report Generation
CV and Pattern Recognition
Creates fake X-rays and reports for doctors.