Using Large Language Models To Translate Machine Results To Human Results
By: Trishna Niraula, Jonathan Stubblefield
Potential Business Impact:
AI writes doctor reports from X-ray pictures.
Artificial intelligence (AI) has transformed medical imaging, with computer vision (CV) systems achieving state-of-the-art performance in classification and detection tasks. However, these systems typically output structured predictions, leaving radiologists responsible for translating results into full narrative reports. Recent advances in large language models (LLMs), such as GPT-4, offer new opportunities to bridge this gap by generating diagnostic narratives from structured findings. This study introduces a pipeline that integrates YOLOv5 and YOLOv8 for anomaly detection in chest X-ray images with a large language model (LLM) to generate natural-language radiology reports. The YOLO models produce bounding-box predictions and class labels, which are then passed to the LLM to generate descriptive findings and clinical summaries. YOLOv5 and YOLOv8 are compared in terms of detection accuracy, inference latency, and the quality of generated text, as measured by cosine similarity to ground-truth reports. Results show strong semantic similarity between AI and human reports, while human evaluation reveals GPT-4 excels in clarity (4.88/5) but exhibits lower scores for natural writing flow (2.81/5), indicating that current systems achieve clinical accuracy but remain stylistically distinguishable from radiologist-authored text.
Similar Papers
Identifying Imaging Follow-Up in Radiology Reports: A Comparative Analysis of Traditional ML and LLM Approaches
Computation and Language
Helps doctors know if patients need more scans.
Towards Explainable Conversational AI for Early Diagnosis with Large Language Models
Artificial Intelligence
Helps doctors diagnose illnesses by talking to patients.
More performant and scalable: Rethinking contrastive vision-language pre-training of radiology in the LLM era
CV and Pattern Recognition
AI reads X-rays and reports for better medical AI.