Evaluating Prompting Strategies with MedGemma for Medical Order Extraction
By: Abhinand Balachandran , Bavana Durgapraveen , Gowsikkan Sikkan Sudhagar and more
Potential Business Impact:
Helps doctors quickly get important patient notes.
The accurate extraction of medical orders from doctor-patient conversations is a critical task for reducing clinical documentation burdens and ensuring patient safety. This paper details our team submission to the MEDIQA-OE-2025 Shared Task. We investigate the performance of MedGemma, a new domain-specific open-source language model, for structured order extraction. We systematically evaluate three distinct prompting paradigms: a straightforward one-Shot approach, a reasoning-focused ReAct framework, and a multi-step agentic workflow. Our experiments reveal that while more complex frameworks like ReAct and agentic flows are powerful, the simpler one-shot prompting method achieved the highest performance on the official validation set. We posit that on manually annotated transcripts, complex reasoning chains can lead to "overthinking" and introduce noise, making a direct approach more robust and efficient. Our work provides valuable insights into selecting appropriate prompting strategies for clinical information extraction in varied data conditions.
Similar Papers
Assessing Large Language Models for Structured Medical Order Extraction
Computation and Language
Helps doctors quickly find patient orders in notes.
Overview of the MEDIQA-OE 2025 Shared Task on Medical Order Extraction from Doctor-Patient Consultations
Computation and Language
Turns doctor talks into patient care orders.
Prompting Strategies for Language Model-Based Item Generation in K-12 Education: Bridging the Gap Between Small and Large Language Models
Computation and Language
Makes computers create good tests for students.