LLM, Reporting In! Medical Information Extraction Across Prompting, Fine-tuning and Post-correction
By: Ikram Belmadani , Parisa Nazari Hashemi , Thomas Sebbag and more
Potential Business Impact:
Helps computers find health words in French text.
This work presents our participation in the EvalLLM 2025 challenge on biomedical Named Entity Recognition (NER) and health event extraction in French (few-shot setting). For NER, we propose three approaches combining large language models (LLMs), annotation guidelines, synthetic data, and post-processing: (1) in-context learning (ICL) with GPT-4.1, incorporating automatic selection of 10 examples and a summary of the annotation guidelines into the prompt, (2) the universal NER system GLiNER, fine-tuned on a synthetic corpus and then verified by an LLM in post-processing, and (3) the open LLM LLaMA-3.1-8B-Instruct, fine-tuned on the same synthetic corpus. Event extraction uses the same ICL strategy with GPT-4.1, reusing the guideline summary in the prompt. Results show GPT-4.1 leads with a macro-F1 of 61.53% for NER and 15.02% for event extraction, highlighting the importance of well-crafted prompting to maximize performance in very low-resource scenarios.
Similar Papers
Supervised Fine-Tuning or In-Context Learning? Evaluating LLMs for Clinical NER
Computation and Language
Helps doctors find patient problems in notes.
Zero-Shot Document-Level Biomedical Relation Extraction via Scenario-based Prompt Design in Two-Stage with LLM
Neural and Evolutionary Computing
Helps computers find health facts without human work.
LLM-based Prompt Ensemble for Reliable Medical Entity Recognition from EHRs
Artificial Intelligence
Helps doctors find important patient info faster.