SemEval-2025 Task 5: LLMs4Subjects -- LLM-based Automated Subject Tagging for a National Technical Library's Open-Access Catalog
By: Jennifer D'Souza , Sameer Sadruddin , Holger Israel and more
Potential Business Impact:
Helps libraries sort science papers automatically.
We present SemEval-2025 Task 5: LLMs4Subjects, a shared task on automated subject tagging for scientific and technical records in English and German using the GND taxonomy. Participants developed LLM-based systems to recommend top-k subjects, evaluated through quantitative metrics (precision, recall, F1-score) and qualitative assessments by subject specialists. Results highlight the effectiveness of LLM ensembles, synthetic data generation, and multilingual processing, offering insights into applying LLMs for digital library classification.
Similar Papers
DNB-AI-Project at SemEval-2025 Task 5: An LLM-Ensemble Approach for Automated Subject Indexing
Computation and Language
Tags library books automatically for better searching.
Annif at the GermEval-2025 LLMs4Subjects Task: Traditional XMTC Augmented by Efficient LLMs
Computation and Language
Helps libraries find books faster using smart computers.
Annif at SemEval-2025 Task 5: Traditional XMTC augmented by LLMs
Computation and Language
Helps libraries automatically sort books by topic.