Adapting AlignScore Mertic for Factual Consistency Evaluation of Text in Russian: A Student Abstract
By: Mikhail Zimin, Milyausha Shamsutdinova, Georgii Andriushchenko
Potential Business Impact:
Checks if Russian text is true.
Ensuring factual consistency in generated text is crucial for reliable natural language processing applications. However, there is a lack of evaluation tools for factual consistency in Russian texts, as existing tools primarily focus on English corpora. To bridge this gap, we introduce AlignRuScore, a comprehensive adaptation of the AlignScore metric for Russian. To adapt the metric, we fine-tuned a RuBERT-based alignment model with task-specific classification and regression heads on Russian and translated English datasets. Our results demonstrate that a unified alignment metric can be successfully ported to Russian, laying the groundwork for robust multilingual factual consistency evaluation. We release the translated corpora, model checkpoints, and code to support further research.
Similar Papers
AlignCheck: a Semantic Open-Domain Metric for Factual Consistency Assessment
Computation and Language
Checks if AI's answers are true or made up.
Examining the Metrics for Document-Level Claim Extraction in Czech and Slovak
Computation and Language
Checks if computer-found facts match human-found facts.
Comparison of Unsupervised Metrics for Evaluating Judicial Decision Extraction
Computation and Language
Checks legal documents automatically for accuracy.