The Mediomatix Corpus: Parallel Data for Romansh Idioms via Comparable Schoolbooks
By: Zachary Hopton , Jannis Vamvas , Andrin Büchler and more
Potential Business Impact:
Helps computers translate between different Romansh languages.
The five idioms (i.e., varieties) of the Romansh language are largely standardized and are taught in the schools of the respective communities in Switzerland. In this paper, we present the first parallel corpus of Romansh idioms. The corpus is based on 291 schoolbook volumes, which are comparable in content for the five idioms. We use automatic alignment methods to extract 207k multi-parallel segments from the books, with more than 2M tokens in total. A small-scale human evaluation confirms that the segments are highly parallel, making the dataset suitable for NLP applications such as machine translation between Romansh idioms. We release the parallel and unaligned versions of the dataset under a CC-BY-NC-SA license and demonstrate its utility for machine translation by training and evaluating an LLM on a sample of the dataset.
Similar Papers
Loci Similes: A Benchmark for Extracting Intertextualities in Latin Literature
Information Retrieval
Finds old book ideas hidden in new writing.
NLP Datasets for Idiom and Figurative Language Tasks
Computation and Language
Helps computers understand jokes and slang.
A Survey of Idiom Datasets for Psycholinguistic and Computational Research
Computation and Language
Helps computers understand tricky sayings and wordplay.