SEAHORSE: A TOOL FOR EVALUATING MULTILINGUAL SUMMARIZATION SYSTEMS

Summary from the TLDR AI newsletter

SEAHORSE is a dataset created to assess the quality of summarization systems in multiple languages. It contains 96,000 summaries that have been rated by humans based on six important aspects: clarity, repetition, grammar, attribution, main ideas, and conciseness. This dataset not only serves as a benchmark for evaluating the performance of automated metrics but also provides a valuable resource for training such metrics, helping researchers advance the field of multilingual summarization evaluation.