中-蒙-藏-维文多文档摘要数据集  

A dataset of Chinese-Mongolian-Tibetan-Uyghur multi-document summaries

作　　者：翁彧 邢天娇[1,2] 叶旭明 刘征 超木日力格 刘轩 WENG Yu;XING Tianjiao;YE Xuming;LIU Zheng;CHAOMU Rilige;LIU Xuan(Information Engineering and the Key Laboratory of Ethnic Language Intelligent Analysis and Security Governance of MOE,Beijing 100081,P.R.China;College of Information Engineering,Minzu University of China,Beijing 100081,P.R.China)

机构地区：[1]民族语言智能分析与安全治理教育部重点实验室,北京100081 [2]中央民族大学信息工程学院,北京100081

出　　处：《中国科学数据（中英文网络版）》2024年第4期85-96,共12页China Scientific Data

基　　金：国家重点研发计划(2020YFB140670002-03);北京市科技计划(Z231100001723002)。

摘　　要：是自然语言处理中的一个关键任务,也是理解和处理大量文档中核心内容的关键技术。尽管目前英文和中文的多文档研究已取得显著进展,低资源语言相关研究却因数据不足而相对滞后。因此,本研究在构建中文多文档摘要数据集的基础上,采取机器翻译加专家校对的方式构建了蒙文、藏文和维文对齐语料,组成了中文、蒙文、藏文和维文版本的面向多文档摘要生成的数据集MMDS。每种语言包含1044个新闻簇(6234篇新闻文章),覆盖2018至2023年间的重大新闻事件。此外,通过人工评价确保了数据集的高质量,使其更加适用于多语言的深度学习模型训练和NLP研究。本数据集的发布,对于促进低资源语言信息处理的发展具有重要的价值。Multi-document summarization is a key task in natural language processing and a crucial technology for understanding and processing the core content within a large volume of documents.Despite significant progress in multi-document research for English and Chinese,research on low-resource languages has been slower due to a lack of data.This paper addresses this gap by first constructing a Chinese multi-document summary dataset and then using machine translation combined,with expert verification,to create aligned corpora in Mongolian,Tibetan,and Uyghur.Based on this,this study produced a multilingual multi-document summary dataset(MMDS)in Chinese,Mongolian,Tibetan,and Uyghur,each containing 1,044 news clusters(6,234 news articles)covering major events from 2018 to 2023.Additionally,the quality of the dataset is ensured through human evaluation,making it well-suited for training deep learning models and conducting NLP research across multiple languages.The release of this dataset is of significant value for promoting the development of information processing in low-resource languages.

关 键 词：多文档摘要 数据集 蒙文 藏文 维文 低资源 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]