第二届中文抽象语义表示解析评测  

The Second Chinese Abstract Meaning Representation Parsing Evaluation

作　　者：李斌[1] 许智星 肖力铭 周俊生 曲维光[3] 薛念文 LI Bin;XU Zhixing;XIAO Liming;ZHOU Junsheng;QU Weiguang;XUE Nianwen(School of Chinese Language and Literature,Nanjing Normal University,Nanjing,Jiangsu 210097,China;Department of Chinese Language and Literature,Peking University,Beijing 100871,China;School of Computer,Electronics and Information,Nanjing Normal University,Nanjing,Jiangsu 210023,China;Computer Science Department,Brandcis University,Waltham,MA 02453,US)

机构地区：[1]南京师范大学文学院,江苏南京210097 [2]北京大学中国语言文学系,北京100871 [3]南京师范大学计算机与电子信息学院,江苏南京210023 [4]布兰迪斯大学计算机科学系,马萨诸塞沃尔瑟姆02453

出　　处：《中文信息学报》2023年第6期33-43,共11页Journal of Chinese Information Processing

基　　金：国家社会科学基金(18BYY127);国家自然科学基金(61772278)。

摘　　要：抽象语义表示是近年来国内外句子语义解析领域的研究热点,国际上已举办了CoNLL2019和CoNLL2020两届跨语言的评测。中文抽象语义表示评测是CoNLL2020的五大任务之一,取得了接近英语的解析效果,但是评测数据和评测指标仍有较大改进空间。为了推动中文抽象语义解析研究,该文在第二十一届中国计算语言学大会技术评测任务研讨会上组织了第二届评测,以新设计的Align-smatch指标为排名标准,采用改进的语义标注方案和标注语料库来进行评测。在基础测试集上,封闭模式的最高F1值为80.00%;盲测集上的表现则相比基础测试集下降了7个百分点左右。本次评测的最佳结果在MRP指标下比上届提高了2.66个百分点。统计发现,整体性能提升主要来源于概念之间的语义关系预测准确率的提高,而语义关系的对齐还有待提升。Meaning Representation as a novel yet efficient approach of semantic parsing has prevailed the pertinent domains.English AMR corpus expanded as there has been two international conferences of cross-lingual semantic parsing evaluation held in CoNLL 2019 and CoNLL 2020 in the past several years.Chinese Abstract Meaning Representation parsing,as one of five tasks in CoNLL 2020,performed almost as well as in English.However,the data and metric used for evaluation can be improved.We hereby hold the Second Chinese Abstract Meaning Representation Parsing Evaluation to promote the research and further construct a better corpus for Chinese Abstract Meaning Representation.With regarding rules and preparations,we adopt high-quality data as the training set and use Align-smatch as the evaluation metric for the first time.According to F-score measurement,the best team scored a 0.8 in closed test and a 0.72 in open test under Align-smatch metric,respectively.And the top result surpasses the SOTA in CoNLL 2020 by 2.66 percentage points under MRP metric.Further study shows that this great progress mainly comes from better relation prediction between concepts,leaving the challenge of utilizing the alignments of semantic relations to be improved.

关 键 词：抽象语义表示 语义解析 评测指标 中文信息处理 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]