古汉语句子切分与句读标记方法研究  

Research on Sentence Segmentation and Punctuation in Ancient Chinese

作　　者：王川[1] 张小红[2] 韩采华[3,4] 

机构地区：[1]河南师范大学计算机与信息技术学院,河南新乡453007 [2]河南财政税务高等专科学校信息工程系,郑州450002 [3]郑州大学省信息网络重点学科开放实验室,郑州450000 [4]河南省广播电视大学,郑州450000

出　　处：《河南大学学报（自然科学版）》2009年第5期525-529,共5页Journal of Henan University:Natural Science

摘　　要：利用自然语言理解技术进行古汉语断句及句读标注的主要挑战是数据稀疏问题.为了解决这一难题,设计了一种六字位标记集,提出了一种基于层叠式条件随机场模型的古文断句与句读标记方法.基于六字位标集,低层模型用观察序列确定句子边界,高层模型同时使用观察序列和低层的句子边界信息进行句读标记.实验在5 M混合古文语料上分别进行了封闭测试和开放测试,封闭测试断句与句读标注的F值分别达到96.48%和91.35%,开放测试断句与句读标注的F值分别达到71.42%和67.67%.Data sparseness is a primary challenge in sentence segmentation and punctuation in ancient Chinese using natural language processing technology. In order to overcome this difficulty, a 6-tag set was designed and a method based on cascaded Conditional Random Fields was proposed. The main idea is as follows： based on the 6-tag set, a low level model determines the boundaries of sentences according to observation sequence and a high level model punctuates sentences taking consideration of both observation sequence and low level＇s results. Close test and open test were done based on approximate 5M mixed corpus respectively. The F measure of sentence segmentation and punctuation are 96.48% and 91.35% respectively in close test, and those are 71.42% and 67.67% respectively in open test.

关 键 词：古汉语 层叠条件随机场 数据稀疏 句子切分 句读标注 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]