基于置信度的藏文人名识别的主动学习模型研究  

Confidence Based Active Learning Model for Tibetan Person Name Recognition

作　　者：王志娟[1,2] 刘飞飞 赵小兵 宋伟[1] WANG Zhijuan;LIU Feifei;ZHAO Xiaobing;SONG Wei(School of Electronics Engineering,Minzu University of China,Beijing 100081,China;National Language Resource Monitoring &Research Center ofMinority Languages,Beijing100081,China;Tomorrow Advancing Life Education Group,Beijing100080,China)

机构地区：[1]中央民族大学信息工程学院,北京100081 [2]国家语言资源监测与研究少数民族语言中心,北京100081 [3]好未来教育科技集团,北京100080

出　　处：《中文信息学报》2019年第8期53-59,共7页Journal of Chinese Information Processing

基　　金：国家自然科学基金(61331013,61501529)

摘　　要：训练语料的标注成本是资源稀缺语言处理研究面临的一个重要问题,通过主动学习(active learning)方法可以选择信息量大、无冗余的语料供人工标注,进而大大降低语料标注成本。该文基于CRF模型给出的标注置信度提出了四种主动学习方法,并通过实验确定了这四种主动学习方法的相关参数。实验显示:选择置信度低于0.7的语料进行人工标注,直到新旧模型标注结果的差异度小于0.01%时,仅需6轮迭代;人工标注3.2MB的语料,藏文人名识别的F值可以达到88%,若要达到该识别效果,基于CRF的监督式学习模型需要标注约10MB的语料,该主动学习方法降低了约66%的语料标注规模。To alleviate the issue of labeling cost of training data for low resource languages,the active learning is a promising method by selecting the informative data without redundancy.Four active learning methods based on the confidence are proposed,with the parameters decided empirically.The experimental results:selecting the data with confidence below 0.7 and 6 iteration of labeling with up to 3.2 MB training data,we can achieve 0.88 F-measure for Tibetan name recognition.Compare with the 10 MB training data for CRF model to achieve the same performance(with no more than 0.01% difference),the active learning approach reduces the annotation scale by 66%.

关 键 词：藏文人名识别 主动学习 置信度 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]