应用条件随机场进行汉语词法分析研究  

Research of applying conditional random fields to Chinese lexical analysis

作　　者：王继曾[1] 罗恒[1] 刘宽[1] 任浩征[1] 

机构地区：[1]兰州理工大学计算机与通信工程学院,甘肃兰州730000

出　　处：《计算机工程与设计》2007年第2期486-488,共3页Computer Engineering and Design

摘　　要：中文分词是一个困难的、重要的被广泛研究的序列数据建模问题。以往应用条件随机场进行汉语分词时,将分词转化为对汉字的标注,造成了大量的冗余的候选切分,以至于在分词过程中大大降低了分词的速度。提出了使用词图作为基础的标记序列来完成汉语的词法分析,这样充分利用了现有的词典资源,在属性框架的选择时也可以方便地融合语言知识,并且长度歧视及状态歧视方面的影响也被减到最小。提出了应用条件随机场来构建统一的汉语词法分析。Chinese word segmentation is a difficult, important and widely-studied sequence modeling problem. Precious applications applying conditional random fields to Chinese words segmentation convert segmentation to character-based begin/inside tagging caused a number of redundant segmentation candidates which makes in the segmenting process segmenting speed slower. Using the words lattice as the fundamental sequence to be tagged to achieve Chinese lexical analysis is presented. Then the lexicon is used efficiently, and language knowledge is integrated easily in feature template selecting, and influences of label and length bias are minimized. A unified approach is presented for Chinese lexical analysis using conditional random fields.

关 键 词：汉语词法分析 条件随机场 最大熵 序列数据标记 属性函数 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]