基于条件随机场的敏感话题检测模型研究  

Research on Sensitive Topic Detection Model Based on Conditional Random Fields

作　　者：翟东海[1,2] 崔静静[1] 聂洪玉[1] 于磊[1] 杜佳[2] 

机构地区：[1]西南交通大学信息科学与技术学院,成都610031 [2]西藏大学工学院,拉萨850000

出　　处：《计算机工程》2014年第8期158-162,167,共6页Computer Engineering

基　　金：国家语委"十二五"科研规划基金资助项目(YB125-49);教育部科学技术研究基金资助重点项目(212167);中央高校基本科研业务费专项资金科技创新基金资助项目(SWJTU12CX096);国家级大学生创新创业训练计划基金资助项目(201210694017)

摘　　要：敏感话题通常包含态度倾向性,且具有一定的先验知识,如何有效利用这些先验知识来判断网络文本的敏感性是敏感话题检测的研究难点和热点。在充分利用条件随机场强大知识拟合能力的基础上,提出一种基于条件随机场的敏感话题检测模型。抽取特征词项,并结合敏感词汇库,将待检测文档和敏感话题类别分别表示为条件随机场中的观察序列和状态序列,再利用敏感话题类别中的先验知识来构造特征函数,从而使观察序列和状态序列建立联系。将待测文档中的特征项根据概率标注为敏感话题类别中的词项,在此过程中采用Viterbi算法对观察序列的可信度进行估计,并依据估计所得的概率值对待测文档中的特征项进行敏感性标注。实例验证结果表明,该算法能够得到较好的准确率、召回率和F度量值。Sensitive topics often contain tendentious attitude and some prior knowledge,and how to effectively use prior knowledge to determine sensitivities of network text is the difficulty and hot spots in sensitive topics detection.Taking full advantage of strong knowledge fitting capability of Conditional Random Fields （CRFs）,this paper proposes a sensitive topic detection model based on CRFs.By extracting feature items,in combination with the sensitive terminology,this approach represents new documents and sensitive topic categories as observation sequence and state sequence of CRFs.Feature function is constructed by using prior knowledge of sensitive topics categories,and observation sequence and state sequence are connected by them.It estimates the credibility of the observation sequence by Viterbi algorithm,so feature items in new documents is marked with items in sensitive topic categories in probability.Experimental results demonstrate that this approach achieves very good results in precision,recall rate and F-measure.

关 键 词：敏感话题检测 条件随机场 特征函数 特征词项 VITERBI算法 敏感性标注 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]