基于边界增强的中文直播弹幕新词发现  

Chinese live-streaming danmaku new word detection based on boundary boosting

作　　者：王雪瑞 刘渊[1] WANG Xue-rui;LIU Yuan(School of Digital Media,Jiangnan University,Wuxi 214122,China)

机构地区：[1]江南大学数字媒体学院,江苏无锡214122

出　　处：《传感器与微系统》2018年第7期142-146,150,共6页Transducer and Microsystem Technologies

基　　金：国家科技支撑计划资助项目(2015BAH54F00);国家自然科学基金资助项目(61672264);国家重点研发计划资助项目(2016YFB0800305)

摘　　要：针对中文直播语料具有长度简短、用语不规范、字母数字混杂等特点,使得通常的新词发现方法效果不佳的问题,在现有基于词内部结合度与边界自由度的无监督新词发现算法基础上,引入了边界增强上下文熵的概念,使其适用于通常出现于句子片段边界位置的词语。在某知名直播平台的弹幕语料上进行了实验,结果表明:方法模型简洁,可有效提取新词,并能够通过调整参数适应不同规模的语料输入,且时空复杂度与执行性能良好。Aiming at problem that danmaku are short, irregular in nature and mixed with alphanumerical characters,which cause it is hard to handle with traditional new word detection methods. On the basis of previous methods based on internal cohesion and boundary freedom of words, the concept of boundary-boosted context entropy is introduced, making context entropy compatible with words mostly appearing at sentence fragment boundaries. Experimental results on a danmaku corpus collected from a well-known live-streaming service show effectiveness despite model simplicity, adaptability to differently sized input by means of parameter tuning, time and spatial complexity are also good, as is execution performance.

关 键 词：直播弹幕 新词发现 内部结合度 互信息 边界自由度 边界增强上下文熵 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]