基于共现的汉语词的分布表示学习与改进  

Distributed Representation Learning and Improvement of Chinese Words Based on Co-occurrence

作　　者：曹学飞[1] 牛倩[1] 王瑞波 王钰 李济洪 CAO Xue-fei;NIU Qian;WANG Rui-bo;WANG Yu;LI Ji-hong(School of Automation and Software Engineering,Shanxi University,Taiyuan 030006,China;School of Modern Educational Technology,Shanxi University,Taiyuan 030006,China)

机构地区：[1]山西大学自动化与软件学院,太原030006 [2]山西大学现代教育技术学院,太原030006

出　　处：《计算机科学》2021年第6期222-226,共5页Computer Science

基　　金：国家自然科学基金(62076156,61806115,61603228);山西省应用基础研究计划(201901D111034)。

摘　　要：词与其上下文的共现矩阵是词的分布表示学习的关键。在构造共现矩阵时,可采用不同方法来度量词与其上下文之间的关联。文中首先介绍了3种词与其上下文的关联度量方法并构造了相应的共现矩阵,使用同一个优化求解框架学习得到词的分布表示,在中文词语类比任务和语义相似性任务上的评价结果显示,GloVe方法的结果最好;然后进一步对GloVe方法进行了改进,通过引入一个超参数校正词与其上下文的共现次数,以使校正后的共现次数近似服从Zip’f分布,并给出了求解该超参数估计值的方法。基于改进后的方法学习得到的词的分布表示在词语类比任务上的准确率提高了0.67%,且在McNemar检验下是显著的;在词语相似性任务上的性能提高了5.6%。此外,将改进后的方法得到的词的分布表示应用到语义角色识别任务中,作为词特征的初始向量得到的F1值相比使用改进前的词的分布得到的F1值也提高了0.15%,且经3×2交叉验证的Bayes检验其提升也较为显著。The co-occurrence matrix of words and their contexts is the key to learning the distributed of words.Different methods can be used to measure the association between words and their contexts when constructing a co-occurrence matrix.In this paper,we firstly introduce three association measures of words and their contexts,construct corresponding co-occurrence matrices and learn the distributed representations of words under a unified optimization framework.The results on semantic similarity and word analogy show that GloVe method is the best.Then,we further introduce a hyperparameter to calibrate the co-occurrences of the words and their contexts based on the Zip’f distribution,and present a method for solving the estimated value of the hyperparameter.The obtained distributed representations of words based on the improved method indicate that the accuracy of the word analogy task has increased by 0.67%,and it is significant under the McNemar test.The correlation coefficient on the word simila-rity task has increased by 5.6%.In addition,the distributed representations of the words learned by the improved method is also applied to the semantic role identification task as the initial vector of word feature,and the F 1 value obtained is also increased by 0.15%.

关 键 词：分布表示 共现 词语类比 词语相似性 Zip’f分布 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]