基于点互信息的全局词向量模型  

A global word vector model based on pointwise mutual information

作　　者：李万理 唐婧尧 薛云[1,2] 胡晓晖 张涛[3] LI Wan-li;TANG Jing-yao;XUE Yun;HU Xiao-hui;ZHANG Tao(School of Physics and Telecommunication Engineering, South China Normal University, Guangzhou 510006, Guangdong, China;Guangdong Provincial Engineering Technology Research Center for Data Science, Guangzhou 510006, Guangdong, China;Guangdong CON-COM Technology CO., LTD, Guangzhou 510640, Guangdong, China)

机构地区：[1]华南师范大学物理与电信工程学院,广东广州510006 [2]广东省数据科学工程技术研究中心,广东广州510006 [3]广东中建普联科技股份有限公司,广东广州510640

出　　处：《山东大学学报（理学版）》2019年第7期100-105,共6页Journal of Shandong University(Natural Science)

基　　金：全国统计科学研究资助项目(2016LY98);广东省科技计划资助项目(2016A010101020,2016A010101021,2016A010101022);深圳市科创委基础研究资助项目(JCYJ20160527172144272);广东省数据科学工程技术研究中心课题(2016KF09,2016KFl0);广东科学技术职业学院科研项目(XJSC2016206);广州市科技计划资助项目(201802010033)

摘　　要：提出了一种基于点互信息的全局词向量训练模型。该模型为了避免GloVe词向量模型中使用条件概率刻画词语关系时所产生的缺点,使用了另一种相关信息--联合概率与边际概率乘积的比值--来刻画词语间的关系。为了验证模型的有效性,在相同条件下,利用GloVe模型和我们的模型训练词向量,然后使用这2种词向量分别进行了word analogy以及similarity的实验。实验表明,模型的准确率在word analogy的Semantic问题中比GloVe模型表现更好,分别在100维、200维、300维的词向量实验中,准确率提升了10.50%、4.43%、1.02%,而在similarity的实验中,模型准确率提升也达5%~6%。结果表明,模型可以更有效地捕捉词语的语义。A global word vector training model based on pointwise mutual information was presented. The model used another correlation information, the ratio of the joint probability and the product of the marginal probability, to depict the relationship between words and avoid the shortcoming of conditional probability. In order to verify the validity of our model, we trained word embedding by GloVe and our model in the same situation and then carried out experiments with word analogy and word similarity separately using these two word embeddings. Experiments showed our model has achieved 10.50%, 4.43%, 1.02% better accuracy rate than the GloVe model does in sematic experiments of word analogy at 100 dimensionality, 200 dimensionality, 300 dimensionality respectively. Accuracy rate has also gained 5%-6% rise in word similarity experiments. The results show that our model can capture semantics of words more effectively.

关 键 词：点互信息 词向量 GLOVE 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]