神经网络语言模型的结构与技术研究评述  

Review on the Structure and Technology of Neural Network Language Model

作　　者：徐昊 易绵竹 XU Hao;YI Mian-zhu(Department of Language Information Processing,Information Engineering Unversity's Luoyang Campus,Luoyang 471000)

机构地区：[1]信息工程大学洛阳校区研究生四大队,洛阳471000 [2]信息工程大学洛阳校区基础系,洛阳471000

出　　处：《现代计算机》2019年第19期18-23,共6页Modern Computer

摘　　要：语言模型是对一种语言中词序列的联合概率函数的学习,传统的统计学习方法在语言模型的训练问题上遭遇维数灾难与上下文有限问题。神经网络语言模型通过深度学习方法训练语言模型,并通过分布式词向量表示解决维数灾难问题,循环神经网络对时序问题的处理能力在语言模型问题上得到应用。介绍几种各有优势的传统语言模型,介绍循环神经网络模型对时序问题的学习,与神经网络语言模型的相关技术。并在PTB数据集上分别使用N-Gram模型和LSTM模型训练英文语言模型,比较两者的困惑度差距。The language model is a study of the joint probability function of word sequences in a language.The traditional statistical learning method encounters dimensional disasters and limited context problems in the training of language models.The neural network language model trains the language model through the deep learning method,and solves the dimensionality disaster problem through the distributed word vector representation.The processing ability of the cyclic neural network for the timing problem is applied to the language model problem.Introduces several traditional language models with advantages,introduces the learning of time series problems by the cyclic neural net.work model,and related technologies of neural network language models.The English language model is trained on the PTB dataset using the N-Gram model and the LSTM model,respectively,and the confusion gap between the two is compared.

关 键 词：语言模型 神经语言模型 循环神经网络 长短时记忆网络 词向量 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]