基于稀疏自学习卷积神经网络的句子分类模型  

Sentence Classification Model Based on Sparse and Self-Taught Convolutional Neural Networks

作　　者：高云龙[1,2] 左万利 王英[1,2] 王鑫[2,3] 

机构地区：[1]吉林大学计算机科学与技术学院,长春130012 [2]符号计算与知识工程教育部重点实验室(吉林大学),长春130012 [3]长春工程学院计算机技术与工程学院,长春130012

出　　处：《计算机研究与发展》2018年第1期179-187,共9页Journal of Computer Research and Development

基　　金：国家自然科学基金项目(60903098;60973040;61602057);国家自然科学基金青年科学基金项目(61300148);吉林省科技厅优秀青年人才基金项目(20170520059JH);吉林省教育厅青年基金项目(2016311);吉林大学研究生创新基金项目(2016184)~~

摘　　要：句子分类模型的建立对于自然语言理解的研究有着十分重要的意义.基于卷积神经网络(convolutional neural networks,CNN)提取数据特征的特点,提出基于稀疏自学习卷积神经网络(sparse and self-taught CNN,SCNN)的句子分类模型.首先,在卷积层排除人为约定的特征map输入,自学习前一层输入的特征矩阵的有效组合,动态捕获句子范围内各个特征的有效关联;然后,在训练过程中利用L1范数增加稀疏性约束,降低模型复杂度;最后,在采样层利用K-Max Pooling选择句子中最大特征的序列,并保留特征之间的相对次序.SCNN可以处理变长的句子输入,模型的建立不依赖于句法、分析树等语言学特征,从而适用于任何一种语言.通过对语料库进行句子分类实验,验证了所提出模型有较好的分类效果.The study and establishment of sentence classification model have an important impact on the study of nature language processing and understanding.In this paper,we propose a sentence classification model named SCNN based on sparse and self-taught convolutional neural networks in extracting characteristics of the features from data in the CNN model.Firstly,in this method,the convolutional layer itself studies the effective combinations from the feature matrices of the previous layers in order to dynamically learn the relationships of data features in the scope of the sentence,eliminating the user-defined feature-map input of the convolutional layers.Secondly,during the unsupervised training process,using L1-norm to increase sparse constraints,the complexity of the proposed model can be effectively decreased,on the contrary,the accuracy of SCNN model can be effectively increased.Finally,by employing K-Max Pooling in the feature extraction layer,the maximal feature sequence can be selected,and relative orders among features can be effectively preserved.SCNN can cope with sentence with variant length,and furthermore,the model can apply to any language due to its independence to any linguistic features like syntax and parse trees.Experiments on the standard corpus dataset show that the proposed model is effective for the task of the sentence classification.

关 键 词：词CNN 稀疏 自学习 分类 L1范数 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]