一种深度置信提升网络集成分类模型  

An Integrated Classification Model Based on Deep Belief Network and Gradient Promotion Strategy

作　　者：宋晓明 曲文龙 阚明阳 汪慎文 SONG Xiaoming;QU Wenlong;KAN Mingyang;WANG Shenwen(School of Information Engineering,Hebei GEO University,Shijiazhuang 050031,China;Machine Learning and Artificial Intelligence Lab,Hebei GEO University,Shijiazhuang 050031,China)

机构地区：[1]河北地质大学信息工程学院,石家庄050031 [2]河北地质大学机器学习与人工智能实验室,石家庄050031

出　　处：《重庆理工大学学报（自然科学）》2021年第1期168-179,共12页Journal of Chongqing University of Technology：Natural Science

基　　金：国家自然科学基金资助项目(61402481);河北省重点研发计划项目(18212005);河北省教育厅自然科学基金重点项目(ZD2018083);河北青年拔尖人才支持计划项目(冀字[2013]17号)。

摘　　要：针对前大数据环境下浅层机器学习模型提取特征能力不足、分类能力有限等问题,提出了一种深度置信提升网络集成分类模型。该模型采用集成深度置信网络(deep belief networks,DBN)进行特征提取,克服了单一模型特征提取能力不足的问题;采用极端梯度提升决策树(eXtreme gradient boosting,XGBoost)和梯度提升决策树(gradient boost decision tree,GBDT)构建深层提升网络,对提取特征进行多层特征深度学习,采用相对多数投票法输出分类结果。基于4个UCI公开数据集和MNIST数据集对提出模型进行了检测实验,实验结果表明:提出模型较深度置信网络、极限梯度提升树和DBN-XGBDT等3种模型的准确率分别提升了13.8%、6.98%和4.18%。Shallow learning model has limited ability to express complex functions,so its generalization ability is restricted.Aiming at this problem,a regression prediction model based on depth gradient boosting is proposed by combining deep learning and ensemble learning.This model uses the integrated Deep Belief Networks(DBN)for feature extraction,which overcomes the lack of feature extraction capability of a single model.The deep network based on extreme gradient boosting decision tree(XGBoost)and gradient boosting decision tree(GBDT)is used to learn multi-level feature deep learning of extracted features,and the relative majority voting method is used to output the classification results.Based on four UCI open data sets and MNIST data sets,the test results show that the accuracy of the proposed model is 13.8%,6.98%and 4.18%higher than that of the DBN,the XGBoost and DBN-XGBDT.

关 键 词：深度置信网络 级联结构 集成学习 分类 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]