一种基于Q-学习算法的增量分类模型  

Incremental Classification Model Based on Q-learning Algorithm

作　　者：刘凌云 钱辉 邢红杰[1] 董春茹[1] 张峰[1] LIU Ling-yun;QIAN Hui;XING Hong-jie;DONG Chun-ru;ZHANG Feng(Hebei Key Laboratory of Machine Learning and Computational Intelligence,College of Mathematics and Information Science,Hebei University,Baoding,Hebei 071002,China)

机构地区：[1]河北大学数学与信息科学学院河北省机器学习与计算智能重点实验室,河北保定071002

出　　处：《计算机科学》2020年第8期171-177,共7页Computer Science

基　　金：国家自然科学基金(61672205);河北省自然科学基金面上项目(F2018201115,F2017201020);河北省教育厅青年基金(QN2017019)。

摘　　要：大数据时代的数据信息呈现持续性、爆炸性的增长,为机器学习算法带来了大量监督样本。然而,这对信息通常不是一次性获得的,且获得的数据标记是不准确的,这对传统的分类模型提出了挑战,而增量学习是一种重要的解决方法。但在增量学习中,样本的标记顺序将严重影响分类器的性能,特别是在分类器分类能力较弱的情况下,传统的增量学习方法容易过早地将噪声数据添加到训练集上,从而影响分类器的精度。为解决这个问题,文中提出一种基于Q-学习算法的增量分类模型。该模型利用强化学习中经典的Q-学习算法来合理选择样本增量序列,削弱噪声数据的负面影响,并实现在学习过程中自主标记样本。同时,为了解决当新增未标记样本集规模较大时,Q-学习中的状态空间与动作空间增大带来的计算复杂度和存储空间呈指数增长的问题,文中进一步给出了批量增量分类模型,有效降低了模型的计算复杂度并节约了存储空间。基于Q-学习算法的增量分类模型融合了增量学习及强化学习的思想,具有分类精度高、实时性强等优点。最后,在3个UCI数据集上进行实验来验证所提模型的有效性,结果表明该模型通过选择新增训练集合的确有助于提升分类器的精度,且由不同增量序列训练得到的分类器精度也有较大差异。基于Q-学习算法的增量分类模型可以利用已有的少量监督信息进行初始训练,通过自主标记样本构造增量训练集,并通过自监督的方式提高分类器的精度。因此,基于Q-学习算法的增量分类模型可被用于解决监督信息缺乏的问题,具有一定的应用价值。The traditional classification models are insufficient to take full advantage of the sequential data with their continuous and explosive growth due to the imprecision of the data.Therefore,the incremental learning is provided to handle this problem.However,the difference sequence of the training samples may have strong impact on performance of a classifier.Especially when the classifier is undertrained,traditional incremental learning method takes the risk of utilizing the noise samples with wrong labels to train the classifier.To overcome this problem,this paper proposes an incremental classification model based on Q-learning algorithm.The model employs the classical Q-learning algorithm in reinforcement learning to select the sequence samples incrementally,which is capable of softening the negative impact of the noise data and labels samples automatically as well.To overcome the problem of computational complexity along with the increasing of state space and action space of Q-learning,an improved batch incremental classification model based on Q-learning algorithm is proposed.Compared with the traditionally trained classifiers,the proposed model combines the ideas of online incremental learning and reinforcement learning,which is able to achieve high accuracy and can be updated online.Finally,the validity of the model is verified on three UCI datasets.The experimental results show that choosing training sets incrementally is helpful to improve the performance of the classifier and the precision of the classifier trained by different incremental training sequences varies greatly as well.The proposed incremental classification model based on Q-learning algorithm can make use of the limited available dataset for supervised initial training,and then construct new-added self-supervised training set based on the Q value of each unlabeled sample to improve the accuracy of the classifier.Therefore,the incremental classification model based on Q-learning algorithm can be used to solve the problem of lack of supervisory

关 键 词：增量学习 Q-学习 在线学习 分类 强化学习 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]