孪生支持向量回归机研究进展  

Survey on Twin Support Vector Regression

作　　者：丁世飞[1,2] 张子晨 郭丽丽 张健[1,2] 徐晓[1,2] DING Shi-fei;ZHANG Zi-chen;GUO Li-i;ZHANG Jian;XU Xiao(School of Computer Science and Technology,China University of Mining and Technology,Xuzhou,Jiangsu 221116,China;Mine Digitization Engineering Research Center of Ministry of Education(China University of Mining and Technology),Xuzhou,Jiangsu 221116,China)

机构地区：[1]中国矿业大学计算机科学与技术学院,江苏徐州221116 [2]矿山数字化教育部工程研究中心(中国矿业大学),江苏徐州221116

出　　处：《电子学报》2023年第4期1117-1134,共18页Acta Electronica Sinica

基　　金：国家自然科学基金(No.61976216,No.62276265)。

摘　　要：孪生支持向量回归机(Twin Support Vector Regression,TSVR or TWSVR)是一种基于统计学习理论的回归算法,它以结构风险最小化原理为理论基础,通过适当地选择函数子集及该子集中的判别函数,使学习机的实际风险达到最小,保证了在有限训练样本上得到的小误差分类器对独立测试集的测试误差仍然较小.孪生支持向量回归机通过将线性不可分样本映射到高维特征空间,使得映射后的样本在该高维特征空间内线性可分,保证了其具有较好的泛化性能.孪生支持向量回归机的算法思想基于孪生支持向量机(Twin Support Vector Machine,TWSVM),几何意义是使所有样本点尽可能地处于两条回归超平面的上(下)不敏感边界之间,最终的回归结果由两个超平面的回归值取平均得到.孪生支持向量回归机需求解两个规模较小的二次规划问题(Quadratic Programming Problems,QPPs)便可得到两条具有较小拟合误差的回归超平面,训练时间和拟合精度都高于传统的支持向量回归机(Support Vector Regression,SVR),且其QPPs的对偶问题存在全局最优解,避免了容易陷入局部最优的问题,故孪生支持向量回归机已成为机器学习的热门领域之一.但孪生支持向量回归机作为机器学习领域的一个较新的理论,其数学模型与算法思想都尚不成熟,在泛化性能、求解速度、矩阵稀疏性、参数选取、对偶问题等方面仍存在进一步改进的空间.本文首先给出了两种孪生支持向量回归机的数学模型与几何意义,然后将孪生支持向量回归机的几个常见的改进策略归纳如下.(1)加权孪生支持向量回归机由于孪生支持向量回归机中每个训练样本受到的惩罚是相同的,但每个样本对超平面的影响不同,尤其是噪声和离群值会使算法性能降低,并且在不同位置的训练样本应给予不同的处罚更为合理,因此考虑在孪生支持向量回归机的每个QPP中引入一个�Twin support vector regression is a regression algorithm based on statistical learning theory.It employs the theoretical principle of structural risk minimization;by appropriately selecting a subset of functions and obtaining the dis⁃criminant functions in that subset,it minimizes the actual risk of the learning machine,ensuring that the small test error of a classifier obtained on a limited training sample remains small on an independent test set.Twin support vector regression en⁃sures better generalization performance because it maps linearly inseparable samples to a high-dimensional feature space,making the mapped samples linearly separable within that high-dimensional feature space.The algorithm of twin support vector regression is based on the twin support vector machine,and the geometric meaning of twin support vector regression is to make all sample points as far as possible between the upper(lower)insensitive boundaries of the two regression hyper⁃planes.The final regression result is then obtained by averaging the regression values of the two hyperplanes.In twin sup⁃port vector regression,it is only necessary to solve two small-scale quadratic programming problems(QPPs)to obtain two regression hyperplanes with small fitting errors.Hence,its training time and fitting accuracy are higher than those of tradi⁃tional support vector regression.Moreover,the dual problems of two QPPs have a globally optimal solution,and this makes it harder for the algorithm to become trapped in local optima.Hence,twin support vector regression has become a popular field of machine learning.However,there is still room for further improvement in the generalization performance,solution speed,matrix sparsity,parameter selection,and dual problem of twin support vector regression.In this paper,the mathemati⁃cal models and geometric meanings of two types of twin support vector regression are presented;then,the following com⁃mon improvement strategies for twin support vector regression are summarized:(1)Weighted twin supp

关 键 词：孪生支持向量回归机 拟合精度 泛化能力 计算时间 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]