基于神经切线核的学件RKME规约  

Learnware Reduced Kernel Mean Embedding Specification Based on Neural Tangent Kernel

作　　者：谭志豪 史浩宇 陈梓轩 姜远[1] TAN Zhi-Hao;SHI Hao-Yu;CHEN Zi-Xuan;JIANG Yuan(State Key Laboratory for Novel Software Technology,Nanjing University,Nanjing 210023)

机构地区：[1]南京大学计算机软件新技术国家重点实验室,南京210023

出　　处：《计算机学报》2024年第6期1232-1243,共12页Chinese Journal of Computers

基　　金：国家自然科学基金(62250069,62176117);江苏省研究生科研与实践创新计划项目(KYCX23_0159)资助.

摘　　要：当前机器学习技术已经在大量领域得到广泛应用,然而仍面临许多亟待解决的问题:依赖大量的训练数据和训练技巧、难以适应环境变化、数据隐私/所有权的保护、灾难性遗忘等等.最近,学件范式使得上述问题同时得到系统性地解决成为可能.在该范式下,用户面临新的机器学习任务时可以通过学件基座系统方便地复用他人的结果,而不必从头开始.学件范式的核心在于规约,规约使得学件基座系统在不接触原始数据的情况下,可以根据用户的需求快速识别出对用户任务有帮助的学件.近期研究均通过缩略核均值嵌入(Reduced Kernel Mean Embedding,RKME)为模型构造规约,并通过构建学件原型系统验证了范式的有效性.在实际中,学件基座系统中往往包含在各种领域任务、数据类型上构建的机器学习模型,而传统的RKME规约面临维度灾难的问题,难以适用于高维数据,例如图像场景.为了拓展RKME规约的适用范围,本文引入神经切线核进行RKME规约构造.为提升方法的高效性,本文进一步通过神经网络高斯过程与随机特征近似,快速为各种模型生成RKME规约.最后,本文在真实数据构建的销量预测、图像分类场景的学件基座系统中进行大量实验验证了所提出方法的有效性和高效性,所提出方法相比于传统RKME规约查搜准确率显著提升近9%,且实验结果表明改进后的规约在图像任务上具有良好的隐私保护性质.代码见:.Machine learning technology has been successfully applied in various fields.Nevertheless,several challenges still need to be addressed.In classic machine learning paradigm,developing a high-quality model for a new task from scratch requires a substantial amount of labeled data,expertise,and computational resources,making the process difficult and costly.Moreover,although source data is crucial for transferring and reusing existing efforts,concerns over data privacy and proprietary generally hinder the sharing of experience among developers.Recently,the learnware paradigm has been developed to systematically tackle these challenges.This paradigm enables users to utilize the learnware dock system and leverage numerous existing high-performing models when faced with new machine learning tasks,instead of building machine learning models from scratch.For high-performing models of any structure from various tasks,a learnware consists of the model itself and a specification which captures the model's specialty,like its statistical properties.In this paradigm,developers worldwide can submit their well-trained models spontaneously into a learnware dock system(formerly known as learnware market).The system uniformly generates a specification for each model to form a learnware and accommodates it.The core of this paradigm lies in the specification,which enables the learnware dock system to identify and assemble existing helpful learnwares to solve new machine learning tasks according to the user requirement.Note that the learnware dock system should be able to preserve the raw data of model developers and users.Recent studies have demonstrated the efficacy of the learnware paradigm with reduced kernel mean embedding(RKME)specification,which makes a good approximation for the distri-bution of training data used by the model without revealing the raw data.In practice,the learnware dock system comprises machine learning models from different domains and various data types,whereas the traditional RKME specification faces curse

关 键 词：学件 学件基座系统 规约 神经切线核 缩略核均值嵌入 

分 类 号：TP181[自动化与计算机技术—控制理论与控制工程]