一种多终端视频流智能识别模型共进演化方法研究  

Research on Co-Eevolution Method of Multi-Terminal Video Stream Intelligent Recognition Models

作　　者：王乐豪 刘思聪 於志文[1,2] 于昊艺 郭斌 WANG Le-Hao;LIU Si-Cong;YU Zhi-Wen;YU Hao-Yi;GUO Bin(School of Computer Science,Northwestern Polytechnical University,Xi'an 710072;Harbin Enginering University,Harbin 150006)

机构地区：[1]西北工业大学计算机学院,西安710072 [2]哈尔滨工程大学,哈尔滨150006

出　　处：《计算机学报》2024年第5期947-970,共24页Chinese Journal of Computers

基　　金：国家自然科学基金重点项目(61960206008);国家杰出青年科学基金(62025205);国家自然科学基金面上项目(62102317)资助。

摘　　要：在泛在的智能物联网终端部署深度模型并提供智能应用/服务受到越来越多关注.但是,受限于终端硬件资源,研究人员从模型轻量化技术入手,为深度模型的轻量化、高精度部署提供技术支撑.然而基于轻量化深度模型的视频应用会面临实际场景中的数据漂移问题,导致推理精度急剧下降,并且该问题在移动场景中尤为显著.边缘辅助的模型在线演化是解决数据漂移问题的一种有效方式,可实现自演化的可成长的智能计算系统.然而,模型演化速度会影响终端模型高精度服务时间占比,从而影响模型全生命周期推理性能.为了提升多终端协同的模型演化精度和速度,本文提出基于软硬一体理念的多终端视频流智能识别模型共进演化方法和系统.一方面,本文提出了新颖的多终端互学习共进演化方法,借助终端新场景数据,克服模型数据异构挑战,实现多终端模型和全局模型的高增益协同演化和共进学习;另一方面,结合互学习算法特点,提出基于存内计算的训练加速方法,利用自适应数据压缩和模型训练优化提升系统性能,在保证演化精度增益的同时加速多个终端模型的演化速度.最后,通过不同真实移动场景下的轻量化模型持续演化任务实验验证,并对比六种基准方法证明NestEvo可以有效减少51.98%演化延迟,并提升42.6%终端轻量化模型平均推理精度.Developing Artificial Intelligence of Things(AloT)technology and boostingthe construction of a ubiquitous computing digital infrastructure system are important directions.In order to overcome the privacy issues brought by cloud computing and meet the needs of low-laten-cy applications,deploying deep models on ubiquitous intelligent IoT terminals to provide intelli-gent applications/services has attracted more and more attention.But the terminal deployment of the deep model has many challenges.Limited by the available resources of the terminal hardware plat-form,researchers start with model compression technology and hardware accelerators to provide technical support for the lightweight and high-quality deployment of deep models.However,the video application based on the deep model will inevitably face the problem of data drift in the actu-al mobile scenes.Moreover,this problem is especially noticeable in mobile scenes and devices because of more severe distribution fluctuations and sparser network structures.Under the influ-ence of data drift,the accuracy of the deep model will decrease significantly,making it difficult to meet the performance requirements.Edge-assisted model online evolution is an effective way to solve the problem of data drift,which can realize an intelligent computing system that can evolve and grow.Previous model evolution systems only focus on improving the accuracy of the terminal model.But in multi-terminal system,the global model is also affected by data drift due to the ore complex and varied scenario data from different terminals,resulting in a decrease in accuracy gain in the system.In order to provide stable and reliable knowledge transfer to the terminal models,it is necessary to use federated learning to evolve the global model.However,traditional federated learning will face multiple challenges of terminal model heterogeneity,and data distribution het-erogeneity in multi-model evolution systems.What's more,the speed of online evolution will af-fect the time proportion of high-accu

关 键 词：数据漂移 模型共进演化 互学习 训练加速方案 存内计算 智能物联网 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]