带有半渐进式分层提取机制的轻量化多任务模型  

Lightweight multi⁃tasking learning model with semi⁃progressive layered extraction mechanism

作　　者：杨程 车文刚[1] YANG Cheng;CHE Wengang(Faculty of Information Engineering and Automation,Kunming University of Science and Technology,Kunming 650500,China)

机构地区：[1]昆明理工大学信息工程与自动化学院,云南昆明650500

出　　处：《现代电子技术》2024年第3期18-24,共7页Modern Electronics Technique

摘　　要：多任务学习目前广泛被应用于各大领域,然而大部分效果较佳的模型都有着复杂的网络层级和架构,导致这些多任务学习模型很难被应用于资源有限的设备上,例如:经费有限但是人口基数大的国家或地区进行人口普查预测、便携设备的翻译等任务。为解决这一问题,提出半渐进式分层提取的轻量化多任务模型。模型首先通过对顶层任务独有的专家模块进行剪枝,将原先负责提取每个独立任务深层信息的工作交由每个任务的塔层模块进行。这一做法使得模型既能轻量化,同时也保留了将任务共享参数和任务独有参数分离及分层次提取信息的特点。为了弥补剪枝后模型性能及准确率上的下降,参考不确定性对损失加权的思想,引入动态联合损失进行优化,使得模型可以不断预测任务之间重要性对每个任务的损失进行权值调整。同时,也对部分超参数进行调优。通过模型在公共数据集UCI人口普查-收入数据集上的评估,最终证明模型有着与轻量化之前不分上下的性能。Currently,the multi⁃tasking learning model is widely used in many fields.However,most models with better effect consist of complex network layers and architectures,which makes it difficult for these multi⁃task learning models to be applied to resource⁃limited devices,for example,countries or regions with limited funds but large population bases carry out census prediction,portable devices carry out translation activities and other tasks.In view of the above,a lightweight multi⁃tasking learning model with semi⁃progressive layered extraction mechanism is proposed.In the model,the top⁃level Expert module of the specific task is pruned first and the work originally responsible for extracting the in⁃depth information of each specific task is entrusted to the Tower module of each task.This approach makes the model lightweight and,at the same time,retains the characteristics of separating the shared parameters of the task and the unique parameters of the task and extracting information hierarchically.Inspired by uncertainty to weigh losses,the dynamic joint loss is optimized to compensate for the decrease in the performance and accuracy of the model after pruning.The model can predict the importance of tasks continuously and adjust the weight of each task.Some hyperparameters are also tuned.The evaluation of the model on the UCI Census⁃Income public dataset finally proves that the model has the same performance as that before lightweight.

关 键 词：多任务学习 渐进式分层提取 轻量化 不确定性损失权重 联合损失优化 UCI 

分 类 号：TN711-34[电子电信—电路与系统] TP391.1[自动化与计算机技术—计算机应用技术]