A Semi-shared Hierarchical Joint Model for Sequence Labeling  

作　　者：LIU Gongshen DU Wei ZHOU Jie LI Jing CHENG Jie 

机构地区：[1]School of Cyber Science and Engineering,Shanghai Jiao Tong University,Shanghai 200240,China [2]State Grid Information&Telecommunication Branch,Beijing 100761,China

出　　处：《Chinese Journal of Electronics》2023年第3期519-530,共12页电子学报（英文版）

基　　金：supported by the Joint Funds of the National Natural Science Foundation of China(U1636112).

摘　　要：Multi-task learning is an essential yet practical mechanism for improving overall performance in various machine learning fields.Owing to the linguistic hierarchy,the hierarchical joint model is a common architecture used in natural language processing.However,in the state-of-the-art hierarchical joint models,higher-level tasks only share bottom layers or latent representations with lower-level tasks thus ignoring correlations between tasks at different levels,i.e.,lower-level tasks cannot be instructed by the higher features.This paper investigates how to advance the correlations among various tasks supervised at different layers in an end-to-end hierarchical joint learning model.We propose a semi-shared hierarchical model that contains cross-layer shared modules and layer-specific modules.To fully leverage the mutual information between various tasks at different levels,we design four different dataflows of latent representations between the shared and layer-specific modules.Extensive experiments on CTB-7 and CONLL-2009 show that our semi-shared approach outperforms basic hierarchical joint models on sequence tagging while having much fewer parameters.It inspires us that the proper implementation of the cross-layer sharing mechanism and residual shortcuts is promising to improve the performance of hierarchical joint natural language processing models while reducing the model complexity.

关 键 词：Multi-task learning Neural networks Natural language processing Cross-layer sharing Lexical semantics Text tagging. 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] TP391.1[自动化与计算机技术—控制科学与工程]