面向医学影像报告生成的门归一化编解码网络  

Gate normalized encoder-decoder network for medical image report generation

作　　者：谭立玮 张淑军[2] 韩琪 郭淇 王鸿雁 TAN Liwei;ZHANG Shujun;HAN Qi;GUO Qi;WANG Hongyan(School of Data Science,Qingdao University of Science and Technology,Qingdao 266061,China;College of Information Sci-ence and Technology,Qingdao University of Science and Technology,Qingdao 266061,China;Qingdao Cadre Health Care Ser-vice,Qingdao 266071,China)

机构地区：[1]青岛科技大学信息科学技术学院,山东青岛266061 [2]青岛科技大学数据科学学院,山东青岛266061 [3]青岛市干部保健服务中心,山东青岛266071

出　　处：《智能系统学报》2024年第2期411-419,共9页CAAI Transactions on Intelligent Systems

基　　金：山东省高等学校青创人才引育计划“人工智能与医学影像分析创新团队”建设项目。

摘　　要：医学影像报告的自动生成可以减轻医生的工作强度,减少误诊或漏诊的情况发生。由于医学影像的独特性,通常病灶比较小,与正常区域灰度差异难以分辨,导致文本生成时关键词的缺失,报告不够准确。对此提出一种面向医学影像报告生成的门归一化编解码网络,通过门控通道变换单元优化视觉特征提取,加强特征间的差异,自动筛选关键特征;提出门归一化算法,沿通道维度整合上下文信息,在浅层网络激活、深层网络抑制通道间神经元活性,过滤无效特征,使文本和视觉语义充分交互,提高报告生成质量。在2种广泛使用的基准数据集IU X-Ray和MIMIC-CXR上的试验结果表明,模型能够取得先进的性能,生成的影像报告也具有更好的视觉语义一致性。Automatic generation of medical image reports can alleviate the workload of doctors and reduce the rate of misdiagnosis or missed diagnosis.Because of the uniqueness of medical images,lesions are usually small,and the gray difference between them and normal areas is hard to differentiate,resulting in loss of keywords in text generation and in-accurate reporting.Herein,a gated normalized encoder-decoder network for medical image report generation is de-veloped,which optimizes visual feature extraction through the gated channel transformation unit,enhances the differ-ence between features,and automatically screens key features.A gate normalization algorithm is designed to combine contextual information along with the channel dimensions,activate the neurons between channels in the shallow net-work,inhibit the neuron activity in the deep network,and filter invalid features,allowing full interaction between text and visual semantics to enhance the quality of report generation.Experimental results on two widely used reference datasets,IU X-Ray and MIMIC-CXR,reveal that the model can achieve advanced performance and generate image re-ports with better visual semantic consistency.

关 键 词：医学影像处理 文本处理 特征提取 信息融合 通道编码 深度学习 报告生成器 灰度差异 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术] R445[自动化与计算机技术—计算机科学与技术]