变尺度特征融合与交叉训练的医学报告生成方法  

Medical Report Generation Method Based on Multi-Scale Feature Fusion and Cross-Training

作　　者：韩琪 张淑军[1] 谭立玮 李劲松 Han Qi;Zhang Shujun;Tan Liwei;and Li Jinsong(College of Information Science and Technology,Qingdao University of Science and Technology,Qingdao 266100)

机构地区：[1]青岛科技大学信息科学技术学院,青岛266100

出　　处：《计算机辅助设计与图形学学报》2024年第5期795-804,共10页Journal of Computer-Aided Design & Computer Graphics

基　　金：山东省高等学校青创人才引育计划“人工智能与医学影像分析创新团队”建设项目.

摘　　要：在对医学影像自动生成文本报告的过程中,针对病灶尺寸小、形状不规则、训练数据量少等因素易导致影像报告出现误诊、漏诊的问题,提出变尺度特征融合与交叉训练的医学报告生成方法.首先将条件全局池化后的粗粒度特征与随机丢弃后的细粒度特征相融合,增强模型对不同尺度病灶的感知能力;然后通过整体数据和局部细节双路交叉训练的策略间接丰富数据集,增强模型的鲁棒性,并在双路中分别使用通道分离思想进一步挖掘影像的通道信息;最后通过多头注意力编解码网络,得到准确的医学报告.在IU-X-Ray和MIMIC-CXR数据集上与其他多种方法进行实验的结果表明,METEOR与BLEU-2分数分别提升5.70%和3.13%,所提方法可以有效地提升生成报告的可读性与准确性.In the process of automatic report generation for medical images,due to the small sizes,irregular shapes of lesions,and small amount of training data,it was easy to lead to misdiagnosis and missed diagno-sis in the reports.This paper proposes a medical report generation method based on multi-scale feature fu-sion and cross-training.Firstly,the method combines the coarse-grained features after conditional global pooling with the fine-grained features after random discarding to enhance the model’s perceptive ability to different lesion scales.Secondly,a two-way cross-training strategy through overall data and local details is used to indirectly enrich the dataset and improve the robustness of the model,while adapting channel sepa-ration principle to better mine channel information separately in the two ways.Finally,an accurate image report is obtained through the multi-head attention encoding and decoding network.Compared with other methods,the scores of METEOR and BLEU-2 are improved by 5.70%and 3.13%on the IU-X-Ray and MIMIC-CXR datasets,respectively.The results show that the proposed method can effectively improve the readability and accuracy of generated reports.

关 键 词：影像报告生成 特征融合 交叉训练 条件池化 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术] R445[自动化与计算机技术—计算机科学与技术]