面向小样本股骨骨折分型的多视角注意力融合方法  

Multi-view attention fusion method for few-shot femoral fracture classification

作　　者：张亚东[1] 汪玲[1] 兰海[2] 翟禹樵 程洪[1] Zhang Yadong;Wang Ling;Lan Hai;Zhai Yuqiao;Cheng Hong(University of Electronic Science and Technology of China,Chengdu 611731,China;Clinical Medical College and Affiliated Hospital of Chengdu University,Chengdu 610081,China)

机构地区：[1]电子科技大学,成都611731 [2]成都大学附属医院,成都610081

出　　处：《中国图象图形学报》2022年第3期784-796,共13页Journal of Image and Graphics

基　　金：国家自然科学基金项目(61971106)。

摘　　要：目的股骨粗隆间骨折是老年人最常见的骨折,不同类型的骨折需要不同的治疗方法。计算机图像识别技术可以辅助医生提高诊断准确率。传统的图像特征提取和机器学习方法,无法实现细粒度、高精度的分类,且少见针对3维图像的骨折分型方法。基于深度学习方法,通常需要大量的样本参与训练才能得出较好的分型性能。针对上述问题,本文提出一种面向小样本、多分类的骨折分型方法。方法将原始CT(computed tomography)分层扫描图像进行3维重建,获取不同视角下的2维图像信息,利用添加注意力机制的多视角深度学习网络融合组合特征,并联合旋转网络获得视角不变特征,最终得到预期分型结果。结果针对自建训练数据集(5类,每类23个样本),实验在4种3维深度学习网络模型上进行比较。基于注意力机制的多视角融合深度学习方法比传统深度学习模型的准确率提高了25%;基于旋转网络的方法比多视角深度学习方法提高8%。通过对比实验表明,提出的多视角融合深度学习方法大大优于传统基于体素的方法,并且也有利于使网络快速收敛。结论在骨折分型中,本文提出的添加注意力机制的多视角融合分型方法优于传统基于体素的深度学习方法,具有更高的准确率和更好的性能。Objective Femoral intertrochanteric fracture is the most common fracture in the elderly.Each type of fracture requires a specific treatment method.Computer imaging techniques,such as X-ray and computerized tomography(CT),are used to help doctors in clinical diagnosis.Considering the complex fracture types and the large number of patients,missed diagnosis or misdiagnosis is incurred.In recent years,the development of computer image recognition technology has helped doctors improve the diagnostic accuracy.Femoral fractures have two types,namely,Arbeitsgemeinschaftfür Osteosynthesefragen(AO)/Orthopaedic Trauma Association(OTA)and six-types.The classification methods can be divided into traditional machine learning methods and deep learning methods.In traditional machine learning methods,man-made features are used for learning to make classification.However,these methods usually cannot achieve fine-grained and high-precision classification,and only a few fracture classification methods can be used for three-dimensional images.The deep learning method usually needs a large number of samples to participate in training to obtain good performance.To solve the above problems,this paper proposes a fracture classification method for small samples and multiple classification.Method An attention-based multi-view fusion network is proposed,in which a data-fusion strategy is used to improve the feature-fusion performance.Firstly,the original CT layered scanning images are reconstructed to three-dimension,and then two-dimensional images are obtained from different viewpoints.Secondly,a multi-view depth learning network with attention mechanism is used to fuse the different features with different viewpoints.Max-pooling,fully connective layer(FC)and rectified linear unit(ReL U)layers are used for learning the weights of different viewpoints.These layers are used to learn the view attention.The max-pooling operator down-sample the H×W×M original samples’tensor to 1×1×M,which is then down-sampled to 1×1×M/r by the FC layer

关 键 词：骨折分型 3维重建 多视角采样 多视角融合 注意力机制 

分 类 号：TP3[自动化与计算机技术—计算机科学与技术] R68[医药卫生—骨科学]