多模态深度学习及其在眼科人工智能的应用展望  

Multi-modal Deep Learning and Its Applications in Ophthalmic Artificial Intelligence

作　　者：李锡荣[1,2,3] LI Xirong(Key Laboratory of DEKE,School of Information,Renmin University of China,Beijing 100872,China;AIMC Lab,School of Information,Renmin University of China,Beijing 100872,China;Vistel AI Lab,Visionary Intelligence Ltd.,Beijing 100872,China)

机构地区：[1]中国人民大学数据工程与知识工程教育部重点实验室,北京100872 [2]中国人民大学信息学院人工智能与媒体计算实验室,北京100872 [3]北京致远慧图科技有限公司人工智能实验室,北京100872

出　　处：《协和医学杂志》2021年第5期602-607,共6页Medical Journal of Peking Union Medical College Hospital

基　　金：北京市自然科学基金面上项目(4202033);北京市自然科学基金-海淀原始创新联合基金(19L2062);北京市科委医药协同创新专项课题(Z191100007719002)。

摘　　要：深度学习的强学习能力和高易用性使其成为当前主流机器学习算法和医学人工智能的核心技术。鉴于医学影像在健康筛查、疾病诊断、精准治疗、预后评估等诸多任务中的关键作用,用于医学影像结构分析与语义理解的深度学习正成为重要的交叉学科研究方向。在临床场景中,医生为了实现更精准的诊断,往往需要同时参考不同类型、不同模态的影像样本进行综合分析和判断。本文介绍面向此类场景的多模态深度学习的基本概念和工作原理,结合具体案例分析多模态深度学习在眼科领域的研究进展、应用情况及技术挑战,并对该技术的应用前景作出展望。Deep learning,for its powerful learning capability and high usability,has been a prevalent algorithm of machine learning and a core technique for artificial intelligence(AI)in medicine and healthcare.Due to the importance of medical imaging in many tasks such as health screening,disease diagnosis,precise treatment,and prognosis prediction,deep learning of structural analysis and semantic understanding for medical images is becoming an important interdisciplinary research direction.In clinical scenarios,in order to achieve a more accurate diagnosis,doctors need to simultaneously refer to multiple modalities of medical imaging for a comprehensive analysis and judgment.This article introduced the basic concepts and working principles of multimodal deep learning in such scenarios,reviewed recent research progress on applying multi-modal deep learning in both generic medical fields and ophthalmology,and discussed technical challenges and also envision potential applications of multi-modal deep learning in AI-assisted ophthalmology.

关 键 词：多模态深度学习 眼科 人工智能 辅助诊断 

分 类 号：R77[医药卫生—眼科] TP18[医药卫生—临床医学]