基于多模态数据融合的农作物病害识别方法  

Crop disease recognition method based on multi-modal data fusion

作　　者：陈维 施昌勇 马传香 CHEN Wei;SHI Changyong;MA Chuanxiang(School of Computer Science and Information Engineering,Hubei University,Wuhan Hubei 430062,China;Hubei Key Laboratory of Big Data Intelligent Analysis and Application(Hubei University),Wuhan Hubei 430062,China)

机构地区：[1]湖北大学计算机与信息工程学院,武汉430062 [2]大数据智能分析与行业应用湖北省重点实验室(湖北大学),武汉430062

出　　处：《计算机应用》2025年第3期840-848,共9页journal of Computer Applications

基　　金：国家自然科学基金资助项目(62102136);湖北省技术创新计划重点项目(2024BAB034);湖北省技术创新计划重大项目(2024BAA008)。

摘　　要：现有的基于深度学习模型的农作物病害识别方法依赖特定农作物病害图像数据集进行图像特征学习,而忽视了文本特征在辅助图像特征学习中的重要性。为了更有效地提高模型对农作物病害图像的特征提取能力及病害识别能力,提出一种基于对比语言-图像预训练和多模态数据融合的农作物病害识别方法(CDR-CLIP)。首先,构建高质量的病害识别图像-文本对数据集,利用文本信息增强农作物病害图像的特征表示;其次,利用多模态融合策略有效结合文本特征与图像特征,以加强模型对病害的判别能力;最后,针对性地设计预训练和微调策略,从而优化模型在特定农作物病害识别任务中的表现。实验结果表明,在PlantVillage和AI Challenger 2018农作物病害数据集上,CDR-CLIP的病害识别准确率分别达到99.31%和87.66%,F1值分别达到99.04%和87.56%;在PlantDoc农作物病害数据集上,CDR-CLIP的平均精度均值mAP@0.5达到51.10%,展现出CDR-CLIP强大的性能优势。Current deep learning-based methods for crop disease recognition rely on specific image datasets of crop diseases for image representation learning,and do not consider the importance of text features in assisting image feature learning.To enhance feature extraction and disease recognition capabilities of the model for crop disease images more effectively,a Crop Disease Recognition method through multi-modal data fusion based on Contrastive Language-Image Pretraining(CDR-CLIP)was proposed.Firstly,high-quality disease recognition image-text pair datasets were constructed to enhance image feature representation through textual information.Then,a multi-modal fusion strategy was applied to integrate text and image features effectively,which strengthened the model capability of distinguishing diseases.Finally,specialized pre-training and fine-tuning strategies were designed to optimize the model’s performance in specific crop disease recognition tasks.Experimental results demonstrate that CDR-CLIP achieves the disease recognition accuracies of 99.31%and 87.66%with F1 values of 99.04%and 87.56%,respectively,on PlantVillage and AI Challenger 2018 crop disease datasets.On PlantDoc dataset,CDR-CLIP achieves the mean Average Precision mAP@0.5 of 51.10%,showing the strong performance advantage of CDR-CLIP.

关 键 词：数据融合 多模态 大语言模型 农作物病害识别 对比学习 

分 类 号：TP391.4[自动化与计算机技术—计算机应用技术]