基于超图神经网络的双模态特征融合藏药材植株识别算法  

Bimodal feature fusion algorithm of Tibetan medicinal materials recognition based on hypergraph neural network

作　　者：陈健福 孙燕[1] 裴九场 CHEN Jianfu;SUN Yan;PEI Jiuchang(Qinghai Minzu University,Xining 810007,China;Qinghai Tibetan Hospital,Xining 810003,China)

机构地区：[1]青海民族大学,青海西宁810007 [2]青海藏医院,青海西宁810003

出　　处：《中草药》2025年第4期1310-1317,共8页Chinese Traditional and Herbal Drugs

基　　金：青海省海南州可持续发展议程创新示范区科技创新平台项目(2024-HN-P04);中国高校产学研创新基金异构智能计算专项(2024HY004)。

摘　　要：目的准确识别藏药材,实现藏药材智能化挖掘及管理。方法提出基于超图的双模态特征融合藏药材植株识别算法HerbiFusionNet模型。首先,利用改进的ResNet152-CA模型提取藏药材图像的空间特征,将基于Transformer架构的BERT模型提取藏药材文本的语义特征,实现2种模态特征的互补与融合;其次,计算融合后特征向量的相似性,构建超图网络;最后,通过超图神经网络捕获藏药材植株复杂关联关系,获得藏药材准确的分类。结果相比于单一模态ResNet-152-CA模型,引入融合双模态特征并基于超图神经网络的HerbiFusionNe模型,藏药材识别准确率为96.28%,其准确率增加了4.40%。提出的HerbiFusionNet模型实证了融合图像和文本的双模态特征利用超图结构挖掘藏药材数据内复杂关系的有效性。结论HerbiFusionNet模型提升了藏药材识别的准确率,能有效捕捉藏药材图像与文本之间的高阶关系,展现了超图神经网络在处理藏药植株复杂数据结构中的优势,为后续深入挖掘“症状-方剂-药材”关系及安全使用奠定了标准化基础,推动了藏药研究和应用的发展。Objective To identify Tibetan medicinal materials accurately and realize the intelligent mining and management of Tibetan medicinal materials.Methods HerbiFusionNet,a hypergraph-based dual-modal feature fusion model for Tibetan medicinal plants recognition,was proposed.Firstly,the improved ResNet152-CA model was used to extract the spatial features of Tibetan medicinal materials images,and the Transformer architecture based BERT model was used to extract the semantic features of Tibetan medicinal materials texts to realize the complementarity and fusion of the two modal features.Then,the similarity of the fused feature vectors was calculated to construct a hypergraph network.Finally,the complex association relationship of Tibetan medicinal plants was captured by hypergraph neural network,and the accurate classification of Tibetan medicinal materials was obtained.Results The experimental results show that,compared with the single-modal ResNet-152-CA model,the HerbiFusionNe model based on hypergraph neural network introduced by fusing dual-modal features has an accuracy rate of 96.28%,which is increased by 4.40%.The HerbiFusionNet model proposed in this study demonstrates the effectiveness of using hypergraph structure to mine complex relationships in Tibetan medicinal materials data by fusing dual-modal features of image and text.Conclusion The HerbiFusionNet model improves the accuracy of Tibetan medicinal materials recognition,can effectively capture the high-order relationship between Tibetan medicinal materials images and texts,and shows the advantages of hypergraph neural network in dealing with the complex data structure of Tibetan medicinal plants.It lays a standardized foundation for further exploration of the“symptoms-prescription-medicinal materials”relationship and safe use,and promotes the development of Tibetan medicine research and application.

关 键 词：藏药材 超图 双模态特征融合 卷积神经网络 HerbiFusionNet模型 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程] R282.5[自动化与计算机技术—控制科学与工程]