基于水质-声音-视觉融合的循环水养殖鱼类摄食强度识别  

Identification of feeding intensity in recirculating aquaculture fish using water quality-sound-vision fusion

作　　者：胡学龙[1] 朱文韬 杨信廷[2,3,4] 王丁弘 潘良 曾昱皓 周超 HU Xuelong;ZHU Wentao;YANG Xinting;WANG Dinghong;PAN Liang;ZENG Yuhao;ZHOU Chao(School of Information Engineering,Yangzhou University,Yangzhou 225127,China;Information Technology Research Center,Beijing Academy of Agriculture and Forestry Sciences,Beijing 100097,China;National Engineering Research Center for Information Technology in Agriculture,Beijing 100097,China;National Engineering Laboratory for Agri-product Quality Traceability,Beijing 100097,China)

机构地区：[1]扬州大学信息工程学院,扬州225127 [2]北京市农林科学院信息技术研究中心,北京100097 [3]国家农业信息化工程技术研究中心,北京100097 [4]农产品质量安全追溯技术及应用国家工程研究中心,北京100097

出　　处：《农业工程学报》2023年第10期141-150,共10页Transactions of the Chinese Society of Agricultural Engineering

基　　金：国家重点研发计划项目(2022YFD2001701);北京市自然科学基金项目(6212007);北京市农林科学院青年基金项目(QNJJ202014);广东省重点领域研发计划项目(2021B0202070001)。

摘　　要：在工厂化循环水养殖中,准确识别鱼类摄食强度是实现精准投喂的前提和关键。水质、视觉、声音等单模态数据均可用于评估摄食强度,但单一模态往往具有片面性,难以完全反映全局特征,存在识别精度低、可移植性差等问题。多模态方法通过融合不同模态的特征,可为摄食强度量化提供新的手段。基于此,为融合鱼类摄食中的“水质-声音-视觉”信息,实现高精度的鱼类摄食强度量化,该研究在多模态Transformer(multimodal transformer,MulT)的基础上,提出一种多模态融合的鱼类摄食强度识别算法Fish-MulT。首先,从输入的水质、声音和视觉数据中提取特征向量;其次,利用多模态转移模块(multimodal transfer module,MMTM)对输入的特征向量进行融合,得到3种融合向量;然后对融合向量添加自适应权重并相加,得到融合模态;最后,利用融合模态将MulT算法中各模态分支的跨模态Transformer(cross-modal transformer)从2个优化为1个。试验结果表明,与MulT算法相比,该研究算法的鱼类摄食强度识别准确率由93.30%提高到95.36%,参数量减少38%。与水质、声音和视觉单模态相比,准确率分别提高68.56、21.65和3.61个百分点。可用于制定精准投喂策略,并为开发智能投喂系统提供技术支持。An accurate and rapid identification of fish feeding intensity can be essential to implement the precise feeding strategies in an industrial recirculating aquaculture system.Feeding intensity can be assessed using single modality indicators,such as water quality,vision,and sound.However,these individual data sources can often provide limited information without the complete picture of feeding behavior.The multimodal data can be expected to incorporate more comprehensive insights,compared with the single modality.Consequently,a more accurate representation of fish feeding intensity can be obtained to reduce some limitations of single modality information.Multi-modal fusion can also offer a novel approach to quantifying the feeding intensity.In this study,a Fish-MulT algorithm was proposed to combine the water quality,sound,and vision data using a Multimodal Transformer algorithm(MulT).Firstly,the feature vectors were extracted from the input water quality,sound,and visual data.Secondly,a Multimodal Transfer Module(MMTM)was employed to fuse the input feature vectors.Adaptive weights were then assigned to the three modalities after fusion.Lastly,the cross-modal transformer of each modal branch in the MulT algorithm was optimized to reduce the number of required transformers from 2 to 1.Furthermore,1293 sets of threemodality data were collected in the experiment,with 70%used as the training set,15%as the validation set,and the remaining 15%as the test set.The dataset was classified into four categories,according to the feeding intensity:"strong","medium","weak",and"none".The improved model was then evaluated using the collected dataset.Experimental results demonstrate that the Fish-MulT algorithm improved the accuracy of the fish feeding intensity identification from 93.30%to 95.36%,while reducing the number of parameters by 38%,compared with the MulT.Moreover,there were significant improvements in the accuracy,compared with the water quality,sound,and vision data,with the increases of 68.56,21.65,and 3.61 percentage

关 键 词：渔业 养殖 循环水 多模态融合 自适应 摄食强度 

分 类 号：S24[农业科学—农业电气化与自动化] S951.2[农业科学—农业工程] TP391.4[自动化与计算机技术—计算机应用技术]