基于Hadoop与RabbitMQ的人脸识别算法测试平台的设计与实现  

Design and Implementation of Test Platform for Face Recognition Algorithm Based on Hadoop and RabbitMQ

作　　者：徐宏宁 刘伟 惠君俊 周金磊 罗雄中 丁长松 XU Hong-ning;LIU wei;HUI Jun-jun;ZHOU Jin-lei;LUO Xiong-zhong;DING Chang-song(School of Informatics,Hunan University of Chinese Medicine,Changsha 410208,China;Hunan Big Data Analysis Laboratory of Chinese Medicine,Changsha 410208,China)

机构地区：[1]湖南中医药大学信息科学与工程学院,湖南长沙410208 [2]湖南省中医药大数据分析实验室,湖南长沙410208

出　　处：《测控技术》2023年第3期38-43,共6页Measurement & Control Technology

基　　金：湖南省教育厅科学研究项目(20C1435);长沙市自然科学基金项目(kq2202260)。

摘　　要：针对人脸识别算法研究过程中测试效率低下的问题,基于分布式技术,设计并实现了通用的分布式大数据测试平台。为了提高人脸识别算法的大数据测试的执行效率,提高测试结果统计计算的执行效率,基于RabbitMQ设计分布式并行执行架构,利用Hadoop集群的MapReduce框架进行分布式并行计算。利用Java语言的Spring框架开发测试平台,将测试代码与测试图片托管于Hadoop集群的HDFS文件系统,实现了测试业务与测试平台的分离,提高了平台的通用性。该测试平台不仅实现了单个测试任务的分布式执行而且满足多个测试任务同时执行,可对测试任务以及测试相关的代码与数据进行有效的管理。与传统测试方法相比,该平台测试效率提高10余倍,测试图片的数量越大测试效率提升越明显。该测试平台具有业务通用性、容量可扩展性,对于其他人工智能算法的大量数据测试具有借鉴意义与参考价值。Aiming at the problem of low test efficiency in the research of face recognition algorithm,a general distributed big data test platform is designed and implemented based on distributed technology.In order to improve the execution efficiency of big data test of face recognition algorithm,and improve the execution efficiency of statistical calculation of test results,a distributed parallel execution architecture is designed based on RabbitMQ,and the MapReduce framework of Hadoop cluster is used.The test platform is developed by using the spring framework of Java language.On this test platform,the test codes and test pictures are hosted in the HDFS file system of Hadoop cluster,which realizes the separation of test business and test platform,also improves the universality of the platform.The test platform not only realizes the distributed execution of a single test task,but also satisfies the simultaneous execution of multiple test tasks,and effectively manages the test tasks and related test codes and test data.Compared with traditional test methods,the test efficiency is improved by more than 10 times.The larger the number of test pictures,the more obvious the improvement of test efficiency.The test platform has business universality and capacity scalability.It has reference significance and reference value for big data test of other artificial intelligence algorithms.

关 键 词：人脸识别算法 测试平台 分布式 HADOOP RabbitMQ 

分 类 号：TP391[自动化与计算机技术—计算机应用技术]