基于MapReduce的Flash P2P VoD系统异常监测  

Flash P2P VoD System Monitoring Based on MapReduce

作　　者：王菁菁[1] 林琛[1,2] 陈珂[3] 江弋[1] 

机构地区：[1]厦门大学信息科学与技术学院,福建厦门361005 [2]厦门大学深圳研究院,广东深圳518000 [3]广东石油化工学院计算机科学与技术系,广东茂名525000

出　　处：《厦门大学学报（自然科学版）》2013年第4期459-465,共7页Journal of Xiamen University：Natural Science

基　　金：国家自然科学基金项目(61102136;61001013);福建省自然科学基金项目(2011J05158;2010J01351);深圳市科技创新基础研究项目(JCYJ20120618155655087)

摘　　要：Peer-to-Peer(P2P)是一种充分利用分布在终端电脑上的边缘性网络资源,包括计算资源、带宽资源、内容资源等,以降低对中央服务器资源的消耗需求的技术,适用于大规模的VoD(video on demand)系统中.基于这样的背景技术,当下诞生了一种很流行的视频点播系统:Flash P2PVoD.然而使用如此广泛的系统并没有一个有效的措施对其进行监控,以不断提升其自身运行质量.基于此背景,提出一种基于MapReduce并行框架的算法模型,用于对系统的运行状态进行实时高效的监控及异常检测,提升系统的效用性,进行高效的处理分析,将消耗大量计算资源的复杂计算通过网络分布到多节点上进行计算,是当前一种行之有效的解决方案.实验结果表明,基于MapReduce的状态监控及异常检测方法能有效地从大吞吐量系统所产生的海量日志中监测出异常表现.Peer-to-Peer （P2P）is a technology that takes full use of the edge of the terminal computer network resources to reduce the consumption of the central server resources demand. These resources include computing resources, bandwidth resources and so on. This technology is suitable for the large-scale video on demand （VoD） system. There is a flash VoD system based on such technology background. However, there is not an effective method to monitor this system effectively. Based on this background, this paper propo- ses a model based on MapReduce framework. Itrs used to effectively monitor the system and anomaly detection,and then enhance the system＇s utility. In the face of massive data generated by web,it has been unable to only use a single host to process and analyze the logs. So,the use of the MapReduce programming model is an effective solution. MapReduce programming model distributes the com- plex calculations which consume a lot of computing resources to multi-node through network. Experimental results show that the a- nomaly detection method based on MapReduce can effectively monitor the abnormal performance from the massive logs generated by high-throughput systems.

关 键 词：FLASH P2P VOD 日志分析 MAPREDUCE 

分 类 号：TP18[自动化与计算机技术—控制理论与控制工程]