分布式监测系统中的重复元素检测机制  

Detection of Persistent Elements in Distributed Monitoring System

作　　者：陆乐 孙玉娥 黄河[1,3] 汪润枝 曹振 Lu Le;Sun Yu'e;Huang He;Wang Runzhi;Cao Zhen(School of Computer Science and Technology,Soochow University,Suzhou,Jiangsu 215131;School of Rail Transportation,Soochow University,Suzhou,Jiangsu 215137;Suzhou Institute for Advanced Study,University of Science and Technology of China,Suzhou,Jiangsu 215123)

机构地区：[1]苏州大学计算机科学与技术学院,江苏苏州215131 [2]苏州大学轨道交通学院,江苏苏州215137 [3]中国科学技术大学苏州研究院,江苏苏州215123

出　　处：《计算机研究与发展》2020年第5期1046-1056,共11页Journal of Computer Research and Development

基　　金：国家自然科学基金面上项目(61672369,61873177,61572342)。

摘　　要：重复元素检测在分布式入侵检测、公众兴趣发掘以及交通状况估计等领域有着重要的应用.现有的检测模型存在误报漏报、通信开销大和局限性大等问题,难以满足分布式应用场景的需要.针对这些问题,以最小化通信开销为目标,设计了一个适用于分布式监测系统的重复元素检测机制.首先,机制主要通过监测器与协调器间多轮次的压缩数据传输筛去了大量无关元素,从而降低了整体通信开销;接着,借助理论推导调整参数保证每一轮筛选的必要性及效果的最优化,并结合了可扩展布隆过滤器和持续流量估计等技术,使机制在面对不同的元素分布状况时,都可以取得良好效果;最后,通过仿真实验结果验证了所设计机制的有效性.The detection of persistent elements has many important applications in the fields of detecting intrusions in a distributed system, finding common interests, measuring the traffic, etc. Most of the existing state-of-the-art studies of detecting persistent elements have some problems such as false or missing report, high communication cost and great limitation so that they can hardly satisfy the requirement of some distributed applications. To solve these problems, the paper proposes a scheme to detect persistent elements in distributed monitoring system with the goal of minimizing the total communication cost during the whole detection process. First, the scheme filters out most of the irrelevant elements to reduce the overall communication overhead through multiple rounds of compressed data transferring between all monitors and the central coordinator. Then, we ensure that each round of the filtering is necessary and can achieve the best performance by adjusting parameters of filtering according to the theoretical analysis and derivation. With the technology of extended Bloom filter and the persistent spread estimation function, the scheme can work well no matter in the balanced environment or in the unbalanced environment. Finally, we perform extensive simulations to study the performance of the proposed mechanism, and the simulation results show the effectiveness of our scheme.

关 键 词：分布式系统 重复元素 布隆过滤器 通信开销最小化 机制设计 

分 类 号：TP393[自动化与计算机技术—计算机应用技术]