基于社区发现的网络异常检测方法  

Network Anomaly Detection Method Based on Community Detection

作　　者：钱爱娟 樊昕 董笑菊[1] 褚衍杰 袁晓如 QIAN Ai-Juan;FAN Xin;DONG Xiao-Ju;CHU Yan-Jie;YUAN Xiao-Ru(Department of Computer Science and Engineering,Shanghai Jiao Tong University,Shanghai 200240;National Key Laboratory of Science and Technology on Blind Signal Processing,Chengdu 610041;Key Laboratory of Machine Perception(Ministry of Education),and School of Artificial Intelligence,Peking University,Beijing 100871;National Engineering Laboratory for Big Data Analysis and Application,Beijing 100871)

机构地区：[1]上海交通大学计算机科学与工程系,上海200240 [2]盲信号处理国家级重点实验室,成都610041 [3]北京大学机器感知与智能教育部重点实验室智能学院,北京100871 [4]大数据分析与应用技术国家工程实验室,北京100871

出　　处：《计算机学报》2022年第4期825-837,共13页Chinese Journal of Computers

基　　金：国家重点研发计划项目(No.2017YFB0701900);国家自然科学基金(No.61100053)资助.

摘　　要：随着互联网的不断普及与网络通信技术的不断进步,网络已经逐渐进入到人们生活的每一个层面,越来越多的网络应用应运而生.但是另一方面,随着当前网络结构的日益复杂,会引起各种各样的网络安全问题,对社会构成了巨大的威胁和挑战.因此,网络安全问题至关重要.其中网络异常检测得到了研究人员的普遍关注.多年来,虽然已有许多异常检测的工作可以一定程度上发现和抵御网络攻击,但是有些方法难以适用于无标签的数据集,有些方法则训练成本过高,无法应用于实时场景.此外,对于细微异常的检测也是现有方法面临的一大问题.考虑到模型可解释性对于很多场景的必要性,本文以可视分析作为基础,提出了基于社区发现的网络异常检测方法,通过一个较为合适的粒度来提高系统对于细微异常的检测能力.该方法首先使用多层常量玻茨模型(CPM)算法对移动时间窗内的网络数据检测社区,并以社区为单位提取特征向量,然后用社区匹配方法将相邻时间步的社区关联起来,通过监控各社区特征的变化情况来检测异常.这种方法既考虑了网络数据作为动态图的特性,又能从一个比较合适的粒度提取特征.此外,系统提供可视化界面来帮助用户确认异常点前后的网络情况、关联异常事件.通过在Vast Challenge 2013挑战三的NetFlow数据集上的实验证明了该方法能够有效地检测更加细微的网络异常,验证了所提方法的有效性.In the rapid development of today,network has gradually integrated into the layers of human lives,and more and more network applications have emerged.However,as the current network structure becomes more and more complex,it will inevitably lead to the occurrence of various network security risks and loopholes,posing a huge threat and challenge to society.In recent years,the number and the strength of network attacks are gradually increased,and the intrusion methods are also updated,making the network information security problem more and more serious and crucial.Therefore,the issue of network security is of utmost importance.Among them,network anomaly detection has received widespread attention from researchers.Over the years,although there have been some anomaly detection researches that can discover and defend network attacks to a certain extent,some of them are difficult to apply to unlabeled datasets,and some methods are too expensive to train and cannot be applied to real-time scenarios.Besides,the detection of subtle anomalies is also an important problem of existing methods.In recent years,the continuous development of network security automated detection technology has reduced a lot of manpower expenditure,while there are also many problems.On the one hand,the scale of training data has brought huge storage costs and time costs;on the other hand,the complex network environment and the concealment of attack methods have led to many false negatives.Therefore,it is necessary to involve humans in the analysis process to make more accurate judgments.In this process,how to systemize and structure the massive and complex data is a key issue,and the cross-cutting research field of network security visualization has emerged as the times require.Given the necessities of model interpretability under several scenes,based on visual analysis,this paper proposes a network anomaly detection method based on community detection,which improves the ability of to detect subtle anomalies through an appropriate level of granula

关 键 词：网络异常检测 可视分析 社区发现 社区匹配 

分 类 号：TP309[自动化与计算机技术—计算机系统结构]