Web渗透测试的信息抓取策略研究  

Research on Information Fetching Strategy of Web Penetration Test

作　　者：宋雅楠 刘萍[1] 

机构地区：[1]陆军军官学院,合肥230031

出　　处：《计算机系统应用》2017年第8期232-237,共6页Computer Systems & Applications

基　　金：合肥市基本建设项目(2013CGFZ1948)

摘　　要：文章研究了在渗透测试中Web站点的信息抓取的问题.针对Web渗透测试对于信息抓取的全面性、高效性的需求,本文通过对Web站点进行研究分析,提出了基于导航链接的网络爬虫策略,并通过减少迭代次数对传统的MD5去重算法进行了改进,优化了URL去重的效率.通过实验验证表明,该爬虫策略的信息抓取覆盖率和网页下载效率均有所提高.This paper investigates the problem of information crawling in Web site in penetration testing. In order to meet the requirement of high efficiency and comprehensiveness of information fetching in Web penetration test, in this paper, we researched and analyzed the Web site, and proposed a web crawler strategy based on navigation link. Besides, to optimize the efficiency of URL de-emphasis, we improved the traditional MD5 de-emphasis algorithm by reducing the number of iterations. The experimental results show that the coverage of information fetching and web page download efficiency are raised with the Strategy of Web crawler.

关 键 词：网络爬虫 Web信息抓取 URL去重 MD5算法 覆盖率 

分 类 号：TP393.0[自动化与计算机技术—计算机应用技术]