基于浏览器测试组件的社交网络数据获取技术研究  

Research on Social Network Data Acquisition Technology Based on Browser Test Components

作　　者：陈学敏[1] 沙灜 

机构地区：[1]中国科学院信息工程研究所,北京100093

出　　处：《信息网络安全》2015年第5期56-61,共6页Netinfo Security

基　　金：中国科学院战略先导专项[XDA06030200];国家科技支撑计划[2012BAH46B03]

摘　　要：社交网络数据获取是社交网络分析重要的前提条件。当前各大社交网络平台对于第三方爬虫的屏蔽措施日益复杂,传统的数据获取手段受到严峻的挑战。文章提出了一种基于浏览器测试组件的社交网络数据获取技术,通过模拟正常用户的行为以规避社交网络对于传统网络爬虫的限制,实现目标数据的高效获取。该系统分别获取了QQ群即时聊天信息和非即时资料信息。对于即时聊天信息获取,经过实验测试和结果对比,发现即时信息采集的采全率达99%以上,准确率达100%。对于非即时资料,分别获取了群公告、群成员列表、群共享文件和共享相册等数据,通过数据抽样对比,采全率和准确率均达到100%。实验证明基于浏览器测试组件的社交网络数据获取技术有其一定的数据获取优势。Social network data acquisition is a vital technology and precondition for public opinion analysis. However, shielding measures of the current major social network platforms are increasingly complexity to third-party crawlers. Traditional data acquisition means are facing increasingly severe challenges. This paper proposes a data acquisition technology based on browser test components, which avoids some limitations that social networks have set up to traditional network crawlers by simulating normal users＇ behaviors, in order to achieve efficiently data acquisition. The system acquires QQ group real-time message information and non real-time information. For real-time message, the result of comparative experiment shows that the overall rate of instant information collection reaches 99%, and the accuracy rate reaches 100%. For non-real-time information, the system acquires the information of group announcement, the list of group members, the group shared file and album. By contrast of data sampling, the result shows that overall rate and accuracy rate all reach 100%. The experiment proves that social network data acquisition technology based on browser test components has its advantages on data acquisition.

关 键 词：社交网络 数据获取 用户行为模拟 浏览器测试组件 

分 类 号：TP393.092[自动化与计算机技术—计算机应用技术]