基于全球整合新闻数据库的开源情报关联与可视化分析  

Correlation and Visualization Analysis of Open Source Intelligence Based on Global Dataset of Events,Location and Tone

作　　者：刘坤佳 陈科第[1] 乔凤才[1] 王晖[1] 

机构地区：[1]国防科学技术大学信息系统与管理学院,长沙410073

出　　处：《情报学报》2017年第2期152-158,共7页Journal of the China Society for Scientific and Technical Information

摘　　要：GDELT是一个实时、开源、全球性的社会事件新闻数据库,它收录全世界从1979年至今的网络、纸质新闻报道中抽取的事件。本文设计并实现了一个GDELT数据库采集器,实时采集GDELT的元数据,并将其存储在HDFS分布式文件系统中;提出了一种基于哈希的方法在Spark中对GDELT的三个主要数据表进行快速的连接操作,得到GDELT联合数据集,以便充分挖掘每个事件的详细信息;最后,以韩国地区的GDELT联合数据集为例进行了时空可视化分析,如事件热度的区域时间分布、关注媒体分布、抽取结果的置信度分布等。为情报科技工作者及相关人员提供了一个新的角度和鳃决方案。The GDELT is a real time database of global human society for open research which monitors the world＇s broadcast, print, and web news since 1979, creating a free open platform for computing on the entire world. First, we designed and realized a data collector, which collects metadata of GDELT database in real time and stores them in a Hadoop Distributed File System （HDFS）. Then, we proposed a hash-based method to correlate Event tables, Mentions tables and GKG tables in GDELT, in order to digest every detailed information of each event. Finally, we took South Korea as example to make spatiotemporal visualization analysis, such as Event Spatiotemporal Heat Map, Distribu- tion of Media Attention and Event Extraction Confidence Dot Map. This all will provide a new perspective and solu- tion for further research.

关 键 词：全球整合新闻数据库 可视化分析 关联分析 

分 类 号：G219.1[文化科学—新闻学] TP311.13[自动化与计算机技术—计算机软件与理论]