面向多数据中心跨节点环境的公安大数据分析方法  

Public Security Big Data Analysis Method for Multi-data Center and Cross-node Environment

作　　者：杨杰[1] 邓旭冉 翟春婕[3] YANG Jie;DENG Xuran;ZHAI Chunjie(Public Security Bureau of Jiangsu Province,Nanjing 210024,China;Public Security Bureau of Xinjiang Production and Construction Corps,Urumqi 830002,China;Department of Information Technology,Nanjing Forest Police College,Nanjing 210023,China)

机构地区：[1]江苏省公安厅,江苏南京210024 [2]新疆生产建设兵团公安局,新疆乌鲁木齐830002 [3]南京森林警察学院信息技术学院,江苏南京210023

出　　处：《中国人民公安大学学报（自然科学版）》2020年第4期68-73,共6页Journal of People’s Public Security University of China(Science and Technology)

基　　金：公安部技术研究计划重点项目(2019JSYJA15);江苏省重点研发计划重大科技示范项目(BE2018754)。

摘　　要：随着公安大数据应用的不断深入,受技术限制、政策制约公安数据资源无法向统一的数据中心物理汇聚已是客观事实,传统基于单一数据中心的公安大数据分析方法正逐步失效。针对上述问题,提出一种面向多数据中心跨节点环境的分布式分析技术框架,并据此实现了常见公安大数据分析应用的跨节点分布式并行增量计算,既避免了数据的跨数据中心传输汇聚和多节点多副本存储,又避免了存量大数据的重复计算,极大提高了分析计算的效率和结果的准确度,也为跨节点分布式存储条件下开展公安大数据分析应用,提供了较为可行的思路和方法。With the wide applications of public security big data,it is an objective fact that public security data resources cannot be physically aggregated into a unified data center due to technical restrictions and policy constraints.The traditional public security big data analysis methods based on a single data center are gradually failing.In response to the above problem,a distributed analysis technology framework for the multi-data-center and cross-node environments was proposed,and based on this,cross-node distributed parallel incremental computing for common public security big data analysis application is realized.It not only avoids the transmission and aggregation of data across data centers and multi-node and multi-copy storage but also avoids the repeated calculation of stored big data.It greatly improved the efficiency of analysis and calculation and the accuracy of results,and also provides a more feasible idea and method for the development of public security big data analysis and application under the condition of cross-node distributed storage.

关 键 词：数据分析 跨节点 分布式 增量 

分 类 号：D631[政治法律—政治学]