基于多相似度融合算法的Pol智能化排重方法  

A method of Pol intelligent rearrangement based on multi-similarity fusion algorithm

作　　者：唐勇 耿鲁静 李世光 孙言成 TANG Yong;GENG Lu-jing;LI Shi-guang;SUN Yan-cheng(China Mobile Group Co.,Ltd.,Beijing 100033,China;China Mobile Group Design Institute Co.,Ltd.,Beijing 100080,China;China Mobile Group Design Institute Co.,Ltd.Shandong Branch,Jinan 250001,China)

机构地区：[1]中国移动通信集团有限公司,北京100033 [2]中国移动通信集团设计院有限公司,北京100080 [3]中国移动通信集团设计院有限公司山东分公司,济南250001

出　　处：《电信工程技术与标准化》2022年第S01期42-46,共5页Telecom Engineering Technics and Standardization

摘　　要：现有的PoI数据清洗工作形态已经不能支持自智网络时代庞大复杂的网络架构,为尽量智能地识别出PoI数据库中全量PoI重复数据,本文提出一种基于多相似度融合算法的PoI智能化排重方法。基于多相似度融合算法的PoI智能化排重方法摒弃了以往方法的单一性,经实验分别验证词形相似度、基于编辑距离的句子相似度和基于权重值融合的相似度可行性,通过大数据分析计算得出各相似度应占权重值,并且经过大数据处理确定3类相似度计算方法的阈值,得到最终的排重算法。实验结果表明重复数据识别效果和效率均得到极大幅度的提升,智能化程度得到进一步提高。The existing Pol data cleaning can no longer support the analysis of complex network architecture,it is essential to promote an intelligent Pol rearrangement method based on multi-similarity fusion algorithm to identify duplicate data in the Pol database.The method abandons the monogeneity of previous methods.The feasibility of word similarity,sentence similarity based on edit distance and similarity based on weight value fusion is verified by experiments.Finally,it is calculated by big data analysis that each similarity should account for the weight value,rather than a simple linear combination.And the rearrangement algorithm is obtained by determine the threshold of similarity calculations.By applying the method,the efficiency of duplicate data identification are greatly improved,and the degree of intelligenceis furtherimproved.

关 键 词：PoI数据 编辑距离 文本相似度 加权融合 

分 类 号：TN915[电子电信—通信与信息系统]