一种基于关联数据的科技项目查重系统  

A Linked-data Based Similarity Detection System for Scientific Projects

作　　者：李善青[1] 安淑荻 邢晓昭[1] LI Shanqing;AN Shudi;XING Xiaozhao(Institute of Scientific and Technical Information of China,Beijing 100038)

机构地区：[1]中国科学技术信息研究所,北京100038

出　　处：《计算机与数字工程》2022年第5期959-963,共5页Computer & Digital Engineering

基　　金：国家自然科学基金项目“大数据挖掘在科技项目查重中的应用研究”(编号:71303223);中国科学技术信息研究所创新研究基金项目“基于大数据的区域科技创新态势识别研究”(编号:QN2020-01)资助。

摘　　要：论文提出了一种基于关联数据的科技项目查重系统,通过整合与科技项目密切相关的项目任务书、科技论文、科技报告和科技成果等信息,建立科技项目的描述模型。采用改进的TextRank算法从摘要和标题等短文本中抽取关键词构建特征向量,并提出了项目相似度的判别方法。利用Hadoop和Spark的分布式计算框架,实现了项目查重系统并进行了相关的实验。结果证明了该方法的有效性和可行性,在给定的实验条件下取得了较高的准确率和召回率。This paper proposes a similarity detection system based on related data and data mining to solve the problem. A representation model is constructed by integrating four types of information including project proposals,scientific papers,technical reports and project’s achievements. Meaningful keywords,extracted from scientific abstracts and titles by a modified TextRank algorithm,are used to construct the feature vectors. A discriminant function is formulated to calculate similarities of projects and implemented a detection system based on Hadoop and Spark framework. The experimental results demonstrates the effectiveness and feasibility of proposed method,which achieves good performance of precision rate and recall rate under the given experimental conditions.

关 键 词：关联数据 科技项目查重 数据挖掘 多源信息整合 Hadoop架构 

分 类 号：TP301.6[自动化与计算机技术—计算机系统结构]