基于Python语言的网络舆情大数据传播特征挖掘研究  

Network Public Opinion Transmission Feature Mining Based on Python Language

作　　者：胡贵恒 张震[2] 陈翠红[1] HU Gui-heng;ZHANG Zhen;CHEN Cui-hong(School of Information Engineering,Anhui Business and Technology College,Hefei 231131,Anhui;School of Application Engineering,Anhui Business and Technology College,Hefei 231131,Anhui)

机构地区：[1]安徽工商职业学院信息工程学院,安徽合肥231131 [2]安徽工商职业学院应用工程学院,安徽合肥231131

出　　处：《陇东学院学报》2025年第2期21-26,共6页Journal of Longdong University

基　　金：安徽省教育厅质量工程“鸿蒙应用开发”(2023sdxx174);安徽省职业与成人教育学会重点课题“‘AI大模型+低代码’下高职院校软件开发类课程教学模式的探索与实践”(AZCJ2024024);“华为·安徽”2025产学合作创新课题“‘四维联动,四链融合’的安徽省现代产业院建设模式与路径的分析与研究”(ZCYJ-01)。

摘　　要：为解决网络舆情大数据传播特征挖掘存在读入延迟等问题,提出基于Python语言的网络舆情大数据传播特征挖掘研究。通过Python语言设计基于scrapy开源结构的改进爬虫算法,爬取网络中的舆情大数据;构建舆情大数据文本空间向量模型,提取数据内的文本特征;采用时间序列模型消除文本特征延时性,通过基于特征词向量的短文本聚类算法,计算短文本之间语义关联性,依据该关联性并通过层次聚类算法挖掘网络舆情大数据传播特征。经实验验证,该方法具有较低的读入延迟,能够挖掘得到舆情大数据的网络关注度、发帖数量以及转发时间频率等传播特征。To solve the problems of reading delay in the feature mining of network public opinion big data transmission,a research on the feature mining of network public opinion big data transmission based on Python language was proposed.An improved crawler algorithm based on Scrapy open source structure is designed by Python language,and the big data of public opinion in the network is crawled.Build a text space vector model for public opinion big data and extract text features from the data.The time-series model is used to eliminate the delay of text features.By using a short text clustering algorithm based on feature word vectors,the semantic correlation between short texts is calculated.Based on the relevance,the transmission characteristics of network public opinion big data are mined by the hierarchical clustering algorithm.Experimental results show that this method has a low read-in delay and can mine the communication characteristics of big data of public opinion,such as network attention,number of posts and forwarding time and frequency.

关 键 词：PYTHON语言 网络舆情 大数据 传播特征挖掘 scrapy开源结构 网络爬虫 

分 类 号：TP311[自动化与计算机技术—计算机软件与理论]