基于深度学习的多模态新闻数据主题发现研究  

Data Analysis and Knowledge Discovery Topic Detecting on Multimodal News Data Based on Deep Learning

作　　者：倪亮 吴鹏[2] 周雪晴 Ni Liang;Wu Peng;Zhou Xueqing(School of Cyber Science&Engineering,Nanjing University of Science&Technology,Nanjing 210094,China;School of Intelligent Manufacturing,Nanjing University of Science&Technology,Nanjing 210094,China;School of Economics&Management,Nanjing University of Science&Technology,Nanjing 210094,China)

机构地区：[1]南京理工大学网络空间安全学院,南京210094 [2]南京理工大学智能制造学院,南京210094 [3]南京理工大学经济管理学院,南京210094

出　　处：《数据分析与知识发现》2024年第3期85-97,共13页Data Analysis and Knowledge Discovery

基　　金：国家自然科学基金项目(项目编号:72274096,71774084);江苏省青蓝工程优秀教学团队项目(项目编号:[2020]10)的研究成果之一。

摘　　要：【目的】基于多模态学习方法,对新闻中文本和图片相结合内容,构建多模态主题模型,自动挖掘新闻中的潜在主题。【方法】采用结合词嵌入的主题模型,从图片和文本两方面进行主题建模,并且使用多模态联合表征学习和协同表征学习的方法进行特征融合。最后,对发现的多模态新闻主题进行可视化分析,结合N15News数据集进行实证研究。【结果】实验结果表明,相对于仅使用文本特征的Label-ETM,多模态主题建模方法可以获得更好的主题的可解释性和多样性。这说明多模态主题建模方法具有一定的可行性与合理性。【局限】本文假设新闻中的图片和文字在语义和主题上是相关的,在弱相关和不相关领域多模态融合方法仍需要改善。【结论】多模态主题建模可以发现不同模态数据之间的联系,提高发现主题的多样性。[Objective]This paper constructs a multimodal topic model combining text and images in news based on multimodal learning methods.It aims to uncover latent topics in the news automatically.[Methods]We constructed a model incorporating word embedding for topics from texts and images.It uses multimodal joint representation learning and coordinate representation learning for feature fusion.We conducted visual analysis for the discovered multimodal news topics.Finally,we examined the new model on the N15News dataset.[Results]Compared to Label-ETM using only text features,the multimodal topic modeling approach can achieve better topic interpretability and diversity.This suggests that the multimodal topic modeling approach is feasible.[Limitations]We assume images and text in news are semantically and thematically related.Multimodal fusion methods need to be improved in weakly related and irrelevant domains.[Conclusions]Multimodal topic modeling can discover connections between different modal data and improve the diversity of discovered topics.

关 键 词：主题模型 多模态联合表征 多模态协同表征 新闻主题发现 

分 类 号：TP393[自动化与计算机技术—计算机应用技术] G250[自动化与计算机技术—计算机科学与技术]