主题模型LDA的多文档自动文摘  

Automatic multi-document summarization based on the latent Dirichlet topic allocation model

作　　者：杨潇[1] 马军[2] 杨同峰[2] 杜言琦[2] 邵海敏[2] 

机构地区：[1]山东经济学院信息管理学院,山东济南250014 [2]山东大学计算机科学与技术学院,山东济南250101

出　　处：《智能系统学报》2010年第2期169-176,共8页CAAI Transactions on Intelligent Systems

基　　金：国家自然科学基金资助项目(60970047);山东省自然科学基金资助项目(Y2008G19);山东省科技计划资助项目(2007GG10001002;2008GG10001026)

摘　　要：近年来使用概率主题模型表示多文档文摘问题受到研究者的关注.LDA(latent dirichlet allocation)是主题模型中具有代表性的概率生成性模型之一.提出了一种基于LDA的文摘方法,该方法以混乱度确定LDA模型的主题数目,以Gibbs抽样获得模型中句子的主题概率分布和主题的词汇概率分布,以句子中主题权重的加和确定各个主题的重要程度,并根据LDA模型中主题的概率分布和句子的概率分布提出了2种不同的句子权重计算模型.实验中使用ROUGE评测标准,与代表最新水平的SumBasic方法和其他2种基于LDA的多文档自动文摘方法在通用型多文档摘要测试集DUC2002上的评测数据进行比较,结果表明提出的基于LDA的多文档自动文摘方法在ROUGE的各个评测标准上均优于SumBasic方法,与其他基于LDA模型的文摘相比也具有优势.The representative problem of multi-document summarization using probabilistic topic models has begun receiving considerable attention. A multi-document summarization method was proposed based on the latent dirichlet allocation （LDA） model, itself a model representative of probabilistic generative topic models. In this method, the number of topics in the LDA model was determined by model perplexity, and the probabilistic sentence distribution on topics and the probabilistic topic distribution on words were obtained by the Gibbs sampling method. The importance of topics was determined by the sum of topic weights on all sentences. Two sentence-scoring methods were proposed, one based on sentence distribution and the other on topic distribution. Evaluated by the recalloriented understudy for gisting evaluation （ROUGE） metrics, results of the both proposed methods surpassed the state-of-the-art SumBasic system and the other two LDA based summarization systems for all the ROUGE scores on the DUC2002 generic multi-document summarization test set.

关 键 词：多文档自动文摘 句子分值计算 主题模型 LDA 主题数目 

分 类 号：TP391.1[自动化与计算机技术—计算机应用技术]