修剪算法的信息几何分析  

Information Geometric Analysis of Pruning Algorithm

作　　者：刘蕴辉[1] 罗四维[1] 黄华[1] 李爱军[1] 

机构地区：[1]北京交通大学计算机与信息技术学院,北京100044

出　　处：《计算机研究与发展》2006年第9期1609-1614,共6页Journal of Computer Research and Development

基　　金：国家自然科学基金项目(60373029);教育部博士点基金项目(20020004020);北京交通大学科技基金项目(2005RC044)

摘　　要：修剪法是确定和优化神经网络结构的重要方法之一.当前对修剪法的研究大多集中在方法描述上,对于修剪法内在机理的研究尚不多见,而研究修剪的内在机理可以为修剪策略提供理论基础和依据.从信息几何的角度研究了修剪法的内在机理,给出了神经网络结构修剪法的信息几何理论解释,利用神经流形参数结构的层次性,将修剪法表述为一系列从当前模型流形到其子流形的信息投影过程,在此基础上提出了新的修剪算法,并给出了算法可行性与有效性的实验验证.Pruning algorithm is an important method to set up and optimize the structure of neural network model. The research on pruning nowadays mostly focuses on the algorithm description while less effort is spent on its immanent mechanism. Research on its mechanism can provide theoretical basis for pruning strategy. The immanent mechanism of pruning is analyzed based on information geometry and a set of theoretical explanation of pruning is given. The pruning process is depicted as a series of information projections from the current model manifold to its submanifolds utilizing the hierarchical structure of neural manifold parameter space. A new pruning algorithm is presented based on the theoretical analysis and its validity and the efficiency is verified by experiments.

关 键 词：修剪法 信息几何 神经流形 信息投影 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]