Energy-information trade-off induces continuous and discontinuous phase transitions in lateral predictive coding  

作　　者：Zhen-Ye Huang Ruyi Zhou Miao Huang Hai-Jun Zhou 

机构地区：[1]Key Laboratory for Theoretical Physics,Institute of Theoretical Physics,Chinese Academy of Sciences,Beijing 100190,China [2]School of Physical Sciences,University of Chinese Academy of Sciences,Beijing 100049,China [3]School of Optics and Photonics,Beijing Institute of Technology,Beijing 100081,China [4]Minjiang Collaborative Center for Theoretical Physics,Minjiang University,Fuzhou 350108,China

出　　处：《Science China(Physics,Mechanics & Astronomy)》2024年第6期79-85,共7页中国科学：物理学、力学、天文学（英文版）

基　　金：supported by the National Natural Science Foundation of China(Grant Nos.12047503,11747601 and 12247104);the National Innovation Institute of Defense Technology(Grant No.22TQ0904ZT01025)。

摘　　要：Lateral predictive coding is a recurrent neural network that creates energy-efficient internal representations by exploiting statistical regularity in sensory inputs.Here,we analytically investigate the trade-off between information robustness and energy in a linear model of lateral predictive coding and numerically minimize a free energy quantity.We observed several phase transitions in the synaptic weight matrix,particularly a continuous transition that breaks reciprocity and permutation symmetry and builds cyclic dominance and a discontinuous transition with the associated sudden emergence of tight balance between excitatory and inhibitory interactions.The optimal network follows an ideal gas law over an extended temperature range and saturates the efficiency upper bound of energy use.These results provide theoretical insights into the emergence and evolution of complex internal models in predictive processing systems.

关 键 词：predictive coding recurrent neural network phase transition internal model free energy 

分 类 号：TP183[自动化与计算机技术—控制理论与控制工程]