神经网络的两种结构优化算法研究

提出了一种基于权值拟熵的“剪枝算法”与权值敏感度相结合的新方法，在“剪枝算法”中将权值拟熵作为惩罚项加入目标函数中，使多层前向神经网络在学习过程中自动约束权值分布，并以权值敏感度作为简化标准，避免了单纯依赖权值大小剪枝的随机性．同时，又针对剪枝算法在优化多输入多输出网络过程中计算量大、效率不高的问题，提出了一种在级联—相关(cascade correlation, CC)算法的基础上从适当的网络结构开始对网络进行构建的快速“构造算法”．仿真结果表明这种快速构造算法在收敛速度、运行效率乃至泛化性能上都更胜一筹．

Two Structure Optimization Algorithms for Neural Networks

Based on pseudo-entropy of weights,a new method is proposed to integrate pruning algorithm with sensitivity weights.The pruning algorithm introduces the pseudo-entropy of weights as a penalty term into the normal objective function,and the distribution of weights is automatically constrained by a multilayer feed-forward neural network during the training process. The weight sensitivity is served as the simplification criteria of pruning to avoid the pruning randomicity caused by only using the weights.Meanwhile,for the problems of heavy computation burden and low efficiency of pruning algorithm in optimizing the multi-input and multi-output networks,a fast constructive algorithm is put forward,which is based on the Cascade-Correlation(CC) algorithm and constructs the new neural network from a proper network structure.The simulation results show that this fast constructive algorithm is a better choice in terms of convergence rate,computational efficiency and even generalization performance.