Pi-sigma神经网络的乘子法随机单点在线梯度算法*

在利用梯度算法训练Pi-sigma神经网络时，存在因权值选取过小导致收敛速度过慢的问题，而采用一般罚函数法虽然可以克服这个缺点，但要求罚因子必须趋近于∞且惩罚项绝对值不可微，从而导致数值求解困难。为克服以上缺点，提出了一种基于乘子法的随机单点在线梯度算法。利用最优化理论方法，将有约束问题转换为无约束问题，利用乘子法来求解网络误差函数。从理论上分析了算法的收敛速度和稳定性，仿真实验结果验证了算法的有效性。

Training Pi-sigma neural network by stochastic simple point online gradient algorithm with Lagrange multiplier method

When the on-line gradient algotithm is used for training Pi-sigma neural netrork, there is a problem that the chosen weights may be very small, resulting in a very slow convergence. The shortcoming can be overcome by the penalty method, but there are the difficulties in numerical solution, caused by the facts that the penalty factor must approach infinity and the absolute value of penalty term is nondifferentiable. Based on Lagrange multipler algorithm, this paper proposed a stochastic simple point on-line gradient algorithm to overcome the deficiencies of small weights and penalty function. Using the optimized theory method, transformed the restrained question into the non-constraint question. Proved the convergence rate and stability of the algorithm. The simulated experimental results indicate that the algorithm is efficient.