基于GaussNewton-NL2SOL法的前馈神经网络及应用

目前基于高斯牛顿法及其衍生算法的前馈神经网络虽然可以达到局部二阶收敛速度,但只对小残量或零残量问题有效,对大残量问题则收敛很慢甚至不收敛.为了实时解决神经网络学习过程中可能遇到的小残量问题和大残量问题,引入NL2SOL优化算法,并与GaussNewton法相结合,构建基于GaussNewton-NL2SOL法的前馈神经网络.仿真实例表明,该神经网络较好地解决了残量问题,具有良好的收敛性和稳定性.

Feed-forward Neural Network Based onGaussNewton-NL2SOL Algorithm and Its Application

Though the feed forward neural network based on GaussNewton algorithm and its derivation will converge with order two, it is only effective toward little residual problem. In order to solve the little and large residual problems at the same time, NL2SOL algorithm is introduced and combined with the GaussNewton algorithm so as to form a feed forward neural network based on GaussNewton-NL2SOL algorithm. The application shows that this neural network can solve the residual problem properly and the convergence and stability of it performs well.