深度学习应用技术研究

本文针对深度学习应用技术进行了研究性综述。详细阐述了RBM（Restricted Boltzmann Machine）逐层预训练后再用BP（back-propagation）微调的深度学习贪婪层训练方法，对比分析了BP算法中三种梯度下降的方式，建议在线学习系统，采用随机梯度下降，静态离线学习系统采用随机小批量梯度下降；归纳总结了深度学习深层结构特征，并推荐了目前最受欢迎的5层深度网络结构设计方法。分析了前馈神经网络非线性激活函数的必要性及常用的激活函数优点，并推荐ReLU （rectified linear units）激活函数。最后简要概括了深度CNNs(Convolutional Neural Networks), 深度RNNs(recurrent neural networks), LSTM(long short-termmemory networks)等新型深度网络的特点及应用场景，并归纳总结了当前深度学习可能的发展方向。

A Study on the Application Technology of Deep Learning

This paper is a review of deep learning algorithms and their applications. It elaborates the greedy layer training algorithm which uses the fine-grained back-propagation (BP) learning following the layer-wise pre-training on each Restricted Boltzmann Machine (RBM) layer. After comparing and analyzing the three ways of gradient descent in the BP algorithm, we suggest applying stochastic gradient descent in online learning and adopting stochastic mini-batch gradient descent in static offline learning. We summarize the characteristic of the network structure in deep learning and recommend the design of state-of-art five-layer network architecture. We also analyze the necessity of the nonlinear activation function in feedforward neural networks and the advantages of the common activation functions, and recommend using ReLU activate function. Finally, the paper provides a brief summary of features and application scenarios of emerging deep neural networks such as Deep CNNs, Deep RNNs and LSTM, as well as the potential directions of future deep learning applications and research.