Modelling commuting time in the US: Bootstrapping techniques to avoid overfitting

The research on commuting has emerged in recent decades, but the issue of whether the empirical techniques used are appropriate has not been analysed. Thus, results from prior research could be based on non-accurate models, leading to misleading conclusions. We apply an algorithmic approach based on bootstrapping, variable selection, and mean absolute prediction errors, which is designed to avoid overfitting. Using the American Time Use Survey, we find that models with a reduced set of explanatory variables have similar accuracy to standard econometric models. Our results shed light on the importance of determining whether models can be overfitted.     

Learn more by training less: systematicity in sentence processing by recurrent networks

Connectionist models of sentence processing must learn to behave systematically by generalizing from a small training set. To what extent recurrent neural networks manage this generalization task is investigated. In contrast to Van der Velde et al. (Connection Sci., 16, pp. 21–46, 2004), it is found that simple recurrent networks do show so-called weak combinatorial systematicity, although their performance remains limited. It is argued that these limitations arise from overfitting in large networks. Generalization can be improved by increasing the size of the recurrent layer without training its connections, thereby combining a large short-term memory with a small long-term memory capacity. Performance can be improved further by increasing the number of word types in the training set.     

电动车用MH-Ni电池剩余电量的预测研究

蓄电池剩余容量为电动汽车可持续行驶提供有力的判据，因此，对它的准确估计具有重要意义。该文利用BP神经网络对剩余电量进行预测，采用试错法确定隐层节点数，尺度化共轭梯度反向传播算法对网络进行训练。通过对仿真结果的分析，发现网络出现了过拟合现象；通过引入BP网络学习能力和泛化能力的不确定关系，尝试建立新的网络；并对误差进行比较，取得最佳的泛化能力；进而对网络进行验证，证明新的网络是成功的。最后对进一步的研究提供思路。     

Color face recognition based on quaternion matrix representation

There are several methods to recognize and reconstruct a human face image. The principal component analysis (PCA) is a successful approach because of its effective extraction of the global feature and excellent reconstruction of face image. However, the crucial shortcomings of PCA are its low recognition rate and overfitting of feature extraction which leads to the dependence of training data on training samples. In this paper, a modified two-dimension principal component analysis (2DPCA) and bidirectional principal component analysis (BDPCA) methods based on quaternion matrix are proposed to recognize and reconstruct a color face image. In these methods, the spatial distribution information of color images is used to represent a color face, and the 2DPCA or BDPCA feature of color face image is extracted by reducing the dimensionality in both column and row directions. A method obtaining orthogonal eigenvector set of quaternion matrix is proposed. Numerous experiments show that the present approach based on quaternion matrix can effectively smooth the overfitting issue and substantially enhance the recognition rate.     

改进CAS性能的多网络表决模型

Fahlman和Lebiere提出的级联相关网络是一个典型的自适应神经网络的增长算法。它具有灵活、高效的特点,但由于该算法存在诸多的不确定因素,致使在其增长过程中引入过多的自由参数,它和随机选取的初始权重是导致单个神经网络过拟合的两个直接原因。本文提出的多网表决模型的基本思想是,利用多个网络来对未知的模式进行表决来确定其解,由于其平均效应,它能够避免单个网络预言带来的偏颇,获得满意的结果,利用我们建立的PC－FARM计算环境,本文还从实验上验证了网络表决模型的优越性。     

Information granularity model for evolving context-based fuzzy system

An information granule has to be translated into significant frameworks of granular computing to realize interpretability–accuracy tradeoff. These two objectives are in conflict and constitute an open problem. A new operational framework to form the evolving information granule (EIG) is developed in this paper, which ensures a compromise between interpretability and reasonable accuracy. The evolving information granule is initiated with the first information granule by translating the knowledge of the entire output domain. The initial information granule is considered an underfitting state with a high approximation error. Then, the EIG starts evolving in the information granule by partitioning the output domain and uses a dynamic constraint to maintain semantic interpretability in the output-contexts. The important criterion in the EIG is to determine the prominent distinction (output-context) in the output domain and realize the distinct information granule that depicts the semantics at the fuzzy partition level. The EIG tends to evolve toward the lower error region and realizes the effective rulebase by avoiding overfitting. The outcome on the synthetic and real-world data using the EIG shows the effectiveness of the proposed system, which outperforms state-of-the art methods.     

Sparsity-promoting polynomial response surface: A new surrogate model for response prediction

Computation-intensive analyses/simulations are becoming increasingly common in engineering design problems. To improve the computation efficiency, surrogate models are used to replace expensive simulations of engineering problems. This paper proposes a new high-fidelity surrogate modeling approach which is called the Sparsity-promoting Polynomial Response Surface (SPPRS). In the SPPRS model, a series of Legendre polynomials is selected as basis functions, and its number is compatible with the sample size so as to enhance the expression ability for complex functional relationships. The coefficients associated with basis functions are estimated using a “sparsity-promoting” regression approach which is an ensemble of two techniques: least squares and ℓ₁-norm regularization. As a result, only these basis functions relevant to explain the function relationship are picked out, and that dedicates to ease the problem of overfitting for training points. With the sparsity-promoting regression approach, such a surrogate model intends to capture both the global trend of the functional variation and a reasonable local accuracy in the neighborhood of training points. Additionally, Latin hypercube design (LHD) is proved conducive to improving the predictive capability of our model. The SPPRS is applied to seven benchmark test functions and a complex engineering problem. The results illustrate the promising benefits of this novel surrogate modeling technique.     

A comprehensive survey on regularization strategies in machine learning

In machine learning, the model is not as complicated as possible. Good generalization ability means that the model not only performs well on the training data set, but also can make good prediction on new data. Regularization imposes a penalty on model’s complexity or smoothness, allowing for good generalization to unseen data even when training on a finite training set or with an inadequate iteration. Deep learning has developed rapidly in recent years. Then the regularization has a broader definition: regularization is a technology aimed at improving the generalization ability of a model. This paper gave a comprehensive study and a state-of-the-art review of the regularization strategies in machine learning. Then the characteristics and comparisons of regularizations were presented. In addition, it discussed how to choose a regularization for the specific task. For specific tasks, it is necessary for regularization technology to have good mathematical characteristics. Meanwhile, new regularization techniques can be constructed by extending and combining existing regularization techniques. Finally, it concluded current opportunities and challenges of regularization technologies, as well as many open concerns and research trends.     

基于Adaboost算法的人脸检测

该文提出了一种基于改进的Adaboost算法的人脸检测方法.Adaboost是一种构建准确分类器的学习算法,它将一族弱学习算法通过一定规则结合成为一个强学习算法,从而通过样本训练得到一个识别准确率理想的分类器.但是,Adaboost在有高噪音样本的情况下,有可能发生过配现象,该文在Adaboost算法的基础上,对其权值更新规则做了改进,并结合PCA进行人脸检测.仿真试验表明,该方法具有良好的性能,同时可以在一定程度上有效防止过配现象的发生.