A study of dynamic meta-learning for failure prediction in large-scale systems

Despite years of study on failure prediction, it remains an open problem, especially in large-scale systems composed of vast amount of components. In this paper, we present a dynamic meta-learning framework for failure prediction. It intends to not only provide reasonable prediction accuracy, but also be of practical use in realistic environments. Two key techniques are developed to address technical challenges of failure prediction. One is meta-learning to boost prediction accuracy by combining the benefits of multiple predictive techniques. The other is a dynamic approach to dynamically obtain failure patterns from a changing training set and to dynamically extract effective rules by actively monitoring prediction accuracy at runtime. We demonstrate the effectiveness and practical use of this framework by means of real system logs collected from the production Blue Gene/L systems at Argonne National Laboratory and San Diego Supercomputer Center. Our case studies indicate that the proposed mechanism can provide reasonable prediction accuracy by forecasting up to 82% of the failures, with a runtime overhead less than 1.0 min.     

A dynamic meta-learning rate-based model for gold market forecasting

In this paper, an improved EMD meta-learning rate-based model for gold price forecasting is proposed. First, we adopt the EMD method to divide the time series data into different subsets. Second, a back-propagation neural network model (BPNN) is used to function as the prediction model in our system. We update the online learning rate of BPNN instantly as well as the weight matrix. Finally, a rating method is used to identify the most suitable BPNN model for further prediction. The experiment results show that our system has a good forecasting performance.     

Adaptive adversarial prototyping network for few-shot prototypical translation

Translating multiple real-world source images to a single prototypical image is a challenging problem. Notably, these source images belong to unseen categories that did not exist during model training. We address this problem by proposing an adaptive adversarial prototype network (AAPN) and enhancing existing one-shot classification techniques. To overcome the limitations that traditional works cannot extract samples from novel categories, our method tends to solve the image translation task of unseen categories through a meta-learner. We train the model in an adversarial learning manner and introduce a style encoder to guide the model with an initial target style. The encoded style latent code enhances the performance of the network with conditional target style images. The AAPN outperforms the state-of-the-art methods in one-shot classification of brand logo dataset and achieves the competitive accuracy in the traffic sign dataset. Additionally, our model improves the visual quality of the reconstructed prototypes in unseen categories. Based on the qualitative and quantitative analysis, the effectiveness of our model for few-shot classification and generation is demonstrated.     

Tracking Context Changes through Meta-Learning

The article deals with the problem of learning incrementally (on-line) in domains where the target concepts are context-dependent, so that changes in context can produce more or less radical changes in the associated concepts. In particular, we concentrate on a class of learning tasks where the domain provides explicit clues as to the current context (e.g., attributes with characteristic values). A general two-level learning model is presented that effectively adjusts to changing contexts by trying to detect (via meta-learning) contextual clues and using this information to focus the learning process. Context learning and detection occur during regular on-line learning, without separate training phases for context recognition. Two operational systems based on this model are presented that differ in the underlying learning algorithm and in the way they use contextual information: METAL(B) combines meta-learning with a Bayesian classifier, while METAL(IB) is based on an instance-based learning algorithm. Experiments with synthetic domains as well as a number of real-world problems show that the algorithms are robust in a variety of dimensions, and that meta-learning can produce substantial increases in accuracy over simple object-level learning in situations with changing contexts.     

A fusion spatial attention approach for few-shot learning

Few-shot learning is a challenging problem in computer vision that aims to learn a new visual concept from very limited data. A core issue is that there is a large amount of uncertainty introduced by the small training set. For example, the few images may include cluttered backgrounds or different scales of objects. Existing approaches mostly address this problem from either the original image space or the embedding space by using meta-learning. To the best of our knowledge, none of them tackle this problem from both spaces jointly. To this end, we propose a fusion spatial attention approach that performs spatial attention in both image and embedding spaces. In the image space, we employ a Saliency Object Detection (SOD) module to extract the saliency map of an image and provide it to the network as an additional channel. In the embedding space, we propose an Adaptive Pooling (Ada-P) module tailored to few-shot learning that introduces a meta-learner to adaptively fuse local features of the feature maps for each individual embedding. The fusion process assigns different pooling weights to the features at different spatial locations. Then, weighted pooling can be conducted over an embedding to fuse local information, which can avoid losing useful information by considering the spatial importance of the features. The SOD and Ada-P modules can be used within a plug-and-play module and incorporated into various existing few-shot learning approaches. We empirically demonstrate that designing spatial attention methods for few-shot learning is a nontrivial task and our method has proven effective for it. We evaluate our method using both shallow and deeper networks on three widely used few-shot learning benchmarks, miniImageNet, tieredImageNet and CUB, and demonstrate very competitive performance.     

Evaluation and selection of group recommendation strategies for collaborative searching of learning objects

Nowadays, there is a wide variety of e-learning repositories that provide digital resources for education in the form of learning objects. Some of these systems provide recommender systems in order to help users in the search for and selection of the learning objects most appropriate to their individual needs. The search for and recommendation of learning objects are usually viewed as a solitary and individual task. However, a collaborative search can be more effective than an individual search in some situations – for example, when developing a digital course between a group of instructors. The problem of recommending learning objects to a group of users or instructors is much more difficult than the traditional problem of recommending to only one individual. To resolve this problem, this paper proposes a collaborative methodology for searching, selecting, rating and recommending learning objects. Additionally, voting aggregation strategies and meta-learning techniques are used in order to automatically obtain the final ratings without having to reach a consensus between all the instructors. A functional model has been implemented within the DELPHOS hybrid recommender system. Finally, various experiments have been carried out using 50 different groups in order to validate the proposed learning object group recommendation approach.     

非平衡数据训练方法概述

现实世界中数据分类的应用通常会遇到数据非平衡的问题,即数据中的一类样本在数量上远多于另一类,例如欺诈检测和文本分类问题等.其中少数类的样本通常具有巨大的影响力和价值,是我们主要关心的对象,称为正类,另一类则称为负类.正类样本与负类样本可能数量上相差极大,这给训练非平衡数据提出了挑战.传统机器训练算法可能会产生偏向多数类的结果,因而对于正类来说,预测的性能可能会很差.本文分析了导致非平衡数据分类性能差的多方面原因,并针对这些原因列出了多种解决方法.     

基于元迁移的太阳能电池板缺陷图像超分辨率重建方法

针对太阳能电池板隐裂缺陷在进行光学检测时存在的特征不明显问题,以及小样本导致的训练不充分问题,提出了基于元迁移的太阳能电池板缺陷图像超分辨率重建方法,采用联合训练方法,利用内部图像和外部大规模图像信息分别作为不同阶段的训练数据。首先将引入的大量数据用于模型的初步训练,学习外部大规模数据的公共特征,然后通过元学习模型MAML进行多任务训练,为快速适应小样本无监督任务寻找一个适合图像内部学习的初始参数,提高模型的泛化能力,最后将预训练参数迁移至改进的ZSSR中进行自监督学习。在DIV2K、Set5、BSD100和太阳能电池板电致发光成像数据集上进行训练,实验结果表明,与传统的CARN,RCAN,IKC,ZSSR方法相比,该方法具有更高的峰值信噪比,最高达到36.66,参数量更小,相比ZSSR降低了70000,图像重建时间更短,相比CARN降低了0.51s,具有更好的重建效果,更高的重建效率。     

Statistical approaches to combining binary classifiers for multi-class classification

One of the popular methods for multi-class classification is to combine binary classifiers. In this paper, we propose a new approach for combining binary classifiers. Our method trains a combining method of binary classifiers using statistical techniques such as penalized logistic regression, stacking, and a sparsity promoting penalty. Our approach has several advantages. Firstly, our method outperforms existing methods even if the base classifiers are well-tuned. Secondly, an estimate of conditional probability for each class can be naturally obtained. Furthermore, we propose selecting relevant binary classifiers by adding the group lasso type penalty in training the combining method.     

Efficient feature size reduction via predictive forward selection

Most of the widely used pattern classification algorithms, such as Support Vector Machines (SVM), are sensitive to the presence of irrelevant or redundant features in the training data. Automatic feature selection algorithms aim at selecting a subset of features present in a given dataset so that the achieved accuracy of the following classifier can be maximized. Feature selection algorithms are generally categorized into two broad categories: algorithms that do not take the following classifier into account (the filter approaches), and algorithms that evaluate the following classifier for each considered feature subset (the wrapper approaches). Filter approaches are typically faster, but wrapper approaches deliver a higher performance. In this paper, we present the algorithm – Predictive Forward Selection – based on the widely used wrapper approach forward selection. Using ideas from meta-learning, the number of required evaluations of the target classifier is reduced by using experience knowledge gained during past feature selection runs on other datasets. We have evaluated our approach on 59 real-world datasets with a focus on SVM as the target classifier. We present comparisons with state-of-the-art wrapper and filter approaches as well as one embedded method for SVM according to accuracy and run-time. The results show that the presented method reaches the accuracy of traditional wrapper approaches requiring significantly less evaluations of the target algorithm. Moreover, our method achieves statistically significant better results than the filter approaches as well as the embedded method.