Incremental learning methods with retrieving of interfered patterns

There are many cases when a neural-network-based system must memorize some new patterns incrementally. However, if the network learns the new patterns only by referring to them, it probably forgets old memorized patterns, since parameters in the network usually correlate not only to the old memories but also to the new patterns. A certain way to avoid the loss of memories is to learn the new patterns with all memorized patterns. It needs, however, a large computational power. To solve this problem, we propose incremental learning methods with retrieval of interfered patterns (ILRI). In these methods, the system employs a modified version of a resource allocating network (RAN) which is one variation of a generalized radial basis function (GRBF). In ILRI, the RAN learns new patterns with a relearning of a few number of retrieved past patterns that are interfered with the incremental learning. We construct ILRI in two steps. In the first step, we construct a system which searches the interfered patterns from past input patterns stored in a database. In the second step, we improve the first system in such a way that the system does not need the database. In this case, the system regenerates the input patterns approximately in a random manner. The simulation results show that these two systems have almost the same ability, and the generalization ability is higher than other similar systems using neural networks and k-nearest neighbors.     

Superimposing neural learning by dynamic and spatial changing weights

In this article, a new neural network model is presented for incremental learning tasks where networks are required to learn new knowledge without forgetting the old. An essential core of the proposed network structure is their dynamic and spatial changing connection weights (DSCWs). A learning scheme is developed for the formulation of the dynamic changing weights, while a structural adaptation is formulated by the spatial changing connecting weights. To avoid disturbing the old knowledge by the creation of new connections, a restoration mechanism is introduced dusing the DSCWs. The usefulness of the proposed model is demonstrated by using a system identification task. This work was presented in part at the 7th International Symposium on Artificial Life and Robotics, Oita, Japan, January 16–18, 2002.     

Dynamic learning from multiple examples for semantic object segmentation and search

We present a novel “dynamic learning” approach for an intelligent image database system to automatically improve object segmentation and labeling without user intervention, as new examples become available, for object-based indexing. The proposed approach is an extension of our earlier work on “learning by example,” which addressed labeling of similar objects in a set of database images based on a single example. The proposed dynamic learning procedure utilizes multiple example object templates to improve the accuracy of existing object segmentations and labels. Multiple example templates may be images of the same object from different viewing angles, or images of related objects. This paper also introduces a new shape similarity metric called normalized area of symmetric differences (NASD), which has desired properties for use in the proposed “dynamic learning” scheme, and is more robust against boundary noise that results from automatic image segmentation. Performance of the dynamic learning procedures has been demonstrated by experimental results.     

An investigation of coopetitive pedagogic design for knowledge creation in Web-based learning

Information and communication technologies (ICTs) have created a supportive environment for collaborative learning at the expense of student motivation and engagement. This study attempts to explore the development of a productive learning atmosphere in the context of Web-based learning. An experiment is conducted with university-level students having homogenous background and coursework by applying heterogeneous pedagogies that create either competitive or collaborative learning atmospheres. The differences in learning atmosphere bring about variations in social presence and enjoyment of learning. The findings show that “coopetition” (defined as collaboration within the group and competition between groups) was the best learning strategy because competition and collaboration stimulated different types of knowledge growth in the knowledge-creation spiral. Competitive learning atmospheres encourage students to develop higher analytic skills, while collaborative learning atmospheres prompt students to demonstrate higher synthetic skills. Because both atmospheres contribute to learning, this study has found that combining both pedagogies in constructing a coopetitive learning atmosphere not only contributes to analytic and synthetic skills, but also raises the overall knowledge level. The findings pinpointed the importance of creating a learning environment that integrates ICTs, learners’ backgrounds, courseware, and pedagogic considerations in the process of increasing knowledge levels.     

A Bayesian attractor network with incremental learning

A realtime online learning system with capacity limits needs to gradually forget old information in order to avoid catastrophic forgetting. This can be achieved by allowing new information to overwrite old, as in a so-called palimpsest memory. This paper describes an incremental learning rule based on the Bayesian confidence propagation neural network that has palimpsest properties when employed in an attractor neural network. The network does not suffer from catastrophic forgetting, has a capacity dependent on the learning time constant and exhibits faster convergence for newer patterns.     

机器人增量学习研究综述

机器人的应用场景正在不断更新换代,数据量也在日益增长.传统的机器学习方法难以适应动态的环境,而增量学习技术能够模拟人类的学习过程,使机器人能利用旧知识来加快新任务的学习,在不遗忘旧技能的前提下学习新的技能.目前对于机器人增量学习的相关研究仍然较少,对此,主要介绍机器人增量学习研究进展.首先,对增量学习进行简介;其次,从参数和模型的角度出发,将当前机器人增量学习主流方法分为变参数方法、变模型方法、混合方法3类,分别对每一类进行论述,并给出相应的增量学习技术在机器人领域中的应用实例;然后,对机器人增量学习中常用的数据集和评价指标进行介绍;最后,对增量学习未来的发展趋势进行展望.     

Designing a semantic bliki system to support different types of knowledge and adaptive learning

Though blogs and wikis have been used to support knowledge management and e-learning, existing blogs and wikis cannot support different types of knowledge and adaptive learning. A case in point, types of knowledge vary greatly in category and viewpoints. Additionally, adaptive learning is crucial to improving one’s learning performance. This study aims to design a semantic bliki system to tackle such issues. To support various types of knowledge, this study has developed a new social software called “bliki” that combines the advantages of blogs and wikis. This bliki system also applies Semantic Web technology to organize an ontology and a variety of knowledge types. To aid adaptive learning, a function called “Book” is provided to enable learners to arrange personalized learning goals and paths. The learning contents and their sequences and difficulty levels can be specified according to learners’ metacognitive knowledge and collaborative activities. An experiment is conducted to evaluate this system and the experimental results show that this system is able to comprehend various types of knowledge and to improve learners’ learning performance.     

An incremental neural learning framework and its application to vehicle diagnostics

This paper presents a framework for incremental neural learning (INL) that allows a base neural learning system to incrementally learn new knowledge from only new data without forgetting the existing knowledge. Upon subsequent encounters of new data examples, INL utilizes prior knowledge to direct its incremental learning. A number of critical issues are addressed including when to make the system learn new knowledge, how to learn new knowledge without forgetting existing knowledge, how to perform inference using both the existing and the newly learnt knowledge, and how to detect and deal with aged learnt systems. To validate the proposed INL framework, we use backpropagation (BP) as a base learner and a multi-layer neural network as a base intelligent system. INL has several advantages over existing incremental algorithms: it can be applied to a broad range of neural network systems beyond the BP trained neural networks; it retains the existing neural network structures and weights even during incremental learning; the neural network committees generated by INL do not interact with one another and each sees the same inputs and error signals at the same time; this limited communication makes the INL architecture attractive for parallel implementation. We have applied INL to two vehicle fault diagnostics problems: end-of-line test in auto assembly plants and onboard vehicle misfire detection. These experimental results demonstrate that the INL framework has the capability to successfully perform incremental learning from unbalanced and noisy data. In order to show the general capabilities of INL, we also applied INL to three general machine learning benchmark data sets. The INL systems showed good generalization capabilities in comparison with other well known machine learning algorithms.     

Negative correlation in incremental learning

Negative Correlation Learning (NCL) has been successfully applied to construct neural network ensembles. It encourages the neural networks that compose the ensemble to be different from each other and, at the same time, accurate. The difference among the neural networks that compose an ensemble is a desirable feature to perform incremental learning, for some of the neural networks can be able to adapt faster and better to new data than the others. So, NCL is a potentially powerful approach to incremental learning. With this in mind, this paper presents an analysis of NCL, aiming at determining its weak and strong points to incremental learning. The analysis shows that it is possible to use NCL to overcome catastrophic forgetting, an important problem related to incremental learning. However, when catastrophic forgetting is very low, no advantage of using more than one neural network of the ensemble to learn new data is taken and the test error is high. When all the neural networks are used to learn new data, some of them can indeed adapt better than the others, but a higher catastrophic forgetting is obtained. In this way, it is important to find a trade-off between overcoming catastrophic forgetting and using an entire ensemble to learn new data. The NCL results are comparable with other approaches which were specifically designed to incremental learning. Thus, the study presented in this work reveals encouraging results with negative correlation in incremental learning, showing that NCL is a promising approach to incremental learning.

The formation of learners’ semiosphere by authentic inquiry with an integrated learning object “Young Scientist”

An integrated learning object, a web-based inquiry environment “Young Scientist” for basic school level is introduced by applying the semiosphere conception for explaining learning processes. The study focused on the development of students’ (n = 30) awareness of the affordances of learning objects (LO) during the 3 inquiry tasks, and their ability of dynamically reconstructing meanings in the inquiry subtasks through exploiting these LO affordances in “Young Scientist”. The problem-solving data recorded by the inquiry system and the awareness questionnaire served as the data-collection methods.It was demonstrated that learners obtain complete awareness of the LO affordances in an integrated learning environment only after several problem-solving tasks. It was assumed that the perceived task-related properties and functions of LOs depend on students’ interrelations with LOs in specific learning contexts. Learners’ overall awareness of certain LO affordances, available in the inquiry system “Young Scientist”, developed with three kinds of patterns, describing the hierarchical development of the semiosphere model for learners. The better understanding of the LO affordances, characteristic to the formation of the functioning semiosphere, was significantly related to the advanced knowledge construction during these inquiry subtasks that presumed translation of information from one semiotic system to another. The implications of the research are discussed in the frames of the development of new contextual gateways for learning with virtual objects. It is assumed that effective LO-based learning has to be organized through pedagogically constrained gateways by manifesting certain LO affordances in the context in order to build up the dynamic semiosphere model for learners.