Hierarchical Learning of Navigational Behaviors in an Autonomous Robot using a Predictive Sparse Distributed Memory

We describe a general framework for learning perception-based navigational behaviors in autonomous mobile robots. A hierarchical behavior-based decomposition of the control architecture is used to facilitate efficient modular learning. Lower level reactive behaviors such as collision detection and obstacle avoidance are learned using a stochastic hill-climbing method while higher level goal-directed navigation is achieved using a self-organizing sparse distributed memory. The memory is initially trained by teleoperating the robot on a small number of paths within a given domain of interest. During training, the vectors in the sensory space as well as the motor space are continually adapted using a form of competitive learning to yield basis vectors that efficiently span the sensorimotor space. After training, the robot navigates from arbitrary locations to a desired goal location using motor output vectors computed by a saliency-based weighted averaging scheme. The pervasive problem of perceptual aliasing in finite-order Markovian environments is handled by allowing both current as well as the set of immediately preceding perceptual inputs to predict the motor output vector for the current time instant. We describe experimental and simulation results obtained using a mobile robot equipped with bump sensors, photosensors and infrared receivers, navigating within an enclosed obstacle-ridden arena. The results indicate that the method performs successfully in a number of navigational tasks exhibiting varying degrees of perceptual aliasing.     

Active Learning for Vision-Based Robot Grasping

Reliable vision-based grasping has proved elusive outside of controlled environments. One approach towards building more flexible and domain-independent robot grasping systems is to employ learning to adapt the robot's perceptual and motor system to the task. However, one pitfall in robot perceptual and motor learning is that the cost of gathering the learning set may be unacceptably high. Active learning algorithms address this shortcoming by intelligently selecting actions so as to decrease the number of examples necessary to achieve good performance and also avoid separate training and execution phases, leading to higher autonomy. We describe the IE-ID3 algorithm, which extends the Interval Estimation (IE) active learning approach from discrete to real-valued learning domains by combining IE with a classification tree learning algorithm (ID-3). We present a robot system which rapidly learns to select the grasp approach directions using IE-ID3 given simplified superquadric shape approximations of objects. Initial results on a small set of objects show that a robot with a laser scanner system can rapidly learn to pick up new objects, and simulation studies show the superiority of the active learning approach for a simulated grasping task using larger sets of objects. Extensions of the approach and future areas of research incorporating more sophisticated perceptual and action representation are discussed     

Qualitative spatial representations from task-oriented perception and exploratory behaviors

We describe motor and perceptual behaviors that have proven useful for indoor navigation of an autonomous mobile robot. These behaviors take advantage of the large amount of structure that characterizes many indoor, office-like environments. Based on pre-existing structural landmarks, a mobile robot has the ability to explore, map, and navigate one among several office buildings sharing similar structural features, while coping with slow environment variations and local dynamics. The mobile robot develops and maintains an internal spatial representation of the environment in terms of a topological and qualitative map. The types of structural features suitable as navigation landmarks largely depend upon the available sensors. Adequate navigation performance is achieved by subdividing perception and navigation into a number of behaviors layered upon a multi-threaded real-time control architecture.     

Acquiring Mobile Robot Behaviors by Learning Trajectory Velocities

The development of robots that learn from experience is a relentless challenge confronting artificial intelligence today. This paper describes a robot learning method which enables a mobile robot to simultaneously acquire the ability to avoid objects, follow walls, seek goals and control its velocity as a result of interacting with the environment without human assistance. The robot acquires these behaviors by learning how fast it should move along predefined trajectories with respect to the current state of the input vector. This enables the robot to perform object avoidance, wall following and goal seeking behaviors by choosing to follow fast trajectories near: the forward direction, the closest object or the goal location respectively. Learning trajectory velocities can be done relatively quickly because the required knowledge can be obtained from the robot's interactions with the environment without incurring the credit assignment problem. We provide experimental results to verify our robot learning method by using a mobile robot to simultaneously acquire all three behaviors.     

Learning locomotion reflexes: A self-supervised neural system for a mobile robot

This article is concerned with an artificial neural system for a mobile robot reactive navigation in an unknown, cluttered environment. Reactive navigation is a process of immediately choosing locomotion actions in response to measured spatial situations, while no planning occurs. A task of a presented system is to provide a steering angle signal letting a robot reach a goal while avoiding collisions with obstacles. Basic reactive navigation methods are briefly characterized, special attention is paid to a neural approach to the considered problem. The authors describe the system's architecture and important details of the algorithm. The main parts of the system are: the Fuzzy ART neural self-organizing classifier, performing a perceptual space partitioning, and a neural associative memory, memorizing the system's experience and superposing influences of different behaviors. Tests show that the learning process, starting from zero, is efficient, despite some initial fluctuations of its effectiveness.     

Shaping robot behavior using principles from instrumental conditioning

Shaping by successive approximations is an important animal training technique in which behavior is gradually adjusted in response to strategically timed reinforcements. We describe a computational model of this shaping process and its implementation on a mobile robot. Innate behaviors in our model are sequences of actions and enabling conditions, and shaping is a behavior editing process realized by multiple editing mechanisms. The model replicates some fundamental phenomena associated with instrumental learning in animals, and allows an RWI B21 robot to learn several distinct tasks derived from the same innate behavior.     

Multi-Layer Hierarchical Rule Learning in Reactive Robot Control Using Incremental Decision Trees

This paper presents a new approach to the intelligent navigation of a mobile robot. The hybrid control architecture described combines properties of purely reactive and behaviour-based systems, providing the ability both to learn automatically behaviours from inception, and to capture these in a distributed hierarchy of decision tree networks. The robot is first trained in the simplest world which has no obstacles, and is then trained in successively more complex worlds, using the knowledge acquired in the previous worlds. Each world representing the perceptual space is thus directly mapped on a unique rule layer which represents in turn the robot action space encoded in a distinct decision tree. A major advantage of the current implementation, compared with the previous work, is that the generated rules are easily understood by human users. The paper demonstrates that the proposed behavioural decomposition approach provides efficient management of complex knowledge, and that the learning mechanism is able to cope with noise and uncertainty in sensory data.     

Towards modelling complex robot training tasks through system identification

Previous research has shown that sensor–motor tasks in mobile robotics applications can be modelled automatically, using NARMAX system identification, where the sensory perception of the robot is mapped to the desired motor commands using non-linear polynomial functions, resulting in a tight coupling between sensing and acting — the robot responds directly to the sensor stimuli without having internal states or memory.However, competences such as for instance sequences of actions, where actions depend on each other, require memory and thus a representation of state. In these cases a simple direct link between sensory perception and the motor commands may not be enough to accomplish the desired tasks. The contribution of this paper to knowledge is to show how fundamental, simple NARMAX models of behaviour can be used in a bootstrapping process to generate complex behaviours that were so far beyond reach.We argue that as the complexity of the task increases, it is important to estimate the current state of the robot and integrate this information into the system identification process. To achieve this we propose a novel method which relates distinctive locations in the environment to the state of the robot, using an unsupervised clustering algorithm. Once we estimate the current state of the robot accurately, we combine the state information with the perception of the robot through a bootstrapping method to generate more complex robot tasks: We obtain a polynomial model which models the complex task as a function of predefined low level sensor–motor controllers and raw sensory data.The proposed method has been used to teach Scitos G5 mobile robots a number of complex tasks, such as advanced obstacle avoidance, or complex route learning.     

Learning landmark triples by experimentation

This article describes a method for learning a set of landmarks suitable for place navigation. The approach is novel in that it exploits the ability of a robot to learn through active perception in the task environment, similar to the learning by experimentation technique developed for LEX (Mitchell et al., 1990). The proposed strategy uses heuristics to select and rank candidate triples, then generates test cases to confirm that the best triple is sufficient. The method supports the use of multiple sensors with different computational and energy costs, where a utility function captures the tradeoff between navigational performance ranking and cost.

Over 100 data points were collected on a mobile robot using a laser barcode reader and computer vision to identify landmarks. The results indicated that active perception and experimentation identified triples with better navigational properties. Furthermore, the learning process is proactive: it was shown to prevent the robot from learning a triple which was not visible over the entire navigational space and/or was not sufficient in practice.     

Visual motor control of a 7 DOF robot manipulator using a fuzzy SOM network

A fuzzy self-organizing map (SOM) network is proposed in this paper for visual motor control of a 7 degrees of freedom (DOF) robot manipulator. The inverse kinematic map from the image plane to joint angle space of a redundant manipulator is highly nonlinear and ill-posed in the sense that a typical end-effector position is associated with several joint angle vectors. In the proposed approach, the robot workspace in image plane is discretized into a number of fuzzy regions whose center locations and fuzzy membership values are determined using a Fuzzy C-Mean (FCM) clustering algorithm. SOM network then learns the inverse kinematics by on-line by associating a local linear map for each cluster. A novel learning algorithm has been proposed to make the robot manipulator to reach a target position. Any arbitrary level of accuracy can be achieved with a number of fine movements of the manipulator tip. These fine movements depend on the error between the target position and the current manipulator position. In particular, the fuzzy model is found to be better as compared to Kohonen self-organizing map (KSOM) based learning scheme proposed for visual motor control. Like existing KSOM learning schemes, the proposed scheme leads to a unique inverse kinematic solution even for a redundant manipulator. The proposed algorithms have been successfully implemented in real-time on a 7 DOF PowerCube robot manipulator, and results are found to concur with the theoretical findings.     

Embedded visual behaviors for navigation

The implementation of a set of visually based behaviors for navigation is presented. The approach, which has been inspired by insect's behaviors, is aimed at building a “library” of embedded visually guided behaviors coping with the most common situations encountered during navigation in an indoor environment. Following this approach, the main goal is no longer how to characterize the environment, but how to embed in each behavior the perceptual processes necessary to understand the aspects of the environment required to generate a purposeful motor output.

The approach relies on the purposive definition of the task to be solved by each of the behaviors and it is based on the possibility of computing visual information during the action. All the implemented behaviors share the same input process (partial information of the image flow field) and the same control variables (heading direction and velocity) to demonstrate both the generality of the approach as well as its efficient use of the computational resources. The controlled mobile base is supposed to move on a flat surface but virtually no calibration is required of the intrinsic and extrinsic parameters of the two cameras and no attempt is made at building a 2D or 3D map of the environment: the only output of the perceptual processes is a motor command.

The first behavior, the centering reflex allows a robot to be easily controlled to navigate along corridors or following walls of a given scene structure. The second behavior extends the system capabilities to the detection of obstacles lying on the pavement in front of the mobile robot. Finally docking behaviors to control the robot to a given position in the environment, with controlled speed and orientation, are presented.

Besides the long-term goal of building a completely autonomous system, these behaviors can have very short-term applications in the area of semi-autonomous systems by taking care of the continuous, tedious control required during routine navigation.     

Reactive navigation of underwater mobile robot using ANFIS approach in a manifold manner

Learning and self-adaptation ability is highly required to be integrated in path planning algorithm for underwater robot during navigation through an unspecified underwater environment. High frequency oscillations during underwater motion are responsible for nonlinearities in dynamic behavior of underwater robot as well as uncertainties in hydrodynamic coefficients. Reactive behaviors of underwater robot are designed considering the position and orientation of both target and nearest obstacle from robot’s current position. Human like reasoning power and approximation based learning skill of neural based adaptive fuzzy inference system (ANFIS) has been found to be effective for underwater multivariable motion control. More than one ANFIS models are used here for achieving goal and obstacle avoidance while avoiding local minima situation in both horizontal and vertical plane of three dimensional workspace. An error gradient approach based on input-output training patterns for learning purpose has been promoted to spawn trajectory of underwater robot optimizing path length as well as time taken. The simulation and experimental results endorse sturdiness and viability of the proposed method in comparison with other navigational methodologies to negotiate with hectic conditions during motion of underwater mobile robot.     

Sonar-based mobile robot navigation through supervised learning on a neural net

For mobile robot navigation in an unknown and changing environment, a reactive approach is both simple to implement and fast in response. A neural net can be trained to exhibit such a behaviour. The advantage is that, it relates the desired motion directly to the sensor inputs, obviating the need of modeling and planning. In this work, a feedforward neural net is trained to output reactive motion in response to ultrasonic range inputs, with data generated artificially on the computer screen. We develop input and output representations appropriate to this problem.A purely reactive robot, being totally insensitive to context, often gets trapped in oscillations in front of a wide object. To overcome this problem, we introduce a notion of memory into the net by including context units at the input layer. We describe the mode of training for such a net and present simulated runs of a point robot under the guidance of the trained net in various situations. We also train a neural net for the navigation of a mobile robot with a finite turning radius. The results of the numerous test runs of the mobile robot under the control of the trained neural net in simulation as well as in experiments carried out in the laboratory, are reported in this paper.     

Attention control learning in the decision space using state estimation

The main goal of this paper is modelling attention while using it in efficient path planning of mobile robots. The key challenge in concurrently aiming these two goals is how to make an optimal, or near-optimal, decision in spite of time and processing power limitations, which inherently exist in a typical multi-sensor real-world robotic application. To efficiently recognise the environment under these two limitations, attention of an intelligent agent is controlled by employing the reinforcement learning framework. We propose an estimation method using estimated mixture-of-experts task and attention learning in perceptual space. An agent learns how to employ its sensory resources, and when to stop observing, by estimating its perceptual space. In this paper, static estimation of the state space in a learning task problem, which is examined in the Webots^TM simulator, is performed. Simulation results show that a robot learns how to achieve an optimal policy with a controlled cost by estimating the state space instead of continually updating sensory information.     

MOTION PLANNING OF AN INTELLIGENT ROBOT USING GA MOTIVATED TEMPORAL ASSOCIATIVE MEMORY

ABSTRACT

The paper discusses the concept of re-planning for a mobile robot in the presence of semidynamic obstacles. The navigational planning is done by employing genetic algorithm until it reaches the goal point. The path segments traversed by the mobile robot are stored by a simple matrix, employing temporal associative memory. During subsequent traversal, the robot utilizes the previously stored matrix to avoid an obstacle path. In case of deadlock, the robot back tracks using TAM and finds alternative paths to reach the goal. This algorithm has been realized on a Pioneer 2DX mobile robot of ActiveMedia Robotic LLC, USA, through client server architecture. The result shows that the robot reaches the goal within a vicinity of a 20 mm radius.     

Incremental supervised learning for mobile robot reactive control

Reactive control for a mobile robot can be defined as a mapping from a perceptual space to a command space. This mapping can be hard-coded by the user (potential fields, fuzzy logic), and can also be learnt. This paper is concerned with supervised learning for perception to action mapping for a mobile robot. Among the existing neural approaches for supervised learning of a function, we have selected the grow and learn network for its properties adapted to robotic problems: incrementality and flexible structure. We will present the results we have obtained with this network using first raw sensor data and then pre-processed measures with the automatic construction of virtual sensors.     

Automated Derivation of Primitives for Movement Classification

We describe a new method for representing human movement compactly, in terms of a linear super-imposition of simpler movements termed primitives. This method is a part of a larger research project aimed at modeling motor control and imitation using the notion of perceptuo-motor primitives, a basis set of coupled perceptual and motor routines. In our model, the perceptual system is biased by the set of motor behaviors the agent can execute. Thus, an agent can automatically classify observed movements into its executable repertoire. In this paper, we describe a method for automatically deriving a set of primitives directly from human movement data.We used movement data gathered from a psychophysical experiment on human imitation to derive the primitives. The data were first filtered, then segmented, and principal component analysis was applied to the segments. The eigenvectors corresponding to a few of the highest eigenvalues provide us with a basis set of primitives. These are used, through superposition and sequencing, to reconstruct the training movements as well as novel ones. The validation of the method was performed on a humanoid simulation with physical dynamics. The effectiveness of the motion reconstruction was measured through an error metric. We also explored and evaluated a technique of clustering in the space of primitives for generating controllers for executing frequently used movements.     

Learning to Perceive and Act by Trial and Error

This article considers adaptive control architectures that integrate active sensory-motor systems with decision systems based on reinforcement learning. One unavoidable consequence of active perception is that the agent's internal representation often confounds external world states. We call this phoenomenon perceptual aliasingand show that it destabilizes existing reinforcement learning algorithms with respect to the optimal decision policy. We then describe a new decision system that overcomes these difficulties for a restricted class of decision problems. The system incorporates a perceptual subcycle within the overall decision cycle and uses a modified learning algorithm to suppress the effects of perceptual aliasing. The result is a control architecture that learns not only how to solve a task but also where to focus its visual attention in order to collect necessary sensory information.     

基于强化学习和模糊逻辑的移动机器人导航

自主导航是移动机器人的一项关键技术。该文采用强化学习结合模糊逻辑的方法实现了未知环境下自主式移动机机器人的导航控制。文中首先介绍了强化学习原理,然后设计了一种未知环境下机器人导航框架。该框架由避碰模块、寻找目标模块和行为选择模块组成。针对该框架,提出了一种基于强化学习和模糊逻辑的学习、规划算法：在对避碰和寻找目标行为进行独立学习后,利用超声波传感器得到的环境信息进行行为选择,使机器人在成功避碰的同时到达目标点。最后通过大量的仿真实验,证明了算法的有效性。     

Mobile Robot Self-Localization without Explicit Landmarks

Localization is the process of determining the robot's location within its environment. More precisely, it is a procedure which takes as input a geometric map, a current estimate of the robot's pose, and sensor readings, and produces as output an improved estimate of the robot's current pose (position and orientation). We describe a combinatorially precise algorithm which performs mobile robot localization using a geometric model of the world and a point-and-shoot ranging device. We also describe a rasterized version of this algorithm which we have implemented on a real mobile robot equipped with a laser rangefinder we designed. Both versions of the algorithm allow for uncertainty in the data returned by the range sensor. We also present experimental results for the rasterized algorithm, obtained using our mobile robots at Cornell. Received November 15, 1996; revised January 13, 1998.