Human–robot skills transfer interfaces for a flexible surgical robot

In minimally invasive surgery, tools go through narrow openings and manipulate soft organs to perform surgical tasks. There are limitations in current robot-assisted surgical systems due to the rigidity of robot tools. The aim of the STIFF-FLOP European project is to develop a soft robotic arm to perform surgical tasks. The flexibility of the robot allows the surgeon to move within organs to reach remote areas inside the body and perform challenging procedures in laparoscopy. This article addresses the problem of designing learning interfaces enabling the transfer of skills from human demonstration. Robot programming by demonstration encompasses a wide range of learning strategies, from simple mimicking of the demonstrator's actions to the higher level imitation of the underlying intent extracted from the demonstrations. By focusing on this last form, we study the problem of extracting an objective function explaining the demonstrations from an over-specified set of candidate reward functions, and using this information for self-refinement of the skill. In contrast to inverse reinforcement learning strategies that attempt to explain the observations with reward functions defined for the entire task (or a set of pre-defined reward profiles active for different parts of the task), the proposed approach is based on context-dependent reward-weighted learning, where the robot can learn the relevance of candidate objective functions with respect to the current phase of the task or encountered situation. The robot then exploits this information for skills refinement in the policy parameters space. The proposed approach is tested in simulation with a cutting task performed by the STIFF-FLOP flexible robot, using kinesthetic demonstrations from a Barrett WAM manipulator.     

Using perspective taking to learn from ambiguous demonstrations

This paper addresses an important issue in learning from demonstrations that are provided by “naïve” human teachers—people who do not have expertise in the machine learning algorithms used by the robot. We therefore entertain the possibility that, whereas the average human user may provide sensible demonstrations from a human’s perspective, these same demonstrations may be insufficient, incomplete, ambiguous, or otherwise “flawed” from the perspective of the training set needed by the learning algorithm to generalize properly. To address this issue, we present a system where the robot is modeled as a socially engaged and socially cognitive learner. We illustrate the merits of this approach through an example where the robot is able to correctly learn from “flawed” demonstrations by taking the visual perspective of the human instructor to clarify potential ambiguities.     

Chaotic metaheuristic algorithms for learning and reproduction of robot motion trajectories

Most of today’s mobile robots operate in controlled environments prone to various unpredictable conditions. Programming or reprogramming of such systems is time-consuming and requires significant efforts by number of experts. One of the solutions to this problem is to enable the robot to learn from human teacher through demonstrations or observations. This paper presents novel approach that integrates Learning from Demonstrations methodology and chaotic bioinspired optimization algorithms for reproduction of desired motion trajectories. Demonstrations of the different trajectories to reproduce are gathered by human teacher while teleoperating the mobile robot in working environment. The learning (optimization) goal is to produce such sequence of mobile robot actuator commands that generate minimal error in the final robot pose. Four different chaotic methods are implemented, namely chaotic Bat Algorithm, chaotic Firefly Algorithm, chaotic Accelerated Particle Swarm Optimization and newly developed chaotic Grey Wolf Optimizer (CGWO). In order to determine the best map for CGWO, this algorithm is tested on ten benchmark problems using ten well-known chaotic maps. Simulations compare aforementioned algorithms in reproduction of two complex motion trajectories with different length and shape. Moreover, these tests include variation of population in swarm and demonstration examples. Real-world experiment on a nonholonomic mobile robot in indoor environment proves the applicability of the proposed approach.

     

Robots in the classroom: Differences in students’ perceptions of credibility and learning between “teacher as robot” and “robot as teacher”

Advancements in technology are bringing robotics into interpersonal communication contexts, including the college classroom. This study was one of the first to examine college students’ communication-related perceptions of robots being used in an instructional capacity. Student participants rated both a human instructor using a telepresence robot and an autonomous social robot delivering the same lesson as credible. However, students gave higher credibility ratings to the teacher as robot, which led to differences between the two instructional agents in their learning outcomes. Students reported more affective learning from the teacher as robot than the robot as teacher, despite controlled instructional performances. Instructional agent type had both direct and indirect effects on behavioral learning. The direct effect suggests a potential machine heuristic in which students are more likely to follow behavioral suggestions offered by an autonomous social robot. The findings generally support the MAIN model and the Computers are Social Actors paradigm, but suggest that future work needs to be done in this area.     

Interactive Teaching for Vision-Based Mobile Robots: A Sensory-Motor Approach

For the last decade, we have been developing a vision-based architecture for mobile robot navigation. Using our bio-inspired model of navigation, robots can perform sensory-motor tasks in real time in unknown indoor as well as outdoor environments. We address here the problem of autonomous incremental learning of a sensory-motor task, demonstrated by an operator guiding a robot. The proposed system allows for semisupervision of task learning and is able to adapt the environmental partitioning to the complexity of the desired behavior. A real dialogue based on actions emerges from the interactive teaching. The interaction leads the robot to autonomously build a precise sensory-motor dynamics that approximates the behavior of the teacher. The usability of the system is highlighted by experiments on real robots, in both indoor and outdoor environments. Accuracy measures are also proposed in order to evaluate the learned behavior as compared to the expected behavioral attractor. These measures, used first in a real experiment and then in a simulated experiment, demonstrate how a real interaction between the teacher and the robot influences the learning process.     

Balancing task allocation in multi-robot systems using K-means clustering and auction based mechanisms

This paper aims to solve the balanced multi-robot task allocation problem. Multi-robot systems are becoming more and more significant in industrial, commercial and scientific applications. Effectively allocating tasks to multi-robots i.e. utilizing all robots in a cost effective manner becomes a tedious process. The current attempts made by the researchers concentrate only on minimizing the distance between the robots and the tasks, and not much importance is given to the balancing of work loads among robots. It is also found from the literature that the multi-robot system is analogous to Multiple Travelling Salesman Problem (MTSP). This paper attempts to develop mechanism to address the above two issues with objective of minimizing the distance travelled by ‘m’ robots and balancing the work load between ‘m’ robots equally. The proposed approach has two fold, first develops a mathematical model for balanced multi-robot task allocation problem, and secondly proposes a methodology to solve the model in three stages. Stage I groups the ‘N’ tasks into ‘n’ clusters of tasks using K-means clustering technique with the objective of minimizing the distance between the tasks, stage II calculates the travel cost of robot and clusters combination, stage III allocates the robot to the clusters in order to utilise all robot in a cost effective manner.     

Multivariate evaluation of interactive robot systems

In designing robot systems for human interaction, designers draw on aspects of human behavior that help them achieve specific design goals. For instance, the designer of an educational robot system may use speech, gaze, and gesture cues in a way that enhances its student’s learning. But what set of behaviors improve such outcomes? How might designers of such a robot system determine this set of behaviors? Conventional approaches to answering such questions primarily involve designers carrying out a series of experiments in which they manipulate a small number of design variables and measure the effects of these manipulations on specific interaction outcomes. However, these methods become infeasible when the design space is large and when the designer needs to understand the extent to which each variable contributes to achieving the desired effects. In this paper, we present a novel multivariate method for evaluating what behaviors of interactive robot systems improve interaction outcomes. We illustrate the use of this method in a case study in which we explore how different types of narrative gestures of a storytelling robot improve its users’ recall of the robot’s story, their ability to retell the robot’s story, their perceptions of and rapport with the robot, and their overall engagement in the experiment.     

Active object tracking using context estimation: handling occlusions and detecting missing targets

When performing visual servoing or object tracking tasks, active sensor planning is essential to keep targets in sight or to relocate them when missing. In particular, when dealing with a known target missing from the sensor’s field of view, we propose using prior knowledge related to contextual information to estimate its possible location. To this end, this study proposes a Dynamic Bayesian Network that uses contextual information to effectively search for targets. Monte Carlo particle filtering is employed to approximate the posterior probability of the target’s state, from which uncertainty is defined. We define the robot’s utility function via information theoretic formalism as seeking the optimal action which reduces uncertainty of a task, prompting robot agents to investigate the location where the target most likely might exist. Using a context state model, we design the agent’s high-level decision framework using a Partially-Observable Markov Decision Process. Based on the estimated belief state of the context via sequential observations, the robot’s navigation actions are determined to conduct exploratory and detection tasks. By using this multi-modal context model, our agent can effectively handle basic dynamic events, such as obstruction of targets or their absence from the field of view. We implement and demonstrate these capabilities on a mobile robot in real-time.

     

Showing a model's eye movements in examples does not improve learning of problem-solving tasks

Eye movement modeling examples (EMME) are demonstrations of a computer-based task by a human model (e.g., a teacher), with the model's eye movements superimposed on the task to guide learners' attention. EMME have been shown to enhance learning of perceptual classification tasks; however, it is an open question whether EMME would also improve learning of procedural problem-solving tasks. We investigated this question in two experiments. In Experiment 1 (72 university students, M_age = 19.94), the effectiveness of EMME for learning simple geometry problems was addressed, in which the eye movements cued the underlying principle for calculating an angle. The only significant difference between the EMME and a no eye movement control condition was that participants in the EMME condition required less time for solving the transfer test problems. In Experiment 2 (68 university students, M_age = 21.12), we investigated the effectiveness of EMME for more complex geometry problems. Again, we found no significant effects on performance except for time spent on transfer test problems, although it was now in the opposite direction: participants who had studied EMME took longer to solve those items. These findings suggest that EMME may not be more effective than regular video examples for teaching procedural problem-solving skills.     

Students’ perceptions of creativity in learning Information Technology (IT) in project groups

This paper explores students’ perceptions of creativity in learning Information Technology (IT) in project groups and the implications of better educating creative IT students for the future. Theoretically, the extension of social psychology research into creativity lays the basis of bringing creativity, learning and IT education into one framework. Empirically, qualitative interviews were carried out with 48 students from three disciplines, including Computer Science (n = 16), Electronic Systems (n = 15) and Medialogy (n = 17) at Aalborg University (AAU) in Denmark, which has a tradition of using problem-based learning (PBL) in student project groups. According to the findings, the students’ perceptions of creativity reflect their domain-related conceptualization and tacit learning experience, with different levels of confidence of being creative persons. As IT plays multiple roles in developing students’ creativity, it can be regarded as a ‘learning partner’. This implies that in the future creativity should be taught more explicitly, helping students to become creative IT talents as a part of their professional identity. It also requires teaching efforts to build a learning environment that stimulates creativity more effectively through more interactions between learners, learning tasks and learning tools.     

Deep imitation learning for 3D navigation tasks

Deep learning techniques have shown success in learning from raw high-dimensional data in various applications. While deep reinforcement learning is recently gaining popularity as a method to train intelligent agents, utilizing deep learning in imitation learning has been scarcely explored. Imitation learning can be an efficient method to teach intelligent agents by providing a set of demonstrations to learn from. However, generalizing to situations that are not represented in the demonstrations can be challenging, especially in 3D environments. In this paper, we propose a deep imitation learning method to learn navigation tasks from demonstrations in a 3D environment. The supervised policy is refined using active learning in order to generalize to unseen situations. This approach is compared to two popular deep reinforcement learning techniques: deep-Q-networks and Asynchronous actor-critic (A3C). The proposed method as well as the reinforcement learning methods employ deep convolutional neural networks and learn directly from raw visual input. Methods for combining learning from demonstrations and experience are also investigated. This combination aims to join the generalization ability of learning by experience with the efficiency of learning by imitation. The proposed methods are evaluated on 4 navigation tasks in a 3D simulated environment. Navigation tasks are a typical problem that is relevant to many real applications. They pose the challenge of requiring demonstrations of long trajectories to reach the target and only providing delayed rewards (usually terminal) to the agent. The experiments show that the proposed method can successfully learn navigation tasks from raw visual input while learning from experience methods fail to learn an effective policy. Moreover, it is shown that active learning can significantly improve the performance of the initially learned policy using a small number of active samples.

     

Recognition, prediction, and planning for assisted teleoperation of freeform tasks

The approach of inferring user’s intended task and optimizing low-level robot motions has promise for making robot teleoperation interfaces more intuitive and responsive. But most existing methods assume a finite set of candidate tasks, which limits a robot’s functionality. This paper proposes the notion of freeform tasks that encode an infinite number of possible goals (e.g., desired target positions) within a finite set of types (e.g., reach, orient, pick up). It also presents two technical contributions to help make freeform UIs possible. First, an intent predictor estimates the user’s desired task, and accepts freeform tasks that include both discrete types and continuous parameters. Second, a cooperative motion planner continuously updates the robot’s trajectories to achieve the inferred tasks by repeatedly solving optimal control problems. The planner is designed to respond interactively to changes in the indicated task, avoid collisions in cluttered environments, handle time-varying objective functions, and achieve high-quality motions using a hybrid of numerical and sampling-based techniques. The system is applied to the problem of controlling a 6D robot manipulator using 2D mouse input in the context of two tasks: static target reaching and dynamic trajectory tracking. Simulations suggest that it enables the robot to reach intended targets faster and to track intended trajectories more closely than comparable techniques.     

Recognizing environments from action sequences using self-organizing maps

In this paper, we describe development of a mobile robot which does unsupervised learning for recognizing an environment from action sequences. We call this novel recognition approach action-based environment modeling (AEM). Most studies on recognizing an environment have tried to build precise geometric maps with high sensitive and global sensors. However such precise and global information may be hardly obtained in a real environment, and may be unnecessary to recognize an environment. Furthermore unsupervised-learning is necessary for recognition in an unknown environment without help of a teacher. Thus we attempt to build a mobile robot which does unsupervised-learning to recognize environments with low sensitive and local sensors. The mobile robot is behavior-based and does wall-following in enclosures (called rooms). Then the sequences of actions executed in each room are transformed into environment vectors for self-organizing maps. Learning without a teacher is done, and the robot becomes able to identify rooms. Moreover, we develop a method to identify environments independent of a start point using a partial sequence. We have fully implemented the system with a real mobile robot, and made experiments for evaluating the ability. As a result, we found out that the environment recognition was done well and our method was adaptive to noisy environments.     

Incremental learning of tasks from user demonstrations, past experiences, and vocal comments.

Since many years the robotics community is envisioning robot assistants sharing the same environment with humans. It became obvious that they have to interact with humans and should adapt to individual user needs. Especially the high variety of tasks robot assistants will be facing requires a highly adaptive and user-friendly programming interface. One possible solution to this programming problem is the learning-by-demonstration paradigm, where the robot is supposed to observe the execution of a task, acquire task knowledge, and reproduce it. In this paper, a system to record, interpret, and reason over demonstrations of household tasks is presented. The focus is on the model-based representation of manipulation tasks, which serves as a basis for incremental reasoning over the acquired task knowledge. The aim of the reasoning is to condense and interconnect the data, resulting in more general task knowledge. A measure for the assessment of information content of task features is introduced. This measure for the relevance of certain features relies both on general background knowledge as well as task-specific knowledge gathered from the user demonstrations. Beside the autonomous information estimation of features, speech comments during the execution, pointing out the relevance of features are considered as well. The results of the incremental growth of the task knowledge when more task demonstrations become available and their fusion with relevance information gained from speech comments is demonstrated within the task of laying a table.     

A passive robot controller aiding human coaching for kinematic behavior modifications

Precise programming of robots for industrial tasks is inflexible to variations and time-consuming. Teaching a kinematic behavior by demonstration and encoding it with dynamical systems that are robust with respect to perturbations, is proposed in order to address this issue. Given a kinematic behavior encoded by Dynamic Movement Primitives (DMP), this work proposes a passive control scheme for assisting kinesthetic modifications of the learned behavior in task variations. It employs the utilization of penetrable spherical Virtual Fixtures (VFs) around the DMP’s virtual evolution that follows the teacher’s motion. The controller enables the user to haptically ‘inspect’ the spatial properties of the learned behavior in SE(3) and significantly modify it at any required segment, while facilitating the following of already learned segments. A demonstration within the VFs could signify that the kinematic behavior is taught correctly and could lead to autonomous execution, with the DMP generating the newly learned reference commands. The proposed control scheme is theoretically proved to be passive and experimentally validated with a KUKA LWR4+ robot. Results are compared with the case of using a gravity compensated robot agnostic of the previously learned task. It is shown that the time duration of teaching and the user’s cognitive load are reduced.     

Teaching robots to cooperate with humans in dynamic manipulation tasks based on multi-modal human-in-the-loop approach

We propose an approach to efficiently teach robots how to perform dynamic manipulation tasks in cooperation with a human partner. The approach utilises human sensorimotor learning ability where the human tutor controls the robot through a multi-modal interface to make it perform the desired task. During the tutoring, the robot simultaneously learns the action policy of the tutor and through time gains full autonomy. We demonstrate our approach by an experiment where we taught a robot how to perform a wood sawing task with a human partner using a two-person cross-cut saw. The challenge of this experiment is that it requires precise coordination of the robot’s motion and compliance according to the partner’s actions. To transfer the sawing skill from the tutor to the robot we used Locally Weighted Regression for trajectory generalisation, and adaptive oscillators for adaptation of the robot to the partner’s motion.