Human-aware planning for robots embedded in ambient ecologies

We address the issue of human–robot cohabitation in smart environments. In particular, the presence of humans in a robot’s work space has a profound influence on how the latter should plan its actions. We propose the use of human-aware planning, an approach in which the robot exploits the capabilities of a sensor-rich environment to obtain information about the (current and future) activities of the people in the environment, and plans its tasks accordingly.Here, we formally describe the planning problem behind our approach, we analyze its complexity and we detail the algorithm of our planner. We then show two application scenarios that could benefit from the techniques described. The first scenario illustrates the applicability of human-aware planning in a domestic setting, while the second one illustrates its use for a robotic helper in a hospital. Finally, we present a five hour-long test run in a smart home equipped with real sensors, where a cleaning robot has been deployed and where a human subject is acting. This test run in a real setting is meant to demonstrate the feasibility of our approach to human–robot interaction.     

Using perspective taking to learn from ambiguous demonstrations

This paper addresses an important issue in learning from demonstrations that are provided by “naïve” human teachers—people who do not have expertise in the machine learning algorithms used by the robot. We therefore entertain the possibility that, whereas the average human user may provide sensible demonstrations from a human’s perspective, these same demonstrations may be insufficient, incomplete, ambiguous, or otherwise “flawed” from the perspective of the training set needed by the learning algorithm to generalize properly. To address this issue, we present a system where the robot is modeled as a socially engaged and socially cognitive learner. We illustrate the merits of this approach through an example where the robot is able to correctly learn from “flawed” demonstrations by taking the visual perspective of the human instructor to clarify potential ambiguities.     

Unfolding of a rectangular cloth from unarranged starting shapes by a Dual-Armed robot with a mechanism for managing recognition error and uncertainty

We propose a method for unfolding a rectangular cloth placed on a table in an arbitrary unarranged shape, using a dual arm robot. There are many situations where the manipulation of fabric products by dual arm robots is slow due to operation complexity. Also, observation of fabric products in unarranged shapes can be fraught with uncertainty, posing further difficulties for robotic manipulation. In this article, we address these problems for our specific task, implementing a ‘pinch and slide motion’ to address the former issue, and an operation selection mechanism implemented as a partially observable Markov decision process to address the latter. We used this approach to let a robot unfold a rectangular cloth, thereby experimentally verifying the effectiveness of our approach.     

Physics-based simulation for manual robot guidance—An eRobotics approach

Manual robot guidance is an intuitive approach to teach robots with human's skills in the loop. It is particularly useful to manufacturers because of its high flexibility and low programming effort. However, manual robot guidance requires compliance control that is generally not available in position-controlled industrial robots. We address this issue from a simulation-driven approach. We systematically capture the interactive dynamic behavior of intelligent robot manipulators within physics-based virtual testbeds, regardless of the type of application. On this basis, we develop structures to equip and employ simulated robots with motion control capabilities that include soft physical interaction control driven in real-time with real external guidance forces. We then transfer the virtual compliant behavior of the simulated robots to their physical counterparts to enable manual guidance. The simulator provides assistance to operators through timely and insightful robot monitoring, as well as meaningful performance indexes. The testbed allows us to swiftly assess guidance within numerous interaction scenarios. Experimental case studies illustrate the practical usefulness of the symbiotic transition between 3D simulation and reality, as pursued by the eRobotics framework to address challenging issues in industrial automation.     

Learning to Acquire Whole-Body Humanoid Center of Mass Movements to Achieve Dynamic Tasks

This paper presents a novel approach for acquiring dynamic whole-body movements on humanoid robots focused on learning a control policy for the center of mass (CoM). In our approach, we combine both a model-based CoM controller and a model-free reinforcement learning (RL) method to acquire dynamic whole-body movements in humanoid robots. (i) To cope with high dimensionality, we use a model-based CoM controller as a basic controller that derives joint angular velocities from the desired CoM velocity. The balancing issue can also be considered in the controller. (ii) The RL method is used to acquire a controller that generates the desired CoM velocity based on the current state. To demonstrate the effectiveness of our approach, we apply it to a ball-punching task on a simulated humanoid robot model. The acquired whole-body punching movement was also demonstrated on Fujitsu's Hoap-2 humanoid robot.     

An AR-assisted Deep Reinforcement Learning-based approach towards mutual-cognitive safe human-robot interaction

With the emergence of Industry 5.0, the human-centric manufacturing paradigm requires manufacturing equipment (robots, etc.) interactively assist human workers to deal with dynamic and complex production tasks. To achieve symbiotic human–robot interaction (HRI), the safety issue serves as a prerequisite foundation. Regarding the growing individualized demand of manufacturing tasks, the conventional rule-based safe HRI measures could not well address the safety requirements due to inflexibility and lacking synergy. To fill the gap, this work proposes a mutual-cognitive safe HRI approach including worker visual augmentation, robot velocity control, Digital Twin-enabled motion preview and collision detection, and Deep Reinforcement Learning-based robot collision avoidance motion planning in the Augmented Reality-assisted manner. Finally, the feasibility of the system design and the performance of the proposed approach are validated by establishing and executing the prototype HRI system in a practical scene.     

Development and applications of rescue robots for explosion accidents in coal mines

It is essential to provide disaster relief assistance after coal mine explosions. Often, it is life‐threatening for rescuers to enter an accident scene blindly; therefore, a coal mine rescue robot (CMRR) has been developed. However, the application of the CMRR has not proven satisfactory after decades of development. To solve this problem, we summarize the reasons for this disappointing state and address the technical challenges of the CMRR. Based on these reasons and the associated technical challenges, two generations of tracked robots have been developed. The China University of Mining Technology‐V (CUMT‐V) (A) robot was first developed and its walking system, body support system, communication system, environmental awareness system, and control system are described in detail. A performance test was performed on the CUMT‐V (A) robot and some problems were encountered. To address these problems, we designed the CUMT‐V (B) robot. The field test was conducted in Shanxi province, China, in August 2016. The application results show that the robot has good adaptability to complex terrain and high reliability in terms of environmental awareness and data transmission. In conclusion, the robot is nearing practical applications.     

A Bayesian exploration-exploitation approach for optimal online sensing and planning with a visually guided mobile robot

We address the problem of online path planning for optimal sensing with a mobile robot. The objective of the robot is to learn the most about its pose and the environment given time constraints. We use a POMDP with a utility function that depends on the belief state to model the finite horizon planning problem. We replan as the robot progresses throughout the environment. The POMDP is high-dimensional, continuous, non-differentiable, nonlinear, non-Gaussian and must be solved in real-time. Most existing techniques for stochastic planning and reinforcement learning are therefore inapplicable. To solve this extremely complex problem, we propose a Bayesian optimization method that dynamically trades off exploration (minimizing uncertainty in unknown parts of the policy space) and exploitation (capitalizing on the current best solution). We demonstrate our approach with a visually-guide mobile robot. The solution proposed here is also applicable to other closely-related domains, including active vision, sequential experimental design, dynamic sensing and calibration with mobile sensors.     

基于缓存策略的Java卡字节码校验

作为一种不可靠的可下载Java程序， Java卡的字节码校验是安全的嵌入式系统所不可缺少的一个部分。由于Java卡本身的空间和运算器的限制，传统的字节码校验并不可取。采用非易失性存储器作为主存， RAM中的一部分空间作为缓存器，并且给出一种相应的Cache调度策略算法，尝试实现一种在卡上的Java卡的字节码校验算法，并且证明了该算法具有较好的可移植性和实现性。     

基于视觉和里程计信息融合的移动机器人自定位

受鸽子定向启发，将装备有全维视觉和里程计等传感器的自主移动机器人的自定位分为两种模式：全维视觉定位模式和里程计定位模式．机器人依据一定准则选择具体的主导定位模式：先试视觉定位，若视觉定位不可得或获得的视觉定位不可靠，则采用里程计定位．针对标记物信息失真问题，应用初步视觉定位结果反推标记物理论值，然后通过比较从原始图像中分离出的可能的标记物信息和反推出来的标记物信息理论值，滤除不可靠的视觉定位．针对运动过程中的机器人自定位，分析了影响定位准确性的信息时间延迟因素．     

Error management for robot programming

Reliability is a serious problem in computer controlled robot systems. Although robots serve successfully in relatively simple applications such as painting and spot welding, their potential in areas such as automated assembly is hampered by the complexity of programming. A program for assembling parts may be logically correct, execute correctly on a simulator, and even execute correctly on a robot most of the time, yet still fail unexpectedly in the face of real world uncertainties. Recovery from such errors is far more complicated than recovery from simple controller errors, since even expected errors can manifest themselves in unexpected ways. In this paper we present a novel approach for improving robot reliability. Instead of anticipating errors, we use knowledge-based programming techniques so that the robot can autonomously exploit knowledge about its task and environment to detect and recover from failures. We describe a system that we have designed and constructed in our robotics laboratory.     

Robot control code generation by task demonstration in a dynamic environment

Within mobile robotics, one of the most dominant relationships to consider when implementing robot control code is the one between the robot’s sensors and its motors. When implementing such a relationship, efficiency and reliability are of crucial importance. The latter aspects often prove challenging due to the complex interaction between a robot and the environment in which it exists, frequently resulting in a time consuming iterative process where control code is redeveloped and tested many times before obtaining an optimal controller. In this paper, we address this challenge by implementing an alternative approach to control code generation, which first identifies the desired robot behaviour and represents the sensor-motor task algorithmically through system identification using the NARMAX modelling methodology. The control code is generated by task demonstration, where the sensory perception and velocities are logged and the relationship that exists between them is then modelled using system identification. This approach produces transparent control code through non-linear polynomial equations that can be mathematically analysed to obtain formal statements regarding specific inputs/outputs. We demonstrate this approach to control code generation and analyse its performance in dynamic environments.     

Evaluating robustness in a two layer simulated robot architecture

Many two layer robot architectures have been proposed and implemented. While justification for the design can be well argued, how does one know it is really a good idea? In this paper, one describes a two layer architecture (reinforcement learning in the bottom layer and POMDP planning at the top) for a simulated robot and summarize a set of three experiments in which one evaluated the design. To address the many difficulties of evaluating robot architectures, one advocates an experimental approach in which design criteria are elucidated and then form the basis for the evaluation experiments. In our case, one tests the implementation for its reliability and generalization (our design criteria) by comparing our architecture to one in which a key component is substituted; in these experiments, one demonstrates significant performance gains on the design criteria for our architecture.     

Goal-oriented dynamic test generation

ContextMemory safety errors such as buffer overflow vulnerabilities are one of the most serious classes of security threats. Detecting and removing such security errors are important tasks of software testing for improving the quality and reliability of software in practice.ObjectiveThis paper presents a goal-oriented testing approach for effectively and efficiently exploring security vulnerability errors. A goal is a potential safety violation and the testing approach is to automatically generate test inputs to uncover the violation.MethodWe use type inference analysis to diagnose potential safety violations and dynamic symbolic execution to perform test input generation. A major challenge facing dynamic symbolic execution in such application is the combinatorial explosion of the path space. To address this fundamental scalability issue, we employ data dependence analysis to identify a root cause leading to the execution of the goal and propose a path exploration algorithm to guide dynamic symbolic execution for effectively discovering the goal.ResultsTo evaluate the effectiveness of our proposed approach, we conducted experiments against 23 buffer overflow vulnerabilities. We observed a significant improvement of our proposed algorithm over two widely adopted search algorithms. Specifically, our algorithm discovered security vulnerability errors within a matter of a few seconds, whereas the two baseline algorithms failed even after 30 min of testing on a number of test subjects.ConclusionThe experimental results highlight the potential of utilizing data dependence analysis to address the combinatorial path space explosion issue faced by dynamic symbolic execution for effective security testing.     

Evolving behavior sequences for a humanoid entertainment robot

One of the most important issues in developing an entertainment robot is human-robot interaction, in which the robot is expected to learn new behaviors specified by the user. In this article we present an imitation-based mechanism to support robot learning, and use evolutionary computing to learn new behavior sequences. We also propose several advanced techniques at the task level and the computational level to evolve complex sequences. To evaluate our approach, we use it to evolve different behaviors for a humanoid robot. The results show the promise of our approach.     

Virtual Chassis for Snake Robots: Definition and Applications

Abstract

Intuitively representing the motion of a snake robot is difficult. This is in part because the internal shape changes that the robot uses to locomote involve the entire body and no single point on the robot intuitively represents the robot’s pose at all times. To address this issue, we present a method of defining body coordinate frames that departs from the typical convention of rigidly fixing a frame to a link on the robot, and instead define a body frame that is based on the averaged position of all of the robot’s links. This averaged frame serves as a virtual chassis that effectively isolates the internal motion of the robot’s shape changes from the external motion, due to the robot’s interaction with its surroundings. This separation of motion allows much simpler models—such as those derived for wheeled vehicles—to accurately approximate the motion of the robot as it moves through the world. We demonstrate the practical advantages of using the virtual chassis body frame by estimating the pitch and roll of a snake robot undergoing dynamic motion by fusing readings from its internal encoders, gyros, and accelerometers with an extended Kalman filter.     

Reinforcement learning in dynamic environment: abstraction of state-action space utilizing properties of the robot body and environment

In this paper, we address the autonomous control of a 3D snake-like robot through the use of reinforcement learning, and we apply it in a dynamic environment. In general, snake-like robots have high mobility that is realized by many degrees of freedom, and they can move over dynamically shifting environments such as rubble. However, this freedom and flexibility leads to a state explosion problem, and the complexity of the dynamic environment leads to incomplete learning by the robot. To solve these problems, we focus on the properties of the actual operating environment and the dynamics of a mechanical body. We design the body of the robot so that it can abstract small, but necessary state-action space by utilizing these properties, and we make it possible to apply reinforcement learning. To demonstrate the effectiveness of the proposed snake-like robot, we conduct experiments; from the experimental results we conclude that learning is completed within a reasonable time, and that effective behaviors for the robot to adapt itself to an unknown 3D dynamic environment were realized.     

Real-time humanoid whole-body remote control framework for imitating human motion based on kinematic mapping and motion constraints

In this paper, we propose a whole-body remote control framework that enables a robot to imitate human motion efficiently. The framework is divided into kinematic mapping and quadratic programming based whole-body inverse kinematics. In the kinematic mapping, the human motion obtained through a data acquisition device is transformed into a reference motion that is suitable for the robot to follow. To address differences in the kinematic configuration and dynamic properties of the robot and human, quadratic programming is used to calculate the joint angles of the robot considering self-collision, joint limits, and dynamic stability. To address dynamic stability, we use constraints based on the divergent component of motion and zero moment point in the linear inverted pendulum model. Simulation using Choreonoid and a locomotion experiment using the HUBO2+ demonstrate the performance of the proposed framework. The proposed framework has the potential to reduce the preview time or offline task computation time found in previous approaches and hence improve the similarity of human and robot motion while maintaining stability.     

Controlling swimming and crawling in a fish robot using a central pattern generator

Online trajectory generation for robots with multiple degrees of freedom is still a difficult and unsolved problem, in particular for non-steady state locomotion, that is, when the robot has to move in a complex environment with continuous variations of the speed, direction, and type of locomotor behavior. In this article we address the problem of controlling the non-steady state swimming and crawling of a novel fish robot. For this, we have designed a control architecture based on a central pattern generator (CPG) implemented as a system of coupled nonlinear oscillators. The CPG, like its biological counterpart, can produce coordinated patterns of rhythmic activity while being modulated by simple control parameters. To test our controller, we designed BoxyBot, a simple fish robot with three actuated fins capable of swimming in water and crawling on firm ground. Using the CPG model, the robot is capable of performing and switching between a variety of different locomotor behaviors such as swimming forwards, swimming backwards, turning, rolling, moving upwards/downwards, and crawling. These behaviors are triggered and modulated by sensory input provided by light, water, and touch sensors. Results are presented demonstrating the agility of the robot and interesting properties of a CPG-based control approach such as stability of the rhythmic patterns due to limit cycle behavior, and the production of smooth trajectories despite abrupt changes of control parameters. The robot is currently used in a temporary 20-month long exhibition at the EPFL. We present the hardware setup that was designed for the exhibition, and the type of interactions with the control system that allow visitors to influence the behavior of the robot. The exhibition is useful to test the robustness of the robot for long term use, and to demonstrate the suitability of the CPG-based approach for interactive control with a human in the loop. This article is an extended version of an article presented at BioRob2006 the first IEEE/RAS-EMBS International Conference on Biomedical Robotics and Biomechatronics.