Multi robot collision avoidance in a shared workspace

This paper presents a decentralised human-aware navigation algorithm for shared human–robot work-spaces based on the velocity obstacles paradigm. By extending our previous work on collision avoidance, we are able to include and avoid static and dynamic obstacles, no matter whether they are induced by other robots and humans passing through. Using various cost maps and Monte Carlo sampling with different cost factors accounting for humans and robots, the approach allows human workers to use the same navigation space as robots. It does not rely on any external positioning sensors and shows its feasibility even in densely packed environments.     

Inverse reinforcement learning-based time-dependent A* planner for human-aware robot navigation with local vision

In an environment where robots coexist with humans, mobile robots should be human-aware and comply with humans' behavioural norms so as to not disturb humans' personal space and activities. In this work, we propose an inverse reinforcement learning-based time-dependent A* planner for human-aware robot navigation with local vision. In this method, the planning process of time-dependent A* is regarded as a Markov decision process and the cost function of the time-dependent A* is learned using the inverse reinforcement learning via capturing humans' demonstration trajectories. With this method, a robot can plan a path that complies with humans' behaviour patterns and the robot's kinematics. When constructing feature vectors of the cost function, considering the local vision characteristics, we propose a visual coverage feature for enabling robots to learn from how humans move in a limited visual field. The effectiveness of the proposed method has been validated by experiments in real-world scenarios: using this approach robots can effectively mimic human motion patterns when avoiding pedestrians; furthermore, in a limited visual field, robots can learn to choose a path that enables them to have the larger visual coverage which shows a better navigation performance.     

Social and collaborative aspects of interaction with a service robot

To an increasing extent, robots are being designed to become a part of the lives of ordinary people. This calls for new models of the interaction between humans and robots, taking advantage of human social and communicative skills. Furthermore, human–robot relationships must be understood in the context of use of robots, and based on empirical studies of humans and robots in real settings. This paper discusses social aspects of interaction with a service robot, departing from our experiences of designing a fetch-and-carry robot for motion-impaired users in an office environment. We present the motivations behind the design of the Cero robot, especially its communication paradigm. Finally, we discuss experiences from a recent usage study, and research issues emerging from this work. A conclusion is that addressing only the primary user in service robotics is unsatisfactory, and that the focus should be on the setting, activities and social interactions of the group of people where the robot is to be used.     

Telenoid android robot as an embodied perceptual social regulation medium engaging natural human–humanoid interaction

The present paper aims to validate our research on human–humanoid interaction (HHI) using the minimalist humanoid robot Telenoid. We conducted the human–robot interaction test with 142 young people who had no prior interaction experience with this robot. The main goal is the analysis of the two social dimensions (“Perception” and “Believability”) useful for increasing the natural behaviour between users and Telenoid. We administered our custom questionnaire to human subjects in association with a well defined experimental setting (“ordinary and goal-guided task”). A thorough analysis of the questionnaires has been carried out and reliability and internal consistency in correlation between the multiple items has been calculated. Our experimental results show that the perceptual behaviour and believability, as implicit social competences, could improve the meaningfulness and the natural-like sense of human–humanoid interaction in everyday life task-driven activities. Telenoid is perceived as an autonomous cooperative agent for a shared environment by human beings.     

Detecting anomalies and de-noising monitoring data from sensors: A smart data approach

When monitoring safety levels in deep pit foundations using sensors, anomalies (e.g., highly correlated variables) and noise (e.g., high dimensionality) exist in the extracted time series data, impacting the ability to assess risks. Our research aims to address the following question: How can we detect anomalies and de-noise monitoring data from sensors in real time to improve its quality and use it to assess geotechnical safety risks? In addressing this research question, we develop a hybrid smart data approach that integrates Extended Isolation Forest and Variational Mode Decomposition models to detect anomalies and de-noise data effectively. We use real-life data obtained from sensors to validate our smart data approach while constructing a deep pit foundation. Our smart data approach can detect anomalies with a root mean square error and signal-to-noise ratio of 0.0389 and 24.09, respectively. To this end, our smart data approach can effectively pre-process data enabling improved decision-making and the management of safety risks.     

Development of an indoor navigation system for a monocular-vision-based autonomous mobile robot

We have developed a technology for a robot that uses an indoor navigation system based on visual methods to provide the required autonomy. For robots to run autonomously, it is extremely important that they are able to recognize the surrounding environment and their current location. Because it was not necessary to use plural external world sensors, we built a navigation system in our test environment that reduced the burden of information processing mainly by using sight information from a monocular camera. In addition, we used only natural landmarks such as walls, because we assumed that the environment was a human one. In this article we discuss and explain two modules: a self-position recognition system and an obstacle recognition system. In both systems, the recognition is based on image processing of the sight information provided by the robot’s camera. In addition, in order to provide autonomy for the robot, we use an encoder and information from a two-dimensional space map given beforehand. Here, we explain the navigation system that integrates these two modules. We applied this system to a robot in an indoor environment and evaluated its performance, and in a discussion of our experimental results we consider the resulting problems.     

Robot navigation in a known environment with unknown moving obstacles

We propose a new type of artificial potential field, that we call hybrid potential field, to navigate a robot in situations in which the environment is known except for unknown and possibly moving obstacles. We show how to compute hybrid potential fields in real time and use them to control the motions of a real robot. Our method is tested on both a real robot and a simulated one. We present a feature matching approach for position error correction that we have validated experimentally with our mobile robot. We show extensive simulation results with up to 50 randomly moving obstacles.     

Some aspects of dynamic 3D representation and control of industrial processes via the Internet

In our research we examine and use 3D representation of industrial processes, for example the novel methods of Incremental Sheet Forming. We also test 3D imaging methods on our industrial robot solving the Rubik's Cube. We have created 3D models of our robots and their environment in our laboratory to examine the behavior of different industrial processes both in the real, and in the 3D virtual environment. We have connected the 3D model with the real system with which we could extend the features of our robots with some services that exits in the virtual space. We have also established synchronized connections of the real and virtual systems, which enables us to control the real robots and machines from its 3D model via the Internet.     

Recognizing environments from action sequences using self-organizing maps

In this paper, we describe development of a mobile robot which does unsupervised learning for recognizing an environment from action sequences. We call this novel recognition approach action-based environment modeling (AEM). Most studies on recognizing an environment have tried to build precise geometric maps with high sensitive and global sensors. However such precise and global information may be hardly obtained in a real environment, and may be unnecessary to recognize an environment. Furthermore unsupervised-learning is necessary for recognition in an unknown environment without help of a teacher. Thus we attempt to build a mobile robot which does unsupervised-learning to recognize environments with low sensitive and local sensors. The mobile robot is behavior-based and does wall-following in enclosures (called rooms). Then the sequences of actions executed in each room are transformed into environment vectors for self-organizing maps. Learning without a teacher is done, and the robot becomes able to identify rooms. Moreover, we develop a method to identify environments independent of a start point using a partial sequence. We have fully implemented the system with a real mobile robot, and made experiments for evaluating the ability. As a result, we found out that the environment recognition was done well and our method was adaptive to noisy environments.     

Coupled dynamical system based arm–hand grasping model for learning fast adaptation strategies

Performing manipulation tasks interactively in real environments requires a high degree of accuracy and stability. At the same time, when one cannot assume a fully deterministic and static environment, one must endow the robot with the ability to react rapidly to sudden changes in the environment. These considerations make the task of reach and grasp difficult to deal with. We follow a Programming by Demonstration (PbD) approach to the problem and take inspiration from the way humans adapt their reach and grasp motions when perturbed. This is in sharp contrast to previous work in PbD that uses unperturbed motions for training the system and then applies perturbation solely during the testing phase. In this work, we record the kinematics of arm and fingers of human subjects during unperturbed and perturbed reach and grasp motions. In the perturbed demonstrations, the target’s location is changed suddenly after the onset of the motion. Data show a strong coupling between the hand transport and finger motions. We hypothesize that this coupling enables the subject to seamlessly and rapidly adapt the finger motion in coordination with the hand posture. To endow our robot with this competence, we develop a coupled dynamical system based controller, whereby two dynamical systems driving the hand and finger motions are coupled. This offers a compact encoding for reach-to-grasp motions that ensures fast adaptation with zero latency for re-planning. We show in simulation and on the real iCub robot that this coupling ensures smooth and “human-like” motions. We demonstrate the performance of our model under spatial, temporal and grasp type perturbations which show that reaching the target with coordinated hand–arm motion is necessary for the success of the task.     

Robot semantic mapping through human activity recognition: A wearable sensing and computing approach

Semantic information can help robots understand unknown environments better. In order to obtain semantic information efficiently and link it to a metric map, we present a new robot semantic mapping approach through human activity recognition in a human–robot coexisting environment. An intelligent mobile robot platform called ASCCbot creates a metric map while wearable motion sensors attached to the human body are used to recognize human activities. Combining pre-learned models of activity–furniture correlation and location–furniture correlation, the robot determines the probability distribution of the furniture types through a Bayesian framework and labels them on the metric map. Computer simulations and real experiments demonstrate that the proposed approach is able to create a semantic map of an indoor environment effectively.     

An application of Lyapunov stability analysis to improve the performance of NARMAX models

Previously we presented a novel approach to program a robot controller based on system identification and robot training techniques. The proposed method works in two stages: first, the programmer demonstrates the desired behaviour to the robot by driving it manually in the target environment. During this run, the sensory perception and the desired velocity commands of the robot are logged. Having thus obtained training data we model the relationship between sensory readings and the motor commands of the robot using ARMAX/NARMAX models and system identification techniques. These produce linear or non-linear polynomials which can be formally analysed, as well as used in place of “traditional robot” control code.In this paper we focus our attention on how the mathematical analysis of NARMAX models can be used to understand the robot’s control actions, to formulate hypotheses and to improve the robot’s behaviour. One main objective behind this approach is to avoid trial-and-error refinement of robot code. Instead, we seek to obtain a reliable design process, where program design decisions are based on the mathematical analysis of the model describing how the robot interacts with its environment to achieve the desired behaviour. We demonstrate this procedure through the analysis of a particular task in mobile robotics: door traversal.     

Maximizing Reward in a Non-Stationary Mobile Robot Environment

The ability of a robot to improve its performance on a task can be critical, especially in poorly known and non-stationary environments where the best action or strategy is dependent upon the current state of the environment. In such systems, a good estimate of the current state of the environment is key to establishing high performance, however quantified. In this paper, we present an approach to state estimation in poorly known and non-stationary mobile robot environments, focusing on its application to a mine collection scenario, where performance is quantified using reward maximization. The approach is based on the use of augmented Markov models (AMMs), a sub-class of semi-Markov processes. We have developed an algorithm for incrementally constructing arbitrary-order AMMs on-line. It is used to capture the interaction dynamics between a robot and its environment in terms of behavior sequences executed during the performance of a task. For the purposes of reward maximization in a non-stationary environment, multiple AMMs monitor events at different timescales and provide statistics used to select the AMM likely to have a good estimate of the environmental state. AMMs with redundant or outdated information are discarded, while attempting to maintain sufficient data to reduce conformation to noise. This approach has been successfully implemented on a mobile robot performing a mine collection task. In the context of this task, we first present experimental results validating our reward maximization performance criterion. We then incorporate our algorithm for state estimation using multiple AMMs, allowing the robot to select appropriate actions based on the estimated state of the environment. The approach is tested first with a physical robot, in a non-stationary environment with an abrupt change, then with a simulation, in a gradually shifting environment.     

Indoor scene recognition by a mobile robot through adaptive object detection

Mobile robotics has achieved notable progress, however, to increase the complexity of the tasks that mobile robots can perform in natural environments, we need to provide them with a greater semantic understanding of their surrounding. In particular, identifying indoor scenes, such as an Office or a Kitchen, is a highly valuable perceptual ability for an indoor mobile robot, and in this paper we propose a new technique to achieve this goal. As a distinguishing feature, we use common objects, such as Doors or furniture, as a key intermediate representation to recognize indoor scenes. We frame our method as a generative probabilistic hierarchical model, where we use object category classifiers to associate low-level visual features to objects, and contextual relations to associate objects to scenes. The inherent semantic interpretation of common objects allows us to use rich sources of online data to populate the probabilistic terms of our model. In contrast to alternative computer vision based methods, we boost performance by exploiting the embedded and dynamic nature of a mobile robot. In particular, we increase detection accuracy and efficiency by using a 3D range sensor that allows us to implement a focus of attention mechanism based on geometric and structural information. Furthermore, we use concepts from information theory to propose an adaptive scheme that limits computational load by selectively guiding the search for informative objects. The operation of this scheme is facilitated by the dynamic nature of a mobile robot that is constantly changing its field of view. We test our approach using real data captured by a mobile robot navigating in Office and home environments. Our results indicate that the proposed approach outperforms several state-of-the-art techniques for scene recognition.     

Reversing the operating system of a Java based smart card

Attacks on smart cards can only be based on a black box approach where the code of cryptographic primitives and operating system are not accessible. To perform hardware or software attacks, a white box approach providing access to the binary code is more efficient. In this paper, we propose a methodology to discover the romized code whose access is protected by the virtual machine. It uses a hooked code in an indirection table. We gained access to the real processor, thus allowing us to run a shell code written in 8051 assembly language. As a result, this code has been able to dump completely the ROM of a Java Card operating system. One of the issues is the possibility to reverse the cryptographic algorithm and all the embedded countermeasures. Finally, our attack is evaluated on different cards from distinct manufacturers.     

Octopus: An Upperware based system for building personal pervasive environments

As of today, there is no operating system suitable for pervasive computing. Such system must integrate and coordinate heterogeneous devices and systems but, at the same time, it should provide a single system image to let the user feel that there is only a single “pervasive” computing environment. Such illusion must consider the Internet as the system backbone, because users move. The challenge is providing a novel system while permitting the seamless integration of traditional legacy systems, which may be required to run on many computers and devices, if only to run their applications. We argue that to build such a system, we should abandon Middleware and use a different technology, that we call Upperware. To back up our claim, we have built an actual system using Upperware: the Octopus. The Octopus has been in use for several years both to build pervasive applications like smart spaces and to provide a general-purpose computing environment. We have been using it through wide area networks, on a daily basis. In this paper we discuss the Upperware approach and present the Octopus as an actual system built out of Upperware, including some evaluation results.     

Geographical information systems for map based navigation in urban environments

In order to solve most of the existing mobile robotics applications, the robot needs some information about its spatial environment encoded in what it has been commonly called a map. The knowledge contained in such a map, whatever approach is used to obtain it, will mainly be used by the robot to gain the ability to navigate in a given environment. We are describing in this paper, a method that allows a robot or team of robots to navigate in large urban areas for which an existing map in a standard human understandable fashion is available. As detailed maps of most urban areas already exist, it will be assumed that a map of the zone where the robot is supposed to work is given, which has not been constructed using the robot’s own sensors. We propose in this paper, the use of an existing Geographical Information System based map of an urban zone so that a robot or a team of robots can connect to this map and use it for navigation purposes. Details of the implemented system architecture as well as a position tracking experiment in a real outdoor environment, a University Campus, are provided.     

Human-aware robot navigation: A survey

Navigation is a basic skill for autonomous robots. In the last years human–robot interaction has become an important research field that spans all of the robot capabilities including perception, reasoning, learning, manipulation and navigation. For navigation, the presence of humans requires novel approaches that take into account the constraints of human comfort as well as social rules. Besides these constraints, putting robots among humans opens new interaction possibilities for robots, also for navigation tasks, such as robot guides. This paper provides a survey of existing approaches to human-aware navigation and offers a general classification scheme for the presented methods.     

Human detection using a mobile platform and novel features derived from a visual saliency mechanism

Human detection is a key ability to an increasing number of applications that operates in human inhabited environments or needs to interact with a human user. Currently, most successful approaches to human detection are based on background substraction techniques that apply only to the case of static cameras or cameras with highly constrained motions. Furthermore, many applications rely on features derived from specific human poses, such as systems based on features derived from the human face which is only visible when a person is facing the detecting camera. In this work, we present a new computer vision algorithm designed to operate with moving cameras and to detect humans in different poses under partial or complete view of the human body. We follow a standard pattern recognition approach based on four main steps: (i) preprocessing to achieve color constancy and stereo pair calibration, (ii) segmentation using depth continuity information, (iii) feature extraction based on visual saliency, and (iv) classification using a neural network. The main novelty of our approach lies in the feature extraction step, where we propose novel features derived from a visual saliency mechanism. In contrast to previous works, we do not use a pyramidal decomposition to run the saliency algorithm, but we implement this at the original image resolution using the so-called integral image. Our results indicate that our method: (i) outperforms state-of-the-art techniques for human detection based on face detectors, (ii) outperforms state-of-the-art techniques for complete human body detection based on different set of visual features, and (iii) operates in real time onboard a mobile platform, such as a mobile robot (15 fps).