A semantically-informed multirobot system for exploration of relevant areas in search and rescue settings

Coordinated multirobot exploration involves autonomous discovering and mapping of the features of initially unknown environments by using multiple robots. Autonomously exploring mobile robots are usually driven, both in selecting locations to visit and in assigning them to robots, by knowledge of the already explored portions of the environment, often represented in a metric map. In the literature, some works addressed the use of semantic knowledge in exploration, which, embedded in a semantic map, associates spatial concepts (like ‘rooms’ and ‘corridors’) with metric entities, showing its effectiveness in improving the total area explored by robots. In this paper, we build on these results and propose a system that exploits semantic information to push robots to explore relevant areas of initially unknown environments, according to a priori information provided by human users. Discovery of relevant areas is significant in some search and rescue settings, in which human rescuers can instruct robots to search for victims in specific areas, for example in cubicles if a disaster happened in an office building during working hours. We propose to speed up the exploration of specific areas by using semantic information both to select locations to visit and to determine the number of robots to allocate to those locations. In this way, for example, more robots could be assigned to a candidate location in a corridor, so the attached rooms can be explored faster. We tested our semantic-based multirobot exploration system within a reliable robot simulator and we evaluated its performance in realistic search and rescue indoor settings with respect to state-of-the-art approaches.     

The leader-follower formation control of nonholonomic mobile robots

In this paper, the leader-waypoint-follower formation is constructed based on relative motion states of nonholonomic mobile robots. Since the robots’ velocities are constrained, we proposed a geometrical waypoint in cone method so that the follower robots move to their desired waypoints effectively. In order to form and maintain the formation of multi-robots, we combine stable tracking control method with receding horizon (RH) tracking control method. The stable tracking control method aims to make the robot’s state errors stable and the RH tracking control method guarantees that the convergence of the state errors tends toward zero efficiently. Based on the methods mentioned above, the mobile robots formation can be maintained in any trajectory such as a straight line, a circle or a sinusoid. The simulation results based on the proposed approaches show each follower robot can move to its waypoint efficiently. To validate the proposed methods, we do the experiments with nonholonomic robots using only limited on-board sensor information.     

Relational Model for Robotic Semantic Navigation in Indoor Environments

The emergence of service robots in our environment raises the need to find systems that help the robots in the task of managing the information from human environments. A semantic model of the environment provides the robot with a representation closer to the human perception, and it improves its human-robot communication system. In addition, a semantic model will improve the capabilities of the robot to carry out high level navigation tasks. This paper presents a semantic relational model that includes conceptual and physical representation of objects and places, utilities of the objects, and semantic relation among objects and places. This model allows the robot to manage the environment and to make queries about the environment in order to do plans for navigation tasks. In addition, this model has several advantages such as conceptual simplicity and flexibility of adaptation to different environments. To test the performance of the proposed semantic model, the output for the semantic inference system is associate to the geometric and topological information of objects and places in order to do the navigation tasks.     

Dynamic control of coordinated redundant robots with torque optimization

The problem of controlling a system of coordinated redundant robots with torque optimization based on joint redundancy is addressed. Local and global optimal control laws, both minimizing joint torque loading, are developed. A general method of load distribution among the coordinated robots is also proposed. The control problem is to regulate the motion of the object held by the coordinated robots and the internal force generated as a result of constraints on the object. The errors in the object motion and internal force converge asymptotically to zero under the proposed optimal control laws, when exact knowledge of the dynamic models is assumed. Furthermore, the robustness of the proposed method to model uncertainty is also analyzed. The motion and internal force errors are uniformly ultimately bounded under the proposed optimal controllers, when uncertainty in the dynamic models is assumed to exist.     

移动机器人编队自修复的切换拓扑控制

针对机器人缺失后的移动机器人编队自修复问题, 构建了结合切换拓扑和交互动力模型的移动机器人编队模型, 通过分析机器人缺失后的拓扑变化情况, 提出了网络切换拓扑控制, 该算法利用递归实现自修复, 并且是收敛的. 通过设计相应的分布式算法, 本文将拓扑控制转化为基于局部交互的递归自修复个体控制, 证明了编队自修复个体控制的稳定性. 最后针对编队任务, 通过仿真验证了切换拓扑控制的有效性, 和其他方法比较具有低恢复时间和低功率消耗的优点.     

Robust joint stem detection and crop‐weed classification using image sequences for plant‐specific treatment in precision farming

Conventional farming still relies on large quantities of agrochemicals for weed management which have several negative side‐effects on the environment. Autonomous robots offer the potential to reduce the amount of chemicals applied, as robots can monitor and treat each plant in the field individually and thereby circumventing the uniform chemical treatment of the whole field. Such agricultural robots need the ability to identify individual crops and weeds in the field using sensor data and must additionally select effective treatment methods based on the type of weed. For example, certain types of weeds can only be effectively treated mechanically due to their resistance to herbicides, whereas other types can be treated trough selective spraying. In this article, we present a novel system that provides the necessary information for effective plant‐specific treatment. It estimates the stem location for weeds, which enables the robots to perform precise mechanical treatment, and at the same time provides the pixel‐accurate area covered by weeds for treatment through selective spraying. The major challenge in developing such a system is the large variability in the visual appearance that occurs in different fields. Thus, an effective classification system has to robustly handle substantial environmental changes including varying weed pressure, various weed types, different growth stages, changing visual appearance of the plants and the soil. Our approach uses an end‐to‐end trainable fully convolutional network that simultaneously estimates plant stem positions as well as the spatial extent of crop plants and weeds. It jointly learns how to detect the stems and the pixel‐wise semantic segmentation and incorporates spatial information by considering image sequences of local field strips. The jointly learned feature representation for both tasks furthermore exploits the crop arrangement information that is often present in crop fields. This information is considered even if it is only observable from the image sequences and not a single image. Such image sequences, as typically provided by robots navigating over the field along crop rows, enable our approach to robustly estimate the semantic segmentation and stem positions despite the large variations encountered in different fields. We implemented and thoroughly tested our approach on images from multiple farms in different countries. The experiments show that our system generalizes well to previously unseen fields under varying environmental conditions—a key capability to deploy such systems in the real world. Compared to state‐of‐the‐art approaches, our approach generalizes well to unseen fields and not only substantially improves the stem detection accuracy, that is, distinguishing crop and weed stems, but also improves the semantic segmentation performance.     

Analysis of Dynamic Characteristics of Water Hydraulic Rotating Angle Self-Servo Robot Joint Actuator

The application of robots as a tool to explore underwater environments has increased in the last decade. Underwater tasks such as inspection, maintenance, and monitoring can be automatized by robots. The understanding of the underwater environments and the object recognition are required features that are becoming a critical issue for these systems. On this work, a method to provide a semantic mapping on the underwater environment is provided. This novel system is independent of the water turbidity and uses acoustic images acquired by Forward-Looking Sonar (FLS). The proposed method efficiently segments and classifies the structures in the scene using geometric information of the recognized objects. Therefore, a semantic map of the scene is created, which allows the robot to describe its environment according to high-level semantic features. Finally, the proposal is evaluated in a real dataset acquired by an underwater vehicle in a marina area. Experimental results demonstrate the robustness and accuracy of the method described in this paper.     

Robot semantic mapping through human activity recognition: A wearable sensing and computing approach

Semantic information can help robots understand unknown environments better. In order to obtain semantic information efficiently and link it to a metric map, we present a new robot semantic mapping approach through human activity recognition in a human–robot coexisting environment. An intelligent mobile robot platform called ASCCbot creates a metric map while wearable motion sensors attached to the human body are used to recognize human activities. Combining pre-learned models of activity–furniture correlation and location–furniture correlation, the robot determines the probability distribution of the furniture types through a Bayesian framework and labels them on the metric map. Computer simulations and real experiments demonstrate that the proposed approach is able to create a semantic map of an indoor environment effectively.     

基于多粒度和语义信息的中文关系抽取

中文关系抽取采用基于字符或基于词的神经网络,现有的方法大多存在分词错误和歧义现象,会不可避免的引入大量冗余和噪音,从而影响关系抽取的结果.为了解决这一问题,本文提出了一种基于多粒度并结合语义信息的中文关系抽取模型.在该模型中,我们将词级别的信息合并进入字符级别的信息中,从而避免句子分割时产生错误;借助外部的语义信息对多义词进行建模,来减轻多义词所产生的歧义现象;并且采用字符级别和句子级别的双重注意力机制.实验表明,本文提出的模型能够有效提高中文关系抽取的准确率和召回率,与其他基线模型相比,具有更好的优越性和可解释性.     

Robust robot knowledge instantiation for intelligent service robots

Robot knowledge is considered to endow service robots with intelligence. In the real environments, robot knowledge needs to represent dynamically changing world. Despite its advantages for semantic knowledge of service robots, robot knowledge may be instantiated and updated by using imperfect sensing data, such as misidentification of object recognition. In case of using commercially available visual recognition system, incorrect knowledge instances are created and changed frequently due to object misidentification and/or recognition failures. In this work, a robust semantic knowledge handling method under imperfect object recognition is proposed to instantiate and update robot knowledge with logical inference by estimating confidence of the object recognition results. The following properties may be applied to determine misidentifications in logical inference: temporal reasoning to represent relationships between time intervals, statistical reasoning with confidence of object recognition results. To show validity of our proposed method, experimental results are illustrated, where commercial visual recognition system is employed.     

Development of a semantic product modeling system for initial hull structure in shipbuilding

In the initial stage of ship design, designers represent geometry, arrangement, and dimension of hull structures, which correspond to product model information, with 2D geometric primitives such as points, lines, arcs, and drawing symbols on 2D drawings. However, designers must translate the product model information defined on the 2D drawings more intelligently in the following design stages. Thus, design semantics could be lost and design processes that follow could be delayed because of errors by mistranslating the information. Here, design semantics mean design intents of the designer, that is, functions and structures which the product must have.In this study, a semantic product model data structure of an initial ship hull structure was proposed, and a semantic product modeling system was developed based on the proposed data structure. The proposed data structure can store semantic product model information such as product design results with the use of 2D wire frame geometrical data, part attributes, and design knowledge. Hence, this information can be used to generate a 3D solid model and production material information for CAPP as needed.The applicability of the proposed data structure and the developed system was verified by applying them to the deadweight 300,000 ton of Very Large Crude oil Carrier’s product modeling procedure. The application results showed that the proposed data structure and the developed system can be efficiently used for overall initial ship design environment.     

Motion-based identification of multiple mobile robots using trajectory analysis in a well-configured environment with distributed vision sensors

Networked mobile robots are able to determine their poses (i.e., position and orientation) with the help of a well-configured environment with distributed sensors. Before localizing the mobile robots using distributed sensors, the environment has to have information on each of the robots?? prior knowledge. Consequently, if the environment does not have information on the prior knowledge of a certain mobile robot then it will not determine its current pose. To solve this restriction, as a preprocessing step for indoor localization, we propose a motion-based identification of multiple mobile robots using trajectory analysis. The proposed system identifies the robots by establishing the relation between their identities and their positions, which are estimated from their trajectories related to each of the paths generated as designated signs. The primary feature of the proposed system is the fact that networked mobile robots are quickly and simultaneously able to determine their poses in well-configured environments. Experimental results show that our proposed system simultaneously identifies multiple mobile robots, and approximately estimates each of their poses as an initial state for autonomous localization.     

Modeling robotic operations controlled by natural language

There are multiple ways to control a robotic system. Most of them require the users to have prior knowledge about robots or get trained before using them. Natural language based control attracts increasing attention due to its versatility and less requirements for users. Since natural language instructions from users cannot be understood by the robots directly, the linguistic input has to be processed into a formal representation which captures the task specification and removes the ambiguity inherent in natural language. For most of existing natural language controlled robotic system, they assume the given language instructions are already in correct orders. However, it is very likely for untrained users to give commands in a mixed order based on their direct observation and intuitive thinking. Simply following the order of the commands can lead to failures of tasks. To provide a remedy for the problem, we propose a novel framework named dependency relation matrix (DRM) to model and organize the semantic information extracted from language input, in order to figure out an executable sequence of subtasks for later execution. In addition, the proposed approach projects abstract language input and detailed sensory information into the same space, and uses the difference between the goal specification and temporal status of the task under implementation to monitor the progress of task execution. In this paper, we describe the DRM framework in detail, and illustrate the utility of this approach with experiment results.     

浆液下机器人定位功能实现

设计液下搅拌机器人的定位系统,选用多传感器来获取定位信息。用单片机(下位机)测得各个传感器信息,通过RS—485总线把信息传给工控机(上位机),工控机对信息进行融合,实时(本系统可达1 s)获得机器人的位置及航向信息。实验验证定位系统可行性,并对测距误差进行了分析,修正后定位误差在10 cm以内,满足对机器人的定位要求。     

Formation tracker design of multiple mobile robots with wheel perturbations: adaptive output-feedback approach

This paper presents a theoretical design approach for output-feedback formation tracking of multiple mobile robots under wheel perturbations. It is assumed that these perturbations are unknown and the linear and angular velocities of the robots are unmeasurable. First, adaptive state observers for estimating unmeasurable velocities of the robots are developed under the robots’ kinematics and dynamics including wheel perturbation effects. Then, we derive a virtual-structure-based formation tracker scheme according to the observer dynamic surface design procedure. The main difficulty of the output-feedback control design is to manage the coupling problems between unmeasurable velocities and unknown wheel perturbation effects. These problems are avoided by using the adaptive technique and the function approximation property based on fuzzy logic systems. From the Lyapunov stability analysis, it is shown that point tracking errors of each robot and synchronisation errors for the desired formation converge to an adjustable neighbourhood of the origin, while all signals in the controlled closed-loop system are semiglobally uniformly ultimately bounded.     

Error management for robot programming

Reliability is a serious problem in computer controlled robot systems. Although robots serve successfully in relatively simple applications such as painting and spot welding, their potential in areas such as automated assembly is hampered by the complexity of programming. A program for assembling parts may be logically correct, execute correctly on a simulator, and even execute correctly on a robot most of the time, yet still fail unexpectedly in the face of real world uncertainties. Recovery from such errors is far more complicated than recovery from simple controller errors, since even expected errors can manifest themselves in unexpected ways. In this paper we present a novel approach for improving robot reliability. Instead of anticipating errors, we use knowledge-based programming techniques so that the robot can autonomously exploit knowledge about its task and environment to detect and recover from failures. We describe a system that we have designed and constructed in our robotics laboratory.     

Shape recovery from robot contour-tracking with force feedback

In this paper we describe a process for shape recovery from robot contour-tracking operations with force feedback. Shape recovery is an important task for self-teaching robots and for exploratory operations in unknown environments. An algorithm which directs a position-controlled robot around an unknown planar contour using the steady-state contact force information is described in this paper. Shape recovery from planar contouring is not a trivial problem. It was found experimentally that there is significant distortion of the original contour if direct kinematics are used to recover the object's shape, as we are unable to recover the exact position of the robot tool owing to the errors present in the kinematic model of the arm and the non-linearities of the drive train. Drive train errors can consist of the joint compliance, gear backlash, and gear eccentricity. A mathematical model of the errors generated by the drive train has been previously addressed. In this paper a compensation process is explored for purposes of planar shape recovery. It was found through experimentation that the joint compliance is most conveniently compensated for in practice. Improvements in the shapes recovered from robot contouring are seen with our compensations. Experimental details and difficulties are also discussed.     

人工智能系统故障分析原理研究

为研究未来系统在人工智能控制下的系统故障预测、预防、控制和恢复能力,提出一种基于信息生态方法论的人工智能系统故障分析方法。将研究对象划分为人、功能、自然和智能系统;以智能系统为核心,研究故障信息、知识和智能安全生成原理;论述了基础故障意识、情感和理智的特点。研究表明,系统故障的人工智能分析必须采用信息生态方法论结合安全科学理论进行。分析原理是基于信息生态方法论,考虑基础故障意识、情感与理智,及即时故障语义信息进行的综合决策与反应,以确保系统在规定条件下完成预定功能。     

基于纹理信息的室内场景语义标注学习方法

针对目前室内场景视频中关键物体的检测、跟踪及信息编辑等方面主要是采用人工处理方式,存在效率低、精度不高等问题,提出了一种基于纹理信息的室内场景语义标注学习方法。首先,采用光流方法获取视频帧间的运动信息,利用关键帧标注和帧间运动信息进行非关键帧的标注初始化;然后,利用非关键帧的图像纹理信息约束及其初始化标注构建能量方程;最后,利用图割方法优化得到该能量方程的解,即为非关键帧语义标注。标注的准确率和视觉效果的实验结果表明,与运动估计法和基于模型的学习法相比较,所提基于纹理信息的室内场景语义标注学习法具有较好的效果。该方法可以为服务机器人、智能家居、应急响应等低时延决策系统提供参考。