Augmented reality-assisted robot programming system for industrial applications

Robots are important in high-mix low-volume manufacturing because of their versatility and repeatability in performing manufacturing tasks. However, robots have not been widely used due to cumbersome programming effort and lack of operator skill. One significant factor prohibiting the widespread application of robots by small and medium enterprises (SMEs) is the high cost and necessary skill of programming and re-programming robots to perform diverse tasks. This paper discusses an Augmented Reality (AR) assisted robot programming system (ARRPS) that provides faster and more intuitive robot programming than conventional techniques. ARRPS is designed to allow users with little robot programming knowledge to program tasks for a serial robot. The system transforms the work cell of a serial industrial robot into an AR environment. With an AR user interface and a handheld pointer for interaction, users are free to move around the work cell to define 3D points and paths for the real robot to follow. Sensor data and algorithms are used for robot motion planning, collision detection and plan validation. The proposed approach enables fast and intuitive robotic path and task programming, and allows users to focus only on the definition of tasks. The implementation of this AR-assisted robot system is presented, and specific methods to enhance the performance of the users in carrying out robot programming using this system are highlighted.     

Transformative CAD based industrial robot program generation

Industrial robots are widely used in various processes of surface manufacturing, such as spray painting, spray forming, rapid tooling, spray coating, and polishing. Robot programming for these applications is still time consuming and costly. Typical teaching methods are not cost effective and efficient. There are many off-line programming methods developed to reduce the robot programming effort. However, these methods suffer many practical issues, such as cable/hose tangling, robot configuration, collision, and reachability. To solve these problems, this paper discusses a new method to generate robot programs. Since industrial robots have been used in production for decades, there are many robot programs for different parts generated by the robot programmers. These robot programs, which contain not only the robot paths, but also the programmers' knowledge and process parameters, can be transformed to generate new robot programs for similar parts. In this paper, a transformative robot program generation method is developed based on the existing ones in the database. Experiments were performed to validate the developed methodology. The results are very promising in reducing the programming efforts in surface manufacturing.     

Artificial intelligence and robotic assembly

Traditionally, most industrial robots are programmed by teaching. The emergence of robot-level programming languages has improved the programmer's ability to describe and modify the robot moves. However, commercially available robot-level programming languages still fall short of the robot user's need to program complex tasks, and consequently, are not widely used in industry. There is an increasing need for integrating sensors feedback into the robot system to provide better perception and for improving the capacity of the robot to reason and make decisions intelligently in real time.The role of artificial intelligence in programming and controlling robots is discussed. Available robot programming systems including robot-level, object-level, and task-level languages are reviewed. The importance of developing intelligent robots in broadening the scope of flexible automation and opening the door to new robotic applications in space, under water and in harsh environments is outlined. The current development and implementation of programming and control systems for intelligent robots, at McMaster University, are explained. A number of research issues are discussed such as (1) automatic task planning, (2) knowledge representation and use, (3) world modeling, (4) reasoning in automatic assembly planning, and (5) vision monitoring of actions. Examples of geometric, functional, and handling reasoning, as they apply to assembly, are provided. The systems described in this paper are being implemented in the center for flexible manufacturing research and development. Several pieces of hardware are used, including a six-axis articulated robot, a grey-level vision system with a multi-camera, Micro VAX II, and a variety of graphics monitors. The languages available for software development include Common LISP, C, OPS5, VAL II, PASCAL, and FORTRAN 77. The domain of application is currently focused on mechanical assembly.     

Problems associated with the off-line programming of robots

The introduction of robots into any organization forms part of a considerable investment in new technology over a range of applications in the search for efficiency and increased productivity. The greater consistency and quality associated with robot operations, compared with that of the human operator, is regarded as an advantage both in relation to the manufacturing process and to the product. However, in order to make the most efficient use of robots, the ability to generate good robot programs must be developed. Traditional robot programming techniques are extremely slow and laborious. Off-line programming by textual input alone is an equally tedious process. In this paper, existing and potential problems associated with off-line programming are examined. Various commercial and experimental robot languages and their relative important features are described. Opinions of manufacturers of CNC (Computer Numerically Controlled) machines and robots concerning both user interfaces and potential candidate users are discussed. Both the training and the abilities of the candidate robot programmers are important aspects of any robot programming system, although obviously much will depend on the sophistication of the particular application. Recommendations are made concerning issues that should be taken into account when developing future off-line programming systems.     

Design of service robots

According to the International Federation of Robotics (IFR), "a service robot is a robot which operates semi or fully autonomously to perform services useful to the well being of human and equipment, excluding manufacturing operations" [1]. These devices are typically complex systems requiring the input of knowledge from numerous disciplines. The authors have been using different software engineering techniques for the last 15 years, integrating new paradigms in the service robot development process as they emerged. This has made it possible to achieve rapid development of applications and subsequent maintenance. During the early years (1993?1998), our effortswere directed at the development of software for various kinds of teleoperated robots to performmaintenance tasks in nuclear power plants [2]; during a second phase (1999?2006),we built applications for ship-hull cleaning robots [3]. All this time, we have been applying all the possibilities of software engineering, from the use of paradigms for structured and object-based programming in early developments to the adoption of the current model-driven approach [model-driven engineering (MDE)] [4].     

Ontology-based state representations for intention recognition in human–robot collaborative environments

In this paper, we describe a novel approach for representing state information for the purpose of intention recognition in cooperative human–robot environments. States are represented by a combination of spatial relationships in a Cartesian frame along with cardinal direction information. This approach is applied to a manufacturing kitting operation, where humans and robots are working together to develop kits. Based upon a set of predefined high-level state relationships that must be true for future actions to occur, a robot can use the detailed state information described in this paper to infer the probability of subsequent actions occurring. This would allow the robot to better help the human with the task or, at a minimum, better stay out of his or her way.     

Realization of sign language motion using a dual-arm/hand humanoid robot

The recent increase in technological maturity has empowered robots to assist humans and provide daily services. Voice command usually appears as a popular human–machine interface for communication. Unfortunately, deaf people cannot exchange information from robots through vocal modalities. To interact with deaf people effectively and intuitively, it is desired that robots, especially humanoids, have manual communication skills, such as performing sign languages. Without ad hoc programming to generate a particular sign language motion, we present an imitation system to teach the humanoid robot performing sign languages by directly replicating observed demonstration. The system symbolically encodes the information of human hand–arm motion from low-cost depth sensors as a skeleton motion time-series that serves to generate initial robot movement by means of perception-to-action mapping. To tackle the body correspondence problem, the virtual impedance control approach is adopted to smoothly follow the initial movement, while preventing potential risks due to the difference in the physical properties between the human and the robot, such as joint limit and self-collision. In addition, the integration of the leg-joints stabilizer provides better balance of the whole robot. Finally, our developed humanoid robot, NINO, successfully learned by imitation from human demonstration to introduce itself using Taiwanese Sign Language.     

A new simulation based robot command library applied to three robots

A new type of high‐level robot command library is presented and demonstrated. Three robot programming languages have been analyzed and new robot command libraries created for three types of robot. The programming of three robots using the new high‐level robot command library demonstrated that it was possible to program robots with different kinematic configurations without the programmer having any knowledge of the physical structure of the robots. The library commands contained simulations of the abilities of the robots as well as having the ability to control the physical robots. This paper shows how simulation can be incorporated into a high‐level robot command library and how the command library can be used for the programming of three industrial robots. ©1999 John Wiley & Sons, Inc.     

机器人感知与控制关键技术及其智能制造应用

智能机器人在服务国家重大需求,引领国民经济发展和保障国防安全中起到重要作用,被誉为“制造业皇冠顶端的明珠”.随着新一轮工业革命的到来,世界主要工业国家都开始加快机器人技术的战略部署.而智能机器人作为智能制造的重要载体,在深入实施制造强国战略,推动制造业的高端化、智能化、绿色化过程中将发挥重要作用.本文从智能机器人的感知与控制等关键技术的视角出发,重点阐述了机器人的三维环境感知、点云配准、位姿估计、任务规划、多机协同、柔顺控制、视觉伺服等共性关键技术的国内外发展现状.然后,以复杂曲面机器人三维测量、复杂部件机器人打磨、机器人力控智装配等机器人智能制造系统为例,阐述了机器人的智能制造的应用关键技术,并介绍了工程机械智能化无人工厂、无菌化机器人制药生产线等典型案例.最后探讨了智能制造机器人的发展趋势和所面临的挑战.     

Safe and efficient motion planning of multiple mobile robots based on artificial potential for human behavior and robot congestion

In order for robots to exist together with humans, safety for the humans has to be strictly ensured. On the other hand, safety might decrease working efficiency of robots. Namely, this is a trade-off problem between human safety and robot efficiency in a field of human–robot interaction. For this problem, we propose a novel motion planning technique of multiple mobile robots. Two artificial potentials are presented for generating repulsive force. The first potential is provided for humans. The von Mises distribution is used to consider the behavioral property of humans. The second potential is provided for the robots. The Kernel density estimation is used to consider the global robot congestion. Through simulation experiments, the effectiveness of the behavior and congestion potentials of the motion planning technique for human safety and robot efficiency is discussed. Moreover, a sensing system for humans in a real environment is developed. From experimental results, the significance of the behavior potential based on the actual humans is discussed. For the coexistence of humans and robots, it is important to evaluate a mutual influence between them. For this purpose, a virtual space is built using projection mapping. Finally, the effectiveness of the motion planning technique for the human–robot interaction is discussed from the point of view of not only robots but also humans.     

Human–Robot Cooperation Using Multi-Agent-Systems

In this paper a new intelligent robot control scheme is presented which enables a cooperative work of humans and robots through direct contact interaction in a partially known environment. Because of the high flexibility and adaptability, the human–robot cooperation is expected to have a wide range of applications in uncertain environments, not only in future construction and manufacturing industries but also in service branches. A multi-agent control architecture gives an appropriate frame for the flexibility of the human–robot-team. Robots are considered as intelligent autonomous assistants of humans which can mutually interact on a symbolic level and a physical level. This interaction is achieved through the exchange of information between humans and robots, the interpretation of the transmitted information, the coordination of the activities and the cooperation between independent system components. Equipped with sensing modalities for the perception of the environment, the robot system KAMRO (Karlsruhe Autonomous Mobile Robot) is introduced to demonstrate the principles of the cooperation among humans and robot agents. Experiments were conducted to prove the effectiveness of our concept.     

车身焊装生产线上多台工业机器人应用技术的研究

本文讨论利用系统集成技术和计算机仿真技术的方法，根据用户提出的工艺要求和操作要求 进行工业机器人、焊接设备以及周边设备的选择，工业机器人焊接工位的设计与安装，自动 控制系统（PLC）的程序编制和机器人程序编制以及对若干关键技术的研究．     

A STEP-compliant Industrial Robot Data Model for robot off-line programming systems

Recently, various robot off-line programming systems have promoted their own robot data models, resulting in a plethora of robot representation methods and unchangeable data files among CAx and robot off-line programming systems. The current paper represents a STEP-compliant Industrial Robot Data Model (IRDM) for data exchange between CAx systems and robot off-line programming systems. Using this novel representation method, most resources involved in a robot manufacturing system can be represented. The geometric and mathematic aspects of industrial robots have been defined in IRDM, so that the robot off-line programming system could have abundant information to represent robots’ kinematic and dynamic behaviors. In order to validate the proposed models and approaches, a prototype robot off-line programming system with 3D virtual environment is presented. The functionalities of IRDM not only have significant meaning for providing a unified data platform for robot simulation systems, but also have the potential capability to represent robot language using the object-oriented concept.     

Error management for robot programming

Reliability is a serious problem in computer controlled robot systems. Although robots serve successfully in relatively simple applications such as painting and spot welding, their potential in areas such as automated assembly is hampered by the complexity of programming. A program for assembling parts may be logically correct, execute correctly on a simulator, and even execute correctly on a robot most of the time, yet still fail unexpectedly in the face of real world uncertainties. Recovery from such errors is far more complicated than recovery from simple controller errors, since even expected errors can manifest themselves in unexpected ways. In this paper we present a novel approach for improving robot reliability. Instead of anticipating errors, we use knowledge-based programming techniques so that the robot can autonomously exploit knowledge about its task and environment to detect and recover from failures. We describe a system that we have designed and constructed in our robotics laboratory.     

Integration of Heterogeneity for Human-Friendly Robotic Operations

The objective of this paper is to develop an analytical scheme to integrate the heterogeneity of human and robot functions to achieve a human-friendly robotic operations. The heterogeneity of human and robot functions can be characterized by the fact that humans are intelligent while robots are fast, powerful and accurate. Humans can use their knowledge and experience to quickly respond to unexpected events, which makes it easy for humans to deal with unstructured environments. In contrast, robots can easily enhance the mechanical power of humans and the ability of humans to work remotely. Therefore, robots are capable of performing precise and repetitive tasks at high speed or in a hazardous environment. The important issue, in light of human/robot heterogeneity, is how to plan and control a robotic operation such that the human and the robot can cooperate in a complementary manner. Thus, a task which cannot be done by either human or robot alone can be performed efficiently and robustly by both. This paper introduces a new paradigm for human/robot interactive systems, heterogeneous function-based human/robot cooperation. A new perceptive action reference frame has been developed in the paper. It matches human perception and robot sensory measurement, and provides a platform for modeling the human/robot cooperative operations. The theoretical results presented in the paper have laid down a foundation for stability analysis as well as a planning and control system design of human/robot integrated systems. The implementations and experimental results have clearly demonstrated the advantages of proposed methods.     

Smart hardware integration with advanced robot programming technologies for efficient reconfiguration of robot workcells

The manufacturing industry is seeing an increase in demand for more custom-made, low-volume production. This type of production is rarely automated and is to a large extent still performed manually. To keep up with the competition and market demands, manufacturers will have to undertake the effort to automate such manufacturing processes. However, automating low-volume production is no small feat as the solution should be adaptable and future proof to unexpected changes in customers’ demands. In this paper, we propose a reconfigurable robot workcell aimed at automating low-volume production. The developed workcell can adapt to the changes in manufacturing processes by employing a number of passive, reconfigurable hardware elements, supported by the ROS-based, modular control software. To further facilitate and expedite the setup process, we integrated intuitive, user-friendly robot programming methods with the available hardware. The system was evaluated by implementing five production processes from different manufacturing industries.     

Reusing Primitive and Acquired Motion Knowledge for Gait Generation of a Six-legged Robot Using Genetic Programming

There has been growing interest in motion planning problems for mobile robots. In this field, the main research is to generate a motion for a specific robot and task without previously acquired motions. However it is too wasteful not to use hard-earned acquired motions for other tasks. Here, we focus on a mechanism of reusing acquired motion knowledge and study a motion planning system able to generate and reuse motion knowledge. In this paper, we adopt a tree-based representation for expressing knowledge of motion, and propose a hierarchical knowledge for realizing a reuse mechanism. We construct a motion planning system using hierarchical knowledge as motion knowledge and using genetic programming as a learning method. We apply a proposed method for the gait generation task of a six-legged locomotion robot and show its availability with computer simulation.     

Human–robot interaction in industrial collaborative robotics: a literature review of the decade 2008–2017

ABSTRACT

Currently, a large number of industrial robots have been deployed to replace or assist humans to perform various repetitive and dangerous manufacturing tasks. However, based on current technological capabilities, such robotics field is rapidly evolving so that humans are not only sharing the same workspace with robots, but also are using robots as useful assistants. Consequently, due to this new type of emerging robotic systems, industrial collaborative robots or cobots, human and robot co-workers have been able to work side-by-side as collaborators to accomplish tasks in industrial environments. Therefore, new human–robot interaction systems have been developed for such systems to be able to utilize the capabilities of both humans and robots. Accordingly, this article presents a literature review of major recent works on human–robot interactions in industrial collaborative robots, conducted during the last decade (between 2008 and 2017). Additionally, the article proposes a tentative classification of the content of these works into several categories and sub-categories. Finally, this paper addresses some challenges of industrial collaborative robotics and explores future research issues.     

Design of a modular assembly of four-footed robots with multiple functions

Modern industries use many types of robots. In addition to general robotic arms, bipedal, tripedal, and quadrupedal robots, which were originally developed as toys, are gradually being used for multiple applications in manufacturing processes. This research begins with establishing the platform for four-footed robots with multiple functions, high sensitivity, and modular assembly and this is how a fundamental model of the industrial robots is constructed. Under additional loads, the four feet of the quadrupedal robot reinforce its carrying ability and reliability compared to bipedal or tripedal robots, which helps it to carry more objects and enhances functionality. Based on different requirements and demands from the manufacturing processes, the highly sensitive four-footed robot provides an expandable interface to add different sensing components. In addition, when combined with a wireless communication module or independent 1.2 GHz radio frequency CCD wireless image transmission system, the user can control the robot remotely and instantly. The design helps the four-footed robot to expand its applications. By assembling and disassembling modules and changing the sensing components, the highly sensitive four-footed robot can be used for different tasks. Moreover, the remote control function of the robot will increase interaction with human beings, so it can become highly become involved in people's lives. The platform of the four-footed robot will become a design reference for the commercialization of different industrial robots, and it will provide the design of industrial robots with more options and useful applications.     

A Robot Swarm Assisting a Human Fire-Fighter

Emergencies in industrial warehouses are a major concern for fire-fighters. The large dimensions, together with the development of dense smoke that drastically reduces visibility, represent major challenges. The GUARDIANS robot swarm is designed to assist fire-fighters in searching a large warehouse. In this paper we discuss the technology developed for a swarm of robots assisting fire-fighters. We explain the swarming algorithms that provide the functionality by which the robots react to and follow humans while no communication is required. Next we discuss the wireless communication system, which is a so-called mobile ad-hoc network. The communication network provides also the means to locate the robots and humans. Thus, the robot swarm is able to provide guidance information to the humans. Together with the fire-fighters we explored how the robot swarm should feed information back to the human fire-fighter. We have designed and experimented with interfaces for presenting swarm-based information to human beings.