Adaptive robot training by programming and guiding

The simplest and most common robot training method is operator-guiding, but this method is inflexible and limited to fixed sequences. Explicit textual robot programming has enjoyed many advances in the last decade and current research implementations are powerful. But explicit programming is complex and programmers must be specially trained. Mixed systems attempt to unite the facility of guiding with the power of programming. However, existing mixed systems exhibit mismatches in the interaction between programming and guiding, because they ignore underlying dependencies between the programmer and operator. An operator may (1) be given insufficient visual cues; (2) lack the required dexterity; or (3) be required to add unforeseen movement sequences to the program. This paper presents the design and implementation of ART (adaptive robot trainer), a prototypical mixed robot training system that eliminates or corrects deficiencies in visual cues and dexterity, and additionally improves the guiding and programming components. Mixed systems and assembly tasks are analysed, to give an effective representation of task state, which in turn motivates the design of ART's language to automate much of the program-guiding interaction. The language allows the programmer to express assembly operations and object-feature relationships in a natural way while providing the system with the information necessary to maintain the task state. The representation also enables the correction of guiding errors, flexibility in the guiding protocol and the generation of meaningful messages to prompt operator actions. These principles in the design and implementation pave the road to more instructable, capable robots.Now at Bell Northern Research, Ottawa.     

IAda: A language for robot programming based on Ada

We present a programming language for robots which we have implemented based on the Ada language. It is an interpreted language which permits dynamic configuration of software. It manipulates Ada tasks and subroutines. One of the Ada tasks is an inference engine of a logic programming language adapted to real-time constraints. We show how the conjunction of Ada tasks, to perform perception and action functions on the robot, to logic programs, for the control of these tasks, both manipulated by the IAda language, gives a powerful environment for robot programming.     

A three-dimensional machine-vision approach for automatic robot programming

This research investigates a novel robot-programming approach that applies machine-vision techniques to generate a robot program automatically. The hand motions of a demonstrator are initially recorded as a long sequence of images using two CCD cameras. Machine-vision techniques are then used to recognize the hand motions in three-dimensional space including open, closed, grasp, release and move. The individual hand feature and its corresponding hand position in each sample image is translated to robot's manipulator-level instructions. Finally a robot plays back the task using the automatically generated program.A robot can imitate the hand motions demonstrated by a human master using the proposed machine-vision approach. Compared with the traditional leadthrough and structural programming-language methods, the robot's user will not have to physically move the robot arm through the desired motion sequence and learn complicated robot-programming languages. The approach is currently focused on the classification of hand features and motions of a human arm and, therefore, is restricted to simple pick-and-place applications. Only one arm of the human master can be presented in the image scene, and the master must not wear long-sleeved clothes during demonstration to prevent false identification. Analysis and classification of hand motions in a long sequence of images are time-consuming. The automatic robot programming currently developed is performed off-line.     

A STEP-compliant Industrial Robot Data Model for robot off-line programming systems

Recently, various robot off-line programming systems have promoted their own robot data models, resulting in a plethora of robot representation methods and unchangeable data files among CAx and robot off-line programming systems. The current paper represents a STEP-compliant Industrial Robot Data Model (IRDM) for data exchange between CAx systems and robot off-line programming systems. Using this novel representation method, most resources involved in a robot manufacturing system can be represented. The geometric and mathematic aspects of industrial robots have been defined in IRDM, so that the robot off-line programming system could have abundant information to represent robots’ kinematic and dynamic behaviors. In order to validate the proposed models and approaches, a prototype robot off-line programming system with 3D virtual environment is presented. The functionalities of IRDM not only have significant meaning for providing a unified data platform for robot simulation systems, but also have the potential capability to represent robot language using the object-oriented concept.     

A novel AR-based robot programming and path planning methodology

This paper discusses the benefits of applying Augmented Reality (AR) to facilitate intuitive robot programming, and presents a novel methodology for planning collision-free paths for an n degree-of-freedom (DOF) manipulator in an unknown environment. The targeted applications are where the end-effector is constrained to move along a visible 3D path/curve, which position is unknown, at a particular orientation with respect to the path, such as arc welding and laser cutting. The methodology is interactive as the human is involved in obtaining the 3D data points of the desired curve to be followed through performing a number of demonstrations, defining the free space relevant to the task, and planning the orientations of the end-effector along the curve. A Piecewise Linear Parameterization (PLP) algorithm is used to parameterize the data points using an interactively generated piecewise linear approximation of the desired curve. A curve learning method based on Bayesian neural networks and reparameterization is used to learn and generate 3D parametric curves from the parameterized data points. Finally, the orientation of the end-effector along the learnt curve is planned with the aid of AR. Two case studies are presented and discussed.     

Novel integrated optimization algorithm for trajectory planning of robot manipulators based on integrated evolutionary programming

Optimal trajectory plarmmg for robot manipulators plays an important role in implementing the high productivity for robots. The performance indexes used in optimal trajectory planning are classified into two roam categories:optimum traveling time and optimum mechanical energy of the actuators. The current trajectory planning algorithms are designed based on one of the above two performance indexes. So far, there have been few planning algorithms designed to satisfy two performance indexes simultaneously. On the other hand, some deficiencies arise in the existing integrated optimization algorithms of trajectory planning.In order to overcome those deficiencies, the integrated optimization algorithms of trajectory planning are presented based on the complete analysis for trajectory planning of robot manipulators. In the algorithm, two object functiom are designed based on the specific weight coefficient method and “ideal point” strategy. Moreover, based on the features of optimization problem, the intensified evolutionary programming is proposed to solve the corresponding optimization model. Especially, for the Stanford Robot, the high-quality solutions are found at a lower cost.     

Integrated architecture for industrial robot programming and control

As robot control systems are traditionally closed, it is difficult to add supplementary intelligence. Accordingly, as based on a new notion of user views, a layered system architecture is proposed. Bearing in mind such industrial demands as computing efficiency and simple factory-floor operation, the control layers are parameterized by means of functional operators consisting of pieces of compiled code that can be passed as parameters between the layers. The required interplay between application-specific programs and built-in motion control is thereby efficiently accomplished. The results from experimental evaluation and several case studies suggest the architecture to be very useful also in an industrial context.     

Dual-coding representations for robot vision programming in Tekkotsu

We describe complementary iconic and symbolic representations for parsing the visual world. The iconic pixmap representation is operated on by an extensible set of “visual routines” (Ullman, 1984; Forbus et al., 2001). A symbolic representation, in terms of lines, ellipses, blobs, etc., is extracted from the iconic encoding, manipulated algebraically, and re-rendered iconically. The two representations are therefore duals, and iconic operations can be freely intermixed with symbolic ones. The dual-coding approach offers robot programmers a versatile collection of primitives from which to construct application-specific vision software. We describe some sample applications implemented on the Sony AIBO. David S. Touretzky is a Research Professor in the Computer Science Department and the Center for the Neural Basis of Cognition at Carnegie Mellon University. He earned his B.A. in Computer Science from Rutgers University in 1978, and his M.S. (1979) and Ph.D. (1984) in Computer Science from Carnegie Mellon. Dr. Touretzky’s research interests are in computational neuroscience, particularly representations of space in the rodent hippocampus and related structures, and high level primitives for robot programming. He is presently developing an undergraduate curriculum in cognitive robotics based on the Tekkotsu software framework described in this article. Neil S. Halelamien earned a B.S. in Computer Science and a B.S. in Cognitive Science at Carnegie Mellon University in 2004, and is currently pursuing his Ph.D. in the Computation & Neural Systems program at the California Institute of Technology. His research interests are in studying vision from both a computational and biological perspective. He is currently using transcranial magnetic stimulation to study visual representations and information processing in visual cortex. Ethan J. Tira-Thompson is a graduate student in the Robotics Institute at Carnegie Mellon University. He earned a B.S. in Computer Science and a B.S. in Human-Computer Interaction in 2002, and an M.S. in Robotics in 2004, at Carnegie Mellon. He is interested in a wide variety of computer science topics, including machine learning, computer vision, software architecture, and interface design. Ethan’s research has revolved around the creation of the Tekkotsu framework to enable the rapid development of robotics software and its use in education. He intends to specialize in mobile manipulation and motion planning for the completion of his degree. Jordan J. Wales is completing a Master of Studies in Theology at the University of Notre Dame. He earned a B.S. in Engineering (Swarthmore College, 2001), an M.Sc. in Cognitive Science (Edinburgh, UK, 2002), and a Postgraduate Diploma in Theology (Oxford, UK, 2003). After a year as a graduate research assistant in Computer Science at Carnegie Mellon, he entered the master’s program in Theology at Notre Dame and is now applying to doctoral programs. His research focus in early and medieval Christianity is accompanied by an interest in medieval and modern philosophies of mind and their connections with modern cognitive science. Kei Usui is a masters student in the Robotics Institute at Carnegie Mellon University. He earned his B.S. in Physics from Carnegie Mellon University in 2005. His research interests are reinforcement learning, legged locomotion, and cognitive science. He is presently working on algorithms for humanoid robots to maintain balance against unexpected external forces.     

A versatile interaction framework for robot programming based on hand gestures and poses

This paper proposes a framework for industrial and collaborative robot programming based on the integration of hand gestures and poses. The framework allows operators to control the robot via both End-Effector (EE) and joint movements and to transfer compound shapes accurately to the robot. Seventeen hand gestures, which cover the position and orientation controls of the robotic EE and other auxiliary operations, are designed according to cognitive psychology. Gestures are classified by a deep neural network, which is pre-trained for two-hand pose estimation and fine-tuned on a custom dataset, achieving a test accuracy of 99%. The index finger’s pointing direction and the hand’s orientation are extracted via 3D hand pose estimation to indicate the robotic EE’s moving direction and orientation, respectively. The number of stretched fingers is detected via two-hand pose estimation to represent decimal digits for selecting robot joints and inputting numbers. Finally, we integrate these three manners seamlessly to form a programming framework.We conducted two interaction experiments. The reaction time of the proposed hand gestures in indicating randomly given instructions is significantly less than that of other gesture sets, such as American Sign Language (ASL). The accuracy of our method in compound shape reconstruction is much better than that of hand movement trajectory-based methods, and the operating time is comparable with that of teach pendants.     

Graphical simulation for sensor based robot programming

The programming of robots is slowly evolving from traditional teach pendant methods to graphical Off-Line Programming (OLP) methods. Graphical simulation tools, such as OLP, are very useful for developing and testing robot programs before they are run on real industrial equipment. OLP systems are also used to develop task level programs. Traditional OLP systems, however, suffer from the limitations of using only position control which does not account for inherent robot inaccuracies and dynamic environments. This paper describes our work on improving and supplementing traditional position control programming methods. A baseline OLP system was implemented at NIST's Automated Manufacturing Research Facility (AMRF). Experience gained in implementing this system showed that an effective OLP system must accurately simulate the real world and must support sensor programming to compensate for real-world changes that cannot be simulated. The developed OLP geometric world model is calibrated using robot mounted ultrasound ranging sensors. This measurement capability produces a baseline geometric model of relatively good static accuracy for off-line programming. The graphical environment must also provide representations of sensor features. For this specific application, force is simulated in order to include force based commands in our robot programs. These sensor based programs are able to run reliably and safely in an unpredictable industrial environment. The last portion of this paper extends OLP and describes the functionality of a complete system for programming complex robot tasks.     

A sensor based technique for automated robot programming

This paper presents the concept of integrating a tactile sensor based, automated part measuring technique and a CAD programming scheme to perform off-line programming for a welding robot. The techniques used to achieve data flow throughout the production process include coordinate system referencing, interactive programming, tactile sensing seam tracing, and coordinate transformation. Together they form the backbone of the idea of integrating measurement and production operations where continuous path geometrical control is required. Techniques for referencing a workpiece with respect to different handling devices, and positioning the part on a suitable fixture, form an important portion of this work. Consequently, the positioning aspect and the data transfer capability between stages in the production process are highlighted. The welding of lap-joint type seams serves as an example. The integration of automated measurement and off-line robot programming actually constitutes a flexible manufacturing system operation that is capable of assuring the required process control. In this regard, linking a robot with an automated part measuring technique forms an important step towards upgrading a robot device from a difficult to program unit to a unit with a high degree of flexibility with rapid, convenient automatic contour programming.     

A blackboard system for the off-line programming of Robots

This paper describes the use of the blackboard architecture for the off-line programming of an IMB 7565 Robot. A blackboard system was implemented in PROLOG and it has been applied successfully for the automatic generation of a control code for the robot to perform the task of block assembly in an environment with an obstacle. The opportunistic type of problem-solving offered by the blackboard architecture has succeeded in obtaining a solution. The user-interface to the system is represented as a knowledge source in the blackboard system, which allows the user to modify the goal specifications during the operation of the blackboard system.     

ADA: A language for robot programming?

Robot programming languages are emerging from their experimental stage and entering an assessment phase. Their main features are illustrated and a parallel with ADA is proposed. The comparison is positive for ADA, in the sense that ADA provides most of the required capabilities. The ability of reasoning on object models and taking decisions will play an increasing role in the future. In this case the role of ADA would possibly change and its interest as robot programming language decrease.     

Multiple robot programming using a concurrent logic language

Advances in robotics has led to the cooperation of multiple robots among themselves and with their industrial automation environment. Efficient interaction with industrial robots thus becomes one of the key factors in the successful utilization of this modern equipment. When multiple manipulators have to be coordinated, there is a need for a new programming approach that facilitates and encompasses the needs of concurrency, synchronization, timing, and communication. Most robot languages have been developed with little attention being given to the integration of the robot with its environment. Currently, there is a gap between the robot capabilities, the task definition environment, and language facilities supplied to use robots.This paper analyzes the needs and then establishes that a concurrent logic programming approach is a step towards achieving a multi-robot knowledgeable task programming. In particular, the FCP dialect of concurrent Prolog is demonstrated, and analyzed.This research is partially supported by the Paul Ivanier Center for research in robots and production management.     

机器人相机空间智能编程方法研究*

针对传统的示教编程方式存在操作复杂,效率低,危险性高等不足,严重限制了工业机器人的推广应用。基于自然的人机交互示教方式,提出了一种基于计算机视觉的相机空间工业机器人智能虚拟编程方法,本方法不需要实际操作示教盒和机器人本体,仅采用辅助示教工具在视觉相机空间示教就实现了工业机器人的虚拟编程。主要研究了实现该方案的关键技术即基于相机空间映射模型的视觉定位技术以及基于K-means聚类算法实现的相机空间映射关系自学习技术。最后,基于自主开发的机器人平台,开展基于相机空间的虚拟智能编程实验,验证了本文提出的相机空间工业机器人智能编程方法的可行性及正确性。     

An online robot trajectory planning and programming support system for industrial use

The manufacturing industry today is still looking for enhancement of their production. Programming of articulated production robots is a major area for improvement. Today, offline simulation modified by manual programming is widely used to reduce production downtimes but requires financial investments in terms of additional personnel and equipment costs. The requirements have been evaluated considering modern manufacturing aspects and a new online robot trajectory planning and programming support system is presented for industrial use. The proposed methodology is executed solely online, rendering offline simulation obsolete and thereby reduces costs. To enable this system, a new cell-based Voronoi generation algorithm, together with a trajectory planner, is introduced. The robot trajectories so achieved are comparable to manually programmed robot programs. The results for a Mitsubishi RV-2AJ five axis industrial robot are presented.     

Direct off-line robot programming via a common CAD package

This paper focuses on intuitive and direct off-line robot programming from a CAD drawing running on a common 3-D CAD package. It explores the most suitable way to represent robot motion in a CAD drawing, how to automatically extract such motion data from the drawing, make the mapping of data from the virtual (CAD model) to the real environment and the process of automatic generation of robot paths/programs. In summary, this study aims to present a novel CAD-based robot programming system accessible to anyone with basic knowledge of CAD and robotics. Experiments on different manipulation tasks show the effectiveness and versatility of the proposed approach.     

Task-oriented programming of large redundant robot motion

Large robots are a new domain of advanced robotics. Examples of their application fields are tasks like operations on large free-form surfaces, especially aircraft cleaning and removing paint from hulls. They are equipped with a programmable robot control comparable to a control system used for industrial robots. However, conventional teach-in methods are not able to manage the complexity of programming large redundant robot operation on free-form geometries. The Fraunhofer IPA has developed an innovative off-line programming system that allows the creation of robot motion programs which satisfy time and energy optimization criteria. This system helps to avoid collisions within the workspace and to fulfill conditions that arise from the robot kinematics and dynamics. This advanced programming system has been successfully used to generate motion programs for the world's largest mobile robot, the aircraft cleaning manipulator SKYWASH. In this context offline programs for eleven different types of aircraft have been developed.     

机器人控制软件的图形化编程系统设计

提出了一种基于图形化编程的机器人软件系统的总体结构设计,包括软件结构的总体设计思路,主要处理从流程图式的图形化结构到类C语言的转换,再将这种类C语言翻译转换为下位机所能识别的目标代码,经过串口通讯下载到下位机中,完成上位机和下位机的信息交互.提出的设计方法和思路已经过严格的测试和检测,实际应用表明,该系统具有较好的灵活性,稳定性和实用性.