Pedestrian navigation using the sense of touch

Haptics is a feedback technology that takes advantage of the human sense of touch by applying forces, vibrations, and/or motions to a haptic-enabled user device such as a mobile phone. Historically, human–computer interaction has been visual, data, or images on a screen. Haptic feedback can be an important modality in Mobile Location-Based Services like – knowledge discovery, pedestrian navigation and notification systems. In this paper we describe a methodology for the implementation of haptics in four distinct prototypes for pedestrian navigation. Prototypes are classified based on the user’s navigation guidance requirements, the user type (based on spatial skills), and overall system complexity. Here haptics is used to convey location, orientation, and distance information to users using pedestrian navigation applications. Initial user trials have elicited positive responses from the users who see benefit in being provided with a “heads up” approach to mobile navigation. We also tested the spatial ability of the user to navigate using haptics and landmark images based navigation. This was followed by a test of memory recall about the area. Users were able to successfully navigate from a given origin to a Destination Point without the use of a visual interface like a map. Results show the users of haptic feedback for navigation prepared better maps (better memory recall) of the region as compared to the users of landmark images based navigation.     

Evaluation of multimodal graphs for blind people

This paper introduces the development of a multimodal data visualisation system and its evaluations. This system is designed to improve blind and visually impaired peoples access to graphs and tables. Force feedback, synthesized speech and non-speech audio are utilised to present graphical data to blind people. Through the combination of haptic and audio representations, users can explore virtual graphs rendered by a computer. Various types of graphs and tables have been implemented, and a three-stage evaluation has been conducted. The experimental results have proven the usability of the system and the benefits of the multimodal approach. The paper presents the details of the development and experimental findings, as well as the changes of role of haptics in the evaluation.     

虚拟现实大空间下多虚拟目标被动触觉交互方法

针对虚拟现实（VR）大空间下为重定向行走的用户提供被动触觉时存在的虚实交互目标无法一一对应的问题,提出了一种用两个物理代理作为触觉代理为多个虚拟目标提供触觉反馈的方法,以在基于人工势场（APF）的重定向行走过程中,交替地满足用户被动触觉的需求。针对重定向行走算法本身以及标定不精确等原因造成的虚实不对齐的问题,对虚拟目标的位置及朝向进行设计并且在交互阶段引入触觉重定向。仿真实验表明对虚拟目标位置和朝向的设计可以大幅降低对齐误差;而用户实验结果证明触觉重定向的引入进一步提升了交互准确性,且能为用户带来更丰富、更具沉浸感的体验。     

Experiments in haptic-based authentication of humans

With the rapid advancement of the technological revolution, computer technology such as faster processors, advanced graphic cards, and multi-media systems are becoming more affordable. Haptics technology is a force/tactile feedback technology growing in disciplines linked to human–computer interaction. Similar to the increasing complexity of silicon-based components, haptics technology is becoming more advanced. On the other hand, currently available commercial haptics interfaces are expensive, and their application is mostly dedicated to enormous research projects or systems. However, the trend of the market is forcing haptic developers to release products for use in conjunction with current keyboards and mice technologies. Haptics allows a user to touch, fell, manipulate, create, and/or alter simulated three-dimensional objects in a virtual environment. Most of the existing applications of haptics are dedicated to hone human physical skills such as sensitive hardware repair, medical procedures, handling hazardous substances, etc. These skills can be trained in a realistic virtual world, and describe human behavioural patterns in human–computer interaction environments. The measurement of such psychomotor patterns can be used to verify a person’s identity by assessing unique-to-the-individual behavioural attributes. This paper explores the unique behaviour exhibited by different users interacting with haptic systems. Through several haptic-based applications, users’ physical attributes output data from the haptic interface for use in the construction of a biometric system.

Factors affecting user performance in haptic assembly

Current computer-aided assembly systems provide engineers with a variety of spatial snapping and alignment techniques for interactively defining the positions and attachments of components. With the advent of haptics and its integration into virtual assembly systems, users now have the potential advantage of tactile information. This paper reports research that aims to quantify how the provision of haptic feedback in an assembly system can affect user performance. To investigate human–computer interaction processes in assembly modeling, performance of a peg-in-hole manipulation was studied to determine the extent to which haptics and stereovision may impact on task completion time. The results support two important conclusions: first, it is apparent that small (i.e. visually insignificant) assembly features (e.g. chamfers) affect the overall task completion at times only when haptic feedback is provided; and second, that the difference is approximately similar to the values reported for equivalent real world peg-in-hole assembly tasks.     

Haptic cues as a utility to perceive and recognise geometry

Research has been conducted on how to aid blind peoples’ perceptions and cognition of scientific data and, specifically, on how to strengthen their background in mathematics as a means of accomplishing this goal. In search of alternate modes to vision, researchers and practitioners have studied the opportunities of haptics alone and in combination with other modes, such as audio. What is already known, and has motivated research in this area, is that touch and vision might form a common brain representation that is shared between the visual and haptic modalities and through haptics learning is active rather than passive. In spite of extensive research on haptics in the areas of psychology and neuropsychology, recent advances and rare experiences in using haptic technology have not caused a transfer from basic knowledge in the area of haptics to learning applications and practical guidelines on how to develop such applications. Thus motivated, this study investigates different haptic effects, such as free space, magnetic effects and the bounded box when blind people are given the task of recognising and manipulating classes of 3D objects with which they have varying familiarity. In parallel, this study investigates the applicability of Sjöström’s guidelines on haptic applications development and uses his problem classification to capture knowledge from the experiments. The results of this study show that users can easily recognise and manipulate familiar objects, albeit with some assistance. There is an indication that users completed tasks faster and needed less assistance with magnetic effects. However, they were not as satisfied with this mode. While the results of this study show that haptics have the potential to allow students to conceptualise 3D objects, much more work is needed to exploit this technology to the fullest. Objects with higher complexity are difficult for students, and, in their opinion, the virtual objects (as presented) leave much room for improvement. Sjöström’s error taxonomy proved useful, and four of five sub-guidelines tested were confirmed to be useful in this study.     

Desktop haptic virtual assembly using physically based modelling

This research investigates the feasibility of using a desktop haptic virtual environment as a design tool for evaluating assembly operations. Bringing virtual reality characteristics to the desktop, such as stereo vision, further promotes the use of this technology into the every day engineering design process. In creating such a system, the affordability and availability of hardware/software tools is taken into consideration. The resulting application combines several software packages including VR Juggler, open dynamics engine (ODE)/open physics abstraction layer (OPAL), OpenHaptics, and OpenGL/GLM/GLUT libraries to explore the benefits and limitations of combining haptics with physically based modelling. The equipment used to display stereo graphics includes a Stereographics emitter, Crystal Eyes shutter glasses, and a high refresh rate CRT Monitor. One or two-handed force feedback is obtained from various PHANTOM haptic devices from SensAble Technologies Inc. The application’s ability to handle complex part interactions is tested using two different computer systems, which approximate the higher and lower end of a typical engineer’s workstation. Different test scenarios are analyzed and results presented.     

Haptics for medical applications

Robots and intelligent machines in the future should adapt themselves autonomously to the open environment in order to realize physical support for human activities. In addition, the physical support by them must be based on the individual’s “action” and “sensation” in order that the physical support becomes really human-friendly. Then, the robots must actively recognize the unknown environment according to the individual’s action. They also have to transmit the environmental information obtained to the individual in harmony with their sensations. Since haptic information is so important, as well as visual information and auditory information, the development of realworld haptics is one of the important key issues for the purpose. Haptic information is inherently bilateral, since an action is always accompanied by a reaction. That means that bilateral control with high transparency is necessary to transmit real-world haptic information artificially. A acceleration-based bilateral controller is one of the solutions for realizing high transparency. There remain many issues to solve for the application of haptics to the physical support of actual human activities. A haptic system with high transparency should obtain flexibility in order to extend its function. This article presents flexible actuation techniques that have high force transferability and flexibility in actuators’ arrangements. Furthermore, in order to support human activities in remote environments, bilateral telehaptics over a network is also described. Finally, this article introduces the fundamental techniques in haptics, including several examples of medical applications, since they are the first target of real-world haptics. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008     

A novel multimodal interface for improving visually impaired people’s web accessibility

This paper introduces a novel interface designed to help blind and visually impaired people to explore and navigate on the Web. In contrast to traditionally used assistive tools, such as screen readers and magnifiers, the new interface employs a combination of both audio and haptic features to provide spatial and navigational information to users. The haptic features are presented via a low-cost force feedback mouse allowing blind people to interact with the Web, in a similar fashion to their sighted counterparts. The audio provides navigational and textual information through the use of non-speech sounds and synthesised speech. Interacting with the multimodal interface offers a novel experience to target users, especially to those with total blindness. A series of experiments have been conducted to ascertain the usability of the interface and compare its performance to that of a traditional screen reader. Results have shown the advantages that the new multimodal interface offers blind and visually impaired people. This includes the enhanced perception of the spatial layout of Web pages, and navigation towards elements on a page. Certain issues regarding the design of the haptic and audio features raised in the evaluation are discussed and presented in terms of recommendations for future work.     

Bridging the HASM: An OWL ontology for modeling the information pathways in haptic interfaces software

Haptics technology has received enormous attention to enhance human computer interaction. The last decade has witnessed a rapid progress in haptic application software development due to the fact that the underlying technology has become mature and has opened up novel research areas. In an attempt to organize the path between cause and effect we envision a need for a standard for haptic application software modeling. In order for the software to better enhance the tactile information sensation, flow and perception and also make interaction between humans and haptics more efficient and natural, we need a formal representation of the haptics domain. This article proposes the use of HASM, a haptic applications software modeling ontology to formally model the haptics domain in order to be used during the specifications and design phases of developing software applications for haptic interfaces. The presented ontology captures the existing knowledge in haptics domain, using OWL, and defines the pathways that the haptic information follows between the human and the machine haptic system, using SWRL rules. The haptic ontology that has been developed will be used as a basis to design effective user interfaces and assist the development of software modeling for haptic devices. A case study is demonstrating how this haptic ontology can be used to design a software model that analyzes the perception of a haptic property of an object by interacting with a haptic device.     

Haptic‐constraint modeling based on interactive metaballs

Adding interactive haptic‐constraint sensations is important in interactive computer gaming and 3D shape design. Usually constraints are set on vertices of the object to drive the deformation. How to simulate dynamic force constraints in interactive design is still a challenging task. In this paper, we propose a novel haptic‐constraint modeling method based on interactive metaballs, during which the haptic‐constraint tools are attracted to the target location and then control the touch‐enabled deformation within the constrained areas. The interactive force feedbacks facilitate designers to accurately deform the target regions and fine carve the details as their intention on the objects. Our work studies how to apply touch sensation in such constrained deformations using interactive metaballs, thus users can truly feel and control the soft‐touch objects during the deforming interactions. Experimental results show that the dynamic sense of touch during the haptic manipulation is intuitively simulated to users, via the interacting interface we have developed. Copyright © 2010 John Wiley & Sons, Ltd.     

Realistic haptic rendering of interacting deformable objects in virtual environments

A new computer haptics algorithm to be used in general interactive manipulations of deformable virtual objects is presented. In multimodal interactive simulations, haptic feedback computation often comes from contact forces. Subsequently, the fidelity of haptic rendering depends significantly on contact space modeling. Contact and friction laws between deformable models are often simplified in up to date methods. They do not allow a "realistic" rendering of the subtleties of contact space physical phenomena (such as slip and stick effects due to friction or mechanical coupling between contacts). In this paper, we use Signorini's contact law and Coulomb's friction law as a computer haptics basis. Real-time performance is made possible thanks to a linearization of the behavior in the contact space, formulated as the so-called Delassus operator, and iteratively solved by a Gauss-Seidel type algorithm. Dynamic deformation uses corotational global formulation to obtain the Delassus operator in which the mass and stiffness ratio are dissociated from the simulation time step. This last point is crucial to keep stable haptic feedback. This global approach has been packaged, implemented, and tested. Stable and realistic 6D haptic feedback is demonstrated through a clipping task experiment.     

Haptic gas pedal feedback

Active driver support systems either automate a control task or present warnings to drivers when their safety is seriously degraded. In a novel approach, utilising neither automation nor discrete warnings, a haptic gas pedal (accelerator) interface was developed that continuously presents car-following support information, keeping the driver in the loop. This interface was tested in a fixed-base driving simulator. Twenty-one drivers between the ages of 24 and 30 years participated in a driving experiment to investigate the effects of haptic gas pedal feedback on car-following behaviour. Results of the experiment indicate that when haptic feedback was presented to the drivers, some improvement in car-following performance was achieved, while control activity decreased. Further research is needed to investigate the effectiveness of the system in more varied driving conditions. Haptics is an under-used modality in the application of human support interfaces, which usually draw on vision or hearing. This study demonstrates how haptics can be used to create an effective driver support interface.     

Haptic gas pedal feedback

Active driver support systems either automate a control task or present warnings to drivers when their safety is seriously degraded. In a novel approach, utilising neither automation nor discrete warnings, a haptic gas pedal (accelerator) interface was developed that continuously presents car-following support information, keeping the driver in the loop. This interface was tested in a fixed-base driving simulator. Twenty-one drivers between the ages of 24 and 30 years participated in a driving experiment to investigate the effects of haptic gas pedal feedback on car-following behaviour. Results of the experiment indicate that when haptic feedback was presented to the drivers, some improvement in car-following performance was achieved, while control activity decreased. Further research is needed to investigate the effectiveness of the system in more varied driving conditions. Haptics is an under-used modality in the application of human support interfaces, which usually draw on vision or hearing. This study demonstrates how haptics can be used to create an effective driver support interface.     

The moderating role of sensation seeking tendency in robotic haptic interfaces

This study examined the interaction effects between haptic force feedback and users’ sensation seeking tendency (i.e. need for sensations) on users’ feelings of presence (i.e. the state in which users experience virtual objects and virtual environments as if they were actual) in robotic haptic interfaces. Users with low sensation seeking tendency felt stronger physical presence and spatial presence in response to force feedback haptic stimuli (versus no force feedback), whereas users with high sensation seeking tendency did not show any difference between the two conditions, thus confirming the moderating role of the users’ sensation seeking tendency in the robotic haptic interface. Theoretical implications for human–computer interaction (HCI) research and managerial implications for the interactive media market are discussed.     

Haptic rendering: introductory concepts

Haptic rendering allows users to "feel" virtual objects in a simulated environment. We survey current haptic systems and discuss some basic haptic-rendering algorithms. In the past decade we've seen an enormous increase in interest in the science of haptics. Haptics broadly refers to touch interactions (physical contact) that occur for the purpose of perception or manipulation of objects. These interactions can be between a human hand and a real object; a robot end-effector and a real object; a human hand and a simulated object (via haptic interface devices); or a variety of combinations of human and machine interactions with real, remote, or virtual objects. Rendering refers to the process by which desired sensory stimuli are imposed on the user to convey information about a virtual haptic object.     

Sharing control between humans and automation using haptic interface: primary and secondary task performance benefits

This paper describes a paradigm for human/automation control sharing in which the automation acts through a motor coupled to a machine's manual control interface. The manual interface becomes a haptic display, continually informing the human about automation actions. While monitoring by feel, users may choose either to conform to the automation or override it and express their own control intentions. This paper's objective is to demonstrate that adding automation through haptic display can be used not only to improve performance on a primary task but also to reduce perceptual demands or free attention for a secondary task. Results are presented from three experiments in which 11 participants completed a lane-following task using a motorized steering wheel on a fixed-base driving simulator. The automation behaved like a copilot, assisting with lane following by applying torques to the steering wheel. Results indicate that haptic assist improves lane following by least 30%, p < .0001, while reducing visual demand by 29%, p < .0001, or improving reaction time in a secondary tone localization task by 18 ms, p = .0009. Potential applications of this research include the design of automation interfaces based on haptics that support human/automation control sharing better than traditional push-button automation interfaces.     

Augmenting graphical user interfaces with haptic assistance for motion-impaired operators

Haptic assistance is an emerging field of research that is designed to improve human–computer interaction (HCI) by reducing error rates and targeting times through the use of force feedback. Haptic feedback has previously been investigated to assist motion-impaired computer users, however, limitations such as target distracters have hampered its integration with graphical user interfaces (GUIs). In this paper two new haptic assistive techniques are presented that utilise the 3DOF capabilities of the Phantom Omni. These are referred to as deformable haptic cones and deformable virtual switches. The assistance is designed specifically to enable motion-impaired operators to use existing GUIs more effectively. Experiment 1 investigates the performance benefits of the new haptic techniques when used in conjunction with the densely populated Windows on-screen keyboard (OSK). Experiment 2 utilises the ISO 9241-9 point-and-click task to investigate the effects of target size and shape. The results of the study prove that the newly proposed techniques improve interaction rates and can be integrated with existing software without many of the drawbacks of traditional haptic assistance. Deformable haptic cones and deformable virtual switches were shown to reduce the mean number of missed-clicks by at least 75% and reduce targeting times by at least 25%.     

Assisting Visually Impaired People to Acquire Targets on a Large Wall-Mounted Display

Large displays have become ubiquitous in our everyday lives, but these displays are designed for sighted people.This paper addresses the need for visually impaired people to access targets on large wall-mounted displays. We developed an assistive interface which exploits mid-air gesture input and haptic feedback, and examined its potential for pointing and steering tasks in human computer interaction（HCI）. In two experiments, blind and blindfolded users performed target acquisition tasks using mid-air gestures and two different kinds of feedback（i.e., haptic feedback and audio feedback）. Our results show that participants perform faster in Fitts＇ law pointing tasks using the haptic feedback interface rather than the audio feedback interface. Furthermore, a regression analysis between movement time（MT） and the index of difficulty（ID）demonstrates that the Fitts＇ law model and the steering law model are both effective for the evaluation of assistive interfaces for the blind. Our work and findings will serve as an initial step to assist visually impaired people to easily access required information on large public displays using haptic interfaces.     

Neuro-cognitively inspired haptic user interfaces

Haptic systems and devices are a recent addition to multimodal systems. These devices have widespread applications such as surgical simulations, medical and procedural training, scientific visualizations, assistive and rehabilitative devices for individuals who have physical or neurological impediments and assistive devices for individuals who are blind. While the potential of haptics in natural human machine interaction is undisputable, the realization of such means is still a long way ahead. There are considerable research challenges to development of natural haptic interfaces. The study of human tactile abilities is a recent endeavor and many of the available systems still do not incorporate the domain knowledge of psychophysics, biomechanics and neurological elements of haptic perception. Development of smart and effective haptic interfaces and devices requires extensive studies that link perceptual phenomena with measurable parameters and incorporation of such domain knowledge in the engineering of haptic interfaces. This paper presents design, development and usability testing of a neuro-cognitively inspired haptic user interface for individuals who are blind. The proposed system design is inspired by neuro-cognitive basis of haptic perception and incorporates the computational aspects and requirements of multimodal information processing system. Usability testing of the system suggests that a biologically inspired haptic user interfaces may form a powerful paradigm for haptic user interface design.