Design evaluation of information appliances using augmented reality-based tangible interaction

In this paper, we propose an approach to tangible augmented reality (AR) based design evaluation of information appliances, which not only exploits the use of tangible objects without hardwired connections to provide better visual immersion and support more tangible interaction, but also facilitates the adoption of a simple and low cost AR environment setup to improve user experience and performance. To enhance the visual immersion, we develop a solution for resolving hand occlusion in which skin color information is exploited with the use of the tangible objects to detect the hand regions properly. To improve the tangible interaction with the sense of touch, we introduce the use of product- and fixture-type objects, which provides the feelings of holding the product in his or her hands and touching buttons with his or her index fingertip in the AR setup. To improve user experience and performance in view of hardware configuration, we devise to adopt a simple and cost-effective AR setup that properly meets guidelines such as viewing size and distance, working posture, viewpoint matching, and camera movement. From experimental results, we found that the AR setup is good to improve the user experience and performance in design evaluation of handheld information appliances. We also found that the tangible interaction combined with the hand occlusion solver in the AR setup is very useful to improve tangible interaction and immersive visualization of virtual products while making the user experience the shapes and functions of the products well and comfortably.