一种高保真视-触觉增强现实系统的搭建

随着增强现实技术的发展,仅包含视觉的增强现实技术已经不能满足人们的需求,而加入触觉信息形成的视-触觉融合的增强现实系统却越来越受到关注。触觉设备是将真实环境和虚拟环境连接起来的纽带。然而,触觉设备在视场中的存在会带来两个问题,一方面触觉设备占据了很大的视觉空间,另一方面触觉设备的定位往往会出现较大的误差,导致了视-触觉增强现实的真实性下降。为了增加使用者的体验感,提出一种高保真视-触觉增强现实环境的搭建方法。首先基于改进的ORB(Oriented FAST and Rotated BRIEF)和KLT(Kanade-Lucas-Tomasi)增强现实跟踪注册算法搭建稳定的视觉增强现实环境。其次提出了一种能校准触笔位置和方向的方法,以提升交互体验。然后提出一种基于力反馈的触觉渲染算法,最后使用改进的全局泊松方程和Criminisi算法对触觉设备进行隐藏和修复,减小因触觉设备存在对用户沉浸感造成的影响。     

基于OpenGL ES的移动增强现实虚实融合

传统移动增强现实系统中虚拟模型和场景之间存在明显的视觉差异,不能够给予用户良好、逼真的真实体验在基于移动增强现实系统中使用OpenGL ES技术来实现高逼真度的虚实场景融合,研究内容主要包括三维虚拟模型的解析、渲染及管理和虚实融合场景图层设计的关系,最终完成高渲染性、高融合度的虚实融合场景的构建.实验结果表明,在OpenGL ES环境下,可以将三维虚拟物体注册到真实场景中,达到了虚实融合的效果,为用户展示出高沉浸度、无缝的视觉体验.     

基于VRML的虚拟交互系统触摸感知控制仿真研究

针对虚拟交互系统触摸感知控制过程中图像存在噪声干扰,导致触摸感知效果下降的问题,提出了基于VRML的虚拟交互系统触摸感知控制方法。将初始的三维立体空间图像转换为二维平面空间图像,将不间断的图像加以变换,得到离散型的数据,通过图像中不同像素点的平均值来取代该点的灰度值,得到无噪声数字图像。针对图像中任意像素点,对其边缘点进行检测,完成边缘图像细节信息识别获取。在上述基础上,对目标图像进行轮廓提取,采用改进后的归一化傅里叶描述子对经过去噪处理的图像进行三维识别,利用VRML重建识别结果,实现虚拟交互系统触摸感知。通过具体的仿真数据充分验证了所提方法的综合有效性。     

空间增强现实中的人机交互技术综述

空间增强现实是增强现实的三大形式之一,它利用投影等方式在物理空间中增强显示虚拟内容.在空间增强现实系统中,用户可以在物理空间与虚拟内容交互,具有良好的环境感知、空间感知与协同感知,交互任务执行过程中认知负荷低.首先以空间增强现实显示方式为例,通过其系统架构介绍空间增强现实的技术原理;然后将空间增强现实中的人机交互技术分类为基于触控的交互、基于手势或姿势的交互、基于控制器的交互、手持投影仪交互、实物交互及多模态交互并进行分析,对相关技术在三维配准、虚实融合等方面带来的挑战进行阐述;同时总结了空间增强现实在运动锻炼、医疗辅助、教育娱乐、艺术等领域的应用状况;最后对空间增强现实中的交互技术发展趋势进行了展望,指出未来要克服现有技术在稳定性、准确性和实时性上存在的缺陷,在交互方式更自然、交互遮挡处理方法更智能、系统使用场景更广泛、与社会文化和心理现象研究相结合以及新型成像方式等方面得到突破和发展.     

增强现实中三维跟踪注册技术概述

三维跟踪注册技术是增强现实领域中的重要关键技术，通过对现实场景中的图像或物体进行跟踪与定位，将虚拟物体按照正确的空间透视关系叠加到真实场景当中。综述了增强现实中的三维跟踪注册技术（基于计算机视觉的跟踪注册技术、基于硬件传感器的跟踪注册技术以及混合跟踪注册技术），详细阐述了不同跟踪注册技术的优势与不足，并介绍了不同跟踪注册技术的当前应用现状，随后探讨了跟踪注册技术的发展趋势与存在的难题，对三维跟踪注册技术的进一步研究进行了展望。     

基于环境映照自动对齐的高质量虚实融合技术

要实现高品质的增强现实效果需要解决虚拟物体与现实场景的光照一致性问题.虽然采用HDR技术能获取场景的环境映照,但需要解决所获取的光照环境信息与真实场景的对齐问题.为此提出一种基于特征自动匹配的环境映照对齐方法.首先采用Affine-SIFT算法和随机抽样一致性算法对环境映照和拍摄场景进行特征匹配并优化匹配结果,然后利用基于运动推断结构的摄像机定标算法求得匹配对的三维位置,从而计算出环境映照与真实场景的对应关系,实现了二者的自动对齐.基于该技术搭建的高真实感实时虚实融合系统采用基于关键帧的相机跟踪技术,可以实时地将虚拟物体注册到拍摄的视频场景里,并允许对其进行实时的交互编辑.在渲染时有效地利用了自动对齐后的光照环境信息,采用重要性采样算法和阴影映射技术实现了实时的高质量渲染.实验结果表明,所搭建的增强现实系统很好地解决了实时虚实融合中的几何一致性和光照一致性问题.     

面向磁悬浮视触觉交互的多速率系统框架

非接触式磁悬浮视触觉交互克服了机械式交互的固有摩擦,具有广阔应用前景,但存在交互过程中虚拟工具穿透物体、图形渲染与触觉渲染速率不一致等问题。针对上述问题,提出面向磁悬浮视触觉交互的多速率系统框架,通过扩展三自由度（3-DOF）单射线触觉渲染方法,利用多射线对虚拟工具进行建模,避免工具穿透,实现六自由度（6-DOF）触觉渲染;通过多速率并行,实现速率不一致模块间相互协同;通过构建映射滤波算法,实现视觉定位数据到虚拟工具位姿的稳定映射。实验结果表明,该系统能有效避免交互过程中的穿透现象,并提供稳定、真实的视触觉反馈。     

基于三维虚拟视觉的高校图书馆导视系统设计

针对高校图书馆导视系统响应时间长、图像重建效率低等问题，提出基于三维虚拟视觉的高校图书馆导视系统设计。首先获取高校图书馆空间数据参数，构建实景CAD平面模型；采用增强现实成像技术合并显示实景与虚拟信息，通过Cook Torrance光照模型构造虚拟结构的全局光照效果；最后采用基于信息反馈范围与方式的交互功能设计方法，结合Virtools软件整合三维虚拟视觉图像与相关标识、声音文件，通过液晶显示屏或手机APP等方式实现用户与导视系统间的交互。实验结果表明，系统界面转换时间控制在1 s以内，视觉模型清晰度更高，可更好地提升用户的交互体验。     

基于微机的主从式机器人场景三维可视化与仿真

文章提出了利用触觉传感信息和视频图像相融合来构造主从式机器人三维数据场的思路,并且在微机上实际构造了仿真环境,文章讨论了基于触觉的三维建模、坐标系转换、融合及绘制等问题,最后通过实际例子展示了三维场景的效果。     

面向ERT大面积触觉传感的自适应优化成像方法

在动态的非结构化环境中,有效地感知物理接触对于智能机器人安全交互至关重要。为了能够检测各种潜在的物理交互,需要在机器人表面部署大面积触觉传感器。目前,现有的大面积触觉传感器主要是通过传感阵列方式实现的,但是大规模部署传感元件在实际应用中存在巨大挑战。电阻层析成像(Electrical Resistance Tomography, ERT)技术作为一种连续传感方式,有望克服传统触觉传感阵列的一些限制。为此,利用ERT设计了一款新型的大面积触觉传感器。在此基础上,提出了一种基于自适应感兴趣区(Region of Interest, ROI)的图像重构算法,将图像重构限制在交互区域内,以提高传感器的空间分辨率。为了验证提出成像算法的有效性,通过仿真与物理实验对其进行了全面评估。实验验证了该算法可以有效提高触觉传感器在交互区域的空间分辨率,使其具有较高的测量精度。实验结果表明,该传感器的平均定位误差为0.823 cm,能够准确地识别8种不同交互模式,其精度高达98.6%。这一研究工作表明,该传感器为机器人具身触觉传感的实现提供了一个新的解决方案。     

Object interaction and task programming by demonstration in visuo-haptic augmented reality

A visuo-haptic augmented reality system is presented for object manipulation and task learning from human demonstration. The proposed system consists of a desktop augmented reality setup where users operate a haptic device for object interaction. Users of the haptic device are not co-located with the environment where real objects are present. A three degrees of freedom haptic device, providing force feedback, is adopted for object interaction by pushing, selection, translation and rotation. The system also supports physics-based animation of rigid bodies. Virtual objects are simulated in a physically plausible manner and seem to coexist with real objects in the augmented reality space. Algorithms for calibration, object recognition, registration and haptic rendering have been developed. Automatic model-based object recognition and registration are performed from 3D range data acquired by a moving laser scanner mounted on a robot arm. Several experiments have been performed to evaluate the augmented reality system in both single-user and collaborative tasks. Moreover, the potential of the system for programming robot manipulation tasks by demonstration is investigated. Experiments show that a precedence graph, encoding the sequential structure of the task, can be successfully extracted from multiple user demonstrations and that the learned task can be executed by a robot system.     

虚拟体空间中的触觉雕刻

目前，在虚拟环境中大多数的信息获取是通过视觉、听觉等非接触感觉获得的。然而缺乏触觉反馈的信息减少了很大一部分的信息源。在看和听之外，能够触摸、感觉和操纵物体，在很大程度上提高了虚拟环境的真实性。该文研究了触觉绘制的基本模型，提出了采用虚平面作为中介实现体数据的实时触觉绘制。并在此基础上探讨了体的局部变形及结合触觉反馈模型，实现了具有触觉反馈的虚拟雕刻交互系统。该系统可应用于融化、燃烧、印记、构造和着色实时交互操作。     

Interactive 3D content insertion in images for multimedia applications

This article addresses the problem of creating interactive mixed reality applications where virtual objects interact in images of real world scenarios. This is relevant to create games and architectural or space planning applications that interact with visual elements in the images such as walls, floors and empty spaces. These scenarios are intended to be captured by the users with regular cameras or using previously taken photographs. Introducing virtual objects in photographs presents several challenges, such as pose estimation and the creation of a visually correct interaction between virtual objects and the boundaries of the scene. The two main research questions addressed in this article include, the study of the feasibility of creating interactive augmented reality (AR) applications where virtual objects interact in a real world scenario using the image detected high-level features and, also, verifying if untrained users are capable and motivated enough to perform AR initialization steps. The proposed system detects the scene automatically from an image with additional features obtained using basic annotations from the user. This operation is significantly simple to accommodate the needs of non-expert users. The system analyzes one or more photos captured by the user and detects high-level features such as vanishing points, floor and scene orientation. Using these features it will be possible to create mixed and augmented reality applications where the user interactively introduces virtual objects that blend with the picture in real time and respond to the physical environment. To validate the solution several system tests are described and compared using available external image datasets.     

Augmented Reality with Back-Projection Systems using Transflective Surfaces

In this paper, we introduce the concept of Extended VR (extending viewing space and interaction space of back-projection VR systems), by describing the use of a hand-held semi-transparent mirror to support augmented reality tasks with back-projection systems. This setup overcomes the problem of occlusion of virtual objects by real ones linked with such display systems. The presented approach allows an intuitive and effective application of immersive or semi-immersive virtual reality tasks and interaction techniques to an augmented surrounding space. Thereby, we use the tracked mirror as an interactive image-plane that merges the reflected graphics, which are displayed on the projection plane, with the transmitted image of the real environment. In our implementation, we also address traditional augmented reality problems, such as real-object registration and virtual-object occlusion. The presentation is complemented by a hypothesis of conceivable further setups that apply transflective surfaces to support an Extended VR environment.     

Haptic Direct-Drive Robot Control Scheme in Virtual Reality

This paper explores the use of a 2-D (Direct-Drive Arm) manipulator for mechanism design applications based on virtual reality (VR). This article reviews the system include a user interface, a simulator, and a robot control scheme. The user interface is a combination of a virtual clay environment and human arm dynamics via robot effector handler. The model of the VR system is built based on a haptic interface device behavior that enables the operator to feel the actual force feedback from the virtual environment just as s/he would from the real environment. A primary stabilizing controller is used to develop a haptic interface device where realistic simulations of the dynamic interaction forces between a human operator and the simulated virtual object/mechanism are required. The stability and performance of the system are studied and analyzed based on the Nyquist stability criterion. Experiments on cutting virtual clay are used to validate the theoretical developments. It was shown that the experimental and theoretical results are in good agreement and that the designed controller is robust to constrained/unconstrained environment.     

增强现实环境下的产品装配引导技术

以增强现实环境下引导产品装配为目标,建立了面向增强装配过程的统一 信息模型,管理文字、几何和产品装配特征等可视化引导信息。采用基于标志物的视觉跟踪 技术实现虚拟零件和视频中真实零件的注册定位,通过建立虚拟和真实装配场景的深度图处 理增强装配场景中虚实物体的遮挡关系。利用虚实零件的注册位置把装配引导信息叠加到装 配视频场景中。并开发了演示系统,分析和说明了增强现实环境下引导产品进行装配的过程。     

Film and television industry cloud exhibition design based on 3D imaging and virtual reality

The three-dimensional virtual scene can provide users with a visual three-dimensional virtual environment, with various multimedia channels such as sound, video, force feedback equipment, etc., to bring users a completely immersive interactive experience. This paper introduces 3D imaging and virtual reality technology in the film and television industry cloud exhibition, and develops a virtual display platform. First of all, this paper divides the registration into two processes: camera calibration and joint calibration of the camera and laser scanner based on the calibration results. Camera calibration can determine the plane model of the calibration board in the camera coordinate system, and the joint calibration uses the RANSAC algorithm to extract the point cloud of the plane model of the calibration board, and then optimizes the distance between the points in the plane model point cloud and the corresponding plane in the camera coordinate system Find the optimal transformation between the two sets of data, and then calculate the registration relationship between the point cloud and the image. Secondly, this article conducts a demand analysis of the film and television industry cloud exhibition platform based on virtual reality technology, including the business goals set by the platform, platform system analysis, overall design, and system operating environment and configuration requirements. This model provides a feasible solution for the visual interaction of the cloud exhibition design of the film and television industry.     

虚拟现实的力触觉交互技术发展现状与趋势

虚拟现实通过模拟人的视觉、听觉、力触觉等,使人处于一种与真实世界非常逼真的虚拟世界中,来感受、体验和评价虚拟世界中的场景和设备。力触觉在虚拟现实环境中有其突出优越性,力触觉使得虚拟现实环境变得真实,是唯一的既可接受周围环境输入又可以对周围环境输出的感知通道,可极大增强可视化表达的效果。虚拟现实的力触觉交互技术包括力触觉再现技术和虚拟环境的力触觉建模。介绍了这两方面的发展现状、存在的问题和今后的发展趋势。     

基于视觉标记的增强现实系统建模及配准误差问题研究

针对当前基于视觉的增强现实系统标记大、成本高的缺陷,提出一套结合简单组合标记的增强现实系统的实现方案.基于弱透视模型,以色彩和形状作为融合特征,研究了利用冗余特征实现标记物体的3D位姿还原方法.为使配准方案能够适应普通的视觉采集设备,提出根据摄像机标定时得到的扭曲参数对渲染后的模型进行再扭曲的配准修正方法.结合OpenCV和OpenGL中的空间变换的概念,给出了虚拟和物理世界配准融合的解决方案,并基于统计学数学模型,提出降低配准时的误差以及提升配准稳定性的方案.实验结果表明,提出的算法能有效地将静态标签的抖动降低至亚像素级.     

Modelling and design of an augmented reality differential drive mobility aid in an enabled environment

In this paper the authors discuss the modelling and design of an augmented reality platform for disabled wheeled mobility studies. This design consists of a virtual environment with two degrees of freedom motion platform and integrated ground contact force feedback. Differential drive mobility users continue to be in touch with reality on their own mobility aid while interacting with virtual objects. The major domain related to the differential drive mobility of the disabled members of society which coincides with the use of manual wheelchairs, electric wheelchairs and mobility scooters. In order to account for environmental and dynamic effects, the wheeled mobility user needs to map the intended trajectory into the virtual world. Motion and inertia force feedback produced on the augmented simulator give the users a haptic sensory stimulus input regarding spatial movement and ground contact forces. The main objective is to model and design an augmented reality platform with real world kinematic and dynamic properties to place a wheeled mobility user closer to real world encounters. The use of the designed augmented reality platform will be beneficial to disabled wheeled mobility users in need of occupational therapist training and evaluation.