Personalized gesture interactions for cyber-physical smart-home environments

A gesture-based interaction system for smart homes is a part of a complex cyber-physical environment, for which researchers and developers need to address major challenges in providing personalized gesture interactions. However, current research efforts have not tackled the problem of personalized gesture recognition that often involves user identification. To address this problem, we propose in this work a new event-driven service-oriented framework called gesture services for cyber-physical environments (GS-CPE) that extends the architecture of our previous work gesture profile for web services (GPWS). To provide user identification functionality, GS-CPE introduces a two-phase cascading gesture password recognition algorithm for gesture-based user identification using a two-phase cascading classifier with the hidden Markov model and the Golden Section Search, which achieves an accuracy rate of 96.2% with a small training dataset. To support personalized gesture interaction, an enhanced version of the Dynamic Time Warping algorithm with multiple gestural input sources and dynamic template adaptation support is implemented. Our experimental results demonstrate the performance of the algorithm can achieve an average accuracy rate of 98.5% in practical scenarios. Comparison results reveal that GS-CPE has faster response time and higher accuracy rate than other gesture interaction systems designed for smart-home environments.     

Intelligent virtual environments for virtual reality art

The development of virtual reality (VR) art installations is faced with considerable difficulties, especially when one wishes to explore complex notions related to user interaction. We describe the development of a VR platform, which supports the development of such installations, from an art+science perspective. The system is based on a CAVE™-like immersive display using a game engine to support visualisation and interaction, which has been adapted for stereoscopic visualisation and real-time tracking. In addition, some architectural elements of game engines, such as their reliance on event-based systems have been used to support the principled definition of alternative laws of Physics. We illustrate this research through the development of a fully implemented artistic brief that explores the notion of causality in a virtual environment. After describing the hardware architecture supporting immersive visualisation we show how causality can be redefined using artificial intelligence technologies inspired from action representation in planning and how this symbolic definition of behaviour can support new forms of user experience in VR.     

Realization of reconfigurable virtual environments for virtual testing

1 Introduction The scalability of virtual environments (VEs) that allow users to change and modify their contents and context depending on application requirements has been a de facto feature of virtual reality technology. Engineering uses such as virtual testing and exper- imenting applications demand VEs to be highly or fully reconfigurable. In reality, however, satisfactory reconfiguration of a virtual environment is yet to be achieved, due to the high accuracy of spatial data in- put and…     

Untethered gesture acquisition and recognition for virtual world manipulation

Humans use a combination of gesture and speech to interact with objects and usually do so more naturally without holding a device or pointer. We present a system that incorporates user body-pose estimation, gesture recognition and speech recognition for interaction in virtual reality environments. We describe a vision-based method for tracking the pose of a user in real time and introduce a technique that provides parameterized gesture recognition. More precisely, we train a support vector classifier to model the boundary of the space of possible gestures, and train Hidden Markov Models (HMM) on specific gestures. Given a sequence, we can find the start and end of various gestures using a support vector classifier, and find gesture likelihoods and parameters with a HMM. A multimodal recognition process is performed using rank-order fusion to merge speech and vision hypotheses. Finally we describe the use of our multimodal framework in a virtual world application that allows users to interact using gestures and speech.     

Video mosaics for virtual environments

As computer-based video becomes ubiquitous with the expansion of transmission, storage, and manipulation capabilities, it will offer a rich source of imagery for computer graphics applications. This article looks at one way to use video as a new source of high-resolution, photorealistic imagery for these applications. If you walked through an environment, such as a building interior, and filmed a video sequence of what you saw you could subsequently register and composite the video images together into large mosaics of the scene. In this way, you can achieve an essentially unlimited resolution. Furthermore, since you can acquire the images using any optical technology, you can reconstruct any scene regardless of its range or scale. Video mosaics can be used in many different applications, including the creation of virtual reality environments, computer-game settings, and movie special effects. I present algorithms that align images and composite scenes of increasing complexity-beginning with simple planar scenes and progressing to panoramic scenes and, finally, to scenes with depth variation. I begin with a review of basic imaging equations and conclude with some novel applications of the virtual environments created using the algorithms presented     

虚拟环境的用户意图捕获

目的虚拟制造环境中需要复杂精确的3D人机交互。目前的虚拟环境(VE)的主要问题是人在交互过程中的认知和操作负荷太重,交互效率亟需提高。解决此问题的重要途径是提高机器的认知能力。方法本文研究了用户意图的分析和抽取,并建立多通道用户意图理解的算法,以此来提高交互效率。结果结合虚拟装配应用给出了典型意图的实验结果并给予分析。通过实验对多通道意图的可用性和可靠性,以及基于意图系统的实时性进行了评估。实验是虚拟装配空间中用户拾取对象意图的实验。当3维鼠标和对象距离为5 000 mm时,传统系统中操作平均耗时5.344 7 s,而基于意图的系统中平均耗时2.326 6 s。基于意图的系统极大地降低了操作的时间和复杂度。结论采用意图驱动的系统情景转换能在虚拟环境工作中有效地降低人的认知负荷,并能很好地帮助系统开发者进行混成系统的建模和分析,降低开发的复杂度。实践结果表明用户意图理解的多通道模型和算法能极大地提高交互式系统的交互自然性和交互效率。该方法不仅适用于本文所用的虚拟装配系统,对于所有的虚拟环境应用场景都有同样的有效性。     

Navigational tools for desktop virtual environment interfaces

Desktop systems typically rely on a two-dimensional (2D) software interface and general purpose hardware (mouse, keyboard) for interaction with a three-dimensional (3D) virtual environment. These interfaces must provide all the functionality required to navigate through and interact with the virtual environment, yet research into the usability aspects of the tools presented on these software interfaces indicates that the majority of users experience some degree of frustration when using them to perform even relatively simple tasks. This paper begins with a study of usability issues for interfaces to virtual environments on desktop systems, and details a series of experiments performed to evaluate the usability of a number of navigational tools. Participants were tested on the time taken to complete a number of navigational tasks with a series of interfaces presenting different navigational tools. The tools presented were a speed control function, a you-are-here (YAH) map, a function enabling the user to mark and teleport to any location within the presented environment, and an undo function. Results indicate that the visual presentation of navigational aids improves navigation performance, in terms of the time taken to complete tasks, and also improves user satisfaction with the desktop system.     

Volumetric virtual environments

Driven by fast development of both virtual reality and volume visualization,we discuss some critical techniques towards building a volumetric VR system,specifically the modeling,rendering,and manipulations of a volumetric scene.Techniques such as voxel-based object simplification,accelerated volume rendering,fast stereo volume rendering,and volumetric “collision detection“ are introduced and improved,with the idea of demonstrating the possibilities and potential benefits of incorporating volumetric models into VR systems.     

Execution infrastructure for normative virtual environments

Virtual Institutions (VIs) have proven to be adequate to engineer applications where participants can be humans and software agents. VIs combine Electronic Institutions (EIs) and 3D Virtual Worlds (VWs). In this context, Electronic Institutions are used to establish the regulations that structure interactions and support software agent participation while Virtual Worlds facilitate human participation. In this paper we propose Virtual Institution eXEcution Environment (VIXEE) as an innovative communication infrastructure for VIs. Using VIXEE to connect Virtual Worlds and EI opens EI to humans, providing a fully operational and comprehensive environment. The main features of the infrastructure are (i) the causal connection between Virtual Worlds and Electronic Institutions, (ii) the automatic generation and update of the VIs' 3D visualization and (iii) the simultaneous participation of users from different virtual world platforms. We illustrate the execution of VIXEE system in a simple eAuction house example and use this example to evaluate the performance of our solution.     

Magic Lenses for augmented virtual environments

The authors developed a Magic Lens framework for Scape, an augmented virtual environment (AVE). They generalize the functional characteristics of Magic Lenses in terms of 3D visualization in AVEs and present two tangible Magic Lens-enabled devices with complementary interface capabilities. The authors also demonstrate their Magic Lens devices through testbed applications relevant to urban planning and medical training. This article explores an interface technique that manages viewing complexity and offers a set of versatile visualization capabilities in 3D augmented virtual environments (AVE).     

Rapid prototyping for distributed virtual environments

The authors combine rapid and exploratory prototyping techniques to capture initially unknown, interdependent requirements in the development of distributed virtual environment applications. They employ containerization to accelerate data exchange     

Priority scheduling for networked virtual environments

When simulating large virtual environments (VEs), contention for limited resources such as the CPU, rendering pipeline, or network bandwidth frequently degrades the system's performance. Whenever such a competition occurs and not all elements that require the resource can be serviced, an approximation must be made to avoid compromising interactive performance. We propose an enhancement to the round-robin approach called Priority Round Robin scheduling (PRR). This algorithm enforces priorities, while retaining the output sensitivity and starvation-free performance of round-robin scheduling. Priorities are set by a user-defined error metric (such as visual error), which the algorithm attempts to minimize. This permits not only scheduling the entities competing for a resource such as updates competing for the network, but also filling the gap between filtering and scheduling techniques. We evaluate our algorithm in a client-server system     

Low-cost telepresence for collaborative virtual environments

We present a novel low-cost method for visual communication and telepresence in a CAVEtrade-like environment, relying on 2D stereo-based video avatars. The system combines a selection of proven efficient algorithms and approximations in a unique way, resulting in a convincing stereoscopic real-time representation of a remote user acquired in a spatially immersive display. The system was designed to extend existing projection systems with acquisition capabilities requiring minimal hardware modifications and cost. The system uses infrared-based image segmentation to enable concurrent acquisition and projection in an immersive environment without a static background. The system consists of two color cameras and two additional b/w cameras used for segmentation in the near-IR spectrum. There is no need for special optics as the mask and color image are merged using image-warping based on a depth estimation. The resulting stereo image stream is compressed, streamed across a network, and displayed as a frame-sequential stereo texture on a billboard in the remote virtual environment     

Coordination components for collaborative virtual environments

This paper deals with the behavior of virtual environments from the collaboration point-of-view, in which actors (human or virtual beings) interact and collaborate by means of interdependent tasks. In this sense, actors may realize tasks that are dependent on tasks performed by other actors, while the interdependencies between tasks (through resource management and temporal relations) delineate the overall behavior of a virtual environment. Our main goal is to propose an approach for the coordination of those behaviors. Initially a generic study of possible interdependencies between collaborative tasks is presented, followed by the formal modeling (using Petri Nets) of coordination mechanisms for those dependencies. In order to implement such mechanisms, an architecture of reusable and pluggable coordination components is also introduced. These components are used in an implementation of a multi-user videogame. The presented approach is a concrete step to create virtual societies of actors that collaborate to reach common goals without the risk of getting involved in conflicting or repetitive tasks.     

Multicast issues for collaborative virtual environments

Multicast networking lets designers use new communication patterns in computer graphics applications. This article introduces multicasting, explains how it differs from more traditional networking, and describes multicasting issues for the application designer     

Topology simplification for polygonal virtual environments

We present a topology simplifying approach that can be used for genus reductions, removal of protuberances, and repair of cracks in polygonal models in a unified framework. Our work is complementary to the existing work on geometry simplification of polygonal datasets and we demonstrate that using topology and geometry simplifications together yields superior multiresolution hierarchies than is possible by using either of them alone. Our approach can also address the important issue of repair of cracks in polygonal models, as well as for rapid identification and removal of protuberances based on internal accessibility in polygonal models. Our approach is based on identifying holes and cracks by extending the concept of α-shapes to polygonal meshes under the L_∞ distance metric. We then generate valid triangulations to fill them using the intuitive notion of sweeping an L_∞ cube over the identified regions     

Rigid body cable for virtual environments

The present paper addresses real-time simulation of cables for virtual environments. A faithful physical model based on constrained rigid bodies is introduced and discretized. The performance and stability of the numerical method are analyzed in details and found to meet the requirements of interactive heavy hoisting simulations. The physical model is well behaved in the limit of infinite stiffness as well as in the elastic regime, and the tuning parameters correspond directly to conventional material constants. The integration scheme mixes the well known St?rmer-Verlet method for the dynamics equations with the linearly implicit Euler method for the constraint equations and enables physical constraint relaxation and stabilization terms. The technique is shown to have superior numerical stability properties in comparison with either chain link systems, or spring and damper models. Experimental results are presented to show that the method results in stable, real-time simulations. Stability persists for moderately large fixed integration step of Delta t = 1/60 s, with hoisting loads of up to 10(5) times heavier than the elements of the cable. Further numerical experiments validating the physical model are also presented.