A Flexible Architecture for Virtual Humans in Networked Collaborative Virtual Environments

Complex virtual human representation provides more natural interaction and communication among participants in networked virtual environments, hence it is expected to increase the sense of being together within the same virtual world. We present a flexible framework for the integration of virtual humans in networked collaborative virtual environments. A modular architecture allows flexible representation and control of the virtual humans, whether they are controlled by a physical user using all sorts of tracking and other devices, or by an intelligent control program turning them into autonomous actors. The modularity of the system allows for fairly easy extensions and integration with new techniques making it interesting also as a testbed for various domains from “classic” VR to psychological experiments. We present results in terms of functionalities, example applications and measurements of performance and network traffic with an increasing number of participants in the simulation.     

Hybrid client-server architecture and control techniques for collaborative product development using haptic interfaces

In this paper, a collaborative product development and prototyping framework is proposed by using distributed haptic interfaces along with deformable objects modeling. Collaborative Virtual Environment (CVE) is a promising technique for industrial product development and virtual prototyping. Network control problems such as network traffic and network delay in communication have greatly limited collaborative virtual environment applications. The problems become more difficult when high-update-rate haptic interfaces and computation intensive deformable objects modeling are integrated into CVEs for intuitive manipulation and enhanced realism. A hybrid network architecture is proposed to balance the computational burden of haptic rendering and deformable object simulation. Adaptive artificial time compensation is used to reduce the time discrepancy between the server and the client. Interpolation and extrapolation approaches are used to synchronize graphic and haptic data transmitted over the network. The proposed techniques can be used for collaborative product development, virtual assembly, remote product simulation and other collaborative virtual environments where both haptic interfaces and deformable object models are involved.     

Research on hybrid synchronization methods in multi-user collaborative VR simulation medical surgery training system

The simulation training system based on VR technology has been applied in clinical medicine due to its immersive interactive experience, economy, and safety. In response to the limitation that most medical surgery training systems can only achieve single user operations, a hybrid synchronous model based multi-user collaborative framework design is proposed to meet the needs of collaborative training in virtual reality teaching processes. Based on the characteristics of frame synchronization and state synchronization, the synchronization method has been improved. Based on frame synchronization, specific state synchronization is used on interactive events to improve the consistency of operation details and real-time action. On this basis, a system prototype has been implemented using Unity 3D, and compared with traditional schemes, analysis shows that it has faster response speed for disconnected reconnection and more stable interaction state.     

Modeling and performance analysis using extended fuzzy-timing Petrinets for networked virtual environments

Despite their attractive properties, networked virtual environments (net-VEs) are notoriously difficult to design, implement, and test due to the concurrency, real-time and networking features in these systems. Net-VEs demand high quality-of-service (QoS) requirements on the network to maintain natural and real-time interactions among users. The current practice for net-VE design is basically trial and error, empirical, and totally lacks formal methods. This paper proposes to apply a Petri net formal modeling technique to a net-VE-NICE (narrative immersive constructionist/collaborative environment), predict the net-VE performance based on simulation, and improve the net-VE performance. NICE is essentially a network of collaborative virtual reality systems called the CAVE-(CAVE automatic virtual environment). First, we introduce extended fuzzy-timing Petri net (EFTN) modeling and analysis techniques. Then, we present EFTN models of the CAVE, NICE, and transport layer protocol used in NICE: transmission control protocol (TCP). We show the possibility analysis based on the EFTN model for the CAVE. Then, by using these models and design/CPN as the simulation tool, we conducted various simulations to study real-time behavior, network effects and performance (latencies and jitters) of NICE. Our simulation results are consistent with experimental data.     

VSculpt : a distributed virtual sculpting environment for collaborative design

A collaborative virtual sculpting system supports a team of geographically separated designers/engineers connected by networks to participate in designing three-dimensional (3D) virtual engineering tools or sculptures. It encourages international collaboration at a minimal cost. However, in order for the system to be useful, two factors need to be addressed: intuitiveness and real-time interaction. Although a lot of effort has been put into developing virtual sculpting environments, only limited work addresses collaborative virtual sculpting. This is because in order to support real-time collaborative virtual sculpting, many challenging issues need to be addressed. We propose a collaborative virtual sculpting framework, called VSculpt. Through adapting some techniques we developed earlier and integrating them with some techniques developed here, the proposed framework provides a real-time intuitive environment for collaborative design. In particular, it addresses issues on efficient rendering and transmission of deformable objects, intuitive object deformation using the CyberGlove and concurrent object deformation by multiple clients. We demonstrate and evaluate the performance of the proposed framework through a number of experiments.     

Performance evaluation of compromised synchronization control mechanism for distributed virtual environment (DVE)

Synchronization in a distributed virtual environment (DVE) involves mechanisms to ensure a consistent view of a virtual world for all participants. Most applications in the DVE are related to collaborative activities that include non-contention and contention cases. Using transmission of update messages is suitable enough to support synchronization for only non-contention activity. The contention activity requires an additional mechanism to control accessing a common object for synchronization. In this paper, we present the compromised synchronization control mechanism to support both non-contention and contention activities. The mechanism employs frequent update event and multiple-lock checking to control the synchronization. Frequent update event is used to support a dynamic virtual world for non-contention activity. Multiple-lock checking is embedded to ensure consistency when accessing the common object is required simultaneously for the contention event. Performance measurement of the compromised synchronization is provided by simulation in terms of locking time, sampling event, number of logical processes, and traffic tolerance. Prototype application is also implemented to compare the result in a small scale level. Based on the simulation and experimental results, the compromised sychronization control mechanism is capable to support up to 100 participants for the non-contention activity. It provides a good performance of supporting the contention activity in a small scale. The mechanism is considered suitable for collaborative application where contention is considered a critical event.     

Simultaneous remote haptic collaboration for assembling tasks

Stand-alone virtual environments (VEs) using haptic devices have proved useful for assembly/disassembly simulation of mechanical components. Nowadays, collaborative haptic virtual environments (CHVEs) are also emerging. A new peer-to-peer collaborative haptic assembly simulator (CHAS) has been developed whereby two users can simultaneously carry out assembly tasks using haptic devices. Two major challenges have been addressed: virtual scene synchronization (consistency) and the provision of a reliable and effective haptic feedback. A consistency-maintenance scheme has been designed to solve the challenge of achieving consistency. Results show that consistency is guaranteed. Furthermore, a force-smoothing algorithm has been developed which is shown to improve the quality of force feedback under adverse network conditions. A range of laboratory experiments and several real trials between Labein (Spain) and Queen’s University Belfast (Northern Ireland) have verified that CHAS can provide an adequate haptic interaction when both users perform remote assemblies (assembly of one user’s object with an object grasped by the other user). Moreover, when collisions between grasped objects occur (dependent collisions), the haptic feedback usually provides satisfactory haptic perception. Based on a qualitative study, it is shown that the haptic feedback obtained during remote assemblies with dependent collisions can continue to improve the sense of co-presence between users with regard to only visual feedback.     

基于协同虚拟环境的装配设计系统

针对分布式环境中复杂产品协同开发的装配设计需求,研究基于协同虚拟环境的装配设计系统的体系结构和关键技术,提出分布式条件下支持多用户实时交互操作的协同虚拟装配方法,包括复杂场景高分辨率实时渲染和多通道沉浸输出方案。以此为基础开发分布式协同虚拟装配环境,并通过面向汽车整车的多用户协同虚拟装配实例验证了系统的有效性。结果证明,用户可在该系统中完成复杂产品的协同虚拟装配设计工作。     

基于上下文的分布式协同过滤推荐技术

针对传统协同过滤推荐技术应用于大规模动态数据集时难以兼顾准确度和效率的问题,提出一种基于上下文的分布式协同过滤推荐技术,引入推荐上下文的概念,并在此基础上充分考虑用户的即时兴趣以提高推荐的准确度,采用评分矩阵的分布式存储和计算以提高推荐的效率。实验结果表明,该分布式协同过滤技术能同时保证推荐的准确度和效率,使其在大规模动态数据集上的应用更具优势。     

网络环境下协同虚拟拆卸训练平台

为实现复杂装备拆卸的多人协同工作,规划协同拆卸虚拟训练的工作流程以及网络环境下多人协同拆卸虚拟训练平台的体系框架;采用基于拆卸混合图的协同拆卸串、并行操作的管理方法,并分析协同拆卸过程中的多维拆卸信息,给出各种拆卸信息的协同可视化方法;协同拆卸虚拟训练原型系统的工作过程及其有效性通过示例进行说明并得到验证。     

Decentralized data access control over consortium blockchains

Blockchain is an emerging data management technology that enables people in a collaborative network to establish trusted connections with the other participants. Recently consortium blockchains have raised interest in a broader blockchain technology discussion. Instead of a fully public, autonomous network, consortium blockchain supports a network where participants can be limited to a subset of users and data access strictly controlled. Access control policies should be defined by the respective data owner and applied throughout the network without requiring a centralized data administrator. As a result, decentralized data access control (DDAC) emerges as a fundamental challenge for such systems. However, we show from a trust model for consortium collaborative networks that current consortium blockchain systems provide limited support for DDAC. Further, the distributed, replicated nature of blockchain makes it even more challenging to control data access, especially read access, compared with traditional DBMSes. We investigate possible strategies to protect data from being read by unauthorized users in consortium blockchain systems using combinations of ledger partitioning and encryption strategies. A general framework is proposed to help inexperienced users determine appropriate strategies under different application scenarios. The framework was implemented on top of Hyperledger Fabric to evaluate feasibility. Experimental results along with a real-world case study contrasted the performance of different strategies under various conditions and the practicality of the proposed framework.     

Collaborative virtual geographic environments: A case study of air pollution simulation

The integration of high dimensional geo-visualization, geo-data management, geo-process modeling and computation, geospatial analysis, and geo-collaboration is a trend in GIScience. The technical platform that matches the trend forms a new framework unlike that of GIS and is conceptualized in this paper as a collaborative virtual geographic environment (CVGE). This paper focuses on two key issues. One is scientific research on CVGE including the concept definition and the conceptual and system framework development. The other is a prototype system development according to CVGE frameworks for air pollution simulation in the Pearl River Delta. The prototype system integrates air pollution source data, air pollution dispersion models, air pollution distribution/dispersion visualization in geographically referenced environments, geospatial analysis, and geo-collaboration. Using the prototype system, participants from geographically distributed locations can join in the shared virtual geographic environment to conduct collaborative simulation of air pollution dispersion. The collaborations supporting this simulation happen on air pollution source editing, air pollution dispersion modeling, geo-visualization of the output of the modeling, and geo-analysis.     

An HCI method to improve the human performance reduced by local-lag mechanism

Local-lag mechanism can maintain consistency for replicated continuous applications, but with a tradeoff of adding delay to local operations. To relieve the negative effects of the delay, this paper proposes an HCI method named echo. With the help of the echo method users can immediately perceive the results of their operations and how large the lag is. In order to evaluate the proposed method, a desktop collaborative virtual environment (CVE) system and a virtual object control task were employed to study the effects of the echo method on human performance (including task completion time, error count, and interaction quality). Experimental results indicate that when the lag exceeds 100 ms the echo method can improve human performance with the effects becoming more evident when a larger lag is used.     

Dynamic Verification of Memory Consistency in Cache-Coherent Multithreaded Computer Architectures

Multithreaded servers with cache-coherent shared memory are the dominant type of machines used to run critical network services and database management systems. To achieve the high availability required for these tasks, it is necessary to incorporate mechanisms for error detection and recovery. Correct operation of the memory system is defined by the memory consistency model. Errors can therefore be detected by checking if the observed memory system behavior deviates from the specified consistency model. Based on recent work, we design a framework for dynamic verification of memory consistency (DVMC). The framework consists of mechanisms to verify three invariants that are proven to guarantee that a specified memory consistency model is obeyed. We describe an implementation of the framework for the SPARCv9 architecture, and we experimentally evaluate its performance using full-system simulation of commercial workloads.     

Hybrid and forward error correction transmission techniques for unreliable transport of 3D geometry

The continual improvement in computer performance together with the prevalence of high-speed network connections having high throughput and moderate latencies enables the deployment of multimedia applications, such as collaborative virtual environments, over wide area networks (WANs). These applications can serve as simulated environments in scenarios such as emergency response training to catastrophic disasters, military training, and entertainment. Many of these systems use 3D graphics for display and may be required to distribute geometric models on demand between participants. Progressive meshes provide an attractive mechanism for such distribution. Previous uses of progressive meshes have sent data using reliable protocols (TCP). However, such protocols have disadvantages in on-demand settings, in that they: (1) use flow control, which limits performance in WANs; (2) add additional bandwidth when there is loss; (3) treat all loss as an indication of congestion; and (4) require feature-rich multicast support, which is not always available. In this paper, we modify progressive mesh models to allow reconstruction even in the event of packet loss. We use these modifications in two transmission schemes, a hybrid transmission that uses TCP and UDP to send packets and a forward error correction transmission scheme that uses redundancy to decode the information sent. We assess the performance of these transmission schemes when deployed on network testbeds that simulate wide area and wireless characteristics.Published online: 9 February 2005 Correspondence to : Bobby Bodenheimer     

A collaborative benchmarking framework for multibody system dynamics

Despite the importance given to the computational efficiency of multibody system (MBS) simulation tools, there is a lack of standard benchmarks to measure the performance of these kinds of numerical simulations. This works proposes a collaborative benchmarking framework to measure and compare the performance of different MBS simulation methods. The framework is made up of two main components: (a) an on-line repository of test problems with reference solutions and standardized procedures to measure computational efficiency and (b) a prototype implementation of a collaborative web-based application to collect, organize and share information about performance results in an intuitive and graphical form. The proposed benchmarking framework has been tested to evaluate the performance of a commercial MBS simulation software, and it proved to be an effective tool to collect and analyze information about the numerous factors which affect the computational efficiency of dynamic simulations of multibody systems.     

Computer-based virtual reality simulator for phacoemulsification cataract surgery training

Recent research in virtual reality indicates that computer-based simulators are an effective technology to use for surgeons learning to improve their surgical skills in a controlled environment. This article presents the development of a virtual reality simulator for phacoemulsification cataract surgery training, which is the most common surgical technique currently being used to remove cataracts from the patient’s eyes. The procedure requires emulsifying the cloudy natural lens of the eye and restoring vision by implanting an artificial lens through a small incision. The four main procedures of cataract surgery, namely corneal incision, capsulorhexis, phacoemulsification, and intraocular lens implantation, are incorporated in the simulator for virtual surgical training by implementing several surgical techniques. The surgical activity that are applied on the anatomy of the human eye, such as incision, grasping, tearing, emulsification, rotation, and implantation, are simulated in the system by using different types of mesh modifications. A virtual reality surgical simulator is developed, and the main procedures of phacoemulsification cataract surgery are successfully simulated in the system. The simulation results of the training system show that the developed simulator is capable of generating a virtual surgical environment with faithful force feedback for medical residents and trainees to conduct their training lessons via the computer using a pair of force-feedback haptic devices. In addition, the successful simulation of the mesh modifications on the human eyeball with visual realism and faithful force feedback throughout the surgical operation shows that the developed simulator is able to serve as a virtual surgical platform for surgeons to train their surgical skills.     

Teaching your system to share

Creating multi-user virtual worlds is a relatively new challenge for developers of virtual reality applications. It is especially challenging to build networked VR systems based on inexpensive PC-based hardware, due to limitations in system software and inefficient networking hardware. This article explains how we created a multi-person shared virtual environment using DOS-based PCs. It also covers some of the design goals in creating a collaborative design tool for creating and modifying virtual buildings. Workroom is a general purpose simulation infrastructure that allows multiple participants to interact in unpredictable ways while minimizing simulation latency     

Mobile Fog Computing by Using SDN/NFV on 5G Edge Nodes

Fog computing provides quality of service for cloud infrastructure. As the data computation intensifies, edge computing becomes difficult. Therefore, mobile fog computing is used for reducing traffic and the time for data computation in the network. In previous studies, software-defined networking (SDN) and network functions virtualization (NFV) were used separately in edge computing. Current industrial and academic research is tackling to integrate SDN and NFV in different environments to address the challenges in performance, reliability, and scalability. SDN/NFV is still in development. The traditional Internet of things (IoT) data analysis system is only based on a linear and time-variant system that needs an IoT data system with a high-precision model. This paper proposes a combined architecture of SDN and NFV on an edge node server for IoT devices to reduce the computational complexity in cloud-based fog computing. SDN provides a generalization structure of the forwarding plane, which is separated from the control plane. Meanwhile, NFV concentrates on virtualization by combining the forwarding model with virtual network functions (VNFs) as a single or chain of VNFs, which leads to interoperability and consistency. The orchestrator layer in the proposed software-defined NFV is responsible for handling real-time tasks by using an edge node server through the SDN controller via four actions: task creation, modification, operation, and completion. Our proposed architecture is simulated on the EstiNet simulator, and total time delay, reliability, and satisfaction are used as evaluation parameters. The simulation results are compared with the results of existing architectures, such as software-defined unified virtual monitoring function and ASTP, to analyze the performance of the proposed architecture. The analysis results indicate that our proposed architecture achieves better performance in terms of total time delay (1800 s for 200 IoT devices), reliability (90%), and satisfaction (90%).     

The multi-user computer-aided design collaborative learning framework

New developments to computer-aided design (CAD) software transform a once solitary modelling task into a collaborative one. The emerging multi-user CAD (MUCAD) systems allow virtual, real-time collaboration, with the potential to expand the learning outcomes and teaching methods of CAD. This paper proposes a MUCAD collaborative learning framework (MUCAD-CLF) to interpret backend analytic data from commercially available MUCAD software. The framework builds on several existing metrics from the literature and introduces newly developed methods to classify CAD actions collected from users’ analytic data. The framework contains two different classification approaches of user actions, categorizing actions by action type (e.g., creating, revising, viewing) and by design space (e.g., constructive, organizing), for comparative analysis. Next, the analytical framework is applied via a collaborative design challenge, corresponding to over 20,000 actions collected from 31 participants. Illustrative analyses utilizing the MUCAD-CLF are presented to demonstrate the resulting insight. Differences in CAD behaviour, indicating differences in learning, are observed between teams made up entirely of novices, entirely of experienced users, or a mix. In pairs of experts and novices, we see both a perceived high-satisfaction apprenticeship experience for the novices and preliminary evidence of an increase in expert design behaviours for the novices. The proposed framework is critical for MUCAD systems to make the most of the educational possibility of combining technical skill-building with team collaboration. Preliminary evidence collected in a fully-virtual design learning activity, and analyzed using the proposed MUCAD-CLF, shows that novice students gain advanced CAD design knowledge when collaborating with experienced teammates. With the user data captured by modern MUCAD software and the MUCAD-CLF presented herein, instructors and researchers can more efficiently assess and visualize students’ performance over the design learning process.