Reality-based interaction affecting mental workload in virtual reality mental arithmetic training

ABSTRACT

The concept of digital game-based learning (DGBL) evolves rapidly together with technological enhancements of virtual reality (VR) and smart phones. However, the mental workload (MWL) that VR-training applications demand and motivational qualities originating from user experience (UX) should be identified in order to create effective and enjoyable training/learning challenges that fit with individual users’ capabilities. This study examined the effects of reality-based interaction (RBI) and VR on measures of student motivation and MWL, in a mental arithmetic game for secondary school pupils. In a randomised controlled trial with sixty school children, a mental arithmetic game was tested with three different interaction and two different presentation methods – VR RBI, VR head-mounted-display tapping and tablet flick-gesture. Results found a significant effect of RBI on MWL but no differences in enjoyment of training were found between VR-experience and tablet training-experience. In fact, adding the gaming-context to the mental arithmetic task created an enjoyable, motivating experience regardless of presentation or interaction-style.     

Semiotics of virtual reality as a communication process

ABSTRACT

The experience of immersive virtual reality (VR) can be considered as a communication process between human beings, mediated by computer systems, which uses visualisation and other sensory stimulation. In this paper, we analyse how VR characteristics can be explored using semiotic theory and, with methods of generative semiotics, we explore aspects of narrative and interaction in VR. We propose a semiotic analysis of VR communication focusing on syntax, semantics and pragmatics and considering also some principles of generative semiotics. The syntactic level is analysed as determined by the characteristics of the visual communication adopted. The semantic of VR is related to the functional model chosen to realise the virtual system. The pragmatic of VR is based on the human–computer interaction that changes the user's role. We explore how these aspects can be characterised in the context of VR communication design and what principles can be adopted for a VR application, and we present an analysis and a classification of the iconic signs that are being used in VR. Moreover, we present a framework that can be used to classify and describe different kinds of virtual reality systems and to better understand communication in VR, and we use it to classify eight popular systems for e-learning and collaboration.     

基于事件的虚拟现实用户体验评估方法研究

近年来,随着虚拟现实(Virtual Reality,VR)技术的逐渐成熟,我国VR产业处于高速增长的态势。虽然VR的发展为内容设计提供了许多新的可能性,但由于VR应用的设计不成熟,VR行业的从业者们仍面临着严峻的挑战,VR中的设计仍需要通过用户使用来验证是否可行。因此,用户体验(User Experience,UX)研究对于成功设计VR软件至关重要。同时,随着用户对内容质量要求的提高,VR内容是否有足够的吸引力将在很大程度上影响用户体验,决定了用户是否会进行更进一步的VR体验。因此,如何评估VR用户体验并找到影响用户体验的内容对于提高用户体验和用户黏性至关重要。目前,对VR的研究集中在硬件和软件的改善上,对内容设计的关注较少,对用户体验的评价也没有统一且明确的标准。文中尝试找出用户体验和用户特性以及VR内容之间的关联。首先,定义了4种类型的VR交互事件,并设计了一份问卷来收集测试者的用户特性及其主观评价。其次,将80名测试者分成两组,在固定的时间内体验两种类型的VR产品,在实验过程中,测试者的客观生理数据与参与实验的整个过程都被记录下来。最后,采用统计学方法和改进的Prism算法,找出了用户特性、游戏交互事件类型与用户体验之间的相关性。文中对VR用户体验规范化评估进行了初步尝试,实验结果可以为VR内容设计人员和开发人员提供参考。     

移动环境下神经功能评价多通道交互模型

随着平板电脑、智能手机、智能手表等智能移动设备的普及,利用便携的智能移动设备随时随地进行健康评价受到了国内外学者的广泛关注.人机交互特有的多通道、交互式、人机协同的计算能够有效地提高移动环境下神经功能评价的准确度.然而,目前很少有研究对人机交互在这一应用场景上发挥的作用进行过充分讨论,更没有形成统一的多通道交互模型.为此,首先分析了目前移动设备上主流的神经功能评价方法,归纳总结出了一套适用该应用场景的交互原语和交互任务.然后,在此基础上提出了移动环境下神经功能评价多通道人机交互模型——MINA（multimodal human-computer interaction model for nerve function assessment in mobile environment),并对该模型的移动医学评价和多通道融合特点进行了分析.最后,依据此模型给出神经系统疾病检测的应用实例.实践证明,MINA能够较好地指导交互式神经功能评价应用的设计和开发,多通道融合的方式能够有效地提高医学评价的准确度.     

A scoping review of the use of lab streaming layer framework in virtual and augmented reality research

Virtual Reality - The use of multimodal data allows excellent opportunities for human–computer interaction research and novel techniques regarding virtual and augmented reality (VR/AR)...     

A survey on virtual reality

Virtual reality (VR) is a scientific method and technology created during the exploration of the nature by human beings to understand, simulate, and better adapt and use the nature. Based on the analysis on the whole process of VR, this paper presents different categories of VR problems and a type of theoretical expression, and abstracts three kinds of scientific and technical problems in VR field. On the basis of foresaid content, this paper also studies current major research objectives, research results and development trend of VR in the aspects of VR modeling method, VR representation technology, human-machine interaction and devices, VR development suites and supporting infrastructure, as well as VR applications. Finally, several theoretical and technical problems that need to be further studied and solved are addressed. Supported by the National Basic Research Program of China (Grant No. 2009CB320805), the National Natural Science Foundation of China (Grant Nos. 60533070, 60503066), and the National High-Tech Research & Development Program of China (Grant Nos. 2006AA01Z333, 2006AA01Z311)     

Web-based learning in the computer-aided design curriculum

Abstract This study attempts to apply the principle of constructivism and virtual reality (VR) technologies to computer-aided design (CAD) curriculum by integrating network, CAD and VR into a web-based learning environment. Through VR technologies, it is expected that the traditional two-dimensional (2D) computer graphics course can be expanded into a three-dimensional (3D) real-time simulation one. VR technologies provide a novel method to enhance user visualisation of complex three-dimensional graphics and environments. Experience and environmental interaction allow users more readily to perceive the dimensional interrelations of graphics which are typically portrayed through static multiview or pictorial representations. A web-based learning system ( WebDeGrator ) has been developed to simulate a computer graphics learning system for learning. Future developments of the proposed web-based learning framework are also discussed.     

基于Simulink上的虚拟现实环境动态系统仿真

介绍了MATLAB中虚拟现实动态系统的开发、运行环境以及虚拟现实工具箱的核心虚拟现实模块,着重介绍了如何利用虚拟现实工具箱在Simulink环境中建立倒立摆控制系统的虚拟现实动态系统仿真的方法,并给出了虚拟现实动态仿真的结果。虚拟现实动态仿真形象逼真,在本地计算机上具有交互性,并在远程计算机上可观察其动态仿真效果。     

The evaluation of user experience of a human walking and a driving simulation in the virtual reality

Virtual reality (VR) has been implemented in various applications such as gaming, e-learning, rehabilitation, etc. Although many studies have been conducted to evaluate the user experiences (UX) of various applications in VR, there are few studies evaluating the UX of VR in a holistic view. Since user experience means holistic feelings caused by attributes like efficiency, effectiveness, pleasure, and attractiveness, it is necessary to evaluate user experience in many respects, including presence, usability, and workload. Therefore, this study aims to evaluate the UX of VR in four various metrics: presence, workload, usability, and flow. In this study, the user experience factor was confirmed through the previous studies comparing visual display terminals (VDT) and head-mounted displays (HMD). In addition, the UX difference between VDT and HMD in two different situations, walking and driving, is analyzed. The data of 26 participants were analyzed and found that there were significant differences in the four studied metrics related to UX. In the driving situation, the usability value was higher in VDT, whereas they were higher in HMD for walking. Comparing the two contexts in HMD, walking was significantly higher in three metrics: presence, usability, and flow. The results show that HMD is better in terms of user experience than VDT, and HMD is more suitable for walking situations than for driving situations. Through this study, in terms of user experience, it is suitable to replace walking tasks with VR when the experiment in a real-world environment is impossible.     

The dynamics of film-induced affect and its effect on the interaction with tablet PCs

Affective states have become a crucial part of human–computer interaction research. Many studies have analysed the impact of the technology on the users' affective states as a part of what is called user experience (UX). We consider the impact of antecedent affective states on interaction with a technological artefact. We induced positive and negative affective states using film clips. Then we analysed the impact of affects on the subsequent interaction with a tablet PC. Results show that positive and negative affective states have different emotional activation patterns. Positive affect was more sensitive for changes in tasks and experimental setting. In addition, these activation patterns affected behaviour for a short time only. These findings are discussed against the background of research into UX dynamics, dynamics of affect, and user-centred design research.     

Virtual reality and its role in removing the barriers that turn cognitive impairments into intellectual disability

Early expectations of the contribution that virtual reality (VR) could make to education far exceeded actual applications. This was largely due to the initial immaturity of the technology and a lack of evidence base on which to base design and utilisation. While the early developments in computer based learning largely concentrated on mainstream education, leaving those with special needs behind, the potential of VR as an educational tool was exploited for those with intellectual disabilities right from the start. This paper describes the empirical evidence that has contributed to the development of educational virtual reality for those with intellectual disabilities: studies on transfer of learning from the virtual to the real world; how teachers might support those using VR; the design of virtual environments and what input/control devices best facilitate use of desktop VR. Future developments and ethical issues are also considered.     

Virtual reality human–robot interaction technology acceptance model for learning direct current and alternating current

In this study, we have developed a set of virtual reality (VR) human–robot interaction technology acceptance model for learning direct current and alternating current, aiming to use VR technology to immerse students in the generation, existence, and flow of electricity. We hope that using VR to transform abstract physical concepts into tangible objects will help students learn and comprehend abstract electrical concepts. The VR technology acceptance model was developed using the Unity 3D game kit to be accessed using the HTC Vive VR headset. The scene models, characters, and objects were created using Autodesk 3DS Max and Autodesk Maya, and the 2D graphics were processed in Adobe Photoshop. The results were evaluated using four metrics for our technology acceptance model. The four metrics include the content, design, interface and media content, and practical requirements. The average score of the content is 4.73. The average score of the design is 4.12. The average score of the interface and media content is 4.34. The average score of the practical requirements is 3.72. All the items on the effectiveness questionnaire of the technology acceptance model had average scores in the range 4.25–4.75. Therefore, all teachers were strongly satisfied with the trial teaching activity. The average score of each statement ranged within 3.58–4.03 for the satisfaction with the teaching material contents. Hence, the students were somewhat satisfied with this teaching activity. The average score of each statement ranged from 3.43 to 4.96 for the satisfaction with the implementation of the technology acceptance model. This result shows that the respondents were generally satisfied with the learning outcomes associated with these materials. The average score per question in this questionnaire was 3.92, and most of the questions have an average score greater than 3.8 for the feedback pertaining to satisfaction with the teaching material contents. In summary, a deeply immersive and interactive game was created using tactile somatosensory devices and VR that aim to utilize and enhance the fun and benefits associated with learning from games.

     

虚拟现实前沿技术在青少年科普教育中的创新与实践

近年来随着计算机图形技术的发展,越来越多的虚拟现实技术被运用到科普领域,其技术核心就是使用计算机模拟虚拟的情景并通过VR设备来体验。其中效果最好、沉浸性最强的设备非VR头戴式头盔莫属。但是在青少年科普活动中,特别是6-12年龄段的青少年群体,传统的VR头戴式头盔存在着一些明显的安全隐患,就连头盔设备官网都会提示12岁以下儿童不适用于该设备。在此前提下,本文研究混合现实技术和全息投影技术在青少年科普教育中的创新和实践。     

Evaluation of quality and personalisation of VR/AR/MR learning systems

ABSTRACT

The paper aims to analyse the problem of quality evaluation and personalisation of virtual reality/augmented reality/mixed reality (VR/AR/MR). First of all, systematic review of relevant scientific literature on the research topic was conducted. After that, findings of the systematic review concerning evaluation of quality and personalisation of VR/AR/MR learning environments are presented. The author’s VR/AR/MR learning systems/environments quality evaluation and personalisation framework is also presented in the paper. Evaluation of quality of VR/AR/MR platforms/environments should be based on (a) applying both expert-centred (top-down) and user-centred (bottom-up) quality evaluation methods and (b) separating ‘internal quality’ criteria, and ‘quality in use’ criteria in the set of quality criteria (model). Personalisation of VR/AR/MR platforms/environments should be based on learners’ models/profiles using students’ learning styles, intelligent technologies, and Semantic Web applications.     

Virtual environments for engineering applications

Virtual reality (VR), provides the user with an ego-centred human-computer interaction environment by presenting the data as a computer-generated 3D virtual environment. This enables the user to be immersed in this world via user position tracking devices and to interact with the data objects in the world in intuitive ways. This paper describes a selection of VR simulations for engineering applications implemented in CLRC which demonstrate the potential of VR interaction techniques to offer quicker and possibly better understanding of spatial relationships and temporal patterns inherent in large data sets. Two of the case studies have been implemented to support engineers communicate and review their designs with scientists, managers and manufacturers and to plan their assembly and maintenance work in hazardous physical environments. The other two applications are visualisation case studies based on data sets generated by computational engineering simulations. The case studies are real world applications, involving end-users of large or complex data sets. Insight gained into the user interaction requirements through the implementation and user comments is guiding ongoing research and development activity and this is discussed briefly.     

Investigating learners’ attitudes toward virtual reality learning environments: Based on a constructivist approach

The use of animation and multimedia for learning is now further extended by the provision of entire Virtual Reality Learning Environments (VRLE). This highlights a shift in Web-based learning from a conventional multimedia to a more immersive, interactive, intuitive and exciting VR learning environment. VRLEs simulate the real world through the application of 3D models that initiates interaction, immersion and trigger the imagination of the learner. The question of good pedagogy and use of technology innovations comes into focus once again. Educators attempt to find theoretical guidelines or instructional principles that could assist them in developing and applying a novel VR learning environment intelligently. This paper introduces the educational use of Web-based 3D technologies and highlights in particular VR features. It then identifies constructivist learning as the pedagogical engine driving the construction of VRLE and discusses five constructivist learning approaches. Furthermore, the authors provide two case studies to investigate VRLEs for learning purposes. The authors conclude with formulating some guidelines for the effective use of VRLEs, including discussion of the limitations and implications for the future study of VRLEs.     

A geoscience perspective on immersive 3D gridded data visualization

We describe visualization software, Visualizer, that was developed specifically for interactive, visual exploration in immersive virtual reality (VR) environments. Visualizer uses carefully optimized algorithms and data structures to support the high frame rates required for immersion and the real-time feedback required for interactivity. As an application developed for VR from the ground up, Visualizer realizes benefits that usually cannot be achieved by software initially developed for the desktop and later ported to VR. However, Visualizer can also be used on desktop systems (unix/linux-based operating systems including Mac OS X) with a similar level of real-time interactivity, bridging the “software gap” between desktop and VR that has been an obstacle for the adoption of VR methods in the Geosciences. While many of the capabilities of Visualizer are already available in other software packages used in a desktop environment, the features that distinguish Visualizer are: (1) Visualizer can be used in any VR environment including the desktop, GeoWall, or CAVE, (2) in non-desktop environments the user interacts with the data set directly using a wand or other input devices instead of working indirectly via dialog boxes or text input, (3) on the desktop, Visualizer provides real-time interaction with very large data sets that cannot easily be viewed or manipulated in other software packages. Three case studies are presented that illustrate the direct scientific benefits realized by analyzing data or simulation results with Visualizer in a VR environment. We also address some of the main obstacles to widespread use of VR environments in scientific research with a user study that shows Visualizer is easy to learn and to use in a VR environment and can be as effective on desktop systems as native desktop applications.     

基于VR设备中IMU的无手交互方法研究

随着头戴式显示设备的发展,在基于虚拟现实(VR,virtual reality)的教育培训中,存在用户与设备间进行交互的场景;针对用户与VR视频中对象的模拟靠近与躲避问题,提出了姿态感知与步态识别相结合的方法;通过姿态感知算法解算用户头部姿态,通过步态识别算法识别出人体的静止与行走状态,进而在行走状态时,将计算所得航向角和固定步长代入三角函数公式进行位置更新,在静止状态时,保持位置不变;实验证明,提出的姿态感知算法可以有效的计算出使用者头部的姿态,与商用惯性测量单元提供的姿态角相比具有1.1×10-2的平均姿态偏差;提出的步态识别算法可以有效地识别出人体的静止与行走状态;所提出的两者结合的交互方法,可以有效地实现虚拟的靠近与躲避.     

Improving speech understanding in communication headsets: Simulation of adaptive subband processing for speech in noise

Speech communication headsets are necessary for many high-noise environments to maintain interaction between individuals and facilitate safe working conditions. However, current hearing protection devices intended to protect hearing health can impede speech communication or expose persons to sound pressure levels (SPLs) that could lead to excessive noise exposure if a communication signal is presented improperly. This paper explores an adaptive subband communication algorithm, based on a delayless subband active noise reduction architecture, intended to adjust the communication channel gain to provide an appropriate speech signal power in relation to the instantaneous environmental noise power. The method monitors SPLs underneath the ear cup of a communication headset to provide a target speech signal-to-noise ratio without exceeding safe noise exposure thresholds. A series of computer simulations derived from a real-world communication headset model are used to compare the method developed with a traditional passive attenuation headset and a commercial active noise reduction design. The simulations demonstrate the ability of the adaptive subband communication algorithm to adjust automatically the speech signal gain for improved intelligibility while maintaining healthy noise exposure levels.Relevance to industryThe electro-acoustic performance of an active speech communication headset is explored by simulation. The concept integrates a subband active noise control algorithm with an adaptive gain control structure to improve speech intelligibility in a noisy environment. The concept automatically selects appropriate communication channel gain levels without exceeding hearing damage thresholds or requiring user input, and is directly applicable to a practical device.