The child behind the character

This paper presents a research approach for guidance and control of story characters by young children. We show that it is possible for a child to be at the same time an actor, performing a role in the story, and an engaged and empathic spectator of the overall story. To illustrate the approach, we describe Teatrix: a collaborative virtual environment for story creation by young children. In Teatrix, children control the actions of their story characters, which are implemented as intelligent agents in order to perform a certain chosen role, i.e., a villain or an hero. During the acting, children control their characters at different levels: at motor level, at behavioral level, at emotional level, and at reflection-level. In order to combine these different types of control, we introduce the concept of time-freeze in the story creation, combined with a tool for reflection, called “hot seating”. In this way, children are able to control their characters more deeply while, at the same time, feel like part of the audience of a play     

严肃游戏中虚拟角色行为建模综述

严肃游戏是计算机游戏一个新的发展方向,可以提供形象互动的模拟教学环境,已经广泛应用于科学教育、康复医疗、应急管理、军事训练等领域。虚拟角色是严肃游戏中模拟具有生命特征的图形实体,行为可信的虚拟角色能够提升用户使用严肃游戏的体验感。严肃游戏中的图形渲染技术已经逐步成熟,而虚拟角色行为建模的研究尚在初级阶段。可信的虚拟角色必须能够具有感知、情绪和行为能力。本文分别从游戏剧情与行为、行为建模方法、行为学习和行为建模评价等4个方面来分析虚拟角色行为建模研究。分析了有限状态机和行为树的特点,讨论了虚拟角色的行为学习方法。指出了强化学习的关键要素,探讨了深度强化学习的应用途径。综合已有研究,归纳了虚拟角色行为框架,该框架主要包括感觉输入、知觉分析、行为决策和动作4大模块。从情感计算的融入、游戏剧情和场景设计、智能手机平台和多通道交互4个角度讨论需要进一步研究的问题。虚拟角色的行为建模需要综合地考虑游戏剧情、机器学习和人机交互技术,构建具有自主感知、情绪、行为、学习能力、多通道交互的虚拟角色能够极大地提升严肃游戏的感染力,更好地体现寓教于乐。     

Predicting users' return to virtual worlds: a social perspective

Virtual worlds or three‐dimensional computer‐based simulated environments have received considerable attention as platforms for entertainment, education and commerce. In contrast to a web site, for example, where a user interacts with a two‐dimensional site, virtual worlds provide a platform in which users can interact with other individuals, sometimes including strangers, within three‐dimensional environments. Virtual worlds afford a form of ‘socialness’ unlike many other technologies, often motivating a user – by virtue of these social experiences – to return. Drawing on the Spatial Model of Interaction and Awareness‐Attention Theory, we demonstrate that besides social aspects, which include social awareness and social perception, flow experience, which is the mental state of being fully absorbed and somewhat lost in time, is an essential ingredient in an individual's decision to return to a virtual world. We also demonstrate how characteristics of the technology are linked to the social aspects experienced in virtual worlds. A laboratory study conducted in a virtual world, Second Life, supports our assertions and identifies state predictors of flow that influence individuals' intentions to return to the virtual world environment.     

Let's be a virtual juggler

Juggling, which uses both hands to keep several objects in the air at once, is admired by anyone who sees it. However, skillful real‐world juggling requires long, hard practice. Therefore, we propose an interesting method to enable anyone to juggle skillfully in the virtual world. In the real world, the human motion has to follow the motion of the moving objects; in the virtual world, the objects' motion can be adjusted together with the human motion. By using this freedom, we have generated a juggling avatar that can follow the user's motion. The user simply makes juggling‐like motions in front of a motion sensor. Our system then searches for juggling motions that closely match the user's motions and connects them smoothly. We then generate moving objects that both satisfy the laws of physics and are synchronized with the synthesized motion of the avatar. In this way, we can generate a variety of juggling animations by an avatar in real time. Copyright © 2016 John Wiley & Sons, Ltd.     

Realism in the design process and credibility of a simulation-based virtual laboratory

The credibility of an instructional simulation is the most important issue in distance education, where it may replace hands‐on activities. This credibility is based in large part upon verisimilitude, a perception strongly influenced by the simulation's realism. This paper presents a case study encompassing the design process of a simulation‐based virtual laboratory, which was guided by a realism principle, and an investigation of its credibility among potential users. We found that many characteristics of the environment associated with the design principle favoured its credibility, but that others had widely varying, even opposite, effects among users. A user's prior experience was shown to play a crucial but intricate role in verisimilitude and credibility judgements.     

Kinematics, Dynamics, Biomechanics: Evolution of Autonomy in Game Animation

The believeable portrayal of character performances is critical in engaging the immersed player in interactive entertainment. The story, the emotion and the relationship between the player and the world they are interacting within are hugely dependent on how appropriately the world's characters look, move and behave. We’re concerned here with the character's motion; with next generation game consoles like Xbox360 ^TM and Playstation®3 the graphical representation of characters will take a major step forward which places even more emphasis on the motion of the character. The behavior of the character is driven by story and design which are adapted to game context by the game's AI system. The motion of the characters populating the game's world, however, is evolving to an interesting blend of kinematics, dynamics, biomechanics and AI drivenmotion planning. Our goal here is to present the technologies involved in creating what are essentially character automata, emotionless and largely brainless character shells that nevertheless exhibit enough "behavior" to move as directed while adapting to the environment through sensing and actuating responses. This abstracts the complexities of low level motion control, dynamics, collision detection etc. and allows the game's artificial intelligence system to direct these characters at a higher level. While much research has already been conducted in this area and some great results have been published, we will present the particular issues that face game developers working on current and next generation consoles, and how these technologies may be integrated into game production pipelines so to facilitate the creation of character performances in games. The challenges posed by the limited memory and CPU bandwidth (though this is changing somewhat with next generation) and the challenges of integrating these solutions with current game design approaches leads to some interesting problems, some of which the industry has solutions for and some others which still remain largely unsolved.     

A systematic framework for on‐line calibration of a head‐mounted projection display for augmented‐reality systems

Abstract— Augmented reality (AR) is a technology in which computer‐generated virtual images are dynamically superimposed upon a real‐world scene to enhance a user's perceptions of the physical environment. A successful AR system requires that the overlaid digital information be aligned with the user's real‐world senses — a process known as registration. An accurate registration process requires the knowledge of both the intrinsic and extrinsic parameters of the viewing device and these parameters form the viewing and projection transformations for creating the simulations of virtual images. In our previous work, an easy off‐line calibration method in which an image‐based automatic matching method was used to establish the world‐to‐image correspondences was presented, and it is able to achieve subpixel accuracy. However, this off‐line method yields accurate registration only when a user's eye placements relative to the display device coincides with locations established during the offline calibration process. A likely deviation of eye placements, for instance, due to helmet slippage or user‐dependent factors such as interpupillary distance, will lead to misregistration. In this paper, a systematic on‐line calibration framework to refine the off‐line calibration results and to account for user‐dependent factors is presented. Specifically, based on an equivalent viewing projection model, a six‐parameter on‐line calibration method to refine the user‐dependent parameters in the viewing transformations is presented. Calibration procedures and results as well as evaluation experiments are described in detail. The evaluation experiments demonstrate the improvement of the registration accuracy.     

Capture the flag: mixed-reality social gaming with smart phones

The author developed a mobile, mixed-reality version of capture the flag, a popular genre of computer game. Our CTF game is novel in three ways. First, the smart phone is the main interface. Using the smart phone, players physically role-play virtual characters who try to capture enemy flags by traversing different landscapes. This approach creates a direct, real-time linkage between the real and virtual worlds. Second, players can move freely in the real world over a wide area while maintaining seamless real-time networked contact with other players in both the real and virtual worlds. Third, CTF explores novel tangible aspects of human physical movement and perception, both in the real-world playing environment and in interaction with the virtual world.     

Interconnecting distributed virtual worlds using Metabots: performance evaluation against the traditional client‐server model

Virtual Worlds evolution is breaking the barriers of virtual isolation, thus allowing users to participate in geographically dispersed and culturally diverse places. At the same time, mobile agents have been established as a solid programming method for heterogeneous networking and computing environments. Our work focuses on the definition of a distributed Virtual World reference platform for enhanced users' experience. Towards interconnecting virtual worlds with mobile agents, we have further enriched the concept of a human‐like appearance avatar. We propose two distributed virtual world architectures, namely loose and tight. In parallel, we present a relevant implementation scheme along with experimental results that prove the performance enhancements achieved against the classic client/server model. Copyright © 2014 John Wiley & Sons, Ltd.     

A multi‐plane optical see‐through head mounted display design for augmented reality applications

Augmented reality (AR) display technology greatly enhances users' perception of and interaction with the real world by superimposing a computer‐generated virtual scene on the real physical world. The main problem of the state‐of‐the‐art 3D AR head‐mounted displays (HMDs) is the accommodation‐vergence conflict because the 2D images displayed by flat panel devices are at a fixed distance from the eyes. In this paper, we present a design for an optical see‐through HMD utilizing multi‐plane display technology for AR applications. This approach manages to provide correct depth information and solves the accommodation‐vergence conflict problem. In our system, a projector projects slices of a 3D scene onto a stack of polymer‐stabilized liquid crystal scattering shutters in time sequence to reconstruct the 3D scene. The polymer‐stabilized liquid crystal shutters have sub‐millisecond switching time that enables sufficient number of shutters to achieve high depth resolution. A proof‐of‐concept two‐plane optical see‐through HMD prototype is demonstrated. Our design can be made lightweight, compact, with high resolution, and large depth range from near the eye to infinity and thus holds great potential for fatigue‐free AR HMDs.     

Video-based interactive storytelling using real-time video compositing techniques

Interactive storytelling systems usually adopt computer graphics to represent virtual story worlds, which facilitates the dynamic generation of visual content. However, the quality of the images and motion produced by these systems is still inferior compared to the high quality experience found in live-action films. Interactive rates in photorealistic rendering for the film industry will not be possible for decades to come. A promising alternative is the replacement of 3D virtual characters with video sequences with real actors. In this paper, we propose a new method for video-based interactive narratives that uses video compositing algorithms that run at truly interactive frame rates. The proposed method is consistent with plots that are generated by nondeterministic planning algorithms. Moreover, we propose a system of artificial intelligent agents that perform the same roles played by filmmaking professionals. A user evaluation of the proposed method is presented. We believe that future improvements of the techniques proposed in this paper represent an important contribution to the quest for new and more immersive forms of interactive cinema.

     

A Versatile Parameterization for Measured Material Manifolds

A popular approach for computing photorealistic images of virtual objects requires applying reflectance profiles measured from real surfaces, introducing several challenges: the memory needed to faithfully capture realistic material reflectance is large, the choice of materials is limited to the set of measurements, and image synthesis using the measured data is costly. Typically, this data is either compressed by projecting it onto a subset of its linear principal components or by applying non‐linear methods. The former requires many components to faithfully represent the input reflectance, whereas the latter necessitates costly extrapolation algorithms. We learn an underlying, low‐dimensional non‐linear reflectance manifold amenable to rapid exploration and rendering of real‐world materials. We can express interpolated materials as linear combinations of the measured data, despite them lying on an inherently non‐linear manifold. This allows us to efficiently interpolate and extrapolate measured BRDFs, and to render directly from the manifold representation. We exploit properties of Gaussian process latent variable models and use our representation for high‐performance and offline rendering with interpolated real‐world materials.     

面向智能驾驶测试的仿真场景构建技术综述

随着汽车智能化程度的不断提高,智能汽车通过环境传感器与周边行驶环境的信息交互与互联更为密切,需应对的行驶环境状况也越来越复杂,包括行驶道路、周边交通和气象条件等诸多因素,具有较强的不确定性、难以重复、不可预测和不可穷尽。限于研发周期和成本、工况复杂多样性,特别是安全因素的考虑,传统的开放道路测试试验或基于封闭试验场的测试难以满足智能驾驶系统可靠性与鲁棒性的测试要求。因此,借助数字虚拟技术的仿真测试成为智能驾驶测试验证一种新的手段,仿真场景的构建作为模拟仿真的重要组成部分,是实现智能驾驶测试中大样本、极限边界小概率样本测试验证的关键技术,这对提升智能驾驶系统的压力和加速测评水平显得尤为重要。面向智能驾驶测试的仿真场景构建技术已成为当前汽车智能化新的研究课题和世界性的研究热点,作为一种新兴技术仍面临许多挑战。本文提出了面向智能驾驶测试的仿真场景构建方法,系统阐述了国内外研究工作的进展与现状,包括场景自动构建方法和交通仿真建模方法,重点分析一些值得深入研究的问题并围绕场景构建技术的发展趋势进行了讨论分析,最后介绍了团队相关研究在2020中国智能驾驶挑战赛仿真赛和世界智能驾驶挑战赛的仿真场景应用情况。     

A modified radiosity algorithm for integrated visual and auditory rendering

A concept on integration of 3D visual and auditory rendering is proposed in this paper. By the auditory rendering we mean the computer acoustical simulation with direct sound outputs that is based on the physical model simulation. The so-called visual rendering means the computer generation of 3D images of real world and even virtual world, but without sounds. By integration of 3D visual and auditory rendering we mean while computer generates a 3D image of physical model of real world, it generates the sound based on the physical model displayed if there is a reverberation-free sound source. Based on some kinds of similarity between the propagation nature of light and sound, a modified radiosity algorithm has been proposed for integrating implementation. The algorithm is suitable for both visual and room auditory rendering. It means for a given enclosure, once the form factors are calculated, they can be applied for calculating both the light radiosities and sound radiosities simultaneously. The features of the algorithm are indicated. The implementation techniques are given in detail. The integration of visual and auditory rendering is a big step toward real world simulation, furthermore it will extend the research direction of visualization in scientific computing (ViSC) to visualization and audiobilization in scientific computing (VAiSC). The combination of VAiSC with virtual reality technique will improve human-computer interface greatly.     

Students' mathematics word problem‐solving achievement in a computer‐based story

The purpose of this study was to investigate the effect of a computer‐based story, which was designed in anchored instruction framework, on sixth‐grade students' mathematics word problem‐solving achievement. Problems were embedded in a story presented on a computer as computer story, and then compared with the paper‐based version of the same story and to a condition that presented the problems as typical, isolated word problems (i.e., a non‐story condition including only problems). One hundred twenty‐eight sixth‐grade students from two public middle schools in Turkey participated in this study. In a pretest–posttest experimental design, students were randomly assigned to one of the treatment groups mentioned earlier in which they solved the same mathematics word problems. A one‐way analysis of covariance was used to analyse students' achievement in the treatment groups. The results indicated that students who solved the problems in the computer story treatment had significantly higher achievement scores than students who solved the problems in the paper story and isolated word problems treatments. In addition, the story was found to be significantly more effective than the non‐story treatment when it was presented on computer.     

面向艺术与设计的虚拟人技术研究

虚拟人(Virtual Human)是人类各种特性(几何、行为、感知、情感、生理、心理、社会性等)在虚拟空间中的数字化表示，是高度真实的人类特性的数字化再现．文中首先介绍面向艺术与设计的虚拟人技术的应用领域和研究维度，然后详细介绍其中的主要研究内容，包括：虚拟人体建模、运动控制技术、虚拟人群仿真、虚拟感知与情感计算，最后指出，在面向艺术与设计的虚拟人技术研究中，艺术与设计领域专家和计算机专家的协作关系．     

Motion Sickness Simulation Based on Sensorimotor Control

Sensorimotor control is an essential mechanism for human motions, from involuntary reflex actions to intentional motor skill learning, such as walking, jumping, and swimming. Humans perform various motions according to different task goals and physiological sensory perception; however, most existing computational approaches for motion simulation and generation rarely consider the effects of human perception. The assumption of perfect perception (i.e., no sensory errors) of existing approaches restricts the generated motion types and makes dynamical reactions less realistic. We propose a general framework for sensorimotor control, integrating a balance controller and a vestibular model, to generate perception‐aware motions. By exploiting simulated perception, more natural responses that are closer to human reactions can be generated. For example, motion sickness caused by the impairments in the function of the vestibular system induces postural instability and body sway. Our approach generates physically correct motions and reasonable reactions to external stimuli since the spatial orientation estimation by the vestibular system is essential to preserve balance. We evaluate our framework by demonstrating standing balance on a rotational platform with different angular speeds and duration. The generated motions show that either faster angular speeds or longer rotational duration cause more severe motion sickness. Our results demonstrate that sensorimotor control, integrating human perception and physically‐based control, offers considerable potential for providing more human‐like behaviors, especially for perceptual illusions of human beings, including visual, proprioceptive, and tactile sensations.     

Control mechanisms and local perception to support autonomous behavior in virtual animated agents

We propose basic mechanisms that can be used as support to perform computer animated agents with autonomous behavior. We provide the agents with local perception and use a control theory approach, dealing with dynamics and kinematics issues, in order to establish a well structured way to control the agent resources. While maintaining the virtual agents with physical specifications and constraints necessary for animation purposes, the mechanisms provide other realistic needs necessary for providing autonomous behavior. In this way, agents can interact with their environment independent of user interaction, learning the characteristics of its habitat and adapting to it in order to survive. We validate the mechanisms by presenting two platforms with different structures (sensors, actuators, and dynamics) which have used them. As practical results, by using the same control structure, the animated agents are able to perform different tasks like learning attention control and pattern categorization based on multi-modal sensory information, learning how to progress onto a rough terrain, and learning how to perform visual monitoring tasks of their environment.     

CLUST: Simulating Realistic Crowd Behaviour by Mining Pattern from Crowd Videos

In this paper, we present a data‐driven approach to simulate realistic locomotion of virtual pedestrians. We focus on simulating low‐level pedestrians' motion, where a pedestrian's motion is mainly affected by other pedestrians and static obstacles nearby, and the preferred velocities of agents (direction and speed) are obtained from higher level path planning models. Before the simulation, collision avoidance processes (i.e. examples) are extracted from videos to describe how pedestrians avoid collisions, which are then clustered using hierarchical clustering algorithm with a novel distance function to find similar patterns of pedestrians' collision avoidance behaviours. During the simulation, at each time step, the perceived state of each agent is classified into one cluster using a neural network trained before the simulation. A sequence of velocity vectors, representing the agent's future motion, is selected among the examples corresponding to the chosen cluster. The proposed CLUST model is trained and applied to different real‐world datasets to evaluate its generality and effectiveness both qualitatively and quantitatively. The simulation results demonstrate that the proposed model can generate realistic crowd behaviours with comparable computational cost.