基于视频与语音的多通道游戏用户界面系统

设计和实现了一套基于视频和语音的多通道游戏用户界面系统，以增强计算机游戏的交互性和游戏用户的沉浸感．系统新创建并有效地整合了基于视频与语音两种交互通道，其中包含脸部模型重建、头部姿态估计、汉语语音识别三个模块，可快速实现个性化的游戏角色脸部模型，并允许游戏用户使用头部姿态和语音命令实时控制游戏角色和游戏进展．测试和应用结果表明：该系统适用于普通游戏用户和实际游戏环境．

A Multi-modal Game Player Interface System Based on Video and Speech

In order to enhance game's interactivity and player's immersion in 3D computer games, we developed an easy-to-use and cost-effective interface system based on video and speech. This system integrates three modules, personalized 3D face modeling, head pose estimation and Chinese speech recognition. The first module allows a player to generate his or her personalized character with minimal time and manual interaction, the second and third modules are incorporated to control the character using player's head poses and speech commands in real-time. We also developed a multiplayer game demo, 3D Chinese chess, as a test bench to test our system. The test results demonstrate that the approaches and the system are feasible and practical.