Effect of robot’s gaze behaviors for attracting and controlling human attention

Controlling someone’s attention can be defined as shifting his/her attention from the existing direction to another. To shift someone’s attention, gaining attention and meeting gaze are two most important prerequisites. If a robot would like to communicate a particular person, it should turn its gaze to him/her for eye contact. However, it is not an easy task for the robot to make eye contact because such a turning action alone may not be effective in all situations, especially when the robot and the human are not facing each other or the human is intensely attending to his/her task. Therefore, the robot should perform some actions so that it can attract the target person and make him/her respond to the robot to meet gaze. In this paper, we present a robot that can attract a target person’s attention by moving its head, make eye contact through showing gaze awareness by blinking its eyes, and directs his/her attention by repeating its eyes and head turns from the person to the target object. Experiments using 20 human participants confirm the effectiveness of the robot actions to control human attention.     

基于手机与无线网络的远程医疗监护系统设计

设计了一种医疗远程监护系统。该系统利用手机作为传感器网络的中心节点,与布置在人体上的传感器节点组成体域无线传感器网络,来实现病人生理参数数据的采集。同时,手机又作为连接远程无线网络的网关,以及病人获得信息的人机接口。利用该系统,病人可以了解自己病情和接收医疗指导信息,医疗机构可以对病人进行远程监护和紧急救助,病人家属可以对病人的病情状态进行查询。系统可针对不同病情的病人灵活地配置传感器、使用方便,适合在医院外对慢性病人的医疗远程监护中使用,具有广阔的应用前景。     

Robust and Rapid Generation of Animated Faces from Video Images: A Model-Based Modeling Approach

We have developed an easy-to-use and cost-effective system to construct textured 3D animated face models from videos with minimal user interaction. This is a particularly challenging task for faces due to a lack of prominent textures. We develop a robust system by following a model-based approach: we make full use of generic knowledge of faces in head motion determination, head tracking, model fitting, and multiple-view bundle adjustment. Our system first takes, with an ordinary video camera, images of a face of a person sitting in front of the camera turning their head from one side to the other. After five manual clicks on two images to indicate the position of the eye corners, nose tip and mouth corners, the system automatically generates a realistic looking 3D human head model that can be animated immediately (different poses, facial expressions and talking). A user, with a PC and a video camera, can use our system to generate his/her face model in a few minutes. The face model can then be imported in his/her favorite game, and the user sees themselves and their friends take part in the game they are playing. We have demonstrated the system on a laptop computer live at many events, and constructed face models for hundreds of people. It works robustly under various environment settings.     

ConChat: a context-aware chat program

ConChat is a context-aware chat program that enriches electronic communication by providing contextual information and resolving potential semantic conflicts between users.ConChat uses contextual information to improve electronic communication. Using contextual cues, users can infer during a conversation what the other person is doing and what is happening in his or her immediate surroundings. For example, if a user learns that the other person is talking with somebody else or is involved in some urgent activity, he or she knows to expect a slower response. Conversely, if the user learns that the other person is sitting in a meeting directly related to the conversation, he or she then knows to respond more quickly. Also, by informing users about the other person's context and tagging potentially ambiguous chat messages, ConChat explores how context can improve electronic communication by reducing semantic conflicts.     

基于视频与语音的多通道游戏用户界面系统

设计和实现了一套基于视频和语音的多通道游戏用户界面系统，以增强计算机游戏的交互性和游戏用户的沉浸感．系统新创建并有效地整合了基于视频与语音两种交互通道，其中包含脸部模型重建、头部姿态估计、汉语语音识别三个模块，可快速实现个性化的游戏角色脸部模型，并允许游戏用户使用头部姿态和语音命令实时控制游戏角色和游戏进展．测试和应用结果表明：该系统适用于普通游戏用户和实际游戏环境．     

Visual Focus of Attention in Non-calibrated Environments using Gaze Estimation

Estimating the focus of attention of a person highly depends on her/his gaze directionality. Here, we propose a new method for estimating visual focus of attention using head rotation, as well as fuzzy fusion of head rotation and eye gaze estimates, in a fully automatic manner, without the need for any special hardware or a priori knowledge regarding the user, the environment or the setup. Instead, we propose a system aimed at functioning under unpretending conditions, only with the usage of simple hardware, like a normal web-camera. Our system is aimed at functioning in a human-computer interaction environment, considering a person is facing a monitor with a camera adjusted on top. To this aim, we propose in this paper two novel techniques, based on local and appearance information, estimating head rotation, and we adaptively fuse them in a common framework. The system is able to recognize head rotational movement, under translational movements of the user towards any direction, without any knowledge or a-priori estimate of the user’s distance from the camera or camera intrinsic parameters.     

A company-office system “Valentine” providing informal communication and personal space based on 3D virtual space and avatars

In this paper we propose a virtual office environment that integrates natural communication and secure private space. The features of this system are the following. (1) This system has a virtual shared room based on the idea of “shared room metaphor” and 3D graphics on an SGI workstation is used for this system. It uses Ethernet media (i.e. real-time audio/video streams). (2) This system implements the field of view of a human by using our “around view” technique. This provides more natural communication between members. (3) “Sound effects” are used to help users feel the presence of other members. For instance, members hear the sound of a door opening when someone logs into our system and the sound of footsteps when someone is walking around our virtual room. (4) At times our system limits the flow of awareness information. A person concentrating on his/her work may not want to perceive excessive awareness of others. To support such situation, we define “awareness space” which restricts the field where other members' awareness is transmitted. Awareness space changes in size with the degree of concentration which is measured through two factors: the movement of a chair and the frequency of keyboard typing. (5) “Headphone metaphor”. A picture of a headphone is attached above a person's image and changes color depending on the degree of concentration. This enables other members to recognize his/her state and can be a criterion as to whether he/she is available to communicate or not. (6) In the virtual space, users are represented as avatars built of 3D polygons and still pictures. The avatars change shape automatically according to the users' action.     

Customization of user interfaces to reduce errors and enhance user acceptance

Customization is assumed to reduce error and increase user acceptance in the human–machine relation. Reconfiguration gives the operator the option to customize a user interface according to his or her own preferences. An experimental study with 72 computer science students using a simulated process control task was conducted. The reconfiguration group (RG) interactively reconfigured their user interfaces and used the reconfigured user interface in the subsequent test whereas the control group (CG) used a default user interface. Results showed significantly lower error rates and higher acceptance of the RG compared to the CG while there were no significant differences between the groups regarding situation awareness and mental workload. Reconfiguration seems to be promising and therefore warrants further exploration.     

Successful and unsuccessful multicommunication episodes: Engaging in dialogue or juggling messages?

This paper describes instances of multicommunicating—or engaging in more than one conversation at a time. It uses a critical incident technique to explore successful and unsuccessful incidents of multicommunicating from the perspective of 201 MBA students. Additionally, we asked which media individuals pair together when multicommunicating. We found very frequent pairing of the telephone (which provides partial compartmentalization but no flexibility of tempo) with electroric text (which provides both compartmentalization and flexibility of tempo). We also found that respondents provide a variety of reasons for labeling a particular episode as “unsuccessful.” In many cases the person seemed to describe an episode as unsuccessful when the person or a communicating partner had exceeded his or her ability to juggle multiple conversations as demonstrated by communication errors.     

A probabilistic multimodal approach for predicting listener backchannels

During face-to-face interactions, listeners use backchannel feedback such as head nods as a signal to the speaker that the communication is working and that they should continue speaking. Predicting these backchannel opportunities is an important milestone for building engaging and natural virtual humans. In this paper we show how sequential probabilistic models (e.g., Hidden Markov Model or Conditional Random Fields) can automatically learn from a database of human-to-human interactions to predict listener backchannels using the speaker multimodal output features (e.g., prosody, spoken words and eye gaze). The main challenges addressed in this paper are automatic selection of the relevant features and optimal feature representation for probabilistic models. For prediction of visual backchannel cues (i.e., head nods), our prediction model shows a statistically significant improvement over a previously published approach based on hand-crafted rules.     

Data Modeling for the Analysis of Health Risks and Human Longevity

We study the factors that influence the health of a person and human longevity with methods of mathematical modeling. We pay special attention to the use of modern data analysis methods that take into account the effects of heterogeneity in the considered groups of people due to genetic, behavioral differences, and presence of concomitant diseases that affect the condition of a person differently. We study the relationship between cause-specific mortality and diseases that the person had at the end of his or her life.     

The effect on sitting posture of a desk with a ten-degree inclination using an adjustable chair and table

By means of continuous recording of the positions of the trunk and head, an investigation was carried out to examine the effect of the Erasmus office-desk - a desk with a 10 degrees inclination - on sitting posture whilst reading and writing, sitting in an adjustable chair at an adjustable table. The posture assumed by a person who is doing mixed reading and writing work at a table with the Erasmus office-desk is more upright than the posture of the same person doing the same work without the desk. This applies for both the head and the trunk. The average position of the trunk was changed from 26.0 degrees to 18.2 degrees when working with the desk placed on the table. The moment of force caused by gravity is proportional to the sine of the angle. Therefore the change of angle results in a decrease of the average moment of force on the back at L5-S1 of 29%. The largest individual decrease found was 86%. The average change of the position of the head was changed from 38.5 degrees to 29.6 degrees , resulting in an average decrease of the moment of force, on C7-T1, of 21%. The largest individual decrease was 51%. Lateroflexion of the back was less often assumed when the subjects were using the desk. The posture assumed by the subjects when they were doing writing work only was generally more upright when working at a desk with a 10 degrees inclination than when working at a flat desk. Marked differences were found between the postures of the subjects. The change of posture while using the desk also showed marked differences for each subject individually.     

Automatic creation of 3D facial models

Model-based encoding of human facial features for narrowband visual communication is described. Based on an already prepared 3D human model, this coding method detects and understands a person's body motion and facial expressions. It expresses the essential information as compact codes and transmits it. At the receiving end, this code becomes the basis for modifying the 3D model of the person and thereby generating lifelike human images. The feature extraction used by the system to acquire data for regions or edges that express the eyes, nose, mouth, and outlines of the face and hair is discussed. The way in which the system creates a 3D model of the person by using the features extracted in the first part to modify a generic head model is also discussed     

Fusing bio-inspired vision data for simplified high level scene interpretation: Application to face motion analysis

This paper proposes to demonstrate the advantages of using certain properties of the human visual system in order to develop a set of fusion algorithms for automatic analysis and interpretation of global and local facial motions. The proposed fusion algorithms rely on information coming from human vision models such as human retina and primary visual cortex previously developed at Gipsa-lab. Starting from a set of low level bio-inspired modules (static and moving contour detector, motion event detector and spectrum analyser) which are very efficient for video data pre-processing, it is shown how to organize them together in order to achieve reliable face motion interpretation. In particular, algorithms for global head motion analysis such as head nods, for local eye motion analysis such as blinking, for local mouth motion analysis such as speech lip motion and yawning and for open/close mouth/eye state detection are proposed and their performances are assessed. Thanks to the use of human vision model pre-processing which decorrelates visual information in a reliable manner, fusion algorithms are simplified and remain robust against traditional video acquisition problems (light changes, object detection failure, etc.).     

AutoTutor: Incorporating Back-Channel Feedback and Other Human-Like Conversational Behaviors into an Intelligent Tutoring System

This paper describes our recent attempts to incorporate human-like conversational behaviors into the dialog moves delivered by an animated pedagogical agent that simulates human tutors. We first present a brief overview of the modules comprising AutoTutor, an intelligent tutoring system. The second section describes a set of conversational behaviors that are being incorporated into AutoTutor. The behaviors of interest involve variations in intonation, head movements, arm and hand movements, facial expressions, eye blinking, gaze direction, and back-channel feedback. The final section presents a recent empirical study concerned with back-channel feedback events during human-to-human tutoring sessions. The back-channel feedback events emitted by tutors are mostly positive (63%), mostly verbal (77%), and immediately follow speech-act boundaries or noun-phrase boundaries (83%). Tutors also deliver back-channelevents at a very high rate when students are emitting dialog, about 13 events per minute. Conversely, 88% of students' back-channel feedback events are head nods, and they occur at unbounded locations (63%).     

Implementation and performance evaluation of a distributed conjugate gradient method in a cloud computing environment

Cloud computing is an emerging technology where information technology resources are provisioned to users in a set of a unified computing resources on a pay per use basis. The resources are dynamically chosen to satisfy a user service level agreement and a required level of performance. A cloud is seen as a computing platform for heavy load applications. Conjugate gradient (CG) method is an iterative linear solver that is used by many scientific and engineering applications to solve a linear system of algebraic equations. CG generates a heavy load of computation, and therefore, it slows the performance of the applications using it. Distributing CG is considered as a way to increase its performance. However, running a distributed CG, based on a standard API, such as Message Passing Interface, in a cloud face many challenges, such as the cloud processing and networking capabilities. In this work, we present an in‐depth analysis of the CG algorithm and its complexity to develop adequate distributed algorithms. The implementation of these algorithms and their evaluation in our cloud environment reveal the gains and losses achieved by distributing the CG. The performance results show that despite the complexity of the CG processing and communication, a speedup gain of at least 1157.7 is obtained using 128 cores compared with National Aeronautics and Space Administration Advanced Supercomputing sequential execution. Given the emergence of clouds, the results in this paper analyzes performance issues when a generic public cloud, along with a standard development library, such as Message Passing Interface, is used for high‐performance applications, without the need of some specialized hardware and software. Copyright © 2012 John Wiley & Sons, Ltd.     

基于小波树的数字图像盲水印技术

提出一种基于特征的新型数字图像盲水印算法。根据小波变换的多分辨特性,通过在低频子带确定根节点的方法从小波系数中提取小波树。利用在小波树的中频节点与其邻域内的系数之间建立关系的方法嵌入水印。,通过密钥控制使水印的嵌入位置随机变化以提高安全性。嵌入的水印具有很好的透明性和鲁棒性,实用性较高。     

Modelling of a Human Driver’s Interaction with Vehicle Automated Steering Using Cooperative Game Theory

The introduction of automated driving systems raised questions about how the human driver interacts with the automated system. Non-cooperative game theory is increasingly used for modelling and understanding such interaction, while its counterpart, cooperative game theory is rarely discussed for similar applications despite it may be potentially more suitable. This paper describes the modelling of a human driver’s steering interaction with an automated steering system using cooperative game theory. The distributed Model Predictive Control approach is adopted to derive the driver’s and the automated steering system’s strategies in a Pareto equilibrium sense, namely their cooperative Pareto steering strategies. Two separate numerical studies are carried out to study the influence of strategy parameters, and the influence of strategy types on the driver’s and the automated system’s steering performance. It is found that when a driver interacts with an automated steering system using a cooperative Pareto steering strategy, the driver can improve his/her performance in following a target path through increasing his/her effort in pursuing his/her own interest under the driver-automation cooperative control goal. It is also found that a driver’s adoption of cooperative Pareto steering strategy leads to a reinforcement in the driver’s steering angle control, compared to the driver’s adoption of non-cooperative Nash strategy. This in turn enables the vehicle to return from a lane-change maneuver to straight-line driving swifter.      

Reading a robot's mind: a model of utterance understanding based on the theory of mind mechanism

The purpose of this paper is to construct a methodology for smooth communication between humans and robots. Here, focus is on a mindreading mechanism, which is indispensable in human-human communication. We propose a model of utterance understanding based on this mechanism. Concretely speaking, we apply the model of a mindreading system to a model of human-robot communication. Moreover, we implement a robot interface system that applies our proposed model. The characteristic of our interface system is its ability to construct a relationship between a human and a robot by a method of having an agent, which interacts with the person, migrate from the mobile PC of the person to the robot. Psychological experiments were carried out to explore the validity of the following hypothesis: By reading a robot's mind based on such a relationship, a person can estimate the robot's intention with ease and, moreover, the person can even understand the robot's unclear utterances made by synthesized speech sounds. The results of the experiments statistically supported our hypothesis.