Cortical processing of human and emoji faces: an ERP analysis

ABSTRACT

Social networks have become an effective tool of interaction, frequently replacing face-to-face communication by using non-verbal emotional cues, such as emojis. The aim of the present study was to investigate whether human and emoji faces are analogously processed cortically. We recorded P100, N170, and LPP event-related potentials (ERPs) in response to happy, neutral, and angry expressions of human and emoji faces in 30 participants. The results showed that P100 and LPP amplitudes were larger in response to human faces but emoji faces generated larger N170 amplitude. Angry faces elicited significantly larger P100 and LPP amplitudes. The neural time-course of processing human and emoji faces was similar, however, human faces generate greater attentional orientation response, emoji faces increase the structural encoding processes, and human faces elicited greater arousal and attentional engagement. These results suggest that the use of emoji faces in computer-mediated communication produces neural responses that are similar to those that are observed in face-to-face communication.     

协同设计中多主体冲突问题的研究

从设计流程和体系结构的角度，对多主体冲突的产生、预防、检测与解决机制进行了系统的研究，提出了一个解决冲突问题的三层结构模型和各层相应的冲突解决策略，从而在控制级及领域级都对冲突进行了有效的避免和控制，该模型及方法在摩托车协同概念设计系统中得到了实现，为智能设计和协同设计奠定了基础。     

Eye movements as indices for the utility of life-like interface agents: A pilot study

We motivate an approach to evaluating the utility of life-like interface agents that is based on human eye movements rather than questionnaires. An eye tracker is employed to obtain quantitative evidence of a user’s focus of attention without distracting from the primary task. The salient feature of our evaluation strategy is that it allows us to measure important properties of a user’s interaction experience on a moment-by-moment basis in addition to a cumulative (spatial) analysis of the user’s areas of interest. We describe a pilot study in which we compare attending behavior of subjects watching the presentation of a computer-generated apartment layout and visualization augmented by three types of media: an animated agent, a text box, and speech only. The investigation of eye movements revealed that deictic gestures performed by the agent are more effective in directing the attentional focus of subjects to relevant interface objects than the media used in the two control conditions, at a slight cost of distracting the user from visual inspection of the object of reference. The results also demonstrate that the presence of an interface agent seemingly triggers natural and social interaction protocols of human users.     

The influence of web aesthetics on customers’ PAD

One key focus of an online retail website is to enhance the consumers’ online shopping behavior. Based on the Stimulus-Organism-Response (S-O-R) framework and pleasure, arousal, and dominance (PAD) emotional model, we investigated the relationship between consumers’ emotional model and purchase behavior from the perspective of web aesthetics, and how web aesthetics affect their purchase behaviors through the emotional model. Using 441 questionnaire responses and structural equation modeling, we verified that both aesthetic formality and aesthetic appeal influence purchase behaviors through the emotional model. In the emotional model, web aesthetics have significant and positive influences on control, which is composed of behavior control, cognitive control, and decisional control. Control indirectly influences pleasure through the mediations of energetic arousal and tense arousal. Additionally, pleasure and searching on other websites positively influences purchasing behavior. The aim of this study was to provide practical recommendations in the establishment of a pattern of web aesthetics that influence consumers’ emotions.     

一种基于情绪激励度的情绪词加权方法

在不同的上下文中,情绪词对情绪的激励程度会发生变化。现有情绪词典中大多数只标注了情绪词的情绪类别而未涉及情绪词的激励度。在极少数标注情绪强度的词典中,所标注的强度未考虑上下文的影响。提出一种根据上下文形成的情境评估情绪词对情绪的激励程度并据此对情绪词加权的方法。通过比较情绪词的共现模式与自身情绪类的分布模式计算情绪词的激励程度。然后根据激励程度计算情绪词的情绪权重并将其用于微博情绪识别。实验结果表明,与现有词典中的情绪强度相比,本文方法计算的情绪权重更准确地描述了情绪词在语料中表达的情绪,有效地提高了情绪分析的精度。并且本文方法还能够有效综合多个词典的优势,进一步提高微博情绪分析的准确率。     

Affect in Computer-Meditated Communication: An Experiment in Synchronous Terminal-to-Terminal Discussion

With the spread of computer networks, communication via computer conferences, electronic mail, and computer bulletin boards will become more common in society, but little is known about the social psychological implications of these technologies. One possibility is a change in physiological arousal, feelings, and expressive behavior-that is, affect. These computer-mediated communication technologies focus attention on the message, transmit social information poorly, and do not have a well-developed social etiquette. Therefore, these technologies might be associated with less attention to others, less social feedback, and depersonalization of the communication setting. In the present study we examined what would happen to feelings and interpersonal behavior in an experiment in which two people met for the first time and discussed a series of questions in order to get to know one another. We measured physiological arousal (pulse and palmar sweat), subjective affect (emotional state and evaluations), and expressive behavior (self-disclosure and uninhibited behavior) in both synchronous computer-mediated and face-to-face discussions. (For comparison purposes, we also examined these effects under high- and low-evaluation anxiety). Communicating by computer did not influence physiological arousal, and it did not change emotions or self-evaluations. However, people who communicated by computer evaluated each other less favorably than did people who communicated face-to-face, they felt and acted as though the setting was more impersonal, and their behavior was more uninhibited. These findings suggest that computer-mediated communication, rather than provoking emotionality per se, elicits asocial or unregulated behavior. Of course, our data are based on a laboratory experiment using just one type of computer-mediated communication, but the results are generally consistent with anecdotal evidence and new field research on how people use computers to communicate in organizations.     

Attentional bias in excessive massively multiplayer online role-playing gamers using a modified Stroop task

There is considerable dispute regarding the nature of excessive or problematic Internet-related behaviour and whether it constitutes a clinical addiction. Classification of excessive gaming is hindered by a lack of experimental research investigating behavioural responses from gamers and comparing these patterns to those found in established addictions. We investigated whether an attentional bias for gaming-related words existed for addicted Massively Multiplayer Online Role-Playing Gamers (MMORPGers) identified using the Addiction-Engagement Questionnaire.Forty frequent MMORPGers (15 female) and 19 non-MMORPGers (eight female) completed a computerised modified Stroop task comprised of game-related, negative and neutral word lists, Addiction-Engagement Questionnaire, Depression, Anxiety and Stress Scale 21, gaming-related variables. The results indicated that addicted MMORPGers had significantly longer reaction times to negative and MMORPG words compared to neutral words, whereas highly engaged and non-MMORPG participants showed no such bias. The presence of an attentional bias in addicted MMORPGers is comparable with research investigating this behavioural response in established addictions.     

Inter-relationships among dominance, energetic and tense arousal, and pleasure, and differences in their impacts under online vs. offline environment

Only a few studies have investigated dominance and observed mixed results. The impact of arousal on pleasure and behavior has also produced inconsistent or no meaningful results. In addition, emotional responses such as dominance, arousal, and pleasure have been assumed to be independent, which have resulted in a void in the relationships research among these constructs. Based on this research tradition, the present study proposed an inter-relationship among dominance, energetic and tense arousal, pleasure, and their impact on intention. A two-dimensional concept of energetic arousal and tense arousal has never been adopted and studied in previous retail research. Dominance, energetic arousal, and tense arousal are introduced to explain inconsistent relationships found in previous retail consumer studies. Results from a survey of 406 consumers (217 from offline store customers and 189 online store users) using PLS demonstrated that that (1) while dominance has a significant positive and/or negative effect on both energetic and tense arousal, it has no impact on pleasure and intention under both offline and online environment; (2) the effect of dominance on tense arousal is not statistically significant in an online shopping environment; (3) both energetic and tense arousal has a positive and/or negative impact on pleasure; (4) energetic arousal affects intention directly and indirectly mediated by pleasure, whereas tense arousal has an indirect effect on intention; and (5) the results found in testing differences in path coefficients under the two different shopping environment suggested that (1) the impact of dominance on energetic arousal is stronger under offline shopping context; (2) the effect of dominance on tense arousal is stronger in online environment; (3) the effect of energetic arousal on pleasure is stronger in offline shopping context; and (4) the impact of tense arousal on pleasure is stronger under offline shopping environment. Theoretical and practical implications are discussed in conclusion.     

Focus tree: modeling attentional information in task-oriented human-machine interaction

This paper introduces a new model of attentional state in task-oriented human-machine interaction. It integrates three lines of research: (i)?neurocognitive understanding of the focus of attention in working memory, (ii)?the notion of attention related to the theory of discourse structure in the field of computational linguistics, and (iii)?investigation of a corpus that comprises recordings of spontaneous speech-based human-machine interaction. The underlying idea was to make a computationally appropriate representation of attentional information that imitates the function of a focus of attention in human perception. The introduced model addresses both the research questions of storage and processing of attentional information. Finally, the paper illustrates the model for concrete interaction domains, and discusses its implementation within a prototype spoken dialogue system.     

Attention aware systems: Theories,applications, and research agenda

Human perceptual and cognitive abilities are limited resources. Attention is the mechanism used to allocate such resources in the most effective way. Current technologies, in addition to allowing fast access to information and people, should be designed to support human attentional processes on which they impose further strain. This paper analyses the issues related to the design of systems capable of such support: attention aware systems. We introduce the research aimed at understanding and modelling human attentional processes, including perceptual and cognitive processes as studied in cognitive psychology, as well as rhetorical, aesthetic, and social aspects related to attentional mechanisms. We analyse current approaches to the design of attention aware systems along three major features: detection of user’s current attentional state, detection and evaluation of possible alternative attentional states, strategies for focus switch or maintenance. Finally, we discuss the most promising research direction for the development of systems capable of supporting human attentional mechanisms.     

A neuroergonomic approach to evaluating mental workload in hypermedia interactions

Neuroergonomics could provide on-line methods for measuring mental effort while the operator interacts with hypermedia. We present an experimental study in which 28 participants interacted with a modified version of an existing Spanish e-commerce website in two searching tasks (Goal oriented shopping and Experiential shopping) that demand different amounts of cognitive resources. Mental workload was evaluated multidimensionally, using subjective rating, an interaction index, and eye-related indices. Eye movements and pupil diameter were recorded. The results showed visual scanning behaviour coincided with subjective test scores and performance data in showing a higher information processing load in Goal oriented shopping. However, pupil diameter was able to detect only the variation in user activation during the interaction task, a finding that replicates previous results on the validity of pupil size as an index of arousal. We conclude that a neuroergonomics approach could be a useful method for detecting variations in operators’ attentional states.

Relevance to industry

These results could provide important information for the development of a new attentional screening tool for the prevention of accidents in several application domains.     

Effects of music tempo and task difficulty on multi-attribute decision-making: An eye-tracking approach

This study examined the effects of music tempo and task difficulty on the performance of multi-attribute decision-making according two alternative perspectives: background music as the arousal inducer vs. the distractor. An eye-tracking based experiment was conducted. Our results supported the arousal inducer perspective that, with the same level of decision time, participants made decisions more accurately with the presentation of faster than slower tempo music. Further, faster tempo music was found to improve the accuracy of harder decision-making only, not that of easier decision-making. More interestingly, our exploratory analysis on eye fixations found the occurrence of adaptive behavior, namely, that the search pattern of participants became more intra-dimensional under the faster tempo music as compared with the slower tempo music.     

Predicting unsafe behaviors at nuclear power plants: An integration of Theory of Planned Behavior and Technology Acceptance Model

Unsafe behaviors, including both errors and violations, have been identified as key contributors to accidents at nuclear power plants. However, the mechanisms behind such unsafe behaviors are not fully understood. This study aimed to investigate how workers' attitude and perception factors would predict errors and violations at nuclear power plants by proposing and validating an unsafe behavior model. The proposed model applied the Theory of Planned Behavior as the ground theoretical model and added factors from Technology Acceptance Model to capture workers’ perception of work regulations. To examine the validity of the proposed model, a total of 178 questionnaires were distributed at two nuclear power plants in China and 171 valid questionnaires were returned. The Structured Equation Modeling (SEM) analysis indicated that the proposed model fitted the data well. The results showed that perceived ease of use and perceived usefulness in following work regulations contributed to a positive attitude, which helped reduce the occurrence of both errors and violations. Moreover, errors were further affected by subjective norm while violations were not. Perceived behavior control was not a significant factor of either errors or violations.These findings suggest that to reduce errors and violations, policymakers should focus on methods to improve the perceived usefulness and perceived ease of use of work regulations, and promote a positive attitude towards safety.     

基于时间窗模型的双向机器人路径规划方法

随着智能化与自动化的发展,机器人应用越来越广泛;多机器人路径规划和协作问题就成为当前关注的焦点。针对以上问题,提出了一种改进的遗传算法。在初始化总群时,通过选择中点的相邻点扩大选择范围确保线路连续。通过种群适应度函数改进路径平滑度防止转向次数过多。为了防止遗传算法陷入局部最优解,通过轮盘赌的方法确保一部分非最优个体。在优化后的遗传算法基础上根据优先级顺序,并结合提出的时间窗模型,对冲突类型进行分类处理并制定对应的协调方案。仿真结果表明,该方法可以有效地解决多机器人的路径冲突问题,通过合理地规划线路提高系统效率。     

Emotional clouds: Showing arousal and valence through the movement and darkness of digital cartoonish clouds

While nowadays the most usual way to show emotions in digital contexts is via virtual characters, its use may raise false expectations (the user attributes human abilities to the virtual character). This paper proposes and explores an approach to express emotions which intends to minimize the user's expectations by using a non-anthropomorphic model. Emotions are represented in terms of arousal and valence dimensions. They are visualized in a simple way through the behaviour and appearance of a series of cartoonish clouds. In particular, the arousal value is expressed through the movement of these clouds (controlled by a flocking algorithm), while the valence value is expressed through their degree of darkness. Furthermore, the paper describes a user experiment which investigated whether the arousal and valence expressed by our model are appropriately interpreted by the users or not. The results suggest that movement and darkness are interpreted as arousal and valence respectively and that they are independent of each other.     

Categorical and dimensional affect analysis in continuous input: Current trends and future directions

In the context of affective human behavior analysis, we use the term continuous input to refer to naturalistic settings where explicit or implicit input from the subject is continuously available, where in a human–human or human–computer interaction setting, the subject plays the role of a producer of the communicative behavior or the role of a recipient of the communicative behavior. As a result, the analysis and the response provided by the automatic system are also envisioned to be continuous over the course of time, within the boundaries of digital machine output. The term continuous affect analysis is used as analysis that is continuous in time as well as analysis that uses affect phenomenon represented in dimensional space. The former refers to acquiring and processing long unsegmented recordings for detection of an affective state or event (e.g., nod, laughter, pain), and the latter refers to prediction of an affect dimension (e.g., valence, arousal, power). In line with the Special Issue on Affect Analysis in Continuous Input, this survey paper aims to put the continuity aspect of affect under the spotlight by investigating the current trends and provide guidance towards possible future directions.     

Sampling behavior in a four instrument monitoring task: effects ofsignal bandwidth and number of events per signal

This study investigates the effect of global signal characteristics, such as bandwidth and number of events on human sampling behavior. It also examines whether the influence of these characteristics on sampling strategy is affected by local (actually observed) signal features, such as the degree with which a sampled signal value falls short of an event region and the rate of change. In the present study four independent, numerically displayed signals were used; two different bandwidths and two different event numbers were chosen. To take a sample, subjects had to use a mouse. The mouse key responses were used as an index of sampling. The study demonstrates that both bandwidth and number of events equally affected the distribution of samples over signals. In addition, it shows that global signal characteristics determine sampling behavior less prominently when the attentional demands brought about by the local signal features become of greater importance. This indicates that not predictability as governed by global signal characteristics as such, but rather predictability given certain local signal features is a crucial factor in determining sampling behavior     

Visual Attention for Rendered 3D Shapes

Understanding the attentional behavior of the human visual system when visualizing a rendered 3D shape is of great importance for many computer graphics applications. Eye tracking remains the only solution to explore this complex cognitive mechanism. Unfortunately, despite the large number of studies dedicated to images and videos, only a few eye tracking experiments have been conducted using 3D shapes. Thus, potential factors that may influence the human gaze in the specific setting of 3D rendering, are still to be understood. In this work, we conduct two eye‐tracking experiments involving 3D shapes, with both static and time‐varying camera positions. We propose a method for mapping eye fixations (i.e., where humans gaze) onto the 3D shapes with the aim to produce a benchmark of 3D meshes with fixation density maps, which is publicly available. First, the collected data is used to study the influence of shape, camera position, material and illumination on visual attention. We find that material and lighting have a significant influence on attention, as well as the camera path in the case of dynamic scenes. Then, we compare the performance of four representative state‐of‐the‐art mesh saliency models in predicting ground‐truth fixations using two different metrics. We show that, even combined with a center‐bias model, the performance of 3D saliency algorithms remains poor at predicting human fixations. To explain their weaknesses, we provide a qualitative analysis of the main factors that attract human attention. We finally provide a comparison of human‐eye fixations and Schelling points and show that their correlation is weak.