高难度水平下单通道与多通道突显的效用性研究

研究在高难度水平条件下，视觉突显、听觉突显和视听多通道突显对被试搜索判断绩效的影响。实验结果与分析表明：不同的突显方式对视觉搜索绩效有明显影响；在高难度水平的视觉搜索任务中，视觉突显和听觉突显均具有较好的效用性，多通道突显的效用性比单通道突显的效用性好。     

Emotion recognition using eigenvalues and Levenberg–Marquardt algorithm-based classifier

In this paper, a simple and computationally efficient approach is proposed for person independent facial emotion recognition. The proposed approach is based on the significant features of an image, i.e., the collection of few largest eigenvalues (LE). Further, a Levenberg–Marquardt algorithm-based neural network (LMNN) is applied for multiclass emotions classification. This leads to a new facial emotion recognition approach (LE-LMNN) which is systematically examined on JAFFE and Cohn–Kanade databases. Experimental results illustrate that the LE-LMNN approach is effective and computationally efficient for facial emotion recognition. The robustness of the proposed approach is also tested on low-resolution facial emotion images. The performance of the proposed approach is found to be superior as compared to the various existing methods.     

融合人脸表情的手语到汉藏双语情感语音转换

针对聋哑人与正常人之间存在的交流障碍问题,提出了一种融合人脸表情的手语到汉藏双语情感语音转换的方法。首先使用深度置信网络模型得到手势图像的特征信息,并通过深度神经网络模型得到人脸信息的表情特征。其次采用支持向量机对手势特征和人脸表情特征分别进行相应模型的训练及分类,根据识别出的手势信息和人脸表情信息分别获得手势文本及相应的情感标签。同时,利用普通话情感训练语料,采用说话人自适应训练方法,实现了一个基于隐Markov模型的情感语音合成系统。最后,利用识别获得的手势文本和情感标签,将手势及人脸表情转换为普通话或藏语的情感语音。客观评测表明,静态手势的识别率为92.8%,在扩充的Cohn-Kanade数据库和日本女性面部表情(Japanese Female Facial Expression,JAFFE)数据库上的人脸表情识别率为94.6%及80.3%。主观评测表明,转换获得的情感语音平均情感主观评定得分4.0分,利用三维情绪模型(Pleasure-Arousal-Dominance,PAD)分别评测人脸表情和合成的情感语音的PAD值,两者具有很高的相似度,表明合成的情感语音能够表达人脸表情的情感。     

Facial Expression Recognition Model Depending on Optimized Support Vector Machine

In computer vision, emotion recognition using facial expression images is considered an important research issue. Deep learning advances in recent years have aided in attaining improved results in this issue. According to recent studies, multiple facial expressions may be included in facial photographs representing a particular type of emotion. It is feasible and useful to convert face photos into collections of visual words and carry out global expression recognition. The main contribution of this paper is to propose a facial expression recognition model (FERM) depending on an optimized Support Vector Machine (SVM). To test the performance of the proposed model (FERM), AffectNet is used. AffectNet uses 1250 emotion-related keywords in six different languages to search three major search engines and get over 1,000,000 facial photos online. The FERM is composed of three main phases: (i) the Data preparation phase, (ii) Applying grid search for optimization, and (iii) the categorization phase. Linear discriminant analysis (LDA) is used to categorize the data into eight labels (neutral, happy, sad, surprised, fear, disgust, angry, and contempt). Due to using LDA, the performance of categorization via SVM has been obviously enhanced. Grid search is used to find the optimal values for hyperparameters of SVM (C and gamma). The proposed optimized SVM algorithm has achieved an accuracy of 99% and a 98% F1 score.     

Bimodal recognition of affective states with the features inspired from human visual and auditory perception system

In this article, attention-based mechanism with the enhancement on biologically inspired network for emotion recognition is proposed. Existing bio-inspired models use multiscale and multiorientation architecture to gain discriminative power and to extract meticulous visual features. Prevailing HMAX model represents S2 layers by randomly selected prototype patches from training samples that increase the computational complexity and degrade the discerning ability. As eyes and mouth regions are the most powerful and reliable cues in determining facial emotions, they serve as the prototype patches for S2 layer in HMAX model. Audio code 4 book is constructed from mel-frequency cepstral coefficients, temporal and spectral features processed by principal component analysis. Audio and video data features are fused to train support vector machine classifier. The attained results on eNTERFACE, surrey audio-visual expressed emotion and acted facial expressions in the wild database datasets ascertain the efficiency of the proposed architecture for emotion recognition.     

Emotion Recognition from Occluded Facial Images Using Deep EnsembleModel

Facial expression recognition has been a hot topic for decades, but high intraclass variation makes it challenging. To overcome intraclass variation for visual recognition, we introduce a novel fusion methodology, in which the proposed model first extract features followed by feature fusion. Specifically, RestNet-50, VGG-19, and Inception-V3 is used to ensure feature learning followed by feature fusion. Finally, the three feature extraction models are utilized using Ensemble Learning techniques for final expression classification. The representation learnt by the proposed methodology is robust to occlusions and pose variations and offers promising accuracy. To evaluate the efficiency of the proposed model, we use two wild benchmark datasets Real-world Affective Faces Database (RAF-DB) and AffectNet for facial expression recognition. The proposed model classifies the emotions into seven different categories namely: happiness, anger, fear, disgust, sadness, surprise, and neutral. Furthermore, the performance of the proposed model is also compared with other algorithms focusing on the analysis of computational cost, convergence and accuracy based on a standard problem specific to classification applications.     

Emotion recognition based on facial components

Machine analysis of facial emotion recognition is a challenging and an innovative research topic in human–computer interaction. Though a face displays different facial expressions, which can be immediately recognized by human eyes, it is very hard for a computer to extract and use the information content from these expressions. This paper proposes an approach for emotion recognition based on facial components. The local features are extracted in each frame using Gabor wavelets with selected scales and orientations. These features are passed on to an ensemble classifier for detecting the location of face region. From the signature of each pixel on the face, the eye and the mouth regions are detected using the ensemble classifier. The eye and the mouth features are extracted using normalized semi-local binary patterns. The multiclass Adaboost algorithm is used to select and classify these discriminative features for recognizing the emotion of the face. The developed methods are deployed on the RML, CK and CMU-MIT databases, and they exhibit significant performance improvement owing to their novel features when compared with the existing techniques.     

负性面部表情影响面孔身份识别的实验研究

为验证负性面部表情对面孔身份识别的干扰效应，采用中国人的面孔表情图片为材料，设计了两个Garner范式实验：实验一重复过去研究采用愤怒与快乐表情图片为材料，实验二采用愤怒与悲伤表情图片为材料。结果发现，愤怒与快乐表情不影响面孔身份识别，而愤怒与悲伤表情影响面孔身份识别，说明负性表情能够影响面孔身份识别，结果支持表情身份非独立加工观。这一结果也弥补了过去研究难以发现表情影响身份识别的不足。     

Picture-Induced EEG Signal Classification Based on CVC Emotion Recognition System

Emotion recognition systems are helpful in human–machine interactions and Intelligence Medical applications. Electroencephalogram (EEG) is closely related to the central nervous system activity of the brain. Compared with other signals, EEG is more closely associated with the emotional activity. It is essential to study emotion recognition based on EEG information. In the research of emotion recognition based on EEG, it is a common problem that the results of individual emotion classification vary greatly under the same scheme of emotion recognition, which affects the engineering application of emotion recognition. In order to improve the overall emotion recognition rate of the emotion classification system, we propose the CSP_VAR_CNN (CVC) emotion recognition system, which is based on the convolutional neural network (CNN) algorithm to classify emotions of EEG signals. Firstly, the emotion recognition system using common spatial patterns (CSP) to reduce the EEG data, then the standardized variance (VAR) is selected as the parameter to form the emotion feature vectors. Lastly, a 5-layer CNN model is built to classify the EEG signal. The classification results show that this emotion recognition system can better the overall emotion recognition rate: the variance has been reduced to 0.0067, which is a decrease of 64% compared to that of the CSP_VAR_SVM (CVS) system. On the other hand, the average accuracy reaches 69.84%, which is 0.79% higher than that of the CVS system. It shows that the overall emotion recognition rate of the proposed emotion recognition system is more stable, and its emotion recognition rate is higher.     

采用图像融合技术的多模式人脸识别

利用图像融合技术实现了基于可见光图像和红外热图像相结合的多模式人脸识别,研究了两种图像在像素级和特征级的融合方法.在像素级,提出了基于小波分解的图像融合方法,实现了两种图像的有效融合.在特征级,采用分别提取两种识别方法中具有较好分类效果的前50%的特征进行特征级的融合.实验表明,经像素级和特征级融合后,识别准确率都较单一图像有很大程度的提高,并且特征级的融合效果明显优于像素级的融合.因此,基于图像融合技术的多模式人脸识别,有效的增加了图像的信息量,是提高人脸识别准确率的有效途径之一.     

Research on Facial Expression Capture Based on Two-Stage Neural Network

To generate realistic three-dimensional animation of virtual character, capturing real facial expression is the primary task. Due to diverse facial expressions and complex background, facial landmarks recognized by existing strategies have the problem of deviations and low accuracy. Therefore, a method for facial expression capture based on two-stage neural network is proposed in this paper which takes advantage of improved multi-task cascaded convolutional networks (MTCNN) and high-resolution network. Firstly, the convolution operation of traditional MTCNN is improved. The face information in the input image is quickly filtered by feature fusion in the first stage and Octave Convolution instead of the original ones is introduced into in the second stage to enhance the feature extraction ability of the network, which further rejects a large number of false candidates. The model outputs more accurate facial candidate windows for better landmarks recognition and locates the faces. Then the images cropped after face detection are input into high-resolution network. Multi-scale feature fusion is realized by parallel connection of multi-resolution streams, and rich high-resolution heatmaps of facial landmarks are obtained. Finally, the changes of facial landmarks recognized are tracked in real-time. The expression parameters are extracted and transmitted to Unity3D engine to drive the virtual character's face, which can realize facial expression synchronous animation. Extensive experimental results obtained on the WFLW database demonstrate the superiority of the proposed method in terms of accuracy and robustness, especially for diverse expressions and complex background. The method can accurately capture facial expression and generate three-dimensional animation effects, making online entertainment and social interaction more immersive in shared virtual space.     

基于注意力机制的LSTM语音情感主要特征选择

传统的语音情感识别方式采用的语音特征具有数据量大且无关特征多的特点,因此选择出与情感相关的语音特征具有重要意义。通过提出将注意力机制结合长短时记忆网络(Long Short Term Memory, LSTM),根据注意力权重进行特征选择,在两个数据集上进行了实验。结果发现:(1)基于注意力机制的LSTM相比于单独的LSTM模型,识别率提高了5.4%,可见此算法有效提高了模型的识别效果;(2)注意力机制是一种有效的特征选择方法。采用注意力机制选择出了具有实际物理意义的声学特征子集,此特征集相比于原有公用特征集在降低了维数的情况下,提高了识别准确率;(3)根据选择结果对声学特征进行分析,发现有声片段长度特征、无声片段长度特征、梅尔倒谱系数(Mel-Frequency Cepstral Coefficient, MFCC)、F0基频等特征与情感识别具有较大相关性。     

Face Recognition Based on Gabor Feature Extraction Followed by FastICA and LDA

Over the past few decades, face recognition has become the most effective biometric technique in recognizing people’s identity, as it is widely used in many areas of our daily lives. However, it is a challenging technique since facial images vary in rotations, expressions, and illuminations. To minimize the impact of these challenges, exploiting information from various feature extraction methods is recommended since one of the most critical tasks in face recognition system is the extraction of facial features. Therefore, this paper presents a new approach to face recognition based on the fusion of Gabor-based feature extraction, Fast Independent Component Analysis (FastICA), and Linear Discriminant Analysis (LDA). In the presented method, first, face images are transformed to grayscale and resized to have a uniform size. After that, facial features are extracted from the aligned face image using Gabor, FastICA, and LDA methods. Finally, the nearest distance classifier is utilized to recognize the identity of the individuals. Here, the performance of six distance classifiers, namely Euclidean, Cosine, Bray-Curtis, Mahalanobis, Correlation, and Manhattan, are investigated. Experimental results revealed that the presented method attains a higher rank-one recognition rate compared to the recent approaches in the literature on four benchmarked face datasets: ORL, GT, FEI, and Yale. Moreover, it showed that the proposed method not only helps in better extracting the features but also in improving the overall efficiency of the facial recognition system.     

基于栈式自编码神经网络的脑电信号情绪识别

针对维度情感模型生理信号情绪识别准确率较低的问题,本文基于DEAP维度情绪生理数据集,利用AR模型功率谱估计方法,提取脑电θ,α,β,γ节律的功率谱密度;采用小波包分解提取脑电小波包系数和能量占比时频特征;通过非线性分析提取脑电样本熵和小波包熵特征.然后,设计栈式自编码神经网络算法对脑电组合特征在效价和唤醒度两个情感维度上进行机器情绪识别.最后,分析了脑电特征、数据均衡以及情感标签对情绪识别结果的影响.仿真结果表明,栈式自编码神经网络用于脑电信号情绪识别的有效性,在情绪效价维度上,脑电情绪平均识别正确率可达80.3%;在唤醒度上,平均识别正确率达81.5%.该研究可为连续维度情绪自动分析和机器识别提供实际应用借鉴.     

类人型机器人动作情感识别研究

目的 为优化人与机器人动作情感交互方式,研究类人型机器人动作单模态情绪表达的可识别性,探究类人型机器人动作情绪识别的影响因素。方法 以类人型机器人NAO为例,采用问卷调查的方式,基于离散情绪模型,获取机器人NAO动作表达情绪的识别性、效价和唤醒度,研究类人型机器人动作的情绪识别性、效价和唤醒度,基于认知匹配理论研究类人型机器人动作与真人模拟动作、真人自然动作情绪表达差异的影响因素。结果 人类能够通过类人型机器人动作的单模态情绪表达,在不同情感语义上进行比较细腻的情绪识别,机器人形态及动作的幅度、速度、力量是情绪识别的影响因素。结论 建立类人型机器人动作与情感语义、效价及唤醒度的关系模型,以及类人型机器人动作情绪能量图,为机器人情感表达和动作交互设计提供较为系统的参考模型。     

面向人脸特征点定位和姿态估计任务协同的DCNN方法

为了提高复杂背景下面部信息的识别性能,提出了一种面向人脸特征点定位和姿态估计任务协同的深度卷积神经网络(DCNN)方法。首先从视频图像中检测出人脸信息;其次设计一个深度卷积网络模型,将人脸特征点定位和姿态估计两个任务协同优化,同时回归得到人脸特征点坐标和姿态角度值,然后融合生成相应的人机交互信息;最后采用公开数据集和实际场景数据进行测试,并与其他现有方法进行比对分析。实验结果表明:该方法在人脸特征点定位和姿态估计上表现出较好的性能,在光照变化、表情变化、部分遮挡等复杂条件下人机交互应用也取得了良好的准确性和鲁棒性,平均处理速度约16帧/s,具备一定的实用性。     

基于自身注意力时空特征的语音情感识别算法

针对语音情感识别中无法对关键的时空依赖关系进行建模，导致识别率低的问题，提出一种基于自身注意力（self-attention）时空特征的语音情感识别算法，利用双线性卷积神经网络、长短期记忆网络和多组注意力（multi-headattention）机制去自动学习语音信号的最佳时空表征。首先提取语音信号的对数梅尔（log-Mel）特征、一阶差分和二阶差分特征合成3D log-Mel特征集作为卷积神经网络的输入；然后综合考虑空间特征和时间依赖性关系，将双线性池化和双向长短期记忆网络的输出融合得到空间-时间特征表征，利用多组注意力机制捕获判别性强的特征；最后利用softmax函数进行分类。在IEMOCAP和EMO-DB数据库上进行实验，结果表明两种数据库的识别率分别为63.12%和87.09%，证明了此方法的有效性。     

基于SVD-TRIM特征和LSSVM人脸识别方法

人脸特征的选择对识别结果起关键作用。传统上只提取较大奇异值特征作为识别特征的人脸识别方法,识别率不高,对表情和姿态变化敏感。SVD-TRIM算法选择的奇异值识别特征融合了人脸整体和局部细节特征,并采用基于＂一对一＂的LSSVM多类分类器分类识别。实验结果表明SVD-TRIM算法选择的识别特征对提高识别率具有较大贡献,且对光照、姿态和表情具有鲁棒性。     

离散标签与维度空间结合的语音数据库设计

建立了一个将离散情感标签与维度情感空间结合起来的汉语情感语音数据库。该数据库由16名母语为汉语的说话人对情感语音进行表演型录制。语音样本是根据中性、愉悦、高兴、沮丧、愤怒、哀伤,以及悲伤等七种离散的情感标签采集而得,每名说话人有336条语音样本。随后由三名标注人在维度空间上对每条语音样本进行标注。最后,根据标注所得的数据来研究这七种情感在维度空间的分布情况,并分析了情感在一致性、集中性和差异性方面的性能。除此以外,还计算了这七种情感的情感识别率。结果显示,三名标注人对该数据库标注的一致性都达到了80%以上,情感之间的可区分度较高,并且七种情感的情感识别率均高于基线水平。因此,该数据库具有较好的情感质量,能够为离散情感标签到维度情感空间的转化提供重要的研究依据。