Variational autoencoder for prosody-based speaker recognition

This paper describes a novel end-to-end deep generative model-based speaker recognition system using prosodic features. The usefulness of variational autoencoders (VAE) in learning the speaker-specific prosody representations for the speaker recognition task is examined herein for the first time. The speech signal is first automatically segmented into syllable-like units using vowel onset points (VOP) and energy valleys. Prosodic features, such as the dynamics of duration, energy, and fundamental frequency ( $F_0$), are then extracted at the syllable level and used to train/adapt a speaker-dependent VAE from a universal VAE. The initial comparative studies on VAEs and traditional autoencoders (AE) suggest that the former can efficiently learn speaker representations. Investigations on the impact of gender information in speaker recognition also point out that gender-dependent impostor banks lead to higher accuracies. Finally, the evaluation on the NIST SRE 2010 dataset demonstrates the usefulness of the proposed approach for speaker recognition.     

基于说话人特有特征集的GMM和i-矢量方法的说话人识别

在说话人识别中,当存在两个或多个发声类似的说话人时,会导致错误识别。为了提高在这种情况下的识别准确率,在音素层次上找出说话人特有的特征,将这些特征的子集构成一个该说话人特有的特征集,然后在这些特征集的基础上用GMM和i-矢量的方法对说话人进行识别。在实验室环境下收集了50个说话人的声音,分别在不同信噪比的环境下进行测试。实验结果表明提出的方法能够提高当存在发声类似的说话人时的识别准确率。     

Reviewing automatic language identification

The Oregon Graduate Institute Multi-language Telephone Speech Corpus (OGI-TS) was designed specifically for language identification research. It currently consists of spontaneous and fixed-vocabulary utterances in 11 languages: English, Farsi, French, German, Hindi, Japanese, Korean, Mandarin, Spanish, Tamil, and Vietnamese. These utterances were produced by 90 native speakers in each language over real telephone lines. Language identification is related to speaker-independent speech recognition and speaker identification in several interesting ways. It is therefore not surprising that many of the recent developments in language identification can be related to developments in those two fields. We review some of the more important recent approaches to language identification against the background of successes in speaker and speech recognition. In particular, we demonstrate how approaches to language identification based on acoustic modeling and language modeling, respectively, are similar to algorithms used in speaker-independent continuous speech recognition. Thereafter, prosodic and duration-based information sources are studied. We then review an approach to language identification that draws heavily on speaker identification. Finally, the performance of some representative algorithms is reported     

面向内容的语音信号压缩感知研究

压缩感知理论依据信号的稀疏性质进行压缩测量,将信号的获取方式从对信号的采样上升为对信息的感知,是信号处理领域的一场革命。本文提出一种基于非确定基字典（Uncertainty Basis Dictionary, UBD）对语音信号进行稀疏表示的方法,将压缩感知理论应用于对语音信号稀疏表示的压缩,并提出了基于求解线性规划问题的方法重构语音信号的算法。通过语音识别、话者识别和情感识别实验,从面向内容分析的角度,研究这种基于压缩感知理论的信息感知方法是否保留了语音信号的主要内容。实验结果表明,语音识别、话者识别和情感识别的准确率,与目前这些领域研究方法得到的结果基本一致,说明基于压缩感知理论的信息感知方法能够很好地获取语音信号的语义、话者和情感方面的信息。      

Feature classification criterion for missing features mask estimation in robust speaker recognition

Currently, many speaker recognition applications must handle speech corrupted by environmental additive noise without having a priori knowledge about the characteristics of noise. Some previous works in speaker recognition have used the missing feature (MF) approach to compensate for noise. In most of those applications, the spectral reliability decision step is performed using the signal to noise ratio (SNR) criterion, which attempts to directly measure the relative signal to noise energy at each frequency. An alternative approach to spectral data reliability has been used with some success in the MF approach to speech recognition. Here, we compare the use of this new criterion with the SNR criterion for MF mask estimation in speaker recognition. The new reliability decision is based on the extraction and analysis of several spectro-temporal features from across the entire speech frame, but not across the time, which highlight the differences between spectral regions dominated by speech and by noise. We call it the feature classification (FC) criterion. It uses several spectral features to establish spectrogram reliability unlike SNR criterion that relies only in one feature: SNR. We evaluated our proposal through speaker verification experiments, in Ahumada speech database corrupted by different types of noise at various SNR levels. Experiments demonstrated that the FC criterion achieves considerably better recognition accuracy than the SNR criterion in the speaker verification tasks tested.     

基于音色一致的语音克隆说话人特征提取方法

当前基于预训练说话人编码器的语音克隆方法可以为训练过程中见到的说话人合成较高音色相似性的语音,但对于训练中未看到的说话人,语音克隆的语音在音色上仍然与真实说话人音色存在明显差别。针对此问题,本文提出了一种基于音色一致的说话人特征提取方法,该方法使用当前先进的说话人识别模型TitaNet作为说话人编码器的基本架构,并依据说话人音色在语音片段中保持不变的先验知识,引入一种音色一致性约束损失用于说话人编码器训练,以此提取更精确的说话人音色特征,增加说话人表征的鲁棒性和泛化性,最后将提取的特征应用端到端的语音合成模型VITS进行语音克隆。实验结果表明,本文提出的方法在2个公开的语音数据集上取得了相比基线系统更好的性能表现,提高了对未见说话人克隆语音的音色相似度。     

基于CNN的连续语音说话人声纹识别

近年来,随着社会生活水平的不断提高,人们对机器智能人声识别的要求越来越高.高斯混合—隐马尔可夫模型(Gaussian of mixture-hidden Markov model,GMM-HMM)是说话人识别研究领域中最重要的模型.由于该模型对大语音数据的建模能力不是很好,对噪声的顽健性也比较差,模型的发展遇到了瓶颈.为了解决该问题,研究者开始关注深度学习技术.引入了CNN深度学习模型研究连续语音说话人识别问题,并提出了CNN连续说话人识别(continuous speaker recognition of convolutional neural network,CSR-CNN)算法.模型提取固定长度、符合语序的语音片段,形成时间线上的有序语谱图,通过CNN提取特征序列,经过奖惩函数对特征序列组合进行连续测量.实验结果表明,CSR-CNN算法在连续—片段说话人识别领域取得了比GMM-HMM更好的识别效果.     

Face Recognition with Integrating Multiple Cues

Automatic face recognition is a challenge task, especially working in practical uncontrolled environments. Over the past two decades, numerous innovative ideas and effective processing approaches had been proposed and developed, e.g. various normalization techniques, intrinsic feature extractions and representation schemes, machine learning methods and recognition mechanisms etc. Those approaches based on different principles had been shown possessing varying degrees of effectiveness in different aspects. It is expected that the techniques of information fusion with integrating the advantages of existing methods will boost the recognition performance. This paper deals with developing effective approaches for face recognition using information fusion techniques based on integrating multiple cues. The multiple stage integrating techniques dedicated to localization of landmark points and pose estimation were presented. The precise data of localization of landmarks and pose estimation provide the essential geometry basics for further processing. A face recognition classifier scheme with integration of multiple feature representation and multiple block region scores is also proposed. The experiment results show that the proposed approach can reduce equal error rate EER significantly, compared with using single feature and single block representations. The proposed approach had been shown possessing the best performance in participating MCFR2011 competition.     

一种语音特征参数子分量分析与有效性评价的新方法

语音信号中包含语义和说话人个性两大特征,其有效提取和强化对语音识别和说话人识别有着非常重要的意义。本文提出了一种语音特征参数中语义和个性特征子分量分析与有效性评价的4S方法,对语义和个性特征的成份比例进行分析,并通过量化指标评判特征参数对语音识别和说话人识别的有效性。运用4S分析方法对目前常用的特征参数LPC, LPCC和MFCC的子分量分析与有效性评价结果表明,所有的特征参数都更多地包含了语义特征信息,语义特征和说话人个性特征的成份比例因子LIR分别为1．30、1．44和1．61,并且,三种参数对语音识别和说话人识别的有效性均呈现出依次提高的特性。     

基于听感量化编码的神经网络语音合成方法研究

针对当前神经网络声学建模中数据混用困难的问题,文中提出了一种基于听感量化编码的神经网络语音合成方法。通过设计听感量化编码模型学习海量语音在音色、语种、情感上的不同差异表征,构建统一的多人数据混合训练的神经网络声学模型。在统一的听感量化编码声学模型内通过数据共享和迁移学习,可以显著降低合成系统搭建的数据量要求,并实现对合成语音的音色、语种、情感等属性的有效控制。提升了神经网络语音合成的质量和灵活性,一小时数据构建语音合成系统自然度可达到4.0MOS分,达到并超过普通说话人水平。     

Impostor Detection in Speaker Recognition Using Confusion‐Based Confidence Measures

In this letter, we introduce confusion‐based confidence measures for detecting an impostor in speaker recognition, which does not require an alternative hypothesis. Most traditional speaker verification methods are based on a hypothesis test, and their performance depends on the robustness of an alternative hypothesis. Compared with the conventional Gaussian mixture model–universal background model (GMM‐UBM) scheme, our confusion‐based measures show better performance in noise‐corrupted speech. The additional computational requirements for our methods are negligible when used to detect or reject impostors.     

Text‐Independent Speaker Verification Using Variational Gaussian Mixture Model

This paper concerns robust and reliable speaker model training for text‐independent speaker verification. The baseline speaker modeling approach is the Gaussian mixture model (GMM). In text‐independent speaker verification, the amount of speech data may be different for speakers. However, we still wish the modeling approach to perform equally well for all speakers. Besides, the modeling technique must be least vulnerable against unseen data. A traditional approach for GMM training is expectation maximization (EM) method, which is known for its overfitting problem and its weakness in handling insufficient training data. To tackle these problems, variational approximation is proposed. Variational approaches are known to be robust against overtraining and data insufficiency. We evaluated the proposed approach on two different databases, namely KING and TFarsdat. The experiments show that the proposed approach improves the performance on TFarsdat and KING databases by 0.56% and 4.81%, respectively. Also, the experiments show that the variationally optimized GMM is more robust against noise and the verification error rate in noisy environments for TFarsdat dataset decreases by 1.52%.     

基于区分性GMM文本无关的话者识别的研究

说话人识别的关键在于如何为集合中的每一个人建立一个能表征该说话人个性特征的声学模型,建模方法将会严重影响系统的性能。基于当今与文本无关的话者识别的主流模型——高斯混合模型（Gaussian Mixture Model,GMM）的基础上,从声学的角度剖析了男女发音的差别,以增加说话人之间的差异性为出发点,引入竞争性思想和通用背景模型（Universal Background Model,UBM）,提出了具有区分性的GMM的建模方法,克服了传统GMM需要大量训练样本的局限性和UBM将说话人强制服从统一分布的弱点。最后实验的对比结果表明,具有区分性的GMM相比传统的高斯混合模型在识别率上有所提高。     

Eigenvoices: A compact representation of speakers in model space

In this article, we present a new approach to modeling speaker-dependent systems. The approach was inspired by the eigenfaces techniques used in face recognition. We build a linear vector space of low dimensionality, called eigenspace, in which speakers are located. The basis vectors of this space are called eigenvoices. Each eigenvoice models a direction of inter-speaker variability. The eigenspace is built during the training phase. Then, any speaker model can be expressed as a linear combination of eigenvoices. The benefits of this technique as set forth in this article reside in the reduction of the number of parameters that describe a model. Thereby we are able to reduce the number of parameters to estimate, as well as computation and/or storage costs. We apply the approach to speaker adaptation and speaker recognition. Some experimental results are supplied.     

A General Bayesian Model for Speaker Verification

This paper presents a general Bayesian model for speaker verification tasks.It is a generative probability model.Due to its simple analytical property,a computationally efficient expectation-maximization algorithm can be derived to obtain the model parameters.A closedform solution,which allows the scalable size of enrollment set,is given in a full Bayesian way for making speaker verification decisions.Factor analysis technique is employed to model the speaker-specific components,then the redundant information in this model will be dropped.Experimental results are evaluated by both equal error rate and minimum detection cost function.The proposed approach shows promising results on the National institute of standards and technology (NIST) Speaker recognition evaluation (SRE) 2010 extended and 2012 core tasks.Significant improvement is obtained when comparing with Gaussian probabilistic linear discriminant analysis,especially under phone-call conditions and mismatched train-test channel conditions.Contrast experimental results with other popular generative probability models are also presented in this paper.     

基于本征音子说话人子空间的说话人自适应算法

本征音子说话人自适应算法在自适应数据量充足时可以取得很好的自适应效果,但在自适应数据量不足时会出现严重的过拟合现象。为此该文提出一种基于本征音子说话人子空间的说话人自适应算法来克服这一问题。首先给出基于隐马尔可夫模型-高斯混合模型(HMM-GMM)的语音识别系统中本征音子说话人自适应的基本原理。其次通过引入说话人子空间对不同说话人的本征音子矩阵间的相关性信息进行建模;然后通过估计说话人相关坐标矢量得到一种新的本征音子说话人子空间自适应算法。最后将本征音子说话人子空间自适应算法与传统说话人子空间自适应算法进行了对比。基于微软语料库的汉语连续语音识别实验表明,与本征音子说话人自适应算法相比,该算法在自适应数据量极少时能大幅提升性能,较好地克服过拟合现象。与本征音自适应算法相比,该算法以较小的性能牺牲代价获得了更低的空间复杂度而更具实用性。     

基于段长分布的HMM语音识别模型

本文针对齐次HMM语音识别模型在使用段长信息时存在的缺陷,形式化地定义了一种适合语音信号描述的自左向右非齐次隐含马尔科夫模型,证明了这种模型的状态转移概率表示与状态段长表示的等效性,并在此基础上提出了基于段长分布的HMM模型(DDBHMM).非特定人连续语音实验结果表明,仅仅利用状态段长信息的DDBHMM语音识别模型比经典HMM模型的性能有了明显的提高(误识率降低了17.8%),展示了DDBHMM的良好的性能,为语音信号的时长、语速、时间断续性以及语音特征的相关性等重要特征的描述和利用开辟了空间.     

基于BP神经网络的说话人识别技术的实现

说话人识别就是从说话人的一段语音中提取出说话人的个性特征,通过对这些个人特征的分析和识别,从而达到对说话人进行辨认或者确认的目的。神经网络是一种基于非线性理论的分布式并行处理网络模型,具有很强的模式分类能力及对不完全信息的鲁棒性,为说话人识别技术提供了一种独特的方法。BP(Back-propagation Neural Network)是一种非循环多级网络训练算法,有输入层,输出层和N个隐含层组成。首先概述了语音识别技术,介绍了BP神经网络训练过程的7个步骤及其模型,如何建立BP神经网络模型。同时介绍了与其相关的特征参数的提取,神经网络的训练和识别过程,最后,通过编程在Linux系统下实现说话人身份的识别。     

Effects of speech coding on speaker verification

The impact of speech coding on automatic speaker verification is investigated. Two different coders, classifiers and parametric representations of speech are considered. It is found that coding degrades mean speaker verification accuracy. Classifications of speech frames, for a given speaker, fluctuate by up to 30% relative to the uncoded case