Blind separation of non-stationary sources using continuous density hidden Markov models

Blind source separation (BSS) has attained much attention in signal processing society due to its ‘blind’ property and wide applications. However, there are still some open problems, such as underdetermined BSS, noise BSS. In this paper, we propose a Bayesian approach to improve the separation performance of instantaneous mixtures with non-stationary sources by taking into account the internal organization of the non-stationary sources. Gaussian mixture model (GMM) is used to model the distribution of source signals and the continuous density hidden Markov model (CDHMM) is derived to track the non-stationarity inside the source signals. Source signals can switch between several states such that the separation performance can be significantly improved. An expectation-maximization (EM) algorithm is derived to estimate the mixing coefficients, the CDHMM parameters and the noise covariance. The source signals are recovered via maximum a posteriori (MAP) approach. To ensure the convergence of the proposed algorithm, the proper prior densities, conjugate prior densities, are assigned to estimation coefficients for incorporating the prior information. The initialization scheme for the estimates is also discussed. Systematic simulations are used to illustrate the performance of the proposed algorithm. Simulation results show that the proposed algorithm has more robust separation performance in terms of similarity score in noise environments in comparison with the classical BSS algorithms in determined mixture case. Additionally, since the mixing matrix and the sources are estimated jointly, the proposed EM algorithm also works well in underdetermined case. Furthermore, the proposed algorithm converges quickly with proper initialization.     

Underdetermined Convolutive Blind Source Separation via Time–Frequency Masking

In this paper, we consider the problem of separation of unknown number of sources from their underdetermined convolutive mixtures via time-frequency (TF) masking. We propose two algorithms, one for the estimation of the masks which are to be applied to the mixture in the TF domain for the separation of signals in the frequency domain, and the other for solving the permutation problem. The algorithm for mask estimation is based on the concept of angles in complex vector space. Unlike the previously reported methods, the algorithm does not require any estimation of the mixing matrix or the source positions for mask estimation. The algorithm clusters the mixture samples in the TF domain based on the Hermitian angle between the sample vector and a reference vector using the well known k -means or fuzzy c -means clustering algorithms. The membership functions so obtained from the clustering algorithms are directly used as the masks. The algorithm for solving the permutation problem clusters the estimated masks by using k-means clustering of small groups of nearby masks with overlap. The effectiveness of the algorithm in separating the sources, including collinear sources, from their underdetermined convolutive mixtures obtained in a real room environment, is demonstrated.     

基于核估计的超定混合共轭盲信号分离方法

当混合信号的个数多于源信号时,盲源分离模型中的混合矩阵被描述为一个超定矩阵,因此不能直接通过估计逆矩阵的方法来得到分离矩阵。针对该线性超定混合情况提出了一种基于共轭梯度的盲源分离方法。该方法基于最小互信息准则,通过对行满秩分离矩阵的奇异值分解而引入了超定盲源分离的代价函数。利用共轭梯度优化算法推导出了迭代计算分离矩阵的更新公式。在每次迭代计算中,利用随机变量概率密度估计的核函数法在线估计分离信号的评价函数。避免了诸多传统盲分离算法中只能凭经验选取特定的非线性函数来代替评价函数的问题。仿真结果验证了所提算法的有效性。     

A parametric density model for blind source separation

In this paper, a parametric mixture density model is employed to be the source prior in blind source separation (BSS). A strict lower bound on the source prior is derived by using a variational method, which naturally enables the intractable posterior to be represented as a gaussian form. An expectation-maximization (EM) algorithm in closed form is therefore derived for estimating the mixing matrix and inferring the sources. Simulation results show that the proposed variational expectation-maximization algorithm can perform blind separation of not only speech source of more sources than mixtures, but also binary source of more sources than mixtures.     

Adaptive weighted orthogonal constrained algorithm for blind source separation

Blind source separation (BSS) consists of recovering the statistically independent source signals from their linear mixtures without knowing the mixing coefficients. Pre-whitening is a useful pre-processing technique in BSS. However, BSS algorithms based on the pre-whitened data lack the equivariance property, one of the significant properties in BSS. By transforming the pre-whitening into a weighted orthogonal constraint condition, this paper proposes a new definition of the contrast function. In light of the constrained optimization method, various weighted orthogonal constrained BSS algorithms with equivariance property are developed. Simulations on man-made signals and practical speech signals show the proposed weighted orthogonal constrained BSS algorithms have better separation ability, convergent speed and steady state performance.     

Blind source separation based on high-resolution time-frequency distributions

Blind source separation (BSS) based on time-frequency distributions (TFDs) exploits the underlying diagonal or off-diagonal structure of TFD matrices to separate the source signals. In this paper, we propose a new signal-independent kernel which is defined in both the time-lag and the Doppler-lag domain and satisfies most of the desirable properties of a TFD. The main objective of this research is to achieve the high resolution and the maximum cross-term reduction with the preferable diagonal or off-diagonal structure of TFD matrices in BSS applications. Moreover, a BSS approach is developed which includes first whitening mixed signals, then constructing a set of TFD matrices using the proposed TFD and the Hough transform, finally a joint diagonalization of a combined set of TFD matrices to estimate the mixing matrix and the source signals. By use of the techniques proposed in this paper, the improved performance of BSS of nonstationary signals has been achieved.     

Maximum contrast analysis for nonnegative blind source separation

In this paper, we propose a maximum contrast analysis (MCA) method for nonnegative blind source separation, where both the mixing matrix and the source signals are nonnegative. We first show that the contrast degree of the source signals is greater than that of the mixed signals. Motivated by this observation, we propose an MCA-based cost function. It is further shown that the separation matrix can be obtained by maximizing the proposed cost function. Then we derive an iterative determinant maximization algorithm for estimating the separation matrix. In the case of two sources, a closed-form solution exists and is derived. Unlike most existing blind source separation methods, the proposed MCA method needs neither the independence assumption, nor the sparseness requirement of the sources. The effectiveness of the new method is illustrated by experiments using X-ray images, remote sensing images, infrared spectral images, and real-world fluorescence microscopy images.     

From separability/identifiability properties of bilinear and linear-quadratic mixture matrix factorization to factorization algorithms

Blind source separation (BSS) and Blind Mixture Identification (BMI) methods typically concern unknown source signals, transferred through a given class of functions with unknown parameter values, which yields mixed observations. Using only these observations, BSS/BMI aims at estimating the source signals and/or mixing parameters. Most investigations concern linear instantaneous mixing functions. They contain two aspects. The first one consists in proposing general BSS/BMI principles, e.g. Independent Component Analysis, Sparse Component Analysis or Nonnegative Matrix Factorization (NMF), and/or deriving associated practical algorithms. The second aspect consists in analyzing the properties resulting from these principles. This is of utmost importance, to determine if the proposed BSS/BMI principles are guaranteed to separate the source signals and to identify the considered mixing model up to acceptable indeterminacies. These separability/identifiability analyses are even more important for nonlinear mixtures, that were shown to potentially yield higher indeterminacies. Among them, bilinear and linear-quadratic mixtures are receiving increasing attention, e.g. due to their application to remote sensing. Especially, extensions of NMF were recently proposed for them, but the resulting separability/identifiability properties were not analyzed. We here address this topic, moreover proceeding further by investigating Bilinear and Linear-Quadratic Mixture Matrix Factorization (BMMF and LQMMF) approaches without nonnegativity constraints. We especially show that, whereas nonlinearity is often considered to be a burden, it yields an essentially unique decomposition under mild conditions for BMMF. On the contrary, full LQMMF is shown to yield spurious solutions, which increases the usefulness of combining it with nonnegativity constraints in applications where data meet these constraints. Algorithms based on this framework are also defined in this paper and their performance is reported.     

基于时频单源点的欠定混合矩阵估计

主要研究了欠定盲源分离中的混合矩阵估计问题。提出了一种检测时频单源点的新方法,通过比较归一化的观测信号时频点的实部和虚部向量来检测时频单源点。与其他时频单源点检测方法相比,该方法简单而有效,同时降低了对检测条件的要求。采用K-means方法估计混合矩阵,通过去除聚类后每一类数据中偏离中心方向较远的数据点,进一步提高了混合矩阵的估计精度。仿真实验表明,与已有欠定混合矩阵估计算法相比,提出的方法有更高的估计精度。     

Source number estimation and separation algorithms of underdetermined blind separation

Recently, sparse component analysis （SCA） has become a hot spot in BSS research. Instead of independent component analysis （ICA）, SCA can be used to solve underdetermined mixture efficiently. Two-step approach （TSA） is one of the typical methods to solve SCA based BSS problems. It estimates the mixing matrix before the separation of the sources. K-means clustering is often used to estimate the mixing matrix. It relies on the prior knowledge of the source number strongly. However, the estimation of the source number is an obstacle. In this paper, a fuzzy clustering method is proposed to estimate the source number and mixing matrix simultaneously. After that, the sources are recovered by the shortest path method （SPM）. Simulations show the availability and robustness of the proposed method.     

An iterative inversion approach to blind source separation

We present an iterative inversion (II) approach to blind source separation (BSS). It consists of a quasi-Newton method for the resolution of an estimating equation obtained from the implicit inversion of a robust estimate of the mixing system. The resulting learning rule includes several existing algorithms for BSS as particular cases giving them a novel and unified interpretation. It also provides a justification of the Cardoso and Laheld (1996) step size normalization. The II method is first presented for instantaneous mixtures and then extended to the problem of blind separation of convolutive mixtures. Finally, we derive the necessary and sufficient asymptotic stability conditions for both the instantaneous and convolutive methods to converge.     

Blind estimation of channel parameters and source components for EEG signals: a sparse factorization approach

In this paper, we use a two-stage sparse factorization approach for blindly estimating the channel parameters and then estimating source components for electroencephalogram (EEG) signals. EEG signals are assumed to be linear mixtures of source components, artifacts, etc. Therefore, a raw EEG data matrix can be factored into the product of two matrices, one of which represents the mixing matrix and the other the source component matrix. Furthermore, the components are sparse in the time-frequency domain, i.e., the factorization is a sparse factorization in the time frequency domain. It is a challenging task to estimate the mixing matrix. Our extensive analysis and computational results, which were based on many sets of EEG data, not only provide firm evidences supporting the above assumption, but also prompt us to propose a new algorithm for estimating the mixing matrix. After the mixing matrix is estimated, the source components are estimated in the time frequency domain using a linear programming method. In an example of the potential applications of our approach, we analyzed the EEG data that was obtained from a modified Sternberg memory experiment. Two almost uncorrelated components obtained by applying the sparse factorization method were selected for phase synchronization analysis. Several interesting findings were obtained, especially that memory-related synchronization and desynchronization appear in the alpha band, and that the strength of alpha band synchronization is related to memory performance.     

Markov and Semi-Markov switching of source appearances for nonstationary independent component analysis.

Independent component analysis (ICA) is currently the most popularly used approach to blind source separation (BSS), the problem of recovering unknown source signals when their mixtures are observed but the actual mixing process is unknown. Many ICA algorithms assume that a fixed set of source signals consistently exists in mixtures throughout the time-series to be examined. However, real-world signals often have such difficult nonstationarity that each source signal abruptly appears or disappears, thus the set of active sources dynamically changes with time. In this paper, we propose switching ICA (SwICA), which focuses on such situations. The proposed approach is based on the noisy ICA formulated as a generative model. We employ a special type of hidden Markov model (HMM) to represent such prior knowledge that the source may abruptly appear or disappear with time. The special HMM setting t hen provides an effect ofvariable selection in a dynamic way. We use the variational Bayes (VB) method to derive an effective approximation of Bayesian inference for this model. In simulation experiments using artificial and realistic source signals, the proposed method exhibited performance superior to existing methods, especially in the presence of noise. The compared methods include the natural-gradient ICA with a nonholonomic constraint, and the existing ICA method incorporating an HMM source model, which aims to deal with general nonstationarities that may exist in source signals. In addition, the proposed method could successfully recover the source signals even when the total number of true sources was overestimated or was larger than that of mixtures. We also propose a modification of the basic Markov model into a semi-Markov model, and show that the semi-Markov one is more effective for robust estimation of the source appearance.     

Blind source separation based on constant modulus criterion and signal mutual information

We address the problem of adaptive blind source separation (BSS) from instantaneous multi-input multi-output (MIMO) channels. It is known that the constant modulus (CM) criterion can be used to extract unknown source signals. However, the existing CM-based algorithms normally extract the source signals in a serial manner. Consequently, the accuracy in extracting each source signal, except for the first one, depends on the accuracy of previous source extraction. This estimation error propagation (accumulation) will cause severe performance degradation. In this letter, we propose a new adaptive separation algorithm that can separate all source signals simultaneously by directly updating the separation matrix. The superior performance of the new algorithm is demonstrated by simulation examples.     

蚁群聚类的欠定盲源分离方法

利用欠定盲源分离情况下稀疏源信号具有直线聚类的特点,提出了一种估计混叠矩阵的新方法。通过对混叠信号进行标准化处理,使混叠信号形成球形簇,将线性聚类转变成致密聚类;利用蚁群聚类算法对其进行搜索得到聚类中心,从而获得对混叠矩阵的精确估计。该方法能实现源信号数目未知情况下的欠定盲源分离,且能推广到三路或更多路观测信号的情况。对语音信号的仿真结果证明,该方法能精确地分离和恢复原始信号。     

Blind source separation with unknown and dynamically changing number of source signals

The contrast function remains to be an open problem in blind source separation (BSS) when the number of source signals is unknown and/or dynamically changed. The paper studies this problem and proves that the mutual information is still the contrast function for BSS if the mixing matrix is of full column rank. The mutual information reaches its minimum at the separation points, where the random outputs of the BSS system are the scaled and permuted source signals, while the others are zero outputs. Using the property that the transpose of the mixing matrix and a matrix composed by m observed signals have the indentical null space with probability one, a practical method, which can detect the unknown number of source signals n, ulteriorly traces the dynamical change of the sources number with a few of data, is proposed. The effectiveness of the proposed theorey and the developed novel algorithm is verified by adaptive BSS simulations with unknown and dynamically changing number of source signals.     

Sparse representation and blind source separation of ill-posed mixtures

Bofill et al. discussed blind source separation (BSS) of sparse signals in the case of two sensors. However, as Bofill et al. pointed out, this method has some limitation. The potential function they introduced is lack of theoretical basis. Also the method could not be extended to solve the problem in the case of more than three sensors. In this paper, instead of the potential function method, a K-PCA method (combining K-clustering with PCA) is proposed. The new method is easy to be used in the case of more than three sensors. It is easy to be implemented and can provide accurate estimation of mixing matrix. Some criterion is given to check the effect of the mixing matrix A . Some simulations illustrate the availability and accuracy of the method we proposed.     

基于二阶统计和时间结构的盲信号分离方法

盲源分离(BSS)的目标就是在混合过程未知的情况下,仅仅依据观测得到的混合信号,恢复出不能直接观测的源信号。针对具有时间结构的源信号,即各个源信号分量满足空间上不相关但时间上相关,提出了一种基于二阶统计量的盲源分离方法。该方法首先对混合信号进行鲁棒预白化处理,其中依据最小描述长度准则对源信号的维数进行估计;然后通过对白化信号的时延协方差矩阵进行奇异值分解(SVD),从而实现源信号的盲分离。仿真中通过对一组语音信号的分离验证了算法的效果,并利用信号干扰比(SIR)和性能指标函数(PI)两个指标定量地对算法的性能进行了度量。     

小波去噪算法在含噪盲源分离中的应用

无噪模型下的盲源分离算法在信噪比较低的情况下并不适用。针对该情况一种解决方案就是先对含有高斯白噪声的混合信号进行去噪预处理,然后使用盲源分离算法进行分离。为此,本文提出了一种适用于信噪比较低条件下的基于平移不变量的小波去噪算法。该算法首先使用高频系数滑动窗口法准确估计含噪混合信号的噪声方差,然后使用Bayesshrink阈值估计算法 得到更加合理的阈值,最后在不降低去噪效果的同时缩小了平移不变量的范围,减少了运算量。实验仿真表明,在信噪比较低的情况下,与传统小波去噪算法相比,该算法可以更加有效地去除噪声,在很大程度上提升盲源分离算法的性能。     

病态盲分离情况下的混合矩阵估计研究

在盲信号分离技术中,当混合矩阵是病态情况时,基于信号稀疏性的两步法可用来解决这一问题,而如何估计混合矩阵则是两步法的关键。提出了一种估计混合矩阵的新方法,即通过搜索重构观测信号采样点,每次只需搜索出少数某源信号取值占优的采样点,就可以通过这些采样点处的重构观测信号数据,估计出混合矩阵的某一列。依次类推,可以估计出整个混合矩阵。该方法估计混合矩阵时对源信号的稀疏度要求较低,其实现算法不需优化过程,计算简单,因此其实用性较高。仿真结果表明了该方法有效,有很好的性能。通过大量的仿真试验给出了方法的定量性能分析。