Monte Carlo methods for signal processing: a review in the statistical signal processing context

In this article, MCMC (Markov chain Monte Carlo methods) and SMC (sequential Monte Carlo methods) are introduced to sample and/or maximize high-dimensional probability distributions. These methods enable to perform likelihood or Bayesian inference for complex non-Gaussian signal processing problems.     

一种基于混合MCMC的时域波达方向估计算法

基于马尔科夫链蒙特卡洛（Markov Chain Monte Carlo,MCMC）方法的时域波达方向估计算法通过构造马尔科夫链的方式来对波达方向进行估计,但是现有的算法在马尔科夫链的收敛速度和结果上并没有表现出很好的鲁棒性。为了优化算法的性能,采用多(短)链并行的方式代替原来的长链生成方式,提高了算法收敛的稳定性;并对特定模型下的构造过程进行分析,优化了状态空间,提高了算法的搜索效率;同时结合多混合的MCMC方法,进一步提高了算法估计的精确度和收敛速度。仿真结果表明,改进后的算法对波达方向估计的准确性和实时性都有很大提升。     

Variational Bayes inference of spatial mixture models for segmentation

Mixture models are commonly used in the statistical segmentation of images. For example, they can be used for the segmentation of structural medical images into different matter types, or of statistical parametric maps into activating and nonactivating brain regions in functional imaging. Spatial mixture models have been developed to augment histogram information with spatial regularization using Markov random fields (MRFs). In previous work, an approximate model was developed to allow adaptive determination of the parameter controlling the strength of spatial regularization. Inference was performed using Markov Chain Monte Carlo (MCMC) sampling. However, this approach is prohibitively slow for large datasets. In this work, a more efficient inference approach is presented. This combines a variational Bayes approximation with a second-order Taylor expansion of the components of the posterior distribution, which would otherwise be intractable to Variational Bayes. This provides inference on fully adaptive spatial mixture models an order of magnitude faster than MCMC. We examine the behavior of this approach when applied to artificial data with different spatial characteristics, and to functional magnetic resonance imaging statistical parametric maps.     

Separation of Non-Negative Mixture of Non-Negative Sources Using a Bayesian Approach and MCMC Sampling

This paper addresses blind-source separation in the case where both the source signals and the mixing coefficients are non-negative. The problem is referred to as non-negative source separation and the main application concerns the analysis of spectrometric data sets. The separation is performed in a Bayesian framework by encoding non-negativity through the assignment of Gamma priors on the distributions of both the source signals and the mixing coefficients. A Markov chain Monte Carlo (MCMC) sampling procedure is proposed to simulate the resulting joint posterior density from which marginal posterior mean estimates of the source signals and mixing coefficients are obtained. Results obtained with synthetic and experimental spectra are used to discuss the problem of non-negative source separation and to illustrate the effectiveness of the proposed method.     

Fully Bayesian estimation of Gibbs hyperparameters for emissioncomputed tomography data

In recent years, many investigators have proposed Gibbs prior models to regularize images reconstructed from emission computed tomography data. Unfortunately, hyperparameters used to specify Gibbs priors can greatly influence the degree of regularity imposed by such priors and, as a result, numerous procedures have been proposed to estimate hyperparameter values, from observed image data. Many of these, procedures attempt to maximize the joint posterior distribution on the image scene. To implement these methods, approximations to the joint posterior densities are required, because the dependence of the Gibbs partition function on the hyperparameter values is unknown. Here, the authors use recent results in Markov chain Monte Carlo (MCMC) sampling to estimate the relative values of Gibbs partition functions and using these values, sample from joint posterior distributions on image scenes. This allows for a fully Bayesian procedure which does not fix the hyperparameters at some estimated or specified value, but enables uncertainty about these values to be propagated through to the estimated intensities. The authors utilize realizations from the posterior distribution for determining credible regions for the intensity of the emission source. The authors consider two different Markov random field (MRF) models-the power model and a line-site model. As applications they estimate the posterior distribution of source intensities from computer simulated data as well as data collected from a physical single photon emission computed tomography (SPECT) phantom     

Generalized Gaussian mixture models as a nonparametric Bayesian approach for clustering using class-specific visual features

Recently, there has been a growing interest in the problem of learning mixture models from data. The reasons and motivations behind this interest are clear, since finite mixture models offer a formal approach to the important problems of clustering and data modeling. In this paper, we address the problem of modeling non-Gaussian data which are largely present, and occur naturally, in several computer vision and image processing applications via the learning of a generative infinite generalized Gaussian mixture model. The proposed model, which can be viewed as a Dirichlet process mixture of generalized Gaussian distributions, takes into account the feature selection problem, also, by determining a set of relevant features for each data cluster which provides better interpretability and generalization capabilities. We propose then an efficient algorithm to learn this infinite model parameters by estimating its posterior distributions using Markov Chain Monte Carlo (MCMC) simulations. We show how the model can be used, while comparing it with other models popular in the literature, in several challenging applications involving photographic and painting images categorization, image and video segmentation, and infrared facial expression recognition.     

Convergence analyses and comparisons of Markov chain Monte Carloalgorithms in digital communications

Markov chain Monte Carlo (MCMC) methods have been applied to the design of blind Bayesian receivers in a number of digital communications applications. The salient features of these MCMC receivers include the following: (a) they are optimal in the sense of achieving minimum symbol error rate; (b) they do not require knowledge of the channel states, nor do they explicitly estimate the channel by employing training signals or decision-feedback; and (c) they are well suited for iterative (turbo) processing in coded systems. We investigate the convergence behavior of several MCMC algorithms (both existing and new ones) in digital communication applications. The geometric convergence property of these algorithms is established by considering only the chains or the marginal chains corresponding to the transmitted digital symbols, which take values from a finite discrete set. We then focus on three specific applications, namely, the MCMC decoders in AWGN channels, ISI channels, and CDMA channels. The convergence rates for these algorithms are computed for small simulated datasets. Different convergence behaviors are observed. It is seen that differential encoding, parameter constraining, collapsing, and grouping are efficient ways of accelerating the convergence of the MCMC algorithms, especially in the presence of channel phase ambiguity     

A Bayesian approach to characterizing uncertainty in inverseproblems using coarse and fine-scale information

The Bayesian approach allows one to easily quantify uncertainty, at least in theory. In practice, however, the Markov chain Monte Carlo (MCMC) method can be computationally expensive, particularly in complicated inverse problems. We present a methodology for improving the speed and efficiency of an MCMC analysis by combining runs on different scales. By using a coarser scale, the chain can run faster (particularly when there is an external forward simulator involved in the likelihood evaluation) and better explore the posterior, being less likely to become stuck in local maxima. We discuss methods for linking the coarse chain back to the original fine-scale chain of interest. The resulting coupled chain can thus be run more efficiently without sacrificing the accuracy achieved at the finer scale     

Fast convergent monte carlo receiver for OFDM systems

The paper investigates the problem of the design of an optimal Orthogonal Frequency Division Multiplexing (OFDM) receiver against unknown frequency selective fading. A fast convergent Monte Carlo receiver is proposed. In the proposed method, the Markov Chain Monte Carlo (MCMC) methods are employed for the blind Bayesian detection without channel estimation. Meanwhile, with the exploitation of the characteristics of OFDM systems, two methods are employed to improve the convergence rate and enhance the efficiency of MCMC algorithms.One is the integration of the posterior distribution function with respect to the associated channel parameters, which is involved in the derivation of the objective distribution function; the other is the intra-symbol differential coding for the elimination of the bimodality problem resulting from the presence of unknown fading channels. Moreover, no matrix inversion is needed with the use of the orthogonality property of OFDM modulation and hence the computational load is significantly reduced. Computer simulation results show the effectiveness of the fast convergent Monte Carlo receiver.     

Exact Bayesian curve fitting and signal segmentation

We consider regression models where the underlying functional relationship between the response and the explanatory variable is modeled as independent linear regressions on disjoint segments. We present an algorithm for perfect simulation from the posterior distribution of such a model, even allowing for an unknown number of segments and an unknown model order for the linear regressions within each segment. The algorithm is simple, can scale well to large data sets, and avoids the problem of diagnosing convergence that is present with Monte Carlo Markov Chain (MCMC) approaches to this problem. We demonstrate our algorithm on standard denoising problems, on a piecewise constant AR model, and on a speech segmentation problem.     

一种基于阵列单通道的DOA估计新方法

为克服阵列多通道系统硬件量大,造价高及通道间存在不一致时性能恶化等不足,提出了一种新的基于阵列单通道的DOA估计方法。首先,通过射频开关控制接收通道轮流对各阵元进行采样建立新的阵列单通道窄带信号空间谱估计模型,接着基于该模型推导了来波方向的后验概率密度函数,然后结合马尔科夫链蒙特卡洛方法(MCMC),实现了DOA的估计。仿真实验结果表明,该方法参数估计性能好,分辨率高,能够处理相干信号。     

Joint Bayesian model selection and estimation of noisy sinusoidsvia reversible jump MCMC

In this paper, the problem of joint Bayesian model selection and parameter estimation for sinusoids in white Gaussian noise is addressed. An original Bayesian model is proposed that allows us to define a posterior distribution on the parameter space. All Bayesian inference is then based on this distribution. Unfortunately, a direct evaluation of this distribution and of its features, including posterior model probabilities, requires evaluation of some complicated high-dimensional integrals. We develop an efficient stochastic algorithm based on reversible jump Markov chain Monte Carlo methods to perform the Bayesian computation. A convergence result for this algorithm is established. In simulation, it appears that the performance of detection based on posterior model probabilities outperforms conventional detection schemes     

非线性功放信道下联合信道估计研究

为了解决非线性放大器在60 GHz毫米波信道中造成的非线性影响,提出了基于马尔科夫蒙特卡洛（Markov Chain Monte Carlo,MCMC）算法的联合信道估计与信号检测技术。采用的是MCMC算法中的Metropolis-Hastings方法,在非线性放大器及信道参数未知的情况下,通过被非线性和噪声污染的输出信号（观测信号）来估计非线性放大器的参数,检测输入信号被称为盲均衡技术。仿真结果给出了非线性参数与真实值的对比图以及随SNR变化的误比特率,性能优越。     

Double Markov random fields and Bayesian image segmentation

Markov random fields are used extensively in model-based approaches to image segmentation and, under the Bayesian paradigm, are implemented through Markov chain Monte Carlo (MCMC) methods. We describe a class of such models (the double Markov random field) for images composed of several textures, which we consider to be the natural hierarchical model for such a task. We show how several of the Bayesian approaches in the literature can be viewed as modifications of this model, made in order to make MCMC implementation possible. From a simulation study, conclusions are made concerning the performance of these modified models     

Reversible jump MCMC for joint detection and estimation of sourcesin colored noise

This paper presents a novel Bayesian solution to the difficult problem of joint detection and estimation of sources impinging on a single array of sensors in spatially colored noise with arbitrary covariance structure. Robustness to the noise covariance structure is achieved by integrating out the unknown covariance matrix in an appropriate posterior distribution. The proposed procedure uses the reversible jump Markov chain Monte Carlo (MCMC) method to extract the desired model order and direction-of-arrival parameters. We show that the determination of model order is consistent, provided a particular hyperparameter is within a specified range. Simulation results support the effectiveness of the method     

MCMC粒子滤波算法研究及应用

粒子滤波算法是一种基于贝叶斯估计的蒙特卡罗方法,适用于非线性非高斯系统的分析,被广泛应用于跟踪、定位等问题的研究中。为了解决粒子滤波算法在重采样后,丧失粒子多样性的问题,本文在粒子滤波算法的重采样步骤后,加入了马尔可夫链蒙特卡罗（Markov Chain Monte Carlo,简称MCMC）移动步骤,增加粒子的多样性。利用粒子滤波算法和MCMC粒子滤波算法对目标跟踪问题进行了仿真,并且通过分析仿真实验结果,比较了两种算法的性能,结果说明加入MCMC粒子滤波算法的性能优于粒子滤波算法。     

A Bayesian transformation model for wavelet shrinkage

Wavelet shrinkage estimators, in general, make the additive normal noise assumption and disregard the nonlinear nature of contamination. We develop Bayesian wavelet shrinkage estimators (based on the power transformations in the linear model) to accommodate a broad class of noise models in image processing applications. We intend to admit, under one roof, the widespread additive model, the product models common in imaging (such as in synthetic aperture radar (SAR) imagery), as well as noise that may exist amid these two extremes. Tactful prior elicitation in this model, such as the simultaneous assignment of mixture priors for wavelet coefficients and the transformation, imparts flexibility and ample insight into the underlying structure. The model permits estimation with unknown noise structure for reasonably unimodal and well-behaved (on the tails) distributions, wherein it can outperform common shrinkage estimators. Extensions with multiple transformations and Markov random field priors are also considered for adaptation to local variations in contamination. Modern Markov chain Monte Carlo (MCMC) Bayesian computation has been used for simulations and several examples are reported.     

Applying Bayesian mixtures-of-experts models to statistical description of smart power semiconductor reliability

Reliability prediction of semiconductor devices gains importance, since demand increases and resources, e.g. time, are restricted. Normally, methods focusing on technology aspects are applied. This work presents a more mathematical approach by using Bayesian statistics. Physical failure inspection and past research indicate that the data follow a bimodal distribution. Therefore, we suggest using a heteroscedastic mixture of two normal distributions to model the given data. To incorporate the dependency on different test settings, linear models are used for the means and the mixing proportion. Gamma distributions are proposed as priors for the model parameters, due to the physical restrictions concerning the sample space. For the variances hierarchical inverse gamma priors are applied. Sampling from the posterior is done by using Monte Carlo Markov Chain methods. The proposed mixtures-of-experts model shows good adaption to the behavior of the measurements as well as good prediction quality.     

Monte Carlo Bayesian Signal Processing for Wireless Communications

Many statistical signal processing problems found in wireless communications involves making inference about the transmitted information data based on the received signals in the presence of various unknown channel distortions. The optimal solutions to these problems are often too computationally complex to implement by conventional signal processing methods. The recently emerged Bayesian Monte Carlo signal processing methods, the relatively simple yet extremely powerful numerical techniques for Bayesian computation, offer a novel paradigm for tackling wireless signal processing problems. These methods fall into two categories, namely, Markov chain Monte Carlo (MCMC) methods for batch signal processing and sequential Monte Carlo (SMC) methods for adaptive signal processing. We provide an overview of the theories underlying both the MCMC and the SMC. Two signal processing examples in wireless communications, the blind turbo multiuser detection in CDMA systems and the adaptive detection in fading channels, are provided to illustrate the applications of MCMC and SMC respectively.     

A new likelihood-based modulation classification algorithm using MCMC

In this paper, a new likelihood-based method for classifying phase-amplitude-modulated signals in Additive White Gaussian Noise (AWGN) is proposed. The method introduces a new Markov Chain Monte Carlo (MCMC) algorithm, called the Adaptive Metropolis (AM) algorithm, to directly generate the samples of the target posterior distribution and implement the multidimensional integrals of likelihood function. Modulation classification is achieved along with joint estimation of unknown parameters by running an ergodic Markov Chain. Simulation results show that the proposed method has the advantages of high accuracy and robustness to phase and frequency offset.