A unified approach to statistical tomography using coordinatedescent optimization

Over the past years there has been considerable interest in statistically optimal reconstruction of cross-sectional images from tomographic data. In particular, a variety of such algorithms have been proposed for maximum a posteriori (MAP) reconstruction from emission tomographic data. While MAP estimation requires the solution of an optimization problem, most existing reconstruction algorithms take an indirect approach based on the expectation maximization (EM) algorithm. We propose a new approach to statistically optimal image reconstruction based on direct optimization of the MAP criterion. The key to this direct optimization approach is greedy pixel-wise computations known as iterative coordinate decent (ICD). We propose a novel method for computing the ICD updates, which we call ICD/Newton-Raphson. We show that ICD/Newton-Raphson requires approximately the same amount of computation per iteration as EM-based approaches, but the new method converges much more rapidly (in our experiments, typically five to ten iterations). Other advantages of the ICD/Newton-Raphson method are that it is easily applied to MAP estimation of transmission tomograms, and typical convex constraints, such as positivity, are easily incorporated.     

Genetic algorithms for finite mixture model based voxel classification in neuroimaging

Finite mixture models (FMMs) are an indispensable tool for unsupervised classification in brain imaging. Fitting an FMM to the data leads to a complex optimization problem. This optimization problem is difficult to solve by standard local optimization methods, such as the expectation-maximization (EM) algorithm, if a principled initialization is not available. In this paper, we propose a new global optimization algorithm for the FMM parameter estimation problem, which is based on real coded genetic algorithms. Our specific contributions are two-fold: 1) we propose to use blended crossover in order to reduce the premature convergence problem to its minimum and 2) we introduce a completely new permutation operator specifically meant for the FMM parameter estimation. In addition to improving the optimization results, the permutation operator allows for imposing biologically meaningful constraints to the FMM parameter values. We also introduce a hybrid of the genetic algorithm and the EM algorithm for efficient solution of multidimensional FMM fitting problems. We compare our algorithm to the self-annealing EM-algorithm and a standard real coded genetic algorithm with the voxel classification tasks within the brain imaging. The algorithms are tested on synthetic data as well as real three-dimensional image data from human magnetic resonance imaging, positron emission tomography, and mouse brain MRI. The tissue classification results by our method are shown to be consistently more reliable and accurate than with the competing parameter estimation methods.     

Data-aided channel estimation for turbo-PIC MIMO detectors

We consider data-aided channel estimation for multiple input multiple output (MIMO) systems when iterative parallel interference cancellation (PIC) is performed for signal detection. We compare some data-aided channel estimation methods based on expectation maximization (EM) algorithm or on hard estimated transmit symbols. In particular, we propose a modified EM-based approach and show that when few iterations are to be performed, it provides considerable performance improvement.     

Maximum-entropy expectation-maximization algorithm for image reconstruction and sensor field estimation.

In this paper, we propose a maximum-entropy expectation-maximization (MEEM) algorithm. We use the proposed algorithm for density estimation. The maximum-entropy constraint is imposed for smoothness of the estimated density function. The derivation of the MEEM algorithm requires determination of the covariance matrix in the framework of the maximum-entropy likelihood function, which is difficult to solve analytically. We, therefore, derive the MEEM algorithm by optimizing a lower-bound of the maximum-entropy likelihood function. We note that the classical expectation-maximization (EM) algorithm has been employed previously for 2-D density estimation. We propose to extend the use of the classical EM algorithm for image recovery from randomly sampled data and sensor field estimation from randomly scattered sensor networks. We further propose to use our approach in density estimation, image recovery and sensor field estimation. Computer simulation experiments are used to demonstrate the superior performance of the proposed MEEM algorithm in comparison to existing methods.     

Maximum-likelihood parameter estimation for unsupervised stochasticmodel-based image segmentation

An unsupervised stochastic model-based approach to image segmentation is described, and some of its properties investigated. In this approach, the problem of model parameter estimation is formulated as a problem of parameter estimation from incomplete data, and the expectation-maximization (EM) algorithm is used to determine a maximum-likelihood (ML) estimate. Previously, the use of the EM algorithm in this application has encountered difficulties since an analytical expression for the conditional expectations required in the EM procedure is generally unavailable, except for the simplest models. In this paper, two solutions are proposed to solve this problem: a Monte Carlo scheme and a scheme related to Besag's (1986) iterated conditional mode (ICM) method. Both schemes make use of Markov random-field modeling assumptions. Examples are provided to illustrate the implementation of the EM algorithm for several general classes of image models. Experimental results on both synthetic and real images are provided.     

An expectation-maximization algorithm based Kalman smoother approach for event-related desynchronization (ERD) estimation from EEG

We consider the problem of event-related desynchronization (ERD) estimation. In existing approaches, model parameters are usually found manually through experimentation, a tedious task that often leads to suboptimal estimates. We propose an expectation-maximization (EM) algorithm for model parameter estimation that is fully automatic and gives optimal estimates. Further, we apply a Kalman smoother to obtain ERD estimates. Results show that the EM algorithm significantly improves the performance of the Kalman smoother. Application of the proposed approach to the motor-imagery EEG data shows that useful ERD patterns can be obtained even without careful selection of frequency bands.     

An Efficient Adaptive Channel Estimation Algorithm for MIMO OFDM Systems—Study of Doppler Spread Tolerance

In this paper we develop an adaptive MIMO channel estimation algorithm for space–time block coded OFDM systems. The presented algorithm is based on Expectation Maximization (EM) technique by decomposing the superimposed received signals into their signal components, and estimating the channel parameters of each signal component separately. We also study and compare our proposed EM-based algorithm with a previously introduced recursive-least-squares based algorithm for MIMO OFDM systems. At each iteration the EM algorithm decomposes the problem of multi-channel estimation into channel estimation for each transmit–receive link. In this paper we also study the Doppler spread tolerance of our proposed algorithm in a fast fading environment, and investigate how it affects the system BER performance.     

Fast numerical modeling of multitransmitter electromagnetic data using multigrid quasi-linear approximation

Multitransmitter electromagnetic (EM) surveys are widely used in remote-sensing and geophysical exploration. The interpretation of the multitransmitter geophysical data requires numerous three-dimensional (3-D) modelings of the responses of the receivers for different geoelectrical models of complex geological formations. In this paper, we introduce a fast method for 3-D modeling of EM data, based on a modified version of quasilinear approximation, which uses a multigrid approach. This method significantly speeds up the modeling of multitransmitter-multireceiver surveys. The developed algorithm has been applied for the interpretation of marine controlled-source electromagnetic (MCSEM) data. We have tested our new method using synthetic problems and for the simulation of MCSEM data for a geoelectrical model of a Gemini salt body.     

Iterative and recursive estimators for hidden Markoverrors-in-variables models

In this paper we propose maximum-likelihood (ML) estimation of errors in variables models with finite-state Markovian disturbances. Such models have applications in econometrics, speech processing, communication systems, and neurobiological signal processing. We derive the maximum likelihood (ML) model estimates using the expectation maximization (EM) algorithm. Then two recursive or “on-line” estimation schemes are derived for estimating such models. The first on-line algorithm is based on the EM algorithm and uses stochastic approximations to maximize the Kullback-Leibler (KL) information measure. The second on-line algorithm we propose is a gradient-based scheme and uses stochastic approximations to maximize the log likelihood     

On the Performance of FFT/DWT/DCT-based OFDM Systems with Chaotic Interleaving and Channel Estimation Algorithms

The efficiency of data transmission over fading channels in orthogonal frequency division multiplexing (OFDM) systems depends on the employed interleaving method. In this study, we propose an improved chaotic interleaving scheme which aims to improve the performance of OFDM system under fading channel. In the proposed scheme, the binary data is interleaved with chaotic Baker map prior to the modulation process. In the sequel, significant degree of encryption is being added during data transmission. The performance of the proposed approach is tested on the conventional fast Fourier transform OFDM, discrete wavelet transform OFDM, and discrete cosine transform OFDM with and without chaotic interleaving. Furthermore, an expectation–maximization (EM) algorithm is proposed for improving channel impulse response (CIR) estimation based on a maximum likelihood principle. The proposed scheme makes use of EM algorithm to update the channel estimates until convergence is reached. The simulation results show the efficiency of the proposed algorithms under Rayleigh fading environments where the symbol error rate essentially coincides with that of the perfect channel case after the fifth EM iteration.     

A statistical multiscale framework for Poisson inverse problems

This paper describes a statistical multiscale modeling and analysis framework for linear inverse problems involving Poisson data. The framework itself is founded upon a multiscale analysis associated with recursive partitioning of the underlying intensity, a corresponding multiscale factorization of the likelihood (induced by this analysis), and a choice of prior probability distribution made to match this factorization by modeling the “splits” in the underlying partition. The class of priors used here has the interesting feature that the “noninformative” member yields the traditional maximum-likelihood solution; other choices are made to reflect prior belief as to the smoothness of the unknown intensity. Adopting the expectation-maximization (EM) algorithm for use in computing the maximum a posteriori (MAP) estimate corresponding to our model, we find that our model permits remarkably simple, closed-form expressions for the EM update equations. The behavior of our EM algorithm is examined, and it is shown that convergence to the global MAP estimate can be guaranteed. Applications in emission computed tomography and astronomical energy spectral analysis demonstrate the potential of the new approach     

Edge-preserving joint motion-disparity estimation in stereo image sequences

An energy-based joint motion and disparity estimation algorithm with an anisotropic diffusion operator is proposed to yield correct and dense displacement vectors. We propose two energy models; the joint estimation model and the simultaneous joint estimation model. In the joint estimation model, we compute the initial disparity in the current frame with the joint estimation constraint, using the left and right motions and the disparity in the previous frame; therefore, the model is prevented from being trapped in the local minima. Then, we regularize this disparity by using our proposed energy model. In the simultaneous joint estimation model, we propose an energy functional that consists of fidelity and smoothing terms for the left and right motions and the joint data terms. We estimate the left and right motions simultaneously in order to increase correctness. We use the Euler–Lagrange equation with variational methods and solve the equation with the finite difference method (FDM) to minimize the energy model. Experimental results show that the proposed algorithm provides accurate motion-disparity maps that reflect the constraints of motion and disparity, and preserve the discontinuities of the object boundaries well.     

Three-dimensional building detection and modeling using astatistical approach

In this paper, we address the problem of building reconstruction in high-resolution stereoscopic aerial imagery. We present a hierarchical strategy to detect and model buildings in urban sites, based on a global focusing process, followed by a local modeling. During the first step, we extract the building regions by exploiting to the full extent the depth information obtained with a new adaptive correlation stereo matching. In the modeling step, we propose a statistical approach, which is competitive to the sequential methods using segmentation and modeling. This parametric method is based on a multiplane model of the data, interpreted as a mixture model. From a Bayesian point of view the so-called augmentation of the model with indicator variables allows using stochastic algorithms to achieve both model parameter estimation and plane segmentation. We then report a Monte Carlo study of the performance of the stochastic algorithm on synthetic data, before displaying results on real data.     

Accelerated iterative reconstruction for positron emission tomography based on the em algorithm for maximum likelihood estimation

The EM method that was originally developed for maximum likelihood estimation in the context of mathematical statistics may be applied to a stochastic model of positron emission tomography (PET). The result is an iterative algorithm for image reconstruction that is finding increasing use in PET, due to its attractive theoretical and practical properties. Its major disadvantage is the large amount of computation that is often required, due to the algorithm's slow rate of convergence. This paper presents an accelerated form of the EM algorithm for PET in which the changes to the image, as calculated by the standard algorithm, are multiplied at each iteration by an overrelaxation parameter. The accelerated algorithm retains two of the important practical properties of the standard algorithm, namely the selfnormalization and nonnegativity of the reconstructed images. Experimental results are presented using measured data obtained from a hexagonal detector system for PET. The likelihood function and the norm of the data residual were monitored during the iterative process. According to both of these measures, the images reconstructed at iterations 7 and 11 of the accelerated algorithm are similar to those at iterations 15 and 30 of the standard algorithm, for two different sets of data. Important theoretical properties remain to be investigated, namely the convergence of the accelerated algorithm and its performance as a maximum likelihood estimator.     

EM Based Data Detection Algorithm for Downlink of a Single Hop Relaying Network

This work proposes an Expectation–Maximization (EM) based joint channel estimation and data detection algorithm and a time sharing hopped cooperative communication for the downlink of a communication network. The algorithm was applied to the downlink of a single hop relaying network. The relaying technique is a variant of the decode and forward technique. However, it is a time sharing single hop cooperative communication technique applied to a multi-carrier code division multiple access used in cellular networks. The proposed EM algorithm is compared with the minimum mean square estimation (MMSE) method using simulation. In addition, the proposed relaying technique is compared with cooperative communication on the considered cellular network for frequency-selective fading channels. Simulation results illustrate that our proposed EM algorithm performs better than the MMSE algorithm and that our proposed relaying technique is better than the normal case and the cooperative diversity case.     

Mixtures of general linear models for functional neuroimaging

We set out a new general framework for making inferences from neuroimaging data, which includes a standard approach to neuroimaging analysis, statistical parametric mapping (SPM), as a special case. The model offers numerous conceptual and statistical advantages that derive from analyzing data at the "cluster level" rather than the "voxel level" and from explicit modeling of the shape and position of clusters of activation. This provides a natural and principled way to pool data from nearby voxels for parameter and variance-component estimation. The model can also be viewed as performing a spatio-temporal cluster analysis. The parameters of the model are estimated using an expectation maximization (EM) algorithm.     

Efficient Huber-Markov edge-preserving image restoration.

The regularization of the least-squares criterion is an effective approach in image restoration to reduce noise amplification. To avoid the smoothing of edges, edge-preserving regularization using a Gaussian Markov random field (GMRF) model is often used to allow realistic edge modeling and provide stable maximum a posteriori (MAP) solutions. However, this approach is computationally demanding because the introduction of a non-Gaussian image prior makes the restoration problem shift-variant. In this case, a direct solution using fast Fourier transforms (FFTs) is not possible, even when the blurring is shift-invariant. We consider a class of edge-preserving GMRF functions that are convex and have nonquadratic regions that impose less smoothing on edges. We propose a decomposition-enabled edge-preserving image restoration algorithm for maximizing the likelihood function. By decomposing the problem into two subproblems, with one shift-invariant and the other shift-variant, our algorithm exploits the sparsity of edges to define an FFT-based iteration that requires few iterations and is guaranteed to converge to the MAP estimate.     

Estimating Dynamic Traffic Matrices by Using Viable Routing Changes

In this paper we propose a new approach for dealing with the ill-posed nature of traffic matrix estimation. We present three solution enhancers: an algorithm for deliberately changing link weights to obtain additional information that can make the underlying linear system full rank; a cyclo-stationary model to capture both long-term and short-term traffic variability, and a method for estimating the variance of origin-destination (OD) flows. We show how these three elements can be combined into a comprehensive traffic matrix estimation procedure that dramatically reduces the errors compared to existing methods. We demonstrate that our variance estimates can be used to identify the elephant OD flows, and we thus propose a variant of our algorithm that addresses the problem of estimating only the heavy flows in a traffic matrix. One of our key findings is that by focusing only on heavy flows, we can simplify the measurement and estimation procedure so as to render it more practical. Although there is a tradeoff between practicality and accuracy, we find that increasing the rank is so helpful that we can nevertheless keep the average errors consistently below the 10% carrier target error rate. We validate the effectiveness of our methodology and the intuition behind it using commercial traffic matrix data from Sprint's Tier-1 backbone.     

Independent Component Analysis Applied to fMRI Data: A Generative Model for Validating Results

Methods for testing and validating independent component analysis (ICA) results in fMRI are growing in importance as the popularity of this model for studying brain function increases. We introduce and apply a synthesis/analysis model for analyzing functional Magnetic Resonance Imaging (fMRI) data using ICA. Classes of signal types relevant to fMRI are described and a statistical approach for validation of simulation results is developed. Additionally, we propose an empirical version of our validation approach to test the performance of various ICA approaches in “hybrid” fMRI data, a mixture of real fMRI data and known (validatable) sources. The synthesis portion of the model assumes statistically independent spatial sources in the brain. We also assume that the fMRI scanner acquires overdetermined data such that there are more time points than hemodynamic brain sources. We propose several signal classes relevant to fMRI and discuss the properties of each. The analysis portion of the model includes several candidates for spatial smoothing, ICA algorithm, and data reduction. We use the Kullback-Leibler divergence between the estimated source distributions and the “true” distributions as a measure of the optimality of the final ICA decomposition. Using this model, we generate fMRI-like data and optimize the analysis stage as a function of ICA algorithm, data reduction scheme, and spatial smoothing. An example of how our synthesis/analysis model can be used in validating an fMRI experiment is demonstrated using simulations and “hybrid” fMRI data.     

Noise Estimation in Magnitude MR Datasets

Estimating the noise parameter in magnitude magnetic resonance (MR) images is important in a wide range of applications. We propose an automatic noise estimation method that does not rely on a substantial proportion of voxels being from the background. Specifically, we model the magnitude of the observed signal as a mixture of Rice distributions with common noise parameter. The expectation-maximization (EM) algorithm is used to estimate all parameters, including the common noise parameter. The algorithm needs initializing values for which we provide some strategies that work well. The number of components in the mixture model also needs to be estimated en route to noise estimation and we provide a novel approach to doing so. Our methodology performs very well on a range of simulation experiments and physical phantom data. Finally, the methodology is demonstrated on four clinical datasets.