Maximum-likelihood parameter estimation of discrete homogeneousrandom fields with mixed spectral distributions

This paper presents a maximum-likelihood solution to the general problem of fitting a parametric model to observations from a single realization of a real valued, 2-D, homogeneous random field with mixed spectral distribution. On the basis of a 2-D Wold-like decomposition, the field is represented as a sum of mutually orthogonal components of three types: purely indeterministic, harmonic, and evanescent. The proposed algorithm provides a complete solution to the joint estimation problem of the random field components. By introducing appropriate parameter transformations, the highly nonlinear least-squares problem that results from the maximization of the likelihood function is transformed into a separable least-squares problem. In this new problem, the solution for the unknown spectral supports of the harmonic and evanescent components reduces the problem of solving for the transformed parameters of the field to linear least squares. Solution of the transformation equations provides a complete solution of the field model parameter estimation problem     

Maximum-likelihood parameter estimation of the harmonic,evanescent, and purely indeterministic components of discretehomogeneous random fields

This paper presents a maximum-likelihood solution to the general problem of fitting a parametric model to observations from a single realization of a two-dimensional (2-D) homogeneous random field with mixed spectral distribution. On the basis of a 2-D Wold-like decomposition, the field is represented as a sum of mutually orthogonal components of three types: purely indeterministic, harmonic, and evanescent. The suggested algorithm involves a two-stage procedure. In the first stage, we obtain a suboptimal initial estimate for the parameters of the spectral support of the evanescent and harmonic components. In the second stage, we refine these initial estimates by iterative maximization of the conditional likelihood of the observed data, which is expressed as a function of only the parameters of the spectral supports of the evanescent and harmonic components. The solution for the unknown spectral supports of the harmonic and evanescent components reduces the problem of solving for the other unknown parameters of the field to a linear least squares. The Cramer-Rao lower bound on the accuracy of jointly estimating the parameters of the different components is derived, and it is shown that the bounds on the purely indeterministic and deterministic components are decoupled. Numerical evaluation of the bounds provides some insight into the effects of various parameters on the achievable estimation accuracy. The performance of the maximum-likelihood algorithm is illustrated by Monte Carlo simulations and is compared with the Cramer-Rao bound     

Bounds on the accuracy of estimating the parameters of discretehomogeneous random fields with mixed spectral distributions

This paper considers the achievable accuracy in jointly estimating the parameters of a real-valued two-dimensional (2-D) homogeneous random field with mixed spectral distribution, from a single observed realization of it. On the basis of a 2-D Wold-like decomposition, the field is represented as a sum of mutually orthogonal components of three types: purely indeterministic, harmonic, and evanescent. An exact form of the Cramer-Rao lower bound on the error variance in jointly estimating the parameters of the different components is derived. It is shown that the estimation of the harmonic component is decoupled from that of the purely indeterministic and the evanescent components. Moreover, the bound on the parameters of the purely indeterministic and the evanescent components is independent of the harmonic component. Numerical evaluation of the bounds provides some insight into the effects of various parameters on the achievable estimation accuracy     

A unified texture model based on a 2-D Wold-like decomposition

A unified texture model that is applicable to a wide variety of texture types found in natural images is presented. This model leads to the derivation of texture analysis and synthesis algorithms designed to estimate the texture parameters and to reconstruct the original texture field from these parameters. The texture field is assumed to be a realization of a regular homogeneous random field, which is characterized in general by a mixed spectral distribution. The texture field is orthogonally decomposed into a purely indeterministic component and a deterministic component. The deterministic component is further orthogonally decomposed into a harmonic component, and a generalized-evanescent component. Both analytical and experimental results show that the deterministic components should be parameterized separately from the purely indeterministic component. The model is very efficient in terms of the number of parameters required to faithfully represent textures. Reconstructed textures are practically indistinguishable from the originals     

Cramer-Rao bound on the estimation accuracy of complex-valuedhomogeneous Gaussian random fields

This paper considers the problem of the achievable accuracy in jointly estimating the parameters of a complex-valued two-dimensional (2-D) Gaussian and homogeneous random field from a single observed realization of it. Based on the 2-D Wold decomposition, the field is modeled as a sum of purely indeterministic, evanescent, and harmonic components. Using this parametric model, we first solve a key problem common to many open problems in parametric estimation of homogeneous random fields: that of expressing the field mean and covariance functions in terms of the model parameters. Employing the parametric representation of the observed field mean and covariance, we derive a closed-form expression for the Fisher information matrix (FIM) of complex-valued homogeneous Gaussian random fields with mixed spectral distribution. Consequently, the Cramer-Rao lower bound on the error variance in jointly estimating the model parameters is evaluated     

The evanescent field transform for estimating the parameters ofhomogeneous random fields with mixed spectral distributions

Parametric modeling and estimation of complex valued homogeneous random fields with mixed spectral distributions is a fundamental problem in two-dimensional (2-D) signal processing. The parametric model under consideration results from the 2-D Wold-type decomposition of the random field. The same model naturally arises as the physical model in problems of space-time adaptive processing of airborne radar. A computationally efficient algorithm for estimating the parameters of the field components is presented. The algorithm is based on a nonlinear operator that uniquely maps each evanescent component to a single exponential. The exponential's spatial frequency is a function of the spectral support parameters of the evanescent component. Hence, employing this operator, the problem of estimating the spectral support parameters of an evanescent field is replaced by the simpler problem of estimating the spatial frequency of a 2-D exponential. The properties of the operator are analyzed. The algorithm performance is illustrated and investigated using Monte Carlo simulations     

Texture coding using a Wold decomposition model

A novel approach for coding textured images is presented. The texture field is assumed to be a realization of a regular homogeneous random field, which can have a mixed spectral distribution. On the basis of a two-dimensional (2-D) Wold-like decomposition, the field is represented as a sum of a purely indeterministic, harmonic, and countable number of evanescent fields. We present an algorithm for estimating and coding the texture model parameters, and show that the suggested algorithm yields high-quality reconstructions at low bit rates. The model and the resulting coding algorithm are seen to be applicable to a wide variety of texture types found in natural images.     

Spectral analysis based on the canonical autoregressivedecomposition

Statistical inference for mixed spectral problems based on a parametric time series model is studied. The model used is based on the canonical autoregressive decomposition (CARD) and represents the underlying random process as the sum of an autoregressive process and sinusoids. Maximum likelihood estimation of the unknown parameters in the model is considered. The entire estimation problem can be shown to require a numerical maximization with respect to only the sinusoidal frequencies. An iterative algorithm to efficiently implement this maximization is presented. This enables us to examine a host of issues associated with a practical implementation of inferential procedures for mixed spectral problems. Some of the topics are accuracy of parameter estimates, selection of model orders, and sensitivity and robustness of the spectral estimates to modeling inaccuracies. The modeling approach, together with the inferential procedures, overcome many of the difficulties encountered in current spectral estimation techniques     

Maximum likelihood parameter estimation of F-ARIMA processes using the genetic algorithm in the frequency domain

This work aims to treat the parameter estimation problem for fractional-integrated autoregressive moving average (F-ARIMA) processes under external noise. Unlike the conventional approaches from the perspective of the time domain, a maximum likelihood (ML) method is developed in the frequency domain since the power spectrum of an F-ARIMA process is in a very explicit and more simple form. However, maximization of the likelihood function is a highly nonlinear estimation problem. Conventional searching algorithms are likely to converge to local maxima under this situation. Since the genetic algorithm (GA) tends to find the globally optimal solution without being trapped at local maxima, an estimation scheme based on the GA is therefore developed to solve the ML parameter estimation problem for F-ARIMA processes from the frequency domain perspective. In the parameter estimation procedure, stability of the F-ARIMA model is ensured, and convergence to the global optimum of the likelihood function is also guaranteed. Finally, several simulation examples are presented to illustrate the proposed estimation algorithm and exhibit its performance.     

A fast algorithm for linear estimation of two- dimensional isotropic random fields

The problem considered involves estimating a two-dimensional isotropic random field given noisy observations of this field over a disk of finite radius. By expanding the field and observations in Fourier series, and exploiting the covariance structure of the resulting Fourier coefficient processes, recursions are obtained for efficiently constructing the linear least-squares estimate of the field as the radius of the observation disk increases. These recursions are similar to the Levinson equations of one-dimensional linear prediction. In the spectral domain they take the form of Schrödinger equations, which are used to give an inverse spectral interpretation of our estimation procedure.     

有限贝塔刘维尔混合模型的变分学习及其应用

由于贝塔刘维尔分布的共轭先验分布中存在积分表达式,贝叶斯估计有限贝塔刘维尔混合模型参数异常困难.本文提出利用变分贝叶斯学习模型参数,采用gamma分布作为近似的先验分布并使用合理的非线性近似技术,得到了后验分布的近似解.与常用的EM算法相比,该方法能够同时估计模型参数和确定分量数,且避免了过拟合的问题.在合成数据集及场景分类问题上进行了大量的实验,实验结果验证了本文所提方法的有效性.     

Maximum likelihood parameter estimation of superimposed chirpsusing Monte Carlo importance sampling

We address the problem of parameter estimation of superimposed chirp signals in noise. The approach used here is a computationally modest implementation of a maximum likelihood (ML) technique. The ML technique for estimating the complex amplitudes, chirping rates, and frequencies reduces to a separable optimization problem where the chirping rates and frequencies are determined by maximizing a compressed likelihood function that is a function of only the chirping rates and frequencies. Since the compressed likelihood function is multidimensional, its maximization via a grid search is impractical. We propose a noniterative maximization of the compressed likelihood function using importance sampling. Simulation results are presented for a scenario involving closely spaced parameters for the individual signals     

纹理分析窗大小的高斯-马尔可夫随机场模型估计方法

在纹理分析中,窗口大小的选择对所提取特征的有效性及计算速度等有很大影响。文中利用高斯－马尔可夫随机场（ＧＭＲＦ）模型对纹理进行描述,采用最小平方误差估计获取纹理图像的随机场参数,并证明了这种估计的一致性。针对估计式在某些情况下可能无解,对该式作了改进,使其在实际应用中总能有解。利用估计的一致性,提出了一种系统估计纹理分析窗口大小的方法,实验表明了这种方法的有效性。     

Scalable data parallel algorithms for texture synthesis using Gibbsrandom fields

This article introduces scalable data parallel algorithms for image processing. Focusing on Gibbs and Markov random field model representation for textures, we present parallel algorithms for texture synthesis, compression, and maximum likelihood parameter estimation, currently implemented on Thinking Machines CM-2 and CM-5. The use of fine-grained, data parallel processing techniques yields real-time algorithms for texture synthesis and compression that are substantially faster than the previously known sequential implementations. Although current implementations are on Connection Machines, the methodology presented enables machine-independent scalable algorithms for a number of problems in image processing and analysis.     

Bounds on maximum likelihood ratio-part I: application to antenna array detection-estimation with perfect wavefront coherence

The multiple hypothesis testing problem of the detection-estimation of an unknown number of independent Gaussian point sources is adequately addressed by likelihood ratio (LR) maximization over the set of admissible covariance matrix models. We introduce nonasymptotic lower and upper bounds for the maximum LR. Since LR optimization is generally a nonconvex multiextremal problem, any practical solution could now be tested against these bounds, enabling a high probability of recognizing nonoptimal solutions. We demonstrate that in many applications, the lower bound is quite tight, with approximate maximum likelihood (ML) techniques often unable to approach this bound. The introduced lower bound analysis is shown to be very efficient in determining whether or not performance breakdown has occurred for subspace-based direction-of-arrival (DOA) estimation techniques. We also demonstrate that by proper LR maximization, we can extend the range of signal-to-noise ratio (SNR) values and/or number of data samples wherein accurate parameter estimates are produced. Yet, when the SNR and/or sample size falls below a certain limit for a given scenario, we show that ML estimation suffers from a discontinuity in the parameter estimates: a phenomenon that cannot be eliminated within the ML paradigm.     

Maximum likelihood estimation of signals in autoregressive noise

Time series modeling as the sum of a deterministic signal and an autoregressive (AR) process is studied. Maximum likelihood estimation of the signal amplitudes and AR parameters is seen to result in a nonlinear estimation problem. However, it is shown that for a given class of signals, the use of a parameter transformation can reduce the problem to a linear least squares one. For unknown signal parameters, in addition to the signal amplitudes, the maximization can be reduced to one over the additional signal parameters. The general class of signals for which such parameter transformations are applicable, thereby reducing estimator complexity drastically, is derived. This class includes sinusoids as well as polynomials and polynomial-times-exponential signals. The ideas are based on the theory of invariant subspaces for linear operators. The results form a powerful modeling tool in signal plus noise problems and therefore find application in a large variety of statistical signal processing problems. The authors briefly discuss some applications such as spectral analysis, broadband/transient detection using line array data, and fundamental frequency estimation for periodic signals     

Power system harmonic signal estimation and retrieval for activepower filter applications

This paper presents a new adaptive infinite impulse response (IIR) line enhancer (LE) comb filter configuration for the purpose of power system harmonic signal estimation and retrieval. The approximate maximum likelihood (AML) algorithm is employed for the parameter update. The proposed solution is characterised by modest computational burden, effective tracking capabilities and provides the retrieved harmonic components with little or no distortion. The retrieved power system harmonics may be obtained on an individual basis or as a composite signal. Practical test results are included which show the performance achieved by the proposed technique     

Characterization and estimation of two-dimensional ARMA models

A class of finite-order two-dimensional autoregressive moving average (ARMA) is introduced that can represent any process with rational spectral density. In this model the driving noise is correlated and need not be Gaussian. Currently known classes of ARMA models or AR models are shown to be subsets of the above class. The three definitions of Markov property are discussed, and the class of ARMA models are precisely stated which have the noncausal and semicausal Markov property without imposing any specific boundary conditions. Next two approaches are considered to estimate the parameters of a model to fit a given image. The first method uses only the empirical correlations and involves the solution of linear equations. The second method is the likelihood approach. Since the exact likelihood function is difficult to compute, we resort to approximations suggested by the toroidal models. Numerical experiments compare the quality of the two estimation schemes. Finally the problem of synthesizing a texture obeying an ARMA model is considered.     

利用期望最大化算法的EMCCD噪声分布模型的参数估计

讨论了电子倍增CCD（EMCCD）图像的噪声来源及其统计特性,建立了混合泊松-高斯噪声分布模型。针对混合泊松-高斯噪声分布模型的极大似然函数难以求解的问题,对噪声模型进行了适当的初始化设置,利用期望最大化算法对噪声模型进行参数估计,有效实现了噪声参数的极大似然估计。Monte Carlo仿真结果及实验结果表明,期望最大化算法估计性能较好,对混合泊松-高斯分布有较好的拟合效果,能得到较高精度的参数估计值。     

Estimation of wavefield power distribution in the remotely sensed environment: Bayesian maximum entropy approach

The problem of estimating, from one random realization of the remotely sensed signal, the spatial spectrum pattern (SSP) of the wavefield sources distributed in the environment is cast in the framework of Bayesian estimation theory. The kernel spectral estimation method that is familiar, for the classical SSP estimation problem, with the Fourier transform operator and white noise in the observations is extended to incorporate spatial correlation in the data, the system-oriented model of the signal formation operator, and the maximum entropy (ME) statistical a priori information about the SSP. To derive the estimate of the SSP, we applied the Bayesian strategy for maximization of the a posteriori probability density function of the randomized ME model of the SSP. The estimator was obtained as a nonlinear adaptive algorithm that also permits a concise robust implementation. The optimal algorithm implies formation of the second-order sufficient statistics of the data and their smoothing by applying the window operator. The new formalism of the sufficient statistics and windows, explaining their adjustment to the metrics in a solution space, a priori nonparametric model and assumed correlation properties of the desired SSP, is developed. Simulation results are included to illustrate the overall performance of the proposed method in an example of application to radar image formation.