Combined instrumental variable and subspace fitting approach toparameter estimation of noisy input-output systems

The paper considers the problem of estimating the parameters of linear discrete-time systems from noise-corrupted input-output measurements, under fairly general conditions: the output and input noises may be auto-correlated and they may be cross-correlated as well. By using the instrumental-variable (IV) principle a covariance matrix is obtained, the singular vectors of which bear complete information on the parameters of the system under study. A weighted subspace fitting (WSF) procedure is then employed on the sample singular vectors to derive estimates of the parameters of the system. The combined IV-WSF method proposed in the present paper is noniterative and simple to use. Its large-sample statistical performance is analyzed in detail and the theoretical results so obtained are used to predict the behavior of the method in samples with practical lengths. Several numerical examples are included to show the agreement between the theoretically predicted and the empirically observed performances     

MIMO system identification: state-space and subspace approximationsversus transfer function and instrumental variables

The identification of multi-input multi-output (MIMO) linear systems has previously received a new impetus with the introduction of the state-space (SS) approach based on subspace approximations. This approach has immediately gained popularity, owing to the fact that it avoids the use of canonical forms, requires the determination of only one structural parameter, and has been empirically shown to yield MIMO models with good accuracy in many cases, However, the SS approach suffers from several drawbacks: there is no well-established rule tied to this approach for determining the structural parameter, and, perhaps more important the SS parameter estimates depend on the data in a rather complicated way, which renders almost futile any attempt to analyze and optimize the performance of the estimator. In this paper, we consider a transfer function (TF) approach based on instrumental variables (IV), as an alternative to the SS approach. We use the simplest canonical TF parameterization in which the denominator is equal to a scalar polynomial times the identity matrix. The analysis and optimization of the statistical accuracy of the TF approach is straightforward. Additionally, a simple test tailored to this approach is devised for estimating the single structural parameter needed. A simulation study, in which we compare the performances of the SS and the TF approaches, shows that the latter can provide more accurate models than the former at a lower computational cost     

Bounds from noisy linear measurements

An estimation problem in which a finite number of linear measurements of an unknown function is available, and in which the only prior information available concerning the unknown function consists of inequality constraints on its magnitude, is ill-posed in that insufficient information is available from which point estimates of the unknown function can be made with any reliability, even with exact measurements. An alternative to point estimation involves the computation of bounds on linear functionals of the unknown function in terms of the measurements. A generalization is described of the bounding technique to problems in which the measurements are inexact. The bounds are defined in terms of a primal optimization problem. A deterministic interpretation of the bounds is given, as well as a probabilistic one for the case of additive Gaussian measurement noise. An unconstrained dual optimization problem is derived that has an interesting data-adaptive filtering interpretation and provides an attractive basis for computation. Several aspects of the primal and dual optimization problems are investigated that have important implications for the reliable computation of the bounds.     

Sequential signal encoding from noisy measurements using quantizerswith dynamic bias control

Signal estimation from a sequential encoding in the form of quantized noisy measurements is considered. As an example context, this problem arises in a number of remote sensing applications, where a central site estimates an information-bearing signal from low-bandwidth digitized information received from remote sensors, and may or may not broadcast feedback information to the sensors. We demonstrate that the use of an appropriately designed and often easily implemented additive control input before signal quantization at the sensor can significantly enhance overall system performance. In particular, we develop efficient estimators in conjunction with optimized random, deterministic, and feedback-based control inputs, resulting in a hierarchy of systems that trade performance for complexity     

Estimation of autoregressive signals from noisy measurements

The paper presents an asymptotically unbiased estimator of autoregressive parameters from noisy observations. The key ingredient in the author's method is that a new and simple scheme for estimation of the variance of the white measurement noise is developed. This estimated variance is then used in conjunction with the known technique for elimination of the least-squares estimation bias when the noise statistics are known a priori. The properties of the method are illustrated by means of some simulated examples     

Modeling intrapersonal deformation subspace using GMM for palmprint identification

In this paper, an efficient model of palmprint identification is presented based on subspace density estimation using Gaussian Mixture Model （GMM）. While a few training samples are available for each person, we use intrapersonal palmprint deformations to train the global GMM instead of modeling GMMs for every class. To reduce the dimension of such variations while preserving density function of sample space, Principle Component Analysis （PCA） is used to find the principle differences and form the Intrapersonal Deformation Subspace （IDS）. After training GMM using Expectation Maximization （EM） algorithm in IDS, a maximum likelihood strategy is carried out to identify a person. Experimental results demonstrate the advantage of our method compared with traditional PCA method and single Gaussian strategy.     

Analysis of subspace fitting and ML techniques for parameterestimation from sensor array data

It is shown that the multidimensional signal subspace method, termed weighted subspace fitting (WSF), is asymptotically efficient. This results in a novel, compact matrix expression for the Cramer-Rao bound (CRB) on the estimation error variance. The asymptotic analysis of the maximum likelihood (ML) and WSF methods is extended to deterministic emitter signals. The asymptotic properties of the estimates for this case are shown to be identical to the Gaussian emitter signal case, i.e. independent of the actual signal waveforms. Conclusions concerning the modeling aspect of the sensor array problem are drawn     

Fast identification of autoregressive signals from noisy observations

The purpose of this brief is to derive, from the previously developed least-squares (LS) based method, a faster convergent approach to identification of noisy autoregressive (AR) stochastic signals. The feature of the new algorithm is that in its bias correction procedure, it makes use of more autocovariance samples to estimate the variance of the additive corrupting noise which determines the noise-induced bias in the LS estimates of the AR parameters. Since more accurate estimates of this corrupting noise variance can be attained at earlier stages of the iterative process, the proposed algorithm can achieve a faster rate of convergence. Simulation results are included that illustrate the good performances of the proposed algorithm.     

On Di's and subspace fitting approaches to direction estimation

The article relates the method for direction estimation in possibly coherent scenarios, which was proposed by Di (1985), to the class of subspace fitting (SSF) methods recently introduced in the literature. It also makes some general comments on the SSF class     

Multichannel blind identification: from subspace to maximumlikelihood methods

A review of blind channel estimation algorithms is presented. From the (second-order) moment-based methods to the maximum likelihood approaches, under both statistical and deterministic signal models. We outline basic ideas behind several new developments, the assumptions and identifiability conditions required by these approaches, and the algorithm characteristics and their performance. This review serves as an introductory reference for this currently active research area     

Robust ISAR imaging based on compressive sensing from noisy measurements

the compressive sensing (CS) based ISAR imaging has exhibited high-resolution imaging quality when faced with limited spatial aperture. However, its performance is significantly dependent on the number of pulses and the noise level. In this paper, from the perspective of promoted sparsity constraint, a novel reconstruction model deducted from Meridian prior (MCS) is proposed. The detailed comparison of the proposed MCS model with the Laplace-prior-based CS model (LCS) is conducted. The Lorentz curve analysis testified the enhanced sparsity of the MCS model. Different from the algorithm for LCS model, in our solution procedure, the variance parameter is iteratively updated until the algorithm converges. Simulations and the ground truth data experiments of ISAR show that, with the decrease of the number of pulses and signal-to-noise ratio, the proposed model exhibits better performance in terms of resolution and amplitude error than that of the LCS model.     

Subspace-based parameter estimation of symmetric noncausalautoregressive signals from noisy measurements

Symmetric noncausal auto-regressive signals (SNARS) arise in several, mostly spatial, signal processing applications. We introduce a subspace fitting approach for parameter estimation of SNARS from noise-corrupted measurements. We show that the subspaces associated with a Hankel matrix built from the data covariances contain enough information to determine the signal parameters in a consistent manner. Based on this result, we propose a multiple signal classification (MUSIC)-like methodology for parameter estimation of SNARS. Compared with the methods previously proposed for SNARS parameter estimation, our SNARS-MUSIC approach is expected to possess a better tradeoff between computational and statistical performances     

System parameter estimation with input/output noisy data andmissing measurements

An investigation is undertaken to examine the parameter estimation problem of linear systems when some of the measurements are unavailable (i.e., missing data) and the probability of occurrence of missing data is unknown a priori. The system input and output data are also assumed to be corrupted by measurement noise, and the knowledge of the noise distribution is unknown. Under the unknown noise distribution and missing measurements, a consistent parameter estimation algorithm [which is based on an l_p norm iterative estimation algorithm-iteratively reweighted least squares (IRLS)] is proposed to estimate the system parameters. We show that if the probability of missing measurement is less than one half, the parameter estimates via the proposed estimation algorithm will converge to the true parameters as the number of data tends to infinity. Finally, several simulation results are presented to illustrate the performance of the proposed l _p norm iterative estimation algorithm. Simulation results indicate that under input/output missing data and noise environment, the proposed parameter estimation algorithm is an efficient approach toward the system parameter estimation problem     

Autoregressive model fitting with noisy data by Akaike's information criterion (Corresp.)

Davisson [131, [141 has considered the problem of determining the "order" of the signal from noisy data. Although interesting theoretically, his result is difficult to use in practice. In this correspondence, we exploit one well-known fact concerning autoregressive (AR) signals plus white noise, and using Akaike's information criterion [15], [17], we have developed one efficient procedure for determining the order of the AR signal from noisy data. The procedure is illustrated numerically using both artificially generated and real data. The connection between the preceding problem and the classical statistical problem of factor analysis is discussed.     

Modeling wireless fading channels via stochastic differential equations: identification and estimation based on noisy measurements

This paper is concerned with modeling and identification of wireless channels using noisy measurements. The models employed are governed by stochastic differential equations (SDEs) in state space form, while the identification method is based on the expectation-maximization (EM) algorithm and Kalman filtering. The algorithm is tested against real channel measurements. The results presented include state space models for the channels, estimates of inphase and quadrature components, and estimates of the corresponding Doppler power spectral densities (DPSDs), from sample noisy measurements. Based on the available measurements, it is concluded that state space models of order two are sufficient for wireless flat fading channel characterization.     

Cardiac artifact subspace identification and elimination in cognitive MEG data using time-delayed decorrelation

To reduce physiological artifacts in magnetoencephalographic (MEG) and electroencephalographic recordings, a number of methods have been applied in the past such as principal component analysis, signal-space projection, regression using secondary information, and independent component analysis. This method has become popular as it does not have constraints such as orthogonality between artifact and signal or the need for a priori information. Applying the time-delayed decorrelation algorithm to raw data from a visual stimulation MEG experiment, we show that several of the independent components can be attributed to the cardiac artifact. Calculating an average cardiac activity shows that physiologically different excitation states of the heart produce similar field distributions in the MEG sensor system. This is equivalent to differing spectral properties of cardiac field distributions in the raw data. As a consequence, the algorithm combines, e.g., the R peak and the T wave of the cardiac cycle into a single component and the one-to-one assignment of each independent component with a physiological source is not justified in this case. To improve the signal quality of visually evoked fields, the multidimensional cardiac artifact subspace is suppressed from the data. To assess the preservation of the evoked signal after artifact suppression, a geometrical and a temporal measure are introduced. The suppression of cardiac and alpha wave artifacts allows, in our experimental setting, the reduction of the number of epochs to one half while preserving the visually evoked signal.     

Improving the convergence of vector fitting for equivalent circuit extraction from noisy frequency responses

The vector fitting (VF) algorithm has become a common tool in electromagnetic compatibility and signal integrity studies. This algorithm allows the derivation of a rational approximation to the transfer matrix of a given linear structure starting from measured or simulated frequency responses. This paper addresses the convergence properties of a VF when the frequency samples are affected by noise. We show that small amounts of noise can seriously impair or destroy convergence. This is due to the presence of spurious poles that appear during the iterations. To overcome this problem we suggest a simple modification of the basic VF algorithm, based on the identification and removal of the spurious poles. Also, an incremental pole addition and relocation process is proposed in order to provide automatic order estimation even in the presence of significant noise. We denote the resulting algorithm as vector fitting with adding and skimming (VF-AS). A thorough validation of the VF-AS algorithm is presented using a Monte Carlo analysis on synthetic noisy frequency responses. The results show excellent convergence and significant improvements with respect to the basic VF iteration scheme. Finally, we apply the new VF-AS algorithm to measured scattering responses of interconnect structures and networks typical of high-speed digital systems.     

Array processing in correlated noise fields based on instrumentalvariables and subspace fitting

Accurate signal parameter estimation from sensor array data is a problem which has received much attention in the last decade. A number of parametric estimation techniques have been proposed in the literature. In general, these methods require knowledge of the sensor-to-sensor correlation of the noise, which constitutes a significant drawback. This difficulty can be overcome only by introducing alternative assumptions that enable separating the signals from the noise. In some applications, the raw sensor outputs can be preprocessed so that the emitter signals are temporally correlated with correlation length longer than that of the noise. An instrumental variable (IV) approach can then be used for estimating the signal parameters without knowledge of the spatial color of the noise. A computationally simple IV approach has recently been proposed by the authors. Herein, a refined technique that can give significantly better performance is derived. A statistical analysis of the parameter estimates is performed, enabling optimal selection of certain user-specified quantities. A lower bound on the attainable error variance is also presented. The proposed optimal IV method is shown to attain the bound if the signals have a quasideterministic character     

Stochastic analysis of gradient adaptive identification ofnonlinear systems with memory for Gaussian data and noisy input andoutput measurements

This paper investigates the statistical behavior of two gradient search adaptive algorithms for identifying an unknown nonlinear system comprised of a discrete-time linear system H followed by a zero-memory nonlinearity g(·). The input and output of the unknown system are corrupted by additive independent noises. Gaussian models are used for all inputs. Two competing adaptation schemes are analyzed. The first is a sequential adaptation scheme where the LMS algorithm is first used to estimate the linear portion of the unknown system. The LMS algorithm is able to identify the linear portion of the unknown system to within a scale factor. The weights are then frozen at the end of the first adaptation phase. Recursions are derived for the mean and fluctuation behavior of the LMS algorithm, which are in excellent agreement with Monte Carlo simulations. When the nonlinearity is modeled by a scaled error function, the second part of the sequential gradient identification scheme is shown to correctly learn the scale factor and the error function scale factor. Mean recursions for the scale factors show good agreement with Monte Carlo simulations. For slow learning, the stationary points of the gradient algorithm closely agree with the stationary points of the theoretical recursions. The second adaptive scheme simultaneously learns both the linear and nonlinear portions of the unknown channel. The mean recursions for the linear and nonlinear portions show good agreement with Monte Carlo simulations for slow learning. The stationary points of the gradient algorithm also agree with the stationary points of the theoretical recursions     

More on autoregressive model fitting with noisy data by Akaike's information criterion (Corresp.)

Tong has proposed an objective method, based on Akaike's information criterion (AIC), for the determination of the order of an autoregressive (AR) model with noisy data; some extensions of this proposed method are discussed.