On computing output-error estimates

In this note, we consider a system with rational transfer function from input to output where this transfer function haspparameters. It is also supposed that the input has a spectrum consisting ofpdistinct points (equivalently, we suppose that the design index =p/2). As proved in [2], this implies that the output-error estimator has the same efficiency as the prediction-error estimator. We present a very convenient algorithm for computing the output-error estimate of the transfer function from input to output.     

On output-error methods for system identification

In this paper are derived consistency and asymptotic normality results for the output-error method of system identification. The output-error estimator has the advantage over the prediction-error estimator of being more easily computable. However, it is shown that the output-error estimator can never be more efficient than the prediction-error estimator. The main result of the paper provides necessary and sufficient conditions for the output-error estimator and the prediction-error estimator to have the same efficiency, irrespective of the spectral density of the noise process.     

Measurement designs to improve the computational efficiency of linear system state estimators

A design procedure for the measurement process is proposed to decrease the computational burden of state estimation in linear discrete-time systems. System order reduction is involved, which is achieved by a derived transformation, yielding an alternative system and measurement process representation that includes time delays in both the system and the measurement process. Without loss of filter optimality, measurement designs yielding reductions in the computational requirements of the Kalman filter applied to the reduced order system are derived. A computational comparison is included to show that the proposed method, as opposed to the direct application of the Kalman filter, can yield large reductions in the filter computation time.     

On the efficient computation of robust regression estimators

The problem of providing efficient and reliable robust regression algorithms is considered. The impact of global optimization methods, such as stopping conditions and clustering techniques, in the calculation of robust regression estimators is investigated. The use of stopping conditions permits us to devise new algorithms that perform as well as the existing algorithms in less time and with adaptive algorithm parameters. Clustering global optimization is shown to be a general framework encompassing many of the existing algorithms.     

Realisation and estimation of piecewise-linear output-error models

Piecewise-linear systems in input/output form can have different switching schedules. In this article, two categories, instant and delayed switching, are analysed. Even though a general piecewise-linear state-space model cannot be converted into input/output form, it is shown that it is possible to find state-space models representing instant and delayed switching. In addition, a prediction-error minimisation (PEM) method for piecewise-linear output-error predictors is derived and it is concluded that the instant-switching model candidate is not necessarily the most suitable for the parameter estimation procedure.     

On the localized estimators and generalized Akaike's criteria

The problem of nonstationary system modeling is considered and the local modeling approach is proposed for its solution. Initially, the concept of localized maximum likelihood estimators is introduced and applied to approximation of time-varying stochastic systems. Two types of such estimators, the first based on the concept of weighting and the second based on the concept of data windowing, are proposed and discussed in some detail in the case of autoregressive systems, Next, the problem of the proper choice of the model structure is considered. It is shown that the criterion for model order selection proposed by Akaike for the case of maximum likelihood estimation (information criterion) can be extended to the case of localized estimators.     

Variance of distribution and efficiency of estimators of the observation probability of transition vectors

The study considers a method of identification of sequences using sample observation probabilities of transition vectors. The variance and properties of point estimators oriented towards applications in inspection systems and systems of statistical tests of digital circuits are determined. The efficiency of empirical probabilities for sampling Bernoulli and multidimensional random events is investigated on the basis of the Rao-Cramer inequality.     

On the asymptotic distribution of the least-squares estimators in unidentifiable models

In order to analyze the stochastic property of multilayered perceptrons or other learning machines, we deal with simpler models and derive the asymptotic distribution of the least-squares estimators of their parameters. In the case where a model is unidentified, we show different results from traditional linear models: the well-known property of asymptotic normality never holds for the estimates of redundant parameters.     

On MAP and MMSE estimators for the co-sparse analysis model

The sparse synthesis model for signals has become very popular in the last decade, leading to improved performance in many signal processing applications. This model assumes that a signal may be described as a linear combination of few columns (atoms) of a given synthesis matrix (dictionary). The Co-Sparse Analysis model is a recently introduced counterpart, whereby signals are assumed to be orthogonal to many rows of a given analysis dictionary. These rows are called the co-support.The Analysis model has already led to a series of contributions that address the pursuit problem: identifying the co-support of a corrupted signal in order to restore it. While all the existing work adopts a deterministic point of view towards the design of such pursuit algorithms, this paper introduces a Bayesian estimation point of view, starting with a random generative model for the co-sparse analysis signals. This is followed by a derivation of Oracle, Minimum-Mean-Squared-Error (MMSE), and Maximum-A-posteriori-Probability (MAP) based estimators. We present a comparison between the deterministic formulations and these estimators, drawing some connections between the two. We develop practical approximations to the MAP and MMSE estimators, and demonstrate the proposed reconstruction algorithms in several synthetic and real image experiments, showing their potential and applicability.     

On the extraction of dead-time controllers and estimators from delay-free parametrizations

An approach to control design for dead-time (DT) systems is proposed. The underlying idea is to treat the DT element not as a part of the generalized plant, but rather as a (causality) constraint imposed upon the controller (estimator). This enables one to use well-understood parametrizations of all delay free controllers in the DT design. In particular, DT controllers can be extracted from such delay-free parametrizations. In the paper, the extraction procedures are developed in both H/sup 2/ and H/sup /spl infin// settings. It is shown that the proposed approach yields simple solution procedures and new transparent and intuitively appealing structures for the resulting controllers and estimators.     

On the design of LMS-based channel estimators using the Doppler spread parameter

An analysis of the optimization of the LMS (Least Mean Square) algorithm for the estimation of time-varying and frequency-selective communication channels is here presented. In contrast to previous works on this subject, in which the step-size optimization is rooted on the assumption of specific channel models, this analysis is much more generic in respect to the modelling of both the Doppler and delay spreadings. Besides, it addresses not only the steady-state performance of channel estimators, but also their transient behaviors. Several useful approximate expressions for designing LMS-based channel estimators are herein derived and validated by comparisons with simulation and with a previous work on this kind of analysis. These expressions are remarkably suitable for practical application, since they depend on a few parameters of the communication system. With regard to the channel variability, the knowledge of the Doppler spread parameter is shown to be enough to optimize the performance of a LMS channel estimator using the analysis here presented.     

Auxiliary model-based least-squares identification methods for Hammerstein output-error systems

The difficulty in identification of a Hammerstein (a linear dynamical block following a memoryless nonlinear block) nonlinear output-error model is that the information vector in the identification model contains unknown variables—the noise-free (true) outputs of the system. In this paper, an auxiliary model-based least-squares identification algorithm is developed. The basic idea is to replace the unknown variables by the output of an auxiliary model. Convergence analysis of the algorithm indicates that the parameter estimation error consistently converges to zero under a generalized persistent excitation condition. The simulation results show the effectiveness of the proposed algorithms.     

On the efficiency of parallel backtracking

Analytical models and experimental results concerning the average case behavior of parallel backtracking are presented. Two types of backtrack search algorithms are considered: simple backtracking, which does not use heuristics to order and prune search, and heuristic backtracking, which does. Analytical models are used to compare the average number of nodes visited in sequential and parallel search for each case. For simple backtracking, it is shown that the average speedup obtained is linear when the distribution of solutions is uniform and superlinear when the distribution of solutions is nonuniform. For heuristic backtracking, the average speedup obtained is at least linear, and the speedup obtained on a subset of instances is superlinear. Experimental results for many synthetic and practical problems run on various parallel machines that validate the theoretical analysis are presented     

On risk concentration for convex combinations of linear estimators

We consider the estimation problem for an unknown vector β ∈ Rp in a linear model Y = Xβ + σξ, where ξ ∈ Rⁿ is a standard discrete white Gaussian noise and X is a known n × p matrix with n ≥ p. It is assumed that p is large and X is an ill-conditioned matrix. To estimate β in this situation, we use a family of spectral regularizations of the maximum likelihood method β^α(Y) = H ^α(X ^T X) β ^?(Y), α ∈ R⁺, where β ^?(Y) is the maximum likelihood estimate for β and {H ^α(·): R⁺ → [0, 1], α ∈ R⁺} is a given ordered family of functions indexed by a regularization parameter α. The final estimate for β is constructed as a convex combination (in α) of the estimates β^α(Y) with weights chosen based on the observations Y. We present inequalities for large deviations of the norm of the prediction error of this method.     

Coupled stochastic gradient identification algorithms for multivariate output-error systems using the auxiliary model

This paper considers the gradient based identification problem of a multivariate output-error system. By using the auxiliary model identification idea and the coupling identification concept, an auxiliary model based stochastic gradient (AM-SG) algorithm and a coupled AM-SG algorithm are presented. The results indicate that the parameter estimation errors converge to zero under the persistent excitation conditions. The simulation examples confirm the theoretical results.     

On the well-posedness of multivariate spectrum approximation and convergence of high-resolution spectral estimators

In this paper, we establish the well-posedness of the generalized moment problems recently studied by Byrnes–Georgiou–Lindquist and coworkers, and by Ferrante–Pavon–Ramponi. We then apply these continuity results to prove the almost sure convergence of a sequence of high-resolution spectral estimators indexed by the sample size.     

On the efficiency of two-bit link chain-code

This note shows that, under reasonable conditions of isotropy, two-bit chain-code is slightly more efficient than three-bit chain-code in terms of total number of bits per unit length.     

On the sampling distribution of resubstitution and leave-one-out error estimators for linear classifiers

Error estimation is a problem of high current interest in many areas of application. This paper concerns the classical problem of determining the performance of error estimators in small-sample settings under a Gaussianity parametric assumption. We provide here for the first time the exact sampling distribution of the resubstitution and leave-one-out error estimators for linear discriminant analysis (LDA) in the univariate case, which is valid for any sample size and combination of parameters (including unequal variances and sample sizes for each class). In the multivariate case, we provide a quasi-binomial approximation to the distribution of both the resubstitution and leave-one-out error estimators for LDA, under a common but otherwise arbitrary class covariance matrix, which is assumed to be known in the design of the LDA discriminant. We provide numerical examples, using both synthetic and real data, that indicate that these approximations are accurate, provided that LDA classification error is not too large.     

On the efficiency of secure XML broadcasting

Secure broadcasting of web documents is becoming a crucial requirement for many web-based applications. Under the broadcast document dissemination strategy, a web document source periodically broadcasts (portions of) its documents to a potentially large community of users, without the need for explicit requests. By secure broadcasting, we mean that the delivery of information to users must obey the access control policies of the document source. Traditional access control mechanisms that have been adapted for XML documents, however, do not address the performance issues inherent in access control. In this paper, a labeling scheme is proposed to support rapid reconstruction of XML documents in the context of a well-known method, called XML pool encryption. The proposed labeling scheme supports the speedy inference of structure information in all portions of the document. The binary representation of the proposed labeling scheme is also investigated. In the experimental results, the proposed labeling scheme is efficient in searching for the location of decrypted information.     

On improving the efficiency of tensor voting

This paper proposes two alternative formulations to reduce the high computational complexity of tensor voting, a robust perceptual grouping technique used to extract salient information from noisy data. The first scheme consists of numerical approximations of the votes, which have been derived from an in-depth analysis of the plate and ball voting processes. The second scheme simplifies the formulation while keeping the same perceptual meaning of the original tensor voting: The stick tensor voting and the stick component of the plate tensor voting must reinforce surfaceness, the plate components of both the plate and ball tensor voting must boost curveness, whereas junctionness must be strengthened by the ball component of the ball tensor voting. Two new parameters have been proposed for the second formulation in order to control the potentially conflictive influence of the stick component of the plate vote and the ball component of the ball vote. Results show that the proposed formulations can be used in applications where efficiency is an issue since they have a complexity of order O(1). Moreover, the second proposed formulation has been shown to be more appropriate than the original tensor voting for estimating saliencies by appropriately setting the two new parameters.