Simplified neural network for generalized least absolute deviation

This paper proposes a simplified neural network for generalized least absolute deviation by transforming its optimization conditions into a system of double projection equations. The proposed network is proved to be stable in the sense of Lyapunov and converges to an exact optimization solution of the original problem for any starting point. Compared with the existing neural networks for generalized least absolute deviation, the new model has the least neurons and low complexity and is suitable to parallel implementation. The validity and transient behavior of the proposed neural network are demonstrated by numerical examples.

     

基于递推近似最小一乘的多传感器系统偏差稳健估计算法

对于多传感器多目标跟踪问题,系统偏差对航迹融合精度有较大影响,因此在信息融合系统中,首先要对各传感器的系统偏差进行估计,而在含错误关联和观测野值的复杂环境下,传统系统偏差估计方法的性能会严重退化.对此,提出一种具有递推形式的近似最小一乘稳健估计算法,以减少异常噪声对偏差估计的不利影响.使用平方根平滑逼近函数替代最小一乘法的目标函数,基于牛顿方向及其秩1修正推导出该方法的递推求解框架.基于条件数分析,证明所提出算法的数值稳定性好于Huber方法.通过两个仿真算例,将所提出算法与已有其他算法进行对比验证.仿真结果表明,在含错误关联和观测野值的条件下,所提出算法可以改善偏差估计精度,并且明显好于已有的其他算法.     

New identification method for Hammerstein models based on approximate least absolute deviation

Disorder and peak noises or large disturbances can deteriorate the identification effects of Hammerstein non-linear models when using the least-square (LS) method. The least absolute deviation technique can be used to resolve this problem; however, its absolute value cannot meet the need of differentiability required by most algorithms. To improve robustness and resolve the non-differentiable problem, an approximate least absolute deviation (ALAD) objective function is established by introducing a deterministic function that exhibits the characteristics of absolute value under certain situations. A new identification method for Hammerstein models based on ALAD is thus developed in this paper. The basic idea of this method is to apply the stochastic approximation theory in the process of deriving the recursive equations. After identifying the parameter matrix of the Hammerstein model via the new algorithm, the product terms in the matrix are separated by calculating the average values. Finally, algorithm convergence is proven by applying the ordinary differential equation method. The proposed algorithm has a better robustness as compared to other LS methods, particularly when abnormal points exist in the measured data. Furthermore, the proposed algorithm is easier to apply and converges faster. The simulation results demonstrate the efficacy of the proposed algorithm.     

Robust object tracking using least absolute deviation

Recently, sparse representation has been applied to object tracking, where each candidate target is approximately represented as a sparse linear combination of target templates. In this paper, we present a new tracking algorithm, which is faster and more robust than other tracking algorithms, based on sparse representation. First, with an analysis of many typical tracking examples with various degrees of corruption, we model the corruption as a Laplacian distribution. Then, a LAD–Lasso optimisation model is proposed based on Bayesian Maximum A Posteriori (MAP) estimation theory. Compared with L1 Tracker and APG-L1 Tracker, the number of optimisation variables is reduced greatly; it is equal to the number of target templates, regardless of the dimensions of the feature. Finally, we use the Alternating Direction Method of Multipliers (ADMM) to solve the proposed optimisation problem. Experiments on some challenging sequences demonstrate that our proposed method performs better than the state-of-the-art methods in terms of accuracy and robustness.     

Maximum likelihood parameter estimation algorithm for controlled autoregressive autoregressive models

A maximum likelihood parameter estimation algorithm is derived for controlled autoregressive autoregressive (CARAR) models based on the maximum likelihood principle. In this derivation, we use an estimated noise transfer function to filter the input–output data. The simulation results show that the proposed estimation algorithm can effectively estimate the parameters of such class of CARAR systems and give more accurate parameter estimates than the recursive generalized least-squares algorithm.     

A continuous-time framework for least squares parameter estimation

This paper proposes a continuous-time framework for the least-squares parameter estimation method through evolution equations. Nonlinear systems in the standard state space representation that are linear in the unknown, constant parameters are investigated. Two estimators are studied. The first one consists of a linear evolution equation while the second one consists of an impulsive linear evolution equation. The paper discusses some theoretical aspects related to the proposed estimators: uniqueness of a solution and an attractive equilibrium point which solves for the unknown parameters. A deterministic framework for the estimation under noisy measurements is proposed using a Sobolev space with negative index to model the noise. The noise can be of large magnitude. Concrete signals issued from an electronic device are used to discuss numerical aspects.     

Constrained least absolute deviation neural networks.

It is well known that least absolute deviation (LAD) criterion or L(1)-norm used for estimation of parameters is characterized by robustness, i.e., the estimated parameters are totally resistant (insensitive) to large changes in the sampled data. This is an extremely useful feature, especially, when the sampled data are known to be contaminated by occasionally occurring outliers or by spiky noise. In our previous works, we have proposed the least absolute deviation neural network (LADNN) to solve unconstrained LAD problems. The theoretical proofs and numerical simulations have shown that the LADNN is Lyapunov-stable and it can globally converge to the exact solution to a given unconstrained LAD problem. We have also demonstrated its excellent application value in time-delay estimation. More generally, a practical LAD application problem may contain some linear constraints, such as a set of equalities and/or inequalities, which is called constrained LAD problem, whereas the unconstrained LAD can be considered as a special form of the constrained LAD. In this paper, we present a new neural network called constrained least absolute deviation neural network (CLADNN) to solve general constrained LAD problems. Theoretical proofs and numerical simulations demonstrate that the proposed CLADNN is Lyapunov stable and globally converges to the exact solution to a given constrained LAD problem, independent of initial values. The numerical simulations have also illustrated that the proposed CLADNN can be used to robustly estimate parameters for nonlinear curve fitting, which is extensively used in signal and image processing.     

Fuzzy regression using least absolute deviation estimators

In fuzzy regression, that was first proposed by Tanaka et al. (Eur J Oper Res 40:389–396, 1989; Int Cong Appl Syst Cybern 4:2933–2938, 1980; IEEE Trans SystMan Cybern 12:903–907, 1982), there is a tendency that the greater the values of independent variables, the wider the width of the estimated dependent variables. This causes a decrease in the accuracy of the fuzzy regression model constructed by the least squares method. This paper suggests the least absolute deviation estimators to construct the fuzzy regression model, and investigates the performance of the fuzzy regression models with respect to a certain errormeasure. Simulation studies and examples show that the proposed model produces less error than the fuzzy regression model studied by many authors that use the least squares method when the data contains fuzzy outliers.     

A generalized pseudoinverse algorithm for unbiased parameter estimation

Unbiased estimates of the parameters of a discrete-time system from noisy measurements of input-output data can be obtained by using a generalized pseudo-inverse algorithm. Because of its recursive nature, the algorithm can be used for on-line identification. Results of simulation are presented comparing this algorithm with earlier methods.     

基于近似偏最小一乘的闭环系统辨识新方法

本文基于近似最小一乘准则和主成分分析,针对反馈通道模型阶次低于前向通道模型阶次且反馈通道不存在噪声的闭环系统,进行了近似偏最小一乘递推辨识算法的推导.为解决最小一乘准则函数不可微的问题,本文算法用确定性可导函数近似代替残差绝对值.近似偏最小一乘辨识算法可以克服基于最小二乘准则的辨识算法在受到满足(SαS)分布的尖峰噪声干扰时残差平方项过大的缺点,具有目标函数可导,计算简单的优点.同时,通过主成分分析去除数据向量各元素之间的线性相关,可以得出模型参数的唯一解.仿真实验表明,本文算法可以对反馈通道模型阶次低于前向通道模型阶次的闭环系统进行直接辨识,抑制了尖峰噪声对辨识结果的影响,具有优良的稳健性,可以更好地应用于闭环系统辨识.     

近似最小一乘意义下的鲁棒卡尔曼滤波器

当存在高污染率的野值观测时,现有的鲁棒卡尔曼滤波器的数值稳定性和抗差能力可能会严重退化.为此,基于近似最小一乘估计和修正的高斯牛顿方法提出一种新的鲁棒卡尔曼滤波器,以减小含野量测对滤波器的不利影响.通过条件数分析和影响函数分析,从理论上证明所提出方法的数值稳定性和抗差能力均好于基于Huber估计的卡尔曼滤波器.通过仿真实验对理论分析结果进行验证.仿真结果表明,在只有少量野值观测的情况下,所提出的滤波器与Huber卡尔曼滤波器的估计性能大致相当;而在含有高污染率的野值观测时,所提出的滤波器的估计性能明显好于Huber卡尔曼滤波器.在仿真实验中还对几种滤波器的计算花费进行了比较,发现所提出滤波器的计算代价小于Huber卡尔曼滤波器的计算代价.     

Estimating the fundamental matrix based on least absolute deviation

The epipolar geometry is the intrinsic projective geometry between two views, and the algebraic representation of it is the fundamental matrix. Estimating the fundamental matrix requires solving an over-determined equation. Many classical approaches assume that the error values of the over-determined equation obey a Gaussian distribution. However, the performances of these approaches may decrease significantly when the noise is large and heterogeneous. This paper proposes a novel technique for estimating the fundamental matrix based on least absolute deviation (LAD), which is also known as the L₁ method. Then a linear iterative algorithm is presented. The experimental results on some indoor and outdoor scenes show that the proposed algorithm yields the accurate and robust estimates of the fundamental matrix when the noise is non-Gaussian.     

Minimum absolute deviation estimation in regression analysis

The use of ordinary least squares estimators (OLS) in regression analysis is widespread. The OLS estimates are, however, very sensitive to the presence of large disturbances. As an alternative to the OLS estimator, the minimum absolute deviation estimator (MAD) is studied. The purpose of this study is, first, to determine the effect of error distributions, with progressively heavier tails starting from the normal distribution and ending with the Cauchy distribution, on the performance of the MAD estimates and the OLS estimates. This provides a framework for when to choose the MAD estimator over the OLS estimator. Second, the effect of some of the other parameters in regression analysis, namely, the unknown parameter vector, the multicollinearity between the independent variables, and the size of the sample on the relative performance of the MAD and OLS estimators is investigated. Some guidelines regarding the choice of the MAD estimator in regression analysis are provided.     

Design of input signals for parameter estimation

The optimum energy-constrained and time-constrained input signal is obtained for estimating the parameters of a system. The output is corrupted by nonstationary, nonwhite additive observation noise, and the observation time is finite. The reproducing kernel Hilbert space formulation is used to obtain the parameter estimates and the error covariance matrix in terms of the input. The performance index, assumed to be a function of the error covariance matrix, is minimized by a variational procedure. A necessary condition for optimality is that the input satisfy a nonlinear Fredholm equation. An example estimates the gain of a single time constant system where the observation noise has an exponential autocorrelation function. For broadband noise, the optimum input is a portion of a sinusoid. For a noise bandwidth narrower than the system bandwidth, the optimum input switches sign as rapidly as possible, but near-optimum performance can be obtained with a relatively high frequency sinusoidal input.     

一种用于跳频信号参数估计的时频表示方法

在时延和频移两个方向上,对跳频信号的模糊函数进行了分析,提出了一种基于跳频信号模糊函数自项特征的时频表示方法。其核函数在信号的模糊域能够有效地滤除噪声和交叉项,并保留绝大部分的自项能量。仿真试验结果证实,与平滑伪维格纳分布相比较,该方法提高了信号项的时频聚集性,具有更好的参数估计性能。     

A revisit to block and recursive least squares for parameter estimation

In this paper, the classical least squares (LS) and recursive least squares (RLS) for parameter estimation have been re-examined in the light of the present day computing capabilities. It has been demonstrated that for linear time-invariant systems, the performance of blockwise least squares (BLS) is always superior to that of RLS. In the context of parameter estimation for dynamic systems, the current computational capability of personal computers are more than adequate for BLS. However, for time-varying systems with abrupt parameter changes, standard blockwise LS may no longer be suitable due to its inefficiency in discarding “old” data. To deal with this limitation, a novel sliding window blockwise least squares approach with automatically adjustable window length triggered by a change detection scheme is proposed. Two types of sliding windows, rectangular and exponential, have been investigated. The performance of the proposed algorithm has been illustrated by comparing with the standard RLS and an exponentially weighted RLS (EWRLS) using two examples. The simulation results have conclusively shown that: (1) BLS has better performance than RLS; (2) the proposed variable-length sliding window blockwise least squares (VLSWBLS) algorithm can outperform RLS with forgetting factors; (3) the scheme has both good tracking ability for abrupt parameter changes and can ensure the high accuracy of parameter estimate at the steady-state; and (4) the computational burden of VLSWBLS is completely manageable with the current computer technology. Even though the idea presented here is straightforward, it has significant implications to virtually all areas of application where RLS schemes are used.     

基于图像处理的跳频信号参数估计

针对复杂的电磁环境对跳频信号检测所产生的影响,提出了一种新的基于图像处理的跳频信号参数估计方法。首先将短时傅里叶变换(STFT)后的时频图处理为二维图像,然后根据电磁环境中噪声在时频图中的表现形式,对时频图进行滤波处理,最后根据对时频图的分析得出跳频信号参数估计。使用真实数据所进行的实验结果表明,所提出的方法能有效地滤除实际跳频信号中的雾态噪声、定频和突发信号,且不需要知道信号的跳速或驻留时间等前提条件,运算原理清晰,时间成本低且具有较高的准确度。目前该方法已应用于某数字接收机产品中,能很好地满足实际需求及测量精度要求。     

Identification algorithm based on the approximate least absolute deviation criteria

Considering the situation that the least-squares (LS) method for system identification has poor robustness and the least absolute deviation (LAD) algorithm is hard to construct, an approximate least absolute deviation (ALAD) algorithm is proposed in this paper. The objective function of ALAD is constructed by introducing a deterministic function to approximate the absolute value function. Based on the function, the recursive equations for parameter identification are derived using Gauss-Newton iterative algorithm without any simplification. This algorithm has advantages of simple calculation and easy implementation, and it has second order convergence speed. Compared with the LS method, the new algorithm has better robustness when disorder and peak noises exist in the measured data. Simulation results show the efficiency of the proposed method.     

全最小一乘准则下的LGA新算法

为克服传统提取数据集中线性结构的LGA对噪声数据比较敏感的缺陷,提出了两种基于稳健的全最小一乘准则下的LGA新算法。首先证明了全最小一乘准则下数据集最优划分的存在性,并据此给出一种有限步终止算法。其次为提高计算速度,根据k-means算法、全最小一乘准则和重抽样方法给出另一种快速收敛算法。通过与传统的LGA和基于Trimmed k-means思想的稳健LGA的比较,仿真结果表明提出的算法具有较好的稳健性,可以在离群数据较多的情形下,同时找出数据集合中的所有强线性结构。     

Multi-innovation stochastic gradient parameter estimation for input nonlinear controlled autoregressive models

This paper proposes a multi-innovation stochastic gradient (MISG) parameter estimation algorithm for an input nonlinear controlled autoregressive (IN-CAR) model, i.e., a Hammerstein nonlinear CAR system, by expanding the innovation length. The analysis and simulation results indicate that the proposed MISG algorithm can generate more accurate parameter estimates for IN-CAR systems compared with the stochastic gradient algorithm.