Extended state space recursive least squares

A new extended state space recursive least squares (ESSRLS) algorithm is proposed for state estimation of nonlinear systems. It is based on state space recursive least squares (SSRLS) approach and uses first order linearization of the system. It inherits the capability of obtaining state estimate without knowledge of process and measurement noise covariance matrices (Q and R respectively). The proposed approach is considered to provide new design option for scenarios where noise statistics and system dynamics vary. ESSRLS is initialized using delayed recursion method and a forgetting factor λ is employed to optimize the performance. The selection of λ can be problem specific as shown through experimental validations. However a value closer to and less than unity is generally recommended. Theoretical bases are validated by applying this algorithm to problems of tracking a non-conservative oscillator, a damped system with amplitude death and a signal modeled by mixture of Gaussian kernels. Simulation results show an MSE performance gain of 20 dB and 23 dB over extended Kalman filter (EKF) and unscented Kalman filter (UKF) while tracking van der Pol oscillator without knowledge about noise variances. The computational complexity of ESSRLS falls within that of EKF and UKF.     

Adaptive recursive algorithm with logarithmic transformation for nonlinear system identification in α-stable noise

Although the least mean pth power (LMP) and normalized LMP (NLMP) algorithms of adaptive Volterra filters outperform the conventional least mean square (LMS) algorithm in the presence of α-stable noise, they still exhibit slow convergence and high steady-state kernel error in nonlinear system identification. To overcome these limitations, an enhanced recursive least mean pth power algorithm with logarithmic transformation (RLogLMP) is proposed in this paper. The proposed algorithm is adjusted to minimize the new cost function with the p-norm logarithmic transformation of the error signal. The logarithmic transformation, which can diminish the significance of outliers under α-stable noise environment, increases the robustness of the proposed algorithm and reduces the steady-state kernel error. Moreover, the proposed method improves the convergence rate by the enhanced recursive scheme. Finally, simulation results demonstrate that the proposed algorithm is superior to the LMP, NLMP, normalized least mean absolute deviation (NLMAD), recursive least squares (RLS) and nonlinear iteratively reweighted least squares (NIRLS) algorithms in terms of convergence rate and steady-state kernel error.     

A Gaussian approximation recursive filter for nonlinear systems with correlated noises

This paper proposes a Gaussian approximation recursive filter (GASF) for a class of nonlinear stochastic systems in the case that the process and measurement noises are correlated with each other. Through presenting the Gaussian approximations about the two-step state posterior predictive probability density function (PDF) and the one-step measurement posterior predictive PDF, a general GASF framework in the minimum mean square error (MMSE) sense is derived. Based on the framework, the GASF implementation is transformed into computing the multi-dimensional integrals, which is solved by developing a new divided difference filter (DDF) with correlated noises. Simulation results demonstrate the superior performance of the proposed DDF as compared to the standard DDF, the existing UKF and EKF with correlated noises.     

Adaptive parameter estimation for a general dynamical system with unknown states

This paper is concerned with the design of a state filter for a time‐delay state‐space system with unknown parameters from noisy observation information. The key is to investigate new identification algorithms for interactive state and parameter estimation of the considered system. Firstly, an observability canonical state‐space model is derived from the original model by linear transformation for the purpose of simplifying the model structure. Secondly, a direct state filter is formulated by minimizing the state estimation error covariance matrix on the basis of the Kalman filtering principle. Thirdly, once the unknown states are estimated, a state filter–based recursive least squares algorithm is proposed for parameter estimation using the least squares principle. Then, a state filter–based hierarchical least squares algorithm is derived by decomposing the original system into several subsystems for improving the computational efficiency. Finally, the numerical examples illustrate the effectiveness and robustness of the proposed algorithms.     

Blind Capon-like adaptive ST-BC MIMO-CDMA receiver based on constant modulus criterion

In this paper, we present a new Capon-like blind receiver based on linearly constrained constant modulus (LCCM) criterion for the multiple-input multiple-output (MIMO) antennas system along with space–time block code (ST-BC) using direct-sequence code division multiple access (CDMA) modulation technique. A time-varying channel model with generalized sidelobe canceller (GSC) associated with the recursive least squares (RLS) algorithm is implemented to reduce the complexity of receiver design. In our derived algorithm, the parameter of constant modulus, α, relating to the desired user power is updated adaptively via stochastic gradient algorithm to track user?s amplitude variation. Also we prove theoretically that in the two-branch filter bank receiver design the weight vector of one branch can be updated simply using the other one, which has been obtained with our proposed CM-GSC-RLS algorithm, with simple pre-calculated transform. Hence computation complexity of the proposed adaptive blind receiver can be further reduced significantly. Via intense simulations it reveals that our proposed scheme has robust performance against the user?s acquisition inaccuracies comparing with current available algorithms.     

Entropy Optimization Filtering for Fault Isolation of Nonlinear Non-Gaussian Stochastic Systems

In this paper, the fault isolation (FI) problem is investigated for nonlinear non-Gaussian systems with multiple faults(or abrupt changes of system parameters) in the presence of noises. By constructing a filter to estimate the states, the FI problem can be reduced to an entropy optimization problem subjected to the non-Gaussian estimation error systems. The design objective for the FI purpose is that the entropy of the estimation error is maximized in the presence of diagnosed fault and is minimized in the presence of the nuisance faults or noises. It is shown that the error dynamics is represented by a nonlinear non-Gaussian stochastic system, for which new relationships are applied to formulate the probability density functions (PDFs) of the stochastic error in terms of the PDFs of the noises and the faults. The Renyi's entropy has been used to simplify the computations in the filtering for the recursive design algorithms. It is noted that the output can be supposed to be immeasurable (but with known stochastic distributions), which is different from the existing results where the output is always measurable for feedback. Finally, simulations are given to demonstrate the effectiveness of the proposed data-driven FI filtering algorithms.     

Maximum Correntropy Kalman Filtering for Non-Gaussian Systems With State Saturations and Stochastic Nonlinearities

This paper tackles the maximum correntropy Kalman filtering problem for discrete time-varying non-Gaussian systems subject to state saturations and stochastic nonlinearities. The stochastic nonlinearities, which take the form of state-multiplicative noises, are introduced in systems to describe the phenomenon of nonlinear disturbances. To resist non-Gaussian noises, we consider a new performance index called maximum correntropy criterion (MCC) which describes the similarity between two stochastic variables. To enhance the “robustness” of the kernel parameter selection on the resultant filtering performance, the Cauchy kernel function is adopted to calculate the corresponding correntropy. The goal of this paper is to design a Kalman-type filter for the underlying systems via maximizing the correntropy between the system state and its estimate. By taking advantage of an upper bound on the one-step prediction error covariance, a modified MCC-based performance index is constructed. Subsequently, with the assistance of a fixed-point theorem, the filter gain is obtained by maximizing the proposed cost function. In addition, a sufficient condition is deduced to ensure the uniqueness of the fixed point. Finally, the validity of the filtering method is tested by simulating a numerical example.      

Principal component extraction using recursive least squareslearning

A new neural network-based approach is introduced for recursive computation of the principal components of a stationary vector stochastic process. The neurons of a single-layer network are sequentially trained using a recursive least squares squares (RLS) type algorithm to extract the principal components of the input process. The optimality criterion is based on retaining the maximum information contained in the input sequence so as to be able to reconstruct the network inputs from the corresponding outputs with minimum mean squared error. The proof of the convergence of the weight vectors to the principal eigenvectors is also established. A simulation example is given to show the accuracy and speed advantages of this algorithm in comparison with the existing methods. Finally, the application of this learning algorithm to image data reduction and filtering of images degraded by additive and/or multiplicative noise is considered.     

递归最小二乘循环神经网络

针对循环神经网络(Recurrent neural networks, RNNs)一阶优化算法学习效率不高和二阶优化算法时空开销过大, 提出一种新的迷你批递归最小二乘优化算法. 所提算法采用非激活线性输出误差替代传统的激活输出误差反向传播, 并结合加权线性最小二乘目标函数关于隐藏层线性输出的等效梯度, 逐层导出RNNs参数的迷你批递归最小二乘解. 相较随机梯度下降算法, 所提算法只在RNNs的隐藏层和输出层分别增加了一个协方差矩阵, 其时间复杂度和空间复杂度仅为随机梯度下降算法的3倍左右. 此外, 本文还就所提算法的遗忘因子自适应问题和过拟合问题分别给出一种解决办法. 仿真结果表明, 无论是对序列数据的分类问题还是预测问题, 所提算法的收敛速度要优于现有主流一阶优化算法, 而且在超参数的设置上具有较好的鲁棒性.     

多算法结合的汽车参数和状态估计方法研究

为实时准确获取汽车参数及状态信息以提高汽车主动安全性能,提出了一种多算法结合的自适应估计算法。该算法将递推最小二乘算法、蚁群优化算法及容积卡尔曼滤波算法进行有效结合,同时将含有不准确模型参数及未知时变噪声的三自由度非线性整车模型作为标称模型。采用递推最小二乘算法实时估计汽车参数,引入蚁群优化算法实时跟踪容积卡尔曼滤波器的过程噪声及量测噪声,根据目标函数对噪声协方差进行寻优,以解决系统的噪声时变问题,从而获取汽车状态的准确估计。基于CarSim/Simulink的仿真实验结果表明,该算法的状态估计精度高,且具备汽车模型参数校正能力,可以满足系统的控制需要。     

基于递推最小二乘与互补滤波的姿态估计

针对基于微机电系统(MEMS)的惯性导航系统中陀螺噪声较大导致姿态漂移的问题,本文基于递推最小二乘(RLS)与互补滤波器提出一种提高姿态估计精度的方法.该方法从陀螺去噪算法和姿态解算原理两个方面提高姿态估计精度:在陀螺去噪方面,为克服传统递推最小二乘的不足,提出一种随机加权的递推最小二乘法,利用随机加权实现对偏差的估计;在姿态解算方面,在传统互补滤波器的基础上通过自适应调整比例-积分(PI)参数来调整滤波器的交接频率,最终得到陀螺积分值的高通滤波和加速度计的低通滤波的叠加.转台静态和动态实验结果表明,使用本文所提方法后,有效降低了陀螺噪声,姿态估计精度明显提升.     

基于RLS与EKF算法的锂电池SOC估计

准确估计荷电状态是电池管理系统高效和安全运行的关键因素之一.以Thevenin模型为基础,运用递推最小二乘法,对模型参数进行估计并且定期更新.采用扩展卡尔曼滤波算法实现了对锂电池荷电状态的估算.仿真结果表明,该估算策略能保持很高的精度,并对观测噪声有很强的抑制作用.     

State estimation in high-mix semiconductor manufacturing

The traditional way of state estimation in semiconductor manufacturing, known as “threaded” state estimation, segregates the process data into different bins and uses the ones that match the current event of the specific context information (such as tools, layers, products) to update the process state. The limitation of threaded state estimation is that a narrowly defined process stream can result in too many different threads and insufficient data for each thread. This limitation becomes more severe in high-mix manufacturing, where there can be many products and many tools. Hence there is great interest in estimation methods that utilize all available data in the analysis. In this work, the characteristics inherent in state estimation of high-mix semiconductor manufacturing processes are analyzed, and a general framework is introduced for the non-threaded state estimation methods, i.e., state estimation without segregating the process data into different bins. The framework is based on the best linear unbiased estimate (BLUE) of a simplified stationary singular Gauss–Markov process, and non-threaded state estimation methods based on the Kalman filter, least squares and recursive least squares (RLS) are analyzed using the general framework. Simulation examples are presented to illustrate the equivalence between different algorithms. As real processes are rarely stationary, modifications to the Kalman filter and RLS are discussed. We show that in non-threaded state estimation, how to regulate the estimate covariance plays a significant role in estimation performance. To handle nonstationary disturbances that often occur in semiconductor processes, Bayesian-enhanced adaptive versions of the Kalman filter and RLS are proposed. Both simulated and industrial nonstationary processes are used to demonstrate the effectiveness of the proposed adaptive methods.     

在不准确方差下带随机系数矩阵的卡尔曼滤波稳定性

针对离散时间线性随机系统,研究了卡尔曼滤波的L2-稳定性问题. 考虑具有这两个特点的系统: 1)系数矩阵是随机的; 2)过程噪声、量测噪声、初始估计误差的方差矩阵不准确. 在系数矩阵有界、条件能观测、初始估计误差有界的假设下, 得到了卡尔曼滤波的L2-稳定性. 同时, 建立了卡尔曼滤波和状态空间最小二乘的等价性, 并在该等价性基础上得到状态空间最小二乘的状态估计误差L2-稳定性. 最后, 数值仿真说明了卡尔曼滤波的有效性.     

Adaptive linear filtering design with minimum symbol error probability criterion

Adaptive digital filtering has traditionally been developed based on the minimum mean square error (MMSE) criterion and has found ever-increasing applications in communications. This paper presents an alternative adaptive filtering design based on the minimum symbol error rate (MSER) criterion for communication applications. It is shown that the MSER filtering is smarter, as it exploits the non-Gaussian distribution of filter output effectively. Consequently, it provides significant performance gain in terms of smaller symbol error over the MMSE approach. Adopting Parzen window or kernel density estimation for a probability density function, a block-data gradient adaptive MSER algorithm is derived. A stochastic gradient adaptive MSER algorithm, referred to as the least symbol error rate, is further developed for sample-by-sample adaptive implementation of the MSER filtering. Two applications, involving single-user channel equalization and beamforming assisted receiver, are included to demonstrate the effectiveness and generality of the proposed adaptive MSER filtering approach.     

Maximum correntropy unscented filter

The unscented transformation (UT) is an efficient method to solve the state estimation problem for a non-linear dynamic system, utilising a derivative-free higher-order approximation by approximating a Gaussian distribution rather than approximating a non-linear function. Applying the UT to a Kalman filter type estimator leads to the well-known unscented Kalman filter (UKF). Although the UKF works very well in Gaussian noises, its performance may deteriorate significantly when the noises are non-Gaussian, especially when the system is disturbed by some heavy-tailed impulsive noises. To improve the robustness of the UKF against impulsive noises, a new filter for non-linear systems is proposed in this work, namely the maximum correntropy unscented filter (MCUF). In MCUF, the UT is applied to obtain the prior estimates of the state and covariance matrix, and a robust statistical linearisation regression based on the maximum correntropy criterion is then used to obtain the posterior estimates of the state and covariance matrix. The satisfying performance of the new algorithm is confirmed by two illustrative examples.     

一种不完全信息下递推辨识方法及收敛性分析

针对信号在网络环境下传输带来不完全信息使得在线参数辨识算法和收敛性困难的问题, 不同于传统递推最小二乘方法, 本文提出了一种不完全信息下递推辨识方法并分析其收敛性. 首先运用伯努利分布刻画引起不完全信息的数据丢包特性, 然后基于辅助模型方法补偿不完全信息并构造了新的数据信息矩阵, 并运用矩阵正交变换性质对数据信息矩阵进行QR分解, 推导了融合网络参数的递推辨识新算法, 理论证明了在不完全信息下递推参数辨识算法的收敛性. 最后仿真结果验证了所提方法的可行性和有效性.     

基于序贯最小二乘的多传感器误差配准方法

为实时估计多传感器系统偏差,针对广义最小二乘(GLS)配准方法不能实时估计传感器偏差的问题,提出了基于序贯最小二乘的多传感器误差估计方法,该方法在GLS配准模型基础上,采用最小二乘的序贯方法来估计系统偏差,不必存储过去的测量数据,能够实时估计系统偏差。仿真结果表明了该方法的有效性。     

Stochastic analysis of the Least Mean Kurtosis algorithm for Gaussian inputs

The Least Mean Kurtosis (LMK) algorithm was initially proposed as an adaptive algorithm that is robust to the observation noise distribution. Good performances of this algorithm have been shown for non-Gaussian additive measurement noise. However, the complexity of the algorithm imposes difficulties for the development of a reasonably complete theoretical stochastic model for its behavior. The purpose of this paper is to contribute to the development of such a model. We study the stochastic behavior of Least Mean Kurtosis (LMK) algorithm for Gaussian inputs and for additive noises with even probability density functions. Deterministic recursions are derived for the adaptive weight error covariance matrix in a very novel manner, leading to a recursive model for the excess mean square error (EMSE) behavior that is shown to be accurate for Gaussian, uniform and binary noise distributions. The analysis results are then used to compare the performances of LMK with the least mean squares (LMS) and least mean fourth (LMF) algorithms under different circumstances.     

Covariance-based fusion filtering for networked systems with random transmission delays and non-consecutive losses

The distributed and centralized fusion filtering problems for multi-sensor networked systems with transmission random one-step delays and non-consecutive packet losses are addressed. The signal evolution model is not required, as only covariance information is used. The measurements of individual sensors, subject to uncertainties modeled by random matrices and correlated noises, are transmitted to local processors through different communication channels and, due to random transmission failures, some of the data packets may be delayed or even definitely lost. The random transmission delays and non-consecutive packet losses are modeled by sequences of Bernoulli variables with different probabilities. By an innovation approach, local least squares linear filtering estimators are obtained by recursive algorithms; the distributed fusion framework is then used to obtain the optimal matrix-weighted combination of the local filters, using the mean squared error as optimality criterion. Also, a recursive least squares linear estimation algorithm is designed within the centralized fusion framework.