Wiener design of adaptation algorithms with time-invariant gains

A design method is presented that extends least mean squared (LMS) adaptation of time-varying parameters by including general linear time-invariant filters that operate on the instantaneous gradient vector. The aim is to track time-varying parameters of linear regression models in situations where the regressors are stationary or have slowly time-varying properties. The adaptation law is optimized with respect to the steady-state parameter error covariance matrix for time-variations modeled as vector-ARIMA processes. The design method systematically uses prior information about time-varying parameters to provide filtering, prediction, or fixed lag smoothing estimates for arbitrary lags. The method is based on a transformation of the adaptation problem into a Wiener filter design problem. The filter works in open loop for slow parameter variations, whereas a time-varying closed loop has to be considered for fast variations. In the latter case, the filter design is performed iteratively. The general form of the solution at each iteration is obtained by a bilateral Diophantine polynomial matrix equation and a spectral factorization. For white gradient noise, the Diophantine equation has a closed-form solution. Further structural constraints result in very simple design equations. Under certain model assumptions, the Wiener designed adaptation laws reduce to LMS adaptation. Compared with Kalman estimators, the channel tracking performance becomes nearly the same in mobile radio applications, whereas the complexity is, in general, much lower     

Stochastic analysis of the LMS algorithm with a saturationnonlinearity following the adaptive filter output

This paper presents a statistical analysis of the least mean square (LMS) algorithm with a zero-memory scaled error function nonlinearity following the adaptive filter output. This structure models saturation effects in active noise and active vibration control systems when the acoustic transducers are driven by large amplitude signals. The problem is first defined as a nonlinear signal estimation problem and the mean-square error (MSE) performance surface is studied. Analytical expressions are obtained for the optimum weight vector and the minimum achievable MSE as functions of the saturation. These results are useful for adaptive algorithm design and evaluation. The LMS algorithm behavior with saturation is analyzed for Gaussian inputs and slow adaptation. Deterministic nonlinear recursions are obtained for the time-varying mean weight and MSE behavior. Simplified results are derived for white inputs and small step sizes. Monte Carlo simulations display excellent agreement with the theoretical predictions, even for relatively large step sizes. The new analytical results accurately predict the effect of saturation on the LMS adaptive filter behavior     

Tracking of time-varying mobile radio channels. II. A case study

For pt.I see ibid., vol.49, p.2207-17 (2001). Low-complexity Wiener LMS (WLMS) adaptation algorithms, of use for channel estimation, have been derived in Lindbom et al. (2001). They are here evaluated on the fast fading radio channels encountered in IS-136 TDMA systems, with the aim of clarifying several issues: How much can channel estimation performance be improved with these tools, as compared to LMS adaptation? When can an improved tracking MSE be expected to result in a meaningful reduction of the bit error rate? Will optimal prediction of future channel estimates significantly improve the equalization? Can one single tracker with fixed gain be used for all encountered Doppler frequencies and SNRs, or must a more elaborate scheme be adopted? These questions are here investigated both analytically and by simulation. An exact analytical expression for the tracking MSE on two-tap FIR channels is presented and utilized. With this tool, the MSE performance and robustness of WLMS algorithms based on different statistical models can be investigated. A simulation study then compares the uncoded bit error rate of detectors, where channel trackers are used in decision directed mode in conjunction with Viterbi algorithms. A Viterbi detector combined with WLMS, based on second order autoregressive fading models possibly combined with integration, provides good performance and robustness at a reasonable complexity     

Adaptive recovery of a chirped sinusoid in noise. II. Performanceof the LMS algorithm

For pt.I see ibid., vol.39, no. 3, p.583-94 (1991). The authors present a methodology for evaluating the tracking behavior of the least-mean square (LMS) algorithm for the nontrivial case of recovering a chirped sinusoid in additive noise. A complete closed-form analysis of the LMS tracking properties for a nonstationary inverse system modeling problem is also presented. The mean-square error (MSE) performance of the LMS algorithm is calculated as a function of the various system parameters. The misadjustment or residual of the adaptive filter output is the excess MSE as compared to the optimal filter for the problem. It is caused by three errors in the adaptive weight vector: the mean lag error between the (time-varying mean) weight and the time-varying optimal weight; the fluctuations of the lag error; and the noise misadjustment which is due to the output noise. These results are important because they represent a precise analysis of a nonstationary deterministic inverse modeling system problem with the input being a colored signal. The results are in agreement with the form of the upper bounds for the misadjustment provided by E. Eweda and O. Macchi (1985) for the deterministic nonstationarity     

A general class of nonlinear normalized adaptive filteringalgorithms

The normalized least mean square (NLMS) algorithm is an important variant of the classical LMS algorithm for adaptive linear filtering. It possesses many advantages over the LMS algorithm, including having a faster convergence and providing for an automatic time-varying choice of the LMS stepsize parameter that affects the stability, steady-state mean square error (MSE), and convergence speed of the algorithm. An auxiliary fixed step-size that is often introduced in the NLMS algorithm has the advantage that its stability region (step-size range for algorithm stability) is independent of the signal statistics. In this paper, we generalize the NLMS algorithm by deriving a class of nonlinear normalized LMS-type (NLMS-type) algorithms that are applicable to a wide variety of nonlinear filter structures. We obtain a general nonlinear NLMS-type algorithm by choosing an optimal time-varying step-size that minimizes the next-step MSE at each iteration of the general nonlinear LMS-type algorithm. As in the linear case, we introduce a dimensionless auxiliary step-size whose stability range is independent of the signal statistics. The stability region could therefore be determined empirically for any given nonlinear filter type. We present computer simulations of these algorithms for two specific nonlinear filter structures: Volterra filters and the previously proposed class of Myriad filters. These simulations indicate that the NLMS-type algorithms, in general, converge faster than their LMS-type counterparts     

Tracking of time-varying mobile radio channels .1. The Wiener LMSalgorithm

Adaptation algorithms with constant gains are designed for tracking smoothly time-varying parameters of linear regression models, in particular channel models occurring in mobile radio communications. In a companion paper, an application to channel tracking in the IS-136 TDMA system is discussed. The proposed algorithms are based on two key concepts. First, the design is transformed into a Wiener filtering problem. Second, the parameters are modeled as correlated ARIMA processes with known dynamics. This leads to a new framework for systematic and optimal design of simple adaptation laws based on prior information. The algorithms can be realized as Wiener filters, called learning filters, or as "LMS/Newton" updates complemented by filters that provide predictions or smoothing estimates. The simplest algorithm, named the Wiener LMS, is presented. All parameters are here assumed governed by the same dynamics and the covariance matrix of the regressors is assumed known. The computational complexity is of the same order of magnitude as that of LMS for regressors which are either white or have autoregressive statistics. The tracking performance is, however, substantially improved     

Analytical performance of the LMS algorithm on the estimation of wide sense stationary channels

The performance of the least mean square (LMS) algorithm on the estimation of time-varying channels is analytically evaluated, using the estimation error correlation matrix, the mean-square weight error (MSWE) and the mean-square estimation error (MSE) as parameters. Expressions for those parameters are obtained from a set of hypotheses usually adopted in the communication systems context. The channel is modeled as a wide sense stationary (WSS) discrete time stochastic field with known autocorrelation. The expressions for the steady-state MSWE and MSE are particularized for the class of WSS channel models, and an original analysis of the optimum LMS step-size parameter for usual channel models is addressed. For the sake of comparison with other works, the analytical step-size optimization for random-walk models is also considered. Several estimates of MSWE curves obtained by computer simulation are compared with analytical results for validation purposes. A very good agreement between simulated and analytical results for both the MSWE expressions and the optimum value of the LMS step-size parameter is shown.     

Optimum Transmitter/Receiver Design for a Narrowband Overlay in Noncoordinated Subscriber Lines

Analytical Performance of the LMS Algorithm on the Estimation of Wide Sense Stationary Channels The performance of the least mean square (LMS) algorithm on the estimation of time-varying channels is analytically evaluated, using the estimation error-correlation matrix, the mean-square weight error (MSWE), and the mean-square estimation error (MSE) as parameters. Expressions for those parameters are obtained from a set of hypotheses usually adopted in the communication systems context. The channel is modeled as a wide sense stationary (WSS) discrete-time stochastic field with known autocorrelation. The expressions for the steady-state MSWE and MSE are specialized for the class of WSS channel models, and an original analysis of the optimum LMS step-size parameter for usual channel models is addressed. For the sake of comparison with other works, the analytical step-size optimization for random-walk models is also considered. Several estimates of MSWE curves obtained by computer simulation are compared with analytical results for validation purposes. A very good agreement between simulated and analytical results for both the MSWE expressions and the optimum value of the LMS step-size parameter is shown.     

Analysis of stochastic gradient tracking of time-varying polynomialWiener systems

This paper presents analytical and Monte Carlo results for a stochastic gradient adaptive scheme that tracks a time-varying polynomial Wiener (1958) system [i.e., a linear time-invariant (LTI) filter with memory followed by a time-varying memoryless polynomial nonlinearity]. The adaptive scheme consists of two phases: (1) estimation of the LTI memory using the LMS algorithm and (2) tracking the time-varying polynomial-type nonlinearity using a second coupled gradient search for the polynomial coefficients. The time-varying polynomial nonlinearity causes a time-varying scaling for the optimum Wiener filter for Phase 1. These time variations are removed for Phase 2 using a novel coupling scheme to Phase 1. The analysis for Gaussian data includes recursions for the mean behavior of the LMS algorithm for estimating and tracking the optimum Wiener filter for Phase 1 for several different time-varying polynomial nonlinearities and recursions for the mean behavior of the stochastic gradient algorithm for Phase 2. The polynomial coefficients are shown to be accurately tracked. Monte Carlo simulations confirm the theoretical predictions and support the underlying statistical assumptions     

Comparative tracking performance of the LMS and RLS algorithms forchirped narrowband signal recovery

This paper studies the comparative tracking performance of the recursive least squares (RLS) and least mean square (LMS) algorithms for time-varying inputs, specifically for linearly chirped narrowband input signals in additive white Gaussian noise. It is shown that the structural differences in the implementation of the LMS and RLS weight updates produce regions where the LMS performance exceeds that of the RLS and other regions where the converse occurs. These regions are shown to be a function of the signal bandwidth and signal-to-noise ratio (SNR). LMS is shown to place a notch in the signal band of the mean lag filter, thus reducing the lag error and improving the tracking performance. For the chirped signal, it is shown that this produces smaller tracking error for small SNR. For high SNR, there is a region of signal bandwidth for which RLS will provide lower error than LMS, but even for these high SNR inputs, LMS always provides superior performance for very narrowband signals     

Efficient subspace-based algorithm for adaptive bearing estimation and tracking

In some practical applications of array processing, the directions of the incident signals should be estimated adaptively, and/or the time-varying directions should be tracked promptly. In this paper, an adaptive bearing estimation and tracking (ABEST) algorithm is investigated for estimating and tracking the uncorrelated and correlated narrow-band signals impinging on a uniform linear array (ULA) based on the subspace-based method without eigendecomposition (SUMWE), where a linear operator is obtained from the array data to form a basis for the space by exploiting the array geometry and its shift invariance property. Specifically, the space is estimated using the least-mean-square (LMS) or normalized LMS (NLMS) algorithm, and the directions are updated using the approximate Newton method. The transient analyses of the LMS and NLMS algorithms are studied, where the "weight" (i.e., the linear operator) is in the form of a matrix and there is a correlation between the "additive noise" and "input data" that involve the instantaneous correlations of the received array data in the updating equation, and the step-size stability conditions are derived explicitly. In addition, the analytical expressions for the mean-square error (MSE) and mean-square deviation (MSD) learning curves of the LMS algorithm are clarified. The effectiveness of the ABEST algorithm is verified, and the theoretical analyses are corroborated through numerical examples. Simulation results show that the ABEST algorithm is computationally simple and has good adaptation and tracking abilities.     

一种新的变步长LMS自适应算法

研究了自适应最小均方误差滤波算法的步长选取问题。在分析现有变步长LMS算法的基础上,给出一种以双曲正切函数的改进形式为变步长的LMS算法。在相同收敛速度的前提下,该算法具有更小的超量均方误差;而在相同超量均方误差的前提下,该算法具有更快的收敛速度。经实验,仿真结果与理论分析相一致,证实了该算法的优越性。     

Adaptive Equalization Techniques for HF Channels

Data transmission at rates of 1.2 kbits/s or higher through voiceband ionospheric channels is subject to impairment from severe linear distortion, fast channel time variations, and severe fading. In this paper, we have focused on the performance of DFE (decision feedback equalization) receivers for communication over 3 kHz bandwidth HF channels. We describe the results of simulations for a wide range of fading rates on simulated and real recorded HF channels, using fractionally spaced DFE receivers. Both LMS (least mean square) and FRLS (fast recursive least squares) adaptation algorithms with periodic restart were evaluated, and both ideal-reference and decision-directed operation was observed. The results indicate that FRLS adaptation yields superior performance to LMS in rapid fading conditions, but that this performance advantage diminishes at low signal-to-noise ratios. Also, fade rates greater than about 1 Hz produced relatively high error rates, irrespective of which adaptation method was employed. Finally, a novel modification of the simple LMS algorithm which improves its tracking ability was evaluated. This involved preceding the LMS DFE receiver with an adaptive lattice whitening filter.     

Statistical properties of the LMS fourier analyzer in the presence of frequency mismatch

The statistical performances of the conventional adaptive Fourier analyzers, such as the least mean square (LMS), the recursive least square (RLS) algorithms, and so forth, may degenerate significantly, if the signal frequencies given to the analyzers are different from the true signal frequencies. This difference is referred to as frequency mismatch (FM). We analyze extensively the performance of the conventional LMS Fourier analyzer in the presence of FM. Difference equations governing the dynamics and closed-form steady-state expression for the estimation mean square error (MSE) of the algorithm are derived in detail. It is revealed that the discrete Fourier coefficient (DFC) estimation problem in the LMS eventually reduces to a DFC tracking one due to the FM, and an additional term derived from DFC tracking appears in the closed-form MSE expression, which essentially deteriorates the performance of the algorithm. How to derive the optimum step size parameters that minimize or mitigate the influence of the FM is also presented, which can be used to perform robust design of step size parameters for the LMS algorithm in the presence of FM. Extensive simulations are conducted to reveal the validity of the analytical results.     

MSE behaviour of biomedical event-related filters [impedancecardiography application]

The mean-squared error (MSE) behaviour for Fourier linear combiner (FLC)-based filters is analyzed, using the independence assumption. The advantage of this analysis is its simplicity compared with previous results. The MSE transient behaviour for this kind of filters is also presented for the first time. Moreover, a time-varying sequence for the least mean square (LMS) algorithm step-size is proposed to provide fast convergence with small misadjustment error. It is shown that for this sequence, the MSE behaves better as the input signal-to-noise ratio (SNR) decreases, but increases with the number of harmonics. Lastly, the authors make a brief analysis on the nonstationary behaviour of these filters, and again they find simple expressions for the MSE behaviour     

Two-stage nondata-aided adaptive linear receivers for DS/CDMA

Closed-form steady-state performance analysis of the signal-to-interference plus noise ratio (SINR) at the output of well-known adaptive implementations of the linear minimum mean-square error (MSE) receiver for direct-sequence code-division multiple access show that nondata-aided (NDA) schemes may suffer from a considerable performance degradation with respect to their data-aided counterparts. Motivated by this fact, we propose a new two-stage NDA scheme where symbol-by-symbol predecisions at the output of a first adaptive stage are used to train a second stage. We derive closed-form steady-state performance analysis for both the two-stage and classical decision-directed schemes, taking into account detection errors in decision-directed adaptation. Our analysis shows that the SINR of the two-stage algorithm is close to optimal over a large range of values, while the SINR of the decision-directed scheme is far from optimal when the optimal SINR is small. Finally, we consider the case of time-varying fading channels. We derive modified recursive least square and least mean square adaptation schemes by considering SINR maximization rather than MSE minimization (that is useless under the assumption of zero-mean random channels). The resulting two-stage receiver shows good tracking properties in heavy near-far conditions (at least for moderate normalized Doppler bandwidth), while the decision-directed receiver may easily loose tracking after deep fades.     

LMS coefficient filtering for Time-varying chirped signals

This paper presents coefficient filtering techniques in the least mean squares (LMS) algorithm to improve adaptive predictor tracking performance for time-varying chirped signals. The example application used in this paper is an electronic support measure (ESM) receiver for detecting radar chirped pulses. The leakage LMS, momentum LMS, and the proposed future-state coefficient (FC-LMS) filtering algorithms have been studied. The leakage LMS algorithm has the ability to remove the memory effect of the initial converged time-varying frequency of the chirped signal, thus improving the radar pulse detection performance. The momentum LMS is able to search for the time-varying optimum weight solution more efficiently, and the FC-LMS uses a parallel technique to retain the LMS throughput while being able to show a better tracking performance for chirped signals compared with the standard LMS algorithm.     

Exploiting sparsity in adaptive filters

This paper studies a class of algorithms called natural gradient (NG) algorithms. The least mean square (LMS) algorithm is derived within the NG framework, and a family of LMS variants that exploit sparsity is derived. This procedure is repeated for other algorithm families, such as the constant modulus algorithm (CMA) and decision-directed (DD) LMS. Mean squared error analysis, stability analysis, and convergence analysis of the family of sparse LMS algorithms are provided, and it is shown that if the system is sparse, then the new algorithms will converge faster for a given total asymptotic MSE. Simulations are provided to confirm the analysis. In addition, Bayesian priors matching the statistics of a database of real channels are given, and algorithms are derived that exploit these priors. Simulations using measured channels are used to show a realistic application of these algorithms     

Tracking of time-varying channels using two-step LMS-type adaptive algorithm

This paper presents a modified version of the two-step least-mean-square (LMS)-type adaptive algorithm motivated by the work of Gazor. We describe the nonstationary adaptation characteristics of this modified two-step LMS (MG-LMS) algorithm for the system identification problem. It ensures stable behavior during convergence as well as improved tracking performance in the smoothly time-varying environments. The estimated weight increment vector is used for the prediction of weight vector for the next iteration. The proposed modification includes the use of a control parameter to scale the estimated weight increment vector in addition to a smoothing parameter used in the two-step LMS (G-LMS) algorithm, which controls the initial oscillatory behavior of the algorithm. The analysis focuses on the effects of these parameters on the lag-misadjustment in the tracking process. The mathematical analysis for a nonstationary case, where the plant coefficients are assumed to follow a first-order Markov process, shows that the MG-LMS algorithm contributes less lag-misadjustment than the conventional LMS and G-LMS algorithms. Further, the stability criterion imposes upper bound on the value of the control parameter. These derived analytical results are verified and demonstrated with simulation examples, which clearly show that the lag-misadjustment reduces with increasing values of the smoothing and control parameters under permissible limits.     

一种基于误差相关函数的改进变步长LMS算法

最小均方算法的收敛速度和稳态误差之间存在矛盾,为此人们提出了各种变步长LMS算法,其中E-LMS算法是将步长与瞬时误差平方相关联,R-LMS算法是将步长与误差的相关函数相关联。E-LMS算法的抗噪性能较差,在低信噪比条件下性能明显变差,R-LMS算法对突变系统的跟踪能力较差。为此文中给出了一种改进的,基于误差相关函数的VSS-LMS算法,该方法利用E-LMS算法的控制步长策略提高算法的跟踪能力。计算机仿真结果显示,该算法能够同时满足抗噪和跟踪两种要求。