Wavelet-based statistical signal processing using hidden Markovmodels

Wavelet-based statistical signal processing techniques such as denoising and detection typically model the wavelet coefficients as independent or jointly Gaussian. These models are unrealistic for many real-world signals. We develop a new framework for statistical signal processing based on wavelet-domain hidden Markov models (HMMs) that concisely models the statistical dependencies and non-Gaussian statistics encountered in real-world signals. Wavelet-domain HMMs are designed with the intrinsic properties of the wavelet transform in mind and provide powerful, yet tractable, probabilistic signal models. Efficient expectation maximization algorithms are developed for fitting the HMMs to observational signal data. The new framework is suitable for a wide range of applications, including signal estimation, detection, classification, prediction, and even synthesis. To demonstrate the utility of wavelet-domain HMMs, we develop novel algorithms for signal denoising, classification, and detection     

基于SWT域改进粒子滤波的SAR图像降斑算法

 粒子滤波(PF)非常适合处理非高斯状态空间模型的滤波问题,而SAR图像的非高斯降斑算法正是粒子滤波的一个有效应用,本文在平稳小波变换(SWT)域上提出了一种基于马尔可夫随机场(MRF)的改进PF的SAR图像降斑算法.新算法首先分析验证了SAR图像在SWT域比在DWT域中利用广义高斯分布(GGD)建模更为精确;然后针对基本PF降斑算法中的粒子整体权重偏差问题,引入MRF重新定义粒子权重,并通过权重更新粒子的采样区间以优化粒子分布;最后为了提高本文降斑算法的实时性,依据小波系数的局部统计特性把图像分为平滑和边缘进行分区域处理.本文针对模拟SAR图像和实测SAR图像进行了仿真,仿真结果和分析表明降斑后的图像能够在去除噪声的同时较好的保持图像的边缘和纹理结构特征,而且分区域处理有效地提高了算法的效率.     

Hidden Markov models for wavelet-based blind source separation.

In this paper, we consider the problem of blind source separation in the wavelet domain. We propose a Bayesian estimation framework for the problem where different models of the wavelet coefficients are considered: the independent Gaussian mixture model, the hidden Markov tree model, and the contextual hidden Markov field model. For each of the three models, we give expressions of the posterior laws and propose appropriate Markov chain Monte Carlo algorithms in order to perform unsupervised joint blind separation of the sources and estimation of the mixing matrix and hyper parameters of the problem. Indeed, in order to achieve an efficient joint separation and denoising procedures in the case of high noise level in the data, a slight modification of the exposed models is presented: the Bernoulli-Gaussian mixture model, which is equivalent to a hard thresholding rule in denoising problems. A number of simulations are presented in order to highlight the performances of the aforementioned approach: 1) in both high and low signal-to-noise ratios and 2) comparing the results with respect to the choice of the wavelet basis decomposition.     

A reconstructing approach to end‐to‐end network traffic based on multifractal wavelet model

This paper studies the reconstructing method of end‐to‐end network traffic. Due to the development of current communication networks, our networks become more complex and heterogeneous. Meanwhile, because of time‐varying nature and spatio‐temporal correlations of the end‐to‐end network traffic, to obtain it accurately is a great challenge. We propose to exploit discrete wavelet transforms and multifractal analysis to reconstruct the end‐to‐end network traffic from time–frequency domain. First, its time–frequency properties can be characterized in detail by discrete wavelet transforms. And then, we combine discrete wavelet transforms and multifractal analysis to reconstruct end‐to‐end network traffic from link loads. Furthermore, our method needs to measure end‐to‐end network traffic to build the statistical model named multifractal wavelet model. Finally, simulation results from the real backbone networks suggest that our method can reconstruct the end‐to‐end network traffic more accurately than previous methods. Copyright © 2013 John Wiley & Sons, Ltd.     

Bivariate shrinkage functions for wavelet-based denoising exploiting interscale dependency

Most simple nonlinear thresholding rules for wavelet-based denoising assume that the wavelet coefficients are independent. However, wavelet coefficients of natural images have significant dependencies. We only consider the dependencies between the coefficients and their parents in detail. For this purpose, new non-Gaussian bivariate distributions are proposed, and corresponding nonlinear threshold functions (shrinkage functions) are derived from the models using Bayesian estimation theory. The new shrinkage functions do not assume the independence of wavelet coefficients. We show three image denoising examples in order to show the performance of these new bivariate shrinkage rules. In the second example, a simple subband-dependent data-driven image denoising system is described and compared with effective data-driven techniques in the literature, namely VisuShrink, SureShrink, BayesShrink, and hidden Markov models. In the third example, the same idea is applied to the dual-tree complex wavelet coefficients.     

On Probability Density for Modeling Video Traffic

Accurate models for variable bit rate (VBR) video traffic need to allow for different frame types present in the video, different activity levels for different frames, and a variable group of pictures (GOP) structure. The temporal as well as the stochastic properties of the trace data need to be captured by any models. We propose some models that capture temporal properties of the data using doubly Markov processes and autoregressive models. We highlight the importance of capturing the stochastic properties of the data accurately, as this leads to significant improvement in the performance of the model. In order to capture the stochastic properties of the traces, the probability density function of the trace data needs to be accurately modeled. Hence, the focus of this paper is on creating autoregressive processes with arbitrary probability densities. We relate this to work in wavelet theory on the solutions to two-scale dilation equations. The performance of our model is evaluated in terms of the stochastic properties of the generated trace as well as using network simulations.     

一种基于瑞利分布的VBR视频流的小波模型

本文提出了一种新型的视频业务流模型,以Haar小波的多分辨率分析为基础,在尺度空间和小波空间分别建模,然后通过小波反变换得出仿真业务流.在最"粗"的尺度空间里,我们根据视频流的概率分布特点,采用基于瑞利(Rayleigh)分布的AR模型对尺度系数建模;在各个小波空间里,采用一般的高斯不相关小波模型(WIG,Wavelet Independent Guassian)建模.由于在尺度空间和小波空间针对各自的特点作了不同的处理,本文模型不但能较好拟合复杂业务流在各个时间尺度的概率分布特性,也能拟合其长时相关的特性.另外,在多尺度排队分析(MSQ,MultiScale Queue)的框架下,我们还推导出了基于本文模型的排队分析的理论结果.最后,通过对实际视频业务流数据仿真实验与排队分析验证了本文模型的有效性.     

Temporal BYY learning for state space approach, hidden Markovmodel, and blind source separation

The temporal Bayesian Yang-Yang (TBYY) learning has been presented for signal modeling in a general state space approach, which provides not only a unified point of view on the Kalman filter, hidden Markov model (HMM), independent component analysis (ICA), and blind source separation (BSS) with extensions, but also further advances on these studies, including a higher order HMM, independent HMM for binary BSS, temporal ICA (TICA), and temporal factor analysis for real BSS without and with noise. Adaptive algorithms are developed for implementation and criteria are provided for selecting an appropriate number of states or sources. Moreover, theorems are given on the conditions for source separation by linear and nonlinear TICA. Particularly, it has been shown that not only non-Gaussian but also Gaussian sources can also be separated by TICA via exploring temporal dependence. Experiments are also demonstrated     

Directional multiscale modeling of images using the contourlet transform.

The contourlet transform is a new two-dimensional extension of the wavelet transform using multiscale and directional filter banks. The contourlet expansion is composed of basis images oriented at various directions in multiple scales, with flexible aspect ratios. Given this rich set of basis images, the contourlet transform effectively captures smooth contours that are the dominant feature in natural images. We begin with a detailed study on the statistics of the contourlet coefficients of natural images: using histograms to estimate the marginal and joint distributions and mutual information to measure the dependencies between coefficients. This study reveals the highly non-Gaussian marginal statistics and strong interlocation, interscale, and interdirection dependencies of contourlet coefficients. We also find that conditioned on the magnitudes of their generalized neighborhood coefficients, contourlet coefficients can be approximately modeled as Gaussian random variables. Based on these findings, we model contourlet coefficients using a hidden Markov tree (HMT) model with Gaussian mixtures that can capture all interscale, interdirection, and interlocation dependencies. We present experimental results using this model in image denoising and texture retrieval applications. In denoising, the contourlet HMT outperforms other wavelet methods in terms of visual quality, especially around edges. In texture retrieval, it shows improvements in performance for various oriented textures.     

SAR speckle reduction using wavelet denoising and Markov random field modeling

The granular appearance of speckle noise in synthetic aperture radar (SAR) imagery makes it very difficult to visually and automatically interpret SAR data. Therefore, speckle reduction is a prerequisite for many SAR image processing tasks. In this paper, we develop a speckle reduction algorithm by fusing the wavelet Bayesian denoising technique with Markov-random-field-based image regularization. Wavelet coefficients are modeled independently and identically by a two-state Gaussian mixture model, while their spatial dependence is characterized by a Markov random field imposed on the hidden state of Gaussian mixtures. The Expectation-Maximization algorithm is used to estimate hyperparameters and specify the mixture model, and the iterated-conditional-modes method is implemented to optimize the state configuration. The noise-free wavelet coefficients are finally estimated by a shrinkage function based on local weighted averaging of the Bayesian estimator. Experimental results show that the proposed method outperforms standard wavelet denoising techniques in terms of the signal-to-noise ratio and the equivalent-number-of-looks measures in most cases. It also achieves better performance than the refined Lee filter.     

Classification of fMRI time series in a low-dimensional subspace with a spatial prior

We propose a new method for detecting activation in functional magnetic resonance imaging (fMRI) data. We project the fMRI time series on a low-dimensional subspace spanned by wavelet packets in order to create projections that are as non-Gaussian as possible. Our approach achieves two goals: it reduces the dimensionality of the problem by explicitly constructing a sparse approximation to the dataset and it also creates meaningful clusters allowing the separation of the activated regions from the clutter formed by the background time series. We use a mixture of Gaussian densities to model the distribution of the wavelet packet coefficients. We expect activated areas that are connected, and impose a spatial prior in the form of a Markov random field. Our approach was validated with in vivo data and realistic synthetic data, where it outperformed a linear model equipped with the knowledge of the true hemodynamic response.     

Analysis of multiscale products for step detection and estimation

We analyze discrete wavelet transform (DWT) multiscale products for detection and estimation of steps. Here the DWT is an over complete approximation to smoothed gradient estimation, with smoothing varied over dyadic scale, as developed by Mallat and Zhong (1992). The multiscale product approach was first proposed by Rosenfeld (1970) for edge detection. We develop statistics of the multiscale products, and characterize the resulting non-Gaussian heavy tailed densities. The results may be applied to edge detection with a false-alarm constraint. The response to impulses, steps, and pulses is also characterized. To facilitate the analysis, we employ a new general closed-form expression for the Cramer-Rao bound (CRB) for discrete-time step-change location estimation. The CRB can incorporate any underlying continuous and differentiable edge model, including an arbitrary number of steps. The CRB analysis also includes sampling phase offset effects and is valid in both additive correlated Gaussian and independent and identically distributed (i.i.d.) non-Gaussian noise. We consider location estimation using multiscale products, and compare results to the appropriate CRB     

Computational methods for hidden Markov tree models-an application to wavelet trees

The hidden Markov tree models were introduced by Crouse et al. in 1998 for modeling nonindependent, non-Gaussian wavelet transform coefficients. In their paper, they developed the equivalent of the forward-backward algorithm for hidden Markov tree models and called it the "upward-downward algorithm". This algorithm is subject to the same numerical limitations as the forward-backward algorithm for hidden Markov chains (HMCs). In this paper, adapting the ideas of Devijver from 1985, we propose a new "upward-downward" algorithm, which is a true smoothing algorithm and is immune to numerical underflow. Furthermore, we propose a Viterbi-like algorithm for global restoration of the hidden state tree. The contribution of those algorithms as diagnosis tools is illustrated through the modeling of statistical dependencies between wavelet coefficients with a special emphasis on local regularity changes.     

The extended alternating fractal renewal process for modelingtraffic in high-speed communication networks

Extensive studies indicate that traffic in high-speed communication networks exhibits long-range dependence (LRD) and impulsiveness, which pose new challenges in network engineering. While many models have appeared for capturing the traffic LRD, fewer models exist that account for impulsiveness as well as LRD. One of the few existing constructive models for network traffic is the celebrated on/off model or the alternating fractal renewal process (AFRP). However, although the AFRP results in aggregated traffic with LRD, it fails to capture impulsiveness, yielding traffic with Gaussian marginal distribution. A new constructive model, namely the extended AFRP (EAFRP), is proposed here, which overcomes the limitations of the AFRP model. We show that for both single-user and aggregated traffic, it results in impulsiveness and long-range dependence, the LRD being defined here in a generalized sense. We provide queueing analysis of the proposed model, which clearly demonstrates the implications of the impulsiveness in traffic engineering, and validate all theoretical findings based on real traffic data     

A long-range dependent model for Internet traffic with power transformation

Internet traffic has been shown to have long-range dependence, and is often modeled by using the fractional Gaussian noise model. The fractional Gaussian noise model can capture the autocorrelation of a real trace, but cannot fit the marginal distribution when the trace has a non-Gaussian marginal distribution. In this letter, we use the inverted Box-Cox transformation to establish a long-range dependent Internet traffic model that can simultaneously capture both the long-range dependence parameter and the marginal distribution of a real trace.     

WPANFIS: combine fuzzy neural network with multiresolution for network traffic prediction

A novel methodology for prediction of network traffic,WPANFIS,which relies on wavelet packet transform(WPT)for multi-resolution analysis and adaptive neuro-fuzzy inference system(ANFIS)is proposed in this article.The widespread existence of self-similarity in network traffic has been demonstrated in earlier studies,which exhibits both long range dependence(LRD)and short range dependence(SRD).Also,it has been shown that wavelet decomposition is an effective tool for LRD decorrelation.The new method uses WPT as extension of wavelet transform which can decoorrelate LRD and make more precisely partition in the high-frequency section of the original traffic.Then ANFIS which can extract useful information from the original traffic is implemented in this study for better prediction performance of each decomposed non-stationary wavelet coefficients.Simulation results show that the proposed WPANFIS can achieve high prediction accuracy in real network traffic environment.     

基于离散小波变换的网络流量多重分形模型

网络流量过程中所蕴含的分形尺度特性对网络性能有显著的影响。因此研究能全面准确地刻画网络流量过程在小时间／空间尺度上的复杂奇异性特征和大时间／空间尺度上的长程依赖性特征的流量模型对Internet网络工程有重要的意义。本文对实测的流量数据(从著名的校园网和国内著名的ISP)进行了分析，利用小波技术构建了一个新的网络流量的多重分形模型，通过模拟验证，发现该新模型能以较简洁的形式捕捉实际网络流量特性，并具有刻画真实流量数据中的多重分形特征的能力。     

基于四元数小波变换域统计模型的图像去噪

图像去噪是图像处理的基本问题,四元数小波变换是1种新的多尺度分析工具.图像经四元数小波变换后,其小波系数不仅在尺度间具有相关性,而且在尺度内也具有一定的相关性.首先利用层内及层间的相关性,用非高斯分布对四元数小波系数进行建模,然后给出分类准则,把小波系数分类为重要系数和不重要系数,再用非高斯分布模型对重要系数与其邻域系...     

一种基于小波变换和FIR神经网络的广域网网络流量预测模型

该文提出了一种基于小波变换和FIR神经网络的广域网网络流量预测模型,首先采用小波分解把网络流量数据分解成小波系数和尺度系数,即高频系数和低频系数,将这些不同频率成分的系数单支重构为高频流量分量和低频流量分量,利用FIR神经网络对这些分量分别进行预测,将合成之后的结果作为原始网络流量的预测。实验结果表明:采用该模型对实际的广域网网络流量数据进行预测,不仅可以得到较快的收敛效果,而且预测性能比现有的小波神经网络和FIR神经网络要好得多。