Sparse structured non-fragile H ∞ controller design for linear systems

This paper presents a study on the problem of designing non-fragile H _∞ controllers with sparse structures for linear continuous-time systems. A new algorithm is proposed to define and further design sparse structured controllers. Firstly, sparse structures are specified from a given fully parameterized H _∞ controller. Then, a three-step design procedure for non-fragile dynamic output feedback H _∞ controllers with the sparse structures is provided. The resulting designs guarantee that the closed-loop system is asymptotically stable and the H _∞ performance from the disturbance to the regulated output is less than a prescribed level. A numerical example is given to illustrate the design methods.     

A novel digital filter structure with minimum roundoff noise

In this paper, a novel structure is derived for efficient implementation of digital filters as well as minimizing the finite word length (FWL) errors. Such a structure is actually an improved version of that reported previously. The performance of this new structure and the famous normalized lattice structure are analyzed by deriving the corresponding expression for the roundoff noise gain. Design examples are presented to illustrate the behavior of the proposed structure and to compare it with some existing ones. It is shown that the proposed structure outperforms the others in terms of minimizing roundoff noise as well as implementation efficiency.     

Noise-robust semi-supervised learning via fast sparse coding

This paper presents a novel noise-robust graph-based semi-supervised learning algorithm to deal with the challenging problem of semi-supervised learning with noisy initial labels. Inspired by the successful use of sparse coding for noise reduction, we choose to give new L₁-norm formulation of Laplacian regularization for graph-based semi-supervised learning. Since our L₁-norm Laplacian regularization is explicitly defined over the eigenvectors of the normalized Laplacian matrix, we formulate graph-based semi-supervised learning as an L₁-norm linear reconstruction problem which can be efficiently solved by sparse coding. Furthermore, by working with only a small subset of eigenvectors, we develop a fast sparse coding algorithm for our L₁-norm semi-supervised learning. Finally, we evaluate the proposed algorithm in noise-robust image classification. The experimental results on several benchmark datasets demonstrate the promising performance of the proposed algorithm.     

Discriminative learning by sparse representation for classification

Recently, sparsity preserving projections (SPP) algorithm has been proposed, which combines l₁-graph preserving the sparse reconstructive relationship of the data with the classical dimensionality reduction algorithm. However, when applied to classification problem, SPP only focuses on the sparse structure but ignores the label information of samples. To enhance the classification performance, a new algorithm termed discriminative learning by sparse representation projections or DLSP for short is proposed in this paper. DLSP algorithm incorporates the merits of both local interclass geometrical structure and sparsity property. That makes it possess the advantages of the sparse reconstruction, and more importantly, it has better capacity of discrimination, especially when the size of the training set is small. Extensive experimental results on serval publicly available data sets show the feasibility and effectiveness of the proposed algorithm.     

Robust multi-view L2 triangulation via optimal inlier selection and 3D structure refinement

This paper presents a new robust approach for multi-view L₂ triangulation based on optimal inlier selection and 3D structure refinement. The proposed method starts with estimating the scale of noise in image measurements, which affects both the quantity and the accuracy of reconstructed 3D points but is overlooked or ignored in existing triangulation pipelines. A new residual-consensus scheme within which the uncertainty of epipolar transfer is analytically characterized by deriving its closed-form covariance is developed to robustly estimate the noise scale. Different from existing robust triangulation pipelines, the issue of outliers is addressed by directly searching for the optimal 3D points that are within either the theoretical correct error bounds calculated by second-order cone programming (SOCP) or the efficiently calculated approximate ranges. In particular, both the inlier selection and 3D structure refinement are realized in an optimal fashion using Differential Evolution (DE) optimization which allows flexibility in the design of the objective function. To validate the performance of the proposed method, extensive experiments using both synthetic data and real image sequences were carried out. Comparing with state-of-the-art robust triangulation strategies, the proposed method can consistently identify more reliable inliers and hence, reconstruct more unambiguous 3D points with higher accuracy than existing methods.     

Improve robustness of sparse PCA by L1-norm maximization

Various sparse principal component analysis (PCA) methods have recently been proposed to enhance the interpretability of the classical PCA technique by extracting principal components (PCs) of the given data with sparse non-zero loadings. However, the performance of these methods is prone to be adversely affected by the presence of outliers and noises. To alleviate this problem, a new sparse PCA method is proposed in this paper. Instead of maximizing the L₂-norm variance of the input data as the conventional sparse PCA methods, the new method attempts to capture the maximal L₁-norm variance of the data, which is intrinsically less sensitive to noises and outliers. A simple algorithm for the method is specifically designed, which is easy to be implemented and converges to a local optimum of the problem. The efficiency and the robustness of the proposed method are theoretically analyzed and empirically verified by a series of experiments implemented on multiple synthetic and face reconstruction problems, as compared with the classical PCA method and other typical sparse PCA methods.     

Recursive Generalized Maximum Correntropy Criterion Algorithm with Sparse Penalty Constraints for System Identification

To address the sparse system identification problem in a non‐Gaussian impulsive noise environment, the recursive generalized maximum correntropy criterion (RGMCC) algorithm with sparse penalty constraints is proposed to combat impulsive‐inducing instability. Specifically, a recursive algorithm based on the generalized correntropy with a forgetting factor of error is developed to improve the performance of the sparsity aware maximum correntropy criterion algorithms by achieving a robust steady‐state error. Considering an unknown sparse system, the l₁‐norm and correntropy induced metric are employed in the RGMCC algorithm to exploit sparsity as well as to mitigate impulsive noise simultaneously. Numerical simulations are given to show that the proposed algorithm is robust while providing robust steady‐state estimation performance.     

<Emphasis Type="Italic">υ</Emphasis>-Support vector machine based on discriminant sparse neighborhood preserving embedding

In this paper, we mainly focus on two issues (1) SVM is very sensitive to noise. (2) The solution of SVM does not take into consideration of the intrinsic structure and the discriminant information of the data. To address these two problems, we first propose an integration model to integrate both the local manifold structure and the local discriminant information into ?₁ graph embedding. Then we add the integration model into the objection function of υ-support vector machine. Therefore, a discriminant sparse neighborhood preserving embedding υ-support vector machine (υ-DSNPESVM) method is proposed. The theoretical analysis demonstrates that υ-DSNPESVM is a reasonable maximum margin classifier and can obtain a very lower generalization error upper bound by minimizing the integration model and the upper bound of margin error. Moreover, in the nonlinear case, we construct the kernel sparse representation-based ?₁ graph for υ-DSNPESVM, which is more conducive to improve the classification accuracy than ?₁ graph constructed in the original space. Experimental results on real datasets show the effectiveness of the proposed υ-DSNPESVM method.     

Robust visual tracking using information theoretical learning

This paper presents a novel online object tracking algorithm with sparse representation for learning effective appearance models under a particle filtering framework. Compared with the state-of-the-art ? ₁ sparse tracker, which simply assumes that the image pixels are corrupted by independent Gaussian noise, our proposed method is based on information theoretical Learning and is much less sensitive to corruptions; it achieves this by assigning small weights to occluded pixels and outliers. The most appealing aspect of this approach is that it can yield robust estimations without using the trivial templates adopted by the previous sparse tracker. By using a weighted linear least squares with non-negativity constraints at each iteration, a sparse representation of the target candidate is learned; to further improve the tracking performance, target templates are dynamically updated to capture appearance changes. In our template update mechanism, the similarity between the templates and the target candidates is measured by the earth movers’ distance(EMD). Using the largest open benchmark for visual tracking, we empirically compare two ensemble methods constructed from six state-of-the-art trackers, against the individual trackers. The proposed tracking algorithm runs in real-time, and using challenging sequences performs favorably in terms of efficiency, accuracy and robustness against state-of-the-art algorithms.     

Multiplicative noise removal via adaptive learned dictionaries and TV regularization

Multiplicative noise removal is a key issue in image processing problem. While a large amount of literature on this subject are total variation (TV)-based and wavelet-based methods, recently sparse representation of images has shown to be efficient approach for image restoration. TV regularization is efficient to restore cartoon images while dictionaries are well adapted to textures and some tricky structures. Following this idea, in this paper, we propose an approach that combines the advantages of sparse representation over dictionary learning and TV regularization method. The method is proposed to solve multiplicative noise removal problem by minimizing the energy functional, which is composed of the data-fidelity term, a sparse representation prior over adaptive learned dictionaries, and TV regularization term. The optimization problem can be efficiently solved by the split Bregman algorithm. Experimental results validate that the proposed model has a superior performance than many recent methods, in terms of peak signal-to-noise ratio, mean absolute-deviation error, mean structure similarity, and subjective visual quality.     

Sparse regularization method for the detection and removal of random-valued impulse noise

In this paper, we propose a novel two-stage algorithm for the detection and removal of random-valued impulse noise using sparse representations. The main aim of the paper is to demonstrate the strength of image inpainting technique for the reconstruction of images corrupted by random-valued impulse noise at high noise densities. First, impulse locations are detected by applying the combination of sparse denoising and thresholding, based on sparse and overcomplete representations of the corrupted image. This stage optimally selects threshold values so that the sum of the number of false alarms and missed detections obtained at a particular noise level is the minimum. In the second stage, impulses, detected in the first stage, are considered as the missing pixels or holes and subsequently these holes are filled-up using an image inpainting method. Extensive simulation results on standard gray scale images show that the proposed method successfully removes random-valued impulse noise with better preservation of edges and other details compared to the existing techniques at high noise ratios.     

Robust formulation for Reliability-based design optimization of structures

The reliability-based design optimization (RBDO) has been widely recognized as a powerful optimization tool under probabilistic constraints, through appropriate modeling of uncertainties. However, the drawback of RBDO is that it does not reflect the ability of the structure to comply with large data variations, unforeseen actions or deterioration mechanisms. On the other hand, the robust design optimization (RDO) reduces the variability of the structural performance, in addition to its mean level. However, RDO does not take direct advantage of the interaction between controllable (product design values) and noise variables (environmental random values), and the obtained results do not accurately indicate what parameter has the highest effect on the performance characteristics. The purpose of this paper is to propose a robust formulation for reliability-based design optimization (RRBDO) that combines the advantages of both optimization procedures and overcomes their weaknesses. The optimization model proposed overcomes the limitations of the existing models without compromising the reliability level, by considering a robust convex objective function and a performance variation constraint. The proposed formulation can consider the total cost of structures and can control structural parameter variations. It takes into account uncertainty and variability in the same mathematical formulation. A numerical solution procedure is also developed, for which results are analyzed and compared with RBDO for several examples of concrete and steel structures.     

联合低秩稀疏的多核子空间聚类算法

针对多核子空间谱聚类算法没有考虑噪声和关系图结构的问题，提出了一种新的联合低秩稀疏的多核子空间聚类算法(JLSMKC)。首先，通过联合低秩与稀疏表示进行子空间学习，使关系图具有低秩和稀疏结构属性；其次，建立鲁棒的多核低秩稀疏约束模型，用于减少噪声对关系图的影响和处理数据的非线性结构；最后，通过多核方法充分利用共识核矩阵来增强关系图质量。7个数据集上的实验结果表明，所提算法JLSMKC在聚类精度(ACC)、标准互信息(NMI)和纯度(Purity)上优于5种流行的多核聚类算法，同时减少了聚类时间，提高了关系图块对角质量。该算法在聚类性能上有较大优势。     

Reconstruction of N-th order nonuniformly sampled bandlimited signals using digital filter banks

In this paper, a digital filter bank structure is proposed for the reconstruction of uniformly sampled bandlimited signals from their N-th order nonuniform samples. The proposed filter bank structure is arrived at after incorporating polyphase-domain filtering operations and discrete Fourier transform (DFT) modulation to an existing filter bank framework.In this paper, an idea is also presented, so that uniform samples can be reconstructed from N-th order nonuniform samples using the structures based on recurrent nonuniform sampling. A comparison of the computational complexity and the signal-to-noise ratio (SNR) performance is also given for various structures existing in the literature.     

Joint weighted nuclear norm and total variation regularization for hyperspectral image denoising

A hyperspectral image is typically corrupted by multiple types of noise including Gaussian noise and impulse noise. On the other hand, a hyperspectral image possesses a high correlation in its spectral dimensions, and its Casorati matrix has a very low rank. Inspired by the recent development of robust principal component analysis, which can be used to remove sparse and arbitrarily large noise from a low-rank matrix, we propose a joint weighted nuclear norm and total variation regularization method to denoise a hyperspectral image data. First, weighted nuclear norm regularization is constructed for sparse noise removal. Total variation regularization is then imposed on each band of the hyperspectral image to further remove the Gaussian noise. A concrete optimization algorithm is developed to implement the two-stage regularization. The combined approach is expected to effectively denoise hyperspectral images even with varying data structures and under varying imaging conditions. Extensive experiments on both simulated and real data sets validate the performance of our proposed method.     

Blind Image Deblurring via Adaptive Optimization with Flexible Sparse Structure Control

Blind image deblurring is a long-standing ill-posed inverse problem which aims to recover a latent sharp image given only a blurry observation. So far, existing studies have designed many effective priors w.r.t. the latent image within the maximum a posteriori (MAP) framework in order to narrow down the solution space. These non-convex priors are always integrated into the final deblurring model, which makes the optimization challenging. However, due to unknown image distribution, complex kernel structure and non-uniform noises in real-world scenarios, it is indeed challenging to explicitly design a fixed prior for all cases. Thus we adopt the idea of adaptive optimization and propose the sparse structure control (SSC) for the latent image during the optimization process. In this paper, we only formulate the necessary optimization constraints in a lightweight MAP model with no priors. Then we develop an inexact projected gradient scheme to incorporate flexible SSC in MAP inference. Besides l_p-norm based SSC in our previous work, we also train a group of denoising convolutional neural networks (CNNs) to learn the sparse image structure automatically from the training data under different noise levels, and we show that CNNs-based SSC can achieve similar results compared with l_p-norm but are more robust to noise. Extensive experiments demonstrate that the proposed adaptive optimization scheme with two types of SSC achieves the state-of-the-art results on both synthetic data and real-world images.

     

Restoration of images corrupted by mixed Gaussian-impulse noise via l1-l0 minimization

In this paper, we study the restoration of images corrupted by Gaussian plus impulse noise, and propose a l₁-l₀ minimization approach where the l₁ term is used for impulse denoising and the l₀ term is used for a sparse representation over certain unknown dictionary of images patches. The main algorithm contains three phases. The first phase is to identify the outlier candidates which are likely to be corrupted by impulse noise. The second phase is to recover the image via dictionary learning on the free-outlier pixels. Finally, an alternating minimization algorithm is employed to solve the proposed minimization energy function, leading to an enhanced restoration based on the recovered image in the second phase. Experimental results are reported to compare the existing methods and demonstrate that the proposed method is better than the other methods.     

Sparse Matrix Block-Cyclic Realignment on Distributed Memory Machines

Modifying the data distribution over the course of a program to adapt to variations in the data access patterns may leads to significant computational benefits in many scientific applications. Therefore, dynamic realignment of data is used to enhance algorithm performance in parallel programs on distributed memory machines. This paper presents a new method aims to the efficiency of block-cyclic data realignment of sparse matrix. The main idea of the proposed technique is first todevelop closed forms for generating the Vector Index Set (VIS) of each source/destination processor. Based on the vector index set and the nonzero structure of sparse matrix, two efficient algorithms,vector2message (v2m) and message2vector (m2v) can be derived. The proposed technique uses v2m to extract nonzero elements from source compressed structures and packs them into messages in the source stage; and uses m2v to unpack each received messages and construct the destination matrix in the destination stage. A significant improvement of this approach is that a processor does not need to determine the complicated sending or receiving data sets for dynamic data redistribution. The indexing cost is reduced obviously. The second advantage of the present techniques is the achievement of optimal packing/unpacking stages consequent upon the informative VIS tables. Another contribution of our methods is the ability to handle sparse matrix redistribution under two disjoint processor grids in the source and destination phases. A theoretical model to analyze the performance of the proposed technique is also presented in this work. To evaluate the performance of our methods, we have implemented the present algorithms on an IBM SP2 parallel machine along with the Histogram method and a dense redistribution strategy. The experimental results show that our technique provides significant improvement for runtime data redistribution of sparse matrices in most test samples.     

Fast Fisher Sparsity Preserving Projections

Recently, there has been a lot of interest in the underlying sparse representation structure in high-dimensional data such as face images. In this paper, we propose two novel efficient dimensionality reduction methods named Fast Sparsity Preserving Projections (FSPP) and Fast Fisher Sparsity Preserving Projections (FFSPP), respectively, which aim to preserve the sparse representation structure in high-dimensional data. Unlike the existing Sparsity Preserving Projections (SPP), where the sparse representation structure is learned through resolving n (the number of samples) time-consuming $ \ell^{ 1} $ norm optimization problems, FSPP constructs a dictionary through classwise PCA decompositions and learns the sparse representation structure under the constructed dictionary through matrix–vector multiplications, which is much more computationally tractable. FFSPP takes into consideration both the sparse representation structure and the discriminating efficiency by adding the Fisher constraint to the FSPP formulation to improve FSPP’s discriminating ability. Both of the proposed methods can boil down to a generalized eigenvalue problem. Experimental results on three publicly available face data sets (Yale, Extended Yale B and ORL), and a standard document collection (Reuters-21578) validate the feasibility and effectiveness of the proposed methods.     

Variational Bayesian sparse additive matrix factorization

Principal component analysis (PCA) approximates a data matrix with a low-rank one by imposing sparsity on its singular values. Its robust variant can cope with spiky noise by introducing an element-wise sparse term. In this paper, we extend such sparse matrix learning methods, and propose a novel framework called sparse additive matrix factorization (SAMF). SAMF systematically induces various types of sparsity by a Bayesian regularization effect, called model-induced regularization. Although group LASSO also allows us to design arbitrary types of sparsity on a matrix, SAMF, which is based on the Bayesian framework, provides inference without any requirement for manual parameter tuning. We propose an efficient iterative algorithm called the mean update (MU) for the variational Bayesian approximation to SAMF, which gives the global optimal solution for a large subset of parameters in each step. We demonstrate the usefulness of our method on benchmark datasets and a foreground/background video separation problem.