Vector-valued image regularization with PDEs: a common framework for different applications

In this paper, we focus on techniques for vector-valued image regularization, based on variational methods and PDE. Starting from the study of PDE-based formalisms previously proposed in the literature for the regularization of scalar and vector-valued data, we propose a unifying expression that gathers the majority of these previous frameworks into a single generic anisotropic diffusion equation. On one hand, the resulting expression provides a simple interpretation of the regularization process in terms of local filtering with spatially adaptive Gaussian kernels. On the other hand, it naturally disassembles any regularization scheme into the smoothing process itself and the underlying geometry that drives the smoothing. Thus, we can easily specialize our generic expression into different regularization PDE that fulfill desired smoothing behaviors, depending on the considered application: image restoration, inpainting, magnification, flow visualization, etc. Specific numerical schemes are also proposed, allowing us to implement our regularization framework with accuracy by taking the local filtering properties of the proposed equations into account. Finally, we illustrate the wide range of applications handled by our selected anisotropic diffusion equations with application results on color images.     

Orthonormal Vector Sets Regularization with PDE's and Applications

We are interested in regularizing fields of orthonormal vector sets, using constraint-preserving anisotropic diffusion PDE's. Each point of such a field is defined by multiple orthogonal and unitary vectors and can indeed represent a lot of interesting orientation features such as direction vectors or orthogonal matrices (among other examples). We first develop a general variational framework that solves this regularization problem, thanks to a constrained minimization of -functionals. This leads to a set of coupled vector-valued PDE's preserving the orthonormal constraints. Then, we focus on particular applications of this general framework, including the restoration of noisy direction fields, noisy chromaticity color images, estimated camera motions and DT-MRI (Diffusion Tensor MRI) datasets.     

各向异性扩散的彩色图像复合滤波方法

将扩散偏微分方程PDE’s推广到彩色图像滤波。分析了基于散度算子和基于迹算子PDE的优缺点,提出了一个新的扩散PDE方法。该方法在各向异性扩散基础上增加了一个震动滤波算子,并对特征根和扩散张量进行了重构,不仅对图像有很好的滤波效果,而且对边缘有增强保护的作用。实验结果表明,提出的方法达到了既有效去除噪声又保留更多细节的目的。     

Reversible Interpolation of Vectorial Images by an Anisotropic Diffusion-Projection PDE

In this paper, a nonlinear model for the interpolation of vector-valued images is proposed. This model is based on an anisotropic diffusion PDE and performs an interpolation that is reversible. The interpolation solution is restricted to the subspace of functions that can recover the discrete input image, after an appropriate smoothing and sampling. The proposed nonlinear diffusion flow lies on this subspace while its strength and anisotropy adapt to the local variations and geometry of image structures. The derived method effectively reconstructs the real image structures and yields a satisfactory interpolation result. Compared to classic and other existing PDE-based interpolation methods, our proposed method seems to increase the accuracy of the result and to reduce the undesirable artifacts, such as blurring, ringing, block effects and edge distortion. We present extensive experimental results that demonstrate the potential of the method as applied to graylevel and color images. The authors acknowledge the financial support of the Future and Emerging Technologies (FET) programme ‘ASPI’ within the Sixth Framework Programme for Research of the European Commission, under FET-Open contract No. 021324.     

An active contour model driven by anisotropic region fitting energy for image segmentation

A novel region active contour model (ACM) for image segmentation is proposed in this paper. In order to perform an accurate segmentation of images with non-homogeneous intensity, the original region fitting energy in the general region-based ACMs is improved by an anisotropic region fitting energy to evolve the contour. Using the local image information described by the structure tensor, this new region fitting energy is defined in terms of two anisotropic fitting functions that approximate the image intensity along the principal directions of variation of the intensity. Therefore, the anisotropic fitting functions extract intensity information more precisely, which enable our model to cope with the boundaries with low-contrast and complicated structures. It is incorporated into a variational formula with a total variation (TV) regularization term with respect to level set function, from which the segmentation process is performed by minimizing this variational energy functional. Experiments on the vessel and brain magnetic resonance images demonstrate the advantages of the proposed method over Chan–Vese (CV) active contours and local binary active contours (LBF) in terms of both efficiency and accuracy.     

Noisy Image Decomposition: A New Structure,Texture and Noise Model Based on Local Adaptivity

These last few years, image decomposition algorithms have been proposed to split an image into two parts: the structures and the textures. These algorithms are not adapted to the case of noisy images because the textures are corrupted by noise. In this paper, we propose a new model which decomposes an image into three parts (structures, textures and noise) based on a local regularization scheme. We compare our results with the recent work of Aujol and Chambolle. We finish by giving another model which combines the advantages of the two previous ones.     

Collaborative image retrieval via regularized metric learning

In content-based image retrieval (CBIR), relevant images are identified based on their similarities to query images. Most CBIR algorithms are hindered by the semantic gap between the low-level image features used for computing image similarity and the high-level semantic concepts conveyed in images. One way to reduce the semantic gap is to utilize the log data of users' feedback that has been collected by CBIR systems in history, which is also called “collaborative image retrieval.” In this paper, we present a novel metric learning approach, named “regularized metric learning,” for collaborative image retrieval, which learns a distance metric by exploring the correlation between low-level image features and the log data of users' relevance judgments. Compared to the previous research, a regularization mechanism is used in our algorithm to effectively prevent overfitting. Meanwhile, we formulate the proposed learning algorithm into a semidefinite programming problem, which can be solved very efficiently by existing software packages and is scalable to the size of log data. An extensive set of experiments has been conducted to show that the new algorithm can substantially improve the retrieval accuracy of a baseline CBIR system using Euclidean distance metric, even with a modest amount of log data. The experiment also indicates that the new algorithm is more effective and more efficient than two alternative algorithms, which exploit log data for image retrieval.     

Image thresholding by variational minimax optimization

In this paper we introduce an adaptive image thresholding technique via minimax optimization of a novel energy functional that consists of a non-linear convex combination of an edge sensitive data fidelity term and a regularization term. While the proposed data fidelity term requires the threshold surface to intersect the image surface only at places with large image gradient magnitude, the regularization term enforces smoothness in the threshold surface. To the best of our knowledge, all the previously proposed energy functional-based adaptive image thresholding algorithms rely on manually set weighting parameters to achieve a balance between the data fidelity and the regularization terms. In contrast, we use minimax principle to automatically find this weighting parameter value, as well as the threshold surface. Our conscious choice of the energy functional permits a variational formulation within the minimax principle leading to a globally optimum solution. The proposed variational minimax optimization is carried out by an iterative gradient descent with exact line search technique that we experimentally demonstrate to be computationally far more attractive than the Fibonacci search applied to find the minimax solution. Our method shows promising results to preserve edge/texture structures in different benchmark images over other competing methods. We also demonstrate the efficacy of the proposed method for delineating lung boundaries from magnetic resonance imagery (MRI).     

Error estimation for Bregman iterations and inverse scale space methods in image restoration

Summary In this paper, we consider error estimation for image restoration problems based on generalized Bregman distances. This error estimation technique has been used to derive convergence rates of variational regularization schemes for linear and nonlinear inverse problems by the authors before (cf. Burger in Inverse Prob 20: 1411–1421, 2004; Resmerita in Inverse Prob 21: 1303–1314, 2005; Inverse Prob 22: 801–814, 2006), but so far it was not applied to image restoration in a systematic way. Due to the flexibility of the Bregman distances, this approach is particularly attractive for imaging tasks, where often singular energies (non-differentiable, not strictly convex) are used to achieve certain tasks such as preservation of edges. Besides the discussion of the variational image restoration schemes, our main goal in this paper is to extend the error estimation approach to iterative regularization schemes (and time-continuous flows) that have emerged recently as multiscale restoration techniques and could improve some shortcomings of the variational schemes. We derive error estimates between the iterates and the exact image both in the case of clean and noisy data, the latter also giving indications on the choice of termination criteria. The error estimates are applied to various image restoration approaches such as denoising and decomposition by total variation and wavelet methods. We shall see that interesting results for various restoration approaches can be deduced from our general results by just exploring the structure of subgradients.      

一种空间自适应正则化图象盲复原算法

图象盲复原所面临的主要问题是可利用信息的不足，所以必须充分利用图象本身及成像系统的先验信息，为此，结合模糊先验辨识的思想，给出了一种新的空间自适应正则化算法，该算法先用交替最小化的迭代方法对模糊进行先验辨识，然后利用辨识结果，用各向异性扩散进行图象复原，算法充分利用了图象及成像系统（或点扩散函数PSF）的分段平滑特性，同时又利用各向异性扩散的概念，使得正则化不仅在程度上，而且在方向上都是空间自适应的，从而能够有效地进行图象盲复原，仿真结果表明，该算法的复原效果优于空间自适应各向同性正则化（SAR）算法，其收敛性能优于空间自适应各向异性正则化（SAAR）算法。     

基于自适应P-Laplace扩散的图像复原

针对图像复原问题提出基于自适应P-Laplace扩散的图像盲复原算法。该算法结合图像的梯度和曲率性质,采用各向异性的空间自适应正则化处理,建立具有非线性和各向异性扩散的正则化方程,使其在恢复目标图像时能自适应地进行梯度平滑和边缘保留。通过最小化能量方案极小化代价函数,同时通过定点交替迭代策略将非线性方程进行线性化处理,快速恢复图像。实验结果证明,该方法能对模糊图像进行有效复原,提高图像的质量。     

A Hierarchical and Contextual Model for Aerial Image Parsing

In this paper we present a hierarchical and contextual model for aerial image understanding. Our model organizes objects (cars, roofs, roads, trees, parking lots) in aerial scenes into hierarchical groups whose appearances and configurations are determined by statistical constraints (e.g. relative position, relative scale, etc.). Our hierarchy is a non-recursive grammar for objects in aerial images comprised of layers of nodes that can each decompose into a number of different configurations. This allows us to generate and recognize a vast number of scenes with relatively few rules. We present a minimax entropy framework for learning the statistical constraints between objects and show that this learned context allows us to rule out unlikely scene configurations and hallucinate undetected objects during inference. A similar algorithm was proposed for texture synthesis (Zhu et al. in Int. J. Comput. Vis. 2:107–126, 1998) but didn’t incorporate hierarchical information. We use a range of different bottom-up detectors (AdaBoost, TextonBoost, Compositional Boosting (Freund and Schapire in J. Comput. Syst. Sci. 55, 1997; Shotton et al. in Proceedings of the European Conference on Computer Vision, pp. 1–15, 2006; Wu et al. in Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8, 2007)) to propose locations of objects in new aerial images and employ a cluster sampling algorithm (C4 (Porway and Zhu, 2009)) to choose the subset of detections that best explains the image according to our learned prior model. The C4 algorithm can quickly and efficiently switch between alternate competing sub-solutions, for example whether an image patch is better explained by a parking lot with cars or by a building with vents. We also show that our model can predict the locations of objects our detectors missed. We conclude by presenting parsed aerial images and experimental results showing that our cluster sampling and top-down prediction algorithms use the learned contextual cues from our model to improve detection results over traditional bottom-up detectors alone.     

Robust object recognition using a color co-occurrence histogram and the spatial relations of image patches

We propose a robust object recognition system where patch-based pyramid images and the spatial relationships among patches are utilized for our image model. In particular, both a color histogram (CH) and a color co-occurrence histogram (CCH) are applied to obtain image features for each patch. The locations of subregions to be tested are decided by a particle filter in our matching process. We show that the performance of object recognition can be improved by using the spatial relationships among patches. To show the validity of our proposed method, we employ input images from various environments as test images. This work was presented in part at the 13th International Symposium on Artificial Life and Robotics, Oita, Japan, January 31–February 2, 2008     

基于PDE''''s的图像平滑方法

传统的图像平滑方法在去除噪声的同时往往会破坏边缘、线务、纹理等图像特征，而基于偏微分方程(PDE’s)的各向异性扩散算法则在抑制噪声的同时能够保持这些特征。本文在Perona ＆ Malik模型基础上引入梯度阈值和高斯平滑核，实验结果表明改进后的平滑方法既能更有效消除孤立噪声点，又可以更好地保持边缘。     

各向异性扩散的图像复原

根据图像复原的性质和非线性约束的原理,提出了各向异性扩散的图像复原新算法。主要思路是根据图像的纹理和边缘结构中梯度特性,采用各向异性的自适应扩散,通过最小化能量方案来极小化代价函数,同时通过定点交替迭代策略将非线性方程进行线性化处理。实验结果表明文中方法能对模糊图像的进行有效复原,提高图像的质量。     

基于抽象匹配流的彩色图像配准模型

在抽象匹配流框架下,构造能够克服大色差问题的彩色图像配准模型.该模型中,数据项采用互相关函数作为2幅图像间的相似性度量,以解决大色差问题;正则项采用各向异性扩散滤波器约束图像演化,从而实现在演化过程中对图像特征的有效保持.扩散滤波器中的扩散系数定义为关于彩色结构张量的函数,以使图像演化能够综合各通道信息,解决了各通道所得位移场不一致而引起的色彩混迭问题.实验结果表明,文中模型对具有大色差的彩色图像能够实现有效配准.     

各向异性导向滤波的红外与可见光图像融合

目的 针对红外与可见光图像融合时易产生边缘细节信息丢失、融合结果有光晕伪影等问题，同时为充分获取多源图像的重要特征，将各向异性导向滤波和相位一致性结合，提出一种红外与可见光图像融合算法。方法 首先，采用各向异性导向滤波从源图像获得包含大尺度变化的基础图和包含小尺度细节的系列细节图；其次，利用相位一致性和高斯滤波计算显著图，进而通过对比像素显著性得到初始权重二值图，再利用各向异性导向滤波优化权重图，达到去除噪声和抑制光晕伪影；最后，通过图像重构得到融合结果。结果 从主客观两个方面，将所提方法与卷积神经网络（convolutional neural network，CNN）、双树复小波变换（dual-tree complex wavelet transform，DTCWT）、导向滤波（guided filtering，GFF）和各向异性扩散（anisotropic diffusion，ADF）等4种经典红外与可见光融合方法在TNO公开数据集上进行实验对比。主观分析上，所提算法结果在边缘细节、背景保存和目标完整度等方面均优于其他4种方法；客观分析上，选取互信息（mutual information，MI）、边缘信息保持度（degree of edge information，Q^AB/F）、熵（entropy，EN）和基于梯度的特征互信息（gradient based feature mutual information，FMI_gradient）等4种图像质量评价指数进行综合评价。相较于其他4种方法，本文算法的各项指标均有一定幅度的提高，MI平均值较GFF提高了21.67%，Q^AB/F平均值较CNN提高了20.21%，EN平均值较CNN提高了5.69%，FMI_gradient平均值较GFF提高了3.14%。结论 本文基于各向异性导向滤波融合算法可解决原始导向滤波存在的细节"光晕"问题，有效抑制融合结果中伪影的产生，同时具有尺度感知特性，能更好保留源图像的边缘细节信息和背景信息，提高了融合结果的准确性。     

Multiphase Soft Segmentation with Total Variation and <Emphasis Type="Italic">H</Emphasis><Superscript>1</Superscript> Regularization

In this paper, we propose a variational soft segmentation framework inspired by the level set formulation of multiphase Chan-Vese model. We use soft membership functions valued in [0,1] to replace the Heaviside functions of level sets (or characteristic functions) such that we get a representation of regions by soft membership functions which automatically satisfies the sum to one constraint. We give general formulas for arbitrary N-phase segmentation, in contrast to Chan-Vese’s level set method only 2 ^m -phase are studied. To ensure smoothness on membership functions, both total variation (TV) regularization and H ¹ regularization used as two choices for the definition of regularization term. TV regularization has geometric meaning which requires that the segmentation curve length as short as possible, while H ¹ regularization has no explicit geometric meaning but is easier to implement with less parameters and has higher tolerance to noise. Fast numerical schemes are designed for both of the regularization methods. By changing the distance function, the proposed segmentation framework can be easily extended to the segmentation of other types of images. Numerical results on cartoon images, piecewise smooth images and texture images demonstrate that our methods are effective in multiphase image segmentation.     

Image segmentation using a multilayer level-set approach

We propose an efficient multilayer segmentation method based on implicit curve evolution and on variational approach. The proposed formulation uses the minimal partition problem as formulated by D. Mumford and J. Shah, and can be seen as a more efficient extension of the segmentation models previously proposed in Chan and Vese (Scale-Space Theories in Computer Vision, Lecture Notes in Computer Science, Vol. 1682, pp. 141–151, 1999, IEEE Trans Image Process 10(2):266–277, 2001), and Vese and Chan (Int J Comput Vis 50(3):271–293, 2002). The set of unknown discontinuities is represented implicitly by several nested level lines of the same function, as inspired from prior work on island dynamics for epitaxial growth (Caflisch et al. in Appl Math Lett 12(4):13, 1999; Chen et al. in J Comput Phys 167:475, 2001). We present the Euler–Lagrange equations of the proposed minimizations together with theoretical results of energy decrease, existence of minimizers and approximations. We also discuss the choice of the curve regularization and conclude with several experimental results and comparisons for piecewise-constant segmentation of gray-level and color images.     

Joint weighted nuclear norm and total variation regularization for hyperspectral image denoising

A hyperspectral image is typically corrupted by multiple types of noise including Gaussian noise and impulse noise. On the other hand, a hyperspectral image possesses a high correlation in its spectral dimensions, and its Casorati matrix has a very low rank. Inspired by the recent development of robust principal component analysis, which can be used to remove sparse and arbitrarily large noise from a low-rank matrix, we propose a joint weighted nuclear norm and total variation regularization method to denoise a hyperspectral image data. First, weighted nuclear norm regularization is constructed for sparse noise removal. Total variation regularization is then imposed on each band of the hyperspectral image to further remove the Gaussian noise. A concrete optimization algorithm is developed to implement the two-stage regularization. The combined approach is expected to effectively denoise hyperspectral images even with varying data structures and under varying imaging conditions. Extensive experiments on both simulated and real data sets validate the performance of our proposed method.