Joint dehazing and denoising for single nighttime image via multi-scale decomposition

Outdoor images taken in the foggy or haze weather conditions are usually contaminated due to the presence of turbid medium in the atmosphere. Moreover, images captured under nighttime haze scenarios will be degraded even further owing to some unexpected factors. However, most existing dehazing methods mainly focus on daytime haze scenes, which cannot effectively remove the haze and suppress the noise for nighttime hazy images. To overcome these intractable problems, a joint dehazing and denoising framework for nighttime haze scenes is proposed based on multi-scale decomposition. First, the glow is removed by using its characteristic of the relative smoothness and the gamma correction operation is employed on the glow-free image for improving the overall brightness. Then, we adopt the multi-scale strategy to decompose the nighttime hazy image into a structure layer and multiple texture layers based on the total variation. Subsequently, the structure layer is dehazed based on the dark channel prior (DCP) and the texture layers are denoised based on color block-matching 3D filtering (CBM3D) prior to enhancement. Finally, the dehazed structure layer and the enhanced texture layers are fused into a dehazing result. Experiments on real-world and synthetic nighttime hazy images reveal that the proposed nighttime dehazing framework outperforms other state-of-the-art daytime and nighttime dehazing techniques.

     

Image Parsing: Unifying Segmentation,Detection, and Recognition

In this paper we present a Bayesian framework for parsing images into their constituent visual patterns. The parsing algorithm optimizes the posterior probability and outputs a scene representation as a parsing graph, in a spirit similar to parsing sentences in speech and natural language. The algorithm constructs the parsing graph and re-configures it dynamically using a set of moves, which are mostly reversible Markov chain jumps. This computational framework integrates two popular inference approaches—generative (top-down) methods and discriminative (bottom-up) methods. The former formulates the posterior probability in terms of generative models for images defined by likelihood functions and priors. The latter computes discriminative probabilities based on a sequence (cascade) of bottom-up tests/filters. In our Markov chain algorithm design, the posterior probability, defined by the generative models, is the invariant (target) probability for the Markov chain, and the discriminative probabilities are used to construct proposal probabilities to drive the Markov chain. Intuitively, the bottom-up discriminative probabilities activate top-down generative models. In this paper, we focus on two types of visual patterns—generic visual patterns, such as texture and shading, and object patterns including human faces and text. These types of patterns compete and cooperate to explain the image and so image parsing unifies image segmentation, object detection, and recognition (if we use generic visual patterns only then image parsing will correspond to image segmentation (Tu and Zhu, 2002. IEEE Trans. PAMI, 24(5):657–673). We illustrate our algorithm on natural images of complex city scenes and show examples where image segmentation can be improved by allowing object specific knowledge to disambiguate low-level segmentation cues, and conversely where object detection can be improved by using generic visual patterns to explain away shadows and occlusions.     

Saliency detection based on self-adaptive multiple feature fusion for remote sensing images

ABSTRACT

Saliency detection has been revealed an effective and reliable approach to extract the region of interest (ROI) in remote sensing images. However, most existing saliency detection methods employing multiple saliency cues ignore the intrinsic relationship between different cues and do not distinguish the diverse contributions of different cues to the final saliency map. In this paper, we propose a novel self-adaptively multiple feature fusion model for saliency detection in remote sensing images to take advantage of this relationship to improve the accuracy of ROI extraction. First, we take multiple feature channels, namely colour, intensity, texture and global contrast into consideration to produce primary feature maps. Particularly, we design a novel method based on dual-tree complex wavelet transform for remote sensing images to generate texture feature pyramids. Then, we introduce a novel self-adaptive multiple feature fusion method based on low-rank matrix recovery, in which the significances of feature maps are ranked by the low rank constraint recovery, and subsequently multiple features’ contributions are allocated adaptively to produce the final saliency map. Experimental results demonstrate that our proposal outperforms the state-of-the-art methods.     

Multiphase Soft Segmentation with Total Variation and <Emphasis Type="Italic">H</Emphasis><Superscript>1</Superscript> Regularization

In this paper, we propose a variational soft segmentation framework inspired by the level set formulation of multiphase Chan-Vese model. We use soft membership functions valued in [0,1] to replace the Heaviside functions of level sets (or characteristic functions) such that we get a representation of regions by soft membership functions which automatically satisfies the sum to one constraint. We give general formulas for arbitrary N-phase segmentation, in contrast to Chan-Vese’s level set method only 2 ^m -phase are studied. To ensure smoothness on membership functions, both total variation (TV) regularization and H ¹ regularization used as two choices for the definition of regularization term. TV regularization has geometric meaning which requires that the segmentation curve length as short as possible, while H ¹ regularization has no explicit geometric meaning but is easier to implement with less parameters and has higher tolerance to noise. Fast numerical schemes are designed for both of the regularization methods. By changing the distance function, the proposed segmentation framework can be easily extended to the segmentation of other types of images. Numerical results on cartoon images, piecewise smooth images and texture images demonstrate that our methods are effective in multiphase image segmentation.     

Diorama Construction From a Single Image

Diorama artists produce a spectacular 3D effect in a confined space by generating depth illusions that are faithful to the ordering of the objects in a large real or imaginary scene. Indeed, cognitive scientists have discovered that depth perception is mostly affected by depth order and precedence among objects. Motivated by these findings, we employ ordinal cues to construct a model from a single image that similarly to Dioramas, intensifies the depth perception. We demonstrate that such models are sufficient for the creation of realistic 3D visual experiences. The initial step of our technique extracts several relative depth cues that are well known to exist in the human visual system. Next, we integrate the resulting cues to create a coherent surface. We introduce wide slits in the surface, thus generalizing the concept of cardboard cutout layers. Lastly, the surface geometry and texture are extended alongside the slits, to allow small changes in the viewpoint which enriches the depth illusion.     

Dynamic Texture Detection Based on Motion Analysis

Motion estimation is usually based on the brightness constancy assumption. This assumption holds well for rigid objects with a Lambertian surface, but it is less appropriate for fluid and gaseous materials. For these materials an alternative assumption is required. This work examines three possible alternatives: gradient constancy, color constancy and brightness conservation (under this assumption the brightness of an object can diffuse to its neighborhood). Brightness conservation and color constancy are found to be adequate models. We propose a method for detecting regions of dynamic texture in image sequences. Accurate segmentation into regions of static and dynamic texture is achieved using a level set scheme. The level set function separates each image into regions that obey brightness constancy and regions that obey the alternative assumption. We show that the method can be simplified to obtain a less robust but fast algorithm, capable of real-time performance. Experimental results demonstrate accurate segmentation by the full level set scheme, as well as by the simplified method. The experiments included challenging image sequences, in which color or geometry cues by themselves would be insufficient.     

Images as Embedded Maps and Minimal Surfaces: Movies,Color, Texture,and Volumetric Medical Images

We extend the geometric framework introduced in Sochen et al. (IEEE Trans. on Image Processing, 7(3):310–318, 1998) for image enhancement. We analyze and propose enhancement techniques that selectively smooth images while preserving either the multi-channel edges or the orientation-dependent texture features in them. Images are treated as manifolds in a feature-space. This geometrical interpretation lead to a general way for grey level, color, movies, volumetric medical data, and color-texture image enhancement.We first review our framework in which the Polyakov action from high-energy physics is used to develop a minimization procedure through a geometric flow for images. Here we show that the geometric flow, based on manifold volume minimization, yields a novel enhancement procedure for color images. We apply the geometric framework and the general Beltrami flow to feature-preserving denoising of images in various spaces.Next, we introduce a new method for color and texture enhancement. Motivated by Gabor's geometric image sharpening method (Gabor, Laboratory Investigation, 14(6):801–807, 1965), we present a geometric sharpening procedure for color images with texture. It is based on inverse diffusion across the multi-channel edge, and diffusion along the edge.     

Image saliency detection using Gabor texture cues

Image saliency analysis plays an important role in various applications such as object detection, image compression, and image retrieval. Traditional methods for saliency detection ignore texture cues. In this paper, we propose a novel method that combines color and texture cues to robustly detect image saliency. Superpixel segmentation and the mean-shift algorithm are adopted to segment an original image into small regions. Then, based on the responses of a Gabor filter, color and texture features are extracted to produce color and texture sub-saliency maps. Finally, the color and texture sub-saliency maps are combined in a nonlinear manner to obtain the final saliency map for detecting salient objects in the image. Experimental results show that the proposed method outperforms other state-of-the-art algorithms for images with complex textures.     

Modeling Textures with Total Variation Minimization and Oscillating Patterns in Image Processing

This paper is devoted to the modeling of real textured images by functional minimization and partial differential equations. Following the ideas of Yves Meyer in a total variation minimization framework of L. Rudin, S. Osher, and E. Fatemi, we decompose a given (possible textured) image f into a sum of two functions u+v, where uBV is a function of bounded variation (a cartoon or sketchy approximation of f), while v is a function representing the texture or noise. To model v we use the space of oscillating functions introduced by Yves Meyer, which is in some sense the dual of the BV space. The new algorithm is very simple, making use of differential equations and is easily solved in practice. Finally, we implement the method by finite differences, and we present various numerical results on real textured images, showing the obtained decomposition u+v, but we also show how the method can be used for texture discrimination and texture segmentation.     

基于Wallis与距离权重增强的无人机影像拼接缝消除

针对无人机影像拼接缝消除困难的问题,提出了一种基于Wallis匀光和距离权重增强的拼接缝消除算法。首先,应用Wallis匀光消除亮度差异,给出Wallis参数设置方法。然后,提出了一种基于空间相关性的距离权重分配方法,通过增强距离对权重的影响,对拼接缝处的局部纹理错位进行全局优化,起到渐入渐出的平滑过渡效果,消除纹理拼接缝。最后,选取7种不同类型影像对本文方法和其他4种经典算法进行对比,并对8条行带影像进行拼接。实验结果表明：本文算法能高效地消除由于曝光差异与几何变形引起的拼接缝,适用于无人机影像快速处理。     

Mathematical Modeling of Textures: Application to Color Image Decomposition with a Projected Gradient Algorithm

In this paper, we are interested in texture modeling with functional analysis spaces. We focus on the case of color image processing, and in particular color image decomposition. The problem of image decomposition consists in splitting an original image f into two components u and v. u should contain the geometric information of the original image, while v should be made of the oscillating patterns of f, such as textures. We propose here a scheme based on a projected gradient algorithm to compute the solution of various decomposition models for color images or vector-valued images. We provide a direct convergence proof of the scheme, and we give some analysis on color texture modeling.     

一种新的图像纹理表示方法

该文基于纹理元直方图导出了纹理的新表示方法,并根据正交镜像滤波器的小波变换能量的转换和量化来定义纹理元。此种,给出了数种小波纹理特征集的实验评价,此方法具有将Brodatz纹理分类的极好性能。同时还研究了数种考虑近似旋转不变性或比例不变性的变换。最后,引入了生成纹理直方图和形成二进制纹理集的特征空间的过程,而且纹理直方图和二进制纹理集的特征空间与颜色直方图是对称的。获得纹理的这些表示旨在为度量纹理的相似性和从图像中抽取纹理区域。     

基于语义分割的红外和可见光图像融合

红外图像即使在低光照条件下,也能根据热辐射的差异将目标与背景区分开来,而可见光图像具有高空间分辨率的纹理细节,此外,红外和可见光图像都含有相应的语义信息.因此,红外与可见光图像融合,需要既保留红外图像的辐射信息,也保留可见光图像的纹理细节,同时,也要反映出二者的语义信息.而语义分割可以将图像转换为带有语义的掩膜,提取源图像的语义信息.提出了一种基于语义分割的红外和可见光图像融合方法,能够克服现有融合方法不能针对性地提取不同区域特有信息的缺点.使用生成式对抗神经网络,并针对源图像的不同区域设计了2种不同的损失函数,以提高融合图像的质量.首先通过语义分割得到含有红外图像目标区域语义信息的掩模,并利用掩模将红外和可见光图像分割为红外图像目标区域、红外图像背景区域、可见光图像目标区域和可见光图像背景区域;然后对目标区域和背景区域分别采用不同的损失函数得到目标区域和背景区域的融合图像;最后将2幅融合图像结合起来得到最终融合图像.实验表明,融合结果目标区域对比度更高,背景区域纹理细节更丰富,提出的方法取得了较好的融合效果.     

A Review of Statistical Approaches to Level Set Segmentation: Integrating Color,Texture, Motion and Shape

Since their introduction as a means of front propagation and their first application to edge-based segmentation in the early 90’s, level set methods have become increasingly popular as a general framework for image segmentation. In this paper, we present a survey of a specific class of region-based level set segmentation methods and clarify how they can all be derived from a common statistical framework. Region-based segmentation schemes aim at partitioning the image domain by progressively fitting statistical models to the intensity, color, texture or motion in each of a set of regions. In contrast to edge-based schemes such as the classical Snakes, region-based methods tend to be less sensitive to noise. For typical images, the respective cost functionals tend to have less local minima which makes them particularly well-suited for local optimization methods such as the level set method. We detail a general statistical formulation for level set segmentation. Subsequently, we clarify how the integration of various low level criteria leads to a set of cost functionals. We point out relations between the different segmentation schemes. In experimental results, we demonstrate how the level set function is driven to partition the image plane into domains of coherent color, texture, dynamic texture or motion. Moreover, the Bayesian formulation allows to introduce prior shape knowledge into the level set method. We briefly review a number of advances in this domain.     

Stereo using monocular cues within the tensor voting framework

We address the fundamental problem of matching in two static images. The remaining challenges are related to occlusion and lack of texture. Our approach addresses these difficulties within a perceptual organization framework, considering both binocular and monocular cues. Initially, matching candidates for all pixels are generated by a combination of matching techniques. The matching candidates are then embedded in disparity space, where perceptual organization takes place in 3D neighborhoods and, thus, does not suffer from problems associated with scanline or image neighborhoods. The assumption is that correct matches produce salient, coherent surfaces, while wrong ones do not. Matching candidates that are consistent with the surfaces are kept and grouped into smooth layers. Thus, we achieve surface segmentation based on geometric and not photometric properties. Surface overextensions, which are due to occlusion, can be corrected by removing matches whose projections are not consistent in color with their neighbors of the same surface in both images. Finally, the projections of the refined surfaces on both images are used to obtain disparity hypotheses for unmatched pixels. The final disparities are selected after a second tensor voting stage, during which information is propagated from more reliable pixels to less reliable ones. We present results on widely used benchmark stereo pairs.     

Image-Based Rendering Using Parameterized Image Varieties

This paper addresses the problem of characterizing the set of all images of a rigid set of m points and n lines observed by a weak perspective or paraperspective camera. By taking explicitly into account the Euclidean constraints associated with calibrated cameras, we show that the corresponding image space can be represented by a six-dimensional variety embedded in R^2(m+n) and parameterized by the image positions of three reference points. The coefficients defining this parameterized image variety (or PIV for short) can be estimated from a sample of images of a scene via linear and non-linear least squares. The PIV provides an integrated framework for using both point and line features to synthesize new images from a set of pre-recorded pictures (image-based rendering). The proposed technique does not perform any explicit three-dimensional scene reconstruction but it supports hidden-surface elimination, texture mapping and interactive image synthesis at frame rate on ordinary PCs. It has been implemented and extensively tested on real data sets.     

Adaptive Compression of Texture Pyramids

High‐quality texture minification techniques, including trilinear and anisotropic filtering, require texture data to be arranged into a collection of pre‐filtered texture maps called mipmaps. In this paper, we present a compression scheme for mipmapped textures which achieves much higher quality than current native schemes by exploiting image coherence across mipmap levels. The basic idea is to use a high‐quality native compressed format for the upper levels of the mipmap pyramid (to retain efficient minification filtering) together with a novel compact representation of the detail provided by the highest‐resolution mipmap. Key elements of our approach include delta‐encoding of the luminance signal, efficient encoding of coherent regions through texel runs following a Hilbert scan, a scheme for run encoding supporting fast random‐access, and a predictive approach for encoding indices of variable‐length blocks. We show that our scheme clearly outperforms native 6:1 compressed texture formats in terms of image quality while still providing real‐time rendering of trilinearly filtered textures.     

Text segmentation using gabor filters for automatic document processing

There is a considerable interest in designing automatic systems that will scan a given paper document and store it on electronic media for easier storage, manipulation, and access. Most documents contain graphics and images in addition to text. Thus, the document image has to be segmented to identify the text regions, so that OCR techniques may be applied only to those regions. In this paper, we present a simple method for document image segmentation in which text regions in a given document image are automatically identified. The proposed segmentation method for document images is based on a multichannel filtering approach to texture segmentation. The text in the document is considered as a textured region. Nontext contents in the document, such as blank spaces, graphics, and pictures, are considered as regions with different textures. Thus, the problem of segmenting document images into text and nontext regions can be posed as a texture segmentation problem. Two-dimensional Gabor filters are used to extract texture features for each of these regions. These filters have been extensively used earlier for a variety of texture segmentation tasks. Here we apply the same filters to the document image segmentation problem. Our segmentation method does not assume any a priori knowledge about the content or font styles of the document, and is shown to work even for skewed images and handwritten text. Results of the proposed segmentation method are presented for several test images which demonstrate the robustness of this technique. This work was supported by the National Science Foundation under NSF grant CDA-88-06599 and by a grant from E. 1. Du Pont De Nemours & Company.     

Query processing issues in region-based image databases

Many modern image database systems adopt a region-based paradigm, in which images are segmented into homogeneous regions in order to improve the retrieval accuracy. With respect to the case where images are dealt with as a whole, this leads to some peculiar query processing issues that have not been investigated so far in an integrated way. Thus, it is currently hard to understand how the different alternatives for implementing the region-based image retrieval model might impact on performance. In this paper, we analyze in detail such issues, in particular the type of matching between regions (either one-to-one or many-to-many). Then, we propose a novel ranking model, based on the concept of Skyline, as an alternative to the usual one based on aggregation functions and k-Nearest Neighbors queries. We also discuss how different query types can be efficiently supported. For all the considered scenarios we detail efficient index-based algorithms that are provably correct. Extensive experimental analysis shows, among other things, that: (1) the 1–1 matching type has to be preferred to the N–M one in terms of efficiency, whereas the two have comparable effectiveness, (2) indexing regions rather than images performs much better, and (3) the novel Skyline ranking model is consistently the most efficient one, even if this sometimes comes at the price of a reduced effectiveness.     

融合深度网络和浅层纹理特征的甲状腺结节癌变超声图像诊断

目的 在甲状腺结节图像中对甲状腺结节进行良恶性分析,对于甲状腺癌的早期诊断有着重要的意义。随着医疗影像学的发展,大部分的早期甲状腺结节可以在超声图像中准确地检测出来,但对于结节的性质仍然缺乏准确的判断。因此,为实现更为准确的早期甲状腺结节良恶性超声图像诊断,避免不必要的针刺或其他病理活检手术、减轻病患生理痛苦和心理压力及其医疗费用,提出一种基于深度网络和浅层纹理特征融合的甲状腺结节良恶性分类新算法。方法 本文提出的甲状腺结节分类算法由4步组成。首先对超声图像进行尺度配准、人工标记以及图像复原去除以增强图像质量。然后,对增强的图像进行数据扩展,并作为训练集对预训练过的GoogLeNet卷积神经网络进行迁移学习以提取图像中的深度特征。同时,提取图像的旋转不变性局部二值模式（LBP）特征作为图像的纹理特征。最后,将深度特征与图像的纹理特征相融合并输入至代价敏感随机森林分类器中对图像进行良恶性分类。结果 本文方法在标准的甲状腺结节癌变数据集上对甲状腺结节图像取得了正确率99.15%,敏感性99.73%,特异性95.85%以及ROC曲线下面积0.997 0的的好成绩,优于现有的甲状腺结节图像分类方法。结论 实验结果表明,图像的深度特征可以描述医疗超声图像中病灶的整体感官特征,而浅层次纹理特征则可以描述超声图像的边缘、灰度分布等特征,将二者统一的融合特征则可以更为全面地描述图像中病灶区域与非病灶区域之间的差异以及不同病灶性质之间的差异。因此,本文方法可以准确地对甲状腺结节进行分类从而避免不必要手术、减轻病患痛苦和压力。