Joint dehazing and denoising for single nighttime image via multi-scale decomposition

Outdoor images taken in the foggy or haze weather conditions are usually contaminated due to the presence of turbid medium in the atmosphere. Moreover, images captured under nighttime haze scenarios will be degraded even further owing to some unexpected factors. However, most existing dehazing methods mainly focus on daytime haze scenes, which cannot effectively remove the haze and suppress the noise for nighttime hazy images. To overcome these intractable problems, a joint dehazing and denoising framework for nighttime haze scenes is proposed based on multi-scale decomposition. First, the glow is removed by using its characteristic of the relative smoothness and the gamma correction operation is employed on the glow-free image for improving the overall brightness. Then, we adopt the multi-scale strategy to decompose the nighttime hazy image into a structure layer and multiple texture layers based on the total variation. Subsequently, the structure layer is dehazed based on the dark channel prior (DCP) and the texture layers are denoised based on color block-matching 3D filtering (CBM3D) prior to enhancement. Finally, the dehazed structure layer and the enhanced texture layers are fused into a dehazing result. Experiments on real-world and synthetic nighttime hazy images reveal that the proposed nighttime dehazing framework outperforms other state-of-the-art daytime and nighttime dehazing techniques.

     

视觉信息损失先验的图像分层去雾方法

目的 针对自然场景下含雾图像呈现出低对比度和色彩失真的问题,提出一种基于视觉信息损失先验的图像去雾算法,将透射图预估转化成求解信息损失函数最小值的目标规划问题。方法 首先通过输入图像的视觉特性将图像划分成含雾浓度不同的3个视觉区域。然后根据含雾图像的视觉先验知识构造视觉信息损失函数,通过像素值溢出映射规律对透射率取值范围进行约束,采用随机梯度下降法求解局部最小透射率图。最后将细化后的全局透射率图代入大气散射模型求解去雾结果。结果 结合现有的典型去雾算法进行仿真实验,本文算法能够有效地复原退化场景的对比度和清晰度,相比于传统算法,本文算法在算法实时性方面提升约20%。结论 本文算法在改善中、浓雾区域去雾效果的同时,提升了透射图预估的效率,对改善雾霾天气下视觉成像系统的能见度和鲁棒性具有重要意义。     

Variational Formulation of Dark Channel Prior for Single Image Dehazing

In this paper, we analyze the single image dehazing problem and propose a new variational method to solve it based on the dark channel prior. In the analysis section, we determine the influence that error in estimation of parameters of the haze degradation model has on the reconstructed image and give conclusions that can be used in designing a dehazing method. After that, we use those conclusions to bias our variational method as well as create a smooth variant of the dark channel prior, so it can be directly used in variational methods as well as potentially deep learning methods. We compare the proposed method quantitatively on a synthetic hazy image dataset as well as qualitatively on real-life hazy images.

     

雾霾图像清晰化算法综述

雾霾图像不仅影响视觉效果,而且模糊不清晰的图像容易为后续识别、理解等高层次任务带来困难。雾霾图像清晰化问题是一个典型的不适定问题,其成像过程难以精确建模,消除图像中的雾霾面临巨大的挑战。近年来,研究者提出大量的图像去雾算法克服雾霾引起的图像降质退化,为全面认识和理解图像清晰化算法,论文对其进行梳理和综述。首先,对雾霾图像清晰化算法进行整理,根据雾霾退化过程是否有模型支持,将清晰化算法分为基于Retinex模型、大气散射模型去雾算法和无模型图像去雾算法。大气散射模型是有模型算法中主流模型,本文详细剖析了模型成像机理,并根据其成像机制揭示大气散射模型容易受大气浓度均匀分布假设的限制,较难处理非均匀雾霾图像问题。基于深度学习的无模型图像去雾算法则不仅可以应对非均匀雾霾图像,而且去雾性能获得了极大地提升。其次,本文汇总了近年来常用去雾数据集,从数据集适应范围、规模、可扩展性等多个维度进行总结。并根据雾霾图像形成方式,对人工合成雾霾数据集和真实拍摄数据集分别从定性和定量的角度探讨了数据集对图像去雾算法的影响。     

基于雾天图像降质模型的优化去雾方法

目的 图像去雾是计算机视觉的重要研究方向,既获得高质量的去雾图像,又保证较低的时间复杂度一直是图像去雾面临的挑战,为此提出了一种基于雾天图像降质模型的优化去雾方法。方法 根据雾天图像降质模型,暗原色作为先验知识,对模型的两个物理量大气光值和透射率进行优化。传统优化算法中通常都是固定其一,优化另一个物理量,与传统方法不同,考虑到大气光和透射率的相关性,采用多元优化策略,将这两个物理量作为互相影响的整体,利用迭代算法进行优化。为保持去雾图像颜色真实、自然,基于对无雾图像的统计特性,多阈值融合的约束条件作为迭代停止的条件,控制优化去雾程度,复原高质量去雾图像。结果 本文方法与其他去雾方法相比,在视觉效果上,图像结构更加清晰,细节更加丰富,色彩更加真实。在客观数据方面,本文方法获得图像的彩色直方图与有雾图像的彩色直方图在形状上更相似,同时在Cones、Herzeliya、House、Dolls对比图像中,本文方法结果图像的信息熵值都比较高,分别为13.801 270、15.490 912、15.395 014、16.276 838,且时间复杂度较He方法（使用软抠图算法优化透射率）降低了3~5倍。结论 本文去雾方法利用迭代算法对大气光和透射率进行多元优化,同时采用多阈值融合约束条件控制优化去雾程度。本文方法在色彩保真度、细节恢复等方面都优于经典算法,同时获得了较好的客观评价数据。实验结果表明,本文方法能够达到主客观都满意的效果。     

基于生成对抗网络的雾霾场景图像转换算法

本文提出了一种新的基于生成对抗网络的雾霾场景图像转换算法.生成对抗网络GAN作为无监督学习的方法,无法实现图像像素与像素之间映射,即生成图像不可控.因此,基于模型的加雾算法存在参数不确定性和应用场景局限性,本文提出了一种新方法的新应用,利用生成对抗网络实现图像转换.该方法基于生成对抗网络GAN模型,改进了GAN的生成器和判别器,进行有监督学习,以训练雾霾图像生成像素与像素之间的映射关系,实现无雾图像与有雾图像之间的转换.以图像加雾为例,本文分别设计了生成网络和判决网络,生成网络用于合成有雾图像,判决网络用于辨别合成的雾霾图像的真伪.考虑到雾霾场景图像转换的对应效果,设计了一种快捷链接沙漏形生成器网络结构,采用无雾图像作为生成网络输入,并输出合成后的有雾霾图像;具体来看,将生成网络分成编码和解码两部分,并通过相加对应间隔的卷积层来保留图像的底层纹理信息.为了更好地检验合成雾霾图像的真实程度,设计了漏斗形全域卷积判决器网络,将合成图像和目标图像分别通过判决器辨别真伪,采用全域卷积,利用神经网络进行多层下采样,最终实现分类判决,辨别图像风格.此外,本文提出了一种新的网络损失函数,通过计算GAN损失和绝对值损失之和,以训练得到更为优秀的图像转换结果.GAN损失函数的作用是使生成对抗网络GAN模型训练更加准确,而雾霾图像合成算法实际上是一个回归问题而非分类问题,生成器的作用不仅是训练判决器更加灵敏,更重要的是要生成与目标图像相似的图像.因此利用优化回归问题的绝对值损失函数,作用是为了准确学习像素间的映射关系,避免出现偏差和失真.最后本文对多类不同图像进行图像的雾霾场景转换并进行评估,分别测试该算法的图像加雾和去雾效果,并与其他算法进行对比测试.对于加雾效果,在合成场景、虚拟场景下,与软件合成效果进行对比,本文算法效果明显比软件合成效果好,不会出现色彩失真;在真实场景下,本文算法与真实拍摄的雾霾天气进行对比,结果十分相近;并且与其他GAN图像转换算法进行对比,本文算法具有明显的优势.同样本文算法在去雾效果上优势也十分明显.结果表明,本文所提基于生成对抗网络的雾霾场景图像转换算法,在主观效果和客观指标上均具有明显优势.     

Dense spatially-weighted attentive residual-haze network for image dehazing

Haze severely affects computer vision algorithms by degrading the quality of captured images and results in image data loss. With several available approaches for dehazing, single image dehazing is most preferred and challenging. We proposed a Dense Spatially-weighted Attentive Residual-haze Network (DSA Net), a novel end-to-end Encoder-decoder architecture to learn the residual haze layer between the hazy and haze-free image. We use encoder-decoder blocks with multiple skip connections to improve feature propagation. Feature Learning block uses a novel Residual Inception fused with Attention (RIA) block to learn the complex non-linearity from features extracted from the encoder part. Learning residual image is more straightforward than the whole haze-free image, and it improves the ability of the network to estimate the haze thickness accurately. DSA Net learns this less complex residual-map from the hazy input image and subtracts it from the input to obtain the dehazed images. Detail ablation study shows the effectiveness of different modules used in our architecture. Experiment results on different haze conditions demonstrate that our method shows significant improvement over other state-of-the-art methods.

     

AIDEDNet: anti-interference and detail enhancement dehazing network for real-world scenes

The haze phenomenon seriously interferes the image acquisition and reduces image quality. Due to many uncertain factors, dehazing is typically a challenge in image processing. The most existing deep learning-based dehazing approaches apply the atmospheric scattering model (ASM) or a similar physical model, which originally comes from traditional dehazing methods. However, the data set trained in deep learning does not match well this model for three reasons. Firstly, the atmospheric illumination in ASM is obtained from prior experience, which is not accurate for dehazing real-scene. Secondly, it is difficult to get the depth of outdoor scenes for ASM. Thirdly, the haze is a complex natural phenomenon, and it is difficult to find an accurate physical model and related parameters to describe this phenomenon. In this paper, we propose a black box method, in which the haze is considered an image quality problem without using any physical model such as ASM. Analytically, we propose a novel dehazing equation to combine two mechanisms: interference item and detail enhancement item. The interference item estimates the haze information for dehazing the image, and then the detail enhancement item can repair and enhance the details of the dehazed image. Based on the new equation, we design an anti-interference and detail enhancement dehazing network (AIDEDNet), which is dramatically different from existing dehazing networks in that our network is fed into the haze-free images for training. Specifically, we propose a new way to construct a haze patch on the flight of network training. The patch is randomly selected from the input images and the thickness of haze is also randomly set. Numerous experiment results show that AIDEDNet outperforms the state-of-the-art methods on both synthetic haze scenes and real-world haze scenes.     

基于级联生成对抗网络的遥感图像去雾方法

针对图像训练对的去雾算法难以应对遥感图像中训练样本对不足,且模型泛化的问题,提出一种基于级联生成对抗网络（GAN）的遥感图像去雾方法。为解决成对遥感数据集的缺失,提出了学习雾生成的U-Net GAN（UGAN）和学习去雾的像素注意GAN（PAGAN）。所提方法通过UGAN学习如何使用未配对的清晰遥感图像和带雾遥感图像集在保留遥感图像细节的同时对无雾图像进行加雾处理,然后引导PAGAN学习如何正确地对此类图像进行去雾。为了减少生成的带雾遥感图像和去雾后遥感图像之间的差异,在PAGAN中加入自我注意机制,用生成器从低分辨率图像中所有位置的细节线索生成高分辨率细节特征,用判别器检查图像远端部分的细节特征是否彼此一致。与特征融合注意网络（FFANet）、门控上下文聚合网络（GCANet）和暗通道先验（DCP）等去雾方法相比,级联GAN方法无需大量成对数据来反复训练网络。实验结果表明该方法能够有效地去除雾和薄云,在目视效果和定量指标上均优于对比方法。     

基于改进暗通道先验的交通图像去雾新方法

针对交通场景图像中由于雾霾导致的图像目标主体不清晰,影响监控效果的问题,提出一种基于导向滤波与自适应色阶调整的改进暗通道图像去雾新方法.首先,基于暗通道原理对原始图像进行映射处理,从而得到大气光成分与透射率的估计值,并利用多维导向滤波方法对大气透射率估计值进行优化处理;然后,根据图像降质过程的逆过程,求解雾霾图像清晰化处理初始结果;最后,利用多通道自适应色阶调整方法进一步优化初始结果,解决初始结果整体亮度较暗、不利于监控系统后期处理的问题.实验结果表明,清晰化处理后的图像具有较高的亮度和对比度值,较好地保留并增强了图像的边缘和细节信息,算法去雾霾效果显著,针对交通场景图像处理的自适应性较高.     

一种基于深度学习的两阶段图像去雾网络

雾是户外图像降质的主要因素之一,图像去雾旨在恢复有雾图像中的内容。基于图像底层特征和先验知识的传统算法去雾效果不稳定。针对以上问题,受深度学习理论的启发,提出一种端到端的两阶段去雾深度神经网络算法。将图像去雾和图像超分辨率重建相结合,先利用编码器-解码器预测低分辨率雾霾残留图像,再利用亚像素卷积和残差块重建出原始分辨率雾霾残留图像,最后预测出原始分辨率无雾图像。在合成和真实有雾图像上的实验表明,该算法在定量评价和定性评价中均优于对比算法。     

Aerial image dehazing using a deep convolutional autoencoder

Aerial images and videos are extensively used for object detection and target tracking. However, due to the presence of thin clouds, haze or smoke from buildings, the processing of aerial data can be challenging. Existing single-image dehazing methods that work on ground-to-ground images, do not perform well on aerial images. Moreover, current dehazing methods are not capable for real-time processing. In this paper, a new end-to-end aerial image dehazing method using a deep convolutional autoencoder is proposed. Using the convolutional autoencoder, the dehazing problem is divided into two parts, namely, encoder, which aims extract important features to dehaze hazy regions and decoder, which aims to reconstruct the dehazed image using the down-sampled image received from the encoder. In this proposed method, we also exploit the superpixels in two different scales to generate synthetic thin cloud data to train our network. Since this network is trained in an end-to-end manner, in the test phase, for each input hazy aerial image, the proposed algorithm outputs a dehazed version without requiring any other information such as transmission map or atmospheric light value. With the proposed method, hazy regions are dehazed and objects within hazy regions become more visible while the contrast of non-hazy regions is increased. Experimental results on synthetic and real hazy aerial images demonstrate the superiority of the proposed method compared to existing dehazing methods in terms of quality and speed.

     

Nighttime image dehazing using color cast removal and dual path multi-scale fusion strategy

Nighttime image dehazing aims to remove the effect of haze on the images captured in nighttime, which however, raises new challenges such as severe color distortion, more complex lighting conditions, and lower contrast. Instead of estimating the transmission map and atmospheric light that are difficult to be accurately acquired in nighttime, we propose a nighttime image dehazing method composed of a color cast removal and a dual path multi-scale fusion algorithm. We first propose a human visual system (HVS) inspired color correction model, which is effective for removing the color deviation on nighttime hazy images. Then, we propose to use dual path strategy that includes an underexposure and a contrast enhancement path for multi-scale fusion, where the weight maps are achieved by selecting appropriate exposed areas under Gaussian pyramids. Extensive experiments demonstrate that the visual effect of the hazy nighttime images in real-world datasets can be significantly improved by our method regarding contrast, color fidelity, and visibility. In addition, our method outperforms the state-of-the-art methods qualitatively and quantitatively.     

基于雾气浓度估计的图像去雾算法

根据雾气浓度的视觉特征,提出一种雾气浓度估计模型.在此基础上,结合大气散射模型,提出一种新的图像去雾算法.首先,基于雾气浓度估计模型计算出雾气浓度量化图,利用模糊聚类算法在量化图中识别出雾气最浓区域并估计出全球光; 然后,对量化图中的“非雾气最浓”区域再次进行聚类处理,根据文中所提最优透射率评价指标估计出每个聚类单元的透射率,将全球光与透射图以及有雾图像导入散射模型,便可达到去雾的目的; 最后,针对去雾后图像较实际场景偏暗,提出一种基于小波域的多尺度锐化算法进行增强处理,以改善其主观视觉质量.实验结果表明,本文算法与现有主流算法相比,具有更好的去雾效果,并且其计算速度也相对较快.     

Comprehensive survey on haze removal techniques

Image haze removal techniques are extensively used in several outdoor applications. Lack of sufficient knowledge that is required to restore hazy images, the existing techniques usually use various attributes and assign constant values to these attributes. Unsuitable assignment to these attributes does not provide desired dehazing results. The primary objective of this review paper is to provide a structured outline of some well-known haze removal techniques. This paper also focuses on the methods which can assign optimal values to image dehazing attributes. The review has revealed that the meta-heuristic techniques can attain the optimistic haze removal parameters and also concurrently develops an optimistic objective function to estimate the depth map efficiently. Finally, this paper describes the various issues and challenges of image dehazing techniques, which are required to be further studied.

     

一种基于稀疏系数匹配学习的图像去雾算法

针对现有基于先验假设的图像去雾算法无法普适性求解问题,提出一种基于稀疏系数匹配学习的图像去雾算法.该算法从图像复原角度出发,将雾天退化模型的求解转换为基于数据库的稀疏系数匹配.之后,从图像增强角度着手,将图像高亮区域对比度恢复量化为反馈迭代问题,进而有效提升图像的视觉效果.实验结果表明,所提出的算法在获得较好去雾结果的同时能够有效提升图像细节和对比度,并具有较强的适用性.     

彩色图像去雾清晰化研究

为了解决雾霾天气的影响下成像设备采集的图像容易出现的降质及色彩失真问题, 并从有雾图像中增强其细节信息, 提高原图像的对比度和清晰度. 将彩色图像RGB通道分别做相应的图像增强算法处理, 全局直方图均衡化把整体的灰度直方图进行均匀分布的处理, 小波变换算法对彩色图像进行多层次分解, 多尺度Retinex算法通过高斯函数做卷积运算对图像做多尺度变换. 实验结果表明, 全局直方图均衡化、小波变换算法和多尺度Retinex算法都能增强雾天图像的景物信息, 有对应于各自的优势和不足. 相比较这3种算法而言, 多尺度Retinex算法得到的去雾图像亮度增强、细节信息突出、失真度小, 能更好地进行去雾增强.     

基于颜色衰减先验的小波融合图像去雾

针对图像去雾问题,在颜色衰减先验基础上提出了一种基于小波融合的单幅图像去雾方法。首先,通过颜色衰减先验假设建立了透射率关于图像亮度、饱和度的线性模型,估计出图像的粗略透射率信息。其次,提取雾图像灰度图的细节信息作为透射率的细节补充。最后,采用小波变换将两者进行融合,得到准确率高的透射率,进而恢复出清晰图像。该方法避免了大气散射系数的人工选择,自动化程度高。并且结合了原图像的特性,提高了透射率的准确性。实验表明该方法泛化效果好,恢复出的图像彩色自然。     

Visibility enhancement and dehazing: Research contribution challenges and direction

Image Dehazing is a fast growing research area with several practical applications. Dehazing improves the image quality that has been affected due to the scattering phenomenon caused by air or water. Extensive work has been carried out in this field. But a significant research gap exists when it comes to large-scale performance evaluation covering a variety of dehazing approaches, datasets, and Image Quality Assessment (IQA) metrics. In this work, we try to fill this gap by reviewing more than 100 methods from different categories. We present a complete taxonomy for image dehazing methods. Haze and IQA datasets were discussed and compared. In addition, we also contribute a real-world dense-to-low high and low-resolution hazy image dataset. Our real hazy images dataset (D2L-Haze) contains 107 natural outdoor dense and moderate haze images. Qualitative and Quantitative evaluation of dehazing approaches allows us to identify their strength and limitations. Several shortcomings of the existing techniques and some open challenges are discussed. To the best of our knowledge, this is the first extensive review that covers all the aspects of Daytime image enhancement and dehazing.