An adaptive fusion approach for infrared and visible images based on NSCT and compressed sensing

A novel nonsubsampled contourlet transform (NSCT) based image fusion approach, implementing an adaptive-Gaussian (AG) fuzzy membership method, compressed sensing (CS) technique, total variation (TV) based gradient descent reconstruction algorithm, is proposed for the fusion computation of infrared and visible images.Compared with wavelet, contourlet, or any other multi-resolution analysis method, NSCT has many evident advantages, such as multi-scale, multi-direction, and translation invariance. As is known, a fuzzy set is characterized by its membership function (MF), while the commonly known Gaussian fuzzy membership degree can be introduced to establish an adaptive control of the fusion processing. The compressed sensing technique can sparsely sample the image information in a certain sampling rate, and the sparse signal can be recovered by solving a convex problem employing gradient descent based iterative algorithm(s).In the proposed fusion process, the pre-enhanced infrared image and the visible image are decomposed into low-frequency subbands and high-frequency subbands, respectively, via the NSCT method as a first step. The low-frequency coefficients are fused using the adaptive regional average energy rule; the highest-frequency coefficients are fused using the maximum absolute selection rule; the other high-frequency coefficients are sparsely sampled, fused using the adaptive-Gaussian regional standard deviation rule, and then recovered by employing the total variation based gradient descent recovery algorithm.Experimental results and human visual perception illustrate the effectiveness and advantages of the proposed fusion approach. The efficiency and robustness are also analyzed and discussed through different evaluation methods, such as the standard deviation, Shannon entropy, root-mean-square error, mutual information and edge-based similarity index.     

Multifocus image fusion based on features contrast of multiscale products in nonsubsampled contourlet transform domain

In this paper, an efficient multifocus image fusion approach is proposed based on local features contrast of multiscale products in nonsubsampled contourlet transform (NSCT) domain. In order to improve the robustness of the fusion algorithm to the noise and select the coefficients of the fused image properly, the multiscale products, which can distinguish edge structures from noise more effectively in NSCT domain, is developed and introduced into image fusion field. The selection principles of different subband coefficients obtained by the NSCT decomposition are discussed in detail. To improve the quality of the fused image, novel different local features contrast measurements, which are proved to be more suitable for human vision system and can extract more useful detail information from source images and inject them into the fused image, are developed and used to select coefficients from the clear parts of subimages to compose coefficients of fused images. Experimental results demonstrate the proposed method performs very well in fusion both noisy and noise-free multifocus images, and outperform conventional methods in terms of both visual quality and objective evaluation criteria.     

A multi-focus image fusion algorithm based on an improved dual-channel PCNN in NSCT domain

This paper presents a multi-focus image fusion algorithm based on dual-channel PCNN in NSCT domain. The fusion algorithm based on multi-scale transform is likely to produce the pseudo-Gibbs effects and it is not effective to fuse the dim or partial bright images. To solve these problems, this algorithm will get a number of different frequency sub-image of the two images by using the NSCT transform, the selection principles of different subband coefficients obtained by the NSCT decomposition are discussed in detail, and the images are fused based on the improved dual-channel PCNN in order to determine the band-pass sub-band coefficient, at last fused image is obtained by using the inverse NSCT transform. Fusion rules based on dual-channel PCNN are used to solve the complexity of the PCNN parameter settings and long computing time problems. The experimental results show that the algorithm has overcome the defects of the traditional multi-focus image fusion algorithm and improved the fusion effect.     

Image fusion based on nonsubsampled contourlet transform for infrared and visible light image

In this paper, a new method based on nonsubsampled contourlet transform (NSCT) is proposed to fuse the infrared image and the visible light image, which will produce a new fused image by which the target can be identified more easily. Firstly, two original images were decomposed into low frequency subband coefficients and the bandpass direction subband coefficients by using NSCT. Secondly, the selection of the low frequency subband coefficient and the bandpass direction subband coefficient is discussed in detail. The low frequency subband coefficients are selected based on the regional visual characteristics. For the selection of bandpass direction subband coefficients, this paper proposes a minimum regional cross-gradient method, and the cross-gradient is gained by calculating the gradient between the pixel of bandpass subbands and the adjacent pixel in the fused image of the low-frequency components. Comparison experiments have been performed on different image sets, and experimental results demonstrate that the proposed method performs better in both subjective and objective qualities.     

一种基于清晰度计算的NSCT域多聚焦图像融合算法

针对多聚焦图像的特点．提出了一种基于清晰度计算的非抽样轮廓波变换（Non-Subsampied Contourlet’Transform,NSCT）域多聚焦图像融合算法。该算法首先对源图像进行NSCT分解．以此克服传统Contourlet变换不具平移不变性的缺点。在分析光学成像中散焦表现形式的基础上．对分解后的低频子带和高频方向子带分别以“邻域梯度”及“合成邻域模值”作为清晰度指标。采用自适应选择法实现对多聚焦图像的融合处理。实验结果表明,该方法不仅能有效融合图像中的“伪影”和“振铃效应”．视觉效果明显优于传统小波和Contourlet方法,且融合图像的熵、交叉熵及均方根交叉熵等客观评价指标也有明显提高。     

基于区域检测与非下采样轮廓波变换的红外与彩色可见光图像融合

针对灰度图像融合的分辨率低及现有的彩色图像融合方法融合的图像色彩不自然、不符合人的视觉感受的特点,在此提出一种基于Snake模型的区域检测和非下采样轮廓波变换(NSCT)的红外与彩色可见光图像融合的方法。首先对彩色可见光图像进行亮度、色度和饱和度(IHS)颜色空间变换提取亮度分量,并用Snake模型对红外图像的目标区域进行检测;然后对亮度分量和目标替换的红外图像应用NSCT分解,对所得到的高频系数采用像素点"绝对值和取大"、低频系数采用基于"亮度重映射技术"的加权融合规则进行融合;通过对融合系数进行NSCT逆变换获得融合图像的亮度分量,最后运用颜色空间逆变换得到融合图像。实验结果表明,所提出的融合方法既能保持可见光图像的高分辨率和自然色彩,又能准确保留红外图像中检测出的目标信息,获得视觉效果较好、综合指标较优的融合图像。     

基于非采样Contourlet变换的遥感图像融合算法

为了使融合后的多光谱图像在尽可能保持原始多光谱图像光谱特性的同时,显著提高空间分辨力,提出了一种基于非采样Contourlet变换(NSCT)的遥感图像融合算法。算法首先对全色波段图像进行非采样Contourlet变换,得到全色波段图像的低频子带系数和各带通方向子带系数;然后针对多光谱图像的每一个波段,将其进行双线性插值后作为融合后多光谱图像的低频子带系数,对全色波段图像的各带通方向子带系数采用基于成像系统物理特性的注入模型(调整系数)进行局部调整后,作为融合后多光谱图像的各带通方向子带系数,从而得到融合后多光谱图像的非采样Contourlet变换系数;最后再经非采样Contourlet逆变换得到该波段具有高空间分辨力的多光谱图像。采用IKONOS卫星遥感图像进行了仿真实验,实验结果表明,该算法在光谱保留和空间质量提高方面优于其它传统的遥感图像融合算法。     

Multiscale top-hat selection transform based infrared and visual image fusion with emphasis on extracting regions of interest

To effectively combine regions of interest in original infrared and visual images, an adaptively weighted infrared and visual image fusion algorithm is developed based on the multiscale top-hat selection transform. First, the multiscale top-hat selection transform using multiscale structuring elements with increasing sizes is discussed. Second, the image regions of the original infrared and visual images at each scale are extracted by using the multiscale top-hat selection transform. Third, the final fusion regions are constructed from the extracted multiscale image regions. Finally, the final fusion regions are combined into a base image calculated from the original images to form the final fusion result. The combination of the final fusion regions uses the adaptive weight strategy, and the weights are adaptively obtained based on the importance of the extracted features. In the paper, we compare seven image fusion methods: wavelet pyramid algorithm (WP), shift invariant discrete wavelet transform algorithm (SIDWT), Laplacian pyramid algorithm (LP), morphological pyramid algorithm (MP), multiscale morphology based algorithm (MSM), center-surround top-hat transform based algorithm (CSTHT), and the proposed multiscale top-hat selection transform based algorithm. These seven methods are compared over five different publicly available image sets using three metrics of spatial frequency, mean gradient, and Q. The results show that the proposed algorithm is effective and may be useful for the applications related to the infrared and visual image fusion.     

一种基于非下采样Contourlet变换多聚焦图像融合算法

针对现有小波类图像融合算法的不足,提出了一种基于非下采样Contourlet变换多聚焦图像融合算法,并在Contourlet域中引入了局部区域可见度以及局部方向能量的概念.针对低频子带系数和各带通方向子带系数分别提出了基于局部区域可见度以及基于局部方向能量的系数选择方案.通过对多聚焦图像融合的仿真实验,表明该算法相对于传统的基于离散小波变换和离散小波框架变换融合算法能够有效减少有用信息的丢失以及虚假信息的引入,同时能够从源图像中提取更多的有用信息并注入到融合图像中, 得到更好视觉效果和更优量化指标的融合图像.     

一种基于方向窗特性的Contourlet域的多聚焦图像融合算法

针对同一场景多聚焦图像的融合问题,提出了一种基于方向区域特性的Contourlet域多聚焦图像融合算法.该算法对图像进行Contourlet变换,分解为不同尺度、不同方向的高低频子带|低频和高频子带分别采用方向区域的方差匹配度和能量作为融合规则|最后通过反变换得到融合图像.结果表明,所提出的方向区域方法能够更好地体现二维图像中的曲线或直线状边缘特征,是一种有效可行的图像融合算法.     

非下采样变换的红外与可见光图像融合

基于非下采样Contourlet变换（NSCT）,提出了一种红外和可见光图像融合算法。针对低频子带系数和各带通方向子带系数分别提出了基于图像物理特征的系数加权选择方式与基于区域能量匹配的系数选择方式,即低频基于区域梯度信息、高频基于区域特征因子的加权与选择结合的图像融合算法。实验结果表明：非下采样Contourlet变换具有较快的运算速度,且经非下采样变换后能量更加集中,可提供更多的图像信息。相对于基于像素的图像融合算法,本文的图像融合算法具有更高的融合性能,是一种更适合图像融合的多尺度几何分析（MGA）工具。     

Region-based fusion of infrared and visible images using nonsubsampled contourlet transform

With the nonsubsampled contourlet transform (NSCT), a novel region-segmentation-based fusion algorithm for infrared (IR) and visible images is presented.The IR image is segmented according to the physical features of the target.The source images are decomposed by the NSCT, and then, different fusion rules for the target regions and the background regions are employed to merge the NSCT coefficients respectively.Finally, the fused image is obtained by applying the inverse NSCT.Experimental results show that the proposed algorithm outperforms the pixel-based methods, including the traditional wavelet-based method and NSCT-based method.     

Multifocus image fusion scheme based on features of multiscale products and PCNN in lifting stationary wavelet domain

Multifocus image fusion aims at overcoming imaging cameras's finite depth of field by combining information from multiple images with the same scene. For the fusion problem of the multifocus image of the same scene, a novel algorithm is proposed based on multiscale products of the lifting stationary wavelet transform (LSWT) and the improved pulse coupled neural network (PCNN), where the linking strength of each neuron can be chosen adaptively. In order to select the coefficients of the fused image properly with the source multifocus images in a noisy environment, the selection principles of the low frequency subband coefficients and bandpass subband coefficients are discussed, respectively. For choosing the low frequency subband coefficients, a new sum modified-Laplacian (NSML) of the low frequency subband, which can effectively represent the salient features and sharp boundaries of the image in the LSWT domain, is an input to motivate the PCNN neurons; when choosing the high frequency subband coefficients, a novel local neighborhood sum of Laplacian of multiscale products is developed and taken as one type of feature of high frequency to motivate the PCNN neurons. The coefficients in the LSWT domain with large firing times are selected as coefficients of the fused image. Experimental results demonstrate that the proposed fusion approach outperforms the traditional discrete wavelet transform (DWT)-based, LSWT-based and LSWT-PCNN-based image fusion methods even though the source image is in a noisy environment in terms of both visual quality and objective evaluation.     

Technique for gray-scale visual light and infrared image fusion based on non-subsampled shearlet transform

A novel image fusion technique based on NSST (non-subsampled shearlet transform) is presented, aiming at resolving the fusion problem of spatially gray-scale visual light and infrared images. NSST, as a new member of MGA (multi-scale geometric analysis) tools, possesses not only flexible direction features and optimal shift-invariance, but much better fusion performance and lower computational costs compared with several current popular MGA tools such as NSCT (non-subsampled contourlet transform). We specifically propose new rules for the fusion of low and high frequency sub-band coefficients of source images in the second step of the NSST-based image fusion algorithm. First, the source images are decomposed into different scales and directions using NSST. Then, the model of region average energy (RAE) is proposed and adopted to fuse the low frequency sub-band coefficients of the gray-scale visual light and infrared images. Third, the model of local directional contrast (LDC) is given and utilized to fuse the corresponding high frequency sub-band coefficients. Finally, the final fused image is obtained by using inverse NSST to all fused sub-images. In order to verify the effectiveness of the proposed technique, several current popular ones are compared over three different publicly available image sets using four evaluation metrics, and the experimental results demonstrate that the proposed technique performs better in both subjective and objective qualities.     

高分辨率遥感影像融合处理技术的对比分析研究

随着遥感技术的发展,获取遥感数据的手段越来越丰富。由各种不同的传感器获取的影像数据与日俱增,在同一地区形成了多时相、多分辨率的影像序列。如何综合各种类型的遥感影像信息,提高遥感数据的利用效益已成为遥感应用的瓶颈问题。多源遥感数据融合技术是解决这一问题的有效手段。高空间分辨率影像数据的多样性和复杂性对遥感信息融合处理技术提出了新的更高的要求。以IKONOS卫星数据为例对其进行了空间分辨率的影像融合。研究中引入了多种融合方法,如IHS变换、主成分分析、小波变换以及基于区域特征的自适应小波包算法。从光谱质量和空间信息的角度出发对融合方法进行了比较研究,分析出了比较适合于IKONOS卫星的高分辨率影像融合的处理方法。     

融合Retinex和离散小波奇异值分解的远距离目标图像清晰化

针对远距离成像系统获取的低照度降质图像增强问题,提出了一种融合Retinex和离散小波奇异值分解的图像清晰化算法。该方法首先利用自适应全尺度Retinex（adaptive full-scale retinex, AFSR）“粗”提取照度分量和反射分量,然后通过离散小波变换将所提取的图像反射分量分解为4个频率子带并估计出低频子带图像的奇异值矩阵,最后应用逆小波变换“精”重建图像。实验结果表明:所提方法处理后的低照度降质图像视觉增强效果较好,在图像对比度、信息熵、平均梯度和边缘密度等客观评价指标方面优于其他经典算法。     

基于数据同化和差分进化算法的图像融合

针对现有融合方法不易根据后续处理目的对融合规则进行自适应调整,不同方法的优点不易综合的问题,提出一个基于数据同化和差分进化算法的图像融合框架.在该框架下,将基于非采样下的Contourlet变换作为模型算子,离散小波变换作为观测算子,可根据后续处理对图像各个属性指标值的依赖程度确定各个属性指标的权重,构造由图像各个属性评价指标的加权和所组成的目标函数,再利用差分进化算法来优化目标函数,从而获取更合适的图像.二组实验从视觉效果和量化指标（标准方差、平均梯度、熵、空间频率及均方根交叉熵）两方面验证了该框架的有效性.     

Fusion method of infrared and visible images based on neighborhood characteristic and regionalization in NSCT domain

On fusing infrared and visible image, the traditional fusion method cannot get the better image quality. Based on neighborhood characteristic and regionalization in NSCT (Nonsubsampled Contourlet Transform) domain, the fusion algorithm was proposed. Firstly, NSCT was adopted to decompose infrared and visible images at different scales and directions for the low and high frequency coefficients, the low frequency coefficients which were fused with improving regional weighted fusion method based on neighborhood energy, and the high-frequency coefficients were fused with multi-judgment rule based on neighborhood characteristic regional process. Finally, the coefficients were reconstructed to obtain the fused image. The experimental results show that, compared with the other three related methods, the proposed method can get the biggest value of IE (information entropy), MI(VI,F) (mutual information from visible image), MI(VI,F) (mutual information from infrared image), MI (sum of mutual information), and Q^AB/F (edge retention). The proposed method can leave enough information in the original images and its details, and the fused images have better visual effects.     

基于离散平稳小波变换的红外图像去噪

提出了一种基于离散平稳小波变换的红外图像去噪方法。在预先不知道噪声方差的前提下,只利用红外图像的输入数据就可以确定所要求的渐近最优阈值。对红外图像进行离散平稳小波变换后,分别对各个分解层的高频子带利用所提出的方法进行迭代去噪,使各个高频子带分别收敛于其最大信噪比。实验结果表明,所提出的方法在有效的去除红外图像噪声的同时,又能较好的保持红外图像的细节部分信息。算法在性能指标和视觉质量上均优于基于离散正交小波变换的阈值去噪方法和传统的中值滤波法。     

多光谱与高分辨率图像融合算法研究

提出了一种多光谱与高分辨率图像融合的新算法.该方法将小波的多分辨分解与IHS变换相结合,它首先利用高分辨率图像各小波面叠加的边缘信息进行区域划分,再通过融合算子实现分区域融合,最后对融合后的强度分量进行IHS反变换,得到一幅同时具有较好空间分辨率和光谱信息的融合图像.实验结果表明,该方法得到的融合图像优于IHS变换法和小波变换法(WT法).