Multi-focus image fusion with Geometrical Sparse Representation

Multi-focus image fusion aims to generate an image with all objects in focus by integrating multiple partially focused images. It is challenging to find an effective focus measure to evaluate the clarity of source images. In this paper, a novel multi-focus image fusion algorithm based on Geometrical Sparse Representation (GSR) over single images is proposed. The main novelty of this work is that it shows the potential of GSR coefficients used for image fusion. Unlike the traditional sparse representation-based (SR) methods, the proposed algorithm does not need to train an overcomplete dictionary and vectorize the signal. In our algorithm, using a single dictionary image, the source images are first represented by geometrical sparse coefficients. Specifically, we employ a weighted GSR model in the sparse coding phase, ensuring the importance of the center pixel. Then, the weighted GSR coefficient is used to measure the activity level of the source image and an average pooling strategy is applied to obtain an initial decision map. Third, the decision map is refined with a simple post-processing. Finally, the fused all-in-focus image is constructed with the refined decision map. Experimental results demonstrate that the proposed method can be competitive with or even superior to the state-of-the-art fusion methods in both subjective and objective comparisons.     

Texture analysis-based multi-focus image fusion using a modified Pulse-Coupled Neural Network (PCNN)

Multi-focus image fusion is an effective method of information fusion that can take a series of source images and obtain a fused image where everything is in focus. In this paper, a multi-focus image fusion method based on image texture that adopts a modified Pulse-Coupled Neural Network (PCNN) approach is proposed. First, the texture of an image is obtained by means of image cartoon and texture decomposition. An ignition image is then acquired by inputting the image textures into a modified PCNN. Ignition images are compared to each other to obtain an initial decision map. A small object detection and bilateral filter is then applied to the initial decision map to reduce noise and enable smoother processing. Finally, the source images and decision map are used to produce the fused image. Experimental results demonstrate that the proposed method effectively preserves the source images information while delivering good image fusion performance.     

Multi-focus image fusion with the all convolutional neural network

A decision map contains complete and clear information about the image to be fused, which is crucial to various image fusion issues, especially multi-focus image fusion. However, in order to get a satisfactory image fusion effect, getting a decision map is very necessary and usually difficult to finish. In this letter, we address this problem with convolutional neural network (CNN), aiming to get a state-of-the-art decision map. The main idea is that the max-pooling of CNN is replaced by a convolution layer, the residuals are propagated backwards by gradient descent, and the training parameters of the individual layers of the CNN are updated layer by layer. Based on this, we propose a new all CNN (ACNN)-based multi-focus image fusion method in spatial domain. We demonstrate that the decision map obtained from the ACNN is reliable and can lead to high-quality fusion results. Experimental results clearly validate that the proposed algorithm can obtain state-of-the-art fusion performance in terms of both qualitative and quantitative evaluations.     

自注意力引导的红外与可见光图像融合算法

为确保源图像中的显著区域在融合图像保持显著,提出了一种自注意力引导的红外与可见光图像融合方法。在特征学习层引入自注意力学习机制获取源图像的特征图和自注意力图,利用自注意力图可以捕获到图像中长距离依赖的特性,设计平均加权融合策略对源图像的特征图进行融合,最后将融合后的特征图进行重构获得融合图像。通过生成对抗网络实现了图像特征编码、自注意力学习、融合规则和融合特征解码的学习。TNO真实数据上的实验表明,学习到注意力单元体现了图像中显著的区域,能够较好地引导融合规则的生成,提出的算法在客观和主观评价上优于当前主流红外与可见光图像融合算法,较好地保留了可见光图像的细节信息和红外图像的红外目标信息。     

基于结构与分解的红外光强与偏振图像融合

在一些特定环境下,红外传感器无法探测到目标时,需要将偏振技术与红外技术相融合。为了获得更清楚的融合图像,采用一种基于多尺度结构分解的图像融合方法实现红外光强与偏振图像融合。该算法提出将红外图像与偏振图分解成3个独立部分：平均强度、信号强度和信号结构。其中平均强度部分,采用一种反正切的权重函数进行融合,信号强度采用最大值的融合原则,而信号结构采用一种基于信号强度幂函数的加权平均方进行融合,最后重构得到融合图像。为了更快进行融合、降低计算的复杂度,将分解过程通过均值滤波代替,再通过上采样与下采样得到最终的融合图像。为了得到更好的融合图像,通过不同融合参数实验对比,选择较优的融合参数。最后实验表明使用所提出的反正切权重函数与融合参数设置,在与传统的多尺度算法的比较中,4项评价指标取得优势,且主观上保留更多的纹理细节、提升对比度以及抑制伪影。     

基于ResNet的红外与低照度可见光图像融合方法

针对传统红外与低照度可见光图像融合后,容易造成目标模糊不清、细节信息缺失等问题,本文提出一种低照度可见光图像预增强与残差网络(Residual Network,ResNet)相结合的图像融合方法。该方法首先利用单尺度Retinex(Single Scale Retinex,SSR)算法对低照度可见光图像进行增强预处理,得到增强的可见光图像。其次,利用ResNet-50分别从增强后的可见光图像和红外图像中提取深度特征。然后,采用L1范数对生成的深度特征进行正则化处理,并通过上采样操作将其分辨率恢复至输入图像大小,得到权重图。最后,使用加权平均策略获取融合图像。实验结果表明,本文算法能更好地保留输入图像的纹理细节和结构信息;使用TNO数据集与现有的三种典型算法对比,该算法融合结果的离散余弦特征互信息(FMIdct)、小波特征互信息(FMIw)、基于噪声评估的融合性能(Nabf)、结构相似度测量(SSIM)四种客观指标总体优于对比算法。     

Gradient-based multiresolution image fusion

A novel approach to multiresolution signal-level image fusion is presented for accurately transferring visual information from any number of input image signals, into a single fused image without loss of information or the introduction of distortion. The proposed system uses a "fuse-then-decompose" technique realized through a novel, fusion/decomposition system architecture. In particular, information fusion is performed on a multiresolution gradient map representation domain of image signal information. At each resolution, input images are represented as gradient maps and combined to produce new, fused gradient maps. Fused gradient map signals are processed, using gradient filters derived from high-pass quadrature mirror filters to yield a fused multiresolution pyramid representation. The fused output image is obtained by applying, on the fused pyramid, a reconstruction process that is analogous to that of conventional discrete wavelet transform. This new gradient fusion significantly reduces the amount of distortion artefacts and the loss of contrast information usually observed in fused images obtained from conventional multiresolution fusion schemes. This is because fusion in the gradient map domain significantly improves the reliability of the feature selection and information fusion processes. Fusion performance is evaluated through informal visual inspection and subjective psychometric preference tests, as well as objective fusion performance measurements. Results clearly demonstrate the superiority of this new approach when compared to conventional fusion systems.     

基于改进小波变换的医学图像融合算法研究

本文主要针对图像经过小波变换后边角上看不清楚和传统的边缘检测算法对噪声敏感的问题,结合医学图像的特点,提出一种小波变换的修正算法。首先采用改进的多结构元多尺度形态学梯度的边缘检测进行图像的预处理,其次对CT图像和MRI图像分别进行两种不同小波基的三层小波分解;接着求其对应分量系数的差值图像,最后按着一定加权融合系数对差值图像进行融合得到最终的融合图像。实验结果和评价参数表明,这种改进的医学图像融合算法相比于传统的多小波融合算法不仅强化了融合图像的边缘和纹理特征,提高了分辨率,而且有效地保留了原图像的信息。     

基于PUCS与DTCWT的红外与弱可见光图像融合

针对红外与弱可见光图像传统融合算法在结果图像中目标不突出、整体对比度降低、边缘及纹理细节不清晰、缺失等问题,本文提出一种基于感知一致性空间（Perception Unified Color Space,PUCS）和双树复小波变换（Dual Tree Complex Wavelet Transform,DTCWT）的融合算法。首先,将红外与弱可见光图像的亮度分量由RGB空间分别转至感知一致性空间得到新的亮度分量以备后续变换处理;接着,将源图像利用DTCWT进行多尺度分解,分别获取各自的低频分量与高频分量;然后,根据不同频带系数特点,提出一种基于区域能量自适应加权的规则对低频子带分量进行融合,采用一种基于拉普拉斯能量和与梯度值向量的规则对不同尺度、方向下高频子带分量进行融合;最后,对融合后的高、低频子带分量进行DTCWT逆变换重构图像,再将其转回至RGB空间以得到最终结果。在不同场景下将本文算法与3种高效融合算法进行对比评价,实验结果表明,本文算法不但在主观视觉上具有显著的目标特征、清晰的背景纹理及边缘细节、整体对比度适宜,而且在8项客观评价指标上也取得了较好的效果。     

基于超像素级卷积神经网络的多聚焦图像融合算法

该文提出了基于超像素级卷积神经网络(sp-CNN)的多聚焦图像融合算法。该方法首先对源图像进行多尺度超像素分割,将获取的超像素输入sp-CNN,并对输出的初始分类映射图进行连通域操作得到初始决策图;然后根据多幅初始决策图的异同获得不确定区域,并利用空间频率对其再分类,得到阶段决策图;最后利用形态学对阶段决策图进行后处理,并根据所得的最终决策图融合图像。该文算法直接利用超像素分割块进行图像融合,其相较以往利用重叠块的融合算法可达到降低时间复杂度的目的,同时可获得较好的融合效果。     

基于超像素级卷积神经网络的多聚焦图像融合算法

该文提出了基于超像素级卷积神经网络(sp-CNN)的多聚焦图像融合算法。该方法首先对源图像进行多尺度超像素分割,将获取的超像素输入sp-CNN,并对输出的初始分类映射图进行连通域操作得到初始决策图;然后根据多幅初始决策图的异同获得不确定区域,并利用空间频率对其再分类,得到阶段决策图;最后利用形态学对阶段决策图进行后处理,并根据所得的最终决策图融合图像。该文算法直接利用超像素分割块进行图像融合,其相较以往利用重叠块的融合算法可达到降低时间复杂度的目的,同时可获得较好的融合效果。     

一种改进的基于区域梯度——能量的图像融合方法

针对区域梯度法、区域能量法重构图像峰值信噪比较低、均方根误差较高的现象,提出了一种改进的基于区域梯度-能量的压缩感知图像融合方法。该方法首先构造正交小波变换矩阵,并使用小波变换使图像稀疏化,然后采用哈达玛矩阵作为测量矩阵对稀疏信号进行测量得到测量值,分别计算测量值对应的梯度值及能量值,依据绝对值取大法及加权平均法对测量值进行融合,最后对融合后的测量值进行图像重构。实验结果表明,该方法比单独使用区域梯度法或区域能量法具有较好的图像融合效果。     

基于深度学习的岩石薄片正交偏光图像多曝光融合算法

针对正交偏光下岩石薄片图像中动态范围较低,无法观察全部颗粒的问题,将深度学习应用于多曝光图像融合算法,通过融合多个曝光度的薄片图像来获取较高的动态范围。首先将低动态范围的多曝光图像序列输入卷积神经网络,然后网络通过优化损失函数获取较好的权重图预测结果,最后由源多曝光图像序列联合权重图加权融合得到具有较高动态范围的薄片图像。对比实验表明,该算法可以有效地提升图像中的颗粒清晰度。     

基于区域间相似度的红外与可见光图像融合算法研究

针对传统的红外图像与可见光图像融合算法存在局部模糊、背景信息不完整的问题,文章提出了一种新的融合算法。使用边缘检测算子实现图像轮廓的提取,同时还进行基于能量的加权融合处理;使用区域间相似度的方法实现信号域的提取,最后根据过信号强度进行图像的融合。为了验证算法的正确性,文章进行了对比测试,同时还使用标准差、信息熵和平均梯度3个参数进行了定量分析,本文方法和传统的加权平均算法相比标准差最大提高106.3%,测试结果表明,本文提出的融合方法融合效果更好,具有一定的实用价值。     

基于能量、梯度与方差的多聚焦图像融合

大多数图像融合算法只就图像的某一个特征进行融合,容易造成其他特征信息损失。针对此问题,提出了一种利用哈尔小波变换的特性,考虑多种区域特征进行融合的策略,将图像进行哈尔小波变换后,根据图像的低频部分集中图像大部份能量的特征,采用梯度和能量相结合,根据图像高频部分反映图像细节的特征,采用区域方差与变换系数相结合的方法进行融合,最后经哈尔小波逆变换得到融合结果。通过对多组多聚焦图像进行融合实验,采用均值、方差、熵和平均梯度4种客观评价指标来评价融合图像效果,结果表明该方法能很好地保留图像信息,融合效果好。     

基于双分解的双通道PCNN红外与可见光图像融合

为解决红外与可见光图像融合过程中存在的对比与清晰度较低和小目标易丢失等问题,提出了基于在双分解模型下的双通道PCNN(dPCNN)图像融合算法。首先对两幅源图像进行预增强处理,通过鲁棒的主成分分析(RPCA)将处理后图像分解为稀疏层与低秩层,接着,再利用非下采用剪切波变换(NSST)对的稀疏层进行多尺度分解得到低频子带与高频子带,然后对低秩层和低频子带采用局部加权能量与拉普拉斯能量两者取大的规则进行融合,对高频子带则利用dPCNN的点火图进行融合,最后将得到的融合成分进行逆变换或合成来得到最终融合图像。实验表明,该算法的融合图像目标信息对比突出、小目标信息明显,对源图像信息保留较好,客观评价指标也明显也优于其他算法,其中互信息有了大幅度的提升,有效地提升了红外与可见光图像的融合效果。     

FFTI: Image inpainting algorithm via features fusion and two-steps inpainting

In view of the faultiness that the existing image inpainting methods fail to make full use of the complete region to predict the missing region features when the object features are seriously missing, resulting in discontinuous features and fuzzy detail texture of the inpainting results, a fine inpainting method of incomplete image based on features fusion and two-steps inpainting (FFTI) is proposed in this paper. Firstly, the dynamic memory networks (DMN+) are used to fuse the external features and internal features of the incomplete image to generate the incomplete image optimization map. Secondly, a generation countermeasure generative network with gradient penalty constraints is constructed to guide the generator to rough repair the optimized incomplete image and obtain the rough repair map of the target to be repaired. Finally, the coarse repair graph is further optimized by the idea of coherence of relevant features to obtain the final fine repair graph. It is verified by simulation on three image data sets with different complexity, and compared with the existing dominant repair model for visual effect and objective data. The experimental results show that the results of the model repair in this paper are more reasonable in texture structure, better than other models in visual effect and objective data, and the Peak Signal-to-Noise Ratio of the proposed algorithm in the most challenging underwater targe dataset is 27.01, the highest Structural Similarity Index is 0.949.     

shearlet变换和区域特性相结合的图像融合

为了提高多模医学图像或多聚焦图像的融合性能,结合shearlet变换能够捕捉图像细节信息的性质,提出了一种基于shearlet变换的图像融合算法。首先,用shearlet变换将已精确配准的两幅原始图像分解,得到低频子带系数和不同尺度不同方向的高频子带系数。低频子带系数使用改进的加权融合算法,用平均梯度来计算加权参量,以此来改善融合图像轮廓模糊度高的问题,高频子带系数采用区域方差和区域能量相结合的融合规则,以得到丰富的细节信息。最后,进行shearlet逆变换得到融合图像。结果表明,此算法在主观视觉效果和客观评价指标上优于其它融合算法。     

基于VGG网络的双波段图像融合方法

针对红外与可见光图像中物体信息具有各自优点的情况,提出了基于VGG网络的红外与可见光图像融合方法来提高对夜间或复杂背景情况下的物体检测识别能力。首先将图像分别输入到一个经过训练得到的VGG网络中,经过不同的卷积层提取各自的特征图;然后将特征图经过ZCA白化处理,去除冗余信息;再通过归一化处理,将特征图的维度降到二维,并通过双三次插值法将其缩放到与源图像尺寸一致;最后通过加权取平均得到融合后的图像。实验结果表明,本文的方法在第四和第五层卷积得到的融合结果优于前三层的融合结果。同时,本文融合方法与其他3种融合方法相比视觉效果较好,在标准差、平均梯度、相关系数、熵值等评价指标上分别平均提升了12.79 %、11.04 %、9.94 %和2.54 %,并且在融合时间上保持在1秒以内。这说明该方法融合效果较好,速度较快,能够较多地保留红外与可见光图像信息和较好地提升目标的显著性。     

基于不可分加性小波与形态学梯度的图像边缘提取方法

针对传统的图像边缘提取方法只强调图像中的水平和垂直边缘的不足,提出了一种基于不可分加性小波和形态学梯度相结合的图像边缘提取方法。根据二维不可分小波理论构造了低通滤波器,利用它对原图像进行加性小波多尺度分解;对低频子图像求形态学梯度,对增强后的高频子图像取模极大值;将所得梯度图与边缘图作加性小波逆变换,得重构后的边缘梯度图;并利用二值形态学方法对其进行处理,得最终结果边缘图。实验结果表明,本文算法可获得较好的边缘图像,与经典的边缘提取方法相比,该方法具有完整性、多方向性、平移不变性和快速性的特点。