A fusion method for infrared–visible image and infrared-polarization image based on multi-scale center-surround top-hat transform

This paper presents a fusion method for infrared–visible image and infrared-polarization image based on multi-scale center-surround top-hat transform which can effectively extract the feature information and detail information of source images. Firstly, the multi-scale bright (dark) feature regions of source images at different scale levels are respectively extracted by multi-scale center-surround top-hat transform. Secondly, the bright (dark) feature regions at different scale levels are refined for eliminating the redundancies by spatial scale. Thirdly, the refined bright (dark) feature regions from different scales are combined into the fused bright (dark) feature regions through adding. Then, a base image is calculated by performing dilation and erosion on the source images with the largest scale outer structure element. Finally, the fusion image is obtained by importing the fused bright and dark features into the base image with a reasonable strategy. Experimental results indicate that the proposed fusion method can obtain state-of-the-art performance in both aspects of objective assessment and subjective visual quality.     

Multiscale top-hat selection transform based infrared and visual image fusion with emphasis on extracting regions of interest

To effectively combine regions of interest in original infrared and visual images, an adaptively weighted infrared and visual image fusion algorithm is developed based on the multiscale top-hat selection transform. First, the multiscale top-hat selection transform using multiscale structuring elements with increasing sizes is discussed. Second, the image regions of the original infrared and visual images at each scale are extracted by using the multiscale top-hat selection transform. Third, the final fusion regions are constructed from the extracted multiscale image regions. Finally, the final fusion regions are combined into a base image calculated from the original images to form the final fusion result. The combination of the final fusion regions uses the adaptive weight strategy, and the weights are adaptively obtained based on the importance of the extracted features. In the paper, we compare seven image fusion methods: wavelet pyramid algorithm (WP), shift invariant discrete wavelet transform algorithm (SIDWT), Laplacian pyramid algorithm (LP), morphological pyramid algorithm (MP), multiscale morphology based algorithm (MSM), center-surround top-hat transform based algorithm (CSTHT), and the proposed multiscale top-hat selection transform based algorithm. These seven methods are compared over five different publicly available image sets using three metrics of spatial frequency, mean gradient, and Q. The results show that the proposed algorithm is effective and may be useful for the applications related to the infrared and visual image fusion.     

基于支持度变换和top-hat分解的双色中波红外图像融合

为了解决用多尺度top-hat分解法融合双色中波红外图像时经常存在对比度提升有限、边缘区域失真较重的问题,提出了基于支持度变换和top-hat分解相结合的融合方法。先用支持度变换法将双色中波图像分解为低频图像和支持度图像序列;再从最后一层低频图像中用多尺度top-hat分解法提取各自的亮信息和暗信息;用灰度值取大法分别融合亮信息和暗信息;通过灰度值归一化和高斯滤波分别增强亮、暗信息融合图像;然后融合两低频图像和亮、暗信息增强图像;将融合图像作为新的低频图像和用灰度值取大法融合得到的支持度融合图像序列进行支持度逆变换,得到最终融合图像。该方法的实验结果同采用单一的支持度变换法融合和多尺度top-hat分解法融合相比,融合图像的对比度提升了11.69%,失真度降低了63.42%,局部粗糙度提高了38.12%。说明提出的从低频图像提取亮暗信息,并经过分别融合、增强,再与低频图像进行融合,能有效破解红外融合图像对比度提升和边缘区域失真度降低之间的矛盾,为提高图像融合质量提供了新方法。     

Image fusion through local feature extraction by using multi-scale top-hat by reconstruction operators

The purpose of image fusion is to combine useful image features of different original images into the final fusion image, which will produce one useful result image for different applications. One of the main difficulties of image fusion is extracting useful image features of different original images. In some cases, useful image features are local image features of the whole image. To efficiently extract local image features and produce an efficient fusion result, an image fusion algorithm based on the extracted local image features by using multi-scale top-hat by reconstruction operators is proposed in this paper. Firstly, multi-scale local feature extraction using multi-scale top-hat by reconstruction operators is discussed. Then, based on the extracted multi-scale local features of different original images, the useful image features for image fusion are constructed. Finally, the constructed useful image features for image fusion are combined into the final fusion image. Experimental results on different types of images show that, the proposed algorithm performs well for image fusion.     

An infrared and visible image fusion method based upon multi-scale and top-hat transforms

The high-frequency components in the traditional multi-scale transform method are approximately sparse, which can represent different information of the details. But in the low-frequency component, the coefficients around the zero value are very few, so we cannot sparsely represent low-frequency image information. The low-frequency component contains the main energy of the image and depicts the profile of the image. Direct fusion of the low-frequency component will not be conducive to obtain highly accurate fusion result. Therefore, this paper presents an infrared and visible image fusion method combining the multi-scale and top-hat transforms. On one hand, the new top-hat-transform can effectively extract the salient features of the low-frequency component. On the other hand, the multi-scale transform can extract highfrequency detailed information in multiple scales and from diverse directions. The combination of the two methods is conducive to the acquisition of more characteristics and more accurate fusion results. Among them, for the low-frequency component, a new type of top-hat transform is used to extract low-frequency features, and then different fusion rules are applied to fuse the low-frequency features and low-frequency background; for high-frequency components, the product of characteristics method is used to integrate the detailed information in high-frequency. Experimental results show that the proposed algorithm can obtain more detailed information and clearer infrared target fusion results than the traditional multiscale transform methods. Compared with the state-of-the-art fusion methods based on sparse representation, the proposed algorithm is simple and efficacious, and the time consumption is significantly reduced.     

An improved fusion algorithm for infrared and visible images based on multi-scale transform

In this paper, an improved fusion algorithm for infrared and visible images based on multi-scale transform is proposed. First of all, Morphology-Hat transform is used for an infrared image and a visible image separately. Then two images were decomposed into high-frequency and low-frequency images by contourlet transform (CT). The fusion strategy of high-frequency images is based on mean gradient and the fusion strategy of low-frequency images is based on Principal Component Analysis (PCA). Finally, the final fused image is obtained by using the inverse contourlet transform (ICT). The experiments and results demonstrate that the proposed method can significantly improve image fusion performance, accomplish notable target information and high contrast and preserve rich details information at the same time.     

Image enhancement using multi scale image features extracted by top-hat transform

To efficiently enhance images, a novel algorithm using multi scale image features extracted by top-hat transform is proposed in this paper. Firstly, the multi scale bright and dim regions are extracted through top-hat transform using structuring elements with the same shape and increasing sizes. Then, two types of multi scale image features, which are the multi scale bright and dim image regions at each scale and the multi scale image details between neighboring scales, are extracted and used to form the final extracted bright and dim image regions. Finally, the image is enhanced through enlarging the contrast between the final extracted bright and dim image features. Experimental results on images from different applications verified that the proposed algorithm could efficiently enhance the contrast and details of image, and produce few noise regions.     

Image enhancement through contrast enlargement using the image regions extracted by multiscale top-hat by reconstruction

An image enhancement algorithm based on multiscale top-hat by reconstruction is proposed in this paper. Firstly, multiscale top-hat by reconstruction using multiscale structuring elements is discussed. Then, multiscale bright and black image regions are extracted. Thirdly, useful image regions for image enhancement are obtained from the extracted multiscale bright and black image regions. Finally, after a base image is calculated from the results of the opening and closing by reconstruction operations, the original image is enhanced through combing the useful image regions into the base image. Experimental results on different types of images show that the proposed algorithm is efficient.     

基于区域检测与非下采样轮廓波变换的红外与彩色可见光图像融合

针对灰度图像融合的分辨率低及现有的彩色图像融合方法融合的图像色彩不自然、不符合人的视觉感受的特点,在此提出一种基于Snake模型的区域检测和非下采样轮廓波变换(NSCT)的红外与彩色可见光图像融合的方法。首先对彩色可见光图像进行亮度、色度和饱和度(IHS)颜色空间变换提取亮度分量,并用Snake模型对红外图像的目标区域进行检测;然后对亮度分量和目标替换的红外图像应用NSCT分解,对所得到的高频系数采用像素点"绝对值和取大"、低频系数采用基于"亮度重映射技术"的加权融合规则进行融合;通过对融合系数进行NSCT逆变换获得融合图像的亮度分量,最后运用颜色空间逆变换得到融合图像。实验结果表明,所提出的融合方法既能保持可见光图像的高分辨率和自然色彩,又能准确保留红外图像中检测出的目标信息,获得视觉效果较好、综合指标较优的融合图像。     

基于兴趣区域掩码卷积神经网络的红外-可见光图像融合与目标识别算法研究

建立权重独立的双通道残差卷积神经网络,对可见光与红外频段下的目标图像进行特征提取,生成多尺度复合频段特征图组。基于像点间的欧式距离计算双频段特征图显著性,根据目标在不同成像频段下的特征贡献值进行自适应融合。通过热源能量池化核与视觉注意力机制,分别生成目标在双频段下的兴趣区域逻辑掩码并叠加在融合图像上,凸显目标特征并抑制非目标区域信息。以端到端识别网络作为基础,利用交叉损失计算策略,对含有注意力掩码的多尺度双频段融合特征图进行目标识别。结果表明,所设计的识别网络能够有效地融合目标红外热源物理特征和可见光图像纹理特征,提高了信息融合深度,保留目标热辐射与纹理特征的同时降低了背景信息干扰,对全天候复杂环境下的多尺度热源目标具有良好的识别精度与鲁棒性。     

Fusion of visible and infrared images using saliency analysis and detail preserving based image decomposition

Image fusion for visible and infrared images is a significant task in image analysis. The target regions in infrared image and abundant detail information in visible image should be both extracted into the fused result. Thus, one should preserve or even enhance the details from original images in fusion process. In this paper, an algorithm using pixel value based saliency detection and detail preserving based image decomposition is proposed. Firstly, the multi-scale decomposition is constructed using weighted least squares filter for original infrared and visible images. Secondly, the pixel value based saliency map is designed and utilized for image fusion in different decomposition level. Finally, the fusion result is reconstructed by synthesizing different scales with synthetic weights. Since the information of original signals can be well preserved and enhanced with saliency extraction and multi scale decomposition process, the fusion algorithm performs robustly and excellently. The proposed approach is compared with other state-of the-art methods on several image sets to verify the effectiveness and robustness.     

基于变换域PCNN的近红外与彩色可见光融合

针对近红外与彩色可见光图像融合后对比度低、细节丢失和颜色失真等问题,提出一种基于多尺度变换和自适应脉冲耦合神经网络(PCNN-pulse coupled neural network,PCNN)的红外与彩色可见光图像融合的新算法。首先将彩色可见光图像转换到HSI（hue saturation intensity)空间,HSI色彩空间包含亮度、色度和饱和度三个分量,并且这三个分量互不相关,因此利用这个特点可对三个分量分别进行处理。将其亮度分量与近红外图像分别进行多尺度变换,变换方法选择Tetrolet变换。变换后分别得到低频和高频分量,针对图像低频分量,提出一种期望最大的低频分量融合规则;针对图像高频分量,采用高斯差分算子调节PCNN模型的阈值,提出一种自适应的PCNN模型作为融合规则。处理后的高低频分量经过Tetrolet逆变换得到的融合图像作为新的亮度图像。然后将新的亮度图像和原始的色度和饱和度分量反向映射到RGB空间,得到融合后的彩色图像。为了解决融合带来的图像平滑化和原始图像光照不均的问题,引入颜色与锐度校正机制(colour and sharpness correction, CSC)来提高融合图像的质量。为了验证方法的有效性,选取了5组分辨率为1 024×680近红外与彩色可见光图像进行试验,并与当前高效的四种融合方法以及未进行颜色校正的本方法进行了对比。实验结果表明,同其他图像融合算法进行对比分析,该方法在有无CSC颜色的情况下均能保留最多的细节和纹理,可见度均大大提高,同时本方法的结果在光照条件较弱的情况下具有更多的细节和纹理,均具有更好的对比度和良好的色彩再现性。在信息保留度、颜色恢复、图像对比度和结构相似性等客观指标上均具有较大优势。     

A fusion algorithm for infrared and visible images based on RDU-PCNN and ICA-bases in NSST domain

The aim of infrared and visible image fusion is to enhance the feature in infrared image and preserve abundant detail information in visible image. Based on the fact that the human sense system accepts external stimulation only when the stimulus intensity is greater than a certain value and the reaction of neuronal cells have obvious regional characters, an image fusion algorithm based on region dual-channel unit-linking pulse coupled neural networks (RDU-PCNN) and independent component analysis (ICA) bases in non-subsampled shearlet transform (NSST) domain for infrared and visible images is proposed. RDU-PCNN we constructed has obvious regional characters and much lower computational costs. We trained ICA-bases using a number of images that the content and statistical properties are similar with the fusion images but applied it as low-frequency ICA-bases, which can reduce calculation complexity. Experimental results demonstrate that the proposed method can significantly improved the fusion quality and need less computational costs.     

基于深度玻尔兹曼模型的红外与可见光图像融合

为了克服红外与可见光图像融合时噪声干扰及易产生伪影导致目标轮廓不鲜明、对比度低的缺点,提出一种基于深度模型分割的图像融合方法.首先,采用深度玻尔兹曼机学习红外与可见光的目标和背景轮廓先验,构建轮廓的深度分割模型,通过Split Bregman迭代算法获取最优能量分割后的红外与可见光图像轮廓;然后再使用非下采样轮廓波变换对源图像进行分解,并针对所分割的背景轮廓采用结构相似度的规则进行系数组合;最后进行非下采样轮廓波反变换重构出融合图像.数值试验证明,该算法可以有效获取目标和背景轮廓均清晰的融合图像,融合结果不但具有较高的对比度,还能抑制噪声影响,具有有效性.     

Top-hat by reconstruction operators based multi-scale multi-structuring element method for multiple linear feature detection with simple post-processing

To efficiently extract all the possible linear features in image, a multi-scale multi-structuring element top-hat by reconstruction operator based algorithm with simple post-processing is proposed in this paper. Multi-scale top-hat by reconstruction operator using multi-scale structuring elements is discussed, firstly. Also, through importing multi-structuring elements with linear shapes at different directions, multi-scale multi-structuring element top-hat by reconstruction operator for linear feature extraction is shown. By using the multi-scales of multi-structuring elements, the method of extracting all the possible linear feature regions in an image is proposed. After extracting the linear feature regions, the final detected linear features, which are expressed as lines with different shapes and lengths, are obtained through image binarisation and refinement. Experimental results on different types of images show that, the proposed algorithm is efficient for linear feature detection and could be widely used in different applications related to multiple linear feature detection.     

一种基于区域特性的红外与可见光图像融合算法

提出了一种基于区域分割和à trous小波变换的红外与可见光图像融合算法.首先,对红外与可见光图像进行区域分割及区域关联,并按关联映射图所划分区域提取红外与可见光图像的的能量信息及梯度信息;然后,对红外与可见光图像进行多尺度à trous小波变换分解,分解后的低频部分按照文中所提出的区域能量比和区域清晰比指标进行区域融合,高频部分采用绝对值取大算子进行融合;最后进行重构得到融合图像.结果表明,该算法既可保持可见光图像的光谱信息,又可有效获取红外图像的热目标信息.     

Real-time visual enhancement for infrared small dim targets in video

Visual enhancement for infrared small dim targets is a standing problem in infrared image processing. Existing approaches cannot enhance the target well and suppress the background simultaneously, especially for targets which are so faint that they are hardly visible. This paper proposes a novel real-time visual enhancement algorithm for infrared small dim targets in video by introducing temporal cues. In this work, Dynamic Programming Algorithm (DPA) is used to detect the target’s trajectory in the video and the target is enhanced through energy accumulation along the trajectory. The shape prior of the small dim target is adopted for background suppression and adaptive merging. Experimental results on real infrared small dim target videos indicate that the proposed algorithm can improve the visual quality of these types of images notably, especially for cases in which the target is hardly visible. In addition, the proposed algorithm takes on average 8.35 ms to process a 320 1 256 image, and thus meets the needs of real-time applications.     

An adaptive fusion approach for infrared and visible images based on NSCT and compressed sensing

A novel nonsubsampled contourlet transform (NSCT) based image fusion approach, implementing an adaptive-Gaussian (AG) fuzzy membership method, compressed sensing (CS) technique, total variation (TV) based gradient descent reconstruction algorithm, is proposed for the fusion computation of infrared and visible images.Compared with wavelet, contourlet, or any other multi-resolution analysis method, NSCT has many evident advantages, such as multi-scale, multi-direction, and translation invariance. As is known, a fuzzy set is characterized by its membership function (MF), while the commonly known Gaussian fuzzy membership degree can be introduced to establish an adaptive control of the fusion processing. The compressed sensing technique can sparsely sample the image information in a certain sampling rate, and the sparse signal can be recovered by solving a convex problem employing gradient descent based iterative algorithm(s).In the proposed fusion process, the pre-enhanced infrared image and the visible image are decomposed into low-frequency subbands and high-frequency subbands, respectively, via the NSCT method as a first step. The low-frequency coefficients are fused using the adaptive regional average energy rule; the highest-frequency coefficients are fused using the maximum absolute selection rule; the other high-frequency coefficients are sparsely sampled, fused using the adaptive-Gaussian regional standard deviation rule, and then recovered by employing the total variation based gradient descent recovery algorithm.Experimental results and human visual perception illustrate the effectiveness and advantages of the proposed fusion approach. The efficiency and robustness are also analyzed and discussed through different evaluation methods, such as the standard deviation, Shannon entropy, root-mean-square error, mutual information and edge-based similarity index.     

基于PHLST的红外与可见光图像融合算法

针对图像融合过程中边缘处理和区域一致性的问题,提出一种基于多重调和局部正弦变换的红外与可见光图像融合新算法.多重调和局部正弦变换的多重调和分量μ代表了图像缓慢变化的"趋势",在空域进行加权融合;残差分量υ体现了源图像的"波动",在傅里叶正弦变换域进行融合,以充分提取可见光图像的细节信息.由于不存在边缘效应,同时残差分量...     

Novel infrared image enhancement technology based on the frequency compensation approach

A novel infrared image enhancement method has been proposed in this paper. Our aim is to develop a detail enhancement method which is focused on the frequency feature of the image. The proposed method is following the most popular strategy of enhancing the infrared images nowadays, but concentrating on the frequency domain. Firstly, the original image is separated by a guided image filter into detail layer and the base layer. Quite unlike the traditional methods, we use the guided image filter to eliminate most of the noise and weak signal of the scenario. Then, by a designed iteration process, the higher frequency of the scenario will be calculated back and add to the detail layer. The noise will not be enhanced because the iteration is only focused on the leftover scenario frequency. We run many tests on the raw data captured by the 320 × 256 HgCdTe cooled thermal imager, and make a comparison between our approach with the previous method of bilateral filtering digital detail enhancement and guided image filtering digital detail enhancement. Figures and analytical data show that our method is better than the previous proposed researches. Our method could effectively process the infrared image with less noise and artifacts, which has potential applications in testing, manufacturing, chemical imaging, night vision, and surveillance security.