| 卷积神经网络结合NSST的红外与可见光图像融合 |
| |
| 引用本文： | 宦克为,李向阳,曹宇彤,陈笑.卷积神经网络结合NSST的红外与可见光图像融合[J].红外与激光工程,2022,51(3):20210139-1-20210139-8. |
| |
| 作者姓名： | 宦克为 李向阳 曹宇彤 陈笑 |
| |
| 作者单位： | 长春理工大学 物理学院，吉林 长春 130022 |
| |
| 基金项目： | 吉林省科技发展计划（20210101158JC） |
| |
| 摘 要： | 传统的多尺度红外与可见光图像融合方法，所提取的图像特征固定，并不能很好的应用于各类复杂的图像环境，而深度学习可以自主选择合适图像特征，改良特征提取单一性问题，因此提出一种基于卷积神经网络与非下采样剪切波变换（NSST）相结合的红外与可见光图像融合方法。首先，用卷积神经网络提取红外目标与背景的二分类图，利用调频（FT）显著性检测算法对分类图进行精准分割，同时，利用NSST将源图像多尺度、多方向进行分解；其次，利用目标显著性结合自适应模糊逻辑算法进行低频子带融合，利用高频系数局部方差对比度方法进行高频子带融合；最后，通过NSST逆变换得到融合后图像。实验结果表明:相比于传统图像融合算法，该方法在信息熵、平均梯度、空间频率、互信息和交叉熵等多个客观评价指标上至少分别提高了0.01%、0.30%、1.43%、2.32%、1.14%。一定程度提高了融合图像对比度，丰富了背景细节信息，更有利于人眼识别，可以广泛的应用于光电侦察、光电告警、多传感器信息融合等光电信息领域。
|
| 关 键 词： | 图像融合 卷积神经网络 显著性提取 非下采样剪切波变换 模糊逻辑 |
| 收稿时间： | 2021-11-22 |
Infrared and visible image fusion of convolutional neural network and NSST |
| |
| Affiliation: | College of Physics, Changchun University of Science and Technology, Changchun 130022, China |
| |
| Abstract: | Traditional multi-scale infrared and visible image fusion methods couldnot be well applied to all kinds of complex image environments, because the extracted image features were fixed. However, deep learning could independently select appropriate image features to solve the unicity in feature extraction of multi-scale methods. Therefore, an infrared and visible image fusion method based on the combination of convolutional neural network and non-subsampled shear wave transform (NSST) was proposed. Firstly, the binary classification map of the infrared target and background was extracted by convolutional neural network, and the classification map was accurately segmented by frequency-tuned (FT) saliency detection algorithm. At the same time, the NSST was used to decompose the source image in multiple scales and directions; Secondly, the target saliency combined with adaptive fuzzy logic algorithm was used for the fusion of low frequency sub-bands, and the high frequency coefficient local variance contrast method was used for the fusion of high frequency sub-bands; Finally, the fused image was obtained through the inverse transformation of NSST. The experiment results show that compared with the traditional image fusion algorithm, this method improves objective evaluation indicators such as information entropy, average gradient, spatial frequency, mutual information and cross entropy at least increased by 0.01%, 0.30%, 1.43%, 2.32%, 1.14%, respectively. The contrast of fusion image is greatly improved, and the background details are enriched, which is more conducive to human eye recognition. It can be widely used in electro-optical reconnaissance, electro-optical warning, multi-sensor information fusion and other electro-optical information fields. |
| |
| Keywords: | |
|
| 点击此处可从《红外与激光工程》浏览原始摘要信息 |
|
点击此处可从《红外与激光工程》下载全文 |
|
