基于CNN的双边融合网络在高光谱图像分类中的应用

doi:10.11959/j.issn.1000-436x.2020238

Abstract

Abstract:

Aiming at the issues of decreasing spatial resolution and feature loss caused by pooling operation in depth CNN-based hyperspectral image classification algorithm,a bilateral fusion block network （DFBN）composed of bilateral fusion blocks was designed.The upper structure of bilateral fusion block was constituted by 1×1 convolution and hyperlink,which was used to transfer local spatial characteristics.The lower structure was constituted by pooling layer,convolutional layer,deconvolution layer and upsampling to enhance the characteristics of efficient discrimination.Experimental results on three benchmark hyperspectral image data sets illustrate that the model is superior to other similar classification models.

Key words: convolutional neural network, hyperspectral images classification, transpose-convolution, upsampling,hyperlink

CLC Number:

TP302

Hongmin GAO,Xueying CAO,Yao YANG,Zaijun HUA,Chenming LI. Application of bilateral fusion model based on CNN in hyperspectral image classification[J]. Journal on Communications, 2020, 41(11): 132-140.

Figures/Tables 18

References 32

[1]	LUO B , YANG C H , CHANUSSOT J ,et al. Crop yield estimation based on unsupervised linear unmixing of multidate hyperspectral imagery[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013,51(1): 162-173.
[2]	YUEN P W T , RICHARDSON M . An introduction to hyperspectral imaging and its application for security,surveillance and target acquisition[J]. Imaging Science Journal, 2010,58(5): 241-253.
[3]	SANDIDGE J C , HOLYER R J . Coastal bathymetry from hyperspectral observations of water radiance[J]. Remote Sensing of Environment, 1998,65(3): 341-352.
[4]	ZHANG L F , ZHANG L P , TAO D C ,et al. Hyperspectral remote sensing image subpixel target detection based on supervised metric learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2014,52(8): 4955-4965.
[5]	LI J , BIOUCAS-DIAS J M , PLAZA A . Semisupervised hyperspectral image segmentation using multinomial logistic regression with active learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2010,48(11): 4085-4098.
[6]	SAMANIEGO L , BARDOSSY A , SCHULZ K . Supervised classification of remotely sensed imagery using a modified KNN technique[J]. IEEE Transactions on Geoscience and Remote Sensing, 2008,46(7): 2112-2125.
[7]	MELGANI F , BRUZZONE L . Classification of hyperspectral remote sensing images with support vector machines[J]. IEEE Transactions on Geoscience and Remote Sensing, 2004,42(8): 1778-1790.
[8]	WANG Q , LIN J Z , YUAN Y . Salient band selection for hyperspectral image classification via manifold ranking[J]. IEEE Transactions on Neural Networks and Learning Systems, 2016,27(6): 1279-1289.
[9]	JIA S , TANG G H , ZHU J S ,et al. A novel ranking-based clustering approach for hyperspectral band selection[J]. IEEE Transactions on Geoscience and Remote Sensing, 2016,54(1): 88-102.
[10]	LICCIARDI G , MARPU P R , CHANUSSOT J ,et al. Linear versus nonlinear PCA for the classification of hyperspectral data based on the extended morphological profiles[J]. IEEE Geoscience and Remote Sensing Letters, 2012,9(3): 447-451.
[11]	VILLA A , BENEDIKTSSON J A , CHANUSSOT J ,et al. Hyperspectral image classification with independent component discriminant analysis[J]. IEEE Transactions on Geoscience and Remote Sensing, 2011,49(12): 4865-4876.
[12]	LIU X , ZHANG B , GAO L R ,et al. A maximum noise fraction transform with improved noise estimation for hyperspectral images[J]. Science in China Series F-information Sciences, 2009,52(9): 1578-1587.
[13]	MAKANTASIS K , KARANTZALOS K , DOULAMIS A ,et al. Deep supervised learning for hyperspectral data classification through convolutional neural networks[C]// 2015 IEEE International Geoscience and Remote Sensing Symposium. Piscataway:IEEE Press, 2015: 4959-4962.
[14]	GHAMISI P , PLAZA J , CHEN Y S ,et al. Advanced spectral classifiers for hyperspectral images:a review[J]. IEEE Geoscience and Remote Sensing Magazine, 2017,5(1): 8-32.
[15]	GAO W , PENG Y . Ideal kernel-based multiple kernel learning for spectral-spatial classification of hyperspectral image[J]. IEEE Geoscience and Remote Sensing Letters, 2017,14(7): 1051-1055.
[16]	CHEN Y , NASRABADI N M , TRAN T D . Hyperspectral image classification via kernel sparse representation[J]. IEEE Transactions on Geoscience and Remote Sensing, 2013,51(1): 217-231.
[17]	CHEN P , NELSON J D B , TOURNERET J Y . Toward a sparse Bayesian Markov random field approach to hyperspectral unmixing and classification[J]. IEEE Transactions on Image Processing, 2017,26(1): 426-438.
[18]	LECUN Y , BENGIO Y , HINTON G . Deep learning[J]. Nature, 2015,521(7553): 436-444.
[19]	KRIZHEVSKY A , SUTSKEVER I , HINTON G E . ImageNet classification with deep convolutional neural networks[J]. Communications of the ACM, 2017,60(6): 84-90.
[20]	GIRSHICK R , DONAHUE J , DARRELL T ,et al. Rich feature hierarchies for accurate object detection and semantic segmentation[C]// 2014 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2014: 580-587.
[21]	CHEN Y S , LIN Z H , ZHAO X ,et al. Deep learning-based classification of hyperspectral data[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2014,7(6): 2094-2107.
[22]	LIU P , ZHANG H , EOM K B . Active deep learning for classification of hyperspectral images[J]. IEEE Journal of Selected Topics in Applied Earth Observations and Remote Sensing, 2017,10(2): 712-724.
[23]	GHAMISI P , MAGGIORI E , LI S T ,et al. New frontiers in spectral-spatial hyperspectral image classification the latest advances based on mathematical morphology,Markov random fields,segmentation,sparse representation,and deep learning[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018,56(3): 10-43.
[24]	LI S T , SONG W W , FANG L Y ,et al. Deep learning for hyperspectral image classification:an overview[J]. IEEE Transactions on Geoscience and Remote Sensing, 2019,57(9): 6690-6709.
[25]	ZHONG Z L , LI J , LUO Z M ,et al. Spectral-spatial residual network for hyperspectral image classification:a 3D deep learning framework[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018,56(2): 847-858.
[26]	FENG F , WANG S T , WANG C Y ,et al. Learning deep hierarchical spatial-spectral features for hyperspectral image classification based on residual 3D-2D CNN[J]. Sensors, 2019,19(23):5276.
[27]	HE K M , ZHANG X Y , REN S Q ,et al. Deep residual learning for image recognition[C]// 2016 IEEE Conference on Computer Vision and Pattern Recognition. Piscataway:IEEE Press, 2016: 770-778.
[28]	LI J J , ZHAO X , LI Y S ,et al. Classification of hyperspectral imagery using a new fully convolutional neural network[J]. IEEE Geoscience and Remote Sensing Letters, 2018,15(2): 292-296.
[29]	MOU L C , GHAMISI P , ZHU X X . Unsupervised spectral–spatial feature learning via deep residual conv-deconv network for hyperspectral image classification[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018,56(1): 391-406.
[30]	MA X R , FU A Y , WANG J ,et al. Hyperspectral image classification based on deep deconvolution network with skip architecture[J]. IEEE Transactions on Geoscience and Remote Sensing, 2018,56(8): 4781-4791.
[31]	LEE H , KWON H . Going deeper with contextual CNN for hyperspectral image classification[J]. IEEE Transactions on Image Processing, 2017,26(10): 4843-4855.
[32]	GAO H M , YANG Y , LEI S ,et al. Multi-branch fusion network for hyperspectral image classification[J]. Knowledge-Based Systems, 2019167:11

Metrics

Recommended 0

No Suggested Reading articles found!

层	内核		输出尺寸
层	内核	IP数据集	PU数据集	SA数据集
图像块	—	21像素×21像素,15层	21像素×21像素,15层	33像素×33像素,21层
卷积层1	5×5	21像素×21像素,16层	21像素×21像素,24层	33像素×33像素,16层
最大池化层	3×3	7像素×7像素,16层	7像素×7像素,24层	11像素×11像素,16层
转置卷积和上采样	3×3	21像素×21像素,16层	21像素×21像素,24层	33像素×33像素,16层
结合层	—	21像素×21像素,32层	21像素×21像素,48层	33像素×33像素,32层
卷积层2	5×5	21像素×21像素,64层	21像素×21像素,96层	33像素×33像素,64层
1×1卷积	1×1	21像素×21像素,64层	21像素×21像素,96层	33像素×33像素,64层
融合层	—	21像素×21像素,64层	21像素×21像素,96层	33像素×33像素,64层
全连接1	Relu		128
全连接2	Relu		64
训练参数	—	3 924 928	5 922 401	9 074 528
双边融合块个数	—	4	3	2

超链接	上采样	转置卷积	OA	AA	KA
×	×	×	97.78%	91.19%	97.46%
×	×	√	98.15%	95.53%	97.89%
×	√	×	98.29%	96.61%	98.05%
√	×	×	97.83%	95.46%	97.53%
×	√	√	98.37%	96.82%	98.14%
√	√	×	98.39%	96.58%	98.17%
√	×	√	98.35%	95.33%	98.12%
√	√	√	98.45%	96.12%	98.23%

训练样本占比	KA	OA	AA
1%	59.53%	64.97%	46.41%
3%	84.88%	86.81%	73.92%
5%	94.03%	94.78%	85.37%
7%	96.70%	97.11%	91.97%
10%	98.23%	98.45%	96.12%

训练样本占比	KA	OA	AA
0.5%	86.57%	89.99%	81.53%
1%	95.10%	96.41%	92.82%
2%	98.33%	98.74%	97.74%
3%	99.09%	99.31%	98.56%
5%	99.53%	99.65%	99.14%

训练样本占比	KA	OA	AA
0.1%	80.32%	82.46%	76.41%
0.5%	98.56%	98.71%	98.64%
1%	99.68%	99.71%	99.62%
2%	99.86%	99.87%	99.80%
3%	99.91%	99.92%	99.91%

Application of bilateral fusion model based on CNN in hyperspectral image classification

RichHTML

PDF

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 18

References 32

Related Articles 15

Metrics

Recommended 0

类别	RPCA-CNN模型	FCNN模型	DCNN模型	SSRN模型	DRN模型	DFBN模型
Alfalfa	34.88%	90.12%	43.66%	93.41%	90.37%	96.10%
Corn-notill	60.91%	93.11%	89.91%	97.59%	94.30%	95.54%
Corn-mintill	57.19%	97.03%	93.68%	98.71%	96.14%	99.35%
Corn	54.32%	93.92%	82.86%	91.22%	85.12%	98.99%
Grass-pasture	92.08%	96.34%	97.31%	99.79%	98.31%	98.77%
Grass-trees	97.63%	97.60%	99.09%	99.70%	98.86%	99.06%
Grass-pasture-mowed	79.20%	99.00%	91.20%	98.60%	93.80%	96.20%
Hay-windrowed	99.93%	99.30%	99.91%	99.84%	99.95%	99.97%
Oats	57.22%	82.78%	69.17%	95.28%	78.06%	73.89%
Soybean-notill	76.05%	97.49%	93.69%	99.25%	98.13%	98.63%
Soybean-mintill	63.95%	98.28%	94.14%	96.30%	99.04%	99.51%
Soybean-clean	48.16%	91.83%	91.22%	97.21%	92.36%	96.47%
Wheat	99.30%	96.32%	99.76%	99.78%	98.57%	97.49%
Woods	94.79%	99.78%	98.78%	99.73%	99.73%	99.93%
Buildings-Grass-Trees-Drives	77.59%	93.78%	85.46%	95.26%	96.71%	98.90%
Stone-Steel-Towers	99.82%	87.20%	96.13%	98.75%	89.17%	89.11%

分类模型	KA	OA	AA
RPCA-CNN模型	70.08%	73.65%	74.56%
FCNN模型	96.15%	96.63%	94.62%
DCNN模型	93.05%	93.91%	89.12%
SSRN模型	97.64%	97.93%	97.53%
DRN模型	96.75%	97.15%	94.29%
DFBN模型	98.23%	98.45%	96.12%

分类模型	KA	OA	AA
RPCA-CNN模型	75.39%	80.94%	85.15%
FCNN模型	84.91%	88.74%	78.32%
DCNN模型	92.40%	94.30%	89.42%
SSRN模型	97.88%	98.40%	97.60%
DRN模型	96.30%	97.21%	95.16%
DFBN模型	98.33%	98.74%	97.74%

分类模型	KA	OA	AA
RPCA-CNN模型	61.45%	64.70%	68.48%
FCNN模型	77.87%	80.21%	81.08%
DCNN模型	84.00%	85.60%	86.76%
SSRN模型	88.64%	89.79%	92.66%
DRN模型	94.87%	95.39%	95.15%
DFBN模型	98.56%	98.71%	98.64%

[1]	Yurong LIAO, Haining WANG, Cunbao LIN, Yang LI, Yuqiang FANG, Shuyan NI. Research progress of deep learning-based object detection of optical remote sensing image [J]. Journal on Communications, 2022, 43(5): 190-203.
[2]	Fan ZHANG, Yun HUANG, Zizhuo FANG, Wei GUO. Lost-minimum post-training parameter quantization method for convolutional neural network [J]. Journal on Communications, 2022, 43(4): 114-122.
[3]	Zhengyu ZHU, Gengwang HOU, Chongwen HUANG, Gangcan SUN, Wanming HAO, Jing LIANG. Systems resource allocation algorithm for RIS-assisted D2D secure communication based on parallel CNN [J]. Journal on Communications, 2022, 43(3): 172-179.
[4]	Hongyan WANG, Libin ZHANG, Guoqiang CHEN, Zumin WANG, Zhiyuan GUAN. Approach of target tracking combining particle filter and metric learning [J]. Journal on Communications, 2021, 42(5): 98-110.
[5]	Nianwen SI, Wenlin ZHANG, Dan QU, Heyu CHANG, Shengxiang LI, Tong NIU. Generalized Grad-CAM attacking method based on adversarial patch [J]. Journal on Communications, 2021, 42(3): 23-35.
[6]	Hongmin GAO, Xueying CAO, Zhonghao CHEN, Zaijun HUA, Chenming LI, Yue CHEN. Hyperspectral image classification method based on multi-scale proximal feature concatenate network [J]. Journal on Communications, 2021, 42(2): 92-102.
[7]	Jun YANG,Jisheng DANG. Semantic segmentation of 3D point cloud based on contextual attention CNN [J]. Journal on Communications, 2020, 41(7): 195-203.
[8]	Meng ZHANG,Haoliang SUN,Peng YANG. Identification of DNS covert channel based on improved convolutional neural network [J]. Journal on Communications, 2020, 41(1): 169-179.
[9]	Xin ZHOU,Xiaoxin HE,Changwen ZHENG. Radio signal recognition based on image deep learning [J]. Journal on Communications, 2019, 40(7): 114-125.
[10]	Jia LI,Xiaochun YUN,Shuhao LI,Yongzheng ZHANG,Jiang XIE,Fang FANG. HTTP malicious traffic detection method based on hybrid structure deep neural network [J]. Journal on Communications, 2019, 40(1): 24-33.
[11]	Linhui LI,Bo QIAN,Jing LIAN,Weina ZHENG,Yafu ZHOU. Study on traffic scene semantic segmentation method based on convolutional neural network [J]. Journal on Communications, 2018, 39(4): 123-130.
[12]	Hengjun WANG,Nianwen SI,Yulong SONG,Yidong SHAN. Neural network model for dependency parsing incorporating global vector feature [J]. Journal on Communications, 2018, 39(2): 53-64.
[13]	Wanliang WANG,Zhuorong LI. Advances in generative adversarial network [J]. Journal on Communications, 2018, 39(2): 135-148.
[14]	Yong WANG,Huiyi ZHOU,Hao FENG,Miao YE,Wenlong KE. Network traffic classification method basing on CNN [J]. Journal on Communications, 2018, 39(1): 14-23.
[15]	Bing-lin ZHAO,Xi MENG,Jin HAN,Jing WANG,Fu-dong LIU. Homology analysis of malware based on graph [J]. Journal on Communications, 2017, 38(Z2): 86-93.

类别	RPCA-CNN模型	FCNN模型	DCNN模型	SSRN模型	DRN模型	DFBN模型
Asphalt	70.04%	86.46%	96.79%	97.40%	97.42%	98.16%
Meadows	84.28%	98.41%	99.23%	99.89%	99.75%	99.86%
Gravel	88.17%	65.25%	67.98%	88.64%	89.02%	96.11%
Trees	92.97%	71.72%	97.03%	97.08%	95.81%	96.60%
Painted metal sheets	100.00%	84.11%	96.50%	99.91%	94.94%	97.22%
Bare Soil	75.27%	91.61%	92.10%	99.54%	99.73%	99.61%
Bitumen	95.69%	61.62%	67.64%	99.55%	94.93%	98.81%
Self-Blocking Bricks	60.09%	86.58%	88.08%	96.44%	88.13%	97.52%
Shadows	99.88%	59.14%	99.41%	99.98%	96.75%	95.78%

类别	RPCA-CNN模型	FCNN模型	DCNN模型	SSRN模型	DRN模型	DFBN模型
Brocoli_green_weeds_1	99.99%	57.71%	77.03%	95.48%	97.36%	99.45%
Brocoli_green_weeds_2	81.90%	78.83%	98.25%	97.97%	99.34%	99.96%
Fallow	22.50%	85.04%	82.33%	87.17%	94.87%	99.97%
Fallow_rough_plow	91.23%	81.69%	98.76%	97.45%	93.77%	98.34%
Fallow_smooth	59.10%	97.54%	92.29%	95.11%	93.54%	98.57%
Stubble	99.53%	93.73%	99.94%	99.90%	98.23%	99.81%
Celery	93.05%	77.67%	98.40%	98.79%	98.16%	99.48%
Grapes_untrained	36.76%	78.56%	69.84%	80.93%	95.05%	96.78%
Soil_vinyard_develop	72.10%	97.90%	99.12%	99.55%	99.70%	100.00%
Corn_senesced_green_weeds	73.37%	98.07%	91.87%	93.60%	97.35%	99.77%
Lettuce_romaine_4wk	60.86%	84.15%	25.40%	86.31%	92.99%	98.25%
Lettuce_romaine_5wk	22.66%	96.44%	98.61%	91.50%	89.96%	99.71%
Lettuce_romaine_6wk	41.34%	84.12%	98.98%	98.09%	89.46%	93.87%
Lettuce_romaine_7wk	97.60%	91.01%	88.94%	93.96%	92.79%	95.75%
Vinyard_untrained	63.91%	55.72%	74.50%	72.31%	89.33%	98.62%
Vinyard_vertical_trellis	86.25%	39.11%	93.87%	94.39%	97.75%	99.87%