基于多模态互补特征学习的遥感影像语义分割

在遥感影像语义分割任务中，数字表面模型可以为光谱数据生成对应的几何表示，能够有效提升语义分割的精度。然而，大部分现有工作仅简单地将光谱特征和高程特征在不同的阶段相加或合并，忽略了多模态数据之间的相关性与互补性，导致网络对某些复杂地物无法准确分割。本文基于互补特征学习的多模态数据语义分割网络进行研究。该网络采用多核最大均值距离作为互补约束，提取两种模态特征之间的相似特征与互补特征。在解码之前互相借用互补特征，增强网络共享特征的能力。在国际摄影测量及遥感探测学会 (international society for photogrammetry and remote sensing, ISPRS)的Potsdam与Vaihingen公开数据集上验证所提出的网络，证明了该网络可以实现更高的分割精度。

Semantic segmentation of remote sensing image based on multimodal complementary feature learning

In the semantic segmentation of remote sensing images, the digital surface model can provide a corresponding geometric representation of the spectral data, which can effectively increase segmentation accuracy. However, most literature studies simply add or merge spectral and elevation features at different stages, ignoring the correlation and complementarity between multimodal data. This makes the network unable to accurately segment some complex features. This paper studies a multimodal data semantic segmentation network based on complementary feature learning. The network uses the multicore maximum mean distance as a complementary constraint to extract similar and complementary features between two modal features. The complementary features are borrowed from each other before decoding to enhance the feature sharing capability of the network. The proposed network is verified on the Potsdam and Vaihingen datasets of ISPRS and achieves higher segmentation accuracy.