基于多模态多级特征聚合网络的光场显著性目标检测

现有基于深度学习的显著性检测算法主要针对二维RGB图像设计，未能利用场景图像的三维视觉信息，而当前光场显著性检测方法则多数基于手工设计，特征表示能力不足，导致上述方法在各种挑战性自然场景图像上的检测效果不理想。提出一种基于卷积神经网络的多模态多级特征精炼与融合网络算法，利用光场图像丰富的视觉信息，实现面向四维光场图像的精准显著性检测。为充分挖掘三维视觉信息，设计2个并行的子网络分别处理全聚焦图像和深度图像。在此基础上，构建跨模态特征聚合模块实现对全聚焦图像、焦堆栈序列和深度图3个模态的跨模态多级视觉特征聚合，以更有效地突出场景中的显著性目标对象。在DUTLF-FS和HFUT-Lytro光场基准数据集上进行实验对比，结果表明，该算法在5个权威评估度量指标上均优于MOLF、AFNet、DMRA等主流显著性目标检测算法。

Light Field Salient Object Detection Based on Multi-modal Multi-level Feature Aggregation Network

Most existing deep learning based saliency detection algorithms focus on 2D RGB images. However, they fail to take advantage of 3D visual information of scenes.Most light field saliency detection methods are based on hand-crafted features, whose feature representation capacity is insufficient.These issues lead to poor performance in many challenging scene images.To remedy these problems, this paper proposes a multi-modal multi-level feature aggregation network based on convolutional neural network for light field salient object detection.To fully exploit 3D visual information, two stream sub-network are designed in parallel to handle all-focus images and depth maps separately.Moreover, several feature aggregation modules are developed to aggregate multi-level features to detect the salient objects in scene.Moreover, several cross-modal feature fusion modules are designed to fuse multi-modal features from all-focus images, focal stack, and depth maps, which can highlight a salient object by utilizing 3D visual information.Comprehensive experimental comparisons were performed on the DUTLF-FS and HFUT-Lytro light field benchmark datasets, and the results reveal that the algorithm outperforms the mainstream salient target detection algorithms, such as MOLF, AFNet, and DMRA on five authoritative evaluation metrics.