显著性引导的目标互补隐藏弱监督语义分割

目的 图像级弱监督语义分割方法利用类别标签训练分割网络，可显著降低标注成本。现有方法大多采用类激活图定位目标物体，然而传统类激活图只能挖掘出物体中最具辨识性的区域，直接将其作为伪标签训练的分割网络精度较差。本文提出一种显著性引导的弱监督语义分割算法，可在获取更完整类激活图的基础上提高分割模型的性能。方法 首先通过显著图对目标进行互补随机隐藏，以获得互补图像对，然后融合互补图像对的类激活图作为监督，提高网络获取完整类激活图的能力。其次引入双重注意力修正模块，利用全局信息修正类激活图并生成伪标签训练分割网络。最后使用标签迭代精调策略，结合分割网络的初始预测、类激活图以及显著图生成更精确的伪标签，迭代训练分割网络。结果 在PASCAL VOC 2012(pattern analysis,statistical modeling and computational learning visual object classes 2012)数据集上进行类激活图生成实验与语义分割实验，所生成的类激活图更加完整，平均交并比有10.21%的提升。语义分割结果均优于对比方法，平均交并比提升6.9%。此外在...

Saliency guided object complementary hiding for weakly supervised semantic segmentation

Objective The fully supervised semantic segmentation method based on deep learning has made remarkable progress,promoting practical applications such as automatic driving and medical image analysis.However,the fully supervised semantic segmentation method depends on the complete pixel-wise annotation,and the construction of large-scale pixel-wise annotation datasets requires a considerable amount of human labor and resources.Researchers have recently attempted to study semantic segmentation based on convenient supervisions,such as bounding boxes,scribbles,points,and image-level labels,to reduce the reliance on accurate annotations.Weakly supervised semantic segmentation based on image-level labels only uses category labels to train the segmentation network,which can significantly reduce the annotation cost.Most of the existing weakly supervised semantic segmentation methods use class activation map(CAM)to locate target objects.On the one hand,the CAM generated by classification networks is sparse and can only focus on the most discriminative areas of objects.Some misactivated pixels are observed in the CAM,which may provide improper guidance for the subsequent segmentation task.On the other hand,the performance of the segmentation network depends on the quality of the pseudo labels.Thus,obtaining the accurate pseudo label also requires the shape and boundary of the object.However,this information cannot be directly and accurately obtained in image-level labels,and guaranteeing the quality of pseudo labels is difficult.A new saliency-guided weakly supervised semantic segmentation algorithm is proposed in this paper to improve the performance of the segmentation model to obtain complete CAMs.Method First,research shows that randomly hiding the target in the image can enhance the capability of the network to locate the complete target.However,part of the image information cannot be used when directly hiding the image at random.By contrast,the complementary hiding method can use all the image information.However,guaranteeing that the target object can be hidden as expected is difficult due to the randomness of the hiding method.Only the background area is randomly hidden in some cases.A saliency-guided object complementary hiding method is proposed in this paper.Through the foreground information provided by the saliency map,the complementary random hiding of the object in the image is performed to obtain the complementary image pairs.The CAMs of the complementary image pairs are then fused as supervision to improve the capability of the network to obtain complete CAMs.Second,the convolution operation in the classification network used to generate CAMs can lead to a local receptive field,which may cause some differences in the corresponding features of the same class objects with changes in scale,illumination,and viewing angle.These differences may result in intra-class inconsistency,negatively affecting the activation and leading to mis-activation in the CAM.In addition,the classification network itself has weak capability to extract complete objects,and achieving good effects in expanding the object area using the object complementary hiding method guided only by saliency is still difficult.Therefore,a dual attention refinement module is introduced to further correct the CAM by the global information,and the obtained CAM is used to generate the pseudo label to train the segmentation network.Prediction results of the segmentation network will have higher accuracy than the original pseudo labels.However,the prediction results also have some noise,which cannot guarantee the performance improvement of the segmentation model by directly using iterative training.Finally,this paper uses the label iteration refinement strategy,combines the initial prediction of the segmentation network,CAM,and saliency map to generate pseudo labels,and iteratively trains the segmentation network to further improve the performance of the segmentation network.Saliency maps can effectively distinguish between foreground and background objects but cannot identify the object categories.CAMs can accurately locate the object categories but lack information regarding the complete shape of the objects.Segmentation network prediction can provide relatively complete information regarding the object boundary but may contain misclassification pixels.The impact of pixel misclassification is markedly reduced by fully utilizing the information provided by the three types of maps to refine the pseudo labels.Result The experiment is divided into two parts to verify the effectiveness of the algorithm.In the first part,the proposed CAM generation algorithm in this paper is verified and compared with other methods.In the second part,the proposed method is compared with several classical weakly supervised semantic segmentation algorithms,and the effectiveness of the modules in the proposed model is analyzed by ablation experiment.The experiments are initially conducted on the PASCAL VOC 2012 dataset.By contrast,the CAM generated by this algorithm is more complete,and its mean intersection over union(mIoU)is improved by 10.21% compared with the baseline.The segmentation network produced better prediction results compared with the six methods,demonstrating a 6.9% improvement over the baseline.Thus,the proposed method outperforms the other methods in 13 categories.With an mIoU value of 92% in the background category,the proposed method achieved the highest performance among other methods,indicating its effective utilization of saliency maps in training.Multi-objective semantic segmentation experiment is also conducted on the COCO 2014 data set.Compared with PASCAL VOC 2012,this dataset has richer categories and contains a larger number of images with multiple object categories,indicating a high demand on the performance of the algorithm.Experimental results show that the value of mIoU is improved by 0.5% on COCO 2014.Conclusion This algorithm can obtain a complete CAM,effectively alleviate the problem of insufficient supervision information,and improve the accuracy of weakly supervised semantic segmentation models.