结合细粒度自注意力的实例图像着色

尽管基于深度学习的图像着色方法已取得显著效果，但仍存在冗余色斑、着色暗淡和颜色偏差三个问题。为此，提出了一种结合细粒度自注意力（fine-grain self-attention，FGSA）的实例图像着色方法。具体地，首先将提取的特征图分为颜色和空间位置，并结合两者拟合提高颜色和图像空间位置的对应关系，以缓解冗余色斑。其次，受光学摄影HDR原理的启发，利用感受野小的卷积核增强或抑制图像的颜色特征，并结合softmax对特征进行动态映射，从而提高对比度，缓解着色暗淡的问题；最后，组合不同的非线性基函数，增加网络对非线性颜色的表达，拟合出最接近真实图像的颜色分布，以解决颜色偏差。大量的实验结果表明，该方法在实例图像着色中取得了良好的效果。特别地，与当前最优的着色方法相比，该方法在特征感知评价指标LPIPS和FID上分别降低了4.1%和7.9%。

Instance image coloring combined with fine-grained self attention

Although deep learning-based image coloring methods have achieved significant results, but there are still suffer from three problems: redundant stain, color dimming, and color deviation. To this end, this paper proposed an instance image coloring method combined with fine-grained attention(fine-grain self-attention, FGSA). Specifically, it firstly divided the extracted feature maps into color and spatial location, and combining the two parts of the fitting improved the correspondence between the color and the spatial location of the image to mitigate the redundant color patches. Secondly, inspired by the principle of HDR for optical photography, it utilized convolutional kernels with small sensory fields to enhance or suppress the color features of the image, and combined them with softmax to dynamically map the features, thus improving contrast and alleviating the darkness of the coloring. Finally, combining different nonlinear basis functions increased the network''s representation of nonlinear colors and fits a color distribution that was closest to the real image to address color bias. Extensive experimental results show that the method proposed in this paper achieves satisfactory results in instance image coloring. In particular, compared with the state-of-the-art methods, the proposed method improves 4.1% and 7.9% in feature perception evaluation indexes LPIPS and FID, respectively.