基于双流循环映射网络的肖像漫画化

肖像风格迁移旨在将参考艺术肖像画中迁移到人物照片上，同时保留人物面部的基本语义结构。然而，由于人类视觉对肖像面部语义结构的敏感性，使得肖像风格迁移任务比一般图像的风格迁移更具挑战性，现有的风格迁移方法未考虑漫画风格的抽象性以及肖像面部语义结构的保持，所以应用到肖像漫画化任务时会出现严重的结构坍塌及特征信息混乱等问题。为此，提出了一个双流循环映射网DSCM。首先，引入了一个结构一致性损失来保持肖像整体语义结构的完整性；其次，设计了一个结合U2-Net的特征编码器在不同尺度下帮助网络捕获输入图像更多有用的特征信息；最后，引入了风格鉴别器来对编码后的风格特征进行鉴别从而辅助网络学习到更接近目标图像的抽象漫画风格特征。实验与五种先进方法进行了定性及定量的比较，该方法均优于其他方法，其不仅能够完整地保持肖像的整体结构和面部的基本语义结构，而且能够充分学习到风格类型。

Portrait caricature based on double-stream cycle mapping network

Portrait artistic style transfer aims to transfer the style from a given reference artistic portrait painting to a portrait photo while preserving the basic semantic structure of the person''s face. However, due to the sensitivity of the human visual system to the facial structure of person, the task of artistic style transfer of portraits is often more challenging than that for general image, especially for caricature type which with more abstract style elements. Existing image style transfer methods, which do not consider the abstraction of the caricature style and the preservation of basic semantic structure of the portrait face, often suffer from serious structural collapse and feature information confusion when applied to the portrait caricature task. To address this problem, this paper proposed a double-stream cycle mapping DSCM(double-stream cycle mapping network) network to portrait caricature. Firstly, based on BeautyGAN, it introduced a structural consistency loss and cooperating with the cycle consistency loss to maintain the integrity of the overall semantic structure of the portrait. Secondly, it designed a feature encoder combined with U2-Net to capture more valuable feature information of input images at different scales. In addition, it further introduced a style discriminator to discriminate the encoded style features to assist the network in learning abstract caricature style features closer to the target image. The experiments conducted qualitative comparisons of five advanced methods, and quantitative comparisons of FID(Frechet inception distance) and PSNR(peak signal to noise ratio) index scores. The experimental results show that this method is superior to other methods. Through extensive experimental verification, the portrait caricature obtained by this method not only maintain the overall structure of the portrait and the basic semantic structure of the face, but also fully learn the abstract style of caricature.