Cartoon synthesis using constrained spreading activation network

In this paper, we propose an approach to synthesize cartoons from the existing cartoon data by controlling the character’s path which is defined by the cartoonists in a background image. First, detailed pre-experiments are conducted in which different cartoon features are extracted and compared. During the pre-experiments, three features extracted from edge, motion and color are demonstrated effectively for evaluating cartoon similarity according to the quantitative analysis. The three features are then fused and a Cartoon Frame Relationship Network is constructed. Based on the graph, we propose a Constrained Spreading Activation Algorithm to select candidate frames which are visually similar to the current frame to generate the next frame. The cartoons are synthesized by choosing the most appropriate frame from the candidates in accordance with the path designed by the cartoonists. When the new cartoons are applied into the background image, our approach coordinates the cartoon character’s size according to the image’s perspective as well. The experiment results demonstrate that the combination of the three proposed features are effective in similarity evaluation, and the candidates selected by Constrained Spreading Activation Algorithm, are more similar to the current frame compared with other algorithms. The results also show that our approach can synthesize visually smooth cartoons from the existing cartoon library.     

Synthesizing style-preserving cartoons via non-negative style factorization

We present a complete framework for synthesizing style-preserving 2D cartoons by learning from traditional Chinese cartoons. In contrast to reusing-based approaches which rely on rearranging or retrieving existing cartoon sequences, we aim to generate stylized cartoons with the idea of style factorization. Specifically, starting with 2D skeleton features of cartoon characters extracted by an improved rotoscoping system, we present a non-negative style factorization (NNSF) algorithm to obtain style basis and weights and simultaneously preserve class separability. Thus, factorized style basis can be combined with heterogeneous weights to re-synthesize style-preserving features, and then these features are used as the driving source in the character reshaping process via our proposed subkey-driving strategy. Extensive experiments and examples demonstrate the effectiveness of the proposed framework.     

EXCOL: an EXtract-and-COmplete layering approach to cartoon animation reusing

We introduce the EXtract-and-COmplete Layering method (EXCOL)--a novel cartoon animation processing technique to convert a traditional animated cartoon video into multiple semantically meaningful layers. Our technique is inspired by vision-based layering techniques but focuses on shape cues in both the extraction and completion steps to reflect the unique characteristics of cartoon animation. For layer extraction, we define a novel similarity measure incorporating both shape and color of automatically segmented regions within individual frames and propagate a small set of user-specified layer labels among similar regions across frames. By clustering regions with the same labels, each frame is appropriately partitioned into different layers, with each layer containing semantically meaningful content. Then, a warping-based approach is used to fill missing parts caused by occlusion within the extracted layers to achieve a complete representation. EXCOL provides a flexible way to effectively reuse traditional cartoon animations with only a small amount of user interaction. It is demonstrated that our EXCOL method is effective and robust, and the layered representation benefits a variety of applications in cartoon animation processing.     

基于形状匹配的卡通运动捕捉

提出一种对传统的卡通动画进行运动捕捉的方法．不同于以前基于骨架模型或者关键形状表示的方法,用整体仿射运动和局部非仿射变形的组合来表示卡通人物的运动,用卡通人物的轮廓表示它的形状;然后直接对相邻的关键帧进行形状匹配,将恢复的运动参数映射到一个目标人物上,使其具有与原来人物相同的运动方式．文中不使用先验模型,通过形状匹配来获取和映射卡通人物的运动,并通过实验验证了该方法的可行性．     

基于递归对齐网络的黑白老卡通高清重制

黑白老卡通视频在数字化的过程中会出现诸如划痕、脏点、模糊和分辨率过低等复合问题,老卡通视频增强是视频增强的一类特殊子问题,当前尚缺乏针对性算法,因此提出一种多帧联合的递归对齐增强网络解决老卡通中的复合问题。首先通过递归结构传递重建历史中的长时域信息对划痕与脏点进行修复,解决了连续性划痕与脏点的处理难题。然后在递归单元中通过基于可变形卷积的对齐模块进行相邻帧特征对齐,改善了网络在卡通大幅度运动场景中提取和补充帧间细节的能力。在递归单元末端设计了级联金字塔结构的多尺度重建模块促进特征聚合,以充分挖掘重建所需的时间和空间细节信息。实验使用峰值信噪比等客观评估标准,在降质数据集和真实老卡通数据集上进行实验测试,并与其他主流方法进行对比。实验结果表明,该方法相比于其他主流视频增强方法有较为明显提升,同时在真实黑白老卡通上可获取高视觉质量的重建结果。     

Semi-supervised kernel density estimation for video annotation

Insufficiency of labeled training data is a major obstacle for automatic video annotation. Semi-supervised learning is an effective approach to this problem by leveraging a large amount of unlabeled data. However, existing semi-supervised learning algorithms have not demonstrated promising results in large-scale video annotation due to several difficulties, such as large variation of video content and intractable computational cost. In this paper, we propose a novel semi-supervised learning algorithm named semi-supervised kernel density estimation (SSKDE) which is developed based on kernel density estimation (KDE) approach. While only labeled data are utilized in classical KDE, in SSKDE both labeled and unlabeled data are leveraged to estimate class conditional probability densities based on an extended form of KDE. It is a non-parametric method, and it thus naturally avoids the model assumption problem that exists in many parametric semi-supervised methods. Meanwhile, it can be implemented with an efficient iterative solution process. So, this method is appropriate for video annotation. Furthermore, motivated by existing adaptive KDE approach, we propose an improved algorithm named semi-supervised adaptive kernel density estimation (SSAKDE). It employs local adaptive kernels rather than a fixed kernel, such that broader kernels can be applied in the regions with low density. In this way, more accurate density estimates can be obtained. Extensive experiments have demonstrated the effectiveness of the proposed methods.     

On combining multiple features for cartoon character retrieval and clip synthesis

How do we retrieve cartoon characters accurately? Or how to synthesize new cartoon clips smoothly and efficiently from the cartoon library? Both questions are important for animators and cartoon enthusiasts to design and create new cartoons by utilizing existing cartoon materials. The first key issue to answer those questions is to find a proper representation that describes the cartoon character effectively. In this paper, we consider multiple features from different views, i.e., color histogram, Hausdorff edge feature, and skeleton feature, to represent cartoon characters with different colors, shapes, and gestures. Each visual feature reflects a unique characteristic of a cartoon character, and they are complementary to each other for retrieval and synthesis. However, how to combine the three visual features is the second key issue of our application. By simply concatenating them into a long vector, it will end up with the so-called "curse of dimensionality," let alone their heterogeneity embedded in different visual feature spaces. Here, we introduce a semisupervised multiview subspace learning (semi-MSL) algorithm, to encode different features in a unified space. Specifically, under the patch alignment framework, semi-MSL uses the discriminative information from labeled cartoon characters in the construction of local patches where the manifold structure revealed by unlabeled cartoon characters is utilized to capture the geometric distribution. The experimental evaluations based on both cartoon character retrieval and clip synthesis demonstrate the effectiveness of the proposed method for cartoon application. Moreover, additional results of content-based image retrieval on benchmark data suggest the generality of semi-MSL for other applications.     

Fuzzy Diffusion Distance Learning for Cartoon Similarity Estimation

In this paper,a novel method called fuzzy diffusion maps (FDM) is proposed to evaluate cartoon similarity,which is critical to the applications of cartoon recognition,cartoon clustering and cartoon reusing.We find that the features from heterogeneous sources have different influence on cartoon similarity estimation.In order to take all the features into consideration,a fuzzy consistent relation is presented to convert the preference order of the features into preference degree,from which the weights are calculated.Based on the features and weights,the sum of the squared differences (L2) can be calculated between any cartoon data.However,it has been demonstrated in some research work that the cartoon dataset lies in a low-dimensional manifold,in which the L2 distance cannot evaluate the similarity directly.Unlike the global geodesic distance preserved in Isomap,the local neighboring relationship preserved in Locally Linear Embedding,and the local similarities of neighboring points preserved in Laplacian Eigenmaps,the diffusion maps we adopt preserve diffusion distance summing over all paths of length connecting the two data.As a consequence,this diffusion distance is very robust to noise perturbation.Our experiment in cartoon classification using Receiver Operating Curves shows fuzzy consistent relation’s excellent performance on weights assignment.The FDM’s performance on cartoon similarity evaluation is tested on the experiments of cartoon recognition and clustering.The results show that FDM can evaluate the cartoon similarity more precisely and stably compared with other methods.     

基于流形距离的半监督判别分析

大量无类别标签的数据具有对分类有用的信息,有效地利用这些信息来提高分类精确度,是半监督分类研究的主要内容.提出了一种基于流形距离的半监督判别分析(semi-supervised discriminant analysis based on manifold distance,简称SSDA)算法,通过定义的流形距离,能够选择位于流形上的数据点的同类近邻点、异类近邻点以及全局近邻点,并依据流形距离定义数据点与其各近邻点之间的相似度,利用这种相似度度量构造算法的目标函数.通过在ORL,YALE人脸数据库上的实验表明,与现有算法相比,数据集通过该算法降维后,能够使基于距离的识别算法具有更高的分类精确度.同时,为了解决非线性降维问题,提出了Kernel SSDA,同样通过实验验证了算法的有效性.     

3-D Ear Modeling and Recognition From Video Sequences Using Shape From Shading

We describe a novel approach for 3-D ear biometrics using video. A series of frames is extracted from a video clip and the region of interest in each frame is independently reconstructed in 3-D using shape from shading. The resulting 3-D models are then registered using the iterative closest point algorithm. We iteratively consider each model in the series as a reference model and calculate the similarity between the reference model and every model in the series using a similarity cost function. Cross validation is performed to assess the relative fidelity of each 3-D model. The model that demonstrates the greatest overall similarity is determined to be the most stable 3-D model and is subsequently enrolled in the database. Experiments are conducted using a gallery set of 402 video clips and a probe of 60 video clips. The results (95.0% rank-1 recognition rate and 3.3% equal error rate) indicate that the proposed approach can produce recognition rates comparable to systems that use 3-D range data. To the best of our knowledge, we are the first to develop a 3-D ear biometric system that obtains a 3-D ear structure from a video sequence.      

Towards perceptual video cropping with curve fitting

Coming with the proliferation of mobile devices, the issue of displaying videos with varying resolutions and aspect ratios has become emerging. To this end, state-of-the-art works typically resort to video retargeting or cropping, i.e. adaptively adjusting the display regions and contents within each video frame to fit for the mobile device screens. However, most existing approaches retain on evaluating the objective visual statistics to select salient parts to be cropped within each frame, with time-consuming constraints to ensure the temporal consistency, which is indeed unsuitable to cope with the massive videos uploaded daily from user-contributed platforms. In this paper, we propose an efficient yet robust video cropping algorithm which merits in two-fold: First, a perceptual Region of Interest selection approach is proposed by employing a novel Interesting Region Score, which is a linear combination of Rate of Focused Attention, Total Saliency Score, and Bias from Center Penalty. Second, we adopt fast curve fitting to seek for an optimal cropping sequence over a set of consecutive video frames within a shot, which avoids the complex optimization with temporal constraints among adjacent frames. Experiments on video sequences from Backkom cartoon and Iron Man II with comparisons against several alternative approaches demonstrate the efficiency and robustness of the proposed algorithm.     

A general learning framework using local and global regularization

In this paper, we propose a general learning framework based on local and global regularization. In the local regularization part, our algorithm constructs a regularized classifier for each data point using its neighborhood, while the global regularization part adopts a Laplacian regularizer to smooth the data labels predicted by those local classifiers. We show that such a learning framework can easily be incorporated into either unsupervised learning, semi-supervised learning, and supervised learning paradigm. Moreover, many existing learning algorithms can be derived from our framework. Finally we present some experimental results to show the effectiveness of our method.     

采用量化离散余弦变换系数检测视频单帧连续多次复制粘贴篡改

目前大多数时域视频帧复制粘贴篡改检测算法都是针对至少20帧以上的视频子序列的复制粘贴篡改,而对单帧复制粘贴篡改无法判断。而根据人眼视觉感知的特性,修改视频内容需要至少15帧以上的帧操作,因此篡改帧想通过单帧复制粘贴篡改来达到想要的效果,必须进行连续多次粘贴操作。为了检测这种篡改方式,针对性地提出了一种基于量化离散余弦变换(DCT)系数的视频单帧连续多次复制-粘贴篡改检测算法。首先,将视频转换为图像,采用量化后的DCT系数作为视频帧图像特征向量,并通过计算巴氏(Bhattacharyya)系数来衡量两相邻帧帧间相似度;再设定阈值来判断两相邻帧帧间相似度是否有异常,最后根据出现相似度异常的帧是否连续,以及连续出现的帧数来判断视频是否经过篡改,并定位篡改位置。实验结果表明,所提算法对不同场景的视频都能检测,不仅检测速度快,而且不受再压缩因素影响,算法的正确率高、漏检率低。     

真实感人脸的形状与表情空间

提出一种建立三维人脸扫描模型参数空间的算法,其中的模板拟和算法基于能量最小的优化机制,通过非线性优化过程求解模板模型在每个顶点上的位移矢量,使之逼近目标模型.优化目标方程由以下测度组成：距离、平滑度以及人脸特征对应,如特征曲线、边界和特征点对等.使用该算法可用于不同人脸以及不同表情模型之间的对应建立,从而获取一致参数化的人脸形状和表情空间.在文中系统中,三维面部特征曲线通过Canny边检测算法自动获取,这样自动检测获取的特征曲线可用于降低三维形状描述的维数,同时完整的面部几何形状通过径向基函数插值得到.在中性人脸和表情人脸模型上所作的一致参数化为许多应用提供了平台,如形状渐变,纹理迁移和表情迁移.考虑到自动提取的特征曲线和二维线画卡通人脸的相似性,使用迭代优化算法实现二维线画卡通人脸姿态到三维真实人脸模型的迁移.     

An approach to computer-supported cartooning

An approach to computer-supported cartooning is described that aims at optimizing the image-related working process by introducing computer-support in the drawing and painting stages of traditional cartoon production, in particular, allowing automatic coloring of a sequence of digitized images. This is achieved by using a shape-matching algorithm to evaluate the similarity of image regions and by performing an optimum region assignment to identify the corresponding ones and propagate the color information through the image sequence. In order to maintain the use of traditional drawing tools, the first step in the proposed system architecture is the processing of scanned animators' drawings to enhance their quality and extract meaningful information. Two different system modules allow either manual coloring of images or computer-assisted automatic painting of an image sequence. Additional modules allow the construction of a vector representation for the images, the generation of in-betweens, and the composing of each cartoon frame. The first stages of the system's architecture — image preprocessing, painting and vectorization — are presented. Special emphasis is placed on the fundamental ideas behind the computer-assisted painting and vectorization steps. The competitiveness of the approach, which requires no special hardware or high-performance workstation, is shown.     

Reinforcement Learning Transfer Based on Subgoal Discovery and Subtask Similarity

This paper studies the problem of transfer learning in the context of reinforcement learning. We propose a novel transfer learning method that can speed up reinforcement learning with the aid of previously learnt tasks. Before performing extensive learning episodes, our method attempts to analyze the learning task via some exploration in the environment, and then attempts to reuse previous learning experience whenever it is possible and appropriate. In particular, our proposed method consists of four stages: 1) subgoal discovery, 2) option construction, 3) similarity searching, and 4) option reusing. Especially, in order to fulfill the task of identifying similar options, we propose a novel similarity measure between options, which is built upon the intuition that similar options have similar stateaction probabilities. We examine our algorithm using extensive experiments, comparing it with existing methods. The results show that our method outperforms conventional non-transfer reinforcement learning algorithms, as well as existing transfer learning methods, by a wide margin.      

基于半监督自动谱聚类算法的网络故障检测

针对网络故障检测中利用先验知识不足和多数谱聚类算法需事先确定聚类数的问题,提出一种新的基于成对约束信息传播与自动确定聚类数相结合的半监督自动谱聚类算法。通过学习一种新的相似性测度函数来满足约束条件,改进NJW聚类算法,对非规范化的Laplacian矩阵特征向量进行自动谱聚类,从而提高聚类性能。在UCI标准数据集和网络实测数据上的实验表明,该算法较相关比对算法聚类准确率更高,可满足网络故障检测的实际需要。     

Real-time Light Animation

Light source animation is a particularly hard field of real‐time global illumination algorithms since moving light sources result in drastic illumination changes and make coherence techniques less effective. However, the animation of small (point‐like) light sources represents a special but practically very important case, for which the reuse of the results of other frames is possible. This paper presents a fast light source animation algorithm based on the virtual light sources illumination method. The speed up is close to the length of the animation, and is due to reusing paths in all frames and not only in the frame where they were obtained. The possible applications of this algorithm are the lighting design and systems to convey shape and features with relighting.     

Computer-assisted coloring by matching line drawings

An approach to automatically color line drawings based on feature matching is proposed. The motivation is that coloring 2D animation is still a labor-intensive process in current cartoon film production. The objective of our work is to investigate how to automatically color an image in a cartoon sequence on the basis of the previous frame. Our method first establishes the matching relationship of two images, after which it automatically paints one of them with the color information of the other using a region-matching algorithm. The region-matching algorithm is based on feature correspondences. The results show that the proposed algorithm can straightforwardly and robustly realize our objective and has a promising future for our next step to further automate conventional animation.