Video Painting via Motion Layer Manipulation

Temporal coherence is an important problem in Non‐Photorealistic Rendering for videos. In this paper, we present a novel approach to enhance temporal coherence in video painting. Instead of painting on video frame, our approach first partitions the video into multiple motion layers, and then places the brush strokes on the layers to generate the painted imagery. The extracted motion layers consist of one background layer and several object layers in each frame. Then, background layers from all the frames are aligned into a panoramic image, on which brush strokes are placed to paint the background in one‐shot. The strokes used to paint object layers are propagated frame by frame using smooth transformations defined by thin plate splines. Once the background and object layers are painted, they are projected back to each frame and blent to form the final painting results. Thanks to painting a single image, our approach can completely eliminate the flickering in background, and temporal coherence on object layers is also significantly enhanced due to the smooth transformation over frames. Additionally, by controlling the painting strokes on different layers, our approach is easy to generate painted video with multi‐style. Experimental results show that our approach is both robust and efficient to generate plausible video painting.     

摄像机运动情况下的运动对象检测

在监控应用中,由于场景是已知的,因此可以使用背景减去法检测运动对象.当摄像机进行扫描和倾斜运动时,需要使用多个图像帧才能完整地表示监控场景.如何组织和索引这些背景帧属于摄像机跟踪问题.提出一种无需摄像机标定的背景帧索引和访问方法.这一方法需要使用图像配准技术估计图像初始运动参数.提出一种屏蔽外点的图像配准算法,综合利用线性回归和稳健回归快速估计初始运动参数.为了快速计算连续帧之间的运动参数,提出一种基于四参数模型的优化算法.利用非参数背景维护模型抑制虚假运动象素.室内和户外实验结果表明本文方法是有效的.     

基于二维透射变换的视频全景图拼接算法

视频序列全景图拼接技术是视频图像分析的关键技术之一。针对帧间运动模型为二维投影变换的常见视频序列,提出了一种全景图拼接算法.该算法首先采用改进的基于RANSAC的特征点匹配算法对相邻帧的运动参数进行估计,接着采用直接法对各帧与全景图间的运动参数进行精确调整,最后运用自适应中值滤波绘制出全景图。实验结果表明,该方法能够生成较高质量的全景图,基本能实现无缝连接。     

一种视频序列拼接的新方法

提出了一种基于视频序列拼接的新方法。首先,利用KLT算法对视频序列中特征点进行提取和跟踪,实现关键帧粗略选取;其次,在选取的关键帧中利用SURF算法进行特征提取,利用最近邻距离比进行特征点匹配,通过RANSAC估计算法求精单映矩阵,并结合关键帧选取判定寻找最佳关键帧;最后,利用级联单映矩阵和加权融合算法实现视频序列拼接。实验验证了该方法的有效性。     

PTZ自主跟踪中的全景视频生成

提出了一种在单PTZ摄像机自主跟踪过程中生成全景视频的方法。该方法在自主跟踪目标的同时,生成目标在大场景上运动的全景视频,可应用于PTZ摄像机监控场所。该方 法将全景视频看作全景背景图像和当前目标图像的叠加：首先利用Mean Shift跟踪方法逐帧获取目标区域图像并保存;然后利用相邻两帧视频图像的竖直方向投影匹配和Harris角 点匹配结果合成全景背景,与传统的配准方法相比,大大降低了匹配运算的复杂度,使全景背景的生成能够实时进行,并记录每帧图像到背景图像的变换参数;最后逐帧将目标区 域图像变换到背景图像上得到全景视频。本文方法与传统的全景视频生成方法相比,无需人工控制摄像机的转动,也无需手工对齐视频帧,整个过程全部自动完成。     

一种视频序列的拼接算法

提出了一种视频序列的拼接算法。首先在同一镜头下的视频序列中提取若干关键帧,利用关键帧拼接表示序列的拼接;其次利用光流场算法计算出的运动位移量引导相邻关键帧间特征点的匹配,并结合随机抽样一致性（RANSAC）鲁棒估计算法和单映矩阵的级联性,得到相邻关键帧、非相邻关键帧间的对应矩阵;最后通过融合实现了序列的无缝拼接。实验验证了这种方法的有效性。     

基于块匹配的鲁棒视频图像镶嵌算法

为了获取宽视野的场景表示,提出了一种基于块匹配的视频图像镶嵌算法,该算法首先采用基于相位相关的块匹配方法估计出视频图像间的运动矢量场,并剔除其中由于图像噪声或运动物体的遮挡而导致外点运动矢量,然后根据图像的运动矢量场确定出图像子块之间的对应点对,进而利用得到的对应点对迭代求解图像间的变换模型参数以实现视频图像的自动镶嵌.针对真实场景的视频图像序列进行实验,获得了较好的镶嵌结果,表明了该算法的有效性.     

Motion Panoramas

In this paper we describe a method for analysing video sequences and for representing them as mosaics or panoramas. Previous work on video mosaicking essentially concentrated on static scenes. We generalize these approaches to the case of a rotating camera observing both static and moving objects where the static portions of the scene are not necessarily dominant, as it has been often hypothesized in the past. We start by describing a robust technique for accurately aligning a large number of video frames under unknown camera rotations and camera settings. The alignment technique combines a feature‐based method (initialization and refinement) with rough motion segmentation followed by a colour‐based direct method (final adjustment). This precise frame‐to‐frame alignment allows the dynamic building of a background representation as well as an efficient segmentation of each image such that moving regions of arbitrary shape and size are aligned with the static background. Thus a motion panorama visualizes both dynamic and static scene elements in a geometrically consistent way. Extensive experiments applied to archived videos of track‐and‐field events validate the approach. Copyright © 2004 John Wiley & Sons, Ltd.     

A new robust video watermarking algorithm based on SURF features and block classification

In this paper, we propose a robust block classification based semi-blind video watermarking algorithm using visual cryptography and SURF (Speed-Up Robust Features) features to enhance the robustness, stability, imperceptibility and real-time performance. A method of selecting the best frames in each shot and the best regions or blocks within best frames is proposed to avoid employing frame–by-frame method for generating owner’s share in order to enhance robustness as well as reducing time complexity. In our method, Owner’s share is generated using the classification of selected robust blocks within the chosen frames along with corresponding watermark information. In extraction process, the SURF features are employed to match the feature points of selected frames with all frames to detect selected frames. Moreover, we resynchronize the embedded regions from distorted video to original sequence using SURF feature points matching. Afterwards, based on these matched feature points, rotation and scaling parameters are estimated next, selected blocks are retrieved using side information being stored eventually, watermark information is reconstructed successfully. Selecting Best frames, best regions, and employing surf features make our method to be highly robust against various kinds of attacks including image processing attacks, geometrical attacks and temporal attacks. Experimental results confirm the superiority of our scheme in case of being applicable in the real world, enhancing robustness and exploiting idea imperceptibility, over previous related methods.     

一种内容完整的视频稳定算法

设计了一种基于可靠特征集合匹配的内容完整的视频稳定算法。为了避免运动前景上的特征点参与运动估计,由经典的KLT(Kanade-Lucas-Tomasi)算法提取特征点,而后基于特征有效性判定规则对特征点集合进行有效性验证以提高特征点的可靠性。利用通过验证的特征点对全局运动进行估计,得到精确的运动参数并据此对视频图像进行运动补偿。对于运动补偿造成的无定义区,首先计算当前帧的定义区与相邻帧的光流,以此为向导腐蚀无定义区;利用拼接的方法,填充仍为无定义区的像素。实验结果表明该算法对于前景物体运动具有较好的鲁棒性并能够生成内容完整的稳定视频序列。     

一种改进的摄像头视频实时拼接方法

针对普通摄像头水平视角较小的问题,通过同时采集具有一定重叠区域的摄像头视频帧图像,基于尺度不变特征变换（ScaleInvariantFeatureTransform,SIFT）特征点,用二分哈希搜索算法（DichotomyBasedHash,DBH）进行匹配,用随机采样一致（RandomSampleConsensus,RANSAC）算法消除误匹配,得到帧图像拼接映射关系。实验结果表明,该方法能有效地实现摄像头视频实时拼接,克服了既有方法在重叠区域小于20％时失效的不足,在重叠区域为10％左右时仍能取得有效的拼接。     

一种用于动态场景的全景表示方法

针对全景图无法表示动态场景这一问题，提出一种用于动态场景的全景图表示方法，将视频纹理和全景图结合起来，构造动态全景图。系统首先将一系列定点拍摄的图像拼接成全景图，然后用摄像机拍摄场景中周期或随机运动的物体，提取视频纹理，最后视频纹理与全景图对准并融合，生成动态全景图。动态全景图既保持静态全景图全视角漫游的优点，又使得场景具有动态的特征，极大地增强漫游的真实感。     

动态背景体育运动视频合成对比*

针对在同一场景下获取的体育运动视频,提出了一种基于全局运动补偿及运动前景区域信息的体育运动视频合成方法。首先,对待合成视频,通过全局运动估计与补偿,将相邻帧在空间上对齐到当前帧。通过计算帧差,得到当前帧中的运动前景区域信息。然后根据两段待合成视频之间背景的相似性,计算并修正全局运动参数,确定待合成对应帧之间的位置关系。最后,依据已经获得的运动前景区域信息,生成合成帧。实验结果表明,该方法可自动合成在同一场景中获得的有相似动态背景的体育视频,保持了前景与背景的清晰度,能清晰地显示运动员动作的差异。     

大场景的视频全景图拼接

为了解决大场景全景图拼接过程中的误差累积问题,提高图像拼接的一致性,提出了一个新的全景图拼接算法。把全局图像配准看作是一个统计估计问题,运用增广的卡尔曼滤波对图像的全局变换参数进行递归地增广和估计。实验结果表明,得到的变换参数是全局一致的,生成的全景图不存在拼接裂缝。拼接算法考虑了各种不确定性,适用于噪声大场景的全景图拼接。     

Real-time detection of steam in video images

In this paper, we present a real-time image processing technique for the detection of steam in video images. The assumption made is that the presence of steam acts as a blurring process, which changes the local texture pattern of an image while reducing the amount of details. The problem of detecting steam is treated as a supervised pattern recognition problem. A statistical hidden Markov tree (HMT) model derived from the coefficients of the dual-tree complex wavelet transform (DT-CWT) in small 48×48 local regions of the image frames is used to characterize the steam texture pattern. The parameters of the HMT model are used as an input feature vector to a support vector machine (SVM) technique, specially tailored for this purpose. By detecting and determining the total area covered by steam in a video frame, a computerized image processing system can automatically decide if the frame can be used for further analysis. The proposed method was quantitatively evaluated by using a labelled image data set with video frames sampled from a real oil sand video stream. The classification results were 90% correct when compared to human labelled image frames. The technique is useful as a pre-processing step in automated image processing systems.     

Mosaicing-by-recognition for video-based text recognition

Text recognition captured in multiple frames by a hand-held video camera is a challenging task because it is possible to capture and recognize a longer line of text while improving the quality of the text image by utilizing the redundancy of the overlapping areas between the frames. For this task, the video frames should be registered, i.e., mosaiced, after compensating for their distortions due to camera shakes. In this paper, a mosaicing-by-recognition technique is proposed where the problems of video mosaicing and text recognition are formulated as a unified optimization problem and solved by a dynamic programming-based optimization algorithm simultaneously and collaboratively. Experimental results indicate that, even if the frames undergo various distortions such as rotation, scaling, translation, and nonlinear speed fluctuation of camera movement, the proposed technique provides fine mosaic image by accurate distortion estimation (around 90% of perfect estimation) and character recognition accuracy (over 95%).     

Eliminating the effect of freezing frames on user perceptive by using a time interleaving technique

Streaming video over a wireless network faces several challenges such as high packet error rates, bandwidth variations, and delays, which could have negative effects on the video streaming and the viewer will perceive a frozen picture for certain durations due to loss of frames. In this study, we propose a Time Interleaving Robust Streaming (TIRS) technique to significantly reduce the frozen video problem and provide a satisfactory quality for the mobile viewer. This is done by reordering the streaming video frames as groups of even and odd frames. The objective of streaming the video in this way is to avoid the losses of a sequence of neighbouring frames in case of a long sequence interruption. We evaluate our approach by using a user panel and mean opinion score (MOS) measurements; where the users observe three levels of frame losses. The results show that our technique significantly improves the smoothness of the video on the mobile device in the presence of frame losses, while the transmitted data are only increased by almost 9% (due to reduced time locality).     

Rectified Surface Mosaics

We approach mosaicing as a camera tracking problem within a known parameterized surface. From a video of a camera moving within a surface, we compute a mosaic representing the texture of that surface, flattened onto a planar image. Our approach works by defining a warp between images as a function of surface geometry and camera pose. Globally optimizing this warp to maximize alignment across all frames determines the camera trajectory, and the corresponding flattened mosaic image. In contrast to previous mosaicing methods which assume planar or distant scenes, or controlled camera motion, our approach enables mosaicing in cases where the camera moves unpredictably through proximal surfaces, such as in medical endoscopy applications.     

基于组反馈融合机制的视频超分辨率模型

视频超分辨率（video super-resolution,VSR）的目的是利用多个相邻帧的信息来生成参考帧的高分辨率版本。现有的许多VSR工作都集中在如何有效地对齐相邻帧以更好地融合相邻帧信息,而很少在相邻帧信息融合这一重要步骤上进行研究。针对该问题,提出了基于组反馈融合机制的视频超分辩模型（GFFMVSR）。具体来说,在相邻帧对齐后,将对齐视频序列输入第一重时间注意力模块;然后,将序列分成几个小组,各小组依次通过组内融合模块实现初步融合。不同小组的融合结果经过第二重时间注意力模块;然后,各小组逐组输入反馈融合模块,利用反馈机制反馈融合不同组别的信息,最后将融合结果输出重建。经验证,该模型具有较强的信息融合能力,在客观评价指标和主观视觉效果上都优于现有的模型。     

空间运动图像序列的跨尺度插值方法

提出跨尺度插值方法,获取高质量、高帧率和高分辨率的运动图像序列,以增强空间对接任务中图像的运动平滑性,流畅地再现空间图像运动细节。提出基于视觉对应的关注域探测方法,有效地缩小插值帧的预测范围,实现关注尺度改变;提出基于一致敏感哈希的帧插值方法,生成高质量的插值帧,实现帧率尺度改变;提出基于区域的运动序列插值方法,提高关注域插值细节,实现分辨率尺度改变。实验结果表明,相比于已有的方法,可取得更好的视觉效果和更高的客观评价指标值。