基于综合特征分析的视频关键帧检索

综合利用了图像的颜色、形状和纹理特征,实现了对视频关键帧进行基于内容的检索。首先研究关键帧的选取、特征匹配等问题,再从视频处理的层次化结构的底层分析入手,构建了视频的连续帧图像序列,运用时间自适应检测法对镜头的关键帧进行了选取,建立了关键帧图像数据库。实验结果证明该方法性能良好。     

一种无人机航拍高清影像快速拼接方法

该文研究了一种无人机航拍高清影像快速拼接方法,主要包括路线规划、视频特定帧抽取、帧图像拼接三步.其中,为得到较精确的对地分辨率,对实验场景进行路线规划;为解决无人机影像冗余帧信息量大的问题,提出了一种基于目标场景和无人机飞行状态参数的抽取算法;为解决配准阶段的耗时问题,用SURF算子仅在重叠区域内进行图像配准,减少配准用时.实验表明,该方法基本达到实时性要求,且拼接图像的视觉效果较好.     

Two-dimensional mesh-based mosaic representation for manipulationof video objects with occlusion

We present a two-dimensional (2-D) mesh-based mosaic representation, consisting of an object mesh and a mosaic mesh for each frame and a final mosaic image, for video objects with mildly deformable motion in the presence of self and/or object-to-object (external) occlusion. Unlike classical mosaic representations where successive frames are registered using global motion models, we map the uncovered regions in the successive frames onto the mosaic reference frame using local affine models, i.e., those of the neighboring mesh patches. The proposed method to compute this mosaic representation is tightly coupled with an occlusion adaptive 2-D mesh tracking procedure, which consist of propagating the object mesh frame to frame, and updating of both object and mosaic meshes to optimize texture mapping from the mosaic to each instance of the object. The proposed representation has been applied to video object rendering and editing, including self transfiguration, synthetic transfiguration, and 2-D augmented reality in the presence of self and/or external occlusion. We also provide an algorithm to determine the minimum number of still views needed to reconstruct a replacement mosaic which is needed for synthetic transfiguration. Experimental results are provided to demonstrate both the 2-D mesh-based mosaic synthesis and two different video object editing applications on real video sequences.     

分布视频编码中基于帧间相关性的自适应关键帧选取算法

针对分布式视频编码(DVC)系统中固定周期关键帧选取(PKFS)方法忽视了帧间相关性的缺陷,提出了一种自适应关键帧选取(AKFS)算法。利用图像特征点检测与匹配的方法,将相邻图像的非匹配点作为帧间相关性的近似,把累积或平均非匹配点数超过阈值的帧判定为关键帧。在此基础上,提出改进的帧内插方案,以适应不同长度序列组的边信息生成;将零运动强度的关联帧合并为一帧图像参与编解码,进一步提高了系统的压缩效率。实验结果表明,对于不同运动特性的序列,本文提出的算法可以明显提升边信息帧的重建质量,使系统的率失真性能提高0.9～2.0 dB,并有效降低了编码传输码率。     

基于压缩感知和熵计算的关键帧提取算法

针对关键帧提取问题,提出了一种基于压缩感知理 论和熵计算的关键帧提取算法, 首先通过构造符合有限等距性质要求的稀疏随机投影矩阵,将高维多尺度帧图像特征变换为 低维多尺度帧图像特征, 并形成视频镜头低维多尺度特征列向量组;然后通过随机权值向量与低维多尺度特征向量的 阿达玛乘积运算生成各 帧图像的匹配特征,并根据匹配特征的相似性度量完成镜头内部的子镜头分割;最后通过交 叉熵计算在每个子镜头 中得到可能的关键帧,并由图像熵计算确定最终的关键帧。实验表明,与传统方法相比,本 文算法提取的关键帧能够更精确、更稳定描述视频镜头内容。     

Key-frame selection for video summarization: an approach of multidimensional time series analysis

This paper presents a novel method of key-frame selection for video summarization based on multidimensional time series analysis. In the proposed scheme, the given video is first segmented into a set of sequential clips containing a number of similar frames. Then the key frames are selected by a clustering procedure as the frames closest to the cluster centres in each resulting video clip. The proposed algorithm is implemented experimentally on a wide range of testing data, and compared with state-of-the-art approaches in the literature, which demonstrates excellent performance and outperforms existing methods on frame selection in terms of fidelity-based metric and subjective perception.     

Intelligent Mobile Video Surveillance System with Multilevel Distillation

This paper proposes a mobile video surveillance system consisting of intelligent video analysis and mobile communication networking. This multilevel distillation approach helps mobile users monitor tremendous surveillance videos on demand through video streaming over mobile communication networks. The intelligent video analysis includes moving object detection/tracking and key frame selection which can browse useful video clips. The communication networking services, comprising video transcoding, multimedia messaging, and mobile video streaming, transmit surveillance information into mobile appliances. Moving object detection is achieved by background subtraction and particle filter tracking. Key frame selection, which aims to deliver an alarm to a mobile client using multimedia messaging service accompanied with an extracted clear frame, is reached by devising a weighted importance criterion considering object clarity and face appearance. Besides, a spatial-domain cascaded transcoder is developed to convert the filtered image sequence of detected objects into the mobile video streaming format. Experimental results show that the system can successfully detect all events of moving objects for a complex surveillance scene, choose very appropriate key frames for users, and transcode the images with a high power signal-to-noise ratio (PSNR).     

Iterative key frame selection in the rate-constraint environment

Video representation through key frames has been addressed frequently as an efficient way of preserving the whole temporal information of sequence with a considerably smaller amount of data. Such compact video representation is suitable for the purpose of video browsing in limited storage or transmission bandwidth environments. In this case, the controllability of the total key frame number (i.e. key frame rate) depending on the storage or bandwidth capacity is an important requirement for the key frame selection method. In this paper, we present a sequential key frame selection method when the number of key frames is given as a constraint. It first selects the pre-determined number of initial key frames and time-intervals. Then, it adjusts the positions of key frames and time-intervals by iteration, which reduces the distortion step by step. Experimental results demonstrate the improved performance of our algorithm over the existing approaches.     

一种新的无人机航拍序列图像快速拼接方法

 无人驾驶飞机航拍序列图像的邻近帧间含有大量的相交区域,这些相交区域是进行基于特征的图像拼接的基础,但是也成为了降低拼接效率的冗余信息.因此,本文提出一种既稳定又具有较小时间开销的无人机航拍序列图像的自动拼接方法.利用图像拼接过程中查找到的匹配特征点,自适应提取出适于航拍序列图像快速拼接的关键帧;在分析了序列图像的拼接时的相交区域的运动模型的基础上,建立了无人机航拍序列图像配准过程中的特征搜索区域预测的卡尔曼滤波器,减少了特征搜索和配准的时间;然后给出了详细无人机航拍序列图像的快速拼接方法,实验结果显示,本文的方法具有较好的拼接效果和拼接效率.     

基于自适应补偿的电子稳像方法

基于车载的光电系统在采集图像的过程中,载车的随机振动会导致视频序列发生抖动,需要通过稳像技术来消除。为解决传统电子稳像方法中存在帧间跳变、机械性选择参考帧,以及边缘信息丢失的问题,提出了一种基于自适应补偿的电子稳像方法。首先,自适应地更新参考帧,后续帧与更新的参考帧进行特征点的提取和匹配,并通过参数迭代方法求解相对于第1帧的全局运动参数;然后,区分不同的全局运动参数:旋转角度、水平、垂直方向的偏移量,并通过运动滤波从全局运动参数中分离出扫描运动,补偿随机抖动;最后,采用后向映射方式实现运动补偿,并运用图像镶嵌,即利用参考基准帧的信息对校正后图像的边缘信息进行补偿。实验结果表明:稳像后视频序列有很好的视觉稳定效果,且提高了峰值信噪比(PSNR)和帧间变换保真度(ITF)。本文的方法具有很好的鲁棒性,在不同的视频场景下都可以实现稳像,可以满足稳像的精度要求。     

HEVC压缩域的视频摘要关键帧提取方法

为了提高关键帧提取的准确率，改善视频摘要的质量，提出了一种HEVC压缩域的视频摘要关键帧提取方法。首先，对视频序列进行编解码，在解码中统计HEVC帧内编码PU块的亮度预测模式数目。然后，特征提取是利用统计得到的模式数目构建成模式特征向量，并将其作为视频帧的纹理特征用于关键帧的提取。最后，利用融合迭代自组织数据分析算法(ISODATA）的自适应聚类算法对模式特征向量进行聚类，在聚类结果中选取每个类内中间向量对应的帧作为候选关键帧，并通过相似度对候选关键帧进行再次筛选，剔除冗余帧，得到最终的关键帧。实验结果表明，在Open Video Project数据集上进行的大量实验验证，该方法提取关键帧的精度为79.9%、召回率达到93.6%、F-score为86.2%，有效地改善了视频摘要的质量。      

Optimized video compression with residual split attention and swin-block artifact contraction

Research in video compression has seen significant advancement in the last several years. However, the existing deep learning-based algorithms continue to be plagued by erroneous motion compression and ineffective motion compensation architectures, resulting in compression errors with a lower rate–distortion trade-off. To overcome these challenges, we present an end-to-end purely deep learning-based video compression method through a set of primary operations (e.g., motion estimation, motion compression, motion compensation, residual compression, and artifact contraction) differently. A deep residual attention split (DRAS) block is introduced for motion compression networks to pay more attention to certain image regions to create more effective features for the decoder while boosting the rate–distortion optimization (RDO) efficiency. A channel residual block (CRB) is proposed in motion compensation to yield a more accurate predicted frame, potentially improving the residual frame. To mitigate the compression errors, an artifact contraction module (ACM) by residual swin convolution UNet block is included in this model to improve the reconstruction quality. To improve the final frame, a buffer is added to fine-tune the previous reference frames. These modules combine with a loss function by assessing the trade-off and enhancing the decoded video quality. A comprehensive ablation study demonstrates the effectiveness of the proposed blocks and modules for video compression. Experimental results show the competitive performance of the proposed method on four benchmark datasets.     

Video key frame extraction through dynamic Delaunay clustering with a structural constraint

Key frame based video summarization has emerged as an important area of research for the multimedia community. Video key frames enable an user to access any video in a friendly and meaningful way. In this paper, we propose an automated method of video key frame extraction using dynamic Delaunay graph clustering via an iterative edge pruning strategy. A structural constraint in form of a lower limit on the deviation ratio of the graph vertices further improves the video summary. We also employ an information-theoretic pre-sampling where significant valleys in the mutual information profile of the successive frames in a video are used to capture more informative frames. Various video key frame visualization techniques for efficient video browsing and navigation purposes are incorporated. A comprehensive evaluation on 100 videos from the Open Video and YouTube databases using both objective and subjective measures demonstrate the superiority of our key frame extraction method.     

一种改进的基于柯西模型的H.264码率控制方法

H.264/AVC编码中的码率控制是通过有效控制输出码流的码率来提高其压缩视频质量的重要技术。本文基于H.264/AVC中的JVT-H017码率控制方案提出了一种改进算法。新算法根据H.264中DCT系数的分布特征,将柯西分布引入到码率控制模块,用更精确的柯西率失真模型取代了原先的二次率失真模型。在此基础上,进一步引入了一种联合PSNR比率和MAD比率进行图像复杂度预测的方法,并依此来调整帧级比特的分配和量化参数,克服了在出现复杂运动或场景切换时,因视频序列相邻帧之间相关性降低而导致的MAD预测失准的情况。实验结果表明,与JVT-H017方案及文献[5]中的算法比较,新的算法不仅具有更精确的码率控制,而且明显改善了输出码率的平稳性及重建图像的PSNR。      

一种OpenCV,VFW及VCM相结合的实时视频处理框架

提出一种OpenCV,VFW和VCM相结合的实时视频处理框架.该框架以OpenCV中集成的DirectShow模块作为摄像机的驱动接口实时采集视频图像,在对图像进行处理后,采用VFW建立视频保存接口,通过VCM技术自动选择当前可行的视频压缩格式,避免了VFW需要弹出压缩格式对话框进行选择的缺点.实验表明,该视频处理框架具有稳定、实时的优点,采用该框架设计的软件已在实际中得到应用.     

A multi-balanced hybrid optimization technique to track objects using rough set theory

In this paper, the process of object detection and tracking is performed by means of five stages, namely frame segregation, shot segmentation, shape and texture feature extraction, object detection in frames through rough set theory and soft computing evolutionary programming with hybrid genetic algorithm particle swarm optimization. In the first stage, the input video file is segregated into number of frames and then the image frame from the specific shots is alone separated in the second stage with the help of DCT transformations. The third phase involves extracting shape and texture features from the shot segmented image frames.     

灰度校正在红外序列图像拼接中的应用

为满足红外图像无缝拼接的需要,提出了一种基于红外序列图像的灰度非均匀性校正算法.该算法利用灰度非均匀区域的相对稳定性,通过对拍摄的大量红外序列图像求平均,获得相机灰度校正的模板图像.然后利用此模板对各帧图像进行校正,实现在同一幅度照射下各像素灰度的一致性.最后利用由HJRG-001型红外热像仪获得的红外视频图像序列进行...     

基于雾线先验的时空关联约束视频去雾算法

现有视频去雾算法由于缺少对视频结构关联约束和帧间一致性分析,容易导致连续帧去雾结果在颜色和亮度上存在突变,同时去雾后的前景目标边缘区域也容易出现退化现象。针对上述问题,该文提出一种基于雾线先验的时空关联约束视频去雾算法,通过引入每帧图像在空间邻域中具有的结构关联性和时间邻域中具有的连续一致性,提高视频去雾算法的求解准确性和鲁棒性。算法首先使用暗通道先验估计每帧图像的大气光向量,并结合雾线先验求取初始透射率图。然后引入加权最小二乘边缘保持平滑滤波器对初始透射率图进行空间平滑,消除奇异点和噪声对估计结果的影响。进一步利用相机参数刻画连续帧间透射率图的时序变化规律,对独立求取的每帧透射率图进行时序关联修正。最后根据雾图模型获得最终的视频去雾结果。定性和定量的对比实验结果表明,该算法下视频去雾结果的帧间过渡更加自然,同时对每一帧图像的色彩还原更加准确,图像边缘的细节信息显示也更加丰富。     

Semi-automatic 2D-to-3D video conversion based on background sprite generation

This paper presents a technique for semi-automatic 2D-to-3D stereo video conversion, which is known to provide user intervention in assigning foreground/background depths for key frames and then get depth maps for non-key frames via automatic depth propagation. Our algorithm treats foreground and background separately. For foregrounds, kernel pixels are identified and then used as the seeds for graph-cut segmentation for each non-key frame independently, resulting in results not limited by objects’ motion activity. For backgrounds, all video frames, after foregrounds being removed, are integrated into a common background sprite model (BSM) based on a relay-frame-based image registration algorithm. Users can then draw background depths for BSM in an integrated manner, thus reducing human efforts significantly. Experimental results show that our method is capable of retaining more faithful foreground depth boundaries (by 1.6–2.7 dB) and smoother background depths than prior works. This advantage is helpful for 3D display and 3D perception.     

基于动态规划的自适应关键帧提取算法

提出一种基于内容的视频检索系统的关键帧提取新算法,把关键帧提取问题建模为一个可以用动态规划算法隶解的全局优化问题.首先建立二值的帧差矩阵来表示低维特征空间中帧与帧之间的相似性度量,然后使用动态规划算法分割帧差矩阵从而提取出关键帧.该算法具有低计算复杂度和对于视频内容的自适应性,而且保持了关键帧的时间顺序.可以方便地根据需要调节关键帧数目.