Video summary generation by visual shielding compressed sensing coding and double-layer affinity propagation

Video summary technology based on keyframe extraction is an effective means to rapidly access video content. Traditional video summary generation technology requires high video resolution, which poses a problem as most existing studies have no targeted solutions for videos that are subject to privacy protection. We propose a novel keyframe extraction algorithm for video data in the visual shielding domain, named visual shielding compressed sensing coding and double-layer affinity propagation (VSCS-DAP). VSCS-DAP involves three main steps. First, the video is compressed by compressed sensing technology to provide a visual shielding effect (protecting the privacy of monitored figures), while the data volume is significantly reduced. Then, pyramid histogram of oriented gradients (PHOG) features are extracted from the compressed video to be clustered by the first step affinity propagation (AP) to gain the summaries of the first stage. Finally, the PHOG and Hist fusion features are extracted from the keyframes of the first stage, and they cluster the fused PHOG-Hist features by the second step AP algorithm to obtain the final output summaries. Experimental results obtained on two common video datasets show that our method exhibits advantages including low redundancy and few missing frames, low computational complexity, strong real-time performance, and robustness to vision-shielded video.     

视频隐私保护技术综述

互联网的快速发展和各类视频应用的日益普及,使视频个人隐私保护得到广泛关注。如何在保证视频应用的同时使隐私信息不被泄露是目前亟待研究和解决的热点问题。针对这些问题从视频隐私保护技术的研究现状进行综述,从隐私提取、隐私区域保护及访问权限控制3个方面进行概括、比较和分析。此外,对与视频隐私保护相关的一些问题,如隐私的定义、感知安全性评价等方面进行了讨论。最后总结了视频隐私保护技术面临的挑战,并对其发展趋势进行展望。     

Augmented keyframe

In surveillance applications, keyframes are usually used to summarize important video contents. However, most traditional keyframe extraction approaches just select some video frames from the input video, while the information in these selected video frames are insufficient and cannot let the users perceive the events happened in the original video easily. In this paper, we propose a novel keyframe generating technique to condense the contents of a surveillance video clip captured by a static camera into a still picture (augmented keyframe). The augmented keyframe is a more meaningful keyframe augmented with representative objects, important contents (human faces, license plate, etc.), motion status (represented as icons) and some marks of the moving objects in a still picture. This new technique consists of two major phases: content extraction and content synthesis. Some testing results of the proposed augmented keyframe are also presented and compared with results obtained by the traditional keyframe approach.     

Visual privacy-preserving level evaluation for multilayer compressed sensing model using contrast and salient structural features

Recognition and classification tasks in images or videos are ubiquitous, but they can lead to privacy issues. People increasingly hope that camera systems can record and recognize important events and objects, such as real-time recording of traffic conditions and accident scenes, elderly fall detection, and in-home monitoring. However, people also want to ensure these activities do not violate the privacy of users or others. The sparse representation classification and recognition algorithms based on compressed sensing (CS) are robust at recognizing human faces from frontal views with varying expressions and illuminations, as well as occlusions and disguises. This is a potential way to perform recognition tasks while preserving visual privacy. In this paper, an improved Gaussian random measurement matrix is adopted in the proposed multilayer CS (MCS) model to realize multiple image CS and achieve a balance between visual privacy-preserving and recognition tasks. The visual privacy-preserving level evaluation for MCS images has important guiding significance for image processing and recognition. Therefore, we propose an image visual privacy-preserving level evaluation method for the MCS model (MCS-VPLE) based on contrast and salient structural features. The basic concept is to use the contrast measurement model based on the statistical mean of the asymmetric alpha-trimmed filter and the salient generalized center-symmetric local binary pattern operator to extract contrast and salient structural features, respectively. The features are fed into a support vector regression to obtain the image quality score, and the fuzzy c-means algorithm is used for clustering to obtain the final evaluated image visual privacy-preserving score. Experiments on three constructed databases show that the proposed method has better prediction effectiveness and performance than conventional methods.     

Data hiding in partially encrypted HEVC video

In this study, an efficient scheme for hiding data directly in partially encrypted versions of high efficiency video coding (HEVC) videos is proposed. The content owner uses stream cipher to selectively encrypt some HEVC‐CABAC bin strings in a format‐compliant manner. Then, the data hider embeds the secret message into the encrypted HEVC videos using the specific coefficient modification technique. Consequently, it can be used in third‐party computing environments (more generally, cloud computing). For security and privacy purposes, service providers cannot access the visual content of the host video. As the coefficient is only slightly modified, the quality of the decrypted video is satisfactory. The encrypted and marked bitstreams meet the requirements of format compatibility, and have the same bit rate. At the receiving end, data extraction can be performed in the encrypted domain or decrypted domain that can be adapted to different application scenarios. Several standard video sequences with different resolutions and contents have been used for experimental evaluation.     

面向机器学习的隐私保护关键技术研究综述

随着信息通信技术的发展,机器学习已经成为多个研究领域与垂直行业必不可少的技术工具。然而,机器学习所需数据中往往包含了大量的个人信息,使其隐私保护面临风险与挑战,受到了越来越多的关注。对现有机器学习下隐私保护法规政策与标准化现状进行梳理,对适用于机器学习的隐私保护技术进行详细介绍与分析。隐私保护算法通常会对数据质量、通信开支与模型表现等造成影响,因此对于隐私保护算法的评估应当进行多维度的综合评估。总结了适用于机器学习应用的隐私保护性能评估指标,并指出隐私保护需要考虑对数据质量、通信开支以及模型准确率等之间的影响。     

姿态估计和跟踪结合的运动视频关键帧提取

提出姿态估计和特定部位跟踪相结合的动作视频关键帧提取算法.首先利用非确定人体部位的时间连续性保持提高基于柔性部件铰接人体模型的单帧图像人体姿态估计准确率,通过实施数据降维得到局部拓扑结构表达能力强的判别性运动特征向量,采用极值判定原理确定候选关键帧集合.然后利用ISODATA动态聚类算法,通过初始聚类中心优化、基于语义的关键帧集合增强等策略确定关键帧.实验表明文中算法具有较高的关键帧提取准确率和召回率,支持基于语义的关键帧提取.提取的视频关键帧可以用于运动视频压缩和批注审阅.     

基于视觉显著计算的视频流媒体渐进式表达方法

为了有效解决视频流媒体传输网络带宽、播出视频质量和用户实时性访问之间的矛盾,本文提出了一种基于视觉显著计算的视频流媒体渐进式表达方法.在视频内容分析和理解的基础上,首先进行场景分类和视觉敏感区域提取;然后根据编码信息确定视频序列中各帧的重要性,估计帧内片层数据重要性;最后基于视觉显著计算的结果提出一种适应网络带宽和质量可伸缩的视频流媒体渐进式表达方法.采用中粒度质量可伸缩（MGS）编码,在模拟网络测试平台上分别针对集中式和分散式视觉敏感区域视频序列进行实验研究,实验结果验证了本文提出的基于视觉显著计算的视频流媒体渐进式表达方法的正确性和有效性.     

A fast algorithm of bitstream extraction using distortion prediction based on simulated annealing

Scalable video streams can be extracted to meet the bandwidth limitation of different networks and end-users. Bitstream extraction is usually performed at the network proxy or gateway during transmission, where a low computational complexity is always preferred. How to quickly and accurately select the best resolution combination for a video to meet different bandwidth requirements by each user is crucial in bitstream extraction. In this paper a fast algorithm of bitstream extraction for scalable video is proposed. The interlayer dependency between the base quality layer and the first quality layer was used to predict the distortion of higher quality layers. When quality of every layer is available, the proposed method searches for the optimized combination of quality layers based on simulated annealing. Experimental results show that the proposed method provides an optimized performance, which is significantly higher than that can be achieved by the basic extraction method. Compared to the quality layer based extraction method in the reference software model of H.264/SVC (i.e., JSVM), the proposed algorithm can greatly decrease the decoding times from 2NT to only 2 without losing rate-distortion performance. Furthermore, the proposed method obtains a more smoothed video quality which is always favorable to the observer.     

Content analysis based smart macroblock rearrangement for error resilience in wireless video transmission

Transmission of compressed video is very sensitive to channel error. The use of resynchronization marks is an efficient method in improving the error resilient performance of a video stream in error-prone environment. In this paper, a smart macroblock rearrangement (SMR) method is proposed to enhance the performance of the resynchronization mark insertion technique for intracoded frames in wireless video transmission. The proposed method makes use of content analysis to rearrange and encode macroblocks between adjacent resynchronization marks. Experiments using the SMR method in both H.263+ and H.264 codecs show that the proposed method outperforms some existing algorithms in both PSNR performance and visual quality.     

基于运动和视差信息的立体视频质量客观评价

在研究人类 立体视觉特性及现有立体图像/视频质量评价算法的基础上,提出了一种基于运动信息和视 差信息的立 体视频质量的客观评价方法。方法包括视频质量评价(VQA)和视频立体感评价(VSSA)两个指 标,其中VQA的估计基于梯度的结构相似度(GSSIM) 算法,并充分考虑了帧内的亮度信息和结构信息、帧间运动信息以及人眼的感知特性对视频 质量的影响, 特别是根据人类的视觉特性,对左右视点的质量赋予了不同的权重;VSSA的估计 是通过计算参考 视频的绝对差值图和降质视频的绝对差值图之间的峰值信噪比(PSNR)而得到。实验结果表明,本文方法对基于H.264 编码的失真视频的评价结果与主观测试有较高的一致性,很好地体现人眼的视觉特性。     

Tsallis entropy-based information measures for shot boundary detection and keyframe selection

Automatic shot boundary detection and keyframe selection constitute major goals in video processing. We propose two different information-theoretic approaches to detect the abrupt shot boundaries of a video sequence. These approaches are, respectively, based on two information measures, Tsallis mutual information and Jensen–Tsallis divergence, that are used to quantify the similarity between two frames. Both measures are also used to find out the most representative keyframe of each shot. The representativeness of a frame is basically given by its average similarity with respect to the other frames of the shot. Several experiments analyze the behavior of the proposed measures for different color spaces (RGB, HSV, and Lab), regular binnings, and entropic indices. In particular, the Tsallis mutual information for the HSV and Lab color spaces with only 8 regular bins for each color component and an entropic index between 1.5 and 1.8 substantially improve the performance of previously proposed methods based on mutual information and Jensen–Shannon divergence.     

Foveation scalable video coding with automatic fixation selection

Image and video coding is an optimization problem. A successful image and video coding algorithm delivers a good tradeoff between visual quality and other coding performance measures, such as compression, complexity, scalability, robustness, and security. In this paper, we follow two recent trends in image and video coding research. One is to incorporate human visual system (HVS) models to improve the current state-of-the-art of image and video coding algorithms by better exploiting the properties of the intended receiver. The other is to design rate scalable image and video codecs, which allow the extraction of coded visual information at continuously varying bit rates from a single compressed bitstream. Specifically, we propose a foveation scalable video coding (FSVC) algorithm which supplies good quality-compression performance as well as effective rate scalability. The key idea is to organize the encoded bitstream to provide the best decoded video at an arbitrary bit rate in terms of foveated visual quality measurement. A foveation-based HVS model plays an important role in the algorithm. The algorithm is adaptable to different applications, such as knowledge-based video coding and video communications over time-varying, multiuser and interactive networks.     

User Friendly Preferential Private Recommendation

To provide preferential protection for users while keeping good service utility, a preferential private recommendation framework ( named as PrefER) is proposed. In this framework, a preferential budget allocation scheme is designed and implemented at the system side to provide multilevel protection. And users' preference is utilized at the user side to improve recommendation performance without increasing users' burden. This framework is generic enough to be employed with other schemes. Recommendation accuracy based on the MovieLens dataset by the collaborative filtering schemes and PrefER are compared and analyzed. The experimental results show that PrefER can provide preferential privacy protection with the improvement of recommendation accuracy.     

基于红外运动目标分割的夜视融合系统设计

为了提高夜视系统的质量以及目标探测性能, 设计了红外视频运动目标与可见光融合夜视侦察系统。系统在硬件处理平台上实现了基于人眼视觉的红外运动目标分割算法和基于目标特性的加权融合算法, 采用红外运动目标分割电路和融合处理电路互联的结构, 首先利用FPGA为核心的红外目标分割电路提取红外视频中的运动目标, 然后将只有红外运动目标的视频输入后端DSP融合处理电路中进行融合处理, 最后从系统中输出一系列目标形态突出、背景细节清晰的融合序列。实验结果表明: 该系统可以提高夜视融合系统目标探测性能和探测概率, 融合结果的各项评价指标提高了90%以上, 有的评价指标甚至提高了7倍以上, 可以很大程度地降低系统的虚警率。     

基于三维变换域频谱差的视频显著性检测算法

显著性区域检测是计算机视觉的重要课题,对视频质量评价和感知视频编码算法优化也至关重要.大多显著性检测算法不能权衡准确率和复杂度,限制了它们在视频预处理和实时处理中的应用.提出了一种基于三维变换域频谱差(3DTDSD)的快速视频显著性检测算法,分别以关键帧及其前一帧为中心建立一定数量图像帧的滑动窗,得到两组3D视频体,用傅里叶变换将两组视频变换到三维频域,两组三维数据之间的差值经过反变换得到显著性图,最后通过连通分析、阈值判断等得到显著区域.频域算法具有运算速度快的特点,实验对比和算法复杂度分析证明了该算法的有效性和快速性.     

一种基于用户面数据采集的视频业务质量优化方法

随着移动视频用户规模和业务规模逐年递增,为了满足更多用户的需求,要求运营商更加重视业务感知,针对此背景提出了一种基于用户面数据采集的视频业务质量优化方法。本文通过研究vMOS评估模型,构建了基于DPI数据的视频业务感知评估体系。通过vMOS评分从无线角度明确了视频质差小区判定标准,构建了日常视频质差小区处理流程,以指导日常无线网络优化,提升无线网络优化的智能化水平。     

基于交互模式的隐私保护方法研究

In network environments, before meaningful interactions can begin, trust may need to be established between two interactive entities in which an entity may ask the other to provide some information involving privacy. Consequently, privacy protection and trust establishment become important in network interactions. In order to protect privacy while facilitating effective interactions, we propose a trust based privacy protection method. Our main contributions in this paper are as follows: (1) We introduce a novel concept of k sensitive privacy as a measure to assess the potential threat of inferring privacy; (2) According to trust and k sensitive privacy evaluation, our proposed method can choose appropriate interaction patterns with lower degree of inferring privacy threat; (3) By considering interaction patterns for privacy protection, our proposed method can overcome the shortcomings of some current privacy protection methods which may result in low interaction success rate. Simulation results show that our method can achieve effective interactions with less privacy loss.     

注意力模型指导下的视频质量评价方法

在视频质量评价方法中,常常需要对人眼视觉系统做出合理的假设,其中注意力模型就是一个很重要的因素。提出了一种在注意力模型指导下的视频质量评价方法,在图像帧的质量评价中加入了显著性区域信息,使之更能符合人眼视觉特性,并兼顾了视频中的运动信息,在一定程度上提高了客观质量评价方法的性能。