On the impact of the GOP size in a temporal H.264/AVC-to-SVC transcoder in baseline and main profile

Scalable video coding is a recent extension of the advanced video coding H.264/AVC standard developed jointly by ISO/IEC and ITU-T, which allows adapting the bitstream easily by dropping parts of it named layers. This adaptation makes it possible for a single bitstream to meet the requirements for reliable delivery of video to diverse clients over heterogeneous networks using temporal, spatial or quality scalability, combined or separately. Since the scalable video coding design requires scalability to be provided at the encoder side, existing content cannot benefit from it. Efficient techniques for converting contents without scalability to a scalable format are desirable. In this paper, an approach for temporal scalability transcoding from H.264/AVC to scalable video coding in baseline and main profile is presented and the impact of the GOP size is analyzed. Independently of the GOP size chosen, time savings of around 63 % for baseline profile and 60 % for main profile are achieved while maintaining the coding efficiency.     

Efficient block-based transparent encryption for H.264/SVC bitstreams

Taking advantage of the inter-layer prediction technique used in H.264/scalable video coding (H.264/SVC), in this paper we propose an efficient block-based encryption scheme (BBES) for encrypting H.264/SVC enhancement layers (ELs). BBES operates in three modes, namely, Intra-MB mode, Group-MB mode and 4Group-MB mode. All the three modes are effective in securing ELs, preserve the “adaptation-transparent” property of H.264/SVC, and are format-compliant to the H.264/SVC bitstream format specifications. Moreover, Intra-MB and Group-MB modes also possess the property we termed as “transcoding transparency”. Experimental results indicate that BBES has low computational complexity and small compression overhead. Thus, BBES is suitable for transparent encryption of H.264/SVC bitstreams in which ELs are encrypted but base layers are left in cleartext.     

Real-time error concealing bitstream adaptation methods for SVC in IPTV systems

The current heterogeneity in networks and devices demands for a high degree of flexibility in IPTV systems for digital television. A scalable video coding scheme (in this paper we focus on H.264/AVC’s scalable video coding extension SVC) accommodates this flexibility from the coding point of view. Because the IP-based network delivery chain in IPTV systems may suffer from packet loss (having a severe impact on the visual quality) it is necessary to provide means for error concealment. In this paper we propose a novel method that performs adaptation on impaired SVC bitstreams so that the resulting adapted bitstream is compliant to the SVC specification and that the reconstruction result at the decoder is equivalent compared to the approach where the error concealment is implemented in the decoder itself. The adapted bitstreams have a significantly higher visual quality while our approach does not require any modification to existing SVC-compliant decoders. The results of several experiments show that the proposed method is extremely fast (over 900 frames/s) and that it introduces a negligible overhead in terms of bit rate (ca. 0.02%).     

Concealment of Whole-Picture Loss in Hierarchical B-Picture Scalable Video Coding

H.264/AVC scalable video coding (H.264/AVC SVC), as the scalable extension of H.264/AVC, offers the flexible adaptivity in terms of spatial, temporal and SNR scalabilities for the generated bitstream. However, such compressed video still suffers from the bad playback quality when packet loss occurs over unreliable networks. In this paper, we present an error concealment algorithm to tackle the whole-picture loss problem in H.264/AVC SVC when hierarchical B-picture coding is used to support temporal scalability. In the proposed algorithm, by taking advantage of the temporal relationship among the adjacent video pictures, the motion information of the lost picture is derived simply and efficiently based on the principle of temporal direct mode. Utilizing the derived motion information, the lost picture is concealed by performing motion compensation on the correctly received temporally previous and future video pictures. The experimental results demonstrate that as a post-processing tool, the proposed error concealment algorithm is able to significantly improve both the objective and subjective qualities of the decoded video pictures in the presence of packet losses when compared to the error concealment algorithm used in H.264/AVC SVC reference software. The proposed method can also be applied to H.264/AVC with hierarchical B-picture coding for error concealment.     

Seamless switching of scalable video bitstreams for efficient streaming

Efficient adaptation to channel bandwidth is broadly required for effective streaming video over the Internet. To address this requirement, a novel seamless switching scheme among scalable video bitstreams is proposed in this paper. It can significantly improve the performance of video streaming over a broad range of bit rates by fully taking advantage of both the high coding efficiency of nonscalable bitstreams and the flexibility of scalable bitstreams, where small channel bandwidth fluctuations are accommodated by the scalability of a single scalable bitstream, whereas large channel bandwidth fluctuations are tolerated by flexible switching between different scalable bitstreams. Two main techniques for switching between video bitstreams are proposed. Firstly, a novel coding scheme is proposed to enable drift-free switching at any frame from the current scalable bitstream to one operated at lower rates without sending any overhead bits. Secondly, a switching-frame coding scheme is proposed to greatly reduce the number of extra bits needed for switching from the current scalable bitstream to one operated at higher rates. Compared with existing approaches, such as switching between nonscalable bitstreams and streaming with a single scalable bitstream, our experimental results clearly show that the proposed scheme brings higher efficiency and more flexibility in video streaming.     

Dyadic spatial resolution reduction transcoding for H.264/AVC

In this paper, we examine spatial resolution downscaling transcoding for H.264/AVC video coding. A number of advanced coding tools limit the applicability of techniques, which were developed for previous video coding standards. We present a spatial resolution reduction transcoding architecture for H.264/AVC, which extends open-loop transcoding with a low-complexity compensation technique in the reduced-resolution domain. The proposed architecture tackles the problems in H.264/AVC and avoids visual artifacts in the transcoded sequence, while keeping complexity significantly lower than more traditional cascaded decoder–encoder architectures. The refinement step of the proposed architecture can be used to further improve rate-distortion performance, at the cost of additional complexity. In this way, a dynamic-complexity transcoder is rendered possible. We present a thorough investigation of the problems related to motion and residual data mapping, leading to a transcoding solution resulting in fully compliant reduced-size H.264/AVC bitstreams.     

基于H.264-SS与FGS结合无漂移比特流切换方案

流媒体应用中,为了满足可用带宽的变化常需要动态地调整比特流码率。由于H.264/AVC支持同步预测(SP)帧,并允许在大范围内的不同质量比特流之间进行切换,而MPEG-4又支持FGS编码,使得比特流小范围可调,因此为了充分利用两者的优点,给出一种将两者融合在一起的解决方案,并采用自适应码率选择方法,使传输的比特流既能适应网络传输带宽的大范围变化,又能灵活适应小范围的带宽波动,且平均峰值信噪比值可较明显提高。     

基于H.264 SVC的视频分层加密算法研究

随着可伸缩视频编码SVC(Scalable Video Coding)的普及,针对可伸缩视频的安全问题也越来越受到重视。目前大部分加密方案都是针对H.264/AVC编码标准来设计的。分析SVC新采用的技术,并率先提出了一种针对其特点的新的分层加密方案。该方案选取不同以往的视频关键信息,如层间预测运动向量,质量可分级关键帧等作为加密对象,根据所选信息的类型,结合用户所需求的安全等级,进行算法设计。试验结果表明该方案具有加密效果好、密钥量较低、实时性高等优点,并可以适应不同安全性需求的应用场合。     

Incremental rate control for H.264 AVC scalable extension

The emerging H.264 Scalable Video Coding (H.264/SVC) requires the rate control algorithm to regulate the output bit rate of all the coarse-grain-scalability, temporal, spatial and combined enhancement layers. In order to address this topic, in this research, we propose an incremental rate control algorithm for H.264/SVC to control each layer’s encoding rate close to the target bit rate. The proposed algorithm introduces a number of efficient methods. First, based on our previous work on H.264/AVC rate control, a Rate-Complexity-Quantization (R-C-Q) model is extended in scalable video coding. Second, a complexity measure for Intra-frames based on their gradient and histogram information is used to precisely determine Quantization Parameters (QPs) for Intra-frames using the R-C-Q model. Third, we adopt an incremental approach to compute QPs of inter-frames. Fourth, a Proportional + Integral + Derivative (PID) buffer controller is presented to provide robust buffer control for each layer of H.264/SVC bitstream. Finally the QPs for hierarchical B-frames are adaptively decided by their neighbor inter-frames. Our extensive simulation results demonstrate that, our algorithm outperforms JVT-W043 rate control algorithm, adopted in the H.264/SVC reference software, by providing more accurate output bit rate for each layer, maintaining stable buffer fullness, reducing frame skipping finally, improving the overall coding quality.     

H.264视频码流分级传输和抗误码性能的研究

网络异构性是面向网络传输的视频编码面临的最大挑战之一,而分层可扩展性编码则在一定程度上解决了这一问题。作为最新的视频编码标准,H.264/AVC很好地继承和发展了该项技术,本文在研究各种分层编码在H.264/AVC标准中的应用情况的基础上,结合差错非均匀保护(UEP-UnequalErrorProtection)技术提出了一种联合可扩展性编码及网络传输方案,实验结果表明该方案提供了更宽范围的可扩展性并具有良好的抗误码性能。     

基于H.264扩展的可伸缩编码技术最新进展

联合视频组(JVT)最近已经完成了基于H.264/AVC标准的可伸缩编码SVC(scalable video coding)标准的第一阶段的制订,并将其作为H.264的第三部分扩展的草稿[2]发布.本文简述了SVC的标准的最新发展和编码架构,重点讲述其中关键的技术.     

Fast video transcoding from H.263 to H.264/MPEG-4 AVC

In the past 10 years detailed works on different video transcoders have been published. However, the new ITU-T Recommendation H.264—also adapted as ISO/IEC MPEG-4 Part 10 (AVC)—provides many new encoding options for the prediction processes that lead to difficulties for low complexity transcoding. In this work we present very fast transcoding techniques to convert H.263 bitstreams into H.264/AVC bitstreams. We will give reasoning, why the proposed pixel domain approach is advantageous in this scenario instead of using a DCT domain transcoder. Our approach results in less than 9% higher data rate at equivalent PSNR quality compared to a full-search approach. But this rate loss allows the reduction of the search complexity by a factor of over 200 for inter frames and still a reduction of over 70% for intra frames. A comparison to a fast search algorithm is given. We also provide simulation results that our algorithm works for transcoding MPEG-2 to H.264/AVC in the aimed scenario.

Semantic scalability using tennis videos as examples

With advances in broadcasting technologies, people are now able to watch videos on devices such as televisions, computers, and mobile phones. Scalable video provides video bitstreams of different size under different transmission bandwidths. In this paper, a semantic scalability scheme with four levels is proposed, and tennis videos are used as examples in experiments to test the scheme. Rather than detecting shot categories to determine suitable scaling options for Scalable Video Coding (SVC) as in previous studies, the proposed method analyzes a video, transmits video content according to semantic priority, and reintegrates the extracted contents in the receiver. The purpose of the lower bitstream size in the proposed method is to discard video content of low semantic importance instead of decreasing the video quality to reduce the video bitstream. The experimental results show that visual quality is still maintained in our method despite reducing the bitstream size. Further, in a user study, we show that evaluators identify the visual quality as more acceptable and the video information as clearer than those of SVC. Finally, we suggest that the proposed scalability scheme in the semantic domain, which provides a new dimension for scaling videos, can be extended to various video categories.     

Dynamic Resource Allocation for MGS H.264/AVC Video Transmission Over Link-Adaptive Networks

In this paper, we address the problem of efficiently allocating network resources to support multiple scalable video streams over a constrained wireless channel. We present a resource allocation framework that jointly optimizes the operation of the link adaptation scheme in the physical layer (PHY), and that of a traffic control module in the network or medium access control (MAC) layer in multirate wireless networks, while satisfying bandwidth/capacity constraints. Multirate networks, such as IEEE 802.16 or IEEE 802.11, adjust the PHY coding and modulation schemes to maintain the reliability of transmission under varying channel conditions. Higher reliability is achieved at the cost of reduced PHY bit-rate which in turn necessitates a reduction in video stream bit-rates. The rate reduction for scalable video is implemented using a traffic control module. Conventional solutions operate unaware of the importance and loss tolerance of data and drop the higher layers of scalable video altogether. In this paper, we consider medium grain scalable (MGS) extension of H.264/AVC video and develop new rate and distortion models that characterize the coded bitstream. Performance evaluations show that our proposed framework results in significant gains over existing schemes in terms of average video PSNR that can reach 3 dB in some cases for different channel SNRs and different bandwidth budgets.      

A scalable H.264/AVC deblocking filter architecture

The deblocking filter (DF) is one of the most complex functional cores of the H.264/AVC and SVC codecs. Its computational cost is heavily dependent on the video profile and the selected scalability level. With the goal of providing faster and better solutions, developers are focused on designing hardware architectures. Thus, it is possible taking advantage of multitasking, reusability and parallelization techniques. In this context, this work proposes a scalable DF architecture that is able to adapt its structure and performance to different video configurations, due to its modular and regular structure. The scalability feature avoids redesigning the whole architecture in case of the environmental demands or the configuration settings change. These facts mean savings in terms of design productivity and silicon area by adapting the necessary logical resources to each condition. Furthermore, regarding the data dependences involved in the H.264/AVC DF algorithm, the proposed architecture relies on an improved version of a traditional wavefront parallelization strategy, also proposed by the authors. This solution reduces the amount of clock cycles needed to filter a video frame as compared to traditional strategies. Implementation results, in an FPGA Virtex-5, demonstrate the performance benefits of this flexible solution as compared to some rigid state-of-the-art deblocking filter approaches.     

基于可伸缩编码的高清IPTV系统

随着移动互联网技术与智能手持设备的发展,IPTV面临多屏融合的需求,可伸缩编码技术可以很好地解决网速和设备分辨率的差异问题．本文设计和实现了一套基于H264／SVC标准的高清实时IPTV系统,通过对视频在时间、空间、质量上进行可伸缩编码,以及TS解封装、AVC解码和TS流传输H264／SVC等模块的实现,通过实时编码,使拥有不同带宽和设备分辨率的用户能并发实时地观看节目．实验证明了系统的有效性和稳定性．     

An improved 3D wavelet-based scalable video coding codec for MC-EZBC

With the rapid growth of modern multimedia applications, 3D wavelet-based scalable video coding (SVC) codec has received considerable attention lately because of its high coding performance and flexibility in bitstream scalability. It combines the motion-compensated temporal filtering (MCTF) together with the spatial decomposition to produce an embedded bitstream offering various levels of video quality over the heterogeneous networks. However, in the existing 3D wavelet-based SVC schemes, where the block types for block matching algorithms are limited, weighting matrices for block-wise motion compensation are fixed, and variations in activities of temporal subbands are not considered in the selection of the Lagrange multiplier for mode decision. In this paper, our major contribution is to provide some recent extensions to the well-known scalable subband/wavelet video codec Motion-Compensated Embedded Zero Block Coding (MC-EZBC) using three novel and content adaptive algorithms. Firstly, the enhanced hierarchical variable size block matching (Enhanced HVSBM) algorithm is proposed for the variable block size motion estimation. Then, the rate-distortion optimization (RDO) based adaptive Lagrange multiplier selection model for mode decision is presented. Finally, we introduce the adaptive weighting matrices design for overlapped block motion compensation (OBMC). Experimental results show that all the three proposed algorithms significantly improve the overall coding performance of MC-EZBC. Comparisons with other popular wavelet-based SVC codecs demonstrate the effectiveness of our improved codec in terms of both video quality assessment and computational complexity.     

Fast macroblock mode decision for H.264/AVC baseline profile video transcoder based on support vector machines

In video transcoding, accuracy and efficiency of macroblock mode decision are critical issues at the re-encoder side due to the changes in frame size, frame rate, and bit rate. In this paper, a fast macroblock mode decision scheme based on support vector machines is proposed for H.264/AVC baseline profile video transcoder. Features including motion vectors, residual data, pre-encoded macroblock modes, and quantization parameters are extracted from incoming bitstream in both of training stage and classification stage. Feature extraction methods are investigated for spatial resolution transcoder, temporal resolution transcoder, and bit-rate transcoder. After off-line training and simplification of support vectors, the obtained support vector machine classifier can determine macroblock mode in the re-encoder accurately. Extensive experiments are carried out on different types of transcoders and results show that the proposed method can save about 80% in computational complexity compared to full mode search algorithm implemented in the latest H.264/AVC reference software (JM17.1), while maximum peak signal-to-noise ratio is degraded by 0.2–1.1?dB depending on different sequences and bit rate.     

Multilayer transcoding with format portability for multicasting of single-layered video

This paper proposes a novel multilayer video transcoding approach for multicasting pre-encoded video to heterogeneous end-systems via diverse grouping of networks. Multilayer transcoding is first addressed by means of multiquality or SNR scalability of the MPEG-2 standard. Frequency domain transcoding and drift-compensated transcoding are derived from the closed-loop and multiloop SNR scalabilities, respectively. The proposed transcoding architectures are verified in terms of eliminating picture drift whilst preserving compatibility with the MPEG-2 SNR decoder. Multilayer transcoding is then addressed by means of multiresolution or spatial scalability of the MPEG-2 standard that supports different video formats. The transcoder retains the full resolution of the incoming video stream in its enhancement layer while generating a low spatio-temporal resolution base-layer compatible with the H.263 video format. Hence providing both multilayer transcoding and video format portability. The resultant video layers are shown to be free from drift with PSNR results comparable to those of the respective scalable encoders.