Rate control for consistent visual quality of H.264/AVC encoding

In this paper, a novel rate control scheme with sliding window basic unit is proposed to achieve consistent or smooth visual quality for H.264/AVC based video streaming. A sliding window consists of a group of successive frames and moves forward by one frame each time. To make the sliding window scheme possible for real-time video streaming, the initial encoder delay inherently in a video streaming system is utilized to generate all the bits of a window in advance, so that these bits for transmission are ready before their due time. The use of initial encoder delay does not introduce any additional delay in video streaming but benefits visual quality as compared to traditional one-pass rate control algorithms of H.264/AVC. Then, a Sliding Window Buffer Checking (SWBC) algorithm is proposed for buffer control at sliding window level and it accords with traditional buffer measurement of H.264/AVC. Extensive experimental results exhibit that higher coding performance, consistent visual quality and compliant buffer constraint can be achieved by the proposed algorithm.     

SSIM-based error-resilient rate-distortion optimization of H.264/AVC video coding for wireless streaming

The SSIM-based rate-distortion optimization (RDO) has been verified to be an effective tool for H.264/AVC to promote the perceptual video coding performance. However, the current SSIM-based RDO is not efficient for improving the perceptual quality of the video streaming application over the error-prone network, because it does not consider the transmission induced distortion in the encoding process. In this paper, a SSIM-based error-resilient RDO scheme for H.264/AVC is proposed to improve the wireless video streaming performance. Firstly, with the help of the SSE-based RDO, we present a low-complexity Lagrange multiplier decision method for the SSIM-based RDO video coding in the error-free environment. Then, the SSIM-based decoding distortion of the user end is estimated at the encoder and is correspondingly introduced into the RDO to involve the transmission induced distortion into the encoding process. Further, the Lagrange multiplier is theoretically derived to optimize the encoding mode selection in the error-resilient RDO process. Experimental results show that the proposed SSIM-based error-resilient RDO can obtain superior perceptual video quality (more structural information) to the traditional SSE-based error-resilient RDO for wireless video streaming at the same bit rate condition.     

A novel rate control algorithm for video coding based on fuzzy-PID controller

Rate control algorithms (RCAs) aim to achieve the best visual quality under the minimum bit rate and the limited buffer size. A self-parameter-tuning fuzzy-PID controller is proposed to reduce the deviation between the target buffer level and the current buffer fullness. Fuzzy logic is used to tune each parameter of the proportional-integral-derivative controller by selecting appropriate fuzzy rules through simulation in H.264/advanced video coding (AVC). To control the quality fluctuation between consecutive frames, a quality controller is adopted. The proposed RCA has been implemented in an H.264/AVC video codec, and our experimental results show that the proposed algorithm achieves smooth target bits while enabling better buffer control and visual quality.     

一种面向H.264/AVC的码率控制算法

码率控制是视频编码中非常重要的技术之一,任何标准离开码率控制其应用都会受到限制.H.264/AVC是目前最新的视频编码标准,本文根据H.264/AVC编码标准的特性及其HRD部分对码率控制的要求,提出了一种新的适合H.264/AVC的码率控制算法,该算法实现了率失真优化与码率控制的结合,使得在达到码率控制的同时也能保证较高的编码效率,同时在码率控制的过程中根据HRD缓冲区状态进行位分配调整,保证了编解码缓冲区既不上溢又不下溢.该算法作为技术提案已被H.264/AVC接受,并集成到H.264/AVC的校验模型软件中.     

Efficient error resilient algorithm for H.264/AVC: mobility management in wireless video streaming

The H.264/AVC standard introduces enhanced error robustness capabilities enabling resilient and reliable transmission of compressed video signals over wireless lossy packet networks. Those robustness capabilities are achieved by integrating some new error resilience tools that are essential for a proper delivery of real-time video services. Those tools include the Intra Refreshing (IR), Arbitrary Slice Ordering (ASO), Sequence Picture Parameter Sets (PPS), Redundant Slices (RS) tools and Flexible Macroblock Ordering (FMO). This paper presents an error resilient algorithm in wireless H.264/AVC streaming. The proposed method merges Reference Frame Selection (RFS), Intra Redundancy Slice and Adaptive Intra Refreshment techniques in order to prevent temporal error propagation in error-phone wireless video streaming. The coding standards only specify the decoding process and the bitstream syntax to allow considerable flexibility for the designers to optimize the encoder for coding performance improvement and complexity reduction. Performance evaluations demonstrate that the proposed encoding algorithm outperforms the conventional H.264/AVC standard. Both subjective and objective visual quality comparative study has been also carried out in order to validate the proposed approach. The proposed method can be used and integrated into H264/AVC without violating the standard.     

Video coding with H.264/AVC: tools, performance, and complexity

H.264/AVC, the result of the collaboration between the ISO/IEC Moving Picture Experts Group and the ITU-T Video Coding Experts Group, is the latest standard for video coding. The goals of this standardization effort were enhanced compression efficiency, network friendly video representation for interactive (video telephony) and non-interactive applications (broadcast, streaming, storage, video on demand). H.264/AVC provides gains in compression efficiency of up to 50% over a wide range of bit rates and video resolutions compared to previous standards. Compared to previous standards, the decoder complexity is about four times that of MPEG-2 and two times that of MPEG-4 Visual Simple Profile. This paper provides an overview of the new tools, features and complexity of H.264/AVC.     

基于代价策略的H.264/AVC半脆弱水印算法

In order to solve some existing problems on detecting malicious tampering by video watermarking based on H.264/AVC video compression standard,a semi-fragile video watermarking algorithm for H.264/AVC based on cost strategy was proposed.To further improve the robustness of the feature codes in semi-fragile video watermarking algorithm,a new generation algorithm for feature codes by eliminating the correlation of them was proposed.In order to meet the requirement that the semi-fragile watermarks should be uniformly embedded into the video frames to locate the tampered region accurately,a minimal cost function was introduced to select a DCT coefficient in the 4 × 4 block residual to embed watermark which had minimal impact on the video quality and bit rates.Experimental results show that the proposed algorithm can significantly reduce impact on video quality and bit rate because of embedding the watermarking by adopting the minimum cost function and it is sensitive to malicious manipulations.The retrieved feature code and watermarking can detect and locate malicious manipulations.     

H.264/AVC video for wireless transmission

H.264/AVC will be an essential component in emerging wireless video applications thanks to its excellent compression efficiency and network-friendly design. However, a video coding standard itself is only one component within the application and transmission environment. Its effectiveness strongly depends on the selection of appropriate modes and parameters at the encoder, at the decoder, as well as in the network. In this paper we introduce the features of the H.264/AVC coding standard that make it suitable for wireless video applications, including features for error resilience, bit rate adaptation, integration into packet networks, interoperability, and buffering considerations. Modern wireless networks provide many different means to adapt quality of service, such as forward error correction methods on different layers and end-to-end or link layer retransmission protocols. The applicability of all these encoding and network features depends on application constraints, such as the maximum tolerable delay, the possibility of online encoding, and the availability of feedback and cross-layer information. We discuss the use of different coding and transport related features for different applications, namely video telephony, video conferencing, video streaming, download-and-play, and video broadcasting. Guidelines for the selection of appropriate video coding tools, video encoder and decoder settings, as well as transport and network parameters are provided and justified. References to relevant research publications and standardization contributions are given.     

Online rate control for video streams

A mechanism for varying the frame rate of pre-encoded video clips online is described. The mechanism relies on two different encoders. An offline encoder creates a high-quality bit stream encoded at 30 fps, as well as separate files containing motion vectors for the same clip at lower frame rates. An online encoder decodes the bit stream (if necessary) and re-encodes it at lower frame rates in real time using the precomputed, stored motion information. Online frame rate control, used in conjunction with dynamic bit rate control, allows clients to solve the rate mismatch between the bandwidth available to them and the bit rate of the pre-encoded bit stream. Clients can therefore choose the amount of trade-off between temporal and spatial quality while resolving the rate mismatch problem. Moreover, online frame rate control provides a natural means for implementing a VCR-like fast forward control for video streaming applications, without increasing bandwidth consumption.An earlier version of this paper appeared in Proceedings of ACM Multimedia, 1999, pp. 141–144.     

结合空时域下采样与重建的H.264/AVC压缩性能优化

为提高H.264/AVC标准在带宽资源严重受限时的压缩效率,采用空时域相结合的编码思路,提出了一种基于运动检测的自适应抽帧方法,并结合空域下采样与重建研究了一种改进的H.264/AVC压缩性能优化框架。在编码端,原视频先空域下采样以减少空间分辨率,然后根据视频运动特征,采用不同抽帧模式自适应地降低帧率,再经H.264/AVC编码,有效降低了编码码率。在解码端,解码视频则采用与抽帧模式相对应的运动估计与补偿插帧方法重建出抽取帧,再利用超分辨率重建技术将视频恢复到原空间分辨率。实验结果表明,所提方法在低码率段的视频压缩性能优于H.264/AVC标准编解码及相关文献方法。     

Fast mode decision algorithm for H.264 inter-prediction

A fast mode decision algorithm for H.264/AVC inter-prediction to reduce computational complexity of the H.264 encoder is presented. Experimental results show that the algorithm can save the entire encoding time by 77% on average while introducing only negligible loss in PSNR value and small increment of bit rate.     

Fast Algorithm and Efficient Architecture for Integer and Fractional Motion Estimation

Motion estimation in H.264/AVC, is done in two parts – integer motion estimation, and fractional motion estimation. Hardware reuse for both parts is inefficient due to the differences between them. In this paper we address the hardware reuse problem by proposing a, fast motion estimation algorithm as well as a pipelined FPGA-based, field programmable system-on-chip (FPSoC), for integer and fractional motion estimation. Our results show that the rate-distortion loss of our algorithm is insignificant when compared to full search in H.264/AVC. Its average Y-PSNR loss is 0.065 dB, its average percentage bit rate increase is 5 %, and its power consumption is 76 mW. Our FPSoC is hardware-efficient, even out-performing some state-of-the-art ASIC implementations. It can support up to high definition 1280?×?720p video at 24Hz. Thus, our proposed algorithm and architecture is suitable for delivery of high quality video on low power devices and low bit rate applications which typically use H.264/AVC baseline profile@levels 1–3.1.     

Quality-optimised MPEG2 video data rate control using fuzzy logictechniques

Fuzzy logic control has been employed to improve the rate control mechanism for a MPEG2 video encoder. The data rate of compressed video is controlled by video encoders for either variable bit rate (VBR) or constant bit rate (CBR) applications. In VBR video transmission, it is considered to be more efficient to regulate the video rate by the video coder than by network management in order to avoid network congestion and maintain stable video quality. This rationale can also be applied to CBR transmission. Two fuzzy-logic-based rate control techniques are proposed which maintain the buffer occupancy within a specified range. In the proposed technique for VBR applications, a video quality measure is taken as the crucial control parameter. In CBR rate control, the video data rate or the buffer occupancy is also considered as a fuzzy logic variable. Proposed techniques are designed to control either data rate or video quality, depending on the mode of transmission, i.e. CBR or VBR for the MPEG2 encoder. The performance is compared to a typical VBR MPEG video coder with fixed quantiser step sizes for VBR and also to the CBR video coder with MPEG2 TM5 at typical channel rates. Simulation results are presented with peak signal-to-noise ratio, data rate variation and buffer occupancy as the performance measures     

Encoder adaptable difference detection for low power video compression in surveillance system

As a state-of-the-art video compression technique, H.264/AVC has been deployed in many surveillance cameras to improve the compression efficiency. However, it induces very high coding complexity, and thus high power consumption. In this paper, a difference detection algorithm is proposed to reduce the computational complexity and power consumption in surveillance video compression by automatically distributing the video data to different modules of the video encoder according to their content similarity features. Without any requirement in changing the encoder hardware, the proposed algorithm provides high adaptability to be integrated into the existing H.264 video encoders. An average of over 82% of overall encoding complexity can be reduced regardless of whether or not the H.264 encoder itself has employed fast algorithms. No loss is observed in both subjective and objective video quality.     

H.264帧间模式选择优化算法

提出一种新的H.264快速模式选择算法：根据块的细节程度和纹理方向选择不同模式集，这些模式集比整个模式集模式数量大大减小，从而提高了模式搜索速度。信噪比的降低与比特率的增加都在可接受范围内，仿真结果证明了算法的有效性。     

New rate-distortion modeling and efficient rate control for H.264/AVC video coding

Rate control (RC) is crucial in controlling compression bit rates and qualities for networked video applications. In this paper, we propose a new rate-distortion (R-D) model and an efficient rate control scheme for H.264/AVC video coding, which elegantly resolve the inter-dependency problem between rate-distortion optimization and rate control by eliminating the need of coding complexity prediction for an inter-frame. The objective is to achieve accurate bit rate, obtain optimal video quality while reducing quality variations and simultaneously handling buffer fullness effectively. The proposed algorithm encapsulates a number of new features, including a coding complexity measure for intra-frames, a rate-distortion model, an accurate quantization parameter (QP) estimation for intra-frames, an incremental quantization parameter calculation method for inter-frames, a proportional+integral+derivative (PID) buffer controller, and an intelligent bit-allocation-balancing technique. Our experimental results demonstrate that the proposed scheme outperforms the JVT-G012 solution by providing accurate rate regulation, effectively reducing frame skipping, and finally improving coding quality by up to 1.80 dB.     

Variable and constant bitrate in a DVC to H.264/AVC transcoder

Mobile-to-mobile video communications constitute one of the main research areas dealing with the dynamic adaptation of traffic generated by video sources. In a framework where one mobile device sends video information to another, both transmitter and receiver should employ video encoders and decoders with low complexity. In this paper, a Variable/Constant Bitrate DVC to H.264/AVC Transcoder is proposed which takes the advantage of both paradigms in terms of low-complexity algorithms on the end-user device side (DVC encoder and H.264/AVC decoder). The proposed transcoder is based on the hypothesis that common DVC GOPs can be converted to H.264/AVC GOPs without significant rate-distortion and bitrate losses, in a flexible way. An in-depth study of the different frame types available in DVC has been carried out in order to exploit the correlation between them and the most suitable GOP pattern in H.264/AVC. Moreover, a dynamic motion estimation technique is proposed in this paper for optimizing the search area for the motion vectors, with the purpose of being used in combination with the GOP mapping approach. Simulation results show that the proposed approaches reduce the DVC to H.264/AVC transcoder complexity by up to 60% on average, while maintaining the coding efficiency in CBR and VBR scenarios, achieving very high quality results over different types of metrics (both objective and subjective). Finally, we conduct a comparative study with all the most prominent DVC transcoding proposals available in the literature, showing that the proposed transcoder achieves the best results (in terms of PSNR and bitrate).