A pixel-wise directional intra prediction method

In a decade where the prominence of video applications has become increasingly prevalent, the optimization of video coding processes remains as important as ever. The directional intra prediction has been proved as an effective tool for intra picture coding. However, in a directional intra prediction based coding structure, there is a considerable gap between the transform efficiency of DCT and the optimal transform KLT. In this paper, we analyze these problems and propose a pixel-wise directional intra prediction (PDIP) method to solve this problem. This method exploits the reconstruction value of the adjacent pixels to predict the current pixel, and keeps the block-based lossy coding structure of H.264/AVC. The proposed method can significantly improve the coding efficiency with little decoding complexity increment.     

Implicit Line-Based Intra 16×16 Prediction for H.264/AVC High-Quality Video Coding

In this paper, we propose an implicit line-based linear intra 16×16 prediction method for high-quality video coding. Typically, in H.264/AVC intra 16×16 vertical and horizontal prediction modes, 256 pixels in the current macroblock are predicted using 16 adjacent boundary pixels of reconstructed neighboring macroblocks. One problem of such a block-based intra prediction is that the prediction error increases as the distance between the reference and current pixels increases. Thus, the prediction accuracy of intra 16×16 vertical and horizontal modes is not sufficient for the mode decision stage. To ensure that the pixels are close to their predictor, we propose a new implicit intra prediction scheme, which makes full use of the correlation between lines instead of blocks. In the proposed method, we perform prediction, transformation, quantization, and reconstruction using the line-of-pixels (LOP) to improve the prediction accuracy. Experimental results show that the proposed algorithm provides an approximately 6.42?% bit-rate reduction compared to the H.264/AVC FRExt high profile, while maintaining the same decoding quality.     

Improved lossless intra coding for H.264/MPEG-4 AVC.

A new lossless intra coding method based on sample-by-sample differential pulse code modulation (DPCM) is presented as an enhancement of the H.264/MPEG-4 AVC standard. The H.264/AVC design includes a multidirectional spatial prediction method to reduce spatial redundancy by using neighboring samples as a prediction for the samples in a block of data to be encoded. In the new lossless intra coding method, the spatial prediction is performed based on samplewise DPCM instead of in the block-based manner used in the current H.264/AVC standard, while the block structure is retained for the residual difference entropy coding process. We show that the new method, based on samplewise DPCM, does not have a major complexity penalty, despite its apparent pipeline dependencies. Experiments show that the new lossless intra coding method reduces the bit rate by approximately 12% in comparison with the lossless intra coding method previously included in the H.264/AVC standard. As a result, the new method is currently being adopted into the H.264/AVC standard in a new enhancement project.     

Edge-oriented uniform intra prediction

We propose an intra prediction solution to block-based image compression. In order to adapt to local image features during intra prediction, we consider the distinct image singularities within the model of piece-wise smooth functions. With such singularities, i.e., edges in this paper, intra prediction can be performed by solving Laplace equations. Moreover, since edges exhibit spatial correlations, we design a rate-distortion optimized method for edge extraction and edge coding. Our edge-oriented intra prediction thus consists of the prediction of smooth regions as well as the prediction of edges. We compare our intra prediction with that in H.264 and achieve superior performance. Our intra prediction can also be integrated into a block-based image coding scheme, which is comparable to JPEG2000 in terms of objective quality. An important advantage of our intra prediction is the improvement in visual quality at low bit-rate due to the preservation of edges.     

Architectural Decomposition of Video Decoders by Meansof an Intermediate Data Stream Format

The microprocessor industry trend towards many-core architectures introduced the necessity of devising appropriately scalable applications. While implementing software based video decoding, the main challenges are the optimized partitioning of decoder operations, efficient tracking of dependencies and resource synchronization for multiple parallel units. The same applies for hardware implementations of video decoders where monolithic approaches anticipate scalability of the design and reusability of already implemented core components.In this paper, we propose an intermediate data stream format (Meta Format Stream) which is suited for architectural decomposition of video decoding by replacing the conventional monolithic decoder architecture design with a pipelined structure. The Meta Format is forward-oriented and self contained and multistandard capable, so that processing of Meta Streams is independent of the originating bit stream. Our approach does not require special coding settings and is applicable to accelerated decoding of any standards-compliant bit stream. A H.264/AVC multiprocessing proposal is presented as a case study for the potential our our concept. The case study combines coarse grained frame-level parallel decoding of the bit stream with fine-grained macroblock level parallelism in the image processing stage.The proposed H.264 decoder achieved speedup factors of up to 7.6 on an 8 core machine with 2-way SMT. We are reporting actual decoding speeds of up to 150 frames per second in 2160p-resolution.     

Efficient error resilient algorithm for H.264/AVC: mobility management in wireless video streaming

The H.264/AVC standard introduces enhanced error robustness capabilities enabling resilient and reliable transmission of compressed video signals over wireless lossy packet networks. Those robustness capabilities are achieved by integrating some new error resilience tools that are essential for a proper delivery of real-time video services. Those tools include the Intra Refreshing (IR), Arbitrary Slice Ordering (ASO), Sequence Picture Parameter Sets (PPS), Redundant Slices (RS) tools and Flexible Macroblock Ordering (FMO). This paper presents an error resilient algorithm in wireless H.264/AVC streaming. The proposed method merges Reference Frame Selection (RFS), Intra Redundancy Slice and Adaptive Intra Refreshment techniques in order to prevent temporal error propagation in error-phone wireless video streaming. The coding standards only specify the decoding process and the bitstream syntax to allow considerable flexibility for the designers to optimize the encoder for coding performance improvement and complexity reduction. Performance evaluations demonstrate that the proposed encoding algorithm outperforms the conventional H.264/AVC standard. Both subjective and objective visual quality comparative study has been also carried out in order to validate the proposed approach. The proposed method can be used and integrated into H264/AVC without violating the standard.     

A Segmentation-Based Chroma Intra Prediction Coding Scheme for H.264/AVC

In this paper, we propose a novel segmentation-based intra prediction coding scheme for low-bitrate video coding. Different coding schemes are separately designed for the luma and chroma components in our proposed method. The traditional block-based coding scheme is still used for the luma components, and the segmentation-based coding scheme is developed for the chroma components. The segmentation operation is used for the reconstructed luma components, which groups similar pixels together and produces a set of homogenous regions. Here, these local and homogenous regions are referred to superpixels. By utilizing the spatial correlation between the luma and chroma planes, we transfer the segmentation result of the luma components to the chroma components, which will not induce any side information in the chroma intra prediction coding. Instead of using the macroblock (MB) as the coding unit, the proposed method implements the chroma intra prediction in each superpixel, and the original pixels in each superpixel are employed to substitute the neighboring reconstructed samples in the prediction process. The experimental results show that the proposed method can achieve an average 0.20 dB and up to 0.63 dB coding gains in comparison to the directional intra prediction scheme for H.264/AVC low-bitrate video coding.     

Mode-Dependent Templates and Scan Order for H.264/AVC-Based Intra Lossless Coding

In H.264/advanced video coding (AVC), lossless coding and lossy coding share the same entropy coding module. However, the entropy coders in the H.264/AVC standard were original designed for lossy video coding and do not yield adequate performance for lossless video coding. In this paper, we analyze the problem with the current lossless coding scheme and propose a mode-dependent template (MD-template) based method for intra lossless coding. By exploring the statistical redundancy of the prediction residual in the H.264/AVC intra prediction modes, more zero coefficients are generated. By designing a new scan order for each MD-template, the scanned coefficients sequence fits the H.264/AVC entropy coders better. A fast implementation algorithm is also designed. With little computation increase, experimental results confirm that the proposed fast algorithm achieves about 7.2% bit saving compared with the current H.264/AVC fidelity range extensions high profile.     

Efficient 45nm ASIC Architecture for Full-Search Free Intra Prediction in Real-Time H.264/AVC Decoder

The standard H.264/AVC Intra frame encoding process has several data dependent and computational intensive coding methodologies that limit the overall encoding speed. It causes not only a high degree of computational complexity but also an unacceptable delay especially for the real-time video applications. Based on DCT properties and spatial activity analysis, low power hardware architecture for high throughput Full-Search Free (FSF) Intra mode selection and direction prediction algorithm is proposed. The FSF Intra prediction Algorithm significantly reduces the computational complexity and the processing run-time required for the H.264/AVC Intra frame prediction process. The ASIC implementation for the proposed architecture is carried out and synthesizing results are obtained. The heavily tested 45nm ASIC design is able to achieve an operating frequency of 140 MHz while limiting the overall power consumption to 9.01 mW, which nominates our proposed FSF Intra prediction architecture for interactive real-time H.264/AVC mobile video decoders.     

Highly Efficient Video Codec for Entertainment‐Quality

We present a novel video codec for supporting entertainment‐quality video. It has new coding tools such as an intra prediction with offset, integer sine transform, and enhanced block‐based adaptive loop filter. These tools are used adaptively in the processing of intra prediction, transform, and loop filtering. In our experiments, the proposed codec achieved an average reduction of 13.35% in BD‐rate relative to H.264/AVC for 720p sequences.     

H.264/AVC解码中帧内预测的硬件实现

根据H．264／AVC的特点，设计出一种适合于帧内预测解码的硬件实现方式，并且引入了帧场自适应模式，有利于提高解码效率，并将该结构配合其他设计好的解码器模块，在FPGA上实现了标准清晰度的H．264视频的实时解码。     

Edge-preserving Intra mode for efficient depth map coding based on H.264/AVC

Depth-image-based-rendering (DIBR) algorithms for 3D video communication systems based on the “multi-view video plus depth” format are very sensitive to the accuracy of depth information. Specifically, edge regions in the depth data should be preserved in the coding/decoding process to ensure good view synthesis performance, which directly affects the overall system performance. This paper proposes a novel scheme for edge-aware Intra depth compression based on the H.264/AVC framework enabled on both Intra (I) and Inter (P) slices. The proposed scheme includes a new Intra mode specifically targeted to depth macroblocks with arbitrarily shaped edges, which are typically not predicted well by the standard Intra modes of H.264/AVC and result in high rate–distortion costs. The proposed algorithm segments edge macroblocks into two regions each approximated by a flat surface. A binary mask identifying the two regions is defined and encoded by means of context-coding with adaptive template selection. As a novel contribution, the proposed mode allows exploiting the correlation with causal neighboring edge macroblocks to improve the performance of context-coding of binary masks and allow significant bit rate savings. The proposed method has been exhaustively compared with different state-of-the-art algorithms for edge-aware depth coding and the results highlight significant improvements in most of the cases, both in terms of reconstructed depth quality, view synthesis performance, and overall texture plus depth rate–distortion performance.     

A low-complexity image compression approach with single spatial prediction mode and transform

The well-known low-complexity JPEG and the newer JPEG-XR systems are based on block-based transform and simple transform-domain coefficient prediction algorithms. Higher complexity image compression algorithms, obtainable from intra-frame coding tools of video coders H.264 or HEVC, are based on multiple block-based spatial-domain prediction modes and transforms. This paper explores an alternative low-complexity image compression approach based on a single spatial-domain prediction mode and transform, which are designed based on a global image model. In our experiments, the proposed single-mode approach uses an average 20.5 % lower bit-rate than a standard low-complexity single-mode image coder that uses only conventional DC spatial prediction and 2-D DCT. It also does not suffer from blocking effects at low bit-rates.     

Fast Interframe mode decision algorithm based on mode mapping and MB activity for MPEG-2 to H.264/AVC transcoding

Recently the latest video coding standard H.264/AVC is widely used for the mobile and low bitrate video codec in the various multimedia terminals. On the other hand, the MPEG-2 MP@HL codec has become the center of digital video contents since it is the standard codec for the Digital TV (DTV). To provide the bridge between the contents in MPEG-2 and mobile terminals, the transcoding of MPEG-2 contents into H.264/AVC format is an inevitable technology in the digital video market. The main bottleneck in the process lies in the computational complexity. In H.264/AVC, the variable block size (VBS) mode decision (MD) is used in the Interframe for the improved performance in the motion compensated prediction. For the macroblock (MB) which cannot be accurately predicted with one motion vector (MV), it is partitioned into smaller blocks and predicted with different MVs. In addition, SKIP and Intra modes are also permitted in the Interframe MD of H.264/AVC to further ameliorate the encoding performance. With the VBS MD technology, the Inter prediction accuracy can be improved significantly. However, the incidental side-effect is the high computational complexity. In this paper, we propose a fast Interframe MD algorithm for MPEG-2 to H.264/AVC transcoding. The relationships between SKIP and Intra modes are detected at first to map these two kinds of modes directly from MPEG-2 to H.264/AVC. And then the MB activity will be scaled by the residual DCT energy obtained from the MPEG-2 decoding process to estimate the block sizes of the MB mode for H.264/AVC Interframe MD. In our proposed method, the original redundant candidate modes can be eliminated effectively, resulting in the reduction of the computational complexity. It can reduce about 85% Rate-to-Distortion Cost (RDCost) computing and 45% entire processing time compared with the well-known cascaded transcoder while maintaining the video quality.     

A high performance and low complexity sampling-based intra coding method for H.264/AVC

The H.264/AVC video coding standard can achieves higher compression performance than previous video coding standards, such as MPEG-2, MPEG-4, and H.263. Especially, in order to obtain the high coding performance in intra pictures, the H.264/AVC encoder employs various directional spatial prediction modes and the rate-distortion (RD) optimization technique inducing high computational complexity. For further improvement in the coding performance with the low computational complexity, we introduce a sampling-based intra coding method. The proposed method generates two sub-images, which are defined as a sampled sub-image and a prediction error sub-image in this paper, from an original image through horizontal or vertical sampling and prediction processes, and then each sub-image is encoded with different intra prediction modes, quantization parameters, and scanning patterns. Experimental results demonstrate that the proposed method significantly improves the intra coding performance and reduces the encoding complexity with the smaller number of the RD cost calculation process.     

Efficient Level and Zero Coding Methods for H.264/AVC Lossless Intra Coding

Since H.264/AVC was designed mainly for lossy video coding, the entropy coding methods in H.264/AVC are not appropriate for lossless video coding. Based on statistical differences of residual data in lossy and lossless coding, we develop efficient level and zero coding methods. Therefore, we design an improved context-based adaptive variable length coding (CAVLC) scheme for lossless intra coding by modifying the relative entropy coding parts in H.264/AVC. Experimental results show that the proposed method provides approximately 6.8% bit saving, compared with the H.264/AVC FRExt high profile.      

Error Concealment Using Intra‐Mode Information Included in H.264/AVC‐Coded Bitstream

The H.264/AVC standard has adopted new coding tools such as intra‐prediction, variable block size, motion estimation with quarter‐pixel‐accuracy, loop filter, and so on. The adoption of these tools enables an H.264/AVC‐coded bitstream to have more information than was possible with previous standards. In this paper, we propose an effective spatial error concealment method with low complexity in H.264/AVC intra‐frame. From information included in an H.264/AVC‐coded bitstream, we use prediction modes of intra‐blocks to recover a damaged block. This is because the prediction direction in each prediction mode is highly correlated to the edge direction. We first estimate the edge direction of a damaged block using the prediction modes of the intra‐blocks adjacent to a damaged block and classify the area inside the damaged block into edge and flat areas. Our method then recovers pixel values in the edge area using edge‐directed interpolation, and recovers pixel values in the flat area using weighted interpolation. Simulation results show that the proposed method yields better video quality than conventional approaches.     

H.264/AVC的精度范围扩展

H.264/AVC标准增加了精度范围扩展Frext部分,提高了HD编码效率,改善了视频质量,扩大了其应用范围,同时对编解码器设计提出了更高要求.介绍Frext的基本框架和编码工具,并与JPEG2000、MPEG-2在高清视频图像压缩性能进行对比,进一步说明Frext具有好的编码性能.     

Restricted H.264/AVC video coding for privacy protected video scrambling

The issue of personal privacy has garnered significant attention with the extensive application of video surveillance systems. Privacy region scrambling is an effective method to protect privacy in video. To ensure that nonprivacy regions are not affected by scrambling, particular methods must be taken to prevent drift error in privacy protected video scrambling. However, existing methods have significantly reduced the coding efficiency. In this paper, we focus on improving coding efficiency while preventing drift error in privacy protected H.264/AVC video scrambling, which is the state-of-the-art coding standard. A restricted video coding scheme is proposed, which involves three parts of Mode Restricted Intra Prediction (MRIP), Search Window Restricted Motion Estimation (SWRME) and Boundary Strength Restricted Deblocking Filtering (BSRDF). Experimental results show that the proposed restricted video coding scheme prevents drift error with higher coding efficiency than others.     

Robust Video Watermarking of H.264/AVC

A robust video watermarking scheme of the state-of-the-art video coding standard H.264/AVC is proposed in this brief. 2-D 8-bit watermarks such as detailed company trademarks or logos can be used as inconvertible watermark for copyright protection. A grayscale watermark pattern is first modified to accommodate the H.264/AVC computational constraints, and then embedded into video data in the compressed domain. With the proposed method, the video watermarking scheme can achieve high robustness and good visual quality without increasing the overall bit-rate. Experimental results show that our algorithm can robustly survive transcoding process and strong common signal processing attacks, such as bit-rate reduction, Gaussian filtering and contrast enhancement