基于多视点视频的差错控制技术

多视点视频数据量巨大,除了有效的压缩编码外,解决其在不可靠的信道如Internet和无线网络上传输的差错控制问题也是一个研究热点。针对多视点视频的三种编码结构,初步探讨了目前已经出现或正在研究中的多视点视频差错控制技术,主要集中在其编码端的差错复原编码和解码端的错误隐藏。     

3D电视系统的构成及发展

主要介绍了3D电视系统的现状与发展,按照由双视点向未来多视点3D电视系统发展的技术路线,依次介绍了各阶段系统的架构.同时,对关键技术的基本原理也做了详细说明,包括过度阶段的帧兼容、多视点视频编码、混合分辨率视频编码、视频与相应深度信息联合编码等解决方案.最后对3D电视系统中有待解决的问题和未来的研究方向进行了总结.     

立体视频编码技术的专利申请分析

立体视频编码技术通过挖掘多个视点之间的时空相关性对立体视频进行编码,以提高传输和存储性能,是目前电视技术的研究热点。关注立体视频编码技术的专利申请,基于CPRSABS(中国专利检索系统文摘数据库)和VEN(虚拟外文数据库)数据库中3D编码技术领域的专利文献,从年度申请数量、申请人总体情况、公司方面专利情况以及科研院所方面的专利情况进行多角度分析,并总结关于3D编码的专利申请特点。     

3D多视点立体显示及其关键技术

作为基于 DTV/HDTV 的二维(2D)显示之后的下一代视频显示技术,三维(3D)多视点立体显示已成为国际上的研究热点之一.为建立多视点立体显示系统,阐述了相关的关键技术,包括:光场表示模型和光场获取系统、高效的与现行视频标准兼容的多视点编码和传输方法、解码端任意位置视点的高效绘制方法、3D显示技术以及多视点自由立体显示.针对上述关键技术,分析了当前国际上的发展趋势及存在的问题,同时提出了一种基于交互式自由立体显示的 3D 视频处理系统的解决方案.     

3D-HEVC中深度图编码技术研究进展

3D-HEVC是为了满足3D视频和自由视点视频的高效编码而最新制定的视频编码标准,它要求同时编码几个视点的纹理视频和深度图.完全采用传统的技术来编码深度图会使得深度图内部锐利边界处产生伪影效应,为此,一些新的针对于深度图的编码工具被开发.详细介绍了这些编码工具,同时介绍了编码深度图时所使用的率失真优化方法.     

新一代基于HEVC的3D视频编码技术

HEVC标准出台后,新一代基于HEVC的多视点加深度编码也将正式推出。基于HEVC的3D视频编码作为HEVC标准的扩展部分,主要面向立体电视和自由立体视频。从该编码方式的基本结构出发,较全面地介绍了视频编码方式、深度图编码方式和对深度图的编码控制三个方面的关键技术,包括视点间运动预测、深度图建模模式和视点合成优化等技术。     

多视点与深度视频编码技术研究综述

多视点与深度视频(MVD)可以有效表示自由视点视频,减少了需传输视点的个数,该表达形式正受到越来越多的关注,如何对MVD进行高效编码尤为重要.介绍了自由视点视频的发展现状及国内外研究概况,详细讨论了深度图像的高效编码技术及多视点与深度联合编码技术,并对多视点与深度视频编码技术进行了总结及展望.     

多视点视频编码的研究现状及其展望

阐述了多视点视频编码(MVC)的主要研究问题.其中首先介绍了MVC的体系结构和发展过程;然后详细讨论了MVC的研究内容,包括预测结构、提高MVC编码效率的技术和高层语法;最后在总结MVC研究现状的基础上,提出了MVC在3D视频应用中的研究思路.     

多视点纹理加深度编码的联合码率控制方法

码率控制技术是多视点视频编码和传输中一个关键的问题。为了提高三维(3D)视频的整体显示质量,包括虚拟视点质量和编码视点质量,提出一种多视点纹理加深度编码的联合码率控制方法。该算法研究了纹理和深度的关系,采用基于模型方法确定最优的纹理和深度之间的码率比例。根据各个视点编码结果的统计规律,不同的视频序列采用不同的视点间比特分配比例。实验结果表明,与目前流行的多视点码率控制算法相比,该算法在计算复杂度基本保持不变的情况下,平均码率控制误差在0.6%以内,客观质量PSNR最高可提高0.65 dB。     

基于多视点视频编码的差错控制算法

 多视点视频编码(Multiview Video Coding,MVC)利用运动估计和视差估计取得了较好的编码性能,但在易错的网络环境下传输MVC视频码流,将导致差错在视点内与视点间进行扩散.针对多视点视频的编码特性,提出了一种端到端的失真度估计模型,并将此模型与率失真优化相结合得到一种基于联合信源信道的编码模式选择算法.实验结果表明该方法能够在易错网络环境下有效的提高多视点视频的传输效率.     

基于3D ResNet-LSTM的多视角人体动作识别方法

在基于视频图像的动作识别中,由于固定视角相机所获取的不同动作视频存在视角差异,会造成识别准确率降低等问题。使用多视角视频图像是提高识别准确率的方法之一,提出基于三维残差网络（3D Residual Network,3D ResNet）和长短时记忆（Long Short-term Memory,LSTM）网络的多视角人体动作识别算法,通过3D ResNet学习各视角动作序列的融合时空特征,利用多层LSTM网络继续学习视频流中的长期活动序列表示并深度挖掘视频帧序列之间的时序信息。在NTU RGB+D 120数据集上的实验结果表明,该模型对多视角视频序列动作识别的准确率可达83.2%。     

A Framework for Representation and Processing of Multi-view Video Using the Concept of Layered Depth Image

The multi-view video is a collection of multiple videos, capturing the same scene at different viewpoints. Since it contains more affluent information than a single video, it can be applied to various applications, such as 3DTV, free viewpoint TV, surveillance, sports matches, and so on. However, the data size of the multi-view video linearly increases as the number of cameras, therefore it is necessary to develop an effective framework to represent, process, and transmit those huge amounts of data. In recent, multi-view video coding is getting lots of attention as efficient video coding technologies are being developed. Although most of multi-view video coding algorithms are based on the state-of-the-art H.264/AVC video coding technology, they do not utilize rich 3-D information. In this paper, we propose a new framework using the concept of layered depth image (LDI), one of the efficient image-based rendering techniques, to efficiently represent and process multi-view video data. We describe how to represent natural multi-view video based on the LDI approach and the overall framework to process those converted data.     

3D视频的错误隐藏快速算法

具有独特编码结构的3D视频在不可靠信道上传输时遭遇丢包后极易造成错误传播,因此研究基于3D视频的错误隐藏技术十分必要。而且,压缩后传输的3D视频数据量仍然较大,这对解码器的解码速度提出了很高的要求。如何在解码端加速进行错误隐藏是研究重点,通过快速定位静止宏块的方法加速错误隐藏速度。实验结果表明该算法在保证不降低视频的主、客观质量的同时,能有效地降低错误隐藏算法的复杂度。     

Generating Virtual Images for Multi-view Video

As for the research in the Free view televi-sion (FTV) and stereo video areas, the accurate multi-view video capture is a expensive precondition. We present a virtual multi-view video capture system based on OpenGL programming. By using this system, a researcher may cre-ate arbitrary 3D scenes, and capture images with depth maps by setting up multiple virtual cameras, whose in-trinsic and extrinsic parameters can be imported from or exported to the outside. We conducted several experiments within it to perform camera calibration, depth estimation and view-points synthesis. The system has been demon-strated to be a powerful tool for evaluation of stereo video algorithms.     

立体视频技术的发展现状综述

围绕立体视频产生、编码、传输以及接收显示。就两路光信号产生视差的立体以及两路视频流的编码技术等方面,结合目前业界主流及其热点问题,作了广泛而深入的重点梳理与阐释。指出多视点视频、编码是发展方向。采用运动和视差联合估计,结合如分形编码、小波变换编码等快速算法并应用于最先进的编码标准中,把被动式通用眼镜作为当前一段时间的立体电视过渡,发展到未来的全息立体。     

Architectures for multi-threaded MVC-compliant multi-view video decoding and benchmark tests

3D video based on stereo/multi-view representations is becoming widely popular. Real-time encoding/decoding of such video is an important concern as the number and spatial/temporal resolution of views increase. We present a systematic method for design and optimization of multi-threaded multi-view video encoding/decoding algorithms using multi-core processors and provide benchmark results for real-time decoding. The proposed multi-core decoding architectures are compliant with the current MVC extension of H.264/AVC international standard, and enable multi-threaded processing with negligible loss of encoding efficiency and minimum processing overhead. Benchmark results show that multi-core processors and multi-threading decoding are necessary for real-time high-definition multi-view video decoding and display.     

Depth map creation and image-based rendering for advanced 3DTV services providing interoperability and scalability

Due to enormous progress in the areas of auto-stereoscopic 3D displays, digital video broadcast and computer vision algorithms, 3D television (3DTV) has reached a high technical maturity and many people now believe in its readiness for marketing. Experimental prototypes of entire 3DTV processing chains have been demonstrated successfully during the last few years, and the motion picture experts group (MPEG) of ISO/IEC has launched related ad hoc groups and standardization efforts envisaging the emerging market segment of 3DTV. In this context the paper discusses an advanced approach for a 3DTV service, which is based on the concept of video-plus-depth data representations. It particularly considers aspects of interoperability and multi-view adaptation for the case that different multi-baseline geometries are used for multi-view capturing and 3D display. Furthermore it presents algorithmic solutions for the creation of depth maps and depth image-based rendering related to this framework of multi-view adaptation. In contrast to other proposals, which are more focused on specialized configurations, the underlying approach provides a modular and flexible system architecture supporting a wide range of multi-view structures.     

Multi-view video coding with view interpolation prediction for 2D camera arrays

An efficient compression algorithm for multi-view video sequences, which are captured by two-dimensional (2D) camera arrays, is proposed in this work. First, we propose a novel prediction structure, called three-dimensional hierarchical B prediction (3DHBP), which can efficiently reduce horizontal inter-view redundancies, vertical inter-view redundancies, and temporal redundancies in multi-view videos. Second, we develop a view interpolation scheme based on the bilateral disparity estimation. The interpolation scheme yields high quality view frames by adapting disparity estimation and compensation procedures using the information in neighboring frames. Simulation results demonstrate that the proposed multi-view video coding algorithm provides significantly better rate–distortion (R–D) performance than the conventional algorithm, by employing the 3DHBP structure and using interpolated view frames as additional reference frames.