End-to-end optimized TCP-friendly rate control for real-time video streaming over wireless multi-hop networks

Rate control is an important issue in video streaming applications. The most popular rate control scheme over wired networks is TCP-Friendly Rate Control (TFRC), which is designed to provide optimal transport service for unicast multimedia delivery based on the TCP Reno’s throughput equation. It assumes perfect link quality, treating network congestion as the only reason for packet losses. Therefore, when used in wireless environment, it suffers significant performance degradation because of packet losses arising from time-varying link quality. Most current research focuses on enhancing the TFRC protocol itself, ignoring the tightly coupled relation between the transport layer and other network layers. In this paper, we propose a new approach to address this problem, integrating TFRC with the application layer and the physical layer to form a holistic design for real-time video streaming over wireless multi-hop networks. The proposed approach can achieve the best user-perceived video quality by jointly optimizing system parameters residing in different network layers, including real-time video coding parameters at the application layer, packet sending rate at the transport layer, and modulation and coding scheme at the physical layer. The problem is formulated and solved as to find the optimal combination of parameters to minimize the end-to-end expected video distortion constrained by a given video playback delay, or to minimize the video playback delay constrained by a given end-to-end video distortion. Experimental results have validated 2–4 dB PSNR performance gain of the proposed approach in wireless multi-hop networks by using H.264/AVC and NS-2.     

SVC-based multisource streaming for robust video transmission in mobile ad hoc networks

Emerging noninfrastructure-based network types like mobile ad-hoc networks (MANETs) are becoming suitable platforms for exchanging/sharing real-time video streams, because of recent progress in routing algorithms, throughput and transmission bit-rate. MANETs are characterized by highly dynamic behavior of the transmission routes and path outage probabilities. In this article a multisource streaming approach is presented to increase the robustness of real-time video transmission in MANETs. For that, video coding as well as channel coding techniques on the application layer are introduced, exploiting the multisource representation of the transferred media. Source coding is based on the scalable video coding (SVC) extension of H.264/MPEG4-AVC with different layers for assigning importance for transmission. Channel coding is based on a novel unequal packet loss protection (UPLP) scheme, which is based on Raptor forward error correction (FEC) codes. While in the presented approach, the reception of a single stream guarantees base quality only, the combined reception enables playback of video at full quality and/or lower error rates. Furthermore, an application layer protocol is introduced for supporting peer-to-peer based multisource streaming in MANETs     

Efficient Media Synchronization Mechanism for SVC Video Transport over IP Networks

The scalable extension of H.264, known as scalable video coding (SVC) has been the main focus of the Joint Video Team's work and was finalized at the end of 2007. Synchronization between media is an important aspect in the design of a scalable video streaming system. This paper proposes an efficient media synchronization mechanism for SVC video transport over IP networks. To support synchronization between video and audio bitstreams transported over IP networks, a real‐time transport protocol/RTP control protocol (RTP/RTCP) suite is usually employed. To provide an efficient mechanism for media synchronization between SVC video and audio, we suggest an efficient RTP packetization mode for inter‐layer synchronization within SVC video and propose a computationally efficient RTCP packet processing method for inter‐media synchronization. By adopting the computationally simple RTCP packet processing, we do not need to process every RTCP sender report packet for inter‐media synchronization. We demonstrate the effectiveness of the proposed mechanism by comparing its performance with that of the conventional method.     

Android移动端电视直播系统的设计与实现

以移动流媒体体系结构和流媒体传输控制协议族为基础,在家庭电视机顶盒和无线路由器的基础上,设计一种基于Android系统框架的电视直播系统.机顶盒通过DVB-C获取稳定的数字信号然后硬转码把音视频数据编码成适合移动端播放的AAC和H.264格式并通过HLS协议实现移动端高清电视直播.经验证,该系统可以在现有的家庭多媒体资源基础上实现移动端电视节目的高清同步观看,具有很好的可行性和商业价值.     

Wireless video streaming with TCP and simultaneous MAC packet transmission (SMPT)

Video streaming is expected to account for a large portion of the traffic in future networks, including wireless networks. It is widely accepted that the user datagram protocol (UDP) is the preferred transport protocol for video streaming and that the transmission control protocol (TCP) is unsuitable for streaming. The widespread use of UDP, however, has a number of drawbacks, such as unfairness and possible congestion collapse, which are avoided by TCP. In this paper we investigate the use of TCP as the transport layer protocol for streaming video in a multi‐code CDMA cellular wireless system. Our approach is to stabilize the TCP throughput over the wireless links by employing a recently developed simultaneous MAC packet transmission (SMPT) approach at the link layer. We study the capacity, i.e. the number of customers per cell, and the quality of service for streaming video in the uplink direction. Our extensive simulations indicate that streaming over TCP in conjunction with SMPT gives good performance for video encoded in a closed loop, i.e. with rate control. We have also found that TCP is unsuitable (even in conjunction with SMPT) for streaming the more variable open‐loop encoded video. Copyright © 2004 John Wiley & Sons, Ltd.     

Radio Link Buffer Management and Scheduling for Wireless Video Streaming

In this paper we compare strategies for joint radio link buffer management and scheduling for wireless video streaming. Based on previous work in this area [8], we search for an optimal combination of scheduler and drop strategy for different end-to-end streaming options including timestamp-based streaming and ahead-of-time streaming, both with variable initial playout delay. We will show that a performance gain versus the two best drop strategies in Liebl et al. [8], i.e. drop the HOL packet or drop the packet with the lowest priority starting from HOL, is possible: Provided that some basic side-information on the structure of the incoming video stream is available, a more sophisticated drop strategy removes packets from an HOL group of packets in such a way that the temporal dependencies usually present in video streams are not violated. This advanced buffer management scheme yields significant improvements for almost all investigated scheduling algorithms and streaming options. In addition, we will demonstrate the importance of fairness among users when selecting a suitable scheduler, especially if ahead-of-time streaming is to be applied: Given a reasonable initial playout delay at the streaming media client, both the overall achievable quality averaged over all users, as well as the individual quality of users with bad channel conditions can be increased significantly by trading off fairness with maximum throughput of the system.     

Rate control for consistent visual quality of H.264/AVC encoding

In this paper, a novel rate control scheme with sliding window basic unit is proposed to achieve consistent or smooth visual quality for H.264/AVC based video streaming. A sliding window consists of a group of successive frames and moves forward by one frame each time. To make the sliding window scheme possible for real-time video streaming, the initial encoder delay inherently in a video streaming system is utilized to generate all the bits of a window in advance, so that these bits for transmission are ready before their due time. The use of initial encoder delay does not introduce any additional delay in video streaming but benefits visual quality as compared to traditional one-pass rate control algorithms of H.264/AVC. Then, a Sliding Window Buffer Checking (SWBC) algorithm is proposed for buffer control at sliding window level and it accords with traditional buffer measurement of H.264/AVC. Extensive experimental results exhibit that higher coding performance, consistent visual quality and compliant buffer constraint can be achieved by the proposed algorithm.     

A Resource Allocation Framework for Scalable Video Broadcast in Cellular Networks

Video streaming is one of the most important applications that will make use of the high data rates offered by 4G networks. The current video transport techniques are already very advanced, and the more immediate problems lie in the joint optimization of video coding, AL-FEC, and PHY rate selection with the goal of enhancing the user perceived quality. In this work we provide an analysis of video broadcast streaming services for different combinations of layered coding and AL-FEC, using a realistic LTE PHY layer. Our simulation results show that the scalable content adaptation given by Scalable Video Coding (SVC) and the scheduling flexibility offered by the 3G-LTE MAC-layer provide a good match for enhanced video broadcast services for next generation cellular networks. Our proposed solution is compared to baseline algorithms and broadcast systems based on H.264/AVC streaming solutions. We emphasize the system quality improvement brought by our solution and discuss implications for a wide-scale practical deployment.     

RD-optimized interactive streaming of multiview video with multiple encodings

This paper presents a rate-distortion (RD) optimized interactive streaming method for multiview video pre-compressed by H.264 Joint Multiview Video Model (JMVM). In the proposed method, multiple encodings are first used to facilitate the flexible server–client interaction. Second, a RD-optimized scheduling strategy is provided to guarantee the optimal view-dependent delivery of multiview video. In the RD-optimized scheduling strategy, a distortion model is proposed to estimate the expected end-to-end distortion by accounting for both coding and packet-loss-induced distortions, as well as rendering-induced distortion. With the end-to-end distortion model, the server can select the optimal encoding combination for transmission. Experimental results demonstrate that the proposed method can achieve a significant end-to-end RD performance improvement over the selective streaming methods with simulcast coding or scalable multiview coding. In addition, it has better error-resilience performance to combat with packet-losses over the Internet protocol (IP) networks.     

Streaming RFID: Robust Stream Transmission over Passive RFID

This paper proposes the streaming radio frequency identification (RFID) protocol to support robust data streaming in a passive communication, which is extended from the ISO18000‐6 Type C RFID standard. By observing and modeling the unique bit error behavior through detailed analysis in this paper, we found that performance is significantly limited by inaccurate and unstable link frequencies as well as low SNR which are inevitable for passive devices. Based on the analysis, we propose a simple and efficient protocol to adaptively insert extra error control sequences in a packet for tolerating tough link condition while maximizing the throughput and preserving the minimal implementation cost. To evaluate effectiveness of our proposal in real‐time streaming applications, we experimented on real‐time H.264 video streaming and prototyped the system on FPGA. To our best knowledge, our paper is the first work to take analytical approach for maximizing the throughput and demonstrate the possibility of the real‐time multimedia streaming transmission in the passive RFID system.     

Priority-Aware Packet Pre-marking for DiffServ Architecture Based on H.264/SVC Video Stream Structure

The H264/SVC codec allows for generation of hierarchical video streams. In the stream of this type video data belonging to different layers have different priority depending on their importance to the quality of the video and the decoding process. This creates new demands on the mechanisms of packet marking, and thus new challenges for the policy guaranteeing QoS parameters, such as those defined in the DiffServ architecture. Therefore, mechanisms of the traffic engineering used in the DiffServ network should, as far as possible, take into account internal distribution of priorities inside video streams. This may be achieved by implementing an appropriate method for packet pre-marking. The paper describes the Weighted Priority Pre-marking (WPP) algorithm for priority-aware SVC video streaming over a DiffServ network. Our solution takes into account the relative importance of the Network Abstraction Layer Units. It also does not require any changes in the implementation of the DiffServ marker algorithm. The results presented confirm that video transmission in the DiffServ domain, based on the WPP packet pre-marking, can provide better perceived video quality than the standard (best effort) streaming of multi-layered SVC video. In addition, a comparison with the transmission of the same video content encoded with the H264/AVC codec also points to the superiority of our proposed method.

     

一种有效降低误差扩散的数据分割方法

提出一种有效降低误差扩散的数据分割方法.该方法改进了H.264划分子流的做法,采用当前编码宏块被错误解码时对下一编码帧产生的误差扩散程度来判别该宏块的优先级,以此划分不同的传输子流.在给定网络带宽和丢包率的情况下,采用率失真优化的码率分配算法对不同传输子流分别施以不同的信道编码保护力度,以保证在相同传输条件下,优先级高的宏块被正确接收的概率最大化,使重建视频流的质量达到最优.该方法实现简单,对宏块的分类更加合理,有效降低了由于包丢失引起的误差扩散效应,改善了重建视频的质量.实验结果表明在相同的带宽和丢包率条件下,本文方法比H.264方法提高视频接收质量大约0.3~0.6dB.     

Toward an improvement of H.264 video transmission over IEEE 802.11e through a cross-layer architecture

The recently developed H.264 video standard achieves efficient encoding over a bandwidth ranging from a few kilobits per second to several megabits per second. Hence, transporting H.264 video is expected to be an important component of many wireless multimedia services, such as video conferencing, real-time network gaming, and TV broadcasting. However, due to wireless channel characteristics and lack of QoS support, the basic 802.11-based channel access procedure is merely sufficient to deliver non-real-time traffic. The delivery should be augmented by appropriate mechanisms to better consider different QoS requirements and ultimately adjust the medium access parameters to the video data content characteristics. In this article we address H.264 wireless video transmission over IEEE 802.11 WLAN by proposing a robust cross-layer architecture that leverages the inherent H.264 error resilience tools (i.e., data partitioning); and the existing QoS-based IEEE 802.11e MAC protocol possibilities. The performances of the proposed architecture are extensively investigated by simulations. Results obtained indicate that compared to 802.11 and 802.11e, our cross-layer architecture allows graceful video degradation while minimizing the mean packet loss and end-to-end delays.     

基于H.265的流媒体播放器的研究与设计

基于网络视频监控系统对降低传输带宽和减少存储空间的需求。本文设计了一种基于H.265的流媒体播放器,深入分析了该流媒体播放器的设计需求和组成部分。该流媒体播放器主要分为数据接收层、数据处理层、数据解码层、数据显示层四个部分,能够完成实时视频的采集以及解码显示。实验结果表明,H.265标准的视频压缩性能相比于H.264将近提升了一倍,极大地降低传输带宽并且减少存储空间。     

Adaptive Rate Optimal Design for Wireless Heterogeneous Networks: Jointly Congestion Discrimination and Queue Updating

In this paper, we propose a novel rate adaptive optimization scheme for streaming media transmission over wireless heterogeneous IP networks. In the proposed adaptive scheme, through the analysis of the packet loss characteristics in wireless channel, we develop the relationship between the packet loss rates and the packet sizes. Furthermore, the scheme detects the nature of packet losses by sending large and small packets alternately, and then adopts an adaptive rate optimization strategy to decrease the network congestion and increase the network throughput. Using congestion discrimination and updating factor, the scheme can adapt to the changes of network states quickly and improve delivery quality of wireless multimedia streaming. Simulation results show that, in comparisons to the existing rate optimization algorithms, our proposed scheme offers significantly improved performance in terms of throughput and network congestion, especially when the channel quality is poor in different network topology environments.     

H.264/AVC video for wireless transmission

H.264/AVC will be an essential component in emerging wireless video applications thanks to its excellent compression efficiency and network-friendly design. However, a video coding standard itself is only one component within the application and transmission environment. Its effectiveness strongly depends on the selection of appropriate modes and parameters at the encoder, at the decoder, as well as in the network. In this paper we introduce the features of the H.264/AVC coding standard that make it suitable for wireless video applications, including features for error resilience, bit rate adaptation, integration into packet networks, interoperability, and buffering considerations. Modern wireless networks provide many different means to adapt quality of service, such as forward error correction methods on different layers and end-to-end or link layer retransmission protocols. The applicability of all these encoding and network features depends on application constraints, such as the maximum tolerable delay, the possibility of online encoding, and the availability of feedback and cross-layer information. We discuss the use of different coding and transport related features for different applications, namely video telephony, video conferencing, video streaming, download-and-play, and video broadcasting. Guidelines for the selection of appropriate video coding tools, video encoder and decoder settings, as well as transport and network parameters are provided and justified. References to relevant research publications and standardization contributions are given.     

SCTP故障恢复机制改进

流传输控制协议（SCTP协议）是一种新的Internet传输层协议,Internet工作组设计SCTP的最初目的是在IP网络上传输PSTN信令消息,而且还能够充当通用传输协议。与传统的传输协议相比,SCTP协议允许在一个单一的连接中传输多个数据子流,这种功能可以大大改善高损耗的环境中多媒体流延迟问题,同时SCTP协议支持多宿功能,能够为网络提供冗余备份功能。对SCTP故障恢复机制进行了改进,充分利用SCTP多宿特性为移动Ad hoc网络提供可靠性保障,使其能够适应移动Ad hoc网络的特点,仿真结果表明：该功能极大地减少了故障恢复时间,提高了其在移动Ad hoc网络中的性能。     

A Practical RTP Packetization Scheme for SVC Video Transport over IP Networks

Scalable video coding (SVC) has been standardized as an extension of the H.264/AVC standard. This paper proposes a practical real‐time transport protocol (RTP) packetization scheme to transport SVC video over IP networks. In combined scalability of SVC, a coded picture of a base or scalable enhancement layer is produced as one or more video layers consisting of network abstraction layer (NAL) units. The SVC NAL unit header contains a (DID, TID, QID) field to identify the association of each SVC NAL unit with its scalable enhancement layer without parsing the payload part of the SVC NAL unit. In this paper, we utilize the (DID, TID, QID) information to derive hierarchical spatio‐temporal relationship of the SVC NAL units. Based on the derivation using the (DID, TID, QID) field, we propose a practical RTP packetization scheme for generating single RTP sessions in unicast and multicast transport of SVC video. The experimental results indicate that the proposed packetization scheme can be efficiently applied to transport SVC video over IP networks with little induced delay, jitter, and computational load.