An automatic algorithm for semantic object generation and temporal tracking

Automatic semantic video object extraction is an important step for providing content-based video coding, indexing and retrieval. However, it is very difficult to design a generic semantic video object extraction technique, which can provide variant semantic video objects by using the same function. Since the presence and absence of persons in an image sequence provide important clues about video content, automatic face detection and human being generation are very attractive for content-based video database applications. For this reason, we propose a novel face detection and semantic human object generation algorithm. The homogeneous image regions with accurate boundaries are first obtained by integrating the results of color edge detection and region growing procedures. The human faces are detected from these homogeneous image regions by using skin color segmentation and facial filters. These detected faces are then used as object seed for semantic human object generation. The correspondences of the detected faces and semantic human objects along time axis are further exploited by a contour-based temporal tracking procedure.     

Saliency model-based face segmentation and tracking in head-and-shoulder video sequences

In this paper, a novel face segmentation algorithm is proposed based on facial saliency map (FSM) for head-and-shoulder type video application. This method consists of three stages. The first stage is to generate the saliency map of input video image by our proposed facial attention model. In the second stage, a geometric model and an eye-map built from chrominance components are employed to localize the face region according to the saliency map. The third stage involves the adaptive boundary correction and the final face contour extraction. Based on the segmented result, an effective boundary saliency map (BSM) is then constructed, and applied for the tracking based segmentation of the successive frames. Experimental evaluation on test sequences shows that the proposed method is capable of segmenting the face area quite effectively.     

Packet prioritization for H.264/AVC video with cyclic intra-refresh line

Insertion of a cyclic intra-refresh line is a lightweight way of mitigating spatio-temporal error propagation in a video stream transmitted over a mobile network. This paper presents low-complexity yet effective prioritization based on slice position within a video frame relative to the cyclic refresh line. Two prioritization schemes are compared. The first is a region-based method, while the second, which is packet-based, improves packet classification. Experimental results indicate that, the packet-based scheme can achieve video quality gains of up to 4 dB, compared to when the scheme is not used. The proposed schemes require no decoder modifications and do not introduce an increase in bitrate or in computational complexity.     

A Probabilistic Network for Facial Feature Verification

In this paper, we present a probabilistic approach to determining whether extracted facial features from a video sequence are appropriate for creating a 3D face model. In our approach, the distance between two feature points selected from the MPEG‐4 facial object is defined as a random variable for each node of a probability network. To avoid generating an unnatural or non‐realistic 3D face model, automatically extracted 2D facial features from a video sequence are fed into the proposed probabilistic network before a corresponding 3D face model is built. Simulation results show that the proposed probabilistic network can be used as a quality control agent to verify the correctness of extracted facial features.     

OFDM系统中视频传输的调度算法研究

针对OFDM系统,提出了一种基于视频内容的跨层调度方案。该方案采用了基于生存期的包排序策略,根据视频包的生存期大小进行排序,保证视频包能在生存期内发送到接收端,减少丢包数目。同时,采用跨层设计的思想,综合考虑了信道状况和视频包特性,如视频包生存期、重要性、解码器采用的错误掩藏方法,提出改进的比例公平调度算法,不仅有效地利用了多用户分集来进一步提高数据吞吐量,也充分考虑了视频的重要性和时延约束。实验结果表明,采用内容感知的跨层调度算法,解码端的视频质量得到有效提高,从而可以提高主观感知质量。     

Flexible and robust packet transport for digital HDTV

The packet-oriented transport approach used in the advanced digital television (ADTV) system for terrestrial HDTV broadcast is described. ADTV achieves robust HDTV delivery on terrestrial simulcast channels via MPEG video compression, prioritization of MPEG data, and `cell-relay' type packet transport in conjunction with a two-tier physical transmission scheme. General design issues relevant to the development of the proposed transport protocol are discussed. ADTV's prioritization algorithm for partitioning MPEG-encoded video into high-priority (HP) and standard-priority (SP) bit streams is outlined. The data transport format supporting these prioritized compressed video bit streams is described. The three principal sublayers of the ADTV transport protocol are discussed in terms of specific functions, impact of system performance, and hardware implementation factors. A proof-of-concept simulation model that incorporates transport encoding and decoding functionality is outlined, and performance evaluation results are given for illustrative transmission scenarios     

基于人脸特征的自适应空域差错掩盖算法

针对在视频通信中,高压缩视频码流对通信信道中由于信道带宽、环境等影响出现的丢包和误码极为敏感,本文提出一种在解码端基于人脸检测的自适应空域差错掩盖算法,根据正确接收帧的人脸位置及人脸运动的趋势预测受损帧人脸的位置,对于受损帧人脸内五官区域的宏块采用水平双线性插值,对于背景区域宏块采用自适应多方向插值算法。实验结果表明与现有算法相比主客观质量都有所提高。     

人脸检测跟踪与特征点定位系统

实现的人脸检测跟踪与特征点定位系统,基于VC++6.0开发平台,使用opencv作为开发工具,有效缩短了系统的开发时间。首先,本系统采用adaboost算法进行人脸检测,通过合理的特征模板的选择实现了人脸的实时检测;其次,人脸跟踪模块选用camshift算法,利用人脸检测模块生成的人脸坐标传递给跟踪模块,实现人脸的自动实时跟踪,同时建立多个camshift跟踪器对多人脸进行跟踪,并有效地解决了人脸遮挡的问题;最后,通过ASM(active shapemodel)算法实现了实时人脸特征点定位。实验结果表明该系统实现的人脸实时检测跟踪及特征点定位,效果明显,可以作为表情分析和情感计算、视频人脸识别开发的基础。     

Error concealment aware rate shaping for wireless video transport

Streaming of video, which is both source- and channel-coded, over wireless networks faces the challenge of time-varying packet loss rate and fluctuating bandwidth. Rate shaping (RS) has been proposed to reduce the bit-rate of a precoded video bitstream to adapt to the real-time bandwidth variation. In our earlier work, rate shaping was extended to consider not only the bandwidth but also the packet loss rate variations. Rate-distortion optimized rate adaptation is performed on the precoded video that is a scalable coded bitstream protected by forward error correction codes. In this paper, we propose an RS scheme that further takes into account the error concealment (EC) method used at the receiver. We refer to this scheme as EC aware RS (ECARS). When performing ECARS, first ECARS needs to know the benefit/gain of sending each part of the precoded video, as opposed to not sending it but reconstructing it by EC. Then given a certain packet loss probability, the expected gain can be derived and be included in the rate-distortion optimization problem formulation. Finally, ECARS performs rate-distortion optimization to adapt the rate of the precoded video. A two-stage rate-distortion optimization approach is proposed to solve the ECARS rate-distortion optimization problem. In addition to ECARS, the precoding process can be EC aware to prioritize the precoded video based on the gain. We present an example EC aware precoding process by means of macroblock prioritization. Experiment results of ECARS together with EC aware precoding are shown to have excellent performance.     

Perceptual quality‐based image communication service framework for wireless sensor networks

Wireless multimedia sensor networks (WMSNs) have an increasing variety of multimedia‐based applications including image and video transmission. In these types of applications, multimedia sensor nodes should ideally maximize perceptual quality and minimize energy expenditures in communication. For the required perceptual quality to be obtained, quality‐aware routing is a key research area in WMSNs. However, mapping the system parameters to the end user's perceptual quality‐of‐service measures is a challenging task because of incomplete identification metrics. Unfortunately, unless disputable assumptions and simplifications are made, optimal routing algorithm is not tractable. In this paper, we propose a novel image transmission framework to optimize both perceptual quality and energy expenditure in WMSNs. Our framework aims to provide acceptable perceptual quality at the end user by using an analytical distortion prediction model that is able to predict the image distortion resulting from any given error pattern. The innovation of the proposed scheme lies in the combined use of a content‐aware packet prioritization with an energy‐aware and quality‐aware routing protocol, named as image quality‐aware routing. Additionally, it does not only propose an energy‐efficient route selection policy but also manages the network load according to the energy residues of nodes, thus leading to a great energy economy. The results reveal that the framework is capable of identifying true metrics for mapping required image quality to network parameters. Copyright © 2011 John Wiley & Sons, Ltd.     

Selective recovery of video packet loss using error concealment

An efficient recovery method using error concealment is proposed for video packet loss in fast packet switching networks. In this method, the receiver detects the damaged picture area caused by packet loss from the structured picture data received, makes error concealments, notifies the transmitter, and continues decoding. The transmitter, having received the notice, calculates the affected picture area in the local decoded picture and continues encoding without using this affected area. In this selective recovery method, video signals are not stopped even if a long propagation delay exists, no additional information is transmitted to error recovery and conventional coding algorithms can be used. The proposed method is suitable for multipoint communication. Simulation results show the affected picture area is localized for a considerable time attesting to the method's effectiveness     

Small‐block interleaving for low‐delay cross‐packet forward error correction over burst‐loss channels

By adding the redundant packets into source packet block, cross‐packet forward error correction (FEC) scheme performs error correction across packets and can recover both congestion packet loss and wireless bit errors accordingly. Because cross‐packet FEC typically trades the additional latency to combat burst losses in the wireless channel, this paper presents a FEC enhancement scheme using the small‐block interleaving technique to enhance cross‐packet FEC with the decreased delay and improved good‐put. Specifically, adopting short block size is effective in reducing FEC processing delay, whereas the corresponding effect of lower burst‐error correction capacity can be compensated by deliberately controlling the interleaving degree. The main features include (i) the proposed scheme that operates in the post‐processing manner to be compatible with the existing FEC control schemes and (ii) to maximize the data good‐put in lossy networks; an analytical FEC model is built on the interleaved Gilbert‐Elliott channel to determine the optimal FEC parameters. The simulation results show that the small‐block interleaved FEC scheme significantly improves the video streaming quality in lossy channels for delay‐sensitive video. Copyright © 2013 John Wiley & Sons, Ltd.     

Gender Classification Based on Multi-Scale and Run-Length Features

Human faces can convey substantial information about a person, such as his or her age, race, identity, gender, and emotions. Such facial information can be obtained through techniques like human facial tracking and detection, facial recognition, gender classification, emotion recognition, as well as age estimation. Of these, gender classification is particularly important due to its diverse applications in the fields such as video surveillance and commercial advertising. In this thesis, we propose a method of gender classification based on run-length histograms. The proposed method uses a run-length histogram to record the position information of pixels, thereby efficiently improves the recognition rate and makes the technique suitable for a big-data multimedia database. The experimental results show that the proposed method can achieve better accuracy than a multi-scale based method can.     

Face detection and tracking using a Boosted Adaptive Particle Filter

A novel algorithm, termed a Boosted Adaptive Particle Filter (BAPF), for integrated face detection and face tracking is proposed. The proposed algorithm is based on the synthesis of an adaptive particle filtering algorithm and the AdaBoost face detection algorithm. An Adaptive Particle Filter (APF), based on a new sampling technique, is proposed. The APF is shown to yield more accurate estimates of the proposal distribution and the posterior distribution than the standard Particle Filter thus enabling more accurate tracking in video sequences. In the proposed BAPF algorithm, the AdaBoost algorithm is used to detect faces in input image frames, whereas the APF algorithm is designed to track faces in video sequences. The proposed BAPF algorithm is employed for face detection, face verification, and face tracking in video sequences. Experimental results show that the proposed BAPF algorithm provides a means for robust face detection and accurate face tracking under various tracking scenarios.     

一种融合流形学习的视频人脸性别识别改进算法

如何有效利用视频中人脸之间的时空连续性信息来克服人脸分辨率低、图像尺度变化大和姿态、光照变化以及遮挡等问题是视频人脸识别的关键所在.提出了一种基于流形学习的视频人脸性别识别算法.该算法不仅可以通过聚类融合学习来挖掘视频内在的连续性信息,同时能发现人脸数据中内在非线性结构信息而获得低维本质的流形结构.在UCSD/Honda和自采集数据库上与静态的算法比较结果表明,所提算法能够获得更好的识别率.     

Per-packet rate adaptation for wireless video

Streaming video over error-prone wireless channels is a challenge as the dynamic network conditions and slow adaptation to channel degradations may affect the quality of the streamed video. Unequal error protection (UEP) can potentially address this issue by considering the importance of each video packet and its impact on the quality of reconstructed video. This paper proposes a cross-layer UEP solution for wireless video streaming over IEEE 802.11 networks. Video packets are prioritized based on the relative importance of the video packet. UEP is achieved by adapting the link layer parameters on a per-packet basis, using inherent forward error correction and adaptive modulation capabilities of the 802.11n network. Experimental results revealed that the proposed solution achieves comparable performance to the state-of-the-art methods at a lower complexity.     

Performance Study on ZigBee‐Based Wireless Personal Area Networks for Real‐Time Health Monitoring

When multiple ZigBee wireless personal area networks (WPANs) are in close proximity to each other, contentions and collisions in transmissions will lead to increased packet delays. However, there is no existing study on how delay performance would be affected in a crowded real‐life environment where each person walking down a busy street would be wearing a ZigBee WPAN. This letter studies the use of ZigBee WPANs in such a real‐life environment for real‐time heart beat monitoring. To be pragmatic, we derived a mobility pattern from the analysis of a real‐life video trace. Then, we estimated the delay performance from the video trace by combining data collected from ZigBee experiments. The results show that the 300 ms packet delay requirement will not be met for only 11% of the time. When failure occurs, it will last for an average duration of 1.4 s.     

Performance modeling of FEC‐based unequal error protection for H.264/AVC video streaming over burst‐loss channels

Unequal error protection systems are a popular technique for video streaming. Forward error correction (FEC) is one of error control techniques to improve the quality of video streaming over lossy channels. Moreover, frame‐level FEC techniques have been proposed for video streaming because of different priority video frames within the transmission rate constraint on a Bernoulli channel. However, various communication and storage systems are likely corrupted by bursts of noise in the current wireless behavior. If the burst losses go beyond the protection capacity of FEC, the efficacy of FEC can be degraded. Therefore, our proposed model allows an assessment of the perceived quality of H.264/AVC video streaming over bursty channels, and is validated by simulation experiments on the NS‐2 network simulator at a given estimate of the packet loss ratio and average burst length. The results suggest a useful reference in designing the FEC scheme for video applications, and as the video coding and channel parameters are given, the proposed model can provide a more accurate evaluation tool for video streaming over bursty channels and help to evaluate the impact of FEC performance on different burst‐loss parameters. Copyright © 2014 John Wiley & Sons, Ltd.     

Reliable tracking of facial features in semantic-based video coding

A new method of tracking the position of important facial features for semantic-based moving image coding is presented. Reliable and fast tracking of the facial features in head-and-shoulders scenes is of paramount importance for reconstruction of the speakers motion in videophone systems. The proposed method is based on eigenvalue decomposition of the sub-images extracted from subsequent frames of the video sequence. The motion of each facial feature (the left eye, the right eye, the nose and the lips) is tracked separately; this means that the algorithm can be easily adapted for a parallel machine. No restrictions, other than the presence of the speaker's face, were imposed on the actual contents of the scene. The algorithm was tested on numerous widely used head-and-shoulders video sequences containing moderate head pan, rotation and zoom, with remarkably good results. Tracking was maintained even when the facial features were occluded. The algorithm can also be used in other semantic-based systems     

Content-adaptive traffic prioritization of spatio-temporal scalable video for robust communications over QoS-provisioned 802.11e networks

In this work a low complexity traffic prioritization strategy to transmit H.264 scalable video coding (SVC) video over 802.11e wireless networks is presented (the approach applies to any DiffServ-like network). The distinguishing feature of the proposed strategy is the ability to adapt the amount of error protection to the changing characteristics of the video content. First, we estimate the perceptual impact of data losses in the different types of enhancement layers, i.e., temporal or spatial, for a large set of H.264/SVC videos. The experiments show that perceptual impairments are highly correlated with the level of motion activity in the video sequence. Then, we propose a content-adaptive traffic prioritization strategy based on the identification of the most important parts of the enhancement layers of the video sequence by means of a low complexity macroblock analysis process. The prioritization algorithm is tested by simulating a realistic 802.11e ad hoc network scenario. Simulation results show that the proposed traffic prioritization strategy consistently outperforms, particularly for dynamic video sequences, fixed a priori approaches, as well as a traditional FEC-based UEP strategy.