MPEG-21中的数字项适配技术研究

数字项适配是MPEG-21标准中一项非常重要的关键技术.本文研究MPEG-21中的数字项适配技术以及媒体独立格式的资源适配机制,实现一套基于MPEG-21 DIA的图像适配系统,提供了数字项解析、适配决策形成以及资源适配等功能.并在此基础之上试验了新的基于感兴趣区域的适配方法.试验结果表明该系统是完整和有效的,采用的新的适配方法是新颖的.     

面向MPEG-21 DIA的适配决策模型

通用多媒体内容访问是未来多媒体服务的常用模式。针对这一访问模式,MPEG-21提出了DIA技术框架,以期实现相关技术标准化。但是,DIA现在只对上下文描述等元数据制定了标准,而适配决策引擎仍然是一个开放问题。针对已有适配决策模型存在的问题,提出了一种适配决策树模型,该模型将适配决策视为序列决策,并用决策树对其建模。为满足多媒体服务的个性化需求,利用用户偏好信息建立适配决策树。此外,利用优化模型构建决策空间,使参与决策的对象数缩小为模态数,从而减少了适配决策时间,满足了多媒体服务的实时性。通过仿真实验证明,该适配决策树模型在决策误差和平均迭代次数两个指标上都优于或相近于已有的适配决策模型。     

基于XML的数字媒体资源管理系统设计与实现

数字媒体资源是当前存储和交换信息的主要形式,对其进行有效的管理和高效的利用对信息和知识具有重大意义.介绍了对各种数字媒体资源(包括文本、图片、音频、视频等)进行存储、管理以及应用的实现方法.满足数字媒体资源拥有者收集、保存、查找、编辑、发布各种信息的功能要求,同时为数字媒体资源的授权使用者提供了在线和简便的内容访问平台,从而实现了对数字媒体资源的高效管理和高效利用.     

浅谈数字媒体设计在教学的应用与实现

数字媒体设计以数字化、网络化、虚拟化和多媒体化为特征,具有丰富的内涵。数字媒体设计包括文本、视频、音频、图片、图像、动画等多种媒体内容。本文就数字媒体设计在教学的应用与实现进行探讨,首先阐述了数字媒体设计技术的内涵,其次就数字媒体设计在教学的应用与实现进行了深入的探讨,具有一定的参考价值。     

计算机角色动画制作中的可视媒体融合综述

近年来,计算机角色动画在虚拟现实、计算机游戏、动画生成、影视特技及运动仿真等多个领域得到广泛深入的应用,同时在数据上涉及到图像、视频、运动捕获数据、三维模型等可视媒体,已经成为一个多种可视媒体融合的研究领域．在计算机角色动画制作中,传统的仅仅利用单一可视媒体进行模型制作和动画生成的方法正在被基于多种可视媒体融合的方法所取代．文中从可视媒体融合的角度对现有的计算机角色动画研究进行综述,并依据可视媒体融合框架进行归纳和分类．     

基于感知运动能量模型的MPEG-21数字项适配

最新的MPEG-21标准定义了一个统一的多媒体框架,其目标是使多媒体资源能在异构网络和终端上进行透明传输和得到充分利用。多媒体适配(Multimedia Adaptation)技术作为实现上述目标的关键技术得到了越来越多的研究。本文将感知运动能量模型与MPEG-21数字项适配框架相结合,研究和实现了基于感知运动能量的H．264／AVC视频丢帧适配技术。本文的研完工作有两个方面的优点：一方面,感知运动能量模型的使用将丢帧适配引起的运动抖动减至最小,保证了较高的用户视觉上的主观感知质量;另一方面,数据处理能力和资源受限的适配节点不需了解具体的H．264／AVC编码语法结构和编解码算法,就能方便地执行丢帧操作,实现了独立于媒体编码格式的视频适配机制,提高了多媒体适配系统的灵活性和可扩展性。     

一种基于点特征的异源SAR图像配准方法

适配性分析是合成孔径雷达(Synthetic aperture radar, SAR)图像匹配的重要研究内容, 它研究图像是否适合选作基准图的问题. 本文研究基于点特征配准方法的异源SAR图像适配性分析, 以特征点的数目及稳健程度来度量SAR图像适配性, 提出基于点特征的异源图像适配性评价算法, 并构建出一种评价特征点稳定性的准则, 以指导选取合适的退化模型.实验结果表明本文所提出的点特征适配性评价机制能有效地度量指定SAR图像的适配性, 从而近似得出保障图源的适配性排序结果.     

高斯混合模型、求解算法及视觉应用综述

高斯混合模型(GMMs)是统计学习理论的基本模型,在可视媒体领域应用广泛。近些年来,随着可视媒体信息的增长和分析技术的深入,GMMs在(纹理)图像分割、视频分析、图像配准、聚类等领域有了进一步的发展。从GMMs的基本模型出发,从理论和应用的角度讨论和分析了GMMs的求解算法,包括EM算法、变化形式等,论述了GMMs的模型选择问题:在线学习和模型约简。在视觉应用领域,介绍了GMMs在图像分段、视频分析、图像配准、图像降噪等领域的扩展模型与方法,详细地阐述了一些最新的典型模型的原理与过程,如用于图像分段的空间约束GMMs、图像配准中的关联点漂移算法。最后,讨论了一些潜在的发展方向与存在的困难问题。     

独立分量分析的图像/视频分析与应用

随着通信和计算机技术的发展,图像和视频信息的应用越来越多.图像和视频信息分析中的一个重要方法是获得合适的特征来逼近人类视觉特性.独立分量分析是一种新的无监督训练方法,它可以在图像和视频的理解方面很好地与人类视觉相匹配.给出了不同的ICA图像/视频分析模型和基于这些模型的独立特征,对多媒体ICA分析和数字小波分析方法进行了对比,对于不同分析方法的计算机仿真给出了不同模型的独立特征,并且给出了基于这些特征在图像和视频水印方案中的应用.应用实验的仿真结果表明,独立特征对于图像和视频水印性能具有较好的改善作用.     

移动服务平台终端智能适配框架的研究

为了满足移动服务平台(MSP)对终端智能适配的要求--智能识别终端,返回用户需要的服务结果,高效地开发移动媒体应用,结合常用开发方法的分析提出了基于移动服务平台终端智能适配框架,介绍了智能适配的框架所采用的模板技术、框架体系结构和框架关键点的设计与实现.在实际的运营中验证智能适配框架具有可复用、易管理、可满足用户人性化要求等良好特性,同时降低了MSP上业务开发难度,解决了平台中多种媒体业务和手机终端间的大量适配问题.     

Seamless video access for mobile devices by content-aware utility-based adaptation

Today’s Internet multimedia services are characterized by heterogeneous networks, a wide range of terminals, diverse user preferences, and varying natural environment conditions. Heterogeneity of terminals, networks, and user preferences impose nontrivial challenges to the Internet multimedia services for providing seamless multimedia access particularly for mobile devices (e.g., laptops, tablet PCs, PDAs, mobile phones, etc.). Thus, it is essential that advanced multimedia technologies are developed to deal with these challenges. One of these technologies is video adaptation, which has gained significant importance with its main objective of enabling seamless access to video contents available over the Internet. Adaptation decision taking, which can be considered as the “brain” of video adaptation, assists video adaptation to achieve this objective. Scalable Video Coding (SVC) offers flexibility for video adaptation through providing a comprehensive set of scalability parameters (i.e., temporal, spatial, and quality) for producing scalable video streams. Deciding the best combination of scalability parameters to adapt a scalable video stream while satisfying a set of constraints (e.g., device specifics, network bandwidth, etc.) poses challenges for the existing adaptation services to enable seamless video access. To ease such challenges, an adaptation decision taking technique employing a utility-based approach to decide on the most adequate scalability parameters for adaptation operations is developed. A Utility Function (UF), which models the relationships among the scalability parameters and weights specifying the relative importance of these parameters considering video content characteristics (i.e., motion activity and structural feature), is proposed to assist the developed technique. In order to perform the developed adaptation decision taking technique, a video adaptation framework is also proposed in this paper. The adaptation experiments performed using the proposed framework prove the effectiveness of the framework to provide an important step towards enabling seamless video access for mobile devices to enhance viewing experience of users.     

Metadata-driven optimal transcoding in a multimedia proxy

An adaptive multimedia proxy is presented which provides (1) caching, (2) filtering, and (3) media gateway functionalities. The proxy can perform media adaptation on its own, either relying on layered coding or using transcoding mainly in the decompressed domain. A cost model is presented which incorporates user requirements, terminal capabilities, and video variations in one formula. Based on this model, the proxy acts as a general broker of different user requirements and of different video variations. This is a first step towards What You Need is What You Get (WYNIWYG) video services, which deliver videos to users in exactly the quality they need and are willing to pay for. The MPEG-7 and MPEG-21 standards enable this in an interoperable way. A detailed evaluation based on a series of simulation runs is provided.     

基于电量状态的移动流媒体码率自适应策略

为提升移动流媒体的用户体验质量(quality of experience,QoE)和设备续航时长,提出一种基于移动设备电量状态的Qo E模型,模型的参数包括初始延迟、重新缓冲、平均视频质量、码率切换平滑度以及设备电量状态.在模型的基础上,给出一种基于网络吞吐量,同时又考虑设备电量状态的码率自适应策略.策略能避免客户端...     

Modeling user-generated contents: an intelligent state machine for user-centric search support

Researchers tend to agree that an increasing quantity of data has caused the complexity and difficulty for information discovery, management, and reuse. An essential factor relates to the increasing channels (i.e., Internet, social media, etc.) for information sharing. Finding information, especially those meaningful or useful one, that meets ultimate goal (or task) of user becomes harder then it is used to be. In this research, issues concerning the use of user-generated contents for individual search support are investigated. In order to make efficient use of user-generated contents, an intelligent state machine, as a hybridization of graph model (Document Graph) and petri-net model (Document Sensitive Petri-Net), is proposed. It is utilized to clarify the vague usage scenario between user-generated contents, such as discussions, posts, etc., and to identify correlations and experiences within them. As a practical contribution, an interactive search algorithm that generates potential solutions for individual is implemented. The feasibility of this research is demonstrated by a series of experiments and empirical studies with around 350,000 user-generated contents (i.e., documents) collected from the Internet and 200 users.     

Modeling user perception of 3D video based on ambient illumination context for enhanced user centric media access and consumption

For enjoying 3D video to its full extent, it is imperative that access and consumption of it is user centric, which in turn ensures improved 3D video perception. Several important factors including video characteristics, users’ preferences, contexts prevailing in various usage environments, etc have influences on 3D video perception. Thus, to assist efficient provision of user centric media, user perception of 3D video should be modeled considering the factors affecting perception. Considering ambient illumination context to model 3D video perception is an interesting research topic, which has not been particularly investigated in literature. This context is taken into account while modeling video quality and depth perception of 3D video in this paper. For the video quality perception model: motion and structural feature characteristics of color texture sequences; and for the depth perception model: luminance contrast of color texture and depth intensity of depth map sequences of 3D video are used as primary content related factors in the paper. Results derived using the video quality and depth perception models demonstrate that these models can efficiently predict user perception of 3D video considering the ambient illumination context in user centric media access and consumption environments.     

A Perceptual Comparison of Empirical and Predictive Region-of-Interest Video

When viewing multimedia presentations, a user only attends to a relatively small part of the video display at any one point in time. By shifting allocation of bandwidth from peripheral areas to those locations where a user's gaze is more likely to rest, attentive displays can be produced. Attentive displays aim to reduce resource requirements while minimizing negative user perception-understood in this paper as not only a user's ability to assimilate and understand information but also his/her subjective satisfaction with the video content. This paper introduces and discusses a perceptual comparison between two region-of-interest display (RoID) adaptation techniques. A RoID is an attentive display where bandwidth has been preallocated around measured or highly probable areas of user gaze. In this paper, video content was manipulated using two sources of data: empirical measured data (captured using eye-tracking technology) and predictive data (calculated from the physical characteristics of the video data). Results show that display adaptation causes significant variation in users' understanding of specific multimedia content. Interestingly, RoID adaptation and the type of video being presented both affect user perception of video quality. Moreover, the use of frame rates less than 15 frames per second, for any video adaptation technique, caused a significant reduction in user perceived quality, suggesting that although users are aware of video quality reduction, it does impact level of information assimilation and understanding. Results also highlight that user level of enjoyment is significantly affected by the type of video yet is not as affected by the quality or type of video adaptation-an interesting implication in the field of entertainment.     

Adaptation in IMMPS as a decision making process

Intelligent Multimedia Presentation Systems (IMMPS) are characterised by their capability to dynamically adapt at run-time, based on the state of several aspects of user-computer interaction, such as the user profile, task and information characteristics, etc. In this paper, we treat adaptation as a decision making process, and propose the employment of established techniques from the domain of Decision Making to assist this process. It is argued that, following this approach, the adaptation process can be easily modified, customised and re-used in different application domains and for different user interaction requirements. The designer is enabled to incorporate different design strategies, which can be dynamically and automatically modified, according to the run-time requirements of the system. The application of the proposed approach is exemplified in the media/modalities allocation process.     

面向上行流媒体的压缩感知视频流技术前沿

上行流媒体在军民融合领域展现出日益重要的新兴战略价值,压缩感知视频流技术体系在上行流媒体应用中具有前端功耗低、容错性好、适用信号广等独特优势,已成为当前可视通信研究的前沿与热点之一。本文从阐述上行流媒体的应用特征出发,从性能指标、并行分块计算成像、低复杂度视频编码、视频重构和语义质量评价等方面,分析了当前针对压缩感知视频流的基础理论与关键技术,对国内外相关的研究进展进行了探究与比较。面向上行流媒体的压缩感知视频流面临着观测效率难控、码流适配困难和重建质量较低等技术挑战。对压缩感知视频流的技术发展趋势进行展望,未来将通过前端与智能云端的分工协作,突破高效率的视频观测与语义质量导引视频重构等关键技术,进一步开拓压缩感知视频流在上行流媒体应用中的定量优势与演进途径。     

Live mobile collaboration for video production: design,guidelines, and requirements

Traditional cameras and video equipment are gradually losing the race with smart phones and small mobile devices that allow video, photo, and audio capturing on the go. Users are now quickly creating movies and taking photos whenever and wherever they go, particularly at concerts and live events (e.g., shows, sport events). Still, in-situ media capturing with such devices poses constraints to any user, especially amateur ones. In this paper, we present the design and evaluation of a mobile video capture suite that allows for cooperative ad hoc production. Our system relies on ad hoc in-situ collaboration offering users the ability to switch between streams and cooperate with each other in order to capture better media with mobile devices. Our main contribution is the real-time awareness that users gain on media capturing endeavors around them and the possibility to collect that data for personal use once the event is over. This contribution is further emphasized by the geo-referenced cues that support the overall user interface and the management of the different media streams. As a secondary contribution, we report on lessons and design guidelines that emerged and apply to in-situ design of rich video collaborative experiences and with the elicitation of functional and usability requirements related to privacy, social connections, and gamification.