4K视频流异构多核的多路分屏方法研究

针对视频多屏显示系统无法依据分屏数量进行超高清视频自适应分屏传输的问题,提出了异构多核的视频流传输方法。采用基于ARM的嵌入式系统实现多任务处理与实时监控;通过FPGA实现对视频流进行接收、转换、处理及分屏输出显示的硬件加速。系统可以根据输入信号实时配置FPGA的工作参数,实现分辨率大小、分屏数量可变的超高清视频流分屏显示。测试结果表明,4K视频多路分屏画面拼接无明显错位、同步一致,较好地满足了视频多路分屏显示要求。     

基于DM8168的小型机载视频记录仪设计

针对飞行器搭载相机视频存储和传输的需求,设计了一种基于异构多核处理器DM8168的小型机载视频记录仪。首先,基于DM8168和FPGA处理器设计小型化低功耗的硬件系统;接着,基于McFW软件架构设计视频采集编码的数据流,充分发挥DM8168多核体系结构的优势,实现两路1080P60视频的采集,并且对每路视频分别进行主次码流的H.264编码;最后,采用GPMC总线,将视频码流传输到FPGA处理器,以实现本地视频存储和远程视频传输。本记录仪体积小功耗低、记录时间长、视频传输时延小,能够很好地满足飞行器多路视频记录的需求。     

FPGA+DSP异构视频处理系统中基于SRIO的数据高效传输方法

数据传输一直是影响嵌入式视频系统实时处理能力的关键环节.随着视频应用的多路化和高清化,混合多处理器结构已成为嵌入式视频处理系统的主要发展趋势,异构处理器之间数据的高效传输对系统性能的影响变得比以前更加突出.文中针对FPGA+ DSP异构视频处理系统中的数据传输问题,在分析处理器结构和视频数据格式特征的基础上,提出了一种基于高速串行接口SRIO(Serial Rapid I/O)的数据高效传输方法.该方法分别以FPGA、DSP作为系统的传输、处理核心,在FPGA处理器上采用视频三分量数据重组方法并使用包头信息较小的SRIO流写事务SWRITE(Streaming Write),简化视频传输格式的同时提高了SRIO视频数据包的传输效率;在DSP处理器上通过预定义接收端数据存储单元和采用简洁的SRIO门铃事务(DOORBELL)应答机制,节约了DSP在传输过程中的时间开销.实验结果表明,文中设计的SRIO高效传输方法在占用较少的FPGA资源的条件下传输速度达理论值的81％以上.     

异构多核的全高清H264解码系统设计

利用异构多核处理器OMAP4430在多媒体视频编解码上的运算优势,实现了全高清H264视频的实时解码。采用virtio缓存队列和RPMsg消息框架来实现基于异步通知的异构多核间数据通信,并结合TI公司IVA-HD多媒体硬件加速引擎和针对多核进行的合理任务分配机制来提高解码器的工作效率。实验结果表明,此方法实现的解码速度达到35F/s,能够完成对1080P全高清视频的实时解码。     

一种异构多核处理器嵌入式实时操作系统构架设计

由于异构多核处理器和多处理器系统及同构多核处理器的构架存在很大差别,应用于多处理器系统的分布式结构以及应用于同构多核系统的主从式结构操作系统不能解决异构多核处理器的实时调度和效率问题。对异构多核处理器的特点及发展趋势进行了研究,提出了一种适用异构多核处理器的多主模式实时操作系统构架。这种构架将通信总线中的多主模式引入多核操作系统构架中,采用对称式结构及组件模式设计操作系统模型,使多核处理器中每个内核都可以作为主核实现对资源、任务的实时管理,提高系统性能,同时可以解决主从式操作系统存在的由于处理器核增多而带来的主内核不能满足系统性能要求的瓶颈问题。通过这种单一构架模型可以进行灵活配置,以适应不同结构及功能要求的处理器内核,降低操作系统开发难度。     

基于FPGA高速实时图像数据处理系统的研究

本文对在FPGA中嵌入Powerpc(hard IP Core),引入linux嵌入式操作系统用来开发嵌入式系统进行了研究,系统实现了高速数据采集、高速视频数据压缩、实时视频数据的网络传输和实时压缩图像数据存储功能,该方法采用linux嵌入式操作系统加FPGA中嵌入的Powerpc硬核处理器进行系统控制层面的处理,还根据FPGA的特点将视频处理等功能用高速硬件实时处理。系统适合于系统空间要求小且视频数据处理量大的应用。     

无线多路视频传输调度协议模型的设计

针对无线视频传输中如何对多个场景实现实时监控的问题,提出一种无线多路视频流传输调度的协议模型。该协议模型主要包括视频传输起始阶段通信双方的确立方式、基于连接状态的多路视频传输调度以及通信数据流的加密控制方法。该模型适用于任何面向IP网络的无线多路视频传输,已在基于窄带CDMA1x网络的无线视频传输系统中取得了良好的应用效果。     

基于FPGA的USB3.0高清视频传输系统的设计

针对USB2. 0接口由于速率和带宽不够而无法满足高清视频实时传输的问题,提出了一种基于FPGA的USB3.0高清视频传输系统方案,并从吞吐速率角度分析了方案的可行性;采用SDRAM缓存视频数据和在视频流中插入帧同步码的方法,保证视频图像的完整和像素对齐;对系统进行了模拟测试和实际高清视频传输测试,测试结果表明:该系统实现了高清视频的实时传输,以320 MB/s的速度稳定传输视频数据。     

基于FPGA的多运动目标检测及硬件实现

针对视频流图像的目标检测,基于帧间差分理论提出了一种实时多运动目标检测算法。根据视频图像在实时处理中的应用,基于FPGA硬件平台并行处理快与可编程的特性,在FPGA上实现算法,经过MATLAB环境算法验证,并在设计的FPGA视频图像处理系统进行实现。该系统主要由图像采集模块、图像缓存模块、图像处理模块及图像显示模块4部分组成,通过OV5640采集图像数据,再对数据进行差分、滤波、形态学等一系列算法处理,存入DDR3中,最后通过VGA进行显示。实验结果表明,该系统在一定测距范围内可稳定跟踪运动目标,可实时显示,多目标检测识别率较高,能实时检测视频流图像中的多运动目标。     

无线视频监控中心的多路视频流调度模型

提出了一种用于无线视频监控中心多路视频流调度的通用模型,以实现对多个场景的实时监控.该模型主要包括:远程监控设备的授权管理策略、视频流对象的工作模型、基于连接管理线程的多路视频流调度及抖动缓存自适应累积策略.目前,该模型已经应用于现场视频无线动态传输系统,取得了良好的应用效果.     

The Application of Video Semantics and Theme Representation in Automated Video Editing

This paper considers the automated generation of humorous video sequences from arbitrary video material. We present a simplified model of the editing process. We then outline our approach to narrativity and visual humour, discuss the problems of context and shot-order in video and consider influences on the editing process. We describe the role of themes and semantic fields in the generation of content oriented video scenes. We then present the architecture of AUTEUR, an experimental system that embodies mechanisms to interpret, manipulate and generate video. An example of a humorous video sequence generated by AUTEUR is described.     

VCMF: A Framework for Video Content Modeling

Current approaches to modeling the structure and semantics of video recordings restrict its reuse. This is because these approaches are either too rigidly structured or too generally structured and so do not represent the structural and semantic regularities of classes of video recordings. This paper proposes a framework which tackles the problem of reuse by supporting the definition of a wide range of models of video recordings and supporting reuse between them. Examples of the framework's use are presented and examined with respect to different kinds of reuse of video, current research, and the development of a toolset to support the framework.     

一种整体的视频匹配方法

给出一种视频时空配准的整体方法,提出一种视频内匹配与视频间匹配相结合的空间配准策略,改进动态时间扭曲方法以用于时间维的对齐.视频内匹配跟踪视频内各帧图像的特征点并记录其轨迹,视频间匹配配准不同视频的帧图像,使用轨迹对应提供图像配准所需的初始特征点对应,根据图像配准得到的特征点对应建立和更新轨迹对应.该匹配策略充分利用了视频的连贯性提高了匹配的稳定性和效率,同时提高了配准视频的连贯性.改进的动态时间扭曲方法通过极小化两段视频的整体距离建立视频之间的帧对应关系,保持视频内部各帧之间的时序关系并能处理非线性偏移     

Model-Based Video Classification toward Hierarchical Representation,Indexing and Access

In this paper, we develop a content-based video classification approach to support semantic categorization, high-dimensional indexing and multi-level access. Our contributions are in four points: (a) We first present a hierarchical video database model that captures the structures and semantics of video contents in databases. One advantage of this hierarchical video database model is that it can provide a framework for automatic mapping from high-level concepts to low-level representative features. (b) We second propose a set of useful techniques for exploiting the basic units (e.g., shots or objects) to access the videos in database. (c) We third suggest a learning-based semantic classification technique to exploit the structures and semantics of video contents in database. (d) We further develop a cluster-based indexing structure to both speed-up query-by-example and organize databases for supporting more effective browsing. The applications of this proposed multi-level video database representation and indexing structures for MPEG-7 are also discussed.     

Columbia's VoD and Multimedia Research Testbed with Heterogeneous Network Support

This paper reports our progress in developing an advanced video-on-demand (VoD) testbed, which will be used to accommodate various multimedia research and applications such as Electronic News on Demand, Columbia's Video Course Network, and Digital Libraries. The testbed supports delivery of MPEG-2 audio/video stored as transport streams over various types of networks, e.g., ATM, Ethernet, and wireless. Both software and hardware video encoders/decoders are used in the testbed. A real-time video pump and a distributed application control protocol (MPEG-2's DSM-CC) have been incorporated. Hardware decoders and set-tops are being used to test wide-area video interoperability. Our VoD testbed also provides an advanced platform for implementing proof-of-concept prototypes of related research. Our current research focus covers video transmission with heterogeneous quality-of-service (QoS) provision, variable bitrate (VBR) traffic modeling, VBR server scheduling, video over Internet, and video transmission over IP-ATM hybrid networks. An important aim is to enhance interoperability. Accommodation of practical multimedia applications and interoperability testing with external VoD systems has also been undertaken.     

Metadata-driven interactive web video assembly

The recent expansion of broadband Internet access led to an exponential increase of potential consumers of video on the Web. The huge success of video upload websites shows that the online world, with its virtually unlimited possibilities of active user participation, is an ideal complement to traditional consumption-only media like TV and DVD. It is evident that users are willing to interact with content-providing systems in order to get the content they desire. In parallel to these developments, innovative tools for producing interactive, non-linear audio-visual content are being created. They support the authoring process alongside management of media and metadata, enabling on-demand assembly of videos based on the consumer’s wishes. The quality of such a dynamic video remixing system mainly depends on the expressiveness of associated metadata. Eliminating the need for manual input as far as possible, we aim at designing a system which is able to automatically enrich its own media and metadata repositories continuously. Currently, video content remixing is available on the Web mostly in very basic forms. Most platforms offer upload and simple modification of content. Although several implementations exist, to the best of our knowledge no solution uses metadata to its full extent to dynamically render a video stream based on consumers’ wishes. With the research presented in this paper, we propose a novel concept to interactive video assembly on the Web. In this approach, consumers may describe the desired content using a set of domain-specific parameters. Based on the metadata the video clips are annotated with, the system chooses clips fitting the user criteria. They are aligned in an aesthetically pleasing manner while the user furthermore is able to interactively influence content selection during playback at any time. We use a practical example to clarify the concept and further outline what it takes to implement a suchlike system.

Detecting Roads in Stabilized Video with the Spatio-Temporal Structure Tensor

Video provides strong cues for automatic road extraction that are not available in static aerial images. In video from a static camera, or stabilized (or geo-referenced) aerial video data, motion patterns within a scene enable function attribution of scene regions. A “road”, for example, may be defined as a path of consistent motion — a definition which is valid in a large and diverse set of environments. The spatio-temporal structure tensor field is an ideal representation of the image derivative distribution at each pixel because it can be updated in real time as video is acquired. An eigen-decomposition of the structure tensor encodes both the local scene motion and the variability in the motion. Additionally, the structure tensor field can be factored into motion components, allowing explicit determination of traffic patterns in intersections. Example results of a real time system are shown for an urban scene with both well-traveled and infrequently traveled roads, indicating that both can be discovered simultaneously. The method is ideal in urban traffic scenes, which are the most difficult to analyze using static imagery.     

海量监控视频快速回放与检索技术

为解决海量监控视频的快速浏览和检索,介绍了一种基于目标索引的视频摘要和检索方法。该方法在光流分析的基础上,在画面的静止区域更新背景,运动的区域利用差分法分割出运动目标图像。经过优化的快速特征匹配和建立运动跟踪模型后,根据目标运动轨迹,按照时空距离进行聚类。在目标图像数据和运动参数进行XML结构化存储为索引的基础上,最后在检索时将符合条件的所有目标图像,按照其原有时间顺序逐帧贴到同一个背景图像中,形成动态的摘要视频。由于该方法剔除了背景中大量的时空冗余信息,可在较短回放时间内浏览全部有用目标,显著提高海量监控视频的查阅效率。     

VisualGREP: A Systematic Method to Compare and Retrieve Video Sequences

Multimedia Tools and Applications - In this paper, we consider the problem of similarity between video sequences. Three basic questions are raised and (partially) answered. Firstly, at what...     

基于VFW的远程视频捕获技术研究

多媒体技术在人们日常生活中的应用越来越广泛,图像、视频、音频等多媒体数据逐渐成为信息处理领域中主要的信息媒体形式。视频捕获技术是信息处理中的重要环节,研究该项技术具有重要的实用价值。文章提出一种基于VFW的远程视频捕获方法。该方法利用VFW捕获视频数据,采用H.263编码标准压缩视频数据,利用面向连接协议的流式套接字实现实时视频流的传输,结合多线程技术实现视频文件播放。然后,基于Windows操作系统设计实现了远程视频捕获系统。实验结果表明,该方法CPU占用率低、内存占用小,可靠性强,具有较好的应用价值。