Improving SMIL with NCM Facilities

SMIL is an XML-based language for specifying documents with temporal and spatial synchronization relationships, recommended by the W3C. However, SMIL has some limitations as a hypermedia authoring language. In order to overcome these limitations, this work presents the integration of SMIL documents with HyperProp, a hypermedia system based on an object-oriented model called NCM (Nested Context Model). As benefits, the integration makes both NCM facilities and HyperProp authoring and formatting tools available for SMIL documents. Furthermore, this work presents how the SMIL language can be extended to incorporate NCM facilities through the inclusion of new XML elements and attributes found in NCL, the NCM declarative language. Among the facilities, we can highlight the possibility of reusing document components and their presentation characteristics, offering flexibility in temporal specifications, adapting a document presentation according to user navigation and specifying n-ary relationships expressing causality or constraint among components.     

ZYX-a multimedia document model for reuse and adaptationof multimedia content

Advanced multimedia applications require adequate support for the modeling of multimedia content by multimedia document models. More and more this support calls for not only the adequate modeling of the temporal and spatial course of a multimedia presentation and its interactions, but also for the partial reuse of multimedia documents and adaptation to a given user context. However, our thorough investigation of existing standards for multimedia document models such as HTML, MHEG, SMIL, and HyTime leads to us the conclusion that these standard models do not provide sufficient modeling support for reuse and adaptation. Therefore, we propose a new approach for the modeling of adaptable and reusable multimedia content, the Z_YX model. The model offers primitives that provide-beyond the more or less common primitives for temporal, spatial, and interaction modeling-a variform support for reuse of structure and layout of document fragments and for the adaptation of the content and its presentation to the user context. We present the model in detail and illustrate the application and effectiveness of these concepts by samples taken from our Cardio-OP application in the domain of cardiac surgery. With the Z_YX model, we developed a comprehensive means for advanced multimedia content creation: support for template-driven authoring of multimedia content and support for flexible, dynamic composition of multimedia documents customized to the user's local context and needs. The approach significantly impacts and supports the authoring process in terms of methodology and economic aspects     

From Database to Web Multimedia Documents

This paper deals with the integration of multimedia and database technologies in order to describe web multimedia documents. We present a middleware to seamlessly handle database accesses as well as compositional, spatial and temporal constraints related to data presentation. Our approach is based on the concept of Templates. A template is a logical presentation unit that merge database queries with layout specifications. We choose an XML and SMIL approach to implement template. Template definition and invocation are mapped into a XML DTD. Each template is then translated into a SMIL document. In this paper, we give an example to show the advantages of our approach.     

Bottom-Up Generation of Hypermedia Documents

Creating hypermedia documents for educational or informational purposes is a time and money consuming task. However, automation, especially for maintaining or adapting large hypermedia documents (e.g. a web site), is a must. This paper presents an approach for the bottom-up generation of hypermedia documents. The main idea is to start with a set of mediaobjects (text, images, video, etc.), select the most appropriate ones, sequence them to build a tree-like structure, integrate them into a set of screen pages, and finally convert each screen page into a single HTML file. Maintaining the resulting document is very easy because it only requires extending or modifying the pool of mediaobjects available and regenerating the document.     

Multimedia Document Models: Sealed Fate or Setting Out for New Shores?

Existing multimedia document models like HTML, MHEG, SMIL, and HyTime lack appropriate modeling primitives to fit the needs of next generation multimedia applications which bring up requirements like reusability of multimedia content in different presentations and contexts, and adaptation to user preferences. In this paper, we motivate and present new requirements stemming from advanced multimedia applications and the resulting consequences for multimedia document models. Along these requirements, we discuss the document model standards HTML, HyTime, MHEG, SMIL, and Z_YX, a new model that has been developed with special focus on reusability and adaptation. The analysis and comparison of the models show the limitations of existing models, point the way to the need for new flexible multimedia document models, and throw light on the many implications on authoring systems, multimedia content management, and presentation.     

Context-aware adaptation of multimedia documents for consistent presentations

We propose in this paper a context-aware adaptation technique for SMIL encoded multimedia documents. Our approach is mainly dedicated to run in compile-time but we discuss also how it can be used in runtime. Our algorithm aims at improving the execution of a multimedia presentation. This is achieved by scheduling object pre-fetchings in the presentation while monitoring bandwidth and memory space availabilities. For this effect, a request pattern computed from the temporal graph of the SMIL specification is exploited in order to apply an optimal policy to select which objects should be pre-fetched, real time delivered or discarded, even. The algorithm can be used as a tool to evaluate and to monitor the QoS of a presentation. At compile time, it makes it possible to assess whether the QoS of a presentation can be guaranteed after enforcing specific context-oriented adaptations. At authoring stage, this algorithm can be used, for instance, to produce different alternative sub-codes, such that each one is adapted to run in a specific context. In runtime, the algorithm can provide a mechanism to monitor the elastic time computation in case of a mismatch occurrence. In order to advocate our proposal, some experimentation results performed on a simulation platform are reported.     

Management of Multi-structured Hypermedia Documents: A Data Model,Query Language,and Indexing Scheme

Structured documents have gained popularity with the advent of documentstructure markupstandards such as SGML, ODA, HyTime, and HTML.Document management systems can provide powerful facilities by maintaining thestructure information of documents.Since the hypermediadocument is also a kind of structured document, wecan apply the results of many studies, whichhave been performed in storing, retrieving, and managing structured documents,to the hypermedia document management.However, more factors should be considered in handling hypermedia documentsbecause they contain multimedia data and also have multiple complex structuressuch as hyperlink networks and spatial/temporal layout structures as well aslogical structures.In this paper, we propose an object-oriented model for multi-structuredhypermediadocuments and multimedia data, and a query language for retrievinghypermedia document elements based on the content and multiple complexstructures.By using unique element identifiers and an indexing scheme whichexploits multiple structures,we can process queries efficiently with minimal storage overheadfor maintaining structure information.     

Cooperative hypermedia an enabling paradigm for cooperative work

This paper describes the integration of computer-supported cooperative work (CSCW), multimedia, and hyperstructures into a single framework calledCooperative Hyper Media (CHM). The concept of CHM supports groups of users acting on a single hypermedia document. Cooperative HyperMedia is a layered concept that integrates several notions related to the production of hypermedia documents, namely, document contents, access, and organization. To test the applicability of the concepts of this approach to CSCW we have realized a generic CHM editing architecture (Co MEdiA), telecommunication mechanism for multimedia objects (Tele Media), and service for cooperative work over long distances for time-dependent media (MISTER COOL).     

Automatic authoring and construction of hypermedia for information retrieval

This paper describes the complete process and a tool for the automatic construction of a multimedia hypertext starting from a large collection of multimedia documents. Through the use of an authoring methodology, the document collection is automatically authored, and the result is a multimedia hypertext, also called a hypermedia, written in hypertext mark-up language (HTML), almost a standard among hypermedia mark-up languages. The resulting hypermedia can be browsed and queried with Mosaic, an interface developed in the framework of the World Wide Web Project. In particular, the set of methods and techniques used for the automatic construction of hypermedia is described in this paper, and their relevance in the context of multimedia information retrieval is highlighted.     

Feature pattern based representation of multimedia documents for efficient knowledge discovery

The rapid growth of multimedia documents has raised huge demand for sophisticated multimedia knowledge discovery systems. The knowledge extraction of the documents mainly relies on the data representation model and the document representation model. As the multimedia document comprised of multimodal multimedia objects, the data representation depends on modality of the objects. The multimodal objects require distinct processing and feature extraction methods resulting in different features with different dimensionalities. Managing multiple types of features is challenging for knowledge extraction tasks. The unified representation of multimedia document benefits the knowledge extraction process, as they are represented by same type of features. The appropriate document representation will benefit the overall decision making process by reducing the search time and memory requirements. In this paper, we propose a domain converting method known as Multimedia to Signal converter (MSC) to represent the multimodal multimedia document in an unified representation by converting multimodal objects as signal objects. A tree based approach known as Multimedia Feature Pattern (MFP) tree is proposed for the compact representation of multimedia documents in terms of features of multimedia objects. The effectiveness of the proposed framework is evaluated by performing the experiments on four multimodal datasets. Experimental results show that the unified representation of multimedia documents helped in improving the classification accuracy for the documents. The MFP tree based representation of multimedia documents not only reduces the search time and memory requirements, also outperforms the competitive approaches for search and retrieval of multimedia documents.     

Integrating the Amsterdam hypermedia model with the standard reference model for intelligent multimedia presentation systems

The standard reference model (SRM) for intelligent multimedia presentation systems describes a framework for the automatic generation of multimedia presentations. This framework, however, lacks an explicit document model of the presentation being generated. The Amsterdam hypermedia model (AHM) describes the document features of a hypermedia presentation explicitly. We take the AHM and use it as a basis for describing in detail the stages of generating a hypermedia presentation within the SRM framework, which we summarise in a table. By doing so, the responsibilities of the individual SRM layers become more apparent.     

A Cooperative Architecture for Hypermedia Editing – CoMEdiA

CoMEdiA is a cooperative hypermedia editing prototype, which enables co-authors to cooperatively produce hypermedia documents. It allows co-authors to communicate their ideas, drafts, guidelines, constraints, and annotations with other co-authors in order to exchange information (remotely or face-to-face), improvepassages and modify notes until a final document is achieved. We did not concentrate on the depth but on the breadth of the features. Our efforts were on integrating and coordinating concepts from collaboration, multimedia, and hyper organization rather than on making a specialized system in any of them. Our first step was to investigate issues in group editing and cooperation. Then we started by having text in our documents, later static images, and 2D-Graphics. This procedure was adopted because we felt these were the most simple media to integrate and process.     

Adapting multimedia Internet content for universal access

Content delivery over the Internet needs to address both the multimedia nature of the content and the capabilities of the diverse client platforms the content is being delivered to. We present a system that adapts multimedia Web documents to optimally match the capabilities of the client device requesting it. This system has two key components. 1) A representation scheme called the InfoPyramid that provides a multimodal, multiresolution representation hierarchy for multimedia. 2) A customizer that selects the best content representation to meet the client capabilities while delivering the most value. We model the selection process as a resource allocation problem in a generalized rate distortion framework. In this framework, we address the issue of both multiple media types in a Web document and multiple resource types at the client. We extend this framework to allow prioritization on the content items in a Web document. We illustrate our content adaptation technique with a web server that adapts multimedia news stories to clients as diverse as workstations, PDA's and cellular phones     

A study on multimedia documents behavior: a notion of equivalence

In this paper we address the problem of comparing multimedia documents, which can be described according to different reference models. If we consider pre- sentations as collections of media items and constraints among them, expressed ac- cording to their reference model, they must be translated to a common formalism in order to compare their temporal behavior and detect if they have a common component (i.e., intersection), if one of them is included in another one (i.e., inclusion), or if they have the same temporal evolution along time (i.e., equivalence). In this paper, we propose the use of automata, to describe the temporal evolution of a document, and the SMIL language as a case study, since this standard allows to describe the same behavior with different sets of tags. In case of behaviorally equivalent SMIL documents, we propose an algorithm to extract the canonical form that represents this behavior.

Multimedia Clip Generation From Documents for Browsing on Mobile Devices

Small displays on mobile handheld devices, such as personal digital assistants (PDAs) and cellular phones, are the bottlenecks for usability of most content browsing applications. Generally, conventional content such as documents and Web pages need to be modified for effective presentation on mobile devices. This paper proposes a novel visualization for documents, called multimedia thumbnails, which consists of text and image content converted into playable multimedia clips. A multimedia thumbnail utilizes visual and audio channels of small portable devices as well as both spatial and time dimensions to communicate text and image information of a single document. The proposed algorithm for generating multimedia thumbnails includes 1) a semantic document analysis step, where salient content from a source document is extracted; 2) an optimization step, where a subset of this extracted content is selected based on time, display, and application constraints; and 3) a composition step, where the selected visual and audible document content is combined into a multimedia thumbnail. Scalability of MMNails that allows generation of multimedia clips of various lengths is also described. A user study is presented that evaluates the effectiveness of the proposed multimedia thumbnail visualization.     

基于结构特性的超媒体模型

将超媒体的结构特性与形式语言学理论相结合,提供了一种有效地分析超媒体动态结构的特性的方法,该方法将超毁体组件集合映射为语言中的抽象符号集合,组件集合之上的链映射为语言中建立单词的方法,超媒体的结构化机械制映射为语言中的抽象符号集号,组件集合之上的链映射为语言中建立单词的方法,超媒体的结构化机制映射为语言中描述单词集合构造机制的语法,超媒体模拟为转换机,“链引导”动作模拟为“匹配对”序列,“链引导”的计算结果产生与上下文无关文法相对应的虚拟文档,从而为进一步研究超媒体的浏鉴语义及其它动态特性提供了理论基础。     

Analysis and verification of SMIL documents

In this paper, we consider the problem of automatic verification of SMIL documents and present a tool which can assist the user in the complex task of authoring a multimedia presentation. The tool is based on a formal semantics defining the temporal aspects of SMIL elements by means of a set of inference rules. The rules, in the spirit of Hoare’s semantics, describe how the execution of a piece of code changes the state of the computation of a player. If any temporal conflict is found, the system returns a message to the user pointing out the element which contains the conflict and its motivation. This helps the user to develop robust and clear code.     

Elastic time computation in QoS-driven hypermedia presentations

The development of hypermedia/multimedia systems requires the implementation of an element, usually known as formatter, which is in charge of receiving the specification of a document and controlling its presentation. Adjustments over the duration of media objects is one of the most important adaptation techniques that hypermedia formatters should implement in order to maintain document spatio-temporal relationships. Elastic time computation accomplishes this goal by stretching or shrinking the ideal duration of media objects. This paper presents new elastic time algorithms for adjusting hypermedia document presentations. The algorithms explore the flexibility offered by some hypermedia models in the definition of media-object durations, choosing objects to be stretched or shrunk in order to obtain the best possible quality of presentation. Our proposals are based on the “out-of-kilter” method for minimum-cost flow problems on temporal graphs. An aggregation procedure enhances the basic algorithm offering more flexibility in modeling real-life situations in comparison with other previous work based on linear programming.