A usage study of retrieval modalities for video shot retrieval

As an information medium, video offers many possible retrieval and browsing modalities, far more than text, image or audio. Some of these, like searching the text of the spoken dialogue, are well developed, others like keyframe browsing tools are in their infancy, and others not yet technically achievable. For those modalities for browsing and retrieval which we cannot yet achieve we can only speculate as to how useful they will actually be, but we do not know for sure. In our work we have created a system to support multiple modalities for video browsing and retrieval including text search through the spoken dialogue, image matching against shot keyframes and object matching against segmented video objects. For the last of these, automatic segmentation and tracking of video objects is a computationally demanding problem which is not yet solved for generic natural video material, and when it is then it is expected to open up possibilities for user interaction with objects in video, including searching and browsing. In this paper we achieve object segmentation by working in a closed domain of animated cartoons. We describe an interactive user experiment on a medium-sized corpus of video where we were able to measure users’ use of video objects versus other modes of retrieval during multiple-iteration searching. Results of this experiment show that although object searching is used far less than text searching in the first iteration of a user’s search it is a popular and useful search type once an initial set of relevant shots have been found.     

A hybrid approach to managing job offers and candidates

The evolution of the job market has resulted in traditional methods of recruitment becoming insufficient. As it is now necessary to handle volumes of information (mostly in the form of free text) that are impossible to process manually, an analysis and assisted categorization are essential to address this issue. In this paper, we present a combination of the E-Gen and Cortex systems. E-Gen aims to perform analysis and categorization of job offers together with the responses given by the candidates. E-Gen system strategy is based on vectorial and probabilistic models to solve the problem of profiling applications according to a specific job offer. Cortex is a statistical automatic summarization system. In this work, E-Gen uses Cortex as a powerful filter to eliminate irrelevant information contained in candidate answers. Our main objective is to develop a system to assist a recruitment consultant and the results obtained by the proposed combination surpass those of E-Gen in standalone mode on this task.     

ReGR: Relation-aware graph reasoning framework for video question answering

As one of the challenging cross-modal tasks, video question answering (VideoQA) aims to fully understand video content and answer relevant questions. The mainstream approach in current work involves extracting appearance and motion features to characterize videos separately, ignoring the interactions between them and with the question. Furthermore, some crucial semantic interaction details between visual objects are overlooked. In this paper, we propose a novel Relation-aware Graph Reasoning (ReGR) framework for video question answering, which first combines appearance–motion and location–semantic multiple interaction relations between visual objects. For the interaction between appearance and motion, we design the Appearance–Motion Block, which is question-guided to capture the interdependence between appearance and motion. For the interaction between location and semantics, we design the Location–Semantic Block, which utilizes the constructed Multi-Relation Graph Attention Network to capture the geometric position and semantic interaction between objects. Finally, the question-driven Multi-Visual Fusion captures more accurate multimodal representations. Extensive experiments on three benchmark datasets, TGIF-QA, MSVD-QA, and MSRVTT-QA, demonstrate the superiority of our proposed ReGR compared to the state-of-the-art methods.     

信息检索系统的相关词提示技术与评测

在Web信息检索中,为了明确用户的查询需求,很多搜索引擎和全文数据库提供了相关词提示功能。本文简要介绍了Web信息检索中相关词提示的获取技术,并对相关词提示效果进行实际调查分析。从关键词库中随机抽取若干关键词,在选定的搜索引擎和全文数据库上进行信息检索,获取抽样关键词的相关提示词。通过关键词检索、人工打分和数据统计,进行查询扩展分析、查询式专指度分析和查准率分析,给出相关词提示在改善检索效果和用户满意度方面的综合评价。     

PDF文档视频和动画添加——科技论文可视化发表探讨

【目的】 在PDF格式科技论文中添加视频和动画等动态视觉表征形式以反映科学实验连续变化的视觉过程。【方法】 采用格式工厂和狸窝全能视频转换器转换并同步合并和编辑视频和动画,再利用Adobe Acrobat XI将视频和动画添加入PDF文档。【结果】 视频和动画转换为FLV、F4V、SWF格式后,可利用Adobe Acrobat XI嵌入到PDF文档中,并可用Adobe Reader播放。【结论】 学术期刊数字出版普遍采用PDF作为标准格式,在PDF格式科技论文中可以嵌入视频、动画,直观呈现科学研究动态视觉过程,从本质上提升科技论文的表现形式和内容,使其变得形象生动和易于理解。     

Best entry points for structured document retrieval—Part II: Types,usage and effectiveness

Structured document retrieval makes use of document components as the basis of the retrieval process, rather than complete documents. The inherent relationships between these components make it vital to support users’ natural browsing behaviour in order to offer effective and efficient access to structured documents. This paper examines the concept of best entry points, which are document components from which the user can browse to obtain optimal access to relevant document components. It investigates at the types of best entry points in structured document retrieval, and their usage and effectiveness in real information search tasks.     

Plan and generate: Explicit and implicit variational augmentation for multi-document summarization of scientific articles

Multi-Document Summarization of Scientific articles (MDSS) is a challenging task that aims to generate concise and informative summaries for multiple scientific articles on a particular topic. However, despite recent advances in abstractive models for MDSS, grammatical correctness and contextual coherence remain challenging issues. In this paper, we introduce EDITSum, a novel abstractive MDSS model that leverages sentence-level planning to guide summary generation. Our model incorporates neural topic model information as explicit guidance and sequential latent variables information as implicit guidance under a variational framework. We propose a hierarchical decoding strategy that generates the sentence-level planning by a sentence decoder and then generates the final summary conditioned on the planning by a word decoder. Experimental results show that our model outperforms previous state-of-the-art models by a significant margin on ROUGE-1 and ROUGE-L metrics. Ablation studies demonstrate the effectiveness of the individual modules proposed in our model, and human evaluations provide strong evidence that our model generates more coherent and error-free summaries. Our work highlights the importance of high-level planning in addressing intra-sentence errors and inter-sentence incoherence issues in MDSS.     

Willingness to pay and experienced utility as measures of affective value of information objects: Users’ accounts

The article discusses two instant measures of affective value of information objects: Willingness-to-Pay (WTP) and Experienced Utility (EU). The potential utility of these measures for information retrieval research is positioned within a broader discussion of distinctions between instant utility and remembered utility and their relationship to immediate as opposed to retrospective subjective evaluations. The paper describes an approach for obtaining instant WTP and EU ratings in document evaluation in contexts of information search and thereafter reports results from a usability study of its implementation. Participants encountered no difficulties in use of the design format and in rating documents using WTP and EU measures. Although two measures were found statistically significantly correlated with each other, participants in the usability study offered different reasons for the ratings of WTP and EU made for the same document.     

The appraisal and management of uncertainty: Implications for information-retrieval systems

Uncertainty is an important idea in information-retrieval (IR) research, but the concept has yet to be fully elaborated and explored. Common assumptions about uncertainty are (a) that it is a negative (anxiety-producing) state and (b) that it will be reduced through information search and retrieval. Research in the domain of uncertainty in illness, however, has demonstrated that uncertainty is a complex phenomenon that shares a complicated relationship with information. Past research on people living with HIV and individuals who have tested positive for genetic risk for different illnesses has revealed that information and the reduction of uncertainty can, in fact, produce anxiety, and that maintaining uncertainty can be associated with optimism and hope. We review the theory of communication and uncertainty management and offer nine principles based on that theoretical work that can be used to influence IR system design. The principles reflect a view of uncertainty as a multi-faceted and dynamic experience, one subject to ongoing appraisal and management efforts that include interaction with and use of information in a variety of forms.     

TCACNet: Temporal and channel attention convolutional network for motor imagery classification of EEG-based BCI

Brain–computer interface (BCI) is a promising intelligent healthcare technology to improve human living quality across the lifespan, which enables assistance of movement and communication, rehabilitation of exercise and nerves, monitoring sleep quality, fatigue and emotion. Most BCI systems are based on motor imagery electroencephalogram (MI-EEG) due to its advantages of sensory organs affection, operation at free will and etc. However, MI-EEG classification, a core problem in BCI systems, suffers from two critical challenges: the EEG signal’s temporal non-stationarity and the nonuniform information distribution over different electrode channels. To address these two challenges, this paper proposes TCACNet, a temporal and channel attention convolutional network for MI-EEG classification. TCACNet leverages a novel attention mechanism module and a well-designed network architecture to process the EEG signals. The former enables the TCACNet to pay more attention to signals of task-related time slices and electrode channels, supporting the latter to make accurate classification decisions. We compare the proposed TCACNet with other state-of-the-art deep learning baselines on two open source EEG datasets. Experimental results show that TCACNet achieves 11.4% and 7.9% classification accuracy improvement on two datasets respectively. Additionally, TCACNet achieves the same accuracy as other baselines with about 50% less training data. In terms of classification accuracy and data efficiency, the superiority of the TCACNet over advanced baselines demonstrates its practical value for BCI systems.     

Learning assessments in search-as-learning: A survey of prior work and opportunities for future research

People often search for information in order to learn something new. In recent years, the “search-as-learning” movement has argued that search systems should be better designed to support learning. Current search systems (especially Web search engines) are largely designed and optimized to fulfill simple look-up tasks (e.g., navigational or fact-finding search tasks). However, they provide less support for searchers working on complex tasks that involve learning. Search-as-learning studies have investigated a wide range of research questions. For example, studies have aimed to better understand how characteristics of the individual searcher, the type of search task, and interactive features provided by the system can influence learning outcomes. Learning assessment is a key component in search-as-learning studies. Assessment materials are used to both gauge prior knowledge and measure learning during or after one or more search sessions. In this paper, we provide a systematic review of different types of assessments used in search-as-learning studies to date. The paper makes the following three contributions. First, we review different types of assessments used and discuss their potential benefits and drawbacks. Second, we review assessments used outside of search-as-learning, which may provide insights and opportunities for future research. Third, we provide recommendations for future research. Importantly, we argue that future studies should clearly define learning objectives and develop assessment materials that reliably capture the intended type of learning. For example, assessment materials should test a participant’s ability to engage with specific cognitive processes, which may range from simple (e.g., memorization) to more complex (e.g., critical and creative thinking). Additionally, we argue that future studies should consider two dimensions that are understudied in search-as-learning: long-term retention (i.e., being able to use what was learned in the long term) and transfer of learning (i.e., being able to use what was learned in a novel context).     

QCS: A system for querying, clustering and summarizing documents

Information retrieval systems consist of many complicated components. Research and development of such systems is often hampered by the difficulty in evaluating how each particular component would behave across multiple systems. We present a novel integrated information retrieval system—the Query, Cluster, Summarize (QCS) system—which is portable, modular, and permits experimentation with different instantiations of each of the constituent text analysis components. Most importantly, the combination of the three types of methods in the QCS design improves retrievals by providing users more focused information organized by topic.We demonstrate the improved performance by a series of experiments using standard test sets from the Document Understanding Conferences (DUC) as measured by the best known automatic metric for summarization system evaluation, ROUGE. Although the DUC data and evaluations were originally designed to test multidocument summarization, we developed a framework to extend it to the task of evaluation for each of the three components: query, clustering, and summarization. Under this framework, we then demonstrate that the QCS system (end-to-end) achieves performance as good as or better than the best summarization engines.Given a query, QCS retrieves relevant documents, separates the retrieved documents into topic clusters, and creates a single summary for each cluster. In the current implementation, Latent Semantic Indexing is used for retrieval, generalized spherical k-means is used for the document clustering, and a method coupling sentence “trimming” and a hidden Markov model, followed by a pivoted QR decomposition, is used to create a single extract summary for each cluster. The user interface is designed to provide access to detailed information in a compact and useful format.Our system demonstrates the feasibility of assembling an effective IR system from existing software libraries, the usefulness of the modularity of the design, and the value of this particular combination of modules.     

Integrating textual and visual information for cross-language image retrieval: A trans-media dictionary approach

This paper explores the integration of textual and visual information for cross-language image retrieval. An approach which automatically transforms textual queries into visual representations is proposed. First, we mine the relationships between text and images and employ the mined relationships to construct visual queries from textual ones. Then, the retrieval results of textual and visual queries are combined. To evaluate the proposed approach, we conduct English monolingual and Chinese–English cross-language retrieval experiments. The selection of suitable textual query terms to construct visual queries is the major issue. Experimental results show that the proposed approach improves retrieval performance, and use of nouns is appropriate to generate visual queries.     

Unifying telescope and microscope: A multi-lens framework with open data for modeling emerging events

Open data is becoming ubiquitous as governments, companies, and even individuals have the option to offer more or less unrestricted access to their non-sensitive data. The benefits of open data, such as accessibility and transparency, have motivated and enabled a large number of research studies and applications in both academia and industry. However, each open data only offers a single perspective, and its potential inherent limitations (e.g., demographic biases) may lead to poor decisions and misjudgments. This paper discusses how to create and use multiple digital lenses empowered by open data, including census data (macro lens), search logs (meso lens), and social data (micro lens), to investigate general real-world events. To reveal the unique angles and perspectives brought by each open lens, we summarize and compare the underpinning open data from eleven dimensions, such as utility, data volume, dynamic variability, and demographic fairness. Then, we propose an easy-to-use and generalized open data driven framework, which automatically retrieves multi-source data, extracts features, and trains machine learning models for the event specified by answering what, when, and where questions. With low labor efforts, the framework’s generalization and automation capabilities guarantee an instant investigation of general events and phenomena, such as disasters, sports events, and political activities. We also conduct two case studies, i.e., the COVID-19 pandemic and Great American Eclipse (see Appendix), to demonstrate its feasibility and effectiveness at different time granularities.     

Ideas: A system for international data exchange and access for science

An operational prototype inhomogeneous distributed database system has been built to provide homogeneous access for researchers and managers of science policy to data on research projects held in three countries. The architecture is sufficiently general not only to accommodate additional countries and databases on research projects, but also to provide a general framework for other international collaborative projects aimed at the provision of information. The design exhibits an optimal compromise between sufficient data to satisfy a retrieval request and the minimal data for transmission to maintain the database.     

Search history support for finding and using information: User interface design recommendations from a user study

Recording search histories, presenting them to the searcher, and building additional interface tools on them offer many opportunities for supporting user tasks in information seeking and use. This study investigated the use of search history information in legal information seeking. Qualitative methods were used to explore how attorneys and law librarians used their memory and external memory aids while searching for information and in transferring to information use. Based on the findings, interface design recommendations were made for information systems.