特征词抽取和相关性融合的伪相关反馈查询扩展

针对现有信息检索系统中存在的词不匹配问题,提出一种基于特征词抽取和相关性融合的伪相关反馈查询扩展算法以及新的扩展词权重计算方法。该算法从前列n篇初检局部文档中抽取与原查询相关的特征词,根据特征词在初检文档集中出现的频度以及与原查询的相关度,将特征词确定为最终的扩展词实现查询扩展。实验结果表明,该方法有效,并能提高和改善信息检索性能。     

基于用户相关反馈的带结构语义的XML查询词扩展

在XML文档的信息检索中,检索质量不高的一个主要原因是用户难以提出准确描述其查询意图的查询表达式,而查询扩展技术被认为是可以帮助用户构建符合其查询意图的查询表达式.本文在XML信息检索中提出了基于用户相关反馈的查询扩展技术,在查询扩展中除了考虑词频因素外还充分考虑了XML文档的结构特点对于扩展查询词选取的影响,包括文档中元素的语义权重、元素所在层次和词项与初始查询词间的距离因素对于扩展查询词选取的影响.实验证明本方法是可行的,且能较好地提高检索结果的准确率.     

问答式信息检索查询优化技术研究

问答式信息检索是新一代搜索引擎,它接收自然语言描述的问题,在文档集合中搜索并返回问题的精确答案.问答式信息检索中,检索模块性能的提高将直接影响问题回答系统的整体性能.本文研究系统中的查询优化技术,包括两种策略:基于模式知识库的查询优化;挖掘Web语义蕴含信息,构建查询扩展资源.本文利用TREC提供的问题集与答案集(TREC8-TREC13)做实验来测试查询优化方法的性能,实验结果表明,相对于传统的查询生成,本文采用的查询优化技术在检索精度上取得了提高,t-test结果证明,系统性能提高统计显著.     

基于伪相关反馈的跨语言查询扩展

相关反馈是一种重要的查询重构技术,本文分析了两类相关反馈技术,一是按用户是否参与可分为伪相关反馈和交互式相关反馈,二是按作用于查询的方式可分为查询扩展与检索词重新加权.在此基础上,本文重点探讨了将相关反馈技术应用于跨语言信息检索,提出了翻译前查询扩展、翻译后查询扩展、翻译前与翻译后相结合的查询扩展三种方法.最后,本文通过伪相关反馈实验对这三种方法进行了比较,实验结果显示,三种跨语言查询扩展方法都能够有效地提高检索结果的精度,其中翻译后查询扩展方法相对更优越.此外,查询式的长度对不同跨语言查询扩展方法产生着不同程度的影响.     

一种基于词间关联规则挖掘的查询扩展方法

针对现有信息检索系统中存在的词不匹配问题,提出一种基于词间关联规则的查询扩展算法,该算法利用现有挖掘算法自动对前列初检文档进行词间关联挖掘,提取含有原查询词的词间关联规则,从中提取扩展词,实现查询扩展。实验结果表明,该算法能改善和提高信息检索系统的查全率和查准率,具有很高的应用价值,与未进行查询扩展时相比,采用本文查询扩展算法后,平均准确率提高了13.34%,与传统的局部上下文分析查询扩展算法比较,其平均准确率提高了4.87%。     

相关反馈技术中的检索词排序算法

相关反馈是近年来信息检索领域的研究热点，是自动查询扩展中的一种重要形式，相关反馈主要包括检索词加权和检索词选择。本文介绍了在相关反馈技术中经典的检索词排序算法，对它们带来的性能改进做了比较，并提出了相关反馈的实际应用中需要解决的一些问题。     

领域本体在信息检索中的应用研究

为了改进基于关键词的信息检索方法的局限性,论文研究了一种综合利用领域本体改善信息检索性能的方法.该方法强调通过交互式的方式引导用户一步步逼近其真实的、潜在的检索需求,使用基于编辑距离的词形匹配方法辅助用户查询本体词汇,使用基于概念空间的检索词联想方法帮助用户扩充检索词.使用基于领域本体的词义识别算法来确定文档中的词汇词义.使用XML技术实现用户查询需求和文档标注的规范化标注.实验表明,该方法会有效提升查全率并且会改进查准率.     

搜索引擎中的网页加权索引

如何对索引词加权是信息检索中的一个关键技术。本文针对搜索引擎检索的对象是Web页面这一特点 ,通过分析HTML标签的修饰功能 ,结合传统的tf idf加权公式 ,对网页进行加权索引。实验证明对于精确匹配 ,在查全率较低时系统的查准率有较大的提高。     

基于Apriori改进算法的局部反馈查询扩展

提出面向查询扩展的Apriori改进算法，采用三种剪枝策略，极大提高挖掘效率；针对现有查询扩展存在的缺陷，提出基于Apriori改进算法的局部反馈查询扩展算法，该算法用Apriori改进算法对前列初检文档进行词间关联规则挖掘，提取含有原查询词的词间关联规则，构造规则库，从库中提取扩展词，实现查询扩展。实验结果表明该算法能够提高信息检索性能，与现有算法比较，在相同查全率水平级下其平均查准率有了明显提高。     

基于单文档的上下文查询信息抽取*

提出一种结合全局分析和局部分析从单篇文档中抽取查询信息的算法。利用全局分析提取用户的查询兴趣，通过局部分析消除查询词的歧义性。实验结果表明，该方法能较全面反映用户查询的上下文信息，提高查询的相关度。     

Improving document clustering using Okapi BM25 feature weighting

We investigate the effect of feature weighting on document clustering, including a novel investigation of Okapi BM25 feature weighting. Using eight document datasets and 17 well-established clustering algorithms we show that the benefit of tf-idf weighting over tf weighting is heavily dependent on both the dataset being clustered and the algorithm used. In addition, binary weighting is shown to be consistently inferior to both tf-idf weighting and tf weighting. We investigate clustering using both BM25 term saturation in isolation and BM25 term saturation with idf, confirming that both are superior to their non-BM25 counterparts under several common clustering quality measures. Finally, we investigate estimation of the k1 BM25 parameter when clustering. Our results indicate that typical values of k1 from other IR tasks are not appropriate for clustering; k1 needs to be higher.     

Evolving local and global weighting schemes in information retrieval

This paper describes a method, using Genetic Programming, to automatically determine term weighting schemes for the vector space model. Based on a set of queries and their human determined relevant documents, weighting schemes are evolved which achieve a high average precision. In Information Retrieval (IR) systems, useful information for term weighting schemes is available from the query, individual documents and the collection as a whole. We evolve term weighting schemes in both local (within-document) and global (collection-wide) domains which interact with each other correctly to achieve a high average precision. These weighting schemes are tested on well-known test collections and are compared to the traditional tf-idf weighting scheme and to the BM25 weighting scheme using standard IR performance metrics. Furthermore, we show that the global weighting schemes evolved on small collections also increase average precision on larger TREC data. These global weighting schemes are shown to adhere to Luhn’s resolving power as both high and low frequency terms are assigned low weights. However, the local weightings evolved on small collections do not perform as well on large collections. We conclude that in order to evolve improved local (within-document) weighting schemes it is necessary to evolve these on large collections.     

A novel neighborhood based document smoothing model for information retrieval

In this paper, a novel neighborhood based document smoothing model for information retrieval has been proposed. Lexical association between terms is used to provide a context sensitive indexing weight to the document terms, i.e. the term weights are redistributed based on the lexical association with the context words. A generalized retrieval framework has been presented and it has been shown that the vector space model (VSM), divergence from randomness (DFR), Okapi Best Matching 25 (BM25) and the language model (LM) based retrieval frameworks are special cases of this generalized framework. Being proposed in the generalized retrieval framework, the neighborhood based document smoothing model is applicable to all the indexing models that use the term-document frequency scheme. The proposed smoothing model is as efficient as the baseline retrieval frameworks at runtime. Experiments over the TREC datasets show that the neighborhood based document smoothing model consistently improves the retrieval performance of VSM, DFR, BM25 and LM and the improvements are statistically significant.     

A nonparametric term weighting method for information retrieval based on measuring the divergence from independence

In this article, we introduce an out-of-the-box automatic term weighting method for information retrieval. The method is based on measuring the degree of divergence from independence of terms from documents in terms of their frequency of occurrence. Divergence from independence has a well-establish underling statistical theory. It provides a plain, mathematically tractable, and nonparametric way of term weighting, and even more it requires no term frequency normalization. Besides its sound theoretical background, the results of the experiments performed on TREC test collections show that its performance is comparable to that of the state-of-the-art term weighting methods in general. It is a simple but powerful baseline alternative to the state-of-the-art methods with its theoretical and practical aspects.     

A study of untrained models for multimodal information retrieval

Operational multimodal information retrieval systems have to deal with increasingly complex document collections and queries that are composed of a large set of textual and non-textual modalities such as ratings, prices, timestamps, geographical coordinates, etc. The resulting combinatorial explosion of modality combinations makes it intractable to treat each modality individually and to obtain suitable training data. As a consequence, instead of finding and training new models for each individual modality or combination of modalities, it is crucial to establish unified models, and fuse their outputs in a robust way. Since the most popular weighting schemes for textual retrieval have in the past generalized well to many retrieval tasks, we demonstrate how they can be adapted to be used with non-textual modalities, which is a first step towards finding such a unified model. We demonstrate that the popular weighting scheme BM25 is suitable to be used for multimodal IR systems and analyze the underlying assumptions of the BM25 formula with respect to merging modalities under the so-called raw-score merging hypothesis, which requires no training. We establish a multimodal baseline for two multimodal test collections, show how modalities differ with respect to their contribution to relevance and the difficulty of treating modalities with overlapping information. Our experiments demonstrate that our multimodal baseline with no training achieves a significantly higher retrieval effectiveness than using just the textual modality for the social book search 2016 collection and lies in the range of a trained multimodal approach using the optimal linear combination of the modality scores.     

#nowplaying Madonna: a large-scale evaluation on estimating similarities between music artists and between movies from microblogs

Different term weighting techniques such as $TF\cdot IDF$ or BM25 have been used intensely for manifold text-based information retrieval tasks. Their use for modeling term profiles for named entities and subsequent calculation of similarities between these named entities have been studied to a much smaller extent. The recent trend of microblogging made available massive amounts of information about almost every topic around the world. Therefore, microblogs represent a valuable source for text-based named entity modeling. In this paper, we present a systematic and comprehensive evaluation of different term weighting measures, normalization techniques, query schemes, index term sets, and similarity functions for the task of inferring similarities between named entities, based on data extracted from microblog posts. We analyze several thousand combinations of choices for the above mentioned dimensions, which influence the similarity calculation process, and we investigate in which way they impact the quality of the similarity estimates. Evaluation is performed using three real-world data sets: two collections of microblogs related to music artists and one related to movies. For the music collections, we present results of genre classification experiments using as benchmark genre information from allmusic.com . For the movie collection, we present results of multi-class classification experiments using as benchmark categories from IMDb . We show that microblogs can indeed be exploited to model named entity similarity with remarkable accuracy, provided the correct settings for the analyzed aspects are used. We further compare the results to those obtained when using Web pages as data source.     

A novel term weighting scheme for text classification: TF-MONO

The effective representation of the relationship between the documents and their contents is crucial to increase classification performance of text documents in the text classification. Term weighting is a preprocess aiming to represent text documents better in Vector Space by assigning proper weights to terms. Since the calculation of the appropriate weight values directly affects performance of the text classification, in the literature, term weighting is still one of the important sub-research areas of text classification. In this study, we propose a novel term weighting (MONO) strategy which can use the non-occurrence information of terms more effectively than existing term weighting approaches in the literature. The proposed weighting strategy also performs intra-class document scaling to supply better representations of distinguishing capabilities of terms occurring in the different quantity of documents in the same quantity of class. Based on the MONO weighting strategy, two novel supervised term weighting schemes called TF-MONO and SRTF-MONO were proposed for text classification. The proposed schemes were tested with two different classifiers such as SVM and KNN on 3 different datasets named Reuters-21578, 20-Newsgroups, and WebKB. The classification performances of the proposed schemes were compared with 5 different existing term weighting schemes in the literature named TF-IDF, TF-IDF-ICF, TF-RF, TF-IDF-ICSDF, and TF-IGM. The results obtained from 7 different schemes show that SRTF-MONO generally outperformed other schemes for all three datasets. Moreover, TF-MONO has promised both Micro-F1 and Macro-F1 results compared to other five benchmark term weighting methods especially on the Reuters-21578 and 20-Newsgroups datasets.     

一种用于主题提取的非线性加权方法

主题提取是文本处理的一项重要工作。本文首先分析了主题抽取中加权方法形成时的一些定量问题,然后提出了主题相关词一种非线性加权处理方法,对比实验结果显示它不仅是一种比较稳健的方法,而且能在一定程度上提高主题提取的正确率。     

元素级XML检索模型构建的关键问题与解决方案研究

与传统信息检索不同的是XML要实现元素级的检索,其核心是元素级检索模型的构建。而XML文档内上下文元素的相关性、元素之间信息的重复性以及元素大小的不一性等则是构建模型时面临的核心问题。解决办法是:构建基于BM25元素级XML检索模型,构建基于上下文的元素级XML检索模型BM25E,过滤重复元素,进行可检索元素的选择和太小元素的处理。表1。图1。参考文献19。