一种基于敏感站点处理的Web信息检索系统

本文设计实现了一种Web信息检索系统,面向有特定需求的特殊用户群,采用基于web站点处理的情报采集策略。先对各站点页面随机采样,提取出包含敏感信息页面的web站点,再采集敏感站点中的相关页面生成本地敏感资源库,并对库中的文本页面用改进的TFIDF算法分析处理,以满足用户的查询。该系统能够提高Web页面信息的检索精确度和检测更新率,并可据某一专题方向对Web站点进行简单的自动分类。

Web Information Retrieval System Based on Sensitive Site Processing

The paper studies a Web information retrieval system facing the special users, which applies a sensitive- site- processing policy. Firstly the system randomly copies pages from a group of Web sites using sampling, analyses the URLs from the pages which include the sensitive information, secondly it crawls the sensitive Web sites and copies the relevant pages to a local cache. The sys- tem analyses the pages using a TFIDF algorithm for the user' s query. The system could effectively improve the precision of infor- mation retrieval and maintains the pages up- to- date. It could try automatically and alternatively classify the Web sites.