一种评价搜索引擎信息覆盖率的模型及其验证

搜索引擎的网页搜集子系统通常以WWW的网页构成的有向图结构为依据,循着网页间的链接进行搜集从而扩大信息覆盖面.本文针对这种信息覆盖能力,建立量化模型从多个角度考察搜集系统对WWW信息资源的覆盖程度.文章首先分析了网页搜集不完全性的若干因素,在指出信息覆盖率的研究意义后提出了三类重要的信息覆盖率概念,然后围绕其中的数量和质量覆盖率展开研究工作.在建立"采样-权值计算-验证"的覆盖率评测模型之后,以北大"燕穹"网页信息博物馆为考察对象并获得其网页数据,用不同的方式对中国Web进行采样;然后分别采用PageRank和HITS两种网页权值算法算出其中的重要网页作为样本,从量和质的角度考察"燕穹"系统的信息覆盖率,得到合理的数量和质量覆盖率值,从而验证了"燕穹"系统信息覆盖率结论的合理性和该信息覆盖率评测模型的可靠性.

An Evaluation Model on Information Coverage of Search Engines

Search engines usually get web pages by using links between them.With already massive and ever increasing of web pages,they can only crawl and index a portion of the whole web pages.A model to evaluate their information coverage percentages is presented.We analyze main factors why crawlers can't cover all web information,and put up three kinds of benchmarks to measure the coverage of a search engine.The paper gives out an evaluation model for two of three benchmarks as follows:First,sampling WWW to get many web pages,which are used to check the coverage percentage of quantity through generating random IPs or breadth first search.Second,selecting high qualified pages as samples of important pages,by HITS or PageRank algorithms.Finally,we submit the samples to page database of search engines,and get the coverage percentage.In our research work,we get experimental data from WebInfoMall system of Peking University and compute the coverage percentages of quantity and quality.Using different sampling approaches and algorithms,we get the same results,which can prove our model is right and all the results are exact.