On Finding Templates on Web Collections |
| |
Authors: | Karane Vieira André Luiz da Costa Carvalho Klessius Berlt Edleno S de Moura Altigran S da Silva Juliana Freire |
| |
Affiliation: | (1) Department of Computer Science, Federal University of Amazonas, Manaus, Brazil;(2) School of Computing, University of Utah, Salt Lake City, USA |
| |
Abstract: | Templates are pieces of HTML code common to a set of web pages usually adopted by content providers to enhance the uniformity
of layout and navigation of theirs Web sites. They are usually generated using authoring/publishing tools or by programs that
build HTML pages to publish content from a database. In spite of their usefulness, the content of templates can negatively
affect the quality of results produced by systems that automatically process information available in web sites, such as search
engines, clustering and automatic categorization programs. Further, the information available in templates is redundant and
thus processing and storing such information just once for a set of pages may save computational resources. In this paper,
we present and evaluate methods for detecting templates considering a scenario where multiple templates can be found in a
collection of Web pages. Most of previous work have studied template detection algorithms in a scenario where the collection
has just a single template. The scenario with multiple templates is more realistic and, as it is discussed here, it raises
important questions that may require extensions and adjustments in previously proposed template detection algorithms. We show
how to apply and evaluate two template detection algorithms in this scenario, creating solutions for detecting multiple templates.
The methods studied partitions the input collection into clusters that contain common HTML paths and share a high number of
HTML nodes and then apply a single-template detection procedure over each cluster. We also propose a new algorithm for single
template detection based on a restricted form of bottom-up tree-mapping that requires only small set of pages to correctly
identify a template and which has a worst-case linear complexity. Our experimental results over a representative set of Web
pages show that our approach is efficient and scalable while obtaining accurate results. |
| |
Keywords: | web template detection tree-mapping web IR |
本文献已被 SpringerLink 等数据库收录! |
|