IDENTIFYING EQUIVALENT LINKS ON A PAGE

摘要

An illustrative embodiment of a computer-implemented process for identifying equivalent links on a page responsive to a determination that the crawler has not visited all required universal resource locators, locates a next URL to be crawled to form a current URL and processes the current URL to identify equivalent URLs. Responsive to a determination that the crawler has not visited the current URL, determine whether necessary to crawl all identified equivalent URLS and responsive to a determination that it is necessary to crawl all identified equivalent URLS, adding all equivalent URLs to a list of URLs to be crawled.