发明名称 |
System and method for crawl policy management utilizing IP address and IP address range |
摘要 |
The present invention relates to a method for configuring a policy management protocol for a web crawler, the method further comprising the steps of determining a web space that is to be crawled by a web crawler, wherein the web space is comprised of an IP address and/or a range of IP addresses, and determining additional hostnames that are associated with the IP address and/ range of IP addresses. The method further comprises the steps of configuring the web crawler to crawl the IP address and/ range of IP addresses, and determine additional hostnames that are associated with the IP address or range of IP addresses, and performing a web crawling function upon the determined additional hostnames by the web crawler.
|
申请公布号 |
US7701944(B2) |
申请公布日期 |
2010.04.20 |
申请号 |
US20070625110 |
申请日期 |
2007.01.19 |
申请人 |
INTERNATIONAL BUSINESS MACHINES CORPORATION |
发明人 |
BHAGWAN VARUN;DESAI RAJESH M.;JALAN PIYOOSH |
分类号 |
H04L12/28;G06F7/00;G06F17/30;H04L12/56 |
主分类号 |
H04L12/28 |
代理机构 |
|
代理人 |
|
主权项 |
|
地址 |
|