SYSTEM, METHOD, AND COMPUTER PROGRAM PRODUCT FOR CRAWLING A WEBSITE BASED ON A SCHEME OF THE WEBSITE

摘要

A system, method, and computer program product are provided for crawling a website based on a scheme of the website. In use, a difference between a first content and second content of a website is identified. Additionally, a scheme of the website is identified based on the difference. Furthermore, the website is crawled based on the scheme.

主权项

1. A non-transitory computer readable medium comprising instructions stored thereon that when executed by one or more processors cause the one or more processors to:
determine a difference between a first content and a second content of a single website, the first content and second content identified in a first and second probe of the single website, each probe comprising a request and a corresponding response for information; identify a scheme of the single website based on the difference; and direct crawling of a portion of the single website, the directed crawling based on the identified scheme, and the directed crawling to further identify vulnerability to unwanted activity on the single website.