PREDICTING AND USING UTILITY OF SCRIPT EXECUTION IN FUNCTIONAL WEB CRAWLING AND OTHER CRAWLING

摘要

A program is executed that includes multiple script functions. For a selected script function, the following are performed during program execution. It is determined whether the selected script function should or should not be executed based on a utility corresponding to the selected script function. The utility was determined prior to determining whether the selected script function should be executed. The selected script function is executed in response to a determination the selected script function should be executed. Execution of the selected script function is skipped in response to a determination the selected script function should not be executed. These techniques may be applied in real-time to crawl a program such as a webpage or may be applied using offline learning followed by a real-time crawling of the program. Apparatus, methods, and program products are disclosed.

主权项

1. A method, comprising:
executing a program comprising a plurality of script functions; for a selected script function of the plurality of script functions, performing the following during program execution:
determining whether the selected script function should or should not be executed based on a utility corresponding to the selected script function, wherein the utility was determined prior to determining whether the selected script function should be executed;executing the selected script function in response to a determination the selected script function should be executed; andskipping execution of the selected script function in response to a determination the selected script function should not be executed.