Methods for analyzing dynamic web pages

摘要

A computer-implemented method is provided for searching for files on the Internet. In one embodiment, the method may provide an application crawler that assembles and dynamically instantiates all components of a web page. The instantiated web application may then be analyzed to locate desired components on the web page. This may involve finding and analyzing all clickable items in the application, driving the web application by injecting events, and extracting information from the application and writing it to a file or database.

主权项

1. A method comprising:
loading a web page; identifying an extractor template for extracting information from the web page, wherein the extractor template comprises timing instructions for extracting information from the web page; identifying a first object loaded in a web page; simulating user input that manipulates the identified first object; identifying, by at least one processor, a video loaded in response to the simulated user input; extracting first information from the video; in accordance with the timing instructions of the extractor template, skipping a commercial associated with the video; continuing, in accordance with the timing instructions of the extractor template, to extract information from the video by extracting second information from the video after skipping the commercial associated with the video; and aggregating the first information from the video and the second information from the video in an index.