Method and apparatus for retrieving and processing data

摘要

Data is captured from a web site or other data source. Data is extracted from the web page using a data harvesting script or other data acquisition routine. The extracted data is then normalized and stored in a database. If data cannot be extracted from the web page, a copy of the captured web page is stored without personal information contained in the web page. The data harvesting script is then edited based on an analysis of the captured web page.