Method and system for providing a net effect platform for developing and correcting screen scraping parser scripts

摘要

Individuals using a data management system are provided the capability to make corrections to default values presented in a simulation of at least part of a webpage including default data indicating the layout of the webpage, and/or one or more identified data fields determined by heuristic analysis and/or from data collected from other individuals/users. These contributing individual corrections are then transformed into contributing individual originated parser scripts associated with the webpage. The contributing individual originated parser scripts are then implemented and/or applied to the webpage for all individuals, including the contributing individual.

主权项

1. A computing system implemented method for providing a net effect platform for developing and correcting screen scraping scripts comprising the following, which when executed individually or collectively by any set of one or more processors perform a process including:
obtaining, by a process computing system from a user computing system under the control of an individual, login data used to obtain data associated with the individual from a third party webpage provided by a webpage computing system, the obtained login data including at least an account number associated with the user; accessing, using some or all of the login data by the process computing system, the third party webpage; determining, by the process computing system following an attempt to access the third party webpage, that an error has occurred in retrieving data from the third party webpage; requesting, of the user through communication by the process computing system with the first computing system, that the individual help identify and correct the cause of the error; receiving acknowledgement from the individual that the individual will help identify and correct the cause of the error; generating, by the process computing system, default data associated with a third party webpage, the default data at least indicating the layout of the webpage; generating, by the process computing system using the default data, a mock-up of the third party webpage, the mock-up of the third party webpage including a reproduction of at least part of the third party webpage; providing, to the user computer system by the process computing system communicating with the user computer system, the mock-up of the third party webpage as an interactive interface, wherein the interactive interface configured to allows the individual to add, modify, correct, and rearrange the default data; obtaining, from the interactive interface, additions, modifications, corrections, and/or rearrangements of the default data associated with the third party webpage from the interactive interface, the additions, modifications, corrections, and/or rearrangements of the default data associated with the third party webpage at least including an additions, modifications, corrections, and/or rearrangements of the layout of the webpage; transforming the additions, modifications, corrections, and/or rearrangements of the default data associated with the third party webpage into correction data; transforming the correction data into parser scripts associated with the third party webpage; and using the login data and parser scripts associated with the third party webpage to obtain data associated with the individual from the third party webpage.