Data Mining Method and Apparatus

摘要

A data mining method and apparatus where the method includes determining multiple execution steps of a data mining process, acquiring a correspondence between a physical resource required by each execution step in a running process and a physical resource occupied by input data of the data mining process, determining a node for executing each execution step, determining, according to a maximum amount of data of input data that can be processed by the node for executing each step, a maximum amount of data of input data that can be processed by the distributed system, and processing to-be-mined data in accordance with the data mining process according to the maximum amount of data of the input data that can be processed by the distributed system. The input data is accurately and effectively limited such that normal running of the system can be ensured.

主权项

1. A data mining method, wherein the method is applied to a distributed system, wherein the distributed system comprises at least one node, and wherein the method comprises:
determining multiple execution steps of a data mining process; acquiring a correspondence between a physical resource required by each execution step in a running process and a physical resource occupied by input data of the data mining process; determining a node for executing each execution step, wherein the node provides a physical resource for each execution step; determining, according to the correspondence and a physical resource possessed by a node for executing a corresponding execution step, a maximum amount of data of input data that is capable of being processed by the node for executing each execution step; determining, according to the maximum amount of data of the input data that is capable of being processed by the node for executing each execution step, a maximum amount of data of input data that is capable of being processed by the distributed system; and processing to-be-mined data in accordance with the data mining process according to the maximum amount of data of the input data that is capable of being processed by the distributed system.