| 摘要 |
Disclosed are a method and a system for large volume data processing for solving the problem of system collapse caused by processing delays resulting from a failure of processing a large volume of data within a scheduled time. The method allocates a server to divide a source file into multiple small files, according to a source file naming scheme, and allocates multiple servers to distributedly process the small files. The allocation of servers can be based on the filenames named according to a file naming scheme. The disclosed method deploys multiple servers to divide and process large data files, thereby maximally improving the processing power of the system and ensuring the system to complete the processing of the files within scheduled times. Furthermore, the system promises good scalability.
|
