Dividing and parallel processing record sets using a plurality of sub-tasks executing across different computers

摘要

Disclosed are a data processing system, etc., whereby total processing time of a data process including a parallel process and a serial process is optimized. This data processing system comprises: a task segmenting means for receiving a plurality of tasks which execute, upon a record set which is a set of a plurality of records, a data process which includes parallel processes which a plurality of computers are capable of executing in parallel and serial processes which are post-processes of the parallel processes and is executed by one computer, and, for a parallel process which is the pre-process of a serial process, among the serial processes which are included in the task, which is envisioned as requiring more time to process than another serial process, by subdividing the record set which is to be processed in the parallel process, segmenting the parallel process into a plurality of sub-tasks; and an assignment means for assigning the plurality of sub-tasks to the plurality of computers, such that the plurality of sub-tasks are executed prior to a task other than the task which includes the parallel processes being executed.

主权项

1. A data processing system comprising:
hardware, including a processor and memory; a task dividing unit implemented by the hardware for receiving a plurality of tasks in each of which data processing, including a parallel processing that can be performed by a plurality of computers in parallel and a serial processing that is a post-processing of the parallel processing and performed by one computer, is performed to a record set that is a set of a plurality of records, and for dividing the parallel processing that is a preprocessing of the serial processing to which an expected processing time thereof is greater than that of the serial processing included in another task into a plurality of sub tasks by subdividing the record set that is a processing target of the parallel processing; and an assignment unit implemented by the hardware for assigning the plurality of the sub tasks to the plurality of the computers so that the plurality of the sub tasks are performed before a task other than the task including the parallel processing is performed,
wherein when an expected processing time of a task in the plurality of tasks is longer than an expected processing time of the serial processing with the longest processing time, the parallel processing included in the task is then divided.