APPARATUS FOR SORTING AND COMBINING INTERMEDIATE RESULTS OF MULTIPLE MAP TASKS ON THE DISTRIBUTED PARALLEL PROCESSING SYSTEM AND METHOD THEREOF

摘要

A multiple map task intermediate result queue, a combined device and a method thereof in a distributed parallel processing system removing a duplicate key existing in a whole destination file are provided to reduce task processing load, the number of task output file and total size of an output file transmitted in the distributed parallel processing system. A map task performing unit generates an intermediate result by performing the allocated map task. A reduce task performance removes task transmission schedule information on a transmission schedule intermediate result list(S512). The reduce task performance sets up intermediate result path information about a residual task identifier(S522).