SCHEDULING MAPREDUCE JOBS IN A CLUSTER OF DYNAMICALLY AVAILABLE SERVERS

摘要

There is provided a method, a system and a computer program product for improving performance and fairness in sharing a cluster of dynamically available computing resources among multiple jobs. The system collects at least one parameter associated with availability of a plurality of computing resources. The system calculates, based on the collected parameter, an effective processing time each computing resource can provide to each job. The system allocates, based on the calculated effective processing time, the computing resources to the multiple jobs, whereby the multiple jobs are completed at a same time or an approximate time.

主权项

1. A method for scheduling jobs, the method comprising:
calculating a correlation across a plurality of computing resources, the correlation indicating a fraction of simultaneously available times of every pair of the plurality of computing resources; assigning, according the calculated correlation, the jobs to the plurality of computing resources, wherein a processor coupled to a memory device runs the step of calculating and the step of assigning.