SYSTEM AND METHOD FOR EFFICIENT TASK SCHEDULING IN HETEROGENEOUS, DISTRIBUTED COMPUTE INFRASTRUCTURES VIA PERVASIVE DIAGNOSIS

摘要

A system and method schedules jobs in a cluster of compute nodes. A job with an unknown resource requirement profile is received. The job includes a plurality of tasks. Execution of some of the plurality of tasks is scheduled on compute nodes of the cluster with differing capability profiles. Timing information regarding execution time of the scheduled tasks is received. A resource requirement profile for the job is inferred based on the received timing information and the differing capability profiles. Execution of remaining tasks of the job is scheduled on the compute nodes of the cluster using the resource requirement profile.

主权项

1. A system for scheduling jobs in a cluster of compute nodes, said system comprising:
at least one processor programmed to:
receive a job with an unknown resource requirement profile, the job including a plurality of tasks;schedule execution of some of the plurality of tasks on compute nodes of the cluster with differing capability profiles;receive timing information regarding execution time of the scheduled tasks;infer a resource requirement profile for the job based on the received timing information and the differing capability profiles; andschedule execution of remaining tasks of the job on the compute nodes of the cluster using the resource requirement profile.