DYNAMIC PARALLEL DISTRIBUTED JOB CONFIGURATION IN A SHARED-RESOURCE ENVIRONMENT

摘要

Dynamically adjusting the parameters of a parallel, distributed job in response to changes to the status of the job cluster. Includes beginning execution of a job in a cluster, receiving cluster status information, determining a job performance impact of the cluster status, reconfiguring job parameters based on the performance impact, and continuing execution of the job using the updated configuration. Dynamically requesting a change to the resources of the job cluster for a parallel, distributed job in response to changes in job status. Includes beginning execution of a job in a cluster, receiving job status information, determining a job performance impact, requesting a changed allocation of cluster resources based on the determined job performance impact, reconfiguring one or more job parameters based on the changed allocation, and continuing execution of the job using the updated configuration.

主权项

1. A method comprising:
beginning execution of a parallel, distributed job in a job cluster defined by an initial allocation of cluster resources, the job having an initial configuration of a plurality of job parameters; receiving status information about the cluster; determining, based at least in part on the cluster status information, a performance impact to the job; reconfiguring one or more job parameters of the plurality of job parameters, based, at least in part, on the determined job performance impact; and continuing execution of the job using the updated configuration; wherein: the job includes at least a first job task; and among the plurality of job parameters are parameters for the first job task.