ASYNCHRONOUS SCHEDULING INFORMED BY JOB CHARACTERISTICS AND ANTICIPATORY PROVISIONING OF DATA FOR REAL-TIME, PARALLEL PROCESSING

摘要

Systems and methods are disclosed for scheduling jobs processed in a distributed fashion to realize unharnessed efficiencies latent in the characteristics of the jobs and distributed processing technologies. A job store may be communicatively coupled to a job analyzer. The job analyzer may be operable to read information characterizing a job to identify multiple data blocks to be processed during the job at multiple locations in a cluster of nodes. A scheduling module may use information about the multiple data blocks, their storage locations, their status with respect to being provisioned to processing logic, data blocks to be processed by other jobs, data blocks in cache that have been pre-fetched for a prior job, quality-of-services parameters, and/or job characteristics, such as job size, to schedule the job in relation to other jobs.

主权项

1. A system for scheduling a job may comprise:
a job store operable to receive jobs; a job analyzer communicatively coupled to the job store and operable to identify multiple data blocks to be processed by a job in the job store from characterization information for the job, the multiple data blocks stored at multiple storage locations within a network; and a scheduling module operable to schedule the processing of the job to promote availability of the multiple data blocks at multiple instances of processing logic within the network once processing of the job commences.