Workflow control of reservations and regular jobs using a flexible job scheduler

摘要

A scheduler receives flexible reservation requests for scheduling in a computing environment comprising consumable resources. The flexible reservation request specifies a duration and a required resource. The consumable resources comprise machine resources and floating resources. The scheduler creates a flexible job for the flexible reservation request and places the flexible job in a prioritized job queue for scheduling, wherein the flexible job is prioritizes relative to at least one regular job in the prioritized job queue. The scheduler adds a reservation set to a waiting state for the flexible reservation request. The scheduler, responsive to detecting the flexible job positioned in the prioritized job queue for scheduling next and detecting a selection of consumable resources available to match the at least one required resource for the duration, transfers the selection of consumable resources to the reservation and sets the reservation to an active state.

主权项

1. A method for managing workflow, comprising:
receiving, by a scheduler comprising one or more processors and one or more memories, at least one flexible reservation request for scheduling in a computing environment comprising a plurality of consumable resources, wherein the at least one flexible reservation request specifies a duration and at least one required resource and wherein the plurality of consumable resources comprises at least one machine resource and at least one floating resource; creating, by the scheduler, a flexible job for the at least one flexible reservation request and placing the flexible job in a prioritized job queue for scheduling; adding, by the scheduler, a reservation set to a waiting state for the at least one flexible reservation request; receiving, by the scheduler, at least one regular job not bound to a reservation, wherein the at least one regular job is a job ready to be dispatched for running when the job reaches the highest priority position in the prioritized job queue when the resources required for executing the job becomes available; monitoring, by the scheduler, for each of the at least one regular job and the flexible job, whether each separate selection of resources required for each of the at least one regular job and the flexible job are available; responsive to detecting the flexible job positioned in the prioritized job queue for scheduling next and detecting a selection of the plurality of consumable resources available to match the at least one required resource for the duration, removing, by the scheduler, the flexible job from the prioritized job queue without sending the flexible job to the resource manager to run, transferring the selection of the plurality of consumable resources to the reservation, and setting the reservation to an active state, wherein the reservation is activated as the selection of the plurality of consumable resources become available and has uninterrupted use of the selection of the plurality of consumable resources for the duration for at least one bound job to the reservation; and responsive to receiving, by the scheduler, the at least one bound job for the at least one flexible reservation request and the reservation set to the active state, dispatching the at least one bound job directly to the reservation for running on the selection of the plurality of consumable resources without placement in the prioritized job queue.