Rule-based dynamic resource adjustment for upstream and downstream processing units in response to an intermediate processing unit event

摘要

A method dynamically adjusts the resources available to a processing unit of a distributed computer process executing on a multi-node computer system. The resources for the processing unit are adjusted based on the data other processing units handle or the execution path of code in an upstream or downstream processing unit in the distributed process or application.

主权项

1. A multi-node computer system comprising:
a plurality of compute nodes having compute resources; a plurality of processing units that comprise a distributed process executing on the plurality of compute nodes; a rules registry with a rule that comprises an action with an associated event, where the action indicates an adjustment of compute resources available to a first processing unit and a second processing unit upon detection of the associated event on a third processing unit, wherein the first processing unit is located on a compute node downstream from the third processing unit, and the second processing unit is located on a compute node upstream from the third processing unit, wherein compute resources indicated in the rules registry comprise central processing unit, memory, and network resources, and wherein the rules registry lists actions and events that include the following: increase processing unit resources upon detection of a quantity of input data; decrease processing unit resources upon detection of a quantity of input data; increase processing unit resources upon detection of a specific type of data; decrease processing unit resources upon detection of a specific type of data; increase processing unit resources upon detection of a code point in a processing unit; and decrease processing unit resources upon detection of a code point in a processing unit; an event detection mechanism that detects the occurrence of the associated event in the rules registry; and a resource controller that, in response to detection of the associated event on the third processing unit, dynamically adjusts the compute resources available to the first processing unit and the second processing unit according to the action listed in the rules registry corresponding to the detected associated event by increasing compute resources for the first processing unit and decreasing compute resources for the second processing unit.