SCALABLE MONITORING OF LONG RUNNING MULTI-STEP DATA INTENSIVE WORKLOADS

摘要

Examples perform monitoring of multiple-step, concurrently executed workflows across distributed nodes. Requests from an intermediate node are classified by a load balancer as monitoring or non-monitoring. Non-monitoring requests are handled by any node; however, monitoring requests are distributed to all nodes via a plurality of queues but handled only by nodes executing the subject workflow. The load balancer receives reports from any node executing the subject workflow, and passes the first report to the intermediate node.

主权项

1. A computing system for monitoring multi-step workflows, said system comprising:
an intermediate node; a plurality of queues; a plurality of management nodes; a load balancer that is stored in the memory and, when executed by a processor, monitors multi-step workflows by:
receiving at least one request from the intermediate node for an operation to be performed by one or more of the management nodes;if the received request is a monitoring request, transmitting the received request to a plurality of monitoring queues each associated with at least one of the plurality of management nodes; andif the received request is a non-monitoring request, transmitting the received request to a single non-monitoring queue associated with each of the plurality of management nodes; the plurality of management nodes which, when executed by a processor, each process queued requests by:
accessing at least one of the queued requests;if the accessed request is a non-monitoring request, performing the accessed request; andif the accessed request is a monitoring request, performing the accessed request only if the accessed request is from the monitoring queue associated with the management node.