APPARATUS AND METHOD FOR IDENTIFYING PERFORMANCE BOTTLENECKS IN PIPELINE PARALLEL PROCESSING ENVIRONMENT

摘要

A system identifies a performance bottleneck status in a parallel data processing environment by examining data flow associated with the parallel data processing environment to identify at least one operator, where an operator type is associated with at least one operator, at least one buffer, and a relationship that the buffer has with the operator, where the relationship is associated with the operator type. The system monitors the buffer to determine a buffer status associated with the buffer. The system applies a set of rules to identify an operator bottleneck status associated with the operator. The set of rules is applied to the operator, based on the operator type, the buffer status, and relationship that the buffer has with the operator. The system then determines a performance bottleneck status associated with the parallel data processing environment, based on the operator bottleneck status.

主权项

1. A method for identifying performance bottleneck status in a parallel data processing environment, implemented by a computing processor, the method comprising:
examining, by the computing processor, data flow associated with the parallel data processing environment to identify: i) at least one operator, wherein an operator type is associated with the at least one operator; ii) at least one buffer; and iii) a relationship that the at least one buffer has with the at least one operator, wherein the relationship is associated with the operator type; monitoring, by the computing processor, the at least one buffer to determine a buffer status associated with the at least one buffer; applying, by the computing processor, a set of rules to identify an operator bottleneck status associated with the at least one operator, wherein the set of rules is applied to: i) the at least one operator, based on the operator type; ii) the at least one buffer status; and iii) the relationship that the at least one buffer has with the at least one operator; and determining, by the computing processor, a performance bottleneck status associated with the parallel data processing environment, based on the operator bottleneck status.