Managing processing branches in an operator graph

摘要

Embodiments of the disclosure provide a method and system for processing data such as a stream of tuples. One embodiment is directed toward a method for managing a stream of tuples. The method can include receiving the stream of tuples to be processed by a plurality of stream operators operating on one or more computer processors. The method can include routing the stream of tuples to a plurality of output ports, the output ports each communicate with a processing branch. The method can include receiving performance data of the stream of tuples from a measuring stream operator that measures the processing branch. The method can include determining weights to assign to each output port based on the performance data. The method can also include modifying the routing of the stream of tuples through the plurality of output ports as a function of the weight of each output port.

主权项

1. A system for processing a stream of tuples, each tuple having one or more attributes, comprising:
one or more processing elements operating on a compute node having one or more computer processors, the one or more processing elements configured to processes the attributes from the stream of tuples as a single instance, each processing element having one or more stream operators; a split operator from the one or more processing elements configured to route the stream of tuples, using a routing method, through a plurality of processing branches, each processing branch having one or more stream operators,
wherein a first processing branch of the plurality of processing branches is configured to perform a particular type of processing and produce a different processing result from a second processing branch of the plurality of processing branches, a stream operator configured to perform processing independent from another stream operator; a measuring stream operator that is configured to:
receive the stream of tuples from one or more processing branches from the plurality of processing branches;measure performance data of each tuple from the stream of tuples;determine a weight to assign to each processing branch based off of the performance data;determine the routing method for the split operator based off of the weight; andtransmit the routing method to the split operator.