Triggering window conditions by streaming features of an operator graph

摘要

In a stream computing application, data may be transmitted between operators using tuples. However, the receiving operator may not evaluate these tuples as they arrive but instead wait to evaluate a group of tuples—i.e., a window. A window is typically triggered when a buffer associated with the receiving operator reaches a maximum window size or when a predetermined time period has expired. Additionally, a window may be triggered by a monitoring a tuple rate—i.e., the rate at which the operator receives the tuples. If the tuple rate exceeds or falls below a threshold, a window may be triggered. Further, the number of exceptions, or the rate at which an operator throws exceptions, may be monitored. If either of these parameters satisfies a threshold, a window may be triggered, thereby instructing an operator to evaluate the tuples contained within the window.

主权项

1. A method for processing data, comprising:
receiving streaming data tuples to be processed by a plurality of operators, the operators processing at least a portion of the received data tuples by operation of one or more computer processors; establishing an operator graph of the plurality of operators, the operator graph defining at least one execution path in which a first operator of the plurality of operators is configured to receive data tuples from at least one upstream operator and transmit data tuples to at least one downstream operator; measuring a current rate at which data tuples pass from the upstream operator to the first operator during a period of time; and triggering a data window in at least one operator of the plurality of operators based on the current rate, the window comprising a set of data tuples to be processed by the at least one operator.