STREAMING DATA PATTERN RECOGNITION AND PROCESSING

摘要

When processing data tuples, operators of a streaming application may identify certain tuples as being relevant. To determine relevant tuples, the operators may, for example, process the received tuples and determine if they meet certain thresholds. If so, the tuples are deemed relevant, but if not they are characterized as irrelevant. The streaming application may use a pattern detector to parse the relevant data tuples to identify a pattern, such as a shared trait between the tuples. Based on this commonality, the pattern detector may generate filtering criteria that may be used to process subsequently received tuples. In one embodiment, the filtering criteria identified by one operator is transmitted to a second operator to be used to process tuples received there. Thus, once one of the operators determines a pattern, the operator generates filtering criteria that another, related operator uses for filtering received tuples.

主权项

1. A system, comprising:
a computer processor; and a memory containing a program that, when executed on the computer processor, performs an operation for processing data, comprising:
receiving streaming data tuples to be processed by a plurality of interconnected operators, the operators processing at least a portion of the received data tuples by operation of the computer processor;processing tuples received by a first operator of the plurality of operators to identify relevant tuples that satisfy one or more criteria;evaluating the relevant tuples to determine a pattern based on a commonality shared by at least a portion of the relevant tuples;transmitting the shared commonality to a second operator of the plurality of operators; anddetermining whether data tuples received by the second operator contain the shared commonality.