Event stream processing partitioning

摘要

In an example embodiment, event stream processing is performed by first parsing an input query into a directed acyclic graph (DAG) including a plurality of operator nodes. Then a grouping of one or more of the operator nodes is created. One or more partitions are created, either by the user or automatically, in the DAG by forming one or more duplicates of the grouping. A splitter node is created in the DAG, the splitter node splits data from one or more event streams and distributes it among the grouping and the duplicates of the grouping. Then, the input query is resolved by processing data from one or more event streams using the DAG.

主权项

1. A method for performing event stream processing comprising:
receiving an input query; parsing, by or more processors, the input query into a single directed acyclic graph (DAG) including a plurality of operator nodes, the operator nodes each defining an operator to perform on input data; creating, by one or more processors, a grouping of one or more of the operator nodes; creating, by one or more processors, one or more partitions in the single DAG by forming one or more duplicates of the grouping and placing the one or more duplicates in the single DAG along with the grouping; creating, by one or more processors, a splitter node in the single DAG, the splitter node defining an operator to split data from one or more event streams among the grouping in the single DAG and the one or more duplicates of the grouping in the single DAG; and resolving, by one or more processors, the input query by processing data from one or more event streams using the single DAG, including executing the operator defined by the splitter node to split data from the one or more event streams to the grouping in the single DAG and the one or more duplicates of the grouping in the single DAG, the resolving further comprising executing operators defined by operator nodes in the grouping in the single DAG and the one or more duplicates of the grouping in the single DAG.