Scalable event-driven system

摘要

Processing events in a distributed environment. There is provided an apparatus which includes a central queue for receiving events and tracking their progress through filtering, correlation, and update processing of stateful target objects, at least one filtering node, at least one correlation node and at least one update node. The central queue receives events in their order of arrival, each event having an entry in the central queue. Processing of events is divided into three pipelined stages in the order of filtering, correlating and updating. A result for each event of each of filtering the events, correlating the events and updating the target object is recorded in the central queue as part of the event's queue entry. The method may be performed by one or more computing devices. The exemplary embodiments detect and compensate for any erroneous results due to unsynchronized parallel processing, and guarantee an eventually correct outcome.

主权项

1. A method for processing events in a distributed environment comprising:
providing an apparatus comprising a central queue for receiving and tracking an event, at least one filtering node, at least one correlation node and at least one update node; receiving, by the central queue, a plurality of events in their order of arrival, each event having an entry in the central queue; dividing processing of the events into three pipelined stages in the order of filtering, correlating and updating, the processing of events comprising:
filtering the events, by the at least one filtering node, to determine zero or more kinds for each event such that filtering the events takes place in a parallel processing environment;correlating the events, by the at least one correlating node, and applying a delivery rule to determine a set of state-holding target objects for each event that will consume the event and update their state wherein the target objects hold all known information about the events, such that correlating the events takes place in the parallel processing environment; and
updating the state-holding target objects for each event, by the at least one update node, such that updating takes place in the parallel processing environment; wherein a result for each event of each of filtering the events, correlating the events and updating the target objects is recorded in the central queue as part of the event's queue entry, wherein the updating for each event includes one bit of information for each target object provided to the central queue to indicate that the update would affect the correlating of events subsequent to the each event and wherein the method is performed by one or more computing devices.