Processing of streaming data with keyed aggregation

摘要

Keyed aggregation is used in the processing of streaming data to streamline processing to provide higher throughput and decreased use of resources. The most recent event for each unique replacement key value(s) is maintained. In response to an incoming event having a same key as a previous event, the effect on an aggregation of the previous event is removed. The aggregation is then updated with one or more values from the arriving event and the updated aggregation is output.

主权项

1. A computer program product for processing streaming data in a data processing system, the computer program product comprising:
a non-transitory computer readable storage medium for execution by a processing circuit for performing a method comprising:
receiving a stream of data;aggregating values of events of the stream of data to form an aggregation, the aggregating comprising:
determining whether a new event of the stream of data includes a key which is the same key as a key included in a previous event of the stream of data, the previous event being currently maintained as a stored event, and the previous event including one or more values having an effect on the aggregation, wherein a match between the key included in the new event and the key included in the previous event indicates provision of updated information, in the new event, about the previous event, the updated information for replacing the one or more values having the effect on the aggregation;based on determining the new event includes the same key as the key included in the previous event, replacing the previous event with the new event;undoing the effect of the one or more values included in the previous event on the aggregation; andapplying to the aggregation an effect of one or more values included in the new event; andoutputting the aggregation.