Memory management in a streaming application

摘要

One embodiment is directed to a method for processing a stream of tuples. The method may include receiving a stream of tuples to be processed by a plurality of processing elements operating on one or more computer processors. Each of the processing elements has an associated memory space. In addition, the method may include monitoring the plurality of processing elements. The monitoring may include identifying a first performance metric for a first processing element. The method may include modifying the first processing element based on the first performance metric. The modifying of the first processing element may include employing memory management of the associated memory space.

主权项

1. A system for processing a stream of tuples, comprising:
a plurality of processing elements to receive a stream of tuples, each processing element including one or more stream operators, wherein one or more of the stream operators include code configured to output tuples to one or more other stream operators, the processing elements operating on one or more computer processors, the processing elements forming an operator graph in which tuples flow between stream operators, the operator graph defining one or more execution paths for processing and routing the stream of tuples, each processing element having an associated memory space; and a performance monitor configured to:
monitor the plurality of processing elements, wherein the monitoring includes identifying a first performance metric for a first processing element and identifying the first performance metric for a second processing element; andmodify one or more of the processing elements, wherein the modifying includes modifying the first processing element based on a comparison of a first value of the first performance metric at the first processing element with a second value of the first performance metric at the second processing element, wherein the modification of the first processing element includes employing memory management of a memory space associated with the first processing element.