Increasing performance of a streaming application by running experimental permutations

摘要

A streams manager monitors performance of a streaming application, and when the performance needs to be improved, the streams manager automatically requests virtual machines from a cloud manager. In response, the cloud manager provisions one or more virtual machines in a cloud. The streams manager builds a permutation of a flow graph for the streaming application using the virtual machine(s), runs the permutation in parallel with the corresponding portion of the flow graph, and compares the performance of the original portion of the flow graph to the performance of the permutation. When the performance of the permutation exceeds the performance of the original part of the flow graph, the streams manager modifies the flow graph so the permutation replaces the original part of the flow graph. In this manner a streaming application can dynamically evolve to increase its performance as needed.

主权项

1. A computer-implemented method executed by at least one processor for improving performance of a streaming application, the method comprising:
executing a streaming application that comprises a flow graph that includes a plurality of operators that process a plurality of data tuples; monitoring performance of the streaming application; when performance of the streaming application needs to be improved, requesting a cloud manager to provision at least one virtual machine; generating using the at least one virtual machine a permutation of a portion of the flow graph that provides logical equivalence of tuples of the portion of the flow graph; running the permutation in parallel with the portion of the flow graph; determining whether the performance of the permutation exceeds the performance of the portion of the flow graph; and when the performance of the permutation exceeds the performance of the portion of the flow graph, the streams manager modifies the flow graph to replace the portion of the flow graph with the permutation.