DYNAMIC STREAM COMPUTING TOPOLOGY

摘要

Methods, systems, and apparatus, including computer programs encoded on computer storage media, for dynamic topology generation. One of the methods includes receiving, from a user, a query that specifies a search criterion and a post-processing function prototype. A topology definition is generated from the query that specifies one or more source nodes and at least one processing node, wherein the processing node is specified in the topology to output a stream of data objects corresponding to the data object identifiers and a post-processing node according to the post-processing function prototype, wherein the post-processing node is specified in the topology to receive a stream of data objects output by the processing node, and is specified in the topology to output one or more modified data objects. The topology definition is then provided to a real-time stream computing system for execution.

主权项

1. A computer-implemented method comprising:
receiving, from a user, a query that specifies a search criterion and a post-processing function prototype; generating, from the query, a topology definition that specifies:
one or more source nodes and at least one processing node, wherein the processing node is specified in the topology to receive, from the one or more source nodes, one or more streams of data object identifiers of data objects that satisfy the search criterion, and wherein the processing node is specified in the topology to output a stream of data objects corresponding to the data object identifiers; anda post-processing node according to the post-processing function prototype, wherein the post-processing node is specified in the topology to receive a stream of data objects output by the processing node, perform one or more operations on the data objects according to the post-processing function prototype, and is specified in the topology to output one or more modified data objects; and providing the topology definition to a real-time stream computing system for execution.