发明名称 INCREMENTAL PARALLEL PROCESSING OF DATA
摘要 One example method includes identifying synchronous code including instructions specifying a computing operation to be performed on a set of data; transforming the synchronous code into a pipeline application including one or more pipeline objects; identifying a first input data set on which to execute the pipeline application; executing the pipeline application on a first input data set to produce a first output data set; after executing the pipeline application on the first input data set, identifying a second input data set on which to execute the pipeline application; determining a set of differences between the first input data set and second input data set; and executing the pipeline application on the set of differences to produce a second output data set.
申请公布号 US2016117188(A1) 申请公布日期 2016.04.28
申请号 US201614989867 申请日期 2016.01.07
申请人 Google Inc. 发明人 Bradshaw Robert;Chambers Craig D.;Cooper Ezra;Raniwala Ashish;Perry Frances J.
分类号 G06F9/46;G06F9/38 主分类号 G06F9/46
代理机构 代理人
主权项 1. A computer-implemented method executed by one or more processors, the method comprising: identifying synchronous code including instructions specifying a computing operation to be performed on a set of data; transforming the synchronous code into a pipeline application including one or more pipeline objects, the pipeline application configured to be executed in parallel across a plurality of computing devices, each of the one or more pipeline objects configured to receive an input data set and produce an output data set; identifying a first input data set on which to execute the pipeline application; executing the pipeline application on a first input data set to produce a first output data set, the executing the pipeline application including executing each of the one or more pipeline objects in an order in which a previous pipeline object provides its output data set to a next pipeline object as its input data set; after executing the pipeline application on the first input data set, identifying a second input data set on which to execute the pipeline application; determining a set of differences between the first input data set and second input data set; and executing the pipeline application on the set of differences to produce a second output data set, the executing the pipeline application on the set of differences including executing each of the one or more pipeline objects includes each previous pipeline object in the order providing differences from its previous output data set to the next pipeline object as its input data set, and the second output data set including differences from the first output data set.
地址 Mountain View CA US