Method and apparatus to facilitate discrete-device accelertaion of queries on structured data

摘要

A method for parallel processing of data, including accessing a sub-graph of an execution plan for a query used for accessing a database by a host processor, wherein the execution plan includes operators organized into sub-graphs. The method includes generating a transformed sub-graph by transforming operators from the sub-graph for parallel execution on discrete computing devices. The method includes adding at least one first operator to the transformed sub-graph configured for allocating memory on the discrete computing devices and copying ingress variables to the allocated memory. The method includes adding at least one second operator to the transformed sub-graph for copying egress variables, wherein the transformed sub-graph comprises a plurality of intermediate variables generated during execution of the sub-graph that are fully contained within the transformed sub-graph. The method includes off-loading the transformed sub-graph to the discrete devices for execution.

主权项

1. A computer system for parallel processing of data, comprising:
memory having stored therein computer-executable instructions; and a processor executing said computer-executable instructions including:
receiving a query used for accessing a database;determining an execution plan for said query, wherein said execution plan comprises a plurality of operators organized into one or more sub-graphs, and for determining a first sub-graph of said execution plan;determining a first size of memory of discrete computing devices available for execution of operators in parallel related to said query; andgenerating a transformed first sub-graph by transforming operators from said first sub-graph that when executed on said discrete computing devices in parallel requires less memory than said first size of memory, wherein said transformed first sub-graph comprises a plurality of intermediate variables generated during execution of said transformed first sub-graph that are fully contained within said transformed first sub-graph, upper and lower boundaries of said transformed first sub-graph completely encompass said plurality of intermediate variables, said upper boundary comprises a first set of operators accessing a plurality of ingress variables and said lower boundary comprises a second set of operators outputting a plurality of egress variables, said plurality of intermediate variables are not accessed by operators outside of said transformed first sub-graph and said transformed first sub-graph is off-loaded to discrete devices for execution.