PER-NODE CUSTOM CODE ENGINE FOR DISTRIBUTED QUERY PROCESSING

摘要

Distributed query processing is often performed by a set of nodes that apply MapReduce to a data set and materialize partial results to storage, which are then aggregated to produce the query result. However, this architecture requires a preconfigured set of database nodes; can only fulfill queries that utilize MapReduce processing; and may be slowed down by materializing partial results to storage. Instead, distributed query processing can be achieved by choosing a node for various portions of the query, and generating customized code for the node that only performs the query portion that is allocated to the node. The node executes the code to perform the query portion, and rather than materializing partial results to storage, streams intermediate query results to a next selected node in the distributed query. Nodes selection may be involve matching the details of the query portion with the characteristics and capabilities of the available nodes.

主权项

1. A server that executes a query using a node set of nodes, the server comprising:
a processor; and a memory storing instructions, wherein execution of the instructions by the processor causes the server to:
partition the query into at least two query portions;for respective query portions:
choose, from the node set, a selected node to perform the query portion;generate a query instruction set for the selected node, wherein execution of the query instruction set causes the selected node to:
perform the query portion of the query, andfor query portions that generate an intermediate query result, transmit the intermediate query result to a next selected node of the node set; anddeploy the query instruction set to the selected node; andcause the selected nodes of the node set to execute the query instruction sets that together represent the query.