TECHNIQUES FOR EVALUATING QUERY PREDICATES DURING IN-MEMORY TABLE SCANS

摘要

Techniques are described herein for filtering data from a table during an in-memory scan. Predicates are pushed to in-memory scan to avoid scanning unnecessary columnar units and reduce the overhead of decompressing, row stitching and distributing data during evaluation. Techniques are described herein for generating implied predicates that have conditions on single columns from complex predicates that have multiple conditions on the same column, which can be evaluated during an in-memory scan.;Techniques are also described herein to reduce the overhead of a table scan for processing a join query. When redistributing a first table for performing a hash-join, the nodes performing an in-memory scan of the first table may create a filter that tracks unique values from the join key. Data from the second table is only processed and transferred to other nodes in the cluster if the values from the join key pass through the filter.

主权项

1. A method comprising:
prior to receiving a query with at least one predicate that has work to be performed on a first table, performing the steps of:
dividing data from the first table into a plurality of chunks;populating, into a volatile memory of at least one host node, the plurality of chunks;compressing each given chunk of the plurality of chunks into a given plurality of column-major units; in response to receiving the query with the at least one predicate,
generating from the at least one predicate a condition to evaluate against a particular column-major unit;during an in-memory scan of at least a portion of the first table, comparing data from the particular column-major unit with the condition; andbased on the comparison, filtering data items from the particular column-major unit to produce a first set of intermediate results for the query; wherein the method is performed by one or more nodes.