Efficient set operation execution using a single group-by operation

摘要

Techniques are described herein for efficient set operation execution. According to some embodiments, a request is received to perform a set operation with respect to a first data set and a second data set. The request may identify the first data set, the second data set, and a type of set operation to perform. In response to receiving the request, a hash table is generated in memory from a first set of records in the first data set, and a second set of records from the second data set is probed against the hash table. Based on probing the hash table and the type of set operation identified in the request, records that satisfy the set operation are identified and output from the hash table.

主权项

1. A method for performing a set operation using one or more computing devices, the method comprising:
receiving a request to perform a set operation, wherein the request identifies one or more grouping keys and a set operator indicating how to combine a first data set with a second data set; storing first entries corresponding to the first data set in a hash table based on the one or more grouping keys, each first entry comprising at least a portion of a record in the first data set; probing records from the second data set against the hash table based on the one or more grouping keys to identify matching records from the second data set, each matching record matching a first entry of the first entries corresponding to the first data set; identifying and outputting records that satisfy the set operation based on the first entries in the hash table and the matching records identified.