Hash join using collaborative parallel filtering in intelligent storage with offloaded bloom filters

摘要

Processing resources at a storage system for a database server are utilized to perform aspects of a join operation that would conventionally be performed by the database server. When requesting a range of data units from a storage system, the database server includes join metadata describing aspects of the join operation for which the data is being requested. The join metadata may be, for instance, a bloom filter. The storage system reads the requested data from disk as normal. However, prior to sending the requested data back to the storage system, the storage system analyzes the raw data based on the join metadata, removing a certain amount of data that is guaranteed to be irrelevant to the join operation. The storage system then returns filtered data to the database server. The database system thereby avoids the unnecessary transfer of certain data between the storage system and the database server.

主权项

1. A system comprising:
one or more storage devices storing raw data in data block structures; a storage server configured to respond to input/output (I/O) requests directly from one or more database servers, but not to respond to database commands received directly from database clients; wherein the I/O requests are requests for raw data from specified data block structures; wherein the storage server is configured to respond to the I/O requests from the one or more database servers by reading data blocks from or writing data blocks to the one or more storage devices; and a database server, connected via a network to the storage server; wherein the storage server acts as an intermediary for all data transfers between the database server and the one or more storage devices; wherein the database server is configured to respond to database commands received directly from one or more database clients, but the database server is not capable of directly responding to I/O requests for raw data; wherein the database server is configured to perform:
identifying one or more data blocks, of the data block structures stored on the one or more storage devices, that store raw data that represents a first table;generating join metadata based upon one or more attributes of a second table;sending to the data storage system:
a particular I/O request for the one or more data blocks, anddata indicating the join metadata; wherein the storage server is further configured to perform:
receiving the particular I/O request for the one or more data blocks;reading the one or more data blocks from the one or more storage devices;identifying a portion of the raw data in the one or more data blocks that represents table rows that are guaranteed, based on the join metadata, to not be needed for a join operation with the second table; andreturning, to the database server, a filtered version of the one or more data blocks without at least the identified portion; wherein the database server is further configured to perform:
receiving the filtered version of the one or more data blocks from the storage server; anda join operation between the first table and the second table based on the filtered version of the one or more data blocks.