Extent hashing technique for distributed storage architecture

摘要

In one embodiment, an extent hashing technique is used to efficiently distribute data and associated metadata substantially evenly among nodes of a cluster. The data may be write data associated with a write request issued by a host and received at a node of the cluster. The write data may be organized into one or more extents. A hash function may be applied to the extent to generate a result which may be truncated or trimmed to generate a hash value. A hash space of the hash value may be divided into a plurality of buckets representative of the write data, i.e., the extents, and the associated metadata, i.e., extent metadata. A number of buckets may be assigned to each extent store instance of the nodes to distribute ownership of the buckets, along with their extents and extent metadata, across all of the extent store instances of the nodes.

主权项

1. A method comprising:
organizing write data of write requests into one or more extents, wherein each extent is a block of data that provides a unit of storage on one or more storage devices of a cluster, the write requests processed at a node of the cluster, the cluster having a plurality of nodes; applying a hash function to each extent to generate a hash value; dividing a hash space of the hash value into a plurality of buckets representative of the one or more extents and associated extent metadata, a number of the buckets being less than a number of values in the hash space; and assigning the buckets to extent store instances of the nodes in the cluster based on results of a remainder computation, wherein the remainder computation divides a remainder of the hash value of each extent by the number of buckets using modulo arithmetic, and the results of the remainder calculation are bucket numbers that operate as indexes in a bucket mapping data structure having bucket number entries, wherein each bucket number entry maps to an extent store instance.