Weighted-value consistent hashing for balancing device wear

摘要

Described are techniques that select a physical storage device for storing data. Device weights for physical storage devices are determined in accordance with factor(s) including a storage capacity each physical storage device and an amount of remaining write capacity of each physical storage device representing an amount of remaining writes for an expected lifetime of each physical storage device. A replica set is determined for each physical storage device in accordance with a device weight. Hash value sets are determined for the physical storage devices using a hash function. A first hash value for a first data item is determined using the hash function. The first hash value is mapped to a second hash value included in one of hash value sets associated with a corresponding one of the physical storage devices. The corresponding one of the physical storage devices is selected to store the first data item.

主权项

1. A method of selecting a physical storage device for storing data comprising:
determining a plurality of device weights for a plurality of physical storage devices, wherein a different one of the plurality of device weights is determined for each of the plurality of physical storage devices in accordance with one or more factors including any of a storage capacity of said each physical storage device and an amount of remaining write capacity of said each physical storage device representing an amount of remaining writes for an expected lifetime of said each physical storage device; determining a replica set for each of said plurality of physical storage devices, said replica set for said each physical storage device including a number of replicas in accordance with the different one of the plurality of device weights for said each physical storage device; determining a plurality of hash value sets for said plurality of physical storage devices, wherein an associated one of the plurality of hash value sets is determined for each of said plurality of physical storage devices, said associated one of the plurality of hash values sets for said each physical storage device including hash values determined using a hash function and said replica set of said each physical storage device, wherein each of said hash values is generated as an output of the hash function when using a corresponding one of the replicas of the replica set for said each physical storage device as an input to the hash function; determining a first hash value for a first data item using the hash function; mapping the first hash value to a second hash value included in a first of the plurality of hash value sets associated with a corresponding one of the plurality of physical storage devices; and selecting the corresponding one of the plurality of physical storage devices to store the first data item.