Efficiency sets in a distributed system

摘要

Disclosed are systems, computer-readable mediums, and methods for efficiency sets in a distributed system. A first efficiency set is determined for a first volume of data. Determining the first efficiency set includes selecting block identifiers for data blocks of the first volume, where each block identifier is used to access a particular data block corresponding to the first volume. Determining the first efficiency set further includes applying a mask to the selected block identifiers to mask at least one bit of each selected block identifier. The first efficiency set is compared to a second efficiency set for a second data store, and based on the comparison, an amount of unique data blocks of the first volume is approximated.

主权项

1. A storage system comprising:
a server comprising one or more processors configured to:
determine a first efficiency set for a first volume of data, wherein to determine the first efficiency set the one or more processors are configured to:
select block identifiers for data blocks of the first volume, wherein each block identifier is used to access a particular data block corresponding to the first volume; andapply a first mask to the selected block identifiers to mask at least one bit of each selected block identifier;compare the first efficiency set to a second efficiency set for a second data store, wherein the second efficiency set is generated with a second mask;approximate, based on the comparison, an amount of unique data blocks of the first volume;compare the first efficiency set to a plurality of efficiency sets, wherein each of the plurality of efficiency sets corresponds to a particular cluster of the storage system and comprises masked blocked identifiers; anddetermine, based on the comparison of the first efficiency set to the plurality of efficiency sets, a cluster on which to move the first volume based upon the efficiency set that corresponds with the cluster and the first efficiency set having the maximum overlap of masked block identifiers.