Systems and methods for de-duplication in storage systems

摘要

In accordance with embodiments of the present disclosure, a storage system may include a storage array comprising one or more storage resources, a processor communicatively coupled to the storage array, and a de-duplication module comprising instructions embodied on a computer-readable medium communicatively coupled to the processor. The de-duplication module may be configured to, when read and executed by the processor: generate a fingerprint for an item of data stored on the storage array; identify a partition for the fingerprint; associate the partition with a hardware instance selected from a plurality of hardware instances, wherein each particular hardware instance comprises one or more information handling resources; and query the selected hardware instance to determine if the fingerprint exists on the hardware instance.

主权项

1. A storage system, comprising:
a storage array comprising one or more physical storage resources; a processor communicatively coupled to the storage array; a de-duplication module comprising instructions embodied on a non-transitory computer-readable medium communicatively coupled to the processor, the de-duplication module configured to, when read and executed by the processor: generate a fingerprint for an item of data stored on the storage array; identify a partition for the fingerprint; associate the partition with a selected hardware instance selected from a plurality of hardware instances, wherein each particular hardware instance comprises one or more information handling resources; and query the selected hardware instance to determine whether the fingerprint exists on the hardware instance; wherein the de-duplication module configured to, in order to identify each partition, calculate a function to identify a partition for each fingerprint: wherein the function is operable to map all possible fingerprints to a corresponding partition; and the function operable to map each fingerprint to a partition such that the number of fingerprints associated with each partition would be approximately equal.