Data deduplication using multiple devices

摘要

Among other things, a technique of data deduplication using multiple devices includes presenting, to host devices, a quantity of logical data storage greater than physical data storage available on a storage area network (SAN) device, by deduplicating at least some of the data received from the host devices for storage at the SAN device in a manner that is coordinated with over-provisioning capability of the SAN device.

主权项

1. A system for managing data-related requests between a host and a storage area network (SAN) device, the system configured to:
receive data describing a first set of logical units of the SAN device, the first set of logical units together representing a total quantity of logical data storage that is greater than a total quantity of physical data storage available on the SAN device, where at least some pages of the first set of logical units are not allocated to the physical data storage; present one or more second logical units to the host, wherein the second logical units correspond to one or more logical units of the first set of logical units; receive, from the host, a request to write data to a particular second logical unit, where the request identifies a particular portion of data and a location of the particular second logical unit; partition the particular portion of data into one or more blocks; create an index for the particular portion of data, the index comprising, for each of the blocks, an identity of the block and a corresponding location field; identify, from among the blocks, at least one duplicative block that is already stored in the physical data storage but stored at a location other than the location specified by the request, and in response,
record, for each of the duplicative blocks, the other location that already stores the block in the corresponding location field of the index, and transmit, to the SAN device, a new request to write data, where the new request identifies the non-duplicative blocks and a location of the one or more logical units of the first set of logical units corresponding to the particular second logical unit specified by the request, and where the new request comprises an indication to the SAN device to write the non-duplicative blocks by, at least in part, prioritizing writing the non-duplicative blocks to partially filled pages of the first set of logical units over empty pages of the first set of logical units; and record, for each of the non-duplicative blocks, the location of the block in the corresponding location field of the index.