System and method for computing message digests

摘要

A data de-duplication approach leverages acceleration hardware in SSDs for performing digest computations used in de-duplication operations and support on behalf of an attached host, thereby relieving the host from the computing burden of the digest computation in de-duplication (de-dupe) processing. De-dupe processing typically involve computation and comparison of message digests (MD) and/or hash functions. Such MD functions are often also employed for cryptographic operations such as encryption and authentication. Often, SSDs include onboard hardware accelerators for MD functions associated with security features of the SSDs. However, the hardware accelerators may also be invoked for computing a message digest result and returning the result to the host, effectively offloading the burden of MD computation from the host, similar to an external hardware accelerator, but without redirecting the data since the digest computation is performed on a data stream passing through the SSD for storage.

主权项

1. A method of data duplication detection, comprising:
receiving a write request for a block of data at a storage device coupled to a host for providing non-volatile data storage; receiving, from the host, a command to compute a message digest of the block of data in conjunction with fulfilling the write request; invoking an onboard hardware accelerator configured for message digest computation based on the command from the host and applied to data in the block of data and traversing a data path from the host to a storage medium in a storage device, wherein the onboard hardware accelerator is an encryption engine disposed on the storage device and configured for encryption, decryption, and secure hash computation; duplicating the block of data by snooping the data from a write stream of data to be written to the storage device; computing, in the onboard hardware accelerator, a message digest of the data, by identifying the block of snooped data and applying a digest function to the block, from the traversed data path as the data passes on the data path from the host to the storage medium in the storage device for storage further comprising noninvasive snooping of the data to be stored, snooping occurring as the data is received at the storage device from the host, wherein the message digest computation is a stateless, flow-through computation applied to data as it passes from the post to the storage device for storage on a media component of the storage device; storing the data on the storage medium in the storage device; and returning the computed message digest from the onboard hardware accelerator to the host.