ACCELERATED DEDUPLICATION

摘要

Mechanisms are provided for accelerated data deduplication. A data stream is received an input interface and maintained in memory. Chunk boundaries are detected and chunk fingerprints are calculated using a deduplication accelerator while a processor maintains a state machine. A deduplication dictionary is accessed using a chunk fingerprint to determine if the associated data chunk has previously been written to persistent memory. If the data chunk has previously been written, reference counts may be updated but the data chunk need not be stored again. Otherwise, datastore suitcases, filemaps, and the deduplication dictionary may be updated to reflect storage of the data chunk. Direct memory access (DMA) addresses are provided to directly transfer a chunk to an output interface as needed.

主权项

1. A system comprising:
a processor configured to maintain a state machine designating a plurality of memory areas for chunk boundary identification and chunk fingerprinting for a data stream; and a deduplication accelerator configured to access the designated memory areas to delineate a respective chunk boundary and determine a respective chunk fingerprint for each of a plurality of data chunks in a single pipeline phase.