DATA DEDUPLICATION IN A DISPERSED STORAGE SYSTEM

摘要

An efficient data deduplication method for use in a dispersed storage network (DSN). After a data object is received for storage in the DSN, it is determined whether a substantially identical data object has previously been encrypted and stored. The determination may be made, for example, by comparing an encryption key reference value relating to the data object to key reference information stored in DSN memory. If not detected, the data object is encrypted using an encryption key based on the data object. The encrypted data object is then compressed and stored. The encryption key and a key reference value are also stored as encoded key slices in DSN memory. If the data object was previously stored, it is encrypted using a retrieved encryption key that is substantially identical to the data object. The data object may then be compressed for storage using a pattern based data compression function.

主权项

1. A data encoding and compression method for execution by a computing device, wherein the method comprises:
receiving a storage request regarding storing a data object in dispersed storage network (DSN) memory; determining that a substantially identical data object is not stored in the DSN memory; encrypting the data object using an encryption key to produce encrypted data, wherein the encryption key is based, at least in part, on the data object; compressing the encrypted data using a pattern based data compression function to produce compressed data; storing the compressed data; dispersed storage error encoding the encryption key to produce a plurality of sets of encoded key slices; and facilitating storage of the plurality of sets of encoded key slices in response to the storage request.