System and method for distributed deduplication of encrypted chunks

摘要

The present disclosure relates to an advantageous system and related methods for distributed deduplication of encrypted chunks. One embodiment relates to a method for storing encrypted chunks in which an encryption key is generated independently from a chunk payload. With this method, two encrypted chunks are identifiable as having identical chunk payloads even when the chunk payloads are encrypted with different encryption keys. Other embodiments, aspects and features are also disclosed.

主权项

1. A method of putting an encrypted chunk to an object storage system supporting distributed deduplication, the method comprising:
calculating a cryptographic hash to fingerprint an unencrypted payload by a client/proxy (a client, or proxy acting on behalf of the client), after optional compression has been applied at the client/proxy; sending a chunk put request to a chunk server that identifies the chunk to be put by a fingerprint, wherein the chunk server was selected either because it is co-located with the client/proxy or by a consistent hashing algorithm.