Methods and Systems for Key Sharding of Objects Stored in Distributed Storage System

摘要

The present disclosure also provides systems and methods for sharding objects stored in a distributed storage system. In accordance with one embodiment disclosed herein, a key sharding technique is used. Key sharding is an advantageously efficient technique when dealing with an object containing a collection of key-value records. In accordance with an embodiment of the invention, referenced chunks identified by the key shards may each store a subset of the collection of the key-value records, and the key-value records in the subset have key hashes that have a range of matching bits in common. One embodiment disclosed herein provides a method of performing a delta edit of a named object stored in a distributed storage system in which a payload of the named object is stored in key shards. Other embodiments, aspects and features are also disclosed.

主权项

1. A method of performing a delta edit of a named object stored in a distributed storage system, the method comprising:
storing, in the distributed storage system, a payload of the named object in key shards that are defined by key-shard chunk references, wherein the payload for the named object comprises a collection of key-value records, and wherein referenced chunks identified by the key shards each stores a subset of the collection of the key-value records, where the key-value records in the subset have key hashes that have a range of matching bits in common; receiving, by a gateway server, a request for a set of delta edits to be applied to the named object, wherein each delta edit specifies an addition or a deletion of a key-value record; determining, by the gateway server, relevant key shards to which the delta edits apply; and updating the relevant key shards, while not updating other key shards for the named object.