Weakly synchronized garbage collection and compaction for aggregated, replicated object stores

摘要

Managing consistency of object replicas is performed at a first instance of a distributed storage system. The first instance performs garbage collection on a shard that includes a first plurality of object chunks, thereby removing a second plurality of object chunks from the shard. This leaves a third plurality of object chunks in the shard, where the first plurality of object chunks is the union of the second and third pluralities of object chunks. The first instance sends a first list of identifiers to a second instance of the distributed storage system. The second instance has a replica of the shard. The first list of identifiers specifies the object chunks in the third plurality of object chunks. The second instance removes all object chunks from the replica of the shard that are not included in the first list.

主权项

1. A method for managing consistency of object replicas in a distributed storage system, comprising:
at a first instance of the distributed storage system, having one or more processors and memory, wherein the memory stores one or more programs for execution by the one or more processors:
performing garbage collection and compaction on a first shard comprising a first plurality of object chunks, thereby removing a second plurality of object chunks from the first shard and leaving a third plurality of object chunks in the first shard, wherein the first plurality of object chunks is the union of the second and third pluralities of object chunks;sending a first list of identifiers to a second instance of the distributed storage system, wherein the second instance has a replica of the first shard and wherein the first list of identifiers specifies the object chunks in the third plurality of object chunks; and at the second instance, having one or more processors and memory, wherein the memory stores one or more programs for execution by the one or more processors:
compacting the replica of the first shard to remove all object chunks that are not included in the first list.