CLUSTER STORAGE USING SUBSEGMENTING FOR EFFICIENT STORAGE

摘要

Cluster storage is disclosed. A data stream or a data block is received. The data stream or the data block is broken into segments. For each segment, a cluster node is selected, and a portion of the segment smaller than the segment is identified that is a duplicate of a portion of a segment already managed by the cluster node.

主权项

1. A system for cluster storage comprising:
a processor configured to:
break a data stream or a data block into segments; andfor a segment:
select a cluster node, wherein a cluster comprises a plurality of cluster nodes, wherein each node comprises a data storage system that stores data by storing a candidate segment or a reference to a stored copy of the candidate segment that is already stored by the cluster;determine that the segment is similar to a previously stored segment on the cluster node using a first set of tags; anddetermine whether a subsegment of the segment is identical to a previously stored subsegment on the cluster node using a second set of tags; anda memory coupled to the processor and configured to provide the processor with instructions.