Asymmetric distributed data storage system

摘要

Asymmetric distributed replicated data storage systems and methods are described herein. The storage system includes zones that are independent, and autonomous. The zones include nodes that are independent and autonomous. The nodes include storage devices. When a data item is stored, it is partitioned into a plurality of data objects and a plurality of parity objects using erasure coding. The data objects and parity objects are spread across all nodes and zones in the storage system asymmetrically such that a first zone includes all of the data objects and no parity objects while the remaining zones include subsets of the data objects and all of the parity objects. The systems and methods provide for data resiliency while keeping the amount of storage space required relatively low.

主权项

1. A data storage system comprising:
a plurality of storage zones, each storage zone comprising a plurality of nodes wherein each node comprises a plurality of storage devices and a controller, the controller including a processor and memory a first node of a plurality of nodes included in a first zone of the plurality of zones, the first node having instructions which when executed cause a first processor included in a first controller in the first node to perform actions including:
receiving a storage request from a client of the system to store a data item, the storage request including a data item reference and storage criteriapartitioning the data item into a plurality of data objects according to distribution instructions included in the storage criteria, wherein each data object includes a storage policy identifier and a data portion and the storage policy identifier defines replication of the data object among the storage zones of the data storage systemcreating a plurality of parity objects according to the distribution instructions included in the storage criteriastoring all of the data objects in the first storage zonestoring a first subset of the data objects and all of the parity objects in a second storage zonestoring a second subset of the data objects and all of the parity objects in a third storage zonewherein the second subset of the data objects and the first subset of the data objects have partly different constituent data objects according to the storage criteriamaintaining a latency map that includes a time for each node to reach other nodes in the storage zones.