FAILURE RESILIENT DISTRIBUTED REPLICATED DATA STORAGE SYSTEM

摘要

A failure resilient distributed replicated data storage system is described herein. The storage system includes zones that are independent, and autonomous from each other. The zones include nodes that are independent and autonomous. The nodes include storage devices. When a data item is stored, it is partitioned into a plurality of data objects and a plurality of parity objects calculated. Reassembly instructions are created for the data item. The data objects and parity objects are spread across all nodes and zones in the storage system. Reassembly instructions are also spread across the zones. When a read request is received, the data item is prepared from the lowest latency nodes according to the reassembly instructions. This provides for data resiliency while keeping the amount of storage space required relatively low.

主权项

1. A system comprising:
a plurality of storage zones, each storage zone comprising a plurality of nodes wherein each node comprises a plurality of storage devices and a controller, the controller including a processor and memory a first node of a plurality of nodes included in a first zone of the plurality of zones, the first node having instructions which when executed cause a first processor included in a first controller in the first node to perform actions including:
receiving a storage request from a client of the system to store a data itempartitioning the data item into a plurality of data objects, each object including a storage policy identifier and a data portioncreating a plurality of parity objects for the data itemcreating reassembly instructions for the data itemstoring groups of some of the data objects and groups of some of the parity objects and the reassembly instructions among the nodes included in the storage zones according to a predetermined set of criteria.wherein each of the data objects are included in at least one of the groups of data objects.wherein each of the parity objects are included in at least one of the groups of parity objects.wherein the predefined set of criteria ensures that the data item can be reconstructed by a accessing a subset of the plurality of data objects and the plurality of the parity objects.