Resilient distributed replicated data storage system

摘要

A resilient distributed replicated data storage system is described herein. The storage system includes zones that are independent, and autonomous from each other. The zones include nodes that are independent and autonomous. The nodes include storage devices. When a data item is stored, it is partitioned into a plurality of data objects and a plurality of parity objects are calculated. Reassembly instructions are created for the data item. The data objects, parity objects and reassembly instructions are spread across nodes and zones in the storage system according to a policy for the data item. When a zone is inaccessible, a virtual zone is created and used until the intended zone is available. When a read request is received, the data item is prepared from the lowest latency nodes according to the reassembly instructions, and a virtual zone is accessed in place of a real zone when the real zone is inaccessible.

主权项

1. A system comprising:
a plurality of storage zones, each storage zone comprising a plurality of nodes wherein each node comprises a plurality of storage devices and a controller, the controller including a processor and memory a first node of a plurality of nodes included in a first zone of the plurality of zones, the first node having instructions stored thereon which when executed cause a first processor included in a first controller in the first node to perform actions including:
receiving a storage request from a client of the system to store a data itempartitioning the data item into a plurality of data objects, each object including a storage policy identifier and a data portioncreating a plurality of parity objects for the data itemcreating reassembly instructions for the data itemstoring groups of some of the data objects and groups of some of the parity objects and the reassembly instructions among the nodes included in the storage zones according to a predetermined set of criteria, the storing including
evaluating a latency for each storage zone to which the data item is to be storedperforming the storing in lowest latency orderevaluating each storage zone to determine whether the storage zone is accessiblewhen one of the storage zones is not accessible, creating a virtual zone in the first zone to store the data objects, parity objects and reassembly instructions intended for the inaccessible storage zonechecking whether the storage zone corresponding to the virtual zone is accessiblewhen the storage zone corresponding to the virtual zone remains inaccessible, waiting a system defined amount of timerepeating the checking and the waiting until the storage zone corresponding to the virtual zone becomes accessiblewhen the storage zone corresponding to the virtual zone is accessible moving the data objects, parity objects and reassembly instructions from the virtual zone to the intended storage zone.