Distributed system for fault-tolerant data storage

摘要

Fault-tolerant storage is provided using a distributed data storage system that receives input data from clients and divides that data into data blocks for storage. The data blocks are processed using a coding scheme that generates redundant level one error correction blocks (L1EC Blocks). The L1EC blocks enable the reconstruction of one or more damaged or inaccessible data blocks, and the L1EC blocks and the data blocks are divided into distribution sets and stored at a plurality of data storage locations. At each data storage location additional level two error correction blocks (L2EC blocks) are generated that provide local data redundancy. Upon detecting a data disruption event, an inaccessible data storage location is identified and the elements that were stored at the inaccessible data storage location are reconstructed.

主权项

1. A method comprising:
storing each of a plurality of distribution sets at a different one of a plurality of data storage locations, each distribution set comprising one or more elements, each element selected from a plurality of data blocks comprising a portion of input data or a plurality of level one error correction blocks (L1EC blocks) generated from the plurality of data blocks, the L1EC blocks comprising recovery information so that a damaged one of the data blocks is recoverable from one or more of the L1EC blocks and one or more of the plurality of data blocks other than the damaged data block; detecting a data disruption event; identifying an inaccessible data storage location in the plurality of data storage locations; and reconstructing the elements that were stored at the inaccessible data storage location using the elements stored at one or more of the plurality of data storage locations other than the inaccessible data storage location.