Method for lock-free clustered erasure coding and recovery of data across a plurality of data stores in a network

摘要

The present invention provides a distributed clustering method to allow multiple active instances of consistency management processes that apply the same encoding scheme to be cooperative and function collectively. The techniques described herein facilitate an efficient method to apply an erasure encoding and decoding scheme across dispersed data stores that receive constant updates. The technique can be applied on many forms of distributed persistent data stores to provide failure resiliency and to maintain data consistency and correctness.

主权项

1. An erasure encoding method comprising:
providing a plurality of computer systems forming a distributed computer network, each computer system having a management process operating thereon; providing a plurality of distributed persistent memories, each persistent memory interoperably coupled to at least one of the computer systems and operative to receive constant updates; distributing via an erasure encoding process a plurality of data sets across the distributed persistent memories, each data set including at least one data block and at least one check sum block; initializing the management processes to operate as a cluster; wherein during the initializing of the management processes to operate as a cluster, each management process:
agrees to enter into an initialization mode;enters into initialization mode;suspends regular run time processing; andregisters its data-set management-process management responsibility; identifying the distributed persistent memories associated with each data set of the plurality of data sets; classifying whether the at least one data block and at least one checksum block of each data set are good; verifying if there are enough good blocks to recover each data set of the plurality of data sets; performing a consistency check of the good blocks; and recovering each data set of the plurality of data sets using the good blocks.