Data layout for recovery and durability

摘要

A Metadata server described herein is configured to generate a metadata table optimized for data durability and recovery. In generating the metadata table, the metadata server associates each possible combination of servers with one of the indices of the table, thereby ensuring that each server participates in recovery in the event of a server failure. In addition, the metadata server may also associate one or more additional servers with each index to provide added data durability. Upon generating the metadata table, the metadata server provides the metadata table to clients or servers. Alternatively, the metadata server may provide rules and parameters to clients to enable those clients to identify servers storing data items. The clients may use these parameters and an index as inputs to the rules to determine the identities of servers storing or designated to store data items corresponding to the index.

主权项

1. A method comprising:
receiving, by a computing device, a total number of servers in a system and one or more rules defining a virtual metadata table of the system; determining, by the computing device, identities of a subset of the servers by utilizing at least an index to the virtual metadata table, the index associated with one or more data items stored on the subset of the servers, and the total number of servers in the system as inputs to the one or more rules defining the virtual metadata table, wherein the one or more rules include:
a first formula to identify a first server of the subset of servers based at least in part on the index and the total number of servers in the system;a second formula to identify a second server of the subset of servers based at least in part on the index and a first output of the first formula; anda third formula to identify a third server of the subset of servers based at least in part on the first output of the first formula and a second output of the second formula; and generating, by the computing device, a metadata table corresponding to the virtual metadata table, wherein data at the computing device is stored in the subset of servers based at least in part on the metadata table.