Prioritizing data reconstruction in distributed storage systems

摘要

A method of prioritizing data for recovery in a distributed storage system includes, for each stripe of a file having chunks, determining whether the stripe comprises high-availability chunks or low-availability chunks and determining an effective redundancy value for each stripe. The effective redundancy value is based on the chunks and any system domains associated with the corresponding stripe. The distributed storage system has a system hierarchy including system domains. Chunks of a stripe associated with a system domain in an active state are accessible, whereas chunks of a stripe associated with a system domain in an inactive state are inaccessible. The method also includes reconstructing substantially immediately inaccessible, high-availability chunks having an effective redundancy value less than a threshold effective redundancy value and reconstructing the inaccessible low-availability and other inaccessible high-availability chunks, after a threshold period of time.

主权项

1. A method of prioritizing data for recovery in a distributed storage system, the method comprising:
determining, by a computing processor, an effective redundancy value for each stripe of a file, the file comprising stripes having chunks, the effective redundancy value based on the chunks and any system domains associated with the corresponding stripe, the distributed storage system having a system hierarchy comprising system domains, each system domain having an active state or an inactive state; when a system domain is in the inactive state, updating, by the computing processor, the effective redundancy value for each stripe of the file associated with that system domain; and causing, by the computing processor, reconstruction of a stripe when the effective redundancy value of the stripe is less than a threshold effective redundancy value, wherein the system hierarchy comprises system levels comprising:
a first system level corresponding to host machines of data processing devices, non-transitory memory devices, or network interface controllers, each host machine having a system domain:a second system level corresponding to power deliverers, communication deliverers, or cooling deliverers of racks housing the host machines, each power deliverer, communication deliverer, or cooling deliverer of the rack having a system domain;a third system level corresponding to power deliverers, communication deliverers, or cooling deliverers of cells having associated racks, each power deliverer, communication deliverer, or cooling deliverer of the cell having a system domain; anda fourth system level corresponding to a distribution center module of the cells, each distribution center module having a system domain.