Efficient Data Reads From Distributed Storage Systems

摘要

A method of distributing data in a distributed storage system includes receiving a file and dividing the received file into chunks. The chunks are data-chunks and non-data chunks. The method further includes grouping chunks into a group and determining a distribution of the chunks of the group among storage devices of the distributed storage system based on a maintenance hierarchy of the distributed storage system. The maintenance hierarchy includes hierarchical maintenance levels and maintenance domains. Each maintenance domain has an active state or an inactive state; and each storage device is associated with at least one maintenance domain. The method also includes distributing the chunks of the group to the storage devices based on the determined distribution. The chunks of the group are distributed across multiple maintenance domains to maintain an ability to reconstruct chunks of the group when a maintenance domain is in the inactive state.

主权项

1. A method of distributing data in a distributed storage system, the method comprising:
receiving, at data processing hardware, a file; dividing, by the data processing hardware, the received file into chunks, the chunks being data-chunks and non-data chunks; grouping, by the data processing hardware, chunks into a group; determining, by the data processing hardware, a distribution of the chunks of the group among storage devices of the distributed storage system based on a maintenance hierarchy of the distributed storage system, the maintenance hierarchy comprising hierarchical maintenance levels and maintenance domains, each maintenance domain having an active state or an inactive state, each storage device associated with at least one maintenance domain; and distributing, by the data processing hardware, the chunks of the group to the storage devices based on the determined distribution, the chunks of the group being distributed across multiple maintenance domains to maintain an ability to reconstruct chunks of the group when a maintenance domain is in the inactive state.