WIDE SPREADING DATA STORAGE ARCHITECTURE

摘要

Technology is disclosed for a data storage architecture for providing enhanced storage resiliency for a data object. The data storage architecture can be implemented in a single-tier configuration and/or a multi-tier configuration. In the single-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data fragments, which are stored across many storage devices. In the multi-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data segments, which are sent to one or more tiers of storage nodes. Each of the storage nodes further encodes the data segment to generate many data fragments representing the data segment, which are stored across many storage devices associated with the storage node. The I/O operations for rebuilding the data in case of device failures is spread across many storage devices, which minimizes the wear of a given storage device.

主权项

1. A computer-implemented method comprising:
receiving, at a storage management computer node of a storage management system, a write request including a data object; encoding, by the storage management computer node, the data object to generate a first specified number of multiple encoded data fragments, the encoded data fragments representing the data object, the first specified number of encoded data fragments including a second specified number of the encoded data fragments using which the data object can be regenerated; determining, by the storage management computer node, a storage layout of the encoded data fragments for storing the encoded data fragments at multiple storage devices, the storage devices grouped into multiple storage shelves, wherein a number of the storage devices is equal to or greater than the first specified number of the encoded data fragments; and transmitting, by the storage management computer node and based on the storage layout, to at least a subset of the storage shelves to store the encoded data fragments at the storage devices.