HIERARCHICAL DATA STORAGE ARCHITECTURE

摘要

Technology is disclosed for a data storage architecture for providing enhanced storage resiliency for a data object. The data storage architecture can be implemented in a single-tier configuration and/or a multi-tier configuration. In the single-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data fragments, which are stored across many storage devices. In the multi-tier configuration, a data object is encoded, e.g., based on an erasure coding method, to generate many data segments, which are sent to one or more tiers of storage nodes. Each of the storage nodes further encodes the data segment to generate many data fragments representing the data segment, which are stored across many storage devices associated with the storage node. The I/O operations for rebuilding the data in case of device failures is spread across many storage devices, which minimizes the wear of a given storage device.

主权项

1. A computer-implemented method comprising:
receiving, at a storage management computer node of a storage management system, a write request including a data object; encoding, by the storage management computer node, the data object to generate multiple encoded data segments representing the data object; determining, by the storage management computer node, a storage layout for storing the encoded data segments in a storage subsystem associated with the storage management system; transmitting, by the storage management computer node and based on the storage layout, distinct encoded data segments to different storage computer nodes; and at each of the storage computer nodes that received an encoded data segment of the encoded data segments,
encoding the encoded data segment to generate multiple encoded data fragments representing the encoded data segment, andstoring the encoded data fragments of the encoded data segment across multiple storage devices associated with the corresponding storage computer node, the storage devices of the storage computer nodes belonging to the storage subsystem.