Storage device selection for database partition replicas

摘要

A system that implements a data storage service may store data in multiple replicated partitions on respective storage nodes. The selection of the storage nodes (or storage devices thereof) on which to store the partition replicas may be performed by administrative components that are responsible for partition management and resource allocation for respective groups of storage nodes (e.g., based on a global view of resource capacity or usage), or the selection of particular storage devices of a storage node may be determined by the storage node itself (e.g., based on a local view of resource capacity or usage). Placement policies applied at the administrative layer or storage layer may be based on the percentage or amount of provisioned, reserved, or available storage or IOPS capacity on each storage device, and particular placements (or subsequent operations to move partition replicas) may result in an overall resource utilization that is well balanced.

主权项

1. A distributed data storage system, comprising:
a plurality of storage nodes, the plurality of storage nodes comprising one or more storage devices or logical storage volumes; and a plurality of compute nodes that implement a plurality of administrative nodes for the distributed data storage system; a first administrative node of the plurality of administrative nodes, the first administrative node configured to:
receive metadata collected by a second administrative node about one or more of the plurality of storage nodes;update global metadata maintained by the first administrative node according to the received metadata;place at least a portion of a first table in the distributed data storage system, wherein to place the portion of the first table, the first administrative node is configured to:
access the global metadata to determine a placement on a first storage node of the plurality of storage nodes for the portion of the first table, the placement based, at least in part, on the global metadata;confirm placement of the portion of the first table with the first storage node; andstore the portion of the first table at the first storage node.