Metadata management for fixed content distributed data storage

摘要

An archival storage cluster of symmetric nodes includes a metadata management system that organizes metadata objects. Each metadata object may have a unique name, and metadata objects are organized into regions. A region is selected by hashing one or more object attributes and extracting a given number of bits of the resulting hash value. The number of bits may be controlled by a configuration parameter. Each region is stored redundantly. A region comprises a set of region copies. In particular, there is one authoritative copy of the region, and zero or more backup copies. The number of backup copies may be controlled by a configuration parameter. Region copies are distributed across the nodes of the cluster to balance the number of authoritative region copies per node, and the number of total region copies per node. Backup region copies are maintained synchronized to their associated authoritative region copy.

主权项

1. A method for a system including a plurality of nodes which store a content data by associating with a metadata object, the method comprising the steps of:
generating a map that identifies a node that stores an authoritative copy of a region for the metadata object, and zero or more nodes that store zero or more backup copies of the region for the metadata object, the number of the backup copies being determined based on a metadata tolerable points of failure (TPOF) configuration parameter; distributing the map across the plurality of nodes so that each node can identify a location of the authoritative copy or backup copy; after a state of the authoritative copy or the backup copy is changed, updating the map to reflect the change of the state; distributing the updated map across the plurality of nodes; and processing an access to the metadata object on each node of the plurality of nodes by referring to the distributed map.