Virtual multi-cluster clouds

摘要

An improved scalable object storage system includes methods and systems allowing multiple clusters to work together. Users working with a first cluster, or with a multi-cluster gateway, can ask for services and have the request or data transparently proxied to a second cluster. This gives transparent cross-cluster replication, as well as multi-cluster compute or storage farms based upon spot availability or various provisioning policies. Vendors providing a cloud storage “frontend” can provide multiple backends simultaneously. In one embodiment, a multi-cluster gateway can have a two, three, or higher-level ring that transparently matches an incoming request with the correct cluster. In the ring, a request is first mapped to an abstract “partition” based on a consistent hash function, and then one or more constrained mappings map the partition number to an actual resource. In another embodiment, the multi-cluster gateway is a dumb gateway, and the rings are located only at the cluster level.

主权项

1. A system for coordinating a plurality of clusters, the system comprising:
a first cluster including a first plurality of information processing devices; a second cluster including a second plurality of information processing devices; and a multi-cluster gateway coupled to the first and second clusters, wherein the multi-cluster gateway includes a ring that performs a first hashing function and a second hashing function responsive to a request to interact with the system, wherein the request includes an object, wherein the ring performs the first hashing function on an attribute of the object, and an output of the first hashing function is a partition identifier of a partition corresponding to computing resources available at one of the first cluster and the second cluster, and wherein the partition identifier includes a plurality of portions, wherein the ring performs the second hashing function on each portion of the plurality of portions, and an output of the second hashing function identifies a storage pool location for a portion of the plurality of portions on which the second hashing function is applied, and wherein the storage pool location is located in a responsive cluster from one of the first cluster and the second cluster, and wherein the multi-cluster gateway sends a message to the responsive cluster.