Scalable partitioning in a multilayered data service framework

摘要

A method and system is provided for maintaining customer data in a data store system utilizing a scalable partitioning framework. More specifically, the data store of a customer service system is partitioned (divided) into multiple partitions by a partitionable key of customer data so that each partition owns a subset of the customer data. By maintaining several partitions that store a subset of the customer data exclusive to each other, blackout or brownout problems can be local to one partition and, thus, the availability for the entire system will be increased. Moreover, a set of partitionable keys, a minimal unit to be moved between partitions, is grouped and associated with a partition. By eliminating direct dependencies between the partitions and the partitionable keys, the system can have great flexibility with regard to migrating customer data between partitions and adding a new partition.

主权项

1. A system for managing customer information in scalable partitioned data stores, the system comprising:
a plurality of data stores, wherein individual data stores of the plurality of data stores include data corresponding to at least one of a plurality of geographic locations, wherein individual geographic locations of the plurality of geographic locations are associated with a single data store from the plurality of data stores, and wherein the individual data stores of the plurality of data stores contain: a plurality of partitions storing customer data corresponding to customers, a plurality of buckets containing partitionable keys, wherein individual partitionable keys correspond to a customer, and at least one mapping table which represents mapping information between the plurality of partitions and the plurality of buckets, wherein the mapping information maps individual partitions to one or more of the plurality of buckets, wherein the at least one mapping table is distinct from mapping tables of other data stores of the plurality of data stores; at least one computing device in communication with the plurality of data stores, the at least one computing device configured to:
identify a bucket currently associated with a source geographic location and that is available to be associated with a destination geographic location,
wherein association of the bucket with the destination geographic location includes association of the bucket with a destination partition of a destination data store that is associated with the destination geographic location,wherein the destination partition is a partition of the destination data store to which the bucket is migrating, andwherein the at least one mapping table associated with the destination data store excludes mapping information mapping the identified bucket to the plurality of partitions of the destination data store;identify a source partition of a source data store that is currently associated with the identified bucket and with the source geographic location,
wherein the source partition is a partition of the source data store from which the bucket is migrating, andwherein the at least one mapping table associated with the source data store contains mapping information mapping the identified bucket to the source partition;update the at least one mapping table associated with the source data store to indicate that the identified bucket is in a migration process between the source geographic location and the destination geographic location;move customer data associated with customer identifiers from the source partition of the source data store corresponding to the source geographic location to the destination partition of the destination data store corresponding to the destination geographic location, wherein the customer identifiers are provided from the identified bucket; andafter moving the customer data:
update the at least one mapping table associated with the source data store to remove mapping information corresponding to the identified bucket; andupdate the at least one mapping table associated with the destination data store to add mapping information corresponding to the identified bucket and mapping the identified bucket to the destination partition.