System and method for load balancing in a distributed system by dynamic migration

摘要

A system and method for load balancing between components of a distributed data grid. The system and method support dynamic data migration of selected data partitions in response to detection of hot spots in the data grid which degrade system performance. In embodiments, the system and method relies upon analysis of per-partition performance statistics for both the identification of data nodes which would benefit from data migration and the selection of data nodes for migration. Tuning of the data migration thresholds and method provides for optimizing throughput of the data grid to avoid degradation of performance resulting from load-induced hot spots.

主权项

1. A method for load balancing in a distributed data grid comprising a plurality of nodes each node storing a plurality of partitions of data stored in the distributed data grid each partition comprising a plurality of data objects, the method comprising:
collecting per-partition performance statistics for each partition of said plurality of partitions stored on said plurality of nodes wherein said per-partition performance statistics include per-partition wait time and execution time; calculating a mean per-partition wait time for each node of the plurality of nodes; analyzing each node of the plurality of nodes by comparing per-partition wait times for said each node to said mean per-partition wait times for said each node, and determining that said each node has homogenous per-partition wait times if the per-partition wait times for said each node are clustered around said mean per-partition wait time for said each node; identifying a source node within the plurality of nodes in the distributed data grid, wherein the source node has exceptional wait times compared to other nodes of the plurality of nodes, and homogeneous per-partition wait times; analyzing the plurality of partitions on the source node to identify hot partitions having a disproportionately large share of execution time load compared to other of the plurality of partitions stored by the source node; and alleviating load on the source node by migrating one or more partitions other than identified hot partitions from the source node to a target node.