Logical Data Shuffling

摘要

Embodiments relate to data shuffling by logically rotating processing nodes. The nodes are logically arranged in a two or three dimensional matrix. Every time two of the nodes in adjacent rows of the matrix are positionally aligned, these adjacent nodes exchange data. The positional alignment is a logical alignment of the nodes. The nodes are logically arranged and rotated, and data is exchanged in response to the logical rotation.

主权项

1. A method comprising:
arranging a plurality of computing nodes in a cluster, the arrangement including a coordinated communication topology, and a static distribution of data within the topology; pre-determining a step sequence, a step encompassing an alignment of nodes into pairs within the topology, wherein the node pair includes at least one partner node; exchanging data among the aligned node pairs, wherein the data exchange supports bi-directional exchange between node pairs; and re-distributing data among nodes in the topology in response to completion of the step-sequence.