Identifying data communications algorithms of all other tasks in a single collective operation in a distributed processing system

摘要

Topology mapping in a distributed processing system, the distributed processing system including a plurality of compute nodes, each compute node having a plurality of tasks, each task assigned a unique rank, including: assigning each task to a geometry defining the resources available to the task; selecting, from a list of possible data communications algorithms, one or more algorithms configured for the assigned geometry; and identifying, by each task to all other tasks, the selected data communications algorithms of each task in a single collective operation.

主权项

1. A method of topology mapping in a distributed processing system, the distributed processing system including a plurality of compute nodes, each compute node executing a plurality of tasks, each task assigned a unique rank, the method comprising:
assigning each task to a geometry, wherein the geometry comprises a collection of compute nodes having distinct data communications abilities and resources available to tasks assigned to the geometry, wherein data communications abilities include one or more data communications algorithms supported by each compute nodes; selecting, by each task from a list of possible data communications algorithms, one or more of the possible data communications algorithms supported by the compute node upon which the task is executing in the assigned geometry; identifying, by each task, the selected data communications algorithms of all other tasks in a single collective operation, wherein the identification includes the steps of:
setting, by each task a string of bits where each bit represents a data communications algorithm available for the task being executed on the compute node, a bit to true for each algorithm selected by the task; andperforming, by all the tasks, an allreduce operation with a bitwise AND of all string of bits associated with each task; and performing, by each task, communication with all other tasks using one or more data communication algorithms common to all tasks.