Distributed parallel computation with acceleration devices

摘要

A method for distributed computing between a host computer and at least one accelerator device interconnected through a network includes profiling a data transfer rate and a computation rate for a range of data sizes to find an optimal chunk size for the data transfer through the network; splitting or aggregating a size of the data stored in a memory in the host computer for encapsulating the data into a chunk with the optimal chunk size; dispatching the encapsulated data to the accelerator device; and instructing pipeline computation to the accelerator device with respect to the encapsulated data received.

主权项

1. A method for distributed computing between a host computer and at least one accelerator device interconnected through a network, the method comprising:
profiling a data transfer rate and a computation rate for a range of data sizes to find an optimal chunk size for the data transfer through the network; splitting or aggregating a size of a data stored in a memory of the host computer for encapsulating the data into a chunk with the optimal chunk size; dispatching the encapsulated data to the accelerator device; and instructing pipeline computation to the accelerator device with respect to the encapsulated data received, wherein the optimal chunk size is determined to be the data size where an overlapping ratio between the data transfer rate and the computation rate is closest to 1 and if there are multiple such data sizes for which the overlapping ratio is closest to 1, determining the optimal chunk size as the data size that has the highest data transfer rate and that which is between a minimum data size and a maximum data size transmitted during profiling.