Matrix Vector Multiply Techniques

摘要

Techniques are disclosed relating to parallel computing. In some embodiments, fine-grained data communication facilitates operations on large data sets such as multiplication of a sparse matrix by a vector. In this example, a first data set (the matrix) and a second data set (the vector) are distributed across multiple processing nodes. Performance of the overall multiplication operation may require communication of data among the processing nodes. In various embodiments, fine-grained communication of this data may reduce processing times and/or power consumption by avoiding congestion.

主权项

1. A method for performing a particular operation using first and second sets of input data, using a system comprising a plurality of processing nodes, wherein each of the plurality of processing nodes includes a respective local memory and a respective local network interface (NI), wherein the plurality of processing nodes are configured to communicatively couple to a network via the NIs, the method comprising:
storing different portions of the first set of input data in different ones of the processing nodes; storing different portions of the second set of input data in different ones of the processing nodes; first performing, by each of the processing nodes, an operation on at least a portion of its stored portion of the first set of input data and portion of the second set of input data; sending, by the processing nodes to other ones of the processing nodes, packets that include: an address in a remote processing node's memory, an operation code, and a value in a payload, wherein the value is generated by an operation of the first performing; second performing, by the processing nodes, operations indicated by the operation code in received packets, wherein the operations are performed on the value and data in the address corresponding to the received packets; and storing results of the second performing.