Multithreaded apparatus and methods for implementing parallel networks

摘要

Apparatus and methods for developing parallel networks. In some implementations, a network may be partitioned into multiple partitions, wherein individual portions are being executed by respective threads executed in parallel. Individual portions may comprise multiple neurons and synapses. In order to reduce cross-thread traffic and/or reduce number of synchronization locks, network may be partitioned such that for given network portion, the neurons and the input synapses into neurons within the portion are executed within the same thread. Synapse update rules may be configured to allow memory access for postsynaptic neurons and forbid memory access to presynaptic neurons. Individual threads may be afforded pairs of memory buffers configured to effectuate asynchronous data input/output to/from thread. During an even iteration of network operation, even buffer may be utilized to store data generated by the thread during even iteration. Odd buffer may be utilized to read data, generated by other threads during the preceding odd iteration.

主权项

1. A method of operating a network comprising a plurality of nodes and a plurality of node-to-node connections by a computerized apparatus, the method comprising:
configuring individual ones of the plurality of connections, each adapted to communicate data from a source node to a target node of the plurality of nodes, and to allow access to memory associated with the target node; partitioning the plurality of nodes into at least a first node portion and a second node portion, individual ones of the plurality of nodes being characterized by a node memory; selecting a first portion of the plurality of connections configured to provide an input to individual nodes of the first node portion; and selecting a second portion of the plurality of connections configured to provide the input to individual nodes of the second node portion; wherein the first node portion and the first portion of the plurality of connections are assigned to a first parallel executor; and wherein the second node portion and the second portion of the plurality of connections are assigned to a second parallel executor; wherein:
an operation of the first portion of the plurality of connections and the first node portion by the first parallel executor is characterized by first and second operation stages configured to be executed immediately sequentially with one another;an operation of the second portion of the plurality of connections and the second node portion by the second parallel executor is characterized by third and fourth operation stages configured to be executed immediately sequentially with one another;the first, second, third, and fourth operation stages have varied execution times; andan operation of a combination of the first and the second operation stages is configured to occur in parallel with an operation of a combination of the third and fourth operation stages.