System and method for parallelizing convolutional neural networks

摘要

A parallel convolutional neural network is provided. The CNN is implemented by a plurality of convolutional neural networks each on a respective processing node. Each CNN has a plurality of layers. A subset of the layers are interconnected between processing nodes such that activations are fed forward across nodes. The remaining subset is not so interconnected.

主权项

1. A convolutional neural network system implemented by one or more computers, wherein the convolutional neural network system is configured to receive an input image and to generate a classification for the input image, and wherein the convolutional neural network system comprises:
a sequence of neural network layers, wherein the sequence of neural network layers comprises:
a first convolutional layer configured to receive a first convolutional layer input derived from the input image and to process the first convolutional layer input to generate a first convolved output;a first max-pooling layer immediately after the first convolutional layer in the sequence configured to pool the first convolved output to generate a first pooled output;a second convolutional layer immediately after the max-pooling layer in the sequence configured to receive the first pooled output and to process the first pooled output to generate a second convolved output, anda plurality of fully-connected layers after the second convolutional layer in the sequence configured to receive an output derived from the second convolved output and to collectively process the output derived from the second convolved output to generate a sequence output for the input image.