Tensor deep stacked neural network

摘要

A tensor deep stacked neural (T-DSN) network for obtaining predictions for discriminative modeling problems. The T-DSN network and method use bilinear modeling with a tensor representation to map a hidden layer to the predication layer. The T-DSN network is constructed by stacking blocks of a single hidden layer tensor neural network (SHLTNN) on top of each other. The single hidden layer for each block then is separated or divided into a plurality of two or more sections. In some embodiments, the hidden layer is separated into a first hidden layer section and a second hidden layer section. These multiple sections of the hidden layer are combined using a product operator to obtain an implicit hidden layer having a single section. In some embodiments the product operator is a Khatri-Rao product. A prediction is made using the implicit hidden layer and weights, and the output prediction layer is consequently obtained.

主权项

1. A computing device comprising:
a processor; and memory that comprises a tensor deep stacked neural network (T-DSN), wherein the T-DSN comprises:
a hidden layer that comprises:
a first set of hidden units that comprises a first plurality of hidden units; anda second set of hidden units that comprises a second plurality of hidden units;an output layer that comprises a plurality of output units;a weight tensor that maps the hidden layer to the output layer, the tensor defines respective weights between:
a product of each pair of hidden units formed between the first plurality of hidden units and the second plurality of hidden units, such that each pair of hidden units comprises a hidden unit in the first plurality of hidden units and a hidden unit in the second plurality of hidden units; andeach output unit in the output units, wherein outputs at the output units are based upon the respective weights defined by the weight tensor and input data, and wherein the computing device is further configured to perform a classification task based upon the outputs at the output units.