Multi-modal neural network for universal, online learning

摘要

In one embodiment, the present invention provides a neural network comprising multiple modalities. Each modality comprises multiple neurons. The neural network further comprises an interconnection lattice for cross-associating signaling between the neurons in different modalities. The interconnection lattice includes a plurality of perception neuron populations along a number of bottom-up signaling pathways, and a plurality of action neuron populations along a number of top-down signaling pathways. Each perception neuron along a bottom-up signaling pathway has a corresponding action neuron along a reciprocal top-down signaling pathway. An input neuron population configured to receive sensory input drives perception neurons along a number of bottom-up signaling pathways. A first set of perception neurons along bottom-up signaling pathways drive a first set of action neurons along top-down signaling pathways. Action neurons along a number of top-down signaling pathways drive an output neuron population configured to generate motor output.

主权项

1. A method comprising:
interconnecting a plurality of neural nodes via an interconnect network of multiple signaling pathways arranged in a lattice, wherein each neural node of the plurality of neural nodes comprises a plurality of neurons, and wherein the plurality of neural nodes comprise:
a first set of neural nodes comprising:
a first neural node for receiving a first sensory input of a first sensory modality;a second neural node for receiving a second sensory input of a second sensory modality that is different from the first sensory modality; anda third neural node for generating a first motor output of a first motor modality; anda second set of neural nodes, wherein at least one neural node of the second set of neural nodes is interconnected with at least two neural nodes of the first set of neural nodes via a first set of signaling pathways and a second set of signaling pathways of the interconnect network, wherein the first set of signaling pathways propagates signals including the first sensory input and the second sensory input in a first direction, wherein the second set of signaling pathways propagates signals including the first motor output in a direction opposite of the first direction, and wherein each signaling pathway of the second set of signaling pathways has a reciprocal signaling pathway in the first set of signaling pathways; and cross-associating the first sensory modality, the second sensory modality, and the first motor modality by exchanging signals between the plurality of neural nodes via the interconnect network, wherein the cross-associating comprises:
determining whether the first motor output is a first type of motor output or a second type of motor output;in response to determining the first motor output is a first type of motor output, propagating the first motor output via at least one signaling pathway of the interconnect network that applies a first learning rule to learn the first motor output; andin response to determining the first motor output is a second type of motor output, propagating the first motor output via at least one signaling pathway of the interconnect network that applies a second learning rule that is different from the first learning rule to unlearn the first motor output; wherein each signaling pathway has a corresponding weight based in part on signals propagating along a reciprocal signaling pathway; wherein at least one neural node generates a signal in response to receiving one or more signals from one or more other neural nodes; and wherein each neural node of the second set of neural nodes exchanges signals with at least two neural nodes of the first set of neural nodes.