Apparatus and method for decoding sensory and cognitive information from brain activity

摘要

Decoding and reconstructing a subjective perceptual or cognitive experience is described. A first set of brain activity data produced in response to a first brain activity stimulus is acquired from a subject using a brain imaging device. An encoding model is used to convert the brain activity data into a corresponding set of predicted response values. A second set of brain activity data produced in response to a second brain activity stimulus is acquired from a subject and decoded using a decoding distribution derived from the encoding model, and the probability the second set of brain activity data corresponds to said predicted response values is determined. The second set of brain activity stimuli is then reconstructed based on the probability of correspondence between the second set of brain activity data and the predicted response values.

主权项

1. A method for decoding and reconstructing a subjective perceptual or cognitive experience, the method comprising:
acquiring a first set of brain activity data from a first subject, wherein said first set of brain activity data is acquired using a first brain imaging device, and wherein said first set of brain activity data is produced in response to a first set of brain activity stimuli; converting said first set of brain activity data into a corresponding set of encoding model parameter values for each of one or more linearizing feature spaces; acquiring a second set of brain activity data from said first subject or a second subject, wherein said second set of brain activity data is acquired using a second brain imaging device, and wherein said second set of brain activity data is produced in response to a second set of brain activity stimuli; creating a decoding database comprising one or more items; generating a corresponding set of predicted brain activity signals for each of the one or more items in the decoding database using the set of encoding model parameter values corresponding to each of the one or more linearizing features spaces; determining a probability for each of the one or more items that said second set of brain activity data corresponds to said set of predicted brain activity signals corresponding to the item; selecting at least one item from the decoding database based on the probability determined for each of the one or more items in the decoding database; and producing a reconstructed set of brain activity stimuli based on the selected at least one item.