Nature Neuroscience has an intriguing fMRI brain scanning study where the researchers could work out what sort of silent video clip the volunteers were watching by observing activity in the part of their brains specialised for perceiving sound.
Although silent, the video clips were all chosen to ‘imply’ sound by depicting things such as a howling dog, a piano key being struck or coins being dropped into a drinking glass. This reliably caused activity in the auditory cortex as the brain ‘simulated’ likely sounds.
The researchers used an analysis technique called ‘multivariate pattern analysis’ that can pick out brain activity patterns associated with different types of experience.
In this study, the analysis was set up to work as a ‘classifier’ where the research team entered both the brain scanning data and what video clips the participants were viewing for part of the experiment, and the ‘classifier’ then tried to guess which types of picture were being viewed for the rest of the experiment just by using the brain scan patterns it had learnt earlier.
After being trained on a sample of the data, the ‘classifier’ could identify which type of silent clip (animal, musical instrument or object) the person was watching by analysing the pattern of activity in the auditory part of the brain.
This is a lovely demonstration of how the brain ‘simulates’ the type of neural activity that would normally be triggered by other senses to help flesh out what it is experiencing.
Perhaps the earliest demonstration of this was from several studies that reported activity in parts of the brain specialised for visual perception that was triggered when you try and picture objects in your imagination.
These visualisation studies are an example of where you are consciously trying to ‘simulate’ another sense through active imagination but, as this new study shows, this sensory ‘filling’ in by the brain also seems to happen automatically.
Studies on ‘implicit motion’ provide another demonstration of this. For example, being shown a still picture that implies movement (such as a ball being dropped) will cause activity in V5, an area specialised for motion perception.