A reader writes:
I’ve recently discovered that I can play a video game while listening to spoken word audio (podcasts).
The game, AntiGrav, uses the body (via a cam which is interpreted as movements). It’s physically demanding and demands quick visual recognition and response– ie. flailing arms about and generally looking like an idiot. Terrific game.
The podcasts on the other hand are fairly intellectually engaging. However, I find that I cannot just sit and listen to them… I need to be doing something else. I can’t do programming work or read blogs/web pages, because I get overwhelmed by the two language-based inputs.
So I’m able to turn off the game music / effects and listen, while playing and do as well as I would listening to the game soundtrack.
This seems a suprising result, and I gather that they use different parts of the brain. Care to comment?
Good question – it is a little suprising that you can do both at once. I think the answer is not so much that they involve different input modalities (one visual, one auditory), but that the two tasks involve different types of processing which do not require a change of the ‘representational code’ between input and output.
With the game you get visual input and rapidly convert it into movements. With the podcast you’re hearing linguistic inputs, and – although you aren’t responding in any overt way – you need to understand it and remember what happened over (i presume) a longer space over time than with the game.
As with most real life situations there is a whole bunch of things going on here, but i’ll try and break it down a bit.
Cognitive psychologists use simple versions of what you are doing here to find out what kind of mental abilities group together – called a ‘dual-task methodology’. There’s an example described in “Talk To Yourself”, Hack #61 – the experimenters wanted to find out if a certain ability relied on language so they got the participants to continuously repeat a news report what they heard over a loud-speaker. Not a complex task, but one that stopped them using their language faculty. Conversely an equally complex task (clapping their hands in time to a rhythm they heard over a loud-speaker) didn’t affect their language faculty, even though it was an equally difficult in itself.
Using this two-things-at-once-and-see-what-happens approach, we know that (not too surprisingly) visual and spatial tasks interfere with each other, as do auditory and linguistic tasks. But visual and spatial tasks don’t generally interfere with auditory and linguistic ones (and vice-versa).
So one part of the story is that tasks of different sensory forms can often be dealt with similatanously. The next part is that sometimes tasks in different sensory modalities still interfere with each other if their content requires ‘strategic’ skills – things like complex planning and memory for events longer than a few seconds. Even if the inputs and outputs of both tasks may be being dealt with by modality-specific subroutines (ie one visuo-spatial, one auditory-verbal), they can still interfere with each other if they both call on our modality-general resources (what some psychologists call ‘the central executive’, which – as far as i can tell – essentially means ‘the bit that does all the complex stuff we don’t understand’: planning, organising timing, attention and memory. And stuff).
I’m guessing that, in your case, only the podcasts put a heavy load on your planning and reasoning abilities, although (and i can’t say, because I don’t know the game) AntiGrav may require momentary slices of attention. And this leads on to two major caveats about dual-task performance.
With most dual-task experiments performed in the lab, people are made to use all of their capacity to do a particular kind of task, and then psychologists look at what else they can’t do. In real life we are often performing under-capacity, and so can dynamically shift our ‘mental attention’ between the things we are doing. The podcasts may be intellectually demanding, but do you really miss so much if you move your focus to the game for 500 milliseconds? So, often when it looks like you are doing two things at once, you are really doing two things intersliced with each other.
The second, and related, major caveat is that we can often learn to do tasks, or bits of tasks, automatically. As we practice a skill (driving is a good example) we master more and more of the components so that we can do them as one thing, without out needing to deliberately focus our attention to oversee what we are doing. During one of my first driving lessons I was so busy trying to work out what order to check the mirrors in, how to coordinate the accelerator and clutch, and how to avoid steering the car into anything else, that I completely didn’t hear the engine complaining as i tried to do 40 mph in first gear. Everything else took up so much of my planning/coordinating resources that I had no spare attention to remember to change gears, or even to notice the ugly noise that resulted because I hadn’t (this example may make less sense to drivers of automatics, you lucky things). My instructor angrily pointed out the sound that was obvious to him but I’d completely missed. Now, having learnt to drive, my keeping-in-the-middle-of-the-road, changing gear, checking mirrors, and pedal control subroutines are all solidly over-learnt. Even if i did forget to change gears I would have spare attention to notice the sound of the engine.
I’m guessing there’s bits of the AntiGrav game which are similarly automatic for you. Chunks – movements, responses – which you can just decide to do and they happen without much further thought. The wonderful thing about our brains is that the more we do something the more and more of it becomes encapsulated into chunks. There’s a seminal cognitive psychology experiment in which typists were asked to touch type one document, while listening to an essay being read over ear-phones (and about which they were asked questions about at the end). At first the typists could do one task or the other, either typing at their normal speed or listening to the essay and answering the questions at the end to 90%-100% accuracy. After enough hours (and hours) of practice they could do both at once!
The last major thing that I think is relevant to this story is the input and output ‘codes’ of the two tasks. By ‘codes’ i mean the form that the information is represented in. We know that visual and spatial tasks interfere with each other if they are in conflict, so we infer that there is some single system that is responsible for processing them. Likewise auditory information and spoken outputs. The two tasks have parallel input-output streams which do not cross over between mental modules. I predict that if your game required you to remember verbal labels, or to make responses by saying things aloud, you wouldn’t be able to listen to your podcast while playing – nor would you ever be able to learn to do both things at once with the same skill as you could do both seperately.
A similar thing has been shown in a far simpler situation, the Stroop task (“Confuse Color Identification With Mixed Signals”, Hack #55). Normally, in this experiment, your ability to name our loud the colour of coloured-inks is confused by the colour-words that the colour-inks spell out . What you have is a visual input (the colour of the ink) which you are trying to convert into a verbal output (the name of the colour). You are translating across representational codes – and it’s difficult, people slow down and start making mistakes. If you make it so that the participants indicate their reponse with a movement the task becomes far easier . Now, conversely, the hard task is reading the word – this is normally the easy thing, the skill we’ve spent thousands of hours practicing, but because it involves translating across representaitonal codes (from the verbally-encoded semantics of the written word into the physical movement) it becomes more difficult.
So these factors, I think, probably explain the suprise that you can play the game and listen to the podcast at the same time.
One final caveat – although you say you play the game just as well when listening to the podcasts, this may not always be true. A close study of your scores might show a small drop when listening to podcasts. There’s some similarity here to people using mobile phones when driving (discussed a bit in “Don’t Divide Attention Across Locations”, Hack #54). Research has shown that in general people are very good at coordinating their driving and phone use, scheduling more attention to driving as it is needed, but if you truly are doing both at once there is a dip in performance, which may be all that’s needed to cause an accident.
Apologies if this is a little over-long and involved. I hope it is clear(ish).
 Sorry if this is confusing. Look at this and it may help make it clear
 I did an experiment that shows this for my PhD thesis (it’s discussed in chapter 5, but i wouldn’t really recommend anyone reads it )