Looking at the evidence behind a recent news story
The headlines
The Los Angeles Times: People trust eyes – not ears – when judging musicians
Classic FM: Classical singers judged by actions not voice
Nature: Musicians’ appearances matter more than their sound
The story
If you wanted to pick out the musician who won a prestigious classical music competition would you listen to a clip of them playing or watch a silent video of them performing the same piece of music?
Most of us would go for an audio clip rather than video, and we’d be wrong. In a series of experiments, Chia-Jung Tsay from University College London, showed that both novices and expert musicians were better able to pick out the winners when they watched rather than listened to them.
The moral, we’re told, is that how you look is more important than how you sound, even in elite classical music competitions.
What they actually did
Dr Tsay, herself a classically trained musician, used footage from real international classical music competitions. She took the top three finalists and asked volunteers to pick out the real winner – with a cash incentive – by looking at video without sound, sound without video, or both.
Over a series of experiments she showed that people think that audio will be more informative than video, but actually people are able to pick the real winner when watching video clips. But they aren’t able to do this when listening to audio clips (these test subjects only perform at the level of chance). The shocking thing is that when people get sound and video clips, which notionally contain more information, they still perform at chance. The implication being that they would do better if they could block their ears and ignore the sound.
Follow up experiments suggested that people’s ability to pick winners depended on their being able to pick out things associated with “stage presence”. A video reduced to line drawings, designed to remove details and emphasise motion, still allowed people to pick out winners at an above chance rate. Another experiment showed that asking people to identify the “most confident, creative, involved, motivated, passionate, and unique performer” tallied with the real winners.
How plausible is this?
We’re a visual species. How things look really matters, as everyone who has dressed up for an interview knows. It’s also not uncommon for us to be misled into believing that how something looks isn’t as important as it really is (here’s an example: judging wine by the labels rather than the taste).
What is less plausible is the spin put on the story by the headlines. We all know that looks are important, but how can they really be more important than sound in a classical music competition? The most important thing really is the sound, but this research resonates with a popular cliché about how irrational we are.
Tom’s take
The secret to why these experiments give the results they do is in this detail: the judgement that people were asked to make was between the top three finalists in prestigious international competitions. In other words, each of these musicians is among the best in the world at what they do. The best of the best even.
In all probability there is a minute difference between their performances on any scale of quality. The paper itself admits that the judges themselves often disagree about who the winner is in these competitions.
The experimental participants were not scored according to some abstract ability to measure playing quality, but according to how well they were able to match real-world competition outcome.
The experiments show that matching the judges in these competitions can be done based on sight but not on sound. This isn’t because sight reveals playing quality, but because sight gives the experimental participants similar biases to the real judges. The real expert judges are biased by how the performers look – and why not, since there is probably so little to choose between them in terms of how they sound?
This is why the conclusion, spelt out in the original paper, is profoundly misleading: “The findings demonstrate that people actually depend primarily on visual information when making judgements about music performance”. It remains completely plausible that most of us, most of the time, judge music on how it sounds, just like we assumed before this research came out.
In ambiguous cases we might rely on looks over sounds – even the experts among us. This is a blow to musicians who thought it was always just about sound – but isn’t a revelation to the rest of us who knew that when choices are hard, whether during the job interview or the music competition, looks matter.
Read more
The original paper: Sight over sound in the judgment of music performance. Tsay, C-J (2013), Proceedings of the National Academy of Sciences
Special mention for the BBC and reporter Melissa Hogenboom who were the only people, as far as I know, who managed to report this story with an accurate headline: Sight dominates sound in music competition judging
The interaction between the senses is an active and fascinating research area. Read more from the Crossmodal Research Laboratory at the Univeristy of Oxford and Cross-modal perception of music network at the University of Sheffield
This article was originally published at The Conversation.
Read the original article.
Mind Hacks 
I would never discount the bias toward attractive people in subjective judging processes, but another possible factor is the visual excitement of the performance. Piano virtuoso Alexander Toradze, for example, plays with amazing force and passion – at times, his energy makes him seem to levitate a few inches above the bench. While one can appreciate his virtuosity in an audio recording, seeing a live performance is far more electrifying. (E.g., http://youtu.be/QC45mdf8PRI)
I’ve also seen technically brilliant pianists who, in concert, didn’t themselves seem to be moved by the music. If one watched video-only performances by pianists with these contrasting styles, both pros and amateurs might well choose the more expressive performers.
Another example of how discussion of results can not be interpreted correctly unless context is understood.
Say what you want…but without context you understand only what your confirmation bias gives you.
An interesting test would be to rate still photos for good looks and see if those rankings still matched the competition judges.
I find myself agreeing with what visibility said. Some pianists turn playing into sort of a performance art. With their bursting displays of emotion. It’s not just about playing perfectly, it involves sudden arching of the back, hunching, even making expressive grimaces. Whether or not it is on purpose is irrelevant, it brings a visual aspect to their performance, and that can definitely make a difference.
If they showed them still photos and asked them to decide the winner, I would be more inclined to believe that looks was the deciding factor.
Okay, my concentration level’s not great today but let’s see if I understand: novices and expert volunteers were tested to see if they could match the competition judges using either just video or audio.
So the question is, did anyone test the competition judges with a blind evaluation? If among the same competitors they awarded one winner based on audio alone then awarded completely different winners based on visual and audio, there’s your answer.
This article interestingly reminds me of the Kennedy-Nixon presidential debate of 1960. It was the first debate to be shown on television, opening up a whole new medium through which the public could experience it. Those who listened to the debate on the radio tended to say that Nixon won the debate, whereas those who watched it on TV tended to say that Kennedy won. Kennedy ended up winning the debate overall. This shows the importance of appearance in a different context: politics.
I would agree with visibility that, judging by context, people make decisions by appearance because that conveys certain abstract, subconscious things to us like vitality, leadership and charisma… which could be more important that technical skill.
I find it interesting how much we rely on appearances. Just the other day I was learning about how based on first impressions, people subconsciously think that “attractive” people are smarter,happier,and even more emotionally stable than everyone else. We as humans tend to focus on the presence generated by someone more than their actual abilities.