My latest column for BBC Future. The original is here. I was hesitant to write this at first, since nobody loves a problemmatiser, but I figured that something in support of team “I think you’ll find its a bit more complicated than that” couldn’t hurt, and there’s an important general point about the way facts about behaviour are built from data in the final paragraphs, and why theory is important.
Recent reports say curved glasses make you drink beer quicker. But, we must be cautious about drawing simple conclusions from single studies.
We all love a neat science story, but even rock solid facts can be less revealing than they seem. Let’s take an example of a piece of psychology research reported recently: the idea that people drink faster from curved glasses.
Hundreds of news sources around the globe covered the findings, many of them changing the story slightly to report that people drink more (rather than faster) from a curved glass. At first it seems like a straightforward piece of psychology research, with clear implications: curved glasses will make pacing yourself harder, so you’ll end up drinking more than you should. Commentators agreed with the research (funded by Alcohol Research UK) – beverage manufacturers were probably onto this before, and will now be rushing to make us take our favourite tipple out of a curved glass.
But before we change our drinking habits or restock our glass collections, let’s look at what the scientists actually did.
Luckily for us the team of researchers from the University of Bristol, UK, published their paper in an open access journal, which means the research details are free for all to read.
The Bristol team invited participants into the lab and asked them to drink lager (or lemonade) from a straight class or a curved one, while watching a nature documentary (a BBC one, I’m happy to report). They also asked their volunteers to judge when the glass was half full. The results of both were clear, participants finished their drink of lager sooner in the curved glass. They also judged the halfway point as being lower down the curved glass than the straight glass – suggesting a reason for the faster drinking: if people thought the glass was fuller than it really was when then they would underestimate the rate at which they were drinking.
Human factor
Now this is all well and good, but there are many reasons why the results don’t mean that we can make people drink more by changing the shape of their glass. Importantly, none of these reasons would have to do with this research being wrong or inexpertly done. I’m absolutely certain that if we did the study ourselves we’d find exactly the same thing.
No, the reasons you can’t jump to conclusions from this kind of study is because, inevitably, a single study can only test one aspect of the world, under one set of circumstances. This makes it hard to draw general conclusions of the sort that get reported. Notice how the psychologists measured one thing (rate of drinking lager), for just two different glasses, over a single drink, for one set of people (volunteers in Bristol, in 2012), and yet a generalised truth stating that “people drink more from curved glasses” emerged from a specific set of circumstances.
Now obviously, the aim of science is to come up with answers to questions that become generalised truths, but psychology is a domain in which it is fiendishly hard to establish them. If you are studying a simple system, then cause and effect is relatively easy to establish. For instance, the harder you throw a rock, the further it tends to travel in distance. The relation between the force you put in and the acceleration of the rock you get out is straightforward. Add a human factor into the equation, however, and such simple relations begin to disappear. (Please don’t experiment by throwing rocks at people.)
To see how this limits the conclusions that can be drawn from the drinks study, think of even the most trivial factor that could change these results. Would you get the same result if people drank ale rather than lager? Probably. If they drank two pints rather than one? Maybe. If they drank in groups rather than watching TV (arguably closer to the circumstances of most drinking)? Who knows! It seems to me perfectly plausible that a social situation would produce different effects than a solo-drinking experience.
We could carry on. Would the effect be the same if we tried it in Minneapolis? In Lagos? In Kuala Lumpur, Reykjavik or Alice Springs? Most psychology studies are carried out on urban, affluent, students of the western world – a culturally unusual group, if you take a global or historical perspective. All the subjects studied were “social drinkers”, presumably with some learnt associations about curved and straight glasses. Maybe the Brits had learnt that expensive beer came in curved glasses. If this is the case, the result might be true for everyone who has a history of drinking from straight glasses in the UK, but not for other cultures where alcohol isn’t drunk like that.
Little things, big effect
Software entrepreur Jim Manzi calls the rate at which small changes can have surprising effects on outcomes, and the consequent difficulty in drawing general conclusions, “causal density”. It’s because human psychology and social life is so causally dense that we can’t simply take straight reports that X affects Y and apply them across the board. But there are hundreds of these relationships reported all the time from the annals of psychology: glass shape affects drinking time, taller men are better paid, holding a hot drink makes you like someone, and so on. Surface effects like these are vulnerable to small changes in circumstances that might remove, or even reverse, the effect you’re relying on.
Psychology researchers know all these arguments, and that’s why they’re cautious about drawing simple conclusions from single studies. The challenge of psychology is to track down those results that actually do generalise across different situations.
The way to do this is to report findings that are about theories, not just about effects. The Bristol researchers show the way in their paper: as well as testing drinking speed, they relate it to people’s ability to estimate how full a glass is. They could have just measured drinking speed, but they knew they had to relate it to a theory about what people really believed to come up with a strong conclusion.
If we can find the right principles that affect people’s actions, then we can draw conclusions that cut across situations. Unless we know the reasons why someone does something, we’ll be tricked time and time again when we try to infer from what they do.
It’s not often you see someone licking a brain in a rock n’ roll video and get to think to yourself “well, there’s a funny story behind that”, but this is one of those occasions.
The New York Times has an 
“It’s no secret” says the promotional material “that several professional footballers live in Repton Park”, presumably unaware that one of London’s most luxurious housing developments used to be a psychiatric hospital.
Like most neuroscientists, when you’ve had a hard day at the lab, you get home and just want to scatter a few rose petals into a warm bath, put on your favourite whale song tape, and sink into the tub.
Mind Hacks 