My latest column for BBC Future. The original is here. I was hesitant to write this at first, since nobody loves a problemmatiser, but I figured that something in support of team “I think you’ll find its a bit more complicated than that” couldn’t hurt, and there’s an important general point about the way facts about behaviour are built from data in the final paragraphs, and why theory is important.
Recent reports say curved glasses make you drink beer quicker. But, we must be cautious about drawing simple conclusions from single studies.
We all love a neat science story, but even rock solid facts can be less revealing than they seem. Let’s take an example of a piece of psychology research reported recently: the idea that people drink faster from curved glasses.
Hundreds of news sources around the globe covered the findings, many of them changing the story slightly to report that people drink more (rather than faster) from a curved glass. At first it seems like a straightforward piece of psychology research, with clear implications: curved glasses will make pacing yourself harder, so you’ll end up drinking more than you should. Commentators agreed with the research (funded by Alcohol Research UK) – beverage manufacturers were probably onto this before, and will now be rushing to make us take our favourite tipple out of a curved glass.
But before we change our drinking habits or restock our glass collections, let’s look at what the scientists actually did.
Luckily for us the team of researchers from the University of Bristol, UK, published their paper in an open access journal, which means the research details are free for all to read.
The Bristol team invited participants into the lab and asked them to drink lager (or lemonade) from a straight class or a curved one, while watching a nature documentary (a BBC one, I’m happy to report). They also asked their volunteers to judge when the glass was half full. The results of both were clear, participants finished their drink of lager sooner in the curved glass. They also judged the halfway point as being lower down the curved glass than the straight glass – suggesting a reason for the faster drinking: if people thought the glass was fuller than it really was when then they would underestimate the rate at which they were drinking.
Now this is all well and good, but there are many reasons why the results don’t mean that we can make people drink more by changing the shape of their glass. Importantly, none of these reasons would have to do with this research being wrong or inexpertly done. I’m absolutely certain that if we did the study ourselves we’d find exactly the same thing.
No, the reasons you can’t jump to conclusions from this kind of study is because, inevitably, a single study can only test one aspect of the world, under one set of circumstances. This makes it hard to draw general conclusions of the sort that get reported. Notice how the psychologists measured one thing (rate of drinking lager), for just two different glasses, over a single drink, for one set of people (volunteers in Bristol, in 2012), and yet a generalised truth stating that “people drink more from curved glasses” emerged from a specific set of circumstances.
Now obviously, the aim of science is to come up with answers to questions that become generalised truths, but psychology is a domain in which it is fiendishly hard to establish them. If you are studying a simple system, then cause and effect is relatively easy to establish. For instance, the harder you throw a rock, the further it tends to travel in distance. The relation between the force you put in and the acceleration of the rock you get out is straightforward. Add a human factor into the equation, however, and such simple relations begin to disappear. (Please don’t experiment by throwing rocks at people.)
To see how this limits the conclusions that can be drawn from the drinks study, think of even the most trivial factor that could change these results. Would you get the same result if people drank ale rather than lager? Probably. If they drank two pints rather than one? Maybe. If they drank in groups rather than watching TV (arguably closer to the circumstances of most drinking)? Who knows! It seems to me perfectly plausible that a social situation would produce different effects than a solo-drinking experience.
We could carry on. Would the effect be the same if we tried it in Minneapolis? In Lagos? In Kuala Lumpur, Reykjavik or Alice Springs? Most psychology studies are carried out on urban, affluent, students of the western world – a culturally unusual group, if you take a global or historical perspective. All the subjects studied were “social drinkers”, presumably with some learnt associations about curved and straight glasses. Maybe the Brits had learnt that expensive beer came in curved glasses. If this is the case, the result might be true for everyone who has a history of drinking from straight glasses in the UK, but not for other cultures where alcohol isn’t drunk like that.
Little things, big effect
Software entrepreur Jim Manzi calls the rate at which small changes can have surprising effects on outcomes, and the consequent difficulty in drawing general conclusions, “causal density”. It’s because human psychology and social life is so causally dense that we can’t simply take straight reports that X affects Y and apply them across the board. But there are hundreds of these relationships reported all the time from the annals of psychology: glass shape affects drinking time, taller men are better paid, holding a hot drink makes you like someone, and so on. Surface effects like these are vulnerable to small changes in circumstances that might remove, or even reverse, the effect you’re relying on.
Psychology researchers know all these arguments, and that’s why they’re cautious about drawing simple conclusions from single studies. The challenge of psychology is to track down those results that actually do generalise across different situations.
The way to do this is to report findings that are about theories, not just about effects. The Bristol researchers show the way in their paper: as well as testing drinking speed, they relate it to people’s ability to estimate how full a glass is. They could have just measured drinking speed, but they knew they had to relate it to a theory about what people really believed to come up with a strong conclusion.
If we can find the right principles that affect people’s actions, then we can draw conclusions that cut across situations. Unless we know the reasons why someone does something, we’ll be tricked time and time again when we try to infer from what they do.
8 thoughts on “BBC Column: Can glass shape really affect how fast you drink?”
Fantastic article. Critique and reporting like this are the reason I follow this blog in the first place.
I’m here to attest to the fact that, as a former resident (or survivor!) of Bristol University’s neighbourhood, that students there can drink alcohol fast out of anything, at anytime, under any circumstances!
Seriously though, good catch. Sloppy science, or at least reportage, does no one any good.
Just a tangential point also about the dangers of making apparently definitive conclusions. You write “Psychology researchers know all these arguments, and that’s why they’re cautious about drawing simple conclusions from single studies.”
So why is it that clinical psychologists make snap conclusions from single studies (i.e. patients) all too often and sometimes to the harm of the ‘study subject’?
The Phytopractor blogged about this and he wanted to know if they controlled for optimists and pessimists. 😉
Great reporting on how science works. Can we really have too much of this? I don’t think so. Although I’m curious how the original hypothesis came about.
I heard somewhere (possibly in The Economist?) that this is one of the reasons why mturk is being exploited so much for international psych research (other than it’s cheap to do). A lot of the psych literature (and the fMRI literature, too) is based on (supposedly) healthy 18-24 y/o students; if it’s in a western uni and it’s literature older than two decades then there’s often a serious racial bias to boot. Seems your white western student has all sorts of different values and behaviors than people in other demographics. (Shocker!)
My kin in the neuroimaging realm need to be acutely aware of our demographic biases when we’re showing how it’s the XYZ network of colored blobs that indicates you’re better at playing the piano left-handed after you’ve been watching Simpsons videos in the scanner for half an hour. FMRI results in particular tend to get generalized out of all proportion. Caveats abound, not least that one about the test subjects lying in a dark, confined tube with clanking noises all around…
@amelie – fun fact from the paper, both groups (curved and straight glasses) underestimated the amount of booze in the glasses. In other words, the average is to think the glass is *less* than half full (or more than half empty, depending!)
@TomStafford that really is interesting. Thank you and to @MarcusMunafo for the backstory.
I’m one of the authors on this paper, and this is a really great critical analysis. It illustrates the problems we face in almost every experiment we run as psychologists.
The idea for this study came out of a conversation between myself (I research tobacco and alcohol use, mainly, from a number of perspectives) and a vision scientist (with an interest in psychophysics). Not surprisingly, it was over a drink, and we were wondering why there seemed to be such an increase in the number of odd-shaped, branded glasses in which beers were being served. I think it’s quite a nice example of how people from different research backgrounds can come together to try to begin to answer complex questions.
Of course there are lots of limitations (not least the sample – psychology has been described as the science of psychology undergraduate students), and I would never want to over-state the results. There’s also perhaps an interesting discussion to be had about whether we (consciously or unconsciously) over-stated our findings because of the current impact agenda in UK research. On the other hand, there is at least a possibility that our results will translate to the real world. We’re obviously planning follow-up studies.
In the UK one of the strongest determinants of drinking rate is whether you are drinking in a round. Drinking alone, with few external influences, helps to isolate specific effects, but it also means the results can’t easily be generalised. So we have to strike a balance between experimental purity and generalisability. As a first step, we generally err on the side of the former, but ultimately we need to get to the latter. I think psychology often does a pretty poor job of this and stops the development of an idea too early.
Anyway – thanks for a really considered analysis. I agree there’s far too little of this, and it makes a refreshing change from the media coverage, which ranges from the quite good to the pretty poor, but is almost always fairly shallow.
What twitter’s character limit was a little too short to say was:
What I really like about your article Tom is that the last bit is an ace attack on positivism.
The reporting of psychology stuff in the press is always positivist, and that is what gives it its magical flavour. It all subtly promotes the idea that the ideas in our heads are kinda illusions. We may think that we do things because of stuff that we are thinking or feeling, but REALLY we are being governed by forces beyond our control, which special authority figures called scientists can reveal to us. It is thus an attack our dignity and autonomy, and on the view of us as reasonable creatures who can be talked to as well as controlled with prodding etc (and thus many other things, including – I think – a subtle attack on democracy).
The second you relate drinking speed to people’s ability to estimate how full a glass is – to a theory about what people believe – you pull it back from that kind of magical positivism, and it slots back into the normal folk psychology that is compatible with our being creatures of reason, meaning and dignity.