Statistical fallacy impairs post-publication mood

banksyNo scientific paper is perfect, but a recent result on the affect of mood on colour perception is getting a particularly rough ride post-publication. Thorstenson and colleagues published their paper this summer in Psychological Science, claiming that people who were sad had impaired colour perception along the blue-yellow colour axis but not along the red-green colour axis. Pubpeer – a site where scholars can anonymously discuss papers after publication – has a critique of the paper, which observes that the paper commits a known flaw in its analysis.

The flaw, anonymous comments suggest, is that a difference between the two types of colour perception is claimed, but this isn’t actually tested by the paper – instead it shows that mood significantly affects blue-yellow perception, but does not significantly affect red-green perception. If there is enough evidence that one effect is significant, but not enough evidence for the second being significant, that doesn’t mean that the two effects are different from each other. Analogously, if you can prove that one suspect was present at a crime scene, but can’t prove the other was, that doesn’t mean that you have proved that the two suspects were in different places.

This mistake in analysis  – which is far from unique to this paper – is discussed in a classic 2011 paper by Nieuwenhuis and colleagues: Erroneous analyses of interactions in neuroscience: a problem of significance. At the time of writing the sentiment on Pubpeer is that the paper should be retracted – in effect striking it from the scientific record.

With commentary like this, you can see why Pubpeer has previously been the target of legal action by aggrieved researchers who feel the site unfairly maligns their work.

(h/t to Daniël Lakens and jjodx on twitter)

UPDATE 5/11/15: It’s been retracted

1 thought on “Statistical fallacy impairs post-publication mood”

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s