A group of black bloc researchers fed up with the lack of interest in replicating psychology studies has set up a strike force called the The Reproducibility Project that will recreate all 2008 studies from three major cognitive science journals.
That sound you can hear. That’s shit hitting the fan.
The Chronicle of Higher Education covers the project that’ll check-out the replicability of well-known studies.
So why not check? Well, for a lot of reasons. It’s time-consuming and doesn’t do much for your career to replicate other researchers’ findings. Journal editors aren’t exactly jazzed about publishing replications. And potentially undermining someone else’s research is not a good way to make friends.
Brian Nosek knows all that and he’s doing it anyway. Nosek, a professor of psychology at the University of Virginia, is one of the coordinators of the project. He’s careful not to make it sound as if he’s attacking his own field. “The project does not aim to single out anybody,” he says. He notes that being unable to replicate a finding is not the same as discovering that the finding is false. It’s not always possible to match research methods precisely, and researchers performing replications can make mistakes, too.
But still. If it turns out that a sizable percentage (a quarter? half?) of the results published in these three top psychology journals can’t be replicated, it’s not going to reflect well on the field or on the researchers whose papers didn’t pass the test. In the long run, coming to grips with the scope of the problem is almost certainly beneficial for everyone. In the short run, it might get ugly.
Unfortunately, psychology and science in general still see a non-replication as a failure (in fact, we even use the term ‘failed replication’).
This is clearly nonsense and checking the original finding is equally as valuable if the new data agree with, or disagree with, the original study.
Sadly, we’ll have to change the attitude of several generations of scientists to reset this rusty conceptual switch.
The Reproducibility Project have just got frustrated with the entrenched attitudes and have manned the barricades. And who cam blame them?
Link to Chronicle article on The Reproducibility Project.