The wrong sort of discussion

The Times Higher Education has an article on post-publication peer review, and whether it will survive legal challenges

The legal action launched by a US scientist who claims that anonymous comments questioning his science cost him a lucrative job offer has raised further questions about the potential for post-publication peer review to replace pre-publication review.

The article chimes with comments made by several prominent Psychologists who have been at the centre of controversies and decried the way their work has been discussed outside of the normal channels of the academic journals.

Earlier this year the head of a clinical trial of Tamiflu wrote to the British Medical Journal to protest that a BMJ journalist had solicited independent critique of the stats used in his work – “going beyond the reasonable response to a press release”.

John Bargh (Yale University) in his now infamous ‘nothing in their heads’ blogpost accused the open access journal PLoS of lacking “the usual high scientific journal standards of peer-review scrutiny”, and accussed Ed Yong – laughably – of “superficial online science journalism”. He concluded:

“I am not so much worried about the impact on science of essentially self-published failures to replicate as much as I’m worried about your ability to trust supposedly reputable online media sources for accurate information on psychological science.”

Simone Schnall (University of Cambridge) is a social psychologist whose work has also been at the centre of the discussion about replication (backstory, independent replication of her work recently reported). She has recently written that ‘no critical discussion is possible’ on social media, where ‘judgments are made quickly nowadays in social psychology and definitively’.

See also this comment from a scientist when a controversial paper which suggested that many correlations in fMRI studies of social psychological constructs were impossibly high was widely discussed before publication: . “I was shocked, this is not the way that scientific discourse should take place.”

The common theme is a lack of faith in the uncontrolled scientific discussion that now happens in public, before and after publication in the journal-sanctioned official record. Coupled, perhaps, with a lack of faith in other people to understand – let alone run – psychological research. Scientific discussion has always been uncontrolled, of course, the differences now are in how open the discussion is, and who takes part. Pre social media, ‘insider’ discussions of specialist topics took place inside psychology departments, and at conference dinners and other social gatherings of researchers. My optimistic take is that social media allows access to people who would not normally have it due to constraints on geography, finance or privilege. Social media means that if you’re in the wrong institution, aren’t funded, or if you have someone to look after at home that means you can’t fly to the conference, you can still experience and contribute to specialist discussions – that’s a massive and positive change and one we should protect as we work out how scientific discussion should take place in the 21st century.

Link: Simone Schnall’s comments in full: blog, video

Previously: Stafford, T., & Bell, V. (2012). Brain network: social media and the cognitive scientist. Trends in Cognitive Sciences, 16(10), 489–490. doi:10.1016/j.tics.2012.08.001

Previously What Jason Mitchell’s ‘On the emptiness of failed replications’ gets right, which includes some less optimistic notes on the current digital disruption of scholarly ways of working