Serendipity in psychological research

micDorothy Bishop has an excellent post ‘Ten serendipitous findings in psychology’, in which she lists ten celebrated discoveries which occurred by happy accident.

Each discovery is interesting in itself, but Prof Bishop puts the discoveries in the context of the recent discussion about preregistration (declaring in advance what you are looking for and how you’ll look). Does preregistration hinder serendipity? Absolutely not says Bishop, not least because the context of ‘discovery’ is never a one-off experiment.

Note that, in all cases, having made the initial unexpected observation – either from unstructured exploratory research, or in the course of investigating something else – the researchers went on to shore up the findings with further, hypothesis-driven experiments. What they did not do is to report just the initial observation, embellished with statistics, and then move on, as if the presence of a low p-value guaranteed the truth of the result.

(It’s hard not to read into these comments a criticism of some academic journals which seem happy to publish single experiments reporting surprising findings.)

Bishop’s list contains 3 findings from electrophysiology (recording brain cell activity directly with electrodes), which I think is notable. In these cases neural recording acts in the place of a microscope, allowing fairly direct observation of the system the scientist is investigating at a level of detail hitherto unavailable. It isn’t surprising to me that given a new tool of observation, the prepared mind of the scientists will make serendipitous discoveries. The catch is whether, for the rest of psychology, such observational tools exist. Many psychologists use their intuition to decide where to look, and experiments to test whether their intuition is correct. The important serendipitous discoveries from electrophysiology suggest that measures which are new ways of observing, rather than merely tests of ideas, must also be important for psychological discoveries. Do such observational measures exist?

Images of ultra-thin models need your attention to make you feel bad

I have a guest post over at the BPS Research Digest, covering research on the psychological effects of pictures of ultra-thin fashion models.

A crucial question is whether the effect of these thin-ideal images is automatic. Does the comparison to the models, which is thought to be the key driver in their negative effects, happen without our intention, attention or both? Knowing the answer will tell us just how much power these images have, and also how best we might protect ourselves from them.

It’s a great study from the lab of Stephen Want (Ryerson University). For the full details of the research, head over: Images of ultra-thin models need your attention to make you feel bad

Update: Download the preprint of the paper, and the original data here

CBT is becoming less effective, like everything else

‘Researchers have found that Cognitive Behavioural Therapy is roughly half as effective in treating depression as it used to be’ writes Oliver Burkeman in The Guardian, arguing that this is why CBT is ‘falling out of favour’. It’s worth saying that CBT seems as popular as ever, but even if it was in decline, it probably wouldn’t be due to diminishing effectiveness – because this sort of reduction in effect is common across a range of treatments.

Burkeman is commenting on a new meta-analysis that reports that more recent trials of CBT for depression find it to be less effective than older trials but this pattern is common as treatments are more thoroughly tested. This has been reported in antipsychotics, antidepressants and treatments for OCD to name but a few.

Interestingly, one commonly cited reason treatments become less effective in trials is because response to placebo is increasing, meaning many treatments seem to lose their relative potency over time.

Counter-intuitively, for something considered to be ‘an inert control condition’ the placebo response is very sensitive to the design of the trial, so even comparing placebo against several rather than one active treatment can affect placebo response.

This has led people to suggest lots of ‘placebo’ hacks. “In clinical trials,” noted one 2013 paper in Drug Discovery, “the placebo effect should be minimized to optimize drug–placebo difference”.

It’s interesting that it is still not entirely clear whether this approach is ‘revealing’ the true effects of the treatment or just another way of ‘spinning’ trials for the increasingly worried pharmaceutical and therapy industries.

The reasons for the declining treatment effects over time are also likely to include different types of patients selected into trials, more methodologically sound research practices meaning less chance of optimistic measuring and reporting, the fact that if chance gives you a falsely inflated treatment effect first time round it is more likely to be re-tested than initially less impressive first trials, and the fact that older known treatments might bring a whole load of expectations with them that brand new treatments don’t.

The bottom line is that lots of our treatments, across medicine as a whole, have quite modest effects when compared to placebo. But if placebo represents an attempt to address the problem, it provides quite a boost to the moderate effects that the treatment itself brings.

So the reports of the death of CBT have been greatly exaggerated but this is mostly due to the fact that lots of treatments start to look less impressive when they’ve been around for a while. This is less due to them ‘losing’ their effect and more likely due to us more accurately measuring their true but more modest effect over time.

Phantasmagoric neural net visions

dreaming neural network imageA starling galley of phantasmagoric images generated by a neural network technique has been released. The images were made by some computer scientists associated with Google who had been using neural networks to classify objects in images. They discovered that by using the neural networks “in reverse” they could elicit visualisations of the representations that the networks had developed over training.

These pictures are freaky because they look sort of like the things the network had been trained to classify, but without the coherence of real-world scenes. In fact, the researchers impose a local coherence on the images (so that neighbouring pixels do similar work in the image) but put no restraint on what is globally represented.

The obvious parallel is to images from dreams or other altered states – situations where ‘low level’ constraints in our vision are obviously still operating, but the high-level constraints – the kind of thing that tries to impose an abstract and unitary coherence on what we see – is loosened. In these situations we get to observe something that reflects our own processes as much as what is out there in the world.

Link: The researchers talk about their ‘dreaming neural networks’
Gallery: Inceptionism: Going deeper into Neural Networks

Explore our back pages

At our birthday party on Thursday I told people how I’d crunched the stats for the 10 years of mindhacks.com posts. Nearly 5000 posts, and over 2 million words – an incredible achievement (for which 96% of the credit should go to Vaughan).

In 2010 we had an overhaul (thanks JD for this, and Matt for his continued support of the tech side of the site). I had a look at the stats, which only date back till then, and pulled out our all time most popular posts. Here they are:

topten

Something about the enthusiasm of last Thursday inspired me to put the links the top ten posts on a wiki. Since it is a wiki anyone can jump in and edit, so if there are any bits of the mindhacks.com back catalogue that you think are worth leaving a placeholder to, feel free to add it. Vaughan and I will add links to a few of our favourite posts, so check back and see how it is coming along.

Link: Mind Hacks wiki

Quasi-stability

Yesterday, before I got here, my dad was trying to fix an invisible machine. By all accounts, he began working on the phantom device quite intently, but as his repairs began to involve the hospice bed and the tubes attached to his body, he was gently sedated, and he had to leave it, unresolved.

This was out-of-character for my father, who I presumed had never encountered a machine he couldn’t fix. He built model aeroplanes in rural New Zealand, won a scholarship to go to university, and ended up as an aeronautical engineer for Air New Zealand, fixing engines twice his size. More scholarships followed and I first remember him completing his PhD in thermodynamics, or ‘what heat does’, as he used to describe it, to his six-year-old son.

When he was first admitted to the hospice, more than a week go, he was quite lucid – chatting, talking, bemoaning the slow pace of dying. “Takes too long,” he said, “who designed this?” But now he is mostly unconscious.

Occasionally though, moments of lucidity dodge between the sleep and the confusion. “When did you arrive?” he asked me in the early hours of this morning, having woken up wanting water. Once the water was resolved he was preoccupied about illusory teaspoons lost among the bedclothes, but then chatted in feint short sentences to me and my step-mum before drifting off once more.

Drifting is a recent tendency, but in the lucidity he has remained a proud engineer. It’s more of a vocation, he always told his students, than a career.

Last week, when the doctors asked if he would speak to medical trainees, he was only too happy to have a final opportunity to teach. Even the consultants find his pragmatic approach to death somewhat out of the ordinary and they funnelled eager learners his way where he engaged with answering their questions and demonstrating any malfunctioning components.

“When I got here”, he explained to them, “I was thermodynamically unstable but now I think I’m in a state of quasi-stability. It looks like I have achieved thermal equilibrium but actually I’m steadily losing energy.”

“I’m not sure”, I said afterwards, “that explaining your health in terms of thermodynamics is exactly what they’re after.”

“They’ll have to learn,” he said, “you can’t beat entropy.”


Postscript

My dad finally returned to entropy on the afternoon of Friday 31st October, with his family and a half-read book on nanoscience by his side.

Dr Murray Alan Bell, 30th January 1945 – 31st October 2014, Engineer (by vocation as much as by career)

Hallucinating astronauts

I’ve got a piece in The Observer about the stresses, strains and mind-bending effects of space flight.

NASA considers behavioural and psychiatric conditions to be one of the most significant risks to the integrity of astronaut functioning and there is a surprisingly long history of these difficulties adversely affecting missions.

Perhaps more seriously, hallucinations have been associated with the breakdown of crew coherence and space mission stress. In 1976, crew from the Russian Soyuz-21 mission were brought back to Earth early after they reported an acrid smell aboard the Salyut-5 space station. Concerns about a possible fluid leak meant the replacement crew boarded with breathing equipment, but no odour or technical problems were found. Subsequent reports of “interpersonal issues” and “psychological problems” in the crew led Nasa to conclude the odour was probably a hallucination. Other Russian missions were thought to be have been halted by psychological problems, but the US space programme has not been without difficulties. During the Skylab 4 mission, long hours, exhaustion and disagreements with mission control resulted in the crew switching off their radio and spending a day ignoring Nasa while watching the Earth’s surface pass by.

The piece also tackles a curious form of hallucination caused by cosmic rays and the detrimental effects of zero-gravity of brain function, as well as some curious Freudian theories from pre-space flight 1950s about the potential psychological consequences of leaving ‘Mother Earth’.

Enjoy!
 

Link to Observer article on psychological challenges of astronauts.