Web of illusion: how the internet affects our confidence in what we know

The internet can give us the illusion of knowledge, making us think we are smarter than we really are. Fortunately, there may be a cure for our arrogance, writes psychologist Tom Stafford.

The internet has a reputation for harbouring know-it-alls. Commenters on articles, bloggers, even your old school friends on Facebook all seem to swell with confidence in their understanding of exactly how the world works (and they are eager to share that understanding with everyone and anyone who will listen). Now, new research reveals that just having access to the world’s information can induce an illusion of overconfidence in our own wisdom. Fortunately the research also shares clues as to how that overconfidence can be corrected.

Specifically, we are looking at how the internet affects our thinking about what we know, a topic psychologists call metacognition. When you know you are boasting, you are being dishonest, but you haven’t made any actual error in estimating your ability. If you sincerely believe you know more than you do then you have made an error. The research suggests that an illusion of understanding may actually be incredibly common, and that this metacognitive error emerges in new ways in the age of the internet.

In a new paper, Matt Fisher of Yale University, considers a particular type of thinking known as transactive memory, which is the idea that we rely on other people and other parts of the world – books, objects – to remember things for us. If you’ve ever left something you needed for work by the door the night before, then you’ve been using transactive memory.

Part of this phenomenon is the tendency to then confuse what we really know in our personal memories, with what we have easy access to, the knowledge that is readily available in the world, or with which we are merely familiar without actually understanding in depth. It can feel like we understand how a car works, the argument goes, when in fact we are merely familiar with making it work. I press the accelerator and it goes forward, neglecting to realise that I don’t really know how it goes forward.

Fisher and colleagues were interested in how this tendency interacts with the internet age. They asked people to provide answers to factual questions, such as “Why are there time zones?”. Half of the participants were instructed to look up the answers on the internet before answering, half were told not to look up the answers on the internet. Next, all participants were asked how confidently they could explain the answers to a second series of questions (seperate, but also factual, questions such as “Why are cloudy nights warmer?” or “How is vinegar made?”).

Sure enough, people who had just been searching the internet for information were significantly more confident about their understanding of the second set of questions. Follow up studies confirmed that these people really did think the knowledge was theirs: they were still more confident if asked to indicate their response on a scale representing different levels of understanding with pictures of brain-scan activity (a ploy that was meant to emphasise that the information was there, in their heads). The confidence effect even persisted when the control group were provided answer material and the internet-search group were instructed to search for a site containing the exact same answer material. Something about actively searching for information on the internet specifically generated an illusion that the  knowledge was in the participants’ own heads.

If the feeling of controlling information generates overconfidence in our own wisdom, it might seem that the internet is an engine for turning us all into bores. Fortunately another study, also published this year, suggests a partial cure.

Amanda Ferguson of the University of Toronto and colleagues ran a similar study, except the set-up was in reverse: they asked participants to provide answers first and, if they didn’t know them, search the internet afterwards for the correct information (in the control condition participants who said “I don’t know” were let off the hook and just moved on to the next question). In this set up, people with access to the internet were actually less willing to give answers in the first place than people in the no internet condition. For these guys, access to the internet shut them up, rather than encouraging them to claim that they knew it all. Looking more closely at their judgements, it seems the effect wasn’t simply that the fact-checking had undermined their confidence. Those that knew they could fall back on the web to check the correct answer didn’t report feeling less confident within themselves, yet they were still less likely to share the information and show off their knowledge.

So, putting people in a position where they could be fact-checked made them more cautious in their initial claims. The implication I draw from this is that one way of fighting a know-it-all, if you have the energy, is to let them know that they are going to be thoroughly checked on whether they are right or wrong. It might not stop them researching a long answer with the internet, but it should slow them down, and diminish the feeling that just because the internet knows some information, they do to.

It is frequently asked if the internet is changing how we think. The answer, this research shows, is that the internet is giving new fuel to the way we’ve always thought. It can be both a cause of overconfidence,  when we mistake the boundary between what we know and what is available to us over the web, and it can be a cause of uncertainty, when we anticipate that we’ll be fact-checked using the web on the claims we make. Our tendencies to overestimate what we know, to use information that is readily available as a substitute for our own knowledge, and to worry about being caught out are all constants on how we think. The internet slots into this tangled cognitive ecosystem, from which endless new forms evolve.

This is my BBC Future column from earlier this week. The original is here

Statistical fallacy impairs post-publication mood

banksyNo scientific paper is perfect, but a recent result on the affect of mood on colour perception is getting a particularly rough ride post-publication. Thorstenson and colleagues published their paper this summer in Psychological Science, claiming that people who were sad had impaired colour perception along the blue-yellow colour axis but not along the red-green colour axis. Pubpeer – a site where scholars can anonymously discuss papers after publication – has a critique of the paper, which observes that the paper commits a known flaw in its analysis.

The flaw, anonymous comments suggest, is that a difference between the two types of colour perception is claimed, but this isn’t actually tested by the paper – instead it shows that mood significantly affects blue-yellow perception, but does not significantly affect red-green perception. If there is enough evidence that one effect is significant, but not enough evidence for the second being significant, that doesn’t mean that the two effects are different from each other. Analogously, if you can prove that one suspect was present at a crime scene, but can’t prove the other was, that doesn’t mean that you have proved that the two suspects were in different places.

This mistake in analysis  – which is far from unique to this paper – is discussed in a classic 2011 paper by Nieuwenhuis and colleagues: Erroneous analyses of interactions in neuroscience: a problem of significance. At the time of writing the sentiment on Pubpeer is that the paper should be retracted – in effect striking it from the scientific record.

With commentary like this, you can see why Pubpeer has previously been the target of legal action by aggrieved researchers who feel the site unfairly maligns their work.

(h/t to Daniël Lakens and jjodx on twitter)

UPDATE 5/11/15: It’s been retracted

How the magic of cinema unlocked one man’s coma-bound world

Image from NIH. Click for source.An Alfred Hitchcock film helped to prove one patient had been conscious while in a coma-like state for 16 years. The discovery shows that neuroscience may still have lots to learn from the ancient art of storytelling, says Tom Stafford.

If brain injury steals your consciousness then you are in a coma: we all know that. What is less well known is that there exist neighbouring states to the coma, in which victims keep their eyes open, but show no signs of consciousness. The vegetative state, or ‘unresponsive wakefulness syndrome’, is one in which the patient may appear to be awake, and even goes to sleep at times, but otherwise shows no reaction to the world. Patients who do inconsistently respond, such as by flinching when their name is called, or following a bright object with their eyes, are classified as in a ‘minimally conscious state’. Both categories of patients show no signs of deliberate actions, or sustained reaction to the environment, and until recently there was no way for anyone to discern their true, inner, level of consciousness.

The fear is that, like the ‘locked-in syndrome’ that can occur after strokes, these patients may be conscious, but are just unable to show it. The opposite possibility is that these patients are as unconscious as someone in the deepest coma, with only circuitry peripheral to consciousness keeping their eyes open and producing minimal responses automatically.

In the last 10 years, research spearheaded by cognitive neuroscientist Adrian Owen has transformed our understanding of these shadowlands of consciousness. There is now evidence, obtained using brain scans, that some patients (around one in five) in these ‘wakeful coma’ states have conscious awareness. If asked to imagine playing tennis, the brain areas specifically controlling movement become active. If asked to imagine finding their way around their house, the brain regions involved in navigation become active. Using these signals a small minority of patients have even communicated with the outside world, with the brain scanner helping observers to mind-read their answers to questions.

The practical and ethical implications of these findings are huge, not least for the treatment of the hundreds of thousands of people who are in hospitals around the world in these conditions right now.

But the meaning of the research is still hotly debated. One issue is that the mind reading uses neural responses to questions or commands, and careful controls are needed to ensure that their patients’ brains aren’t just responding automatically without any actual conscious involvement. A second issue, and one that cannot be controlled away, is that the method used may tell us that these patients are capable of responding, but it doesn’t tell us much about the quality of conscious experience they are having. How alert, aware and focused they are is hard to discern.

In a relatively new study, a post-doctoral fellow at Owen’s lab, Lorina Naci, has used cinema to show just how sophisticated conscious awareness can be in a ‘minimally conscious’ patient.

The trick they used involved an 8 minute edit of “Bang! You’re dead”, a 1961 episode of “Alfred Hitchcock Presents”. In the film, a young boy with a toy gun obsession wanders around aiming and firing at people. Unbeknownst to him, and the adults he aims at, on this day he has found a real gun and it has a live bullet in the chamber.

The film works because of this hidden knowledge we, the viewers, have. Knowing about the bullet, a small boy’s mundane antics become high drama, as he unwittingly involves unsuspecting people in round after deadly round of Russian roulette.

Naci showed the film to healthy participants. To a separate group she showed a scrambled version involving rearranged one-second segments. This ‘control’ version was important because it contained many of the same features as the original; the same visual patterns, the same objects, the same actions. But it lacked the crucial narrative coherence – the knowledge of the bullet – which generated the suspense.

Using brain scanning, and the comparison of the two versions of the film, Naci and colleagues were able to show that the unscrambled, suspenseful version activated nearly every part of the cortex. Everything from primary sensory areas, to motor areas, to areas involved in memory and anticipation were engaged (as you might hope from a film from one of the masters of storytelling). The researchers were particularly interested in a network of activity that rose and fell in synchrony across ‘executive’ areas of the brain – those known to be involved in planning, anticipation, and integrating information from different sources. This network, they found, responded to the moments of highest suspense in the film; the moments when the boy was about to fire, for example. These were the moments you could only find so dramatic if you were following the plot.

Next the researchers showed the film to two patients in wakeful comas. In one, the auditory cortex became activated, but nothing beyond this primary sensory region. Their brain was responding to sounds, perhaps automatically, but there was no evidence of more complex processing. But in a second patient, who had been hospitalised and non-responsive for 16 years, his brain response matched those of the healthy controls who’d seen the film. Like them, activity across the cortex rose and fell with the action of the film, indicating an inner consciousness rich enough to follow the plot.

The astounding result should make us think carefully about how we treat such patients and marks an advance on the arsenal of techniques we can use to connect to the inner lives of non-responsive patients. It also shows how cognitive neuroscience can benefit from the use of more complex stimuli, such as movies, rather than the typically boring visual patterns and simple button-press responses that scientists usually use to probe the mysteries of the brain.

The genius of this research is that to test for the rich consciousness of the patient who appears unresponsive you need to use rich stimuli. The Hitchcock film was perfect because of its ability to create drama by what we believe and expect not because of what we merely see.

My BBC Future column from last week. The original is here. The original paper is: Naci, L., Cusack, R., Anello, M., Owen, A. M. A common neural code for similar conscious experiences in different individuals. PNAS. 2014;111(39):14277–82.

Images of ultra-thin models need your attention to make you feel bad

I have a guest post over at the BPS Research Digest, covering research on the psychological effects of pictures of ultra-thin fashion models.

A crucial question is whether the effect of these thin-ideal images is automatic. Does the comparison to the models, which is thought to be the key driver in their negative effects, happen without our intention, attention or both? Knowing the answer will tell us just how much power these images have, and also how best we might protect ourselves from them.

It’s a great study from the lab of Stephen Want (Ryerson University). For the full details of the research, head over: Images of ultra-thin models need your attention to make you feel bad

Update: Download the preprint of the paper, and the original data here

The reproducibility of psychological science

The Reproducibility Project results have just been published in Science, a massive, collaborative, ‘Open Science’ attempt to replicate 100 psychology experiments published in leading psychology journals. The results are sure to be widely debated – the biggest result being that many published results were not replicated. There’s an article in the New York Times about the study here: Many Psychology Findings Not as Strong as Claimed, Study Says

This is a landmark in meta-science : researchers collaborating to inspect how psychological science is carried out, how reliable it is, and what that means for how we should change what we do in the future. But, it is also an illustration of the process of Open Science. All the materials from the project, including the raw data and analysis code, can be downloaded from the OSF webpage. That means that if you have a question about the results, you can check it for yourself. So, by way of example, here’s a quick analysis I ran this morning: does the number of citations of a paper predict how large the effect size will be of a replication in the Reproducibility Project? Answer: not so much


That horizontal string of dots along the bottom is replications with close to zero-effect size, and high citations for the original paper (nearly all of which reported non-zero and statistically significant effects). Draw your own conclusions!

Link: Reproducibility OSF project page

Link: my code for making this graph (in python)

Intuitions about free will and the brain

Libet’s classifc experiment on the neuroscience of free will tells us more about our intuition than about our actual freedom

It is perhaps the most famous experiment in neuroscience. In 1983, Benjamin Libet sparked controversy with his demonstration that our sense of free will may be an illusion, a controversy that has only increased ever since.

Libet’s experiment has three vital components: a choice, a measure of brain activity and a clock.

The choice is to move either your left or right arm. In the original version of the experiment this is by flicking your wrist; in some versions of the experiment it is to raise your left or right finger. Libet’s participants were instructed to “let the urge [to move] appear on its own at any time without any pre-planning or concentration on when to act”. The precise time at which you move is recorded from the muscles of your arm.

The measure of brain activity is taken via electrodes on the scalp. When the electrodes are placed over the motor cortex (roughly along the middle of the head), a different electrical signal appears between right and left as you plan and execute a movement on either the left or right.

The clock is specially designed to allow participants to discern sub-second changes. This clock has a single dot, which travels around the face of the clock every 2.56 seconds. This means that by reporting position you are reporting time. If we assume you can report position accurately to 5 degree angle, that means you can use this clock to report time to within 36 milliseconds – that’s 36 thousandths of a second.

Putting these ingredients together, Libet took one extra vital measurement. He asked participants to report, using the clock, exactly the point when they made the decision to move.

Physiologists had known for decades that a fraction of a second before you actually move the electrical signals in your brain change. So it was in Libet’s experiment, a fraction of a second before participants moved, a reliable change could be recorded using the electrodes. But the explosive result was when participants reported deciding to move. This occurred in between the electric change in the brain and the actual movement. This means, as sure as cause follows effect, that the feeling of deciding couldn’t be a timely report of whatever was causing the movement. The electrode recording showed that the decision had – in some sense – already been made before the participants were aware of having taken action. The brain signals were changing before the subjective experience of taking a decision occurred.

Had participants’ brains already made the decision? Was the feeling of choosing just an illusion? Controversy has raged ever since. There is far more to the discussion about neuroscience and free will than this one experiment, but its simplicity has allowed it to capture the imagination of many who think our status as biological creatures limits our free will, as well as those who argue that free will survives the challenge of our minds being firmly grounded in our biological brains.

Part of the appeal of the Libet experiment is due to two pervasive intuitions we have about the mind. Without these intuitions the experiment doesn’t seem so surprising.

The first intuition is the feeling that our minds are a separate thing from our physical selves – a natural dualism that pushes us to believe that the mind is a pure, abstract place, free from biological constraints. A moment’s thought about the last time you were grumpy because you were hungry shatters this illusion, but I’d argue that it is still a persistent theme in our thinking. Why else would we be the least surprised that it is possible to find neural correlates of mental events? If we really believed, in our heart of hearts, that the mind is based in the brain, then we would know that every mental change must have a corresponding change in the brain.

The second pervasive intuition, which makes us surprised by the Libet experiment, is the belief that we know our own minds. This is the belief that our subjective experience of making decisions is an accurate report of how that decision is made. The mind is like a machine – as long as it runs right, we are happily ignorant of how it works. It is only when mistakes or contradictions arise that we’re drawn to look under the hood: Why didn’t I notice that exit? How could I forget that person’s name? Why does the feeling of deciding come after the brain changes associated with decision making?

There’s no reason to think that we are reliable reporters of every aspect of our minds. Psychology, in fact, gives us lots of examples of where we often get things wrong. The feeling of deciding in the Libet experiment may be a complete illusion – maybe the real decision really is made ‘by our brains’ somehow – or maybe it is just that the feeling of deciding is delayed from our actual deciding. Just because we erroneously report the timing of the decision, doesn’t mean we weren’t intimately involved in it, in whatever meaningful sense that can be.

More is written about the Libet experiment every year. It has spawned an academic industry investigating the neuroscience of free will. There are many criticisms and rebuttals, with debate raging about how and if the experiment is relevant to the freedom of our everyday choices. Even supporters of Libet have to admit that the situation used in the experiment may be too artificial to be a direct model of real everyday choices. But the basic experiment continues to inspire discussion and provoke new thoughts about the way our freedom is rooted in our brains. And that, I’d argue, is due to the way it helps us confront our intuitions about the way the mind works, and to see that things are more complex than we instinctively imagine.

This is my latest column for BBC Future. The original is here. You may also enjoy this recent post on mindhacks.com Critical strategies for free will experiments

Critical strategies for free will experiments

waveBenjamin Libet’s experiment on the neuroscience of free will needs little introduction. (If you do need an introduction, it’s the topic of my latest column for BBC Future). His reports that the subjective feeling of making a choice only come after the brain signals indicating a choice has been made are famous, and have produced controversy ever since they were published in the 1980s.

For a simple experiment, Libet’s paradigm admits to a large number of interpretations, which I think is an important lesson. Here are some common, and less common, critiques of the experiment:

The Disconnect Criticism

The choice required from Libet’s participants was trivial and inconsequential. Moreover, they were specifically told to make the choice without any reason (“let the urge [to move] appear on its own at any time without any pre-planning or concentration on when to act”). A common criticism is that this kind of choice has little to tell us about everyday choices which are considered, consequential or which are actively trying to involve ourselves in.

The timing criticism(s)

Dennett discusses how the original interpretation of the experiment assumes that the choosing self exists at a particular point and at particular time – so, for example, maybe in some central ‘Cartesian Theatre’ in which information from motor cortex and visual cortex come together, but crucially, does not have direct report of (say) the information about timing gathered by the visual cortex. Even in a freely choosing self, there will be timing delays as information on the clock time is ‘connected up’ with information on when the movement decision was made. These, Dennett argues, could produce the result Libet saw without indicating a fatal compromise for free choice.

My spin on this is that the Libet result shows, minimally, that we don’t accurately know the timing of our decisions, but inaccurate judgements about the timing of decisions doesn’t mean that we don’t actually make the decisions themselves that are consequential.

Spontaneous activity

Aaron Schurger and colleagues have a nice paper in which they argue that Libet’s results can be explained by variations in spontaneous activity before actions are taken. They argue that the movement system is constantly experiencing sub-threshold variation in activity, so that at any particular point in time you are more or less close to performing any particular act. Participants in the Libet paradigm, asked to make a spontaneous act, take advantage of this variability – effectively lowering their threshold for action and waiting until the covert fluctuations are large enough to trigger a movement. Importantly, this reading weakens the link between the ‘onset’ of movements and the delayed subjective experience of making a movement. If the movement is triggered by random fluctuations (observable in the rise of the electrode signal) then there isn’t a distinct ‘decision to act’ in the motor system, so we can’t say that the subjective decision to act reliably comes afterwards.

The ‘only deterministic on average’ criticism

The specific electrode signal which is used to time the decision to move in the brain is called the readiness potential (RP). Electrode readings are highly variable, so the onset of the RP is a statistical artefact, produced by averaging over many trials (40 in Libet’s case). This means we lose the ability to detect, trial-by-trial, the relation between the brain activity related to movement and the subjective experience. Libet reports this in his original paper [1] (‘only the average RP for the whole series could be meaningfully recorded’, p634). On occasion the subjective decision time (which Libet calls W) comes before the time of even the average RP, not after (p635: “instances in which individual W time preceded onset time of averaged RP numbered zero in 26 series [out of 36] – which means that 28% of series saw at least one instance of W occurring before the RP).

The experiment showed strong reliability, but not complete reliability (the difference is described by Libet as ‘generally’ occurring and as being ‘fairly consistent’, p636). What happened next to Libet’s result is a common trick of psychologists. A statistical pattern is discovered and then reality is described as if the pattern is the complete description: “The brain change occurs before the choice”.

Although such generalities are very useful, they are misleading if we forget that they are only averagely true, not always true. I don’t think Libet’s experiment would have the imaginative hold if the result was summarised as “The brain change usually occurs before the choice”.

A consistent, but not universal, pattern in the brain before a choice has the flavour of a prediction, rather than a compulsion. Sure, before we make a choice there are antecedents in the brain – it would be weird if there weren’t – but since these don’t have any necessary consequence for what we choose, so what?

To my mind the demonstration that you can use fMRI to reproduce the Libet effect but with brain signals changing up to 10 seconds before the movement (and an above chance accuracy at predicting the movement made), only reinforces this point. We all believe that the mind has something to do with the brain, so finding patterns in the brain at one point which predict actions in the mind at a later point isn’t surprising. The fMRI result, and perhaps Libet’s experiment, rely as much on our false intuition about dualism as conclusively demonstrating anything new about freewill.

Link: my column Why do we intuitively believe we have free will?