Deeper into forensic bias

For the recent Observer article on forensic science and the psychological biases that affect it, I spoke to cognitive scientist Itiel Dror about his work.

I could only include some brief quotes from a more in-depth exchange, so for those wanting more on the psychology of forensic examining, here’s Dror on how evidence can be skewed and why these effects have been ignored for so long.

What do you think has been the turning point for the forensic science community in terms of beginning to accept the role of cognitive bias in interpretation of evidence?

I think the clear cut scientific research with actual forensic examiners which was a within-subject experimental design, showing that the *same* expert, examining the *same* evidence, can reach different conclusions when they are affected by bias. The problem was also demonstrated in fingerprinting and DNA, very robust forensic domains.

I think you are very right to say that they have ‘began’. There has been a change, for example, the UK Forensic Regulator is now onboard. But there is still a way to go.

Which area of forensic science do you think is currently most susceptible to cognitive bias?

It will be the forensic science areas in which, as I like to say, the human examiner is the main instrument of analysis. These are most of the forensic domains: fingerprinting, DNA, CCTV images, firearms, shoe and tire marks, document examination, and so on. When there is no instrument that says ‘match’ or ‘no-match’ and it is in the ‘eye of the beholder’ to make the judgement, then subjectivity comes in, and is open to cognitive bias.

Essentially, forensic areas in which there are no objective criteria: where it is the forensic expert who compares visual patterns and determines if they are ‘sufficiently similar’ or ‘sufficiently consistent’. For example, whether two fingerprints were made by the same finger, whether two bullets were fired from the same gun, whether two signatures were made by the same person. Such determinations are governed by a variety of cognitive processes.

The cognitive nature of subjectivity is that it can be influenced and biased by extraneous contextual information. Forensic scientists work within a variety of such influences: from knowing the nature and details of the crime, to being indirectly pressurized by detectives, from seeing the ‘target’, to working within and as part of the police, from computer generated meta-data, to appearing in courts within an adversarial criminal justice system, and so on. The contextual influences are many and they come in many forms, some of which are subtle. So, many, most of the forensic areas are vulnerable.

It seems there is a reluctance to change procedures to minimise cognitive bias. Where does the resistance come from?

There are still forensic examiners who think that are immune to context and do not understand, let alone accept, the existence and danger of cognitive bias. They often confuse ‘bias’ (as in being racist, anti-Semitic etc) with cognitive bias; and this makes some of them think that it is an ethical issue. Forensic examiners rarely, if at all, receive training in this area and in the rare occasions that they do, they get bad training from people who do not specialise in providing training about cognitive bias in forensics.

The forensic community, as the military, police, and so on, are all very hard to change; there is a strong culture within those organisations. It is especially hard to promote change when errors are not as apparent as in other domains. If the police shoot an innocent person, then they very quickly know that they made a mistake, if a surgeon amputates the wrong leg, then they know very quickly that they made a mistake. In contrast, in the forensic domain, in real criminal cases, we do not know the ground truth, and do not really know if a mistake has happened or not. Only in very rare and special circumstances do errors surface (as in the Mayfield and McKie cases).

The courts have basically for the most part blindly accepted most of the forensic evidence. So, the examiners see no reason to change, if the courts accepts their evidence, then that is that. This may be changing. The hope is that judges will be more aware of the danger of cognitive bias and not accept forensic conclusions that are tainted with bias.
 

Link to further reading from Itiel Dror.