The fallibility of eyewitness memory

Eyewitness testimony is commonly regarded as very high quality evidence. But recent research has shown there are many ways memories of events can become contaminated. This article is based on a presentation to the NZ Skeptics conference in Wellington, 27 September 2009.

In 2003, a woman was tragically attacked and raped after leaving a bar in Christchurch. She remembered her assailant as a man with “rat- like” features. Later, she chose the police suspect from a photographic lineup, indicating that she was “90 percent sure” that he was her assailant. This identification became the central piece of evidence that convicted Aaron Farmer. But, in June 2007, Mr Farmer was exonerated after DNA proved that he could not have been the rapist – he had spent almost three years in prison.

Unfortunately, Mr Farmer’s case is not an isolated incident. Decades of legal and psychological research have shown that eyewitness identification error is the leading cause of wrongful conviction. Recently the former High Court judge, Sir Thomas Thorp, published an extensive review of legal research on miscarriages of justice. In that paper, he estimated that there are at least 20 innocent people in New Zealand prisons, and he emphasised eyewitness error as a leading cause of convictions. This conclusion fits neatly with exoneration data from the Innocence Project, based in New York. Since 1992, the Innocence Project has exonerated over 250 wrongfully convicted people, over 75 percent of whom were identified by at least one eyewitness.

How can human memory be so fragile as to lead a witness to choose an innocent person from a lineup? Over 30 years of research has shed light on this question. Ultimately, this research has shown that memory can go wrong in several ways. The best way to understand these errors is to think of memory as a three-stage process:
[1] encoding,
[2] retention, and
[3] recall.

At the encoding stage, information is perceived and transferred from the environment, through our senses. These perceptual processes allow us to lay down memory traces. Next, those traces are retained for a period of time. Of course this retention stage can last for anywhere between seconds and years, until finally we recall that information from memory. It is important to know that any one of these three stages can go awry.

Encoding

Encoding depends heavily on our ability to pay attention to information in the environment. However, our attentional systems are limited. We can only pay attention to a few things at once. Anything that does not receive the requisite amount of attention does not have the chance to make it through the encoding phase of memory.

Furthermore, many variables, such as stress, can limit our attentional processes even more. As a result, witnesses will often not pay attention to details that could be forensically relevant. For example, a witness under stress may pay particular attention to the weapon being brandished by the offender, rather than paying attention to his facial details. If this is the case, those facial details may never be stored in memory, and if information is not stored, it cannot be recalled later.

Retention

The information that makes it into memory can be distorted easily. Perhaps the best known psychological science research in this field is the misinformation effect pioneered by Elizabeth Loftus. This research shows that a simple suggestion can change witnesses’ memories. In a typical misinformation experiment, there are three stages.

First, participants watch a simulated crime, such as a man stealing a maths book from a bookstore. After a delay, participants are exposed to post- event information (PEI), which is usually a narrative describing the simulated crime. For some participants, the PEI is accurate but generic (eg, “the man stole a book”), and for others the details are misleading (eg, “the man stole a science book”).

Finally, participants are questioned to determine their memory’s accuracy for the event. These participants are often specifically told to ignore everything they read in the narrative and only rely on what they saw during the event. Typically, those participants who read misleading details during the PEI have less accurate memories than those who read generic information.

This research shows the ease with which a person’s memory can be changed. Decades of research have shown that people can come to remember having seen a crime when in fact they have seen an innocuous event. Using this paradigm people can even come to remember having seen an innocuous event, when in fact they have seen a crime. Witnesses can often be exposed to misleading details from co- witnesses, suggestive interviewing techniques or sometimes, media reports of the crime. Any of these sources can lead witnesses to remember details that did not happen.

Recall

Psychological science has also shown that the way we test witnesses can also affect their memories for what they have seen. Some of the most prolific research in this field has examined the way that we test witnesses’ memories for offenders’ faces using the lineup technique. Photographic lineups are the most common method of testing eyewitness recall for offenders.

Usually, a lineup depicts a police suspect surrounded by known innocent people – known as distracters. A witness chooses a person from a lineup in the same way that a person chooses an option from multiple- choice question. When people choose the correct answer from a multiple- choice question it is considered evidence that they recognised the correct answer by relying on memory; and when witnesses choose the suspect from a montage, it is considered evidence that they recognised the suspect from the crime scene.

However, people do not always rely on their memory in either multiple- choice questions or lineups. A multiple-choice question can be biased towards the correct answer, as in this example:

What is the capital of Burundi?

Most people cannot rely solely on their memory to answer this question. Now consider these choices:

(a) Paris;
(b) Sydney;
(c) Wellington;
(d) Bujumbura.

You probably chose the correct answer (d), not because you had a memory for Burundi’s capital, but because you used a process of elimination to choose that answer. Similarly, a lineup is sometimes constructed so that witnesses do not need to rely on their memory for the offender; instead, they use a process of elimination – the suspect becomes the Bujumbura of the lineup.

Lineup bias

The danger arises when the wrong person is suspected of a crime and then included in a biased lineup. Research shows that witnesses will often choose from a lineup, even when the actual offender is not present. If the lineup has been constructed in a biased way (like the multiple choice question above), witnesses are even more likely to choose from the lineup. It is misidentifications like these that often lead to wrongful convictions.

Taken together, this research shows that witnesses’ memories are susceptible to several sources of error. As such, we need to ensure that we collect and test witnesses’ memories with scientifically valid interview and lineup techniques. Scientific recommendations regarding best practice procedures for witness evidence have been available for several decades, but few jurisdictions worldwide have taken them up. This lack of recognition for scientific validation is surprising given the relatively fast uptake of forensic science methods, such as DNA testing.

As a result, the best way to think of witness memory evidence is like biological evidence at a crime scene. If we were unlucky enough to stumble across a bloody crime scene, most people would be careful not to contaminate the scene by trampling through the blood spatter patterns, or handling any evidence. Similarly, we should treat witness memory with the same caution. When a witness has been exposed to a crime, we should not contaminate their memories with suggestive questioning and biased lineups. Instead, we should collect and preserve their memories with scenically valid techniques. Only then can we hope to reduce the increasing number of wrongful convictions caused by erroneous witness evidence.

Legal Evidence

Skepticism is very much concerned with assessing the quality of evidence in support of a particular claim. But evidence means different things to different people. In the first of a two-part series, Jim Ring examines the legal profession’s view of the matter.

Our knowledge of the world is never certain, so what should count as evidence that a claim is true? For many people trained in science the legal system contains some very strange anomalies. Material can be accepted as evidence in court even though it would be given no weight at all in a scientific context.

People are sometimes imprisoned because juries are prepared to accept the unsupported word of witnesses. David Dougherty was in jail three years for raping a young neighbour. She said he had done so. No other evidence pointed to a conviction; tests on DNA were inconclusive. Later, better tests indicated that the girl had had sex with a man other than Dougherty.

Peter Ellis (the Christchurch crèche case) was jailed because some small children told the court he had done dreadful things. The same children told questioners that some women had done dreadful things, but they were not believed. They also told stories about Ellis that could not possibly be true and these were edited out of the story put to the jury.

In science experiments a control is a most important feature. Unfortunately controls are generally ruled inadmissible by judges. In the Ellis case there was an excellent control for the children’s credibility. How reliable were the children’s stories about the women, and how reliable were their stories about Satanism and black magic?

Eyewitness accounts are given great weight in our justice system; yet in the US it has been estimated that mistaken eyewitness evidence is responsible for about 80 per cent of the wrongful convictions that occur. The trial of Scott Watson (Marlborough Sounds New Year murders) was unusual in that eyewitness evidence was treated in a different manner. The prosecution held that part of the evidence of one of their own witnesses was in error. He had a distinct memory of a two-masted yacht; the prosecution argued that he must have seen one with a single mast. In future will eyewitness evidence be treated as less conclusive?

In New Zealand a cell-mate can claim to have received a confession of guilt from a person awaiting trial. That such an anecdote can be accepted as evidence in court would be beyond belief – except it happens!

It was reported last year in the UK that a judge instructed a jury not to bother about statistics and just to use their common sense (let us hope this report was inaccurate). But common sense is not a reliable guide in complex matters. Science and statistics often produce results that are quite counter-intuitive.

A conviction is only supposed to take place when evidence is beyond all reasonable doubt. In New Zealand people are sometimes convicted in spite of reasonable doubt.

Forum

Where Were the Hunters?

The account of the meeting between the Moa hunters and the Christchurch Skeptics was interesting, but contained some very odd statements. How many skeptics had done any hunting, I wonder? The account reads as though there were no experienced hunters present who could challenge some of the statements made. That is rather like examining key-benders without a magician present. However, the account, like many UFO sightings, contains several inconsistencies which are not obvious to the inexperienced.

I have shot many hundred deer, plus pigs, goats, chamois, wild cattle and sheep so I count as an experienced hunter.

While hunting, one sees difficult-to-identify objects all the time. It is very hard to spot animals unless they move, even when cover is light. Any bush, rock or shadow that is approximately the right colour needs to be scanned. Many a deer or pig turns out to be something inanimate when examined through binoculars or telescope. It will perhaps astound the inexperienced to be told that this applies even to objects which are very close.

Sometimes one could swear the thing moved.

One of the odd things about the story is that there is no mention of binoculars or telescope; were they not used?

“His rifle didn’t even go near his shoulder”. This implies no telescope and suggests he is not a very serious hunter. My rifle, on a hunting trip, would be constantly at my shoulder — not to shoot but to carefully examine objects through the ‘scope. It is essential not to shoot until one is certain of the identification.

“The beast was unmistakably not a deer.” That is simply the voice of inexperience. If the party had no binoculars and no telescope then their story cannot be taken seriously however close the object was. Anyone with moderate experience in looking at wildlife should know that the human eye without the aid of magnification is incapable of such assessment.

To my horror, I once found I had shot a goat, not a deer. I had wrongly identified a very close goat with a distant deer in excellent light! The bullet had struck high of course. But I was certain of my target; I was just wrong. The diagnosis was simply that I needed to start wearing my glasses and have done so ever since.

In a long career of shooting there will inevitably be a number of targets which were allowed to escape because positive identification was impossible. I can think of several — I am still not sure what they were. But I never thought they were extinct birds. I suggest it takes a particular mindset to make such an identification.

If this is the first Unidentified Running Object the party had seen then they simply lack experience, however many years they claim.

Jim Ring, Nelson

Missing Address

In the December Skeptic you have a note by B. Premanand asking New Zealand skeptics to help their Indian counterparts by subscribing to Indian Skeptic. However, he does not give an address. Do you have one?

Gordon Hewitt

Apologies for omitting the address. It is:

B. Premanand
Convenor, Indian CSICOP
10 Chettipalayam Road
Podanur 641-023
Tamilnada
India

Skepticism and Miracles

This article is an abridged version of the fourth article in a series on philosophy and the paranormal. Here Dr Grey discusses David Hume’s analysis of miracles and his view that belief in miraculous events is always unjustified. He also investigates the nature, virtues and dangers of different skeptical viewpoints.

Hume’s Razor

What is a miracle? In the vernacular we speak of “miraculous” escapes and the like, to characterise events which are extremely unlikely — at odds with the normal course of experience. A miracle in this weak sense just means a very improbable event.

David Hume, in his famous essay “On Miracles”, had a stronger sense of “miracle” in mind, namely something which violates a law of nature. It is in this sense that miracles have commonly featured in religious systems of belief, as the means by which God has been thought to have demonstrated His presence or His power to His chosen people.

The question which Hume addresses is: are we justified in believing that miracles have in fact occurred? He argues for the very strong conclusion that we are never justified in believing that a miracle has ever occurred.

Hume is not claiming to show that miracles have never occurred. Proving negative existence claims is notoriously problematic. Hume’s claim is the importantly different one that we are never rationally justified in believing that miracles have occurred. That is, Hume is addressing the epistemological issue of what it is rational to believe, rather than the metaphysical question of what is and is not possible in our sort of world.

The argument has two parts. First, Hume argues that the evidence against miracles is usually very strong. (And according to one of Hume’s epistemological maxims “a wise man proportions his belief to the evidence.”) We therefore have to weigh the evidence that a miraculous event happened against the evidence that it did not.

In evaluating testimony for miracles, Hume advances the following principle which, echoing the famous methodological principle commonly attributed to William of Ockham (c. 1285-1349), has been called Hume’s Razor. The principle is:

No testimony is sufficient to establish a miracle unless that testimony be of such a kind that its falsehood would be more miraculous than the fact which it endeavours to establish.

The second stage of the argument claims that even though the evidence in favour of miracles might outweigh the evidence against them, in practice this never happens. Hume maintains that there never was a miraculous event established on sufficiently strong grounds to warrant rational belief in its occurrence. There are four factors which undermine the credibility of any claim in support of the miraculous.

First there is the problem of witness credibility. Witnesses who testify as to the occurrence of miracles are seldom totally above suspicion of either having been deceived or of the intent to deceive. In evaluating their testimony, we must always choose between believing that a miracle occurred or believing that the witnesses were deceived or deceitful.

According to Hume, no miracle has ever been “attested by a sufficient number of men, of such unquestioned good sense, education and learning as to secure us against all delusion in themselves; of such undoubted integrity as to place them beyond all suspicion of any design to deceive others…” All claims for miracles, that is, suffer from what we call a credibility gap.

The second problem which Hume identifies is human credulity. There is a natural human affinity for the novel, the surprising and the marvelous. Recognising this propensity for credulity, we must take note of and be guided by the following maxims in evaluating claims for the miraculous:

  • Objects of which we have no experience resemble those of which we do have experience;
  • What we have found to be most usual is most probable;
  • In case of dispute, give preference to the side favoured by the greatest number of observations.

These sound, if somewhat pedestrian, maxims are frequently overridden by the excitement and novelty which is often the basis of human credulity.

Hume’s third point about miraculous claims is the tribal origins of superstition. Miracles occur mainly in primitive (in Hume’s words “ignorant and barbarous”) nations, or are derived from barbarous and ignorant ancestors. As human understanding develops, we come to reject omens, oracles, astrology, demons, and the like as unhelpful in explaining natural phenomena.

Finally, Hume points to a problem which confronts claims of the miraculous concerning conflicts of testimony. It is impossible that the religious traditions of “ancient Rome, of Turkey, of Siam, and of China” should all of them be established on any solid foundation. Every miracle claimed within any one of these traditions is intended to establish the truth of that tradition and to discredit the claims of the others.

Hume believed that the inclination of mankind towards superstition and the marvelous may receive some check from sense and learning, but he also seems to have believed that it could never be thoroughly extirpated from human nature. Hume, one suspects, would not be surprised to find beliefs in astrology, UFOs, crystals, channeling, and similar credulous ideas in the twentieth century.

The Virtues of Skepticism

Skepticism can be characterised as a critical stance with regard to knowledge (or existence) claims, and a skeptic is one who calls such claims into question. Skepticism can refer to either the critical stance adopted in subjecting knowledge claims to careful scrutiny, or to a state of doubt or disbelief which may be the outcome of such an inquiry.

Being skeptical in the second sense (withholding assent, or suspending belief in a particular claim) need not involve believing the opposite. Skepticism is a matter of doubt rather than denial. If I withhold assent from the claim that God exists, it need not be the case that I believe that God does not exist. I might believe that the evidence is just not strong enough to settle the matter either way.

We need to distinguish between critical and dogmatic skepticism, and between selective and global skepticism, though these terms do not mark absolute distinctions.

Skepticism is dogmatic if assent is withheld a priori, that is, on the basis of prior conviction without considering the evidence. For example, Galileo’s colleagues expressed a perfectly intelligible skepticism about the existence of the moons of Jupiter; they became dogmatic when they refused to look through his telescope.

Skepticism is global if it is general and encompasses all claims to knowledge; selective if it is targeted to specific knowledge claims.

Global skepticism is rare. Perhaps Cratylus, an older contemporary of Plato (c. 428-348 BC), is the most global skeptic recorded in the annals of philosophy. His skepticism is said to have been so extreme that he refused to answer questions and would only wave a weary finger at his interlocutor to indicate that truth was so elusive and ephemeral that it would be useless to reply. (At least, that is what his interlocutors are reputed to have believed him to have been attempting to say).

Skepticism which falls short of the global always has to be qualified by specifying the subject matter to which it is directed. There are various beliefs, for example about rocks and tables, which are immune to skeptical doubts — at least outside philosophy seminar rooms.

At the other extreme there are tooth fairies, Santa Claus and the elixir of life, which most will immediately dismiss as lacking sufficient epistemic warrant. In between (drawing boundaries here will create controversy) there are various disputed cases and claims, such as God, economic rationalism, J-curves, nuclear deterrence and psychic and paranormal phenomena. Disputed cases are also the coal-face of the professional philosopher: on which side of the dividing line, for example, should we put minds, beliefs, desires, meanings, properties, and numbers?

Strange Claims

When confronted with a claim about some strange, paranormal or similarly anomalous phenomena (an accurate premonition, a “near death” or an “out of body” experience, say), we should adopt a scrupulously skeptical approach. By this I do not mean that we must conclude that the experience did not occur, or that whoever had the experience must have been somehow deluded — though do not rule that out either! Rather, we should be alert to the possibility of natural and ordinary explanations of unnatural and extraordinary occurrences.

We must be especially careful in evaluating the evidence which appears to support such anomalous events. While rejecting a dogmatic skepticism which refuses to countenance anomalous events, critical skepticism seeks to gather as much evidence as possible concerning any extraordinary or allegedly paranormal event, claim or theory.

Critical skepticism means keeping an open mind not rejecting disputed claims a priori. We must examine the evidence scrupulously. But it means refusing to accept as true claims for which there is insufficient or ambiguous evidence, and recognising that withholding belief is preferable to accepting claims for which there are not sufficient grounds.

It also means adopting as a methodological maxim the principle that in seeking explanations we should prefer the ordinary to the extraordinary, and the simple to the complex. This is one interpretation of the principle known as “Ockham’s Razor”.

Skepticism is the disposition, or art, of matching belief to evidence. There is at present no convenient antonym for “skeptic”. For convenience, I propose to revive the archaic expression “credulist” to serve this role. A credulist can be understood as someone who is apt to accept claims without sufficient evidence, that is to say, someone whose epistemic standards are too low.

Why be Skeptical?

Etymologically, skeptikos means “inquirer”, and the value of skepticism is that it leads to — and when seriously entertained is usually the result of — a systematic inquiry into the foundations of knowledge. Skeptical arguments play a central role in inquiry, particularly philosophical inquiry where they have been directed not just at eccentric belief systems, but at beliefs which most regard as self-evident. Thus skeptical arguments have been raised about the existence of other minds, knowledge of the past, knowledge of material objects (the “external world”), moral truths, sensory knowledge and even about knowledge of logic and mathematics.

The purpose of skepticism in these cases is (usually) not to raise extravagant doubts (though that is sometimes an unintended consequence), but rather to clarify our understanding of the subject of inquiry. There are apparently more modest forms of skepticism which challenge, for example, theological or metaphysical knowledge. And in some cases (for example, I suggest with respect to astrology or phrenology) skepticism seems to be not merely defensible, but appropriate.

When pressed to extremes, skeptical arguments sometimes turn out to be self-defeating, and the critical pursuit of knowledge leads to a denial that knowledge is attainable. Extreme or global skepticism has often been the consequence of setting unreasonably high standards as to what is acceptable as knowledge — in particular setting absolute certainty (the impossibility of being mistaken) as a requirement for genuine knowledge.

Objections to Skepticism

Routine rejoinders to skeptics are, first, the claim that skeptics demand unrealistically high standards of proof (often accompanied by the charge that the skeptics’ insistence on these standards is unjustified), and, second, an insistence (sometimes dogmatic) that a disputed category of experience (a psychic experience, say) is more certain than any skeptical argument which calls them into question.

Regarding the first point, the insistence of protocols, controls, and repeatability are based on the beliefs that nature is consistent — and human nature often suspect. No one demands 100 per cent repeatability. There are always anomalous observations due to the quirks of experimenters or their apparatus. (Indeed with complex scientific experiments it is a formidable task to get anything to work at all.)

But for any extraordinary claim to gain respectability, it has to be replicable by someone somewhere. A recent example of the failure to meet this requirement was the discrediting of the exciting empirical claim by Pons and Fleischmann about so-called cold fusion.

The problem with psi phenomena is not that it is difficult for careful researchers to get it to work occasionally under rigorously controlled conditions; it is difficult for careful observers to get anything at all that can’t be dismissed as noise, error, wishful thinking, chance and often, sadly, fraud. It is for this reason that the requirements of controlled experiments and repeatability cannot be dismissed as unduly fussy: experience shows that nature does not cheat and that people sometimes do.

There are number of manipulative techniques such as “cold reading”, which are well known to psychics (and magicians), which are used to fool people into believing that there are special psychic powers.

Skepticism should not be confused with cynicism, though it frequently is. A cynic is someone who is inclined to believe the worst about people. Cynicism is however a form of skepticism: it is skepticism about the sincerity or goodness of human motives and actions. Historically, Cynics were a sect founded by Antisthenes, a pupil of Socrates, who condemned wealth and the enjoyments of life. The most extreme and celebrated exponent of the movement was Diogenes.

In the history of philosophy we find skeptical arguments are repeatedly proposed and then attacked and “refuted”. Skepticism is a continuing challenge to dogmatic claims, and helps to maintain the spirit of free inquiry. Without skepticism we would be in danger of failing to distinguish enthusiasm, prejudice and superstition from serious, rational, and well-grounded beliefs, which is essential to the task of making sense of the world.

Perhaps the main danger for skeptics is that they sometimes have difficulty in distinguishing hard and soft data, that they set their standards of epistemic acceptability at too high a level (in contrast to credulists, whose epistemic standards are too low) and may tend to promote their own form of dogmatic conservatism.

The aim of skepticism is to combat doctrinal rigidities which can afflict almost any belief system, but skeptics must remain alert to the possibility of falling prey to rigidities of their own. Skepticism is all about matching belief to evidence. It is a difficult and continuing challenge to maintain the right proportion of skepticism in our inquiries. Only then are we able to steer between the Scylla of dogmatism and the Charybdis of credulism.