Rats, Scientists and Experimental Design

John Riddell learns about some dangerous chemicals

From the TV3 News website:

CHIPS COULD CAUSE CANCER UPDATED: 05:56PM WEDNESDAY 26 JUNE

The World Health Organisation has begun a three-day emergency meeting in Geneva to evaluate the danger of some popular foods. They’re concerned about a recent discovery that certain starchy foods, from deep fried chips to potato chips and bread, contain a chemical called acrylamide. It can cause cancer in rats, but there is no evidence it does the same in people.

There are lots of chemicals in food. Even organic food. Are the chemicals in our food dangerous? Is food safe?

For example many foods contain a chemical called cellulose. How much do we really know about cellulose?

If I want to know if cellulose is dangerous I need to design an experiment. Experimental design can be tricky.

I have limited resources. The best thing to do would be to look at a large sample of individuals. See what happens when they are exposed to small amounts of cellulose over a long period of time. Unfortunately I don’t have the resources to study this way. So I choose to use a high dose of cellulose over a short time period.

I get a rat and put it in a cage. I place the cage about five metres from the base of a 25 metre tall tree. Using a chainsaw, I chop down the tree so that it falls on to the cage containing the rat.

Then I have to repeat the experiment a few times to make sure the first result wasn’t a fluke.

Every single time I have carried out this experiment the rat has died. The conclusion is obvious. Cellulose is dangerous.

Next I must publish the results of this experiment in a “peer reviewed journal”.

This is in the unlikely event I have made a mistake somewhere. If there is something wrong with my experiment, hopefully someone will spot it.

Somebody did. They raised the point that it might not have been the cellulose that killed the rat. Trees also contain other chemicals such as lignin or even water. Maybe it was one of the other chemicals?

Good point. I have to consider the criticism and design a new experiment to answer this question.

Since a tree is mostly water I design another experiment to test if it could have been the water that killed the rat. I put a new rat, in a cage, into 100 litres of water.

And the rat dies.

So I conclude that water is harmful to rats. But this does not necessarily mean that cellulose is not also harmful.

After all, I noticed that while the tree killed the rat instantly, the water took a few minutes. Perhaps there was something in the combination of cellulose and water that made the tree more harmful than merely water alone.

But before I look at the synergistic effects I want to extrapolate these data to look at how dangerous small doses of water might be.

If 100 litres of water causes death in three minutes this suggests 50 litres would cause death in six minutes. Continuing this extrapolation shows that even a small amount of water will significantly reduce the rat’s life expectancy.

Also, what are the implications for humans?

Anecdotal evidence suggests that many humans who have been in large amounts of water have also died. As yet I haven’t yet been able to obtain funding to test this myself. However if water really is dangerous, can we risk even the smallest exposure?

Okay, some scientists have raised doubts about the validity of high dose trials. Just because a high dose of a chemical is harmful, they say this doesn’t necessarily mean that smaller doses are also harmful. And just because it is harmful to rats doesn’t necessarily mean it is harmful to humans.

But can we afford to take the risk? Do we really know the dangers? The results so far are inconclusive. Obviously more research is needed. I just need a bit more funding. However there is one conclusion we can draw.
Scientists don’t like rats, but I suspect the feeling is mutual.
No rats were harmed in the preparation of this article.

That Eureka Moment: How Science Works

This article is drawn from interviews with Allan Coukell on the NZ National Radio science programme “Eureka!” in 2001.

We live in an era where science is universally needed but rarely appreciated, little understood and much misunderstood. This is not just a problem of the wider non-scientific community; science is increasingly specialised and even prestigious scientists may have little awareness of areas of science outside their specialised research niche. Science is typically learned by studying and working in a particular discipline, but often such narrow perspectives don’t allow us to reflect on wider issues about science and appreciate its strengths and weaknesses. Furthermore, the specialized, abstract, nature of much science education all too often alienates many of its victims, while leaving the survivors blind to the limits and problems of their craft. There is irony in the way that science, the ultimate questioning activity, frequently fosters such unquestioning supporters and critics.

What is science?

Obviously science comes in many shapes and sizes and any attempt to provide a “one size fits all” description is bound to fail. Some scientists are engaged in an open-ended exploration of natural phenomena; some spend their lives developing and testing theories or models. Yet more scientists try to find out whether some theoretical entities like quarks are “real”, whilst others are trying to measure properties of the world with greater and greater precision. What, if any, are the unifying features of such a diverse discipline?

Given that science is such a multifarious thing, is it even sensible to ask a question such as “what is science?” Richard Feynman was a brilliant scientist who thought it was. Feynman, winner of the Nobel Prize for Physics in 1965, was not only one of the most brilliant scientists and science teachers of the 20th Century, he also reflected on the nature of science and communicated his perspectives vividly to a wide audience. Here’s how he addresses the question “What is science?”1:

The word is usually used to mean one of three things, or a mixture of them. … Science means, sometimes, a special method of finding things out. Sometimes it means the body of knowledge arising from the things found out. It may also mean the new things you can do when you have found something out, or the actual doing of new things … so the popular definition of science is partly technology too.”

Science and technology are inextricably linked in the public’s eye; it is technology that provides the gadgets to which society becomes addicted. The reliable and informative nature of scientific knowledge underpins modern technology, but science is not simply a means to technology. As Feynman points out, it is crucial to realise that science is an intellectual adventure, a cultural activity that should be undertaken for its own sake:

“The things that have been found out [are] the gold. This is the … pay you get for all the disciplined thinking and the hard work. The work is not done for the sake of an application. It is done for the excitement of what is found out. You cannot understand science and its relation to anything else unless you understand and appreciate the great adventure of our time.”

Science is an adventure. It involves asking questions about the universe, coming up with theories about the way nature works, and testing those theories to see how valid they are. As any scientist knows, it is a challenging activity:

“Trying to understand the way nature works involves a most terrible test of human reasoning ability. It involves subtle trickery, beautiful tightropes of logic on which one has to walk in order not to make a mistake in predicting what will happen.”

Given the complexities of undertaking a scientific investigation, what is it about science that makes it such a powerful way of finding things out about the world? This is Feynman’s view:

“[S]cience as a method of finding out … is based on the principle that observation is the judge of whether something is so or not. All other aspects and char-acteristics of science can be understood directly when we understand that observation is the ultimate and final judge of an idea. But ‘prove’ used in this way really means ‘test’ … the idea should really be translated as ‘The exception tests the rule.’ Or, put another way, ‘The exception proves that the rule is wrong.’ That is the principle of science. If there is an exception to any rule, and if it can be proved by observation, that rule is wrong.”

Fireworks at NASA

Given the variety and complexity of science, scrutinising illustrative episodes of science in action is a good way to understand more about science. Again Feynman provides a lead: he not only discussed science, he exemplified the whole philosophy of questioning the world and testing scientific ideas. Early in 1986, when he was fighting terminal cancer, Feynman was once again thrust into the public eye when he performed one of the most public demonstrations of science during the inquiry into the tragic accident of the space shuttle “Challenger”. Feynman’s role in this investigation provides an illuminating vignette into science.

On January 28th 1986 the space shuttle Challenger was launched, and almost immediately exploded in a horrific fireball. It is salutary to sometimes reflect on the fallibility of science and the icons of technological sophistication. Yet science rose, phoenix-like, from the ashes, due almost exclusively to Feynman’s scientific acumen.2 Within a week of the accident, on February 4th, Feynman was appointed to a committee of inquiry. He immediately began quizzing the engineers at the Jet Propulsion Laboratory where much of the space shuttle technology was developed. On the first day he learned of well-known problems with the shuttle, including cracks in the turbine blades. Feynman also learned of problems with the O-rings – glorified rubber bands thinner than a pencil and more than 10 m long – that sealed the joins between sections of the solid-fuel rockets. A pair of O-rings had to expand to prevent the leak of hot gases during the burning of the solid fuel rockets. However, on some launches one of the O-rings was being scorched. Feynman jotted down some notes: “Once a small hole burn thru generates a large hole very fast! Few seconds catastrophic failure.”

Feynman flew to Washington and quizzed NASA officials, especially about the effects of the unusually cold weather at the launch of the Challenger shuttle. Because the elasticity of the O-rings would decrease at low temperatures, the problems with the O-rings would be exacerbated. Over the weekend Feynman was hot on the O-ring trail and, when the committee reconvened on Monday 12th February, he was frustrated by the inconclusive and evasive testimony of Lawrence Mulloy, the project manager for the solid fuel rockets. That night at dinner, his eyes fell on a glass of iced water and he saw a way to test whether 0oC (the temperature of the Challenger launch) would affect the resilience of the O-rings.

The next day he bought a small C-clamp and pliers. At the hearing Feynman asked for iced water and then broke off a bit of O-ring material as a sample was passed round. He clamped the O-ring material in the C-clamp, then after a short break in the proceedings, Feynman asked to speak to Mulloy. The rest has become a historic exchange captured by the TV cameras:

“I took this stuff that I got out of your seal and I put it in ice-water, and I discovered that when you put some pressure on it for a while and then undo it, it doesn’t stretch back … In other words, for a few seconds at least … there is no resilience in this particular material when it is at a temperature of 32 degrees [Fahrenheit]. I believe that has some significance for our problem.”

Here, in a nutshell, was the heart of the scientific problem. As another great theoretical physicist, Freeman Dyson, commented:

“The public saw with their own eyes how science is done, how a great scientist thinks with his hands, how nature gives a clear answer when a scientist asks her a clear question.”

Stages of the scientific process illuminated by Feynman’s experiment:

  1. Science starts with a problem: in this case the question of what caused the Challenger to explode.
  2. There is background detective work: finding out what is already known (in most scientific investigations this involves extensive literature work; here it involved garnering various streams of evidence such as the previous O-ring problems, the observed puff of smoke from the solid booster after 0.5s of flight, the lower resilience of rubbers at lower temperatures etc).
  3. Hypothesis formulation: that a momentary loss of resilience of the O-ring allowed hot gases to burn through the seal and caused the rocket to leak.
  4. Hypothesis testing via experiment, or observation: testing that the elasticity of the O-ring material was indeed compromised under the conditions of the launch.
  5. Bringing the results into the public arena for critical scrutiny: committee hearings are an unusual forum for discussion; for most research investigations the academic literature is where scientific claims are subject to critical scrutiny.

What does the Challenger inquiry tell us about science?

Without Feynman’s input, the committee of inquiry was likely to have been a whitewash. Most of the establishment would have liked to rubber stamp the worthiness of the shuttle programme. Scientists have to be careful about not falling into the trap of defending the work of a programme they believe in, rather than subjecting it to full critical scrutiny. Feynman, as the consummate scientist, shows that science is not about confirming your prejudices or defending your patch, it is about uncovering truths about the world.

Feynman’s beautiful experiment did not absolutely prove that problems with the O-rings caused the Challenger disaster. However, together with the history of problems with the shuttle and the particular climatic conditions for the launch, the case was proved “beyond reasonable doubt.” Scientific knowledge bears more than a passing resemblance to court proceedings: the more direct the experimental evidence, and the greater the accumulated weight of diverse lines of evidence, the more clear-cut scientific knowledge becomes.

Science is not a method of generating infallible truths about the world; only tyrants claim to do that. Neither is it simply a way of producing just another opinion about the world – no better or worse than any other (as many postmodern social scientists would have us believe). While science does not dispense absolute truths, it does produce the best knowledge we have in areas where we can subject our theories to rigorous tests. Although the theories that survive such tests can never be proved to be true, they are likely to be close to the truth if they survive detailed scientific scrutiny without being proved wrong.

Furthermore, in areas where theories have been well tested and flaws of the theory are exposed, it is often the case that the theory is not thrown out wholesale – instead the previous theory is often found to be a limiting situation for the theory that succeeds it. We are more confident in the predictions of Newtonian mechanics in the wake of Einsteinian mechanics than we were before, since we now clearly understand where it does and does not apply. Similarly we have not dispensed with atomic theory now we know that atoms are comprised of smaller entities.

So the heart of science is criticism, the use of observations and experiments to test our theories and always being able to accept that we might be wrong. The ability to modify our views, in the face of evidence, is a keystone of science.

Perhaps the last word should go to Feynman3:

Science is a way to teach how something gets to be known, what is not known, to what extent things are known (for nothing is known absolutely), how to handle doubt and uncertainty, what the rules of evidence are, how to think about things so that judgments can be made, how to distinguish truth from fraud and from show.”

References

1 The quotations in this section are from Chapter 1 of The Meaning of it All, R. P. Feynman which is drawn from a public lecture that Feynman gave in April 1963.
[Back to text]

2 For a slightly more detailed chronology of Feynman’s participation in the Challenger investigation see Genius, by J. Gleick, pp414-428.
[Back to text]

3 Quoted in Genius, by J. Gleick, p285.
[Back to text]

This article originally appeared in Chem NZ No. 86.

Medical Evidence

In the second of a two-part series, Jim Ring looks at what evidence means to different people

Scientific evidence is often difficult to interpret, in medicine in particular. ‘An Unfortunate Experiment’ was the title given to the treatment for some women after screening for cervical cancer. In this case science was considered by the legal profession and apparently found wanting. The doctor involved was castigated and publicly humiliated for experimenting on humans. But no real experiments were ever done; it appeared he did not understand scientific methodology. Neither did the journalists and legal people involved. The point is that no proper controls were used so it was very poor science.

Were the women disadvantaged? It is difficult to tell, but many were certainly outraged. It generally escaped notice that the surgeon was responding to public pressure for less radical surgery and that a group of patients involved seem to have had on average a slightly better outcome than the norm.

One of the most unfortunate ideas that came out of the long legal case was the emphasis on privacy for the individuals involved which implied their records should not be available for medical study. There is a difference between privacy and anonymity. It is very important to explain to those involved in medical procedures that for medicine to progress it is essential to collect data. Women appeared on TV complaining bitterly that they had been used in an experiment without their consent. But all good medicine is experimental.

We are not much closer to determining whether mass screening for cervical cancer does improve the chances for the screened population and now we have another scandal in New Zealand. Public expectation of screening programs is far in excess of what they can deliver. Efforts to sue Dr Bottrill, and compensation claims from ACC, seem to imply that patients think a false negative reading is necessarily medical error. Women have appeared on TV claiming their lives have been devastated because they had a false negative. Surely this is wrong; they are rightly upset but this is because further tests show a medical problem. Of course some who died might have been saved if an early intervention had resulted from a correct positive reading; however this does not seem to be the main thrust of their complaint.

False Negatives vs False Positives

It is possible to reduce the number of false negative readings at the expense of an increase in the number of false positives. This may seem desirable but there is a cost. In Britain large numbers of women in a screening project reacted very badly to finding they might have a ‘pre-malignant’ condition. This included some members of the medical profession. There is a clear indication that patients were not well informed before screening.

Patients involved in any medical procedure are supposedly asked for their ‘informed consent’. It seems now obvious that ‘informed consent’ is largely lacking during mass screening for both cervical and breast cancer. Several of those involved in the public hearing are surprised to find that screening is less than 100 per cent accurate. All mass screening procedures are likely to have a high error rate as they are designed to be rapid, cheap and simple; leading to more precise testing if there is a positive result. Is a large and expensive inquiry, using legal methods, a suitable way of investigating scientific questions?

Cervical cancer, unlike breast cancer, is strongly correlated with environmental factors. The former is very rare in the general population with a relatively high incidence in a certain sector. However it is politically incorrect to target the high-risk population for screening because the risk correlation is with such factors as poverty, poor hygiene and sexual promiscuity.

A recent case of a gynaecologist accused of misconduct raises some interesting issues. The unfortunate patient would seem to be outside the high-risk group for cervical cancer, thus an assumption may have been made that the correct diagnosis was very unlikely. But no physical vaginal examination was made. Feminist literature once strongly criticised the medical profession for over-use of this procedure, which one writer described as ‘legalised rape’. It would be interesting to know the rate at which this procedure is used today compared with, say, 30 years ago. Is the medical profession responding to crusades in a way that disadvantages patients?

Objections to trials

Medical ethicists – now a profession – have objected to various drug trials saying it is unethical to provide some patients with a placebo that will not improve their condition. This is in effect a claim to certain knowledge – that the drug being trialed is the ideal treatment. Patients receiving a placebo are not disadvantaged when the new drug may do more harm than good. We can sympathise with terminally ill patients who know that they will die in the absence of treatment and where anything seems a better bet than a placebo. But it is essential that drugs be properly tested before being used routinely.

Experiments have even been done in surgery. In 1959 patients were randomly assigned, but all prepared for surgery and the chest cavity opened. Only then did the surgeon open an envelope and follow the instruction; either to perform the procedure or immediately close the chest. Although some ethicists have objected (one stated that such surgery would never take place in the UK), a double-blind study of brain surgery was recently done in the US. Not only did it pass an ethics committee but patients welcomed the chance to take part even though it involved drilling the skulls of both real and placebo patients. In this case there was considerable improvement in those under 60 who had the real operation.

This indicates people may be willing to give consent to risky experiments providing they are given good information.

Most evidence in medicine comes not from experiments but from epidemiology. This requires the collection of huge amounts of data and sometimes produces conflicting results. Two populations, which differ only in the factor under investigation, should be matched and this is difficult to achieve. Recently, in a world-wide study, doubt has been cast on the efficacy of breast-cancer screening. New analysis purports to show that when populations are matched correctly, the screened population has no better chance of survival than an unscreened population.

Demands for safety

Some demand that all medical procedures should be ‘safe’, though curiously this is not required of alternative medicine. Suppose a new drug has fatal consequences for one patient in 100,000. It is quite likely that this will not be discovered during testing. Should such a tiny risk preclude the use of a drug that gives significant benefits to the vast majority of patients? New medicines are introduced when they show a clear advantage over a placebo. When very large numbers are involved in a study it is possible for a drug to show a significant advantage, yet not be worth introducing. Significance is a technical term and it is possible to find an advantage of only 0.1% is ‘significant’, though it may not be worth taking such a product.

It was this confusion that bedevilled early experiments on ESP. Rhine in America and Soal in England recorded the success of subjects guessing unseen cards. The experimenters wrongly assumed controls were unnecessary; instead they compared guesses with a theoretical chance result. A few subjects scored correct guesses at slightly more than chance and because huge numbers of guesses were involved, statistical tests showed these results had ‘significance’. That is, there was a huge probability that the guesses were not simply ‘lucky’.

Enthusiasts then made the enormous leap to say that because the guesses were not due to chance they must be due to a previously undiscovered human faculty, extra-sensory perception or ESP. Disinterested observers, not just skeptics, should have concluded that other explanations, such as poor experimental design, badly recorded results, fatigue, or just plain cheating were more likely. A great deal of time, money and effort was spent pursuing this will-o’-the-wisp.

Whisky Galore

In which John Riddell conducts an entirely unscientific experiment and saves himself quite a bit of money

The first weekend of May each year is the opening of the New Zealand duck shooting season. It is a time when rednecks and yokels gather to harvest the surplus of the duck population. Like their cave dwelling ancestors before them, the males of the tribe gather on the night before the Great Hunt and tell tales of previous years. The youngsters listen in awe to the lies of their elders. There is enough male bonding to excite even the most boring anthropologist.

Being both a redneck and a yokel, I was invited by my mate Tony to one such event. On the Friday night we met up at an abandoned house by a swamp. As part of my contribution to the affair I took a couple of half filled whisky bottles. Now before anyone gets excited about how you shouldn’t mix guns and alcohol, it has to be said that serious shooters know that even a small hangover affects your ability to shoot straight. While some shooters might be too macho to admit that they are safety conscious, none of them want their mates to laugh at them when they miss. Even so it is usual to consume small amounts of high quality hooch.

Now I am getting to the bit that might be interesting to skeptics. I had one bottle of Glenfiddich and one of Wilson’s. For those who don’t know much about whisky, Glenfiddich is a single malt scotch of excellent quality. Wilson’s is a locally distilled drop, also of excellent quality. Just not made in Scotland. Glenfiddich is expensive. Wilson’s is not. The reason is a combination of good marketing and snobbery.

Tony thought it might be fun to switch the contents of the bottles and see if anyone noticed. Now Glenfiddich is much lighter in colour than Wilson’s but it comes in a green bottle and we didn’t think anyone would pick up on it. But when poured into a glass the difference is obvious.

But they didn’t notice.. The bottles passed the evening getting lighter and lighter. Tony and I were drinking Glenfiddich out of the Wilson’s bottle. The rest drank Wilson’s out of the Glenfiddich bottle.

We never did tell them.

This story is not only true, it’s an anecdote. People tell anecdotes not only to entertain but also to make a point. In this case I am trying to make the point that people’s expectations affect their experience. Our friends believed they were drinking Glenfiddich, so they enjoyed

the whisky more than if they thought they were drinking Wilson’s. But there is another possible explanation. It might also be that my mates don’t know squat about whisky.

In this case I think both are true.

Even though anecdotes should be printed on perforated paper, people use them as if they were good

evidence. In fact, they should only be used as a starting point.

After you have heard an anecdote, the next thing you do is form an hypothesis.

Hypo meaning under. Thesis meaning Theory. An hypothesis is less than a Theory. My hypothesis is that people’s expectations affect their experience.

Next I have to look for more evidence. More anecdotes. Off the top of my head, people who expect organic food to be tastier, think they can taste a difference. People who believe in the healing power of prayer, feel better after visiting a faith healer.

Now the fans of organic food can find loads of anecdotes that indicate how wonderful it is.

The problem with an anecdote is that it doesn’t eliminate those other explanations.

It may be true that “organic” food does taste better than conventional produce. Or it might be that people’s expectations affect their experience. After the anecdote, you have to perform an experiment.

Our local newspaper, a few years ago, got three “experts” to try and tell the difference between some organic and conventional produce. They gave them three different foods.

They fared no better than guessing. The difference between a controlled experiment and an anecdote is the experiment eliminates the other explanations.

You can try it yourself. Next time you are at the supermarket, get some organic orange juice, and also some not organic orange juice. Make sure both are juice, not cordial.

Get 10 glasses and label them 1 to 10. In half of them put the organic juice and in the rest out the other.

Write down which juice is in which glass. Now this is the important bit. Get someone else who doesn’t know which is in which glass to do the tasting.

If there is a difference they should get it right nearly every time. If they cannot tell, they will still get it right half the time.

This is the second important bit. Getting it right half the time means they are just guessing.

This type of test has been done often enough for me to be confident there isn’t any difference except for the expectations of the taster.

Which is a bit of a shame. Because even though I know Wilson’s is just as good, We still drank the “good stuff”.

Ghosts, Mediums and the Argument from Omniscience

In which John Riddell reminisces about happy childhood days and reflects on the stories we tell to grown-ups

When I was just a young skeptic our family used to go to big Christmas family get-togethers at my great aunt’s homestead. There were always lots of fun things for kids to do. There was a swimming pool with water the colour of rotting leaves and a ghost room upstairs in the house. When we thought the grown ups weren’t looking, we would sneak upstairs for a peek into the room. The door was nailed closed, but the big kids knew how to bend the nail back and open the door just a bit. Every year a new group of 8- year-olds would try to scare the bewhatsits out of the 5-year-olds with tales of the ghost. There were also stories that the last kids to enter the room had not left alive through the door. Of course, this was because the room had no floorboards and they had slipped off the rafters and fallen through the ceiling to the dining room below.

I don’t remember if we believed the story of the ghost, but I do remember not wanting to slip and put a hole in the dining room ceiling. Since then the house has become an historic place and been done up to cater for wedding receptions and garden parties. The “Ghost Room” has had new floorboards installed and the nail has been replaced with a doorknob. But the stories of the ghost still continue. One of the uncles takes groups of visitors through and keeps them entertained with rumours of the ghost.

The story started as a way of keeping children from going into a dangerous room, and has continued as a means of entertaining the tourists, but with plenty of drunken wedding guests passing through, it won’t be long before we have a sighting.

There are people who believe in ghosts. Life after death and all that.

There was a scientist in the paper the other day who thought he had found evidence for life after death. He interviewed a lot of people who had nearly died on the operating table. According to doctors trying to save their lives, these people had no detectable brain function at some point.

The scientist found that they had memories that he thought were of that time when they had no detectable brain function. A lot like the fourth form.

It would be very cool if there really was good evidence of a warm and fuzzy place after death, but unhappily, this guy hasn’t found it. There are some other explanations. It might be the doctors had more important things to worry about than making sure the patient had absolutely no brain function. So maybe the brains hadn’t really stopped. Or it could be the memories of a bright light and feelings of happiness were manufactured either before or after the no-brainer. The patients didn’t die, so we don’t really know.

The world’s leading authority on Near Death Experience (NDE ) is Susan Blackmore. For more information about NDE, see http://www.arthurchappell.clara.net/ndes.htm. I thought it was interesting that “Children who have almost died don’t see dead friends and relatives on the other side, but live ones. They haven’t lived long enough to know too many people who have died”.

However, before we give up on life everlasting, there are also mediums. They think they talk to people who really are dead. There was a bloke on Discovery Channel who sat in the middle of a circle of people and gave them messages from their dearly departed. But the messages were not very impressive.

I mean if he said something like “Now Susan, I have a message from your mother Sarah-May. She says don’t have an affair with Billy-Bob coz he has a social disease, and if ya do, that no-good husband of yours will find out and then there’ll be trouble.”

If that was the sort of message from the other side even I might get interested. Assuming of course that Billy-Bob really did have the disease, but nobody else knew. Instead of providing information that nobody in the room could know without checking, the medium always comes up with something the client/audience already knows. He says “I’m getting something about Susan”. Anyone in the audience who is called Susan, or who knows a dead Susan automatically assumes he is talking to them. “How did he know that?” Well he didn’t. You told him. The medium proceeds by throwing out a word or phrase and seeing if someone picks up on it. The medium creates an illusion that he is telling the client things that only the client could know. In fact, the client tells the medium, not the other way around.

It just isn’t good evidence. But the client thinks it is good evidence.

They use what is usually called the “Argument from Ignorance”. It goes like this. “I don’t know how he could possibly know that. There couldn’t be a natural explanation. It has to be supernatural.”

The ignorance of how “it” happened is used as evidence. I prefer to call this the “Argument from Omniscience” which goes like this.

“I am infinitely intelligent and I know everything and if I can’t think how this could have occurred naturally, then the explanation must be supernatural.” Now this would be a good argument if the speaker really was infinitely wise and all knowing. Unfortunately I don’t know anyone who is. Even so, this argument is used to explain belief in ghosts, Creationism and lots of other subjects that interest Skeptics. In the case of the medium, the client thinks “I can’t figure out how he did it, so he must be psychic.”

The same applies to people who think they see a ghost. “I know everything, and I can’t figure out what it was if it wasn’t a ghost.” People don’t normally say the bit about knowing everything. They assume it though.

The scientist who thinks that memories of a Near Death Experience (NDE) are good evidence for the afterlife is no different.

Sometimes it is better to just say, “I don’t know”.

Pseudo-medicine

This is a copy of a presentation given to the New Zealand Skeptics 1995 Conference in Auckland

When Denis Dutton asked me to prepare some comments on this topic he gave me a very wide brief covering, “any aspect that strikes your fancy”.

Since he has left the definition and the territory to me, I will indulge myself, knowing that any remark from here on will be controversial.

Over the same time I had the privilege to witness one of history’s recurrent twists, whereby there is a recapitulation of medical behavioural patterns which can be expressed in Darwinian terms. This has provided some of us with the opportunity to observe the consequences arising directly from the ebb and flow of irrational human behaviour.

In the late 1940s I set out to become an engineer, but I meandered into medicine. I retain some interests in the area of the physical sciences and I think I understand why a 747 flies and usually does not fall to bits on take-off or landing. Such deep insight allows me to perceive the distinction between the functioning of an aircraft engineer and that of a traditional doctor. It is mandatory, as well as reasonable, to test the wings of a proposed new aircraft to the point of destruction in an aeronautical laboratory. In most countries, similar destruction of a human being in a physical or psychological sense is forbidden, or at least not discussed openly in public.

The distinction between the two situations does not stop there. In the former instance, a physical object is being tested by engineers and scientists using a fairly soundly based set of facts, many of which will not change as knowledge evolves. However, errors can occur in both the design and testing of an aircraft wing due to the fallibility in human terms of scientists and engineers. Conversely, in the case of interactions between orthodox doctors and patients or clients, the interactions involve two sets of human behaviour. The nett effect is that at least in terms of ephemeral knowledge, there will be a much greater measure of certainty in the case of the aircraft wing testing than there will be in any health professional-patient interaction.

All that seems very obvious, but I can assure you it is not obvious to many who design and manage health services in various parts of the world, nor is it understood by many orthodox clinicians.

These considerations do, however, lead on to recognition of one perspective through which pseudo-medicine can be defined. In discussing pseudo-medicine we are really addressing a pattern of behaviour which is incongruent with principles common to sound aircraft engineering and sound allopathic medicine. Because a set of physically determined factors imposes a very firm set of disciplines upon the aircraft engineer, he or she operates within definable, and fairly closely defined, sets of constraints.

That is not the situation as far as medicine is concerned. An aircraft wing talks back to its designer by performing efficiently or failing. A patient or client exhibits an enormous range of responses to the propositions of a health professional, who operates within loose constraints, extremely wide boundaries and enormous levels of tolerance. Failure to observe what we may loosely term the laws of nature in relation to aircraft wings induces clearly observable and immediate consequences. Errors of logic and application of scientific knowledge or the indulgence of magic and quackery can persist for centuries in terms of medical practice.

My first point then is that the aircraft engineer is brought face-to-face with the realities of certainty and uncertainty from the outset. Such is not the case for health professional patient interactions.

Uncertainty

When confronted by uncertainty, a person who has a sound understanding of rationality and science acknowledges that doubt and ignorance are facts to be accepted and confronted. If we pause to think about that, hopefully a majority of us within medicine will rapidly realise that John Kenneth Gailbraith was correct when he said “when people are least sure, they are often most dogmatic”.

That idea can be extended by the observation that many who are superficially extremely confident suppress their doubts and uncertainties through extremely assertive behaviour and exposition of dogma. Sometimes they are exposed, as happened to Margaret Thatcher when caught on the hop by the BBC, who perceived she really did not know what to do about the political future of Hong Kong after 1997. “…now, when you say that, you don’t have to go into, to say, well now, precisely what is the nature of this link and the nature of the law and so on…”2

The problem with the Thatchers of this world is that during their predominant period of confidence, while they suppress any dangerous urge to admit doubt and uncertainty, they can inflict devastating damage on huge chunks of society and humanity generally. The consequences may be disastrous for many of us and not just for Argentinean sailors.

That arch sceptic, the late Petr Skrabanek, in a signed Lancet editorial entitled “The Epidemiology of Errors”, quoted Lewis Thomas: “A good deal of scientists, many of them in the professional fields of epidemiology and public health, have never learned how to avoid waffling when yes or no are not available, and the only correct answer is, I don’t know”.1 Pseudo-medicine arises when doctors, particularly, are confronted by a problem for which there is no clear-cut answer. Unfortunately in such situations, many doctors while swearing allegiance on the altar of medical science, move into the Thatcher mode. The practice of pseudo-medicine is based on that phenomenon.

Contrary to the viewpoint of a majority of the public and the media, and against the enthusiastic prophesy of many health professionals, areas of uncertainty are going to become more extensive rather than less as we move into the 21st Century. As technology becomes more sophisticated, complex issues concerning its application are going to raise increasing areas of uncertainty. It is not difficult to predict that there will be an increasing tendency for impetuous action to be taken as anxiety levels increase in the face of uncertainty.

Conversely, there may be a decline in recourse to consultation on the basis of “I don’t know, can you help?” Systematisation of doubt, and suppression of uncertainty lead to indulgence in such practices as homeopathy, chelation and a variety of magical and quack practises. I am not going to go into those areas in detail because they have been well traversed at previous annual meetings of this Society. Rather I want to spend the remaining time indicating the pervasiveness of the problem.

If we put aside the really major health disease problems of society based upon deprivation, economic inequality, hopelessness, loneliness and so forth, we are left with the impact upon society of the chronic degenerative diseases of bones, joints, the cardiovascular systems and cancers. These are the happy hunting grounds of pseudomedicine. The operation of total hip replacement has long since passed the equivalent of the testing of the 747 wings, and is now a standard procedure with sufficient experience behind it to make predictability of application to particular people reasonably certain. That does not mean that a host of other factors are not relevant to the decision whether, when and how to operate on a particular patient and to decide who pays to whom how much.

By contrast, the pain relief to be offered to the person on the increasingly lengthening waiting list for a hip operation provides a fertile ground for the exhibition of pseudo-medicine. Physicians like me do not have ideal pain relieving remedies available for prescribing to such patients. Chronic conditions wax and wane in intensity and it is very difficult to match the interplay of useful and dangerous effects of chronic pain management by drugs, against risks of death, disability and a host of economic factors.

The temptation is always there to indulge in the potentially legitimate use of placebo effect, maybe honestly at first with full understanding of what one is doing, and then to slip into the realm of magic. The boundary between rational therapy and pseudomedicine is very fine, and the width of that boundary varies considerably between one realm of therapy and another and between one doctor and another.

My concept of pseudo-medicine, therefore, is that doctors indulge in the practice when they stop saying, “I don’t know”, stop recognising uncertainty, and substitute false, self-deceiving action based on phoney certainty, backed by great enthusiasm and stern dogma. The euphemistic term “art of medicine” is then applied to this particular brand of practice. The words “art” and “medicine” are simultaneously debased.

Nihilism

Commencing early in the nineteenth century, what has been termed scientific and therapeutic nihilism developed initially in France. In the late 19th Century, influential figures from North America and England, including Sir William Osler who typified both environments, threw their weight behind the therapeutic nihilistic movement. This involved a sceptical approach to the practices and remedies of traditional medicine, and called for the application of rational study and controlled observation of the natural history of disease and its modification in various ways. There was considerable opposition to Osler. Rationalism, scepticism and the scientific method itself, are not immune to rigorous querying from a variety of viewpoints. All can be converted into new forms of religion and all are subject to phases in development.

It took about a hundred years for therapeutic nihilism to demolish significant sections of the old pharmacopoeia, continuing use of which was justified and dignified as being part of the art of medicine.

Earlier in the talk I referred to the interaction of two sets of behaviour when doctor meets patient. History is repeating itself at present as the boundaries where medical science and human behaviour meet are becoming a major topic in the more thoughtful pages of the New England Journal of Medicine, Lancet, BMJ and so forth. Interestingly, the predominant theme in this new wave of medical literature centres on the problems of uncertainty.

Jonathon Rees in the BMJ puts it this way. “For any activity dependent on new knowledge, as medicine is on science, the future is uncertain simply because new knowledge always changes the rules of the game. But even if we could dream this problem away, our guesses of the future will be in error because we continue to delude ourselves, outside the laboratory at least, that we understand the present…”3 Herein lies another basis for pseudo-medicine. Heath professionals like to feel confident and to project confidence in terms of their relationships with patients. Pseudo-medicine flourishes on the basis of apparent confidence exhibited by the professional. The stage is being set in my opinion for an increase in the practice of pseudo-medicine.

Anti-orthodoxy

During the 1960s, 70s and 80s there was a wave of revulsion directed against orthodox medicine and particularly to its perceived power. To some extent the evils attributed to the atomic scientists spilled over into public attitudes towards orthodox medicine. What was perceived as unholy power held by the medical profession was seen in terms of a citadel which should be destroyed. We saw the revival of naturalism, herbalism and a return to various magical procedures. One of the major textbooks of so-called holistic medicine claimed restoration of the theory of transmutation of the elements whereby sodium was converted to potassium by plants.

The attack was unconsciously, and by some cynical entrepreneurs consciously, directed at the whole concept of therapeutic nihilism. The wash from this revolution lapped on the thresholds of medical schools initially, and then penetrated the corridors of academic medicine. To the horror of people like me, graduates of our young School of Medicine began openly to practise homeopathy and chelation.

I analyse this situation as being due partly to the failure of us as educators to prepare students to handle the avalanche of evolving knowledge in the fields of biochemistry, molecular and behavioural medicine. We have been overwhelmed and have not known how to handle the situation. Our students have entered a world in which monetarism has gained the ascendancy and they see a desperate need to make a living. Those who choose not to become technocrats, replacing hips and removing cataracts, are the most vulnerable. Many of them have already succumbed. Moreover the ramparts of the citadel have been breached in more significant ways.

Our students face the usual mixture of myth and reality which typifies the real world — we have not prepared them adequately to confront this reality and provided them with teaching to handle the situation calmly and rationally.

There is a current vogue for insisting that doctors must model their approach to patients upon so-called “evidence-based” medical practice.4,5 The general concept implies that resources of the State, in particular, should only be expended in those areas where there is so-called objective proof that expenditure will significantly influence the natural course of a disease process. Impetus has been given to this movement through a failure of classical epidemiological approaches to produce clear-cut answers for handling the problems of middle and old age.

Over the past two to three decades, so-called scientific medicine backed by exhortations of academia has persisted in traversing the pathway so heavily criticised by Skrabanek and others. This trend has to some extent been driven by a need for resource acquisition for some sections of medical epidemiology. Disciplines such as cardiology have been happy to help create and then support a mirage through which scientific medicine is seen as responsible for releasing an accelerating series of miracles which will ultimately bring lifelong happiness to everyone. When confronted by the failure to deliver to the masses, sections of these same disciplines, like clinicians, have resorted to pseudo-science that dangerous ally of pseudo-medicine.

Rather than confronting politicians and the public with a clearly defined list of uncertainties, probabilities and areas of ignorance, as David Naylor from the Institute for Clinical Evaluated Sciences in Ontario has pointed out, they have “continued to produce inflated expectations of outcomes-oriented and evidence-based medicine.”5 Following these pathways, they have resorted, not to metaphysics or alchemy, but rather to meta-analysis and leaps of faith which are presented as scientific truths.

The Real Culprit

As Skrabenek has pointed out, the real culprit in all this is “risk-factor epidemiology”. This brash young infant amongst the medical sciences has continued to feed information and misinformation into the media. To quote Skrabenek again, “by the misuse of language and logic, observed associations are presented as causal links”. He further points out that “risk-factor epidemiology relies on case-control or cohort studies without rigorous standards of design, execution and interpretation, even though such studies are susceptible to at least 56 different biases. … How should one remedy this state of affairs — bigger studies, better measurement of risk factors, more complex statistics? Statistics are no cure for the faulty paradigm of risk-factor epidemiology.”1

It is in these areas that pseudo-science has aided and abetted what I perceive to be a particularly dangerous form of pseudo-medicine. It is in these areas that I perceive the most significant breaching of the ramparts of the citadel of scientifically based medical practice.

How has this come about? I believe it derives from the attributes of human behaviour stressed in the earlier part of this talk. Faced with failure to reach their objectives within a particular time span, many working in cardiovascular, cancer, and degenerative diseases have chosen to cope by denying areas of ignorance and uncertainty. Unfortunately they have gone further and have moved the goal-posts when it suited them. They have extrapolated, simplified and at times gone even further.

An obvious example to quote is the famous Lipid Research Clinics Study referred to in a paper at Palmerston North last year. In this study a somewhat unpleasant drug called cholestyramine was used to treat North American men held to be at particular risk from coronary artery disease due to elevated blood cholesterol levels. Extrapolation from that study was quite extraordinary and media manipulation of enormous magnitude was employed to preach a message intended for the masses when the facts were that such extrapolation was invalid for women and for the great bulk of the population.

You will all probably believe, correctly, that strict standards should apply to evaluation of both old and new therapies. It is a truism that anything short of randomised double-blind trials is regarded by proponents of evidence-based medicine as providing an unreliable base upon which to proceed. The problem is that these worthy objectives are being distorted and the public is not being given a transparent account of the problems.

Hormone Replacement

A classical example at the present time would be the largely male-determined dogma that hormone replacement therapy for post-menopausal women cannot be justified in terms of evidence-based medicine. The pseudo-medical pronouncements in this instance have a very complex background which is not usually presented. HRT in terms of scientific literature has concentrated almost wholly upon the fact that women after the change-in-life tend to catch up on men in terms of manifestations of atherosclerosis. There have been no published results from major double-blind prospective clinical trials of oestrogens alone or oestrogens combined with progesterones testing whether or not this therapy retards the appearance of myocardial infarction (coronary attacks) in post-menopausal women.

Prospective trials have shown that oestrogens make women more comfortable in terms of their nether regions, their skin texture and preservation of femininity itself. There is some soft evidence that osteoporosis may be retarded amongst woman taking HRT. Thus the pseudo-medicine proponents of evidence-based medicine who concentrate solely upon one aspect of hormone replacement, that of the cardiovascular effects, are not indulging in true science.

As Naylor has put it, we live in the era of chronic and expensive diseases. “Until the ongoing revolution in molecular biology pays more concrete dividends, we shall be muddling along with what Lewis Thomas characterised as half-way technologies. However medical muddling is a profitable business…”5 It is profitable for research groups, for industry and particularly for the exponents of pseudo-science and pseudo-medicine.

False Prophet

However it is more complicated than that. The general assumption by the practitioners of pseudo-medicine is that more, and what they term better, data will dispel uncertainty in medical decision making. Those who say these things seem unable to learn even from recent history. Those who put their faith in meta-analysis are following a false prophet. Take the case of magnesium in treatment of myocardial infarction. A meta-analysis published in 1993 is entitled “Intravenous magnesium in acute myocardial infarction. An effective, safe, simple and inexpensive intervention”.6 Two years later, results of another mega trial showed that magnesium was, if not totally ineffective, only minimally so in treatment of myocardial infarction.7 Resorting to big numbers will not necessarily solve problems from which the pseudo-medicine proponents are seeking to escape nor will it satisfy the absolutist neo-nihilists.

The current vogue for meta-analysis has arisen from a problem clearly recognised by both impeccable medical scientists and proponents of pseudo-medicine. This is the sheer cost of answering key questions based upon hypotheses propounded in relation to chronic diseases. Because genetic endowment heavily influences the differences between us, manipulation of the environment, including our internal environment, through drugs or diets will usually produce gains at the margin, which are usually minimal.

Blunderbuss therapy requires treating of the masses, many of whom will not benefit, while others are harmed by the proposals. The passion for evidence based medical practice, given our current range of technologies, must make recourse to fairly desperate measures. Thus meta-analysis has become big business. Like is not being lumped with like. Little lumps and big lumps of data are being gathered together by various groups beavering away upon the basis for their own particular perspectives, all seeking to justify their particular beliefs which are promulgated as gospel to an eagerly awaiting public. Unfortunately, some of the larger lumps so aggregated are themselves curate’s eggs.

A classic example is the so-called MRFIT data. The Multiple Risk Factor Intervention Trial (MRFIT)8 was a massive study mounted in North America, involving screening of either 361,662 or 361,629 men. Data from the MRFIT screenees has contributed very significantly to a number of the meta-analyses.

Werkö from the Swedish Council on Technology Assessment in Health Care has shown clearly that this massive body of data is significantly and seriously flawed.9 There is inconsistency between reports published in different journals simultaneously. The quality control of the basic data is uneven and people using the material seriously have not even bothered to check the relatively simple points investigated by Werkö. Not to do so is a form of scientific laziness, a form of pseudo-science. If these writers have done so and failed to spot the obvious flaws, then their baseline checks have been sloppy. If they have done so, and uncovered the same points as Werkö and chosen to ignore the evidence in front of them, they are true practitioners of pseudo-science and pseudo-medicine.

Meta-analysis has come in for hefty criticism and deservedly so. While its proponents acknowledge that it is a surrogate for the massively expensive prospective studies which are really required, they frequently go way beyond the capacity of the method in terms of the public pronouncements they make. In particular this applies to translation of conclusions relevant to people at special risk, to the advice given to the masses who may not share the same risks or who portray them in only a minor degree. Meta-analysis is now an art form whose scientific significance must be challenged at each stage and with each pronouncement.

Political Involvement

The situation is more sinister than that because politicians through their minions have cottoned on to the value of some of these manipulable analytical techniques. Thus, information gathered in relation to the National Health Service of the United Kingdom is being used to support claims of success of recent government policies. The same types of problem identified by Werkö arise when politicians make use of this type of data. Once politicians and media get into the business of using flawed information, or of distorting sound information for particular purposes, very unhealthy alliances will result.10,11

Our critics are correct in stating that medicine has built a very powerful base within society. Pronouncements by any segment of medicine or its associates are likely to be taken seriously, even in the face of the current wave of mounting scepticism. Epidemiologists and their allies in cardiology have established a major section of the health-disease industry. There are consequences. For instance, an increasing epidemic of osteoporosis in some western countries may well be based upon reduced calcium intake, particularly by women. Dairy products have been the main contributor of calcium in those countries. In contrast to big sections of epidemiology and cardiology, the dairy industry has employed competent nutritionists and made some attempt to keep pace with evolving knowledge of human nutrition. It deserves credit for the burgeoning range of modified milk products, all of which contain calcium. But the damage has been done from within the medical power base. As David Naylor has put it, these difficulties have arisen from the Malthusian growth of uncertainty when multiple technologies combine into clinical strategies and at the public advice level.5

Thoughtful critics of societal development have been drawing attention to these problems. Many advocate a solution through the information revolution, but in terms of the present topic they have failed to perceive that medical information is fragile, patchy and usually imperfect. Like the Lancet editor, I do not believe the consumer watchdog type of approach, with its challenge to the medical powerbase, is going to change the situation at any great speed.11

One healthy fallacy states that the medical powerbase rests solely on possession of scientific information and a monopoly thereof. As I have tried to demonstrate that base is neither secure nor constant. In the health-disease management industry, power does not reside in possession of scientific information. The current success of the inheritors of the old magic, that is the quack acupuncturists, the chelation therapists, many herbalists, naturopaths and so forth, does not reside in a possession of a body of scientific information or a monopoly of its use. This has always been so. Medical power rests as much on uncertainty as it does on technical expertise or possession of a particular body of ephemeral knowledge which will be disproved tomorrow. How can that be so?

The Lancet states it thus, “uncertainty in the face of disease and death fosters a compelling need for patients to trust someone — and a reciprocal authority among doctors. A leap of faith will always be needed. Information does not, and cannot provide all the answers.”11 We thus have a paradox to confront.

Pragmatic Doctors

To return to the aeronautical engineer. Doctors must indeed make decisions, give advice and offer assistance based on limited interpretation of limited evidence. For the foreseeable future doctors must make decisions which will not be derived from carefully controlled prospective randomised clinical trials. They must nevertheless try to make valid decisions. They cannot indulge in the luxury of being inactive in the face of an absence of evidence. That privileged position belongs to the lawyers, the philosophers and the ethicists. In the end doctors have to be pragmatists. Clinical decisions must be made through a plurality of means, each of which must however, undergo “profound interpretative scrutiny”.4

The doctor’s role is more difficult than that of the aircraft wing designer. They must discipline themselves continually to apply medical knowledge in conjunction with their experience and that of their colleagues. “The unifying science of medicine is an inclusive science of interpretation.”4 The black and white situation of 747 wing testing does not occur in medicine. “Medicine is a series of grey zones in which the evidence concerning risk-benefit ratios of competing clinical options is incomplete or contradictory.”5 The grey zones have varying boundaries which change rapidly.

We academics have great difficulty enabling undergraduates and emerging graduates to cope with these phenomena. It is not surprising that many move into pseudo-medicine. It is not surprising that the teaching of orthopaedics is always much more popular with undergraduates than that of clinical medicine. Once again to quote Naylor, “clinical medicine seems to consist of a few things we know, a few things we think we know (but probably don’t) and lots of things we don’t know at all”.5

We academics have to cope with the fact that when evidence alone cannot guide clinical actions, some undergraduates will take up a minimalistic approach whereas others will favour intervention based upon varying balances of inference and experiences and others will turn to pseudo-medicine. Our job as academics is to make emerging clinicians comfortable with a system whereby they can make decisions under conditions of uncertainty.

Over the next decade at least, I believe medical academics will have to confront a somewhat irrational passion for evidence-based medicine and meta-analysis, and we must teach that there are limits to medical evidence and its application. The craft of caring for patients is a legitimate, scientifically appropriate adjunct to medicine. That role is necessary for the comfort and sanity of human society. Osler said, “good clinical medicine will always blend the art of uncertainty with the science of probability.” We need to understand, then to explain what we mean by the term probability.

I shall end with another example. The practice of pseudo-medicine can inflict much discomfort. For instance, young doctors and nurses have considerable difficulty in agreeing to decisions that this patient or that should not be subjected to the indignity of resuscitation procedures, but rather be left to die in peace.

There is a significant and coherent literature indicating that a majority of resuscitation procedures as undertaken in the 1970s and 80s were futile from the outset. The continued pseudo-medical practice in this regard has led to a situation where relatives expect resuscitation procedures to be undertaken. Their concept of power sharing puts heavy pressure on younger doctors to overturn non-resuscitation orders. If the younger doctors submit, an undignified charade ensues. In turn, that situation has created an environment in which aspects of the so-called passive euthanasia debate have become more tangled than was necessary.

I will not dwell further on that point. Rather, I wish to end by emphasising that facing up to uncertainty and accepting areas of ignorance honestly, does not constitute an admission of laziness or incompetence. That, however, is the perspective which sections of the legal profession and society generally are promoting at the present time. If we submit to such pressures and false perspectives we shall end up as we did in relation to the false-confession mistaken-conviction situation, which was discussed at our conference last year.

All professional groups are vulnerable to external influences playing on our own emotional state and anxiety level. Pseudo-medicine thrives in this environment. If we take the subject of evidence in a legal sense we can remind ourselves that a series of techniques have been advised to law authorities over the past century and a number are still in use in the United States, including the polygraph. All have proved to be potentially unreliable, subject to manipulation and all can produce false-positive and false-negative results. If anything their use increases the risk of false confessions.

Those members of the medical and psychology professions whose weakness and pseudo-science has contributed to the situation have much to answer for. Faced with such examples we should have a better understanding of the pervasiveness of the problems of pseudo-medicine and pseudo-science.12 Society needs the NZCSICOP.

Green Peppers

I shall end with the parable of the green peppers. One could term it a parody. The original publication is in the Journal of Irreproducible Results somewhere round about 1955, I think, but I have lost the reference. Some bright workers in Chicago noted that everyone who had eaten green peppers in their youth but had reached the age of 89, had grey hair or white, rotten joints, few teeth, failing eyesight and poor hearing. The main reference in the bibliography was to a guy called Shakespeare somewhere in the early 17th Century. The green pepper eating cohort who had reached the age of 105 were considerably worse off. No-one who had eaten green peppers was alive by the age of 130.

The green pepper industry obviously faltered at that point. However, a subsequent paper which I believe was written but rejected by the same worthy journal, described a restudy of the situation. This showed that people who had eaten green peppers when surveyed at the age of 20 had normal hearing, all their teeth, no lens opacities and sound joints.

In comparison with the older cohorts studied in the first publication, those who had eaten green peppers ten to twenty years earlier showed a mortality rate of 0.05%. Amongst the 90 year old group in the earlier paper, the mortality experienced by that cohort was noted to be 95.2%. Of high significance statistically was the observation that amongst people in that population over the age of 100, only 1% consumed green peppers in the last twenty years. The conclusion was obvious that those who stopped eating green peppers after an interval of twenty years suffered greying and falling of hair, diminished eyesight, reduced hearing, loss of teeth, a very high mortality rate and rotten joints.

Evidence is one thing, quality of evidence another. Intelligent interpretation and carefully planned application of evidence belong to different dimensions. Quality of action based on evidence depends upon the quality of the evidence, its completeness or otherwise, and the quality of the interpretation plus recognition of what is not known and what is not likely to be known over the next years or decades. Life was not meant to be easy.

The practice of medicine combines the twin problems and pleasures inherent in basing action upon adequate evidence on the one hand and inadequate evidence on the other. Practice of the art of medicine is a legitimate activity dependent for its integrity upon the understanding of the dilemmas posed by this dual basis for action and understanding of the nature of science, including the ephemeral nature of scientific knowledge. Pseudo-medicine is practised by those who lack the resolve and energy to face this intellectual challenge.

References

1) Skrabanek P. Lancet 1993; Vol 342: 1502

2) Margaret Thatcher, PM. BBC World Service interview, 1 Nov 1983

3) Rees J. BMJ; Vol 310: 850-853

4) Horton R. Lancet 1995; Vol 346: 3

5) Naylor ED. Lancet 1995; Vol 345: 840-842

6) Yusuf S et al. Circulation 1993; Vol 87: 2043-2046

7) ISSIS-4 etc. Lancet 1995; Vol 345: 669-685

8) MRFIT. JAMA 1982; Vol 248: 1465-1477

9) Werk[oumlaut] L. J. Int. Med 1995; Vol 237: 507-518

10) Wright M. GP Weekly 1995; 2 August: 12-13

11) Lancet 1995; Vol 345: 1449-1450

12) Lancet 1994; Vol 344: 1447-1450

Cargo Cult Science

This is a Feynman Commencement Address given by Richard Feynman at Caltech in 1974. This message is as relevant today as it was 20 years ago, especially for those who add their committed “science” to the cause of apocalyptic environmentalism.

During the Middle Ages there were all kinds of crazy ideas, such as that a piece of rhinoceros horn would increase potency. Then a method was discovered for separating the ideas — which was to try one to see if it worked, and if it didn’t work, to eliminate it. This method became organised, of course, into science. And it developed very well, so that we are now in the scientific age. It is such a scientific age, in fact, that we have difficulty in understanding how witch doctors could ever have existed, when nothing that they proposed ever really worked — or very little of it did.

But even today I meet lots of people who sooner or later get me into a conversation about UFOs, or astrology, or some form of mysticism, expanded consciousness, new types of awareness, ESP, and so forth. And I’ve concluded that it’s not a scientific world.

Most people believe so many wonderful things that I decided to investigate why they did. And what has been referred to as my curiosity for investigation has landed me in a difficulty where I found so much junk that I’m overwhelmed. First I started out by investigating various ideas of mysticism, and mystic experiences. I went into isolation tanks and got many hours of hallucinations, so I know something about that. Then I went to Esalen, which is a hotbed of this kind of thought (it’s a wonderful place; you should go visit there). Then I became overwhelmed. I didn’t realise how much there was.

At Esalen there are some large baths fed by hot springs situated on a ledge about thirty feet above the ocean. One of my most pleasurable experiences has been to sit in one of those baths and watch the waves crashing onto the rocky shore below, to gaze into the clear blue sky above, and to study a beautiful nude as she quietly appears and settles into the bath with me.

One time I sat down in a bath where there was a beautiful girl sitting with a guy who didn’t seem to know her. Right away I began thinking, “Gee! How am I gonna get started talking to” this beautiful nude babe?”

I’m trying to figure out what to say, when the guy says to her, “I’m, uh, studying massage. Could I practice on you?”

“Sure”, she says. They get out of the bath and she lies down on a massage table nearby.

I think to myself, “What a nifty line! I can never think of anything like that!” He starts to rub her big toe. “I think I feel it”, he says. “I feel a kind of dent — is that the pituitary?”

I blurt out, “You’re a helluva long way from the pituitary, man!”

They looked at me, horrified — I had blown my cover — and said, “It’s reflexology!”

I quickly closed my eyes and appeared to be meditating.

That’s just an example of the kind of things that overwhelm me. I also looked into extrasensory perception and PSI phenomena, and the latest craze there was Uri Geller, a man who is supposed to be able to bend keys by rubbing them with his finger. So I went to his hotel room, on his invitation, to see a demonstration of both mind-reading and bending keys.

He didn’t do any mind-reading that succeeded; nobody can read my mind, I guess. And my boy held a key and Geller rubbed it, and nothing happened. Then he told us it works better under water, and so you can picture all of us standing in the bathroom with the water turned on and the key under it, and him rubbing the key with his finger. Nothing happened. So I was unable to investigate that phenomenon.

But then I began to think, what else is there that we believe? (And I thought then about the witch doctors, and how easy it would have been to check on them by noticing that nothing really worked.) So I found things that even our own people believe, such as that we have some knowledge of how to educate.

There are big schools of reading methods and mathematics methods, and so forth, but if you notice, you’ll see the reading scores keep going down — or hardly going up in spite of the fact that we continually use these same people to improve the methods. There’s a witch doctor remedy that doesn’t work. It ought to be looked into; how do they know that their method should work? Another example is how to treat criminals. We obviously have made no progress — lots of theory, but no progress — in decreasing the amount of crime by the method that we use to handle criminals.

Yet these things are said to be scientific. We study them. And I think ordinary people with common sense ideas are intimidated by this pseudoscience. A teacher who has some good idea of how to teach her children to read is forced by the school system to do it some other way — or is even fooled by the school system into thinking that her method is not necessarily a good one. Or a parent of bad boys, after disciplining them in one way or another, feels guilty for the rest of her life because she didn’t do “the right thing”, according to the experts.

So we really ought to look into theories that don’t work, and science that isn’t science.

I think the educational and psychological studies I mentioned are examples of what I would like to call “cargo cult science”.

In the South Seas there is a cargo cult of people. During the war they saw airplanes land with lots of good materials, and they want the same thing to happen now. So they’ve arranged to make things like runways, to put fires along the sides of the runways, to make a wooden hut for a man to sit in, with two wooden pieces on his head like headphones and bars of bamboo sticking out like antennas — he’s the controller — and they wait for the airplanes to land. They’re doing everything right. The form is perfect. It looks exactly the way it looked before. But it doesn’t work. No airplanes land. So I call these things cargo cult science, because they follow all the apparent precepts and forms of scientific investigation, but they’re missing something essential, because the planes don’t land.

Now it behooves me, of course, to tell you what they’re missing. But it would be just about as difficult to explain to the South Sea Islanders how they have to arrange things so that they get some wealth in their system. It is not something simple like telling them how to improve the shapes of the earphones. But there is one feature I notice that is generally missing in cargo cult science. That is the idea that we all hope you have learned in studying science in school — we never explicitly say what this is, but just hope that you catch on by all the examples of scientific investigation.

It is interesting, therefore, to bring it out now and speak of it explicitly. It’s a kind of scientific integrity, a principle of scientific thought that corresponds to a kind of utter honesty — a kind of leaning over backwards. For example, if you’re doing an experiment, you should report everything that you think might make it invalid — not only what you think is right about it: other causes that could possibly explain your results, and things you thought of that you’ve eliminated by some other experiment, and how they worked — to make sure the other fellow can tell they have been eliminated.

Details that could throw doubt on your interpretation must be given, if you know them. You must do the best you can — if you know anything at all wrong, or possibly wrong — to explain it. If you make a theory, for example, and advertise it, or put it out, then you must also put down all the facts that disagree with it, as well as those that agree with it.

There is also a more subtle problem. When you have put a lot of ideas together to make an elaborate theory, you want to make sure, when explaining what it fits, that those things it fits are not just the things that gave you the idea for the theory; but that the finished theory makes something else come out right, in addition.

In summary, the idea is to try to give all of the information to help others to judge the value of your contribution; not just the information that leads to judgement in one particular direction or another.

The easiest way to explain this idea is to contrast it, for example, with advertising. Last night I heard that Wesson oil doesn’t soak through food. Well, that’s true. It’s not dishonest; but the thing I’m talking about is not just a matter of not being dishonest, it’s a matter of scientific integrity, which is another level. The fact that should be added to that advertising statement is that no oils soak through food, if operated at a certain temperature. If operated at another temperature, they all will, including Wesson oil. So it’s the implication which has been conveyed, not the fact, which is true, and the difference is what we have to deal with.

We’ve learned from experience that the truth will come out. Other experimenters will repeat your experiment and find out whether you were wrong or right. Nature’s phenomena will agree or they’ll disagree with your theory. And, although you may gain some temporary fame and excitement, you will not gain a good reputation as a scientist if you haven’t tried to be very careful in this kind of work. And it’s this type of integrity, this kind of care not to fool yourself, that is missing to a large extent in much of the research in cargo cult science.

A great deal of their difficulty is, of course, the difficulty of the subject and the inapplicability of the scientific method to the subject. Nevertheless it should be remarked that this is not the only difficulty. That’s why the planes didn’t land — but they don’t land.

We have learned a lot from experience about how to handle some of the ways we fool ourselves.

One example: Millikan measured the charge on an electron by an experiment with falling oil drops, and got an answer which we now know not to be just right. It’s a little bit off, because he had the incorrect value for the viscosity of air. It’s interesting to look at the history of measurements of the charge of the electron, after Millikan. If you plot them as a function of time, you find that one is a little bigger than Millikan’s, and the next one’s a little bit bigger than that, and the next one’s a little bit bigger than that, until finally they settle down to a number which is higher.

Why didn’t they discover that the new number was higher right away? It’s a thing that scientists are ashamed of — this history — because it’s apparent that people did things like this: when they got a number that was too high above Millikan’s, they thought something must be wrong — and they would look for and find a reason why something might be wrong. When they got a number closer to Millikan’s value, they didn’t look so hard. And so they eliminated the numbers that were too far off, and did other things like that. We’ve learned those tricks nowadays, and now we don’t have that kind of a disease.

But this long history of learning how not to fool ourselves — of having utter scientific integrity — is, I’m sorry to say, something that we haven’t specifically included in any particular course that I know of. We just hope you’ve caught on by osmosis.

The first principle is that you must not fool yourself — and you are the easiest person to fool. So you have to be very careful about that. After you’ve not fooled yourself, it’s easy not to fool other scientists. You just have to be honest in a conventional way after that.

I would like to add something that’s not essential to the science, but something I kind of believe, which is that you should not fool the layman when you’re talking as a scientist. I am not trying to tell you what to do about cheating on your wife, or fooling your girlfriend, or something like that, when you’re not trying to be a scientist, but just trying to be an ordinary human being — we’ll leave those problems up to you and your rabbi.

I’m talking about a specific, extra type of integrity that is not lying, but bending over backwards to show how you’re maybe wrong, that you ought to have when acting as a scientist. And this is our responsibility as scientists, certainly to other scientists, and I think to laymen.

For example, I was a little surprised when I was talking to a friend who was going to go on the radio. He does work on cosmology and astronomy, and he wondered how he would explain what the applications of this work were.

“Well”, I said, “there aren’t any.” He said, “Yes, but then we won’t get support for more research of this kind.”

I think that’s kind of dishonest. If you’re representing yourself as a scientist, then you should explain to the layman what you’re doing — and if they don’t want to support you under those circumstances, then that’s their decision.

One example of the principle is this: if you’ve made up your mind to test a theory, or you want to explain some idea, you should always decide to publish it whichever way it comes out. If we only publish results of a certain kind, we can make the argument look good. We must publish both kinds of results.

I say that’s also important in giving certain types of government advice. Supposing a senator asked you for advice, about whether drilling a hole should be done in his state; and you decide it would be better in some other state. If you don’t publish such a result, it seems to me you’re not giving scientific advice. You’re being used. If your answer happens to come out in the direction the government or the politicians like, they can use it as an argument in their favour; if it comes out the other way, they don’t publish it at all. That’s not giving scientific advice.

Other kinds of errors are more characteristic of poor science. When I was at Cornell, I often talked to the people in the psychology department. One of the students told me she wanted to do an experiment that went something like this — it had been found by others that under certain circumstances, X, rats did something, A. She was curious as to whether, if she changed the circumstances to Y, they would still do A. So her proposal was to do the experiment under circumstances Y and see if they still did A.

I explained to her that it was necessary first to repeat in her laboratory the experiment of the other person — to do it under condition X to see if she could also get result A, and then change to Y and see if A changed. Then she would know that the real difference was the thing she thought she had under control.

She was very delighted with this new idea, and went to her professor. And his reply was, no, you cannot do that, because the experiment has already been done and you would be wasting time. This was still about 1947 or so, and it seems to have been the general policy then to not try to repeat psychological experiments, but only to change the conditions and see what happens.

Nowadays there’s a certain danger of the same thing happening, even in the famous field of physics. I was shocked to hear of an experiment done at the big accelerator at the National Accelerator Laboratory, where a person used deuterium. In order to compare his heavy hydrogen results to what might happen with light hydrogen he had to use data from someone else’s experiment on light hydrogen, which was done on different apparatus. When asked why, he said it was because he couldn’t get time on the program (because there’s so little time and it’s such expensive apparatus) to do the experiment with light hydrogen on this apparatus because there wouldn’t be any new result.

And so the men in charge of programs at NAL are so anxious for new results, in order to get more money to keep the thing going for public relations purposes, they are destroying — possibly — the value of the experiments themselves, which is the whole purpose of the thing. It is often hard for the experimenters there to complete their work as their scientific integrity demands.

Not all experiments in psychology are of this type, however. For example, there have been many experiments running rats through all kinds of mazes, and so on — with little clear result. But in 1937 a man named Young did a very interesting one. He had a long corridor with doors all along one side where the rats came in, and doors along the other side where the food was. He wanted to see if he could train the rats to go in at the third door down from wherever he started them off. No. The rats went immediately to the door where the food had been the time before.

The question was, how did the rats know, because the corridor was so beautifully built and so uniform, that this was the same door as before? Obviously there was something about the door that was different from the other doors.

So he painted the doors very carefully, arranging the textures on the faces of the doors exactly the same. Still the rats could tell. Then he thought maybe the rats were smelling the food, so he used chemicals to change the smell after each run. Still the rats could tell. Then he realised the rats might be able to tell by seeing the lights and the arrangement in the laboratory like any common sense person. So he covered the corridor, and still the rats could tell.

He finally found that they could tell by the way the floor sounded when they ran over it. And he could only fix that by putting his corridor in sand. So he covered one after another of all possible clues and finally was able to fool the rats so that they had to learn to go in the third door. If he relaxed any of his conditions, the rats could tell.

Now, from a scientific stand-point, that is an A-number-one experiment. That is the experiment that makes rat-running experiments sensible, because it uncovers the clues that the rat is really using — not what you think it’s using. And that is the experiment that tells exactly what conditions you have to use in order to be careful and control everything in an experiment with rat-running.

I looked into the subsequent history of this research. The next experiment, and the one after that, never referred to Mr Young. They never used any of his criteria of putting the corridor on sand, or being very careful. They just went right on running rats in the same old way, and paid no attention to the great discoveries of Mr Young, and his papers are not referred to, because he didn’t discover anything about the rats. In fact, he discovered all the things you have to do to discover something about rats. But not paying attention to experiments like that is a characteristic of cargo cult science.

Another example is the ESP experiments of Mr Rhine, and other people. As various people have made criticisms — and they themselves have made criticisms of their own experiments — they improve the techniques so that the effects are smaller and smaller and smaller until they gradually disappear. All the parapsychologists are looking for some experiment that can be repeated — that you can do again and get the same effect — statistically, even. They run a million rats — no, it’s people this time — they do a lot of things and get a certain statistical effect. Next time they try it they don’t get it any more. And now you find a man saying that it is an irrelevant demand to expect a repeatable experiment. This is science?

This man also speaks about a new institution, in a talk in which he was resigning as Director of the Institute of Parapsychology. And, in telling people what to do next, he says that one of the things they have to do is be sure they only train students who have shown their ability to get PSI results to an acceptable extent — not to waste their time on those ambitious and interested students who get only chance results. It is very dangerous to have such a policy in teaching — to teach students only how to get certain results, rather than how to do an experiment with scientific integrity.

So I have just one wish for you — the good luck to be somewhere where you are free to maintain the kind of integrity I have described, and where you do not feel forced by a need to maintain your position in the organisation, or financial support, or so on, to lose your integrity.

May you have that freedom.

Professor Mack and his Amazing Abducting Aliens

An abridged version of the Skeptical Enquirer’s report of the session dealing with “alien abductions” at the Seattle CSICOP Conference on “The Psychology of Belief”

Many of us have been reading articles or commentaries regarding alien abductions and have just wished that someone of real authority would take up the issue and give it a thorough scientific once over. Most of us were encouraged when we learned that John Mack, award winning Professor of Psychology at Harvard University, had taken on the task.

So it was with some surprise that we found that his work Abduction: Human Encounters With Aliens proclaims Professor Mack’s beliefs that many of his patients have been abducted by aliens — and that he is now the most famous spokesman for this cause.

He vigorously defended his claims at the conference and worried some of the audience by suggesting that other cultures have always known there are other realities, other beings, other dimensions. There is a world of other dimensions, of other realities that can cross over into our own world.

Which realities, beings and dimensions he did not say. One would have expected Professor Mack’s work to at least have been well founded in scientific methodology. But this assumption took a bit of a knock when Donna Bassett, a researcher who had participated in the Professor’s research programme, was called up to the platform to speak.

At first Bassett seemed to indicate that she was one of Mack’s genuine abductees. But she quickly announced that since September 1992 she had been only posing as one in order to infiltrate Mack’s project and learn about his research methods.

“I faked it! Women have been doing it for centuries!” she said.

Ms Bassett reported that Professor Mack’s procedures were flawed and he used little or no scientific methodology. During therapy sessions patients would often get together to embellish their stories. They told Professor Mack what he wanted to hear. Of course her most telling point was that the Professor’s research methods had failed to identify that this “patient” was “faking it”.

Needless to say Professor Mack responded in the expected manner.

I am (deeply?) saddened by this…

I am a little bit clearer about this when I am told that [Bassett] was found to play this role by Philip Klass [of the CSICOP Executive Council] — since that’s his purpose, to destroy and undercut the credibility of this work.

That’s right. Sadly indeed for Professor Mack’s on-going future as a TV chat show guest, that’s what science is about.

There were a few more heated exchanges until Robert Baker ended the session on a humorous note by recommending a new direction for this line of research. He explained :

Sixty-nine percent of Americans believe in angels, and 32% claim they have had contact with them. Now that’s a lot better than for alien abductions. I think we ought to investigate angels…

Thoughts on the Longevity of Superstitions

What is it that keeps superstitions going in the face of our increasing knowledge about the world?

There is no easy, let alone absolute, way of telling the difference between a true belief and a false or superstitious one. In order to be able to label a belief a superstition, one would have to be able to define clearly what kind of belief would not be a superstition; or, for that matter, to call something abnormal, one would have to be quite sure what sort of thing would be normal.

However, people are very ready to insist on these distinctions and they tend to do so on the grounds of what seems to me a very mistaken notion. They think that one can distinguish between true and superstitious beliefs in terms of the method by which the beliefs have been arrived at. There is a correct method, it is alleged, and there are incorrect methods. If the correct method is followed, then the belief it leads to must be a true belief. When pressed such people cite “observation” and/or “reason” as the characteristics of a correct method. Both observation and reason are very woolly terms. If one wants to observe, one first has to know what one wants to observe. And then one has to make sure that the observation is not a hallucination, and so forth. There is no finality in “observation”. The method of reason is equally woolly. People differ very widely on what they suppose to be “rational” and in the end it boils down to little more than the invitation: “Be reasonable, think as I do!”

The moment we dismiss the naive notions of observation and/or reason, the notion of “correct” method involves one in a circularity. In order to decide which method would be a correct method of arriving at a true belief about the real world, one would have to know quite a lot as to what that real world is really like. Without such knowledge, there can be no telling what method would be the correct one. But it is precisely our ignorance of that real world and of what it is like that leads us to the search for the correct method.

The history of science provides countless examples of the absence of a correct method. Even a cursory examination of the “method” used by Galileo, Kepler, Newton, Darwin or Einstein will show that they had no real method at all. The most recent and best documented example is the history of the discovery of the molecular structure of DNA.

There was Rosalind Franklin who tried to avoid all adventure and kept making X-ray pictures of DNA, putting her trust in old Francis Bacon, that heaps and heaps of these pictures would ultimately yield knowledge of the molecular structure of the substance X-rayed. And all the while, there were Crick and Watson, wildly speculating and inventing haphazardly and making informed guesses and using Rosalind Franklin’s X-rays merely to confirm or disconfirm their hypotheses.

What makes us think, in the absence of a correct method, that the conclusions of all these people were not superstitions, is the fact that once they had made their discoveries, these discoveries have failed to be falsified. We owe this paramount insight into and understanding of the growth of knowledge to Karl Popper, whose classic book on the subject was first published in Vienna in 1935.

Since there is no correct method, there is no absolute distinction between a true belief and a superstition. At best, we can tell the difference after the discovery or the proposal of a solution has been made. A superstition, after it has been put forward, is either falsified or it is couched in the first place in such a form that nothing whatever could ever falsify it.

A true belief, on the other hand, is, at best, considered true because, although we know what would have to be the case for it to be false, it has so far not yet been falsified. A true belief is only provisionally and hypothetically true and is, for this reason, not absolutely different from a superstition.

Postmodernists

Unfortunately this lack of an absolute difference between superstition and true belief has been exploited by a host of contemporary philosophers — the so-called postmodern or post-structuralist philosophers (Feyerabend, Rorty, Derrida, Foucault, Lyotard, to mention the most famous ones) — who relish telling us that we might just as well hold any belief we like, that there is no difference at all between superstition and true belief, no difference between science and fiction, and that people who claim their superstitions to be “science” are nothing but arrogant imperialists who use their power to ram their superstitions down their victims’ throats.

These “thinkers” maintain that “science” is nothing but the mythology of Western people. They sum all this up by saying that all beliefs, including those we call scientific beliefs, are social constructions and that their chief purpose is not to understand the world, but to act as ideologies which legitimise the exploitation and oppression of minorities, other races, or, in general, of whatever people we dislike. Bigoted heterosexuals construct beliefs which legitimise the persecution of homosexuals, male chauvinists construct beliefs to validate the oppression of women, and so forth.

In New Zealand we have to be specially wary of these postmodern “thinkers” because if we follow them we will end up believing that there is no difference between the myth of Kupe and the theory of Continental Drift. In the so-called minds of these postmodern “thinkers” the theory of Continental Drift is nothing but a belief employed by Europeans to put down people who believe that the North Island was fished up from the bottom of the ocean by Maui.

In spite of the faddishness of these so-called thinkers, who are now riding on a wave of popular acclaim because they make any group with the weirdest superstitions feel “culturally safe”, there is a very hard way of telling the relative, though never the absolute, difference between a superstition and a true belief. The more a belief coheres with other beliefs, the more scientific it is likely to be. The less it coheres and the more parochial it is, the less scientific it is likely to be.

By this standard, the concept of, for example, “Maori Science” (a course of which is part of the curriculum at Victoria University in Wellington!) is a contradiction in terms. If it is parochially Maori, it can, by definition, not be science; and if it is science, it cannot be specific to Maori. This is not to say that Maori had no science, but such science should be called “science among the Maori” not “Maori Science”. People who think of “Maori Science” ought to be reminded of the genocidal mischief caused in the middle of our century in Europe by the notion of “German (i.e., non-Jewish) Physics”.

A belief which claims to be scientific must always be open to criticism, and can never be shielded from criticism on the grounds that it ought to be respected because it is culturally ensconced in an ethnic group. The real obstacle to the progression of scientific knowledge, therefore, is not the absence of a correct method of finding it, but the demand that certain beliefs ought to be exempt from criticism on the grounds of cultural safety.

Superstitions which are parochial, however, do fulfill a social function. They function as charters of societies and hold those societies together as cooperating units and promote solidarity. This is, of course, more true of tribal or primitive societies than of modern, urban and industrial societies. In primitive societies we get the almost paradoxical situation in which a parochial superstitious belief is socially, though not cognitively, more efficient than a non-parochial, scientific belief.

Social Climate

The reason for this seeming paradox is quite easy to grasp. A society has to have boundaries and exclude lots of people. A parochial superstition is more likely to function well as such an exclusion principle than a more scientific belief which coheres with lots and lots of other beliefs.

A scientific belief can never function as an exclusion and boundary-defining principle. There is only one truth, but there are at least as many false beliefs as there are societies. One society could form itself around the belief that insects have nine legs; another, around the principle that insects have ten legs, and so forth. The society which, on the other hand, consists of people who believe that insects have six legs would include just about everybody. The true belief about insects could never be used as a boundary defining principle.

By the standards of evolution, one would expect that societies based on subscription to false belief would not last long, because they might waste their energy praying for rain rather than digging trenches for irrigation. But here again we come across another seeming paradox. The society based on the belief that rain comes from prayer is likely to be a society with strong social bonds and a good feeling of solidarity. That solidarity will make it more able to fend for itself and to compensate for its lack of true knowledge. It may lack food because prayer does not bring rain, but it will make a solid fighting force which can rob food from other people.

Parochial, false beliefs are not a good adaptation to the environment, but they are obliquely or indirectly adaptive because they are a good cement for the formation of the solidarity of robber gangs which can help themselves to food by other means. Such superstition-based societies have great staying power even though they are not good adaptations to the environment. Hence myths and superstitions are not likely ever to die out. Faith-healing may not be a cure for cancer, but it makes a good support group for cancer patients. Table-rapping may not be a suitable form of communicating with departed spirits, but it does make for conviviality.

For further discussion see two books by Peter Münz: Our Knowledge of the Growth of Knowledge, London, Routledge, 1985; Philosophical Darwinism, London, Routledge, 1993; and the following papers: “Popper’s Contribution to the 20th Century”, New Zealand Science Review, 48, 1991; “What is Postmodern, Anyway?” Philosophy and Literature, 16, 1992; “Anne Salmond’s `Two Worlds’ in Postmodern Fancy Dress”, New Zealand Journal of History, forthcoming, 1994.

The Crackpot Index

On open access computer bulletin boards, any entity with a theory can expound on it at length. Many do — usually to a very unappreciative audience. A seemingly-large proportion of such expositions are surprisingly similar in style. The following scale (tentatively attributed to John Baez of Usenet sci.physics) will help readers establish just how crackpotted something is…

A simple method for rating potentially revolutionary contributions to physics.

  1. A -5 point starting credit.
  2. 1 point for every statement that is widely agreed on to be false.
  3. 2 points for every statement that is logically inconsistent.
  4. 5 points for each such statement that is adhered to despite careful correction [by other readers].
  5. 5 points for using a thought experiment that contradicts the results of a widely accepted real experiment.
  6. 5 points for each word in all capital letters (except for those with defective keyboards).
  7. 10 points for each claim that quantum mechanics is fundamentally misguided (without good evidence).
  8. 10 points for each favorable comparison of oneself to Einstein, or claim that special or general relativity are fundamentally misguided (without good evidence).
  9. 10 points for pointing out that one has gone to school, as if this were evidence of sanity.
  10. 20 points for suggesting that you deserve a Nobel prize.
  11. 20 points for each favorable comparison of oneself to Newton or claim that classical mechanics is fundamentally misguided (without evidence).
  12. 20 points for every use of science fiction works or myths as if they were fact.
  13. 20 points for defending yourself by bringing up (real or imagined) ridicule accorded to one’s past theories.
  14. 30 points for each favorable comparison of oneself to Galileo, claims that the Inquisition is hard at work on one’s case, etc..
  15. 30 points for claiming that the “scientific establishment” is engaged in a “conspiracy” to prevent one’s work from gaining its well-deserved fame, or suchlike.
  16. 40 points for claiming one has a revolutionary theory but giving no concrete testable predictions.