In the second of a two-part series, Jim Ring looks at what evidence means to different people
Scientific evidence is often difficult to interpret, in medicine in particular. ‘An Unfortunate Experiment’ was the title given to the treatment for some women after screening for cervical cancer. In this case science was considered by the legal profession and apparently found wanting. The doctor involved was castigated and publicly humiliated for experimenting on humans. But no real experiments were ever done; it appeared he did not understand scientific methodology. Neither did the journalists and legal people involved. The point is that no proper controls were used so it was very poor science.
Were the women disadvantaged? It is difficult to tell, but many were certainly outraged. It generally escaped notice that the surgeon was responding to public pressure for less radical surgery and that a group of patients involved seem to have had on average a slightly better outcome than the norm.
One of the most unfortunate ideas that came out of the long legal case was the emphasis on privacy for the individuals involved which implied their records should not be available for medical study. There is a difference between privacy and anonymity. It is very important to explain to those involved in medical procedures that for medicine to progress it is essential to collect data. Women appeared on TV complaining bitterly that they had been used in an experiment without their consent. But all good medicine is experimental.
We are not much closer to determining whether mass screening for cervical cancer does improve the chances for the screened population and now we have another scandal in New Zealand. Public expectation of screening programs is far in excess of what they can deliver. Efforts to sue Dr Bottrill, and compensation claims from ACC, seem to imply that patients think a false negative reading is necessarily medical error. Women have appeared on TV claiming their lives have been devastated because they had a false negative. Surely this is wrong; they are rightly upset but this is because further tests show a medical problem. Of course some who died might have been saved if an early intervention had resulted from a correct positive reading; however this does not seem to be the main thrust of their complaint.
False Negatives vs False Positives
It is possible to reduce the number of false negative readings at the expense of an increase in the number of false positives. This may seem desirable but there is a cost. In Britain large numbers of women in a screening project reacted very badly to finding they might have a ‘pre-malignant’ condition. This included some members of the medical profession. There is a clear indication that patients were not well informed before screening.
Patients involved in any medical procedure are supposedly asked for their ‘informed consent’. It seems now obvious that ‘informed consent’ is largely lacking during mass screening for both cervical and breast cancer. Several of those involved in the public hearing are surprised to find that screening is less than 100 per cent accurate. All mass screening procedures are likely to have a high error rate as they are designed to be rapid, cheap and simple; leading to more precise testing if there is a positive result. Is a large and expensive inquiry, using legal methods, a suitable way of investigating scientific questions?
Cervical cancer, unlike breast cancer, is strongly correlated with environmental factors. The former is very rare in the general population with a relatively high incidence in a certain sector. However it is politically incorrect to target the high-risk population for screening because the risk correlation is with such factors as poverty, poor hygiene and sexual promiscuity.
A recent case of a gynaecologist accused of misconduct raises some interesting issues. The unfortunate patient would seem to be outside the high-risk group for cervical cancer, thus an assumption may have been made that the correct diagnosis was very unlikely. But no physical vaginal examination was made. Feminist literature once strongly criticised the medical profession for over-use of this procedure, which one writer described as ‘legalised rape’. It would be interesting to know the rate at which this procedure is used today compared with, say, 30 years ago. Is the medical profession responding to crusades in a way that disadvantages patients?
Objections to trials
Medical ethicists – now a profession – have objected to various drug trials saying it is unethical to provide some patients with a placebo that will not improve their condition. This is in effect a claim to certain knowledge – that the drug being trialed is the ideal treatment. Patients receiving a placebo are not disadvantaged when the new drug may do more harm than good. We can sympathise with terminally ill patients who know that they will die in the absence of treatment and where anything seems a better bet than a placebo. But it is essential that drugs be properly tested before being used routinely.
Experiments have even been done in surgery. In 1959 patients were randomly assigned, but all prepared for surgery and the chest cavity opened. Only then did the surgeon open an envelope and follow the instruction; either to perform the procedure or immediately close the chest. Although some ethicists have objected (one stated that such surgery would never take place in the UK), a double-blind study of brain surgery was recently done in the US. Not only did it pass an ethics committee but patients welcomed the chance to take part even though it involved drilling the skulls of both real and placebo patients. In this case there was considerable improvement in those under 60 who had the real operation.
This indicates people may be willing to give consent to risky experiments providing they are given good information.
Most evidence in medicine comes not from experiments but from epidemiology. This requires the collection of huge amounts of data and sometimes produces conflicting results. Two populations, which differ only in the factor under investigation, should be matched and this is difficult to achieve. Recently, in a world-wide study, doubt has been cast on the efficacy of breast-cancer screening. New analysis purports to show that when populations are matched correctly, the screened population has no better chance of survival than an unscreened population.
Demands for safety
Some demand that all medical procedures should be ‘safe’, though curiously this is not required of alternative medicine. Suppose a new drug has fatal consequences for one patient in 100,000. It is quite likely that this will not be discovered during testing. Should such a tiny risk preclude the use of a drug that gives significant benefits to the vast majority of patients? New medicines are introduced when they show a clear advantage over a placebo. When very large numbers are involved in a study it is possible for a drug to show a significant advantage, yet not be worth introducing. Significance is a technical term and it is possible to find an advantage of only 0.1% is ‘significant’, though it may not be worth taking such a product.
It was this confusion that bedevilled early experiments on ESP. Rhine in America and Soal in England recorded the success of subjects guessing unseen cards. The experimenters wrongly assumed controls were unnecessary; instead they compared guesses with a theoretical chance result. A few subjects scored correct guesses at slightly more than chance and because huge numbers of guesses were involved, statistical tests showed these results had ‘significance’. That is, there was a huge probability that the guesses were not simply ‘lucky’.
Enthusiasts then made the enormous leap to say that because the guesses were not due to chance they must be due to a previously undiscovered human faculty, extra-sensory perception or ESP. Disinterested observers, not just skeptics, should have concluded that other explanations, such as poor experimental design, badly recorded results, fatigue, or just plain cheating were more likely. A great deal of time, money and effort was spent pursuing this will-o’-the-wisp.
