How randomised controlled trials can save the world

This is a transcript of a talk given at the Skeptics conference in Auckland last year. Parts of it were also presented at the inaugral lecture for Bruce Arroll on being appointed to a personal chair last October. The title of that talk was Highways Through Uncertainty and will be published in the NZ Family Physician in early 2007. This paper can be found on the internet at www.rnzcgp.org.nz

While I thought my research career had started in 1984, when I published a review on episiotomy in the Canadian Family Physician, it actually started with a question in 1974. At that time I was a medical student in my second year at the University of Auckland. Part of the course involved a child and family study, which involved meeting a woman who was pregnant, attending the delivery and then following the mother and child for two years. I had met the mother at the outpatient clinic in National Women’s Hospital. At 2am on a cool May morning I received a telephone call telling me the study mother was in labour at National Women’s Hospital and that I should make my way there. I was a little sleepy when I arrived at the delivery suite and, never having seen a live birth, I was somewhat overwhelmed by the atmosphere of the hospital, the bright lights, warm rooms and people running around in a busy state. I was not sure what was happening for most of the second stage but just as the baby was about to be born the registrar injected the perineum (area between the legs) with local anaesthetic and picked up a pair of scissors and performed an episiotomy (ie a cut in to the vagina to widen the birth canal ostensibly to expedite the birth of the baby. I was completely unprepared for this and, while the mother and baby were fine, I needed ‘resuscitating’. I was completely shocked by this procedure. The interesting aspect of this was that I did not discuss what had happened with my fellow students and have often wondered how often health professional students have traumatic experiences that they don’t talk about.

National Women’s Hospital

In 1980 I started the Diploma of Obstetrics at National Women’s Hospital. At that time it seemed that every woman having a baby would have an episiotomy. At the same time there was a home birth movement that was reasonably active in Auckland where virtually no woman received an episiotomy. To me this was difficult to explain. I asked many of the consultants and they defended the episiotomy procedure saying that while it was possible to preserve the perineum without performing an episiotomy the woman was likely to experience incontinence later in life. This was partly but not completely convincing to me.

McMaster University Canada 1981 Family Medicine Programme

I had been interested in the undergraduate medical programme at McMaster Medical School as a medical student as I knew it was problem-based and that they had no examinations. In 1980 I was keen to do some training overseas and was looking at doing a masters programme. I wrote to a number of programmes and one of the few that acknowledged me was the Family Medicine Department at McMaster University in Hamilton, Ontario. They said they did not think I was ready for a masters programme but would I consider being a resident (registrar) in family medicine. Needless to say I jumped at the idea of being able to work in Canada. While I was there one of the tutors was Brian Hutchinson who ran a course on critical appraisal. These papers became the ‘How to Read Clinical Journals’ in the Canadian Medical Association Journal. It turned out that McMaster was the centre of clinical epidemiology, which later became known as evidence based medicine. This course gave me some rudimentary skills in critical appraisal.

Lillooet-Central British Columbia

After leaving McMaster University I worked in Fort St James in northern British Columbia in Canada and after about 10 months there moved to Lillooet in south central British Columbia. There I worked with three other family doctors and we ran a small (about 20-bed) hospital. The two senior doctors did anaesthetics and surgery as was common in small towns in Canada at the time. During the evenings and in my spare time I started to research the issue of episiotomy. To get the literature searches and articles I needed I had to write to the Library of the College of Family Physicians of Canada, which was 3000 miles away in London, Ontario. I could find no clinical trials of episiotomy and it appeared to have become a standard practice in the 1920-30s based on a ‘good idea’ rather than on any evidence. I even found a number of articles from Washington State that reported a number of deaths from necrotising fasciitis as a result of episiotomy. In 1984 I published an article titled ‘Episiotomy in low risk patients’ in the Canadian Family Physician. It was the first publication I had written.At the time I did not know there was a prize, known as the literary prize, for the best article in that journal. My paper won it and I often feel that I am like a gambler who wins a big prize early on and goes on to be a problem gambler. This was the case with me, in that I came to like doing systematic reviews and have done many in my career. About the time my paper was published the first randomised controlled trial was completed and this found that a restrictive policy for episiotomy was no more harmful in the short term than a non-restrictive policy. I did not realise at that time that the final author on that randomised trial was Iain Chalmers whom I was to meet 10 years later in Canberra where he was helping the Australians set up their Cochrane Centre. Iain is one of the truly admirable people that I have met in my clinical career. Later research showed that there did not seem to be any long-term problems in terms of incontinence or anything else. Ironically, my first paper received more dissemination than any article I have since written, as a Canadian newspaper chain reported the article and it went to every province in Canada.

One highway through uncertainty

A Cochrane review on the topic of episiotomy reported that there appeared to be a number of benefits to a restrictive policy of episiotomy in terms of pain, suturing and posterior vaginal trauma with an increase in anterior vaginal trauma. I present the Cochrane database as an example of an electronic reference source that provides clinicians with a pathway through uncertainty. The Cochrane database contains only randomised controlled trials. They are becoming increasingly important in modern clinical medicine as they enable us to determine if a treatment is effective and how effective it is. Prior to 2002 it was thought that merely following up a group of patients on treatment and comparing them to those not on treatment would suffice to determine the efficacy of a treatment. A specific example is that of hormone replacement therapy. It was thought that this medication would reduce heart disease in women who took this medication. The cohort of follow-up studies of those who chose to take hormone replacement therapy reported a benefit. However when the large Womens’ Health Initiative study (a randomised controlled trial) was published it showed an increase in heart disease and stroke. It was clear that the apparent health benefits of hormone replacement therapy were due to the fact that it was relatively ‘healthy, wealthy and educated’ women who were given hormone replacement therapy and these women had lower rates of heart disease. Why aren’t randomised trials conducted more often? In recent years there has been an explosion of randomised controlled trials. However they are relatively expensive and time consuming to conduct and this limits their universal usage.

Antibiotics and respiratory tract infections

In 1996 my brother-in-law had a cold and went to his doctor. He was given the broad spectrum antibiotic Augmentin. Two days later he still had symptoms of his cold but now had diarrhoea from the Augmentin. I decided at that point that my mission in life would be to decrease the use of antibiotics for respiratory tract infections. The situation has improved considerably since 1996 when there were 1.2 million prescriptions of Augmentin filled in New Zealand. By 2003 it had decreased to 0.6 million. The cost of antibiotics in 1998 was $36m and by 2003 had fallen to $16m. This was a decrease in both volume and unit cost and a tribute to the efforts of Pharmac (Pharmac annual report 2003).

The antibiotic state of the nation


This is possibly not as good as it could be. A study conducted by Pauline Norris at the School of Pharmacy at Otago University found that 42 percent of the population of a small New Zealand town (population about 12,000) collected a prescription for antibiotics in 2002. I found this figure quite alarming and discussed it with Professor Chris Van Weel from Neimegen University in the Netherlands. He thought the rate of antibiotic use in his country was about threepercent and thought that was too high. I am not sure what the ideal level is but I suspect it should be less than 10 percent.

Our department has done some work on the patient and GP issues for antibiotics. In a comparison of patients 82 percent saw a GP in 1998 to get an antibiotic for a respiratory tract infection and this had decreased to 57 percent in 2003. A survey of GPs over the same period found that 77 percent were prescribing fewer antibiotics than in 1998, two percent more and 21 percent no change. These are encouraging trends. This work triggered interest in the use of delayed prescriptions. We have since published a randomised controlled trial, a systematic review, an editorial and a qualitative study on this topic. The most interesting part of the systematic review was the 75 percent reduction in the use of antibiotics for otitis media in children aged over the age of two years. This has led to a major change in the practice of giving a routine antibiotic in patients with acute otitis media.

Our group conducted a qualitative study on patients who had been in our delayed prescription randomised trial. This was my first involvement with a qualitative study. Two insights emerged from this. We found two GPs who no longer used delayed prescriptions as they had already ‘trained’ their patients not to expect antibiotics for respiratory symptoms. These two GPs would have appeared in a regular questionnaire as non-users and we would not have been aware that there is the opportunity to train a practice. The other insight was that resistance to antibiotics by micro-organisms was an issue for doctors but not for patients.

Augmentin free office

As a result of this work, Dr Tana Fishman and I have created the ‘Augmentin free office’ where, if a doctor wants to prescribe Augmentin, he or she needs to explain the clinical circumstances to a colleague to get their ‘permission’ to prescribe it. This reinforces with students that it is a very broad spectrum drug that should be used with caution.

Friend’s story

I have a story to tell about a friend of mine. She is a 52-year-old European woman who had a cold for a week and during that time she developed bilateral ear pain. She went and saw a young doctor at an accident and medical clinic (whom I hope may have been a house surgeon moonlighting). He could not see her ear drums and said, “I think you have an ear infection”. She was given a week of amoxicillin and given careful instructions on how to take it. She was very satisfied with her care. She called me one week later to say that her ear pain had not gone away and on the phone I told her that she did not have an ear infection and the antibiotics would have done nothing for her. I went around to her place to see her and both her ear drums looked perfectly normal and she had probably never had an ear infection in her life. When I tell students this story I say that she died on day two from the antibiotics (this is not true) to highlight the fact that every year in New Zealand about one to two people die from everyday antibiotics that we use, eg trimethoprim, doxycycline, amoxicillin and augmentin to name a few. The causes are usually due to effects on the bone marrow or liver rather than allergy (the CARM group have not reported results since 1997 but up until then there were at least one or two deaths each year). In terms of pre-test probabilities my friend had a 0.000001 percent chance of having an acute otitis media while her chances of having a bilateral eustachian tube dysfunction would be about 99.999999 percent. Unless the clinician saw a bulging drum this would not change the situation.

Where do these pre-test probabilities come from? They come in my case from experience or from the literature. For the young doctor I can only presume that all the cases of ear pain he had seen were probably otitis media as he would be unlikely to see eustachian tube dysfunction in a hospital setting.

Another highway through uncertainty-numbers needed to treat


Our young doctor in the after hours clinic may have found it helpful to know that the numbers needed to treat to reduce pain at day two in a child with otitis media is 17. If he had thought that is what my friend had had he may have been less enthusiastic about giving her an antibiotic. There is no data in adults as it is such a rare condition. I recently had a medical student say to me that a treatment was moderately effective for the condition we were discussing. My reply to this was what did he mean by ‘moderately effective’.

I would like to suggest that we sometimes need numbers to communicate effectively with our colleagues. There is evidence for this from a letter from the New England Journal of Medicine in 1980. There they asked physicians what they thought the percent risk would be if a person had a moderate risk and they answered between 20 percent and 75 percent. For pathognomonic (which I always thought was absolutely certain) the range was 55 percent to 100 percent. For high probability this was 55 percent to 95 percent. So you can see that we can mean quite different things with words that we think are communicating what we are thinking.

There is controversy about using numbers needed to treat as these can differ considerably from study to study for similar conditions. However, for a person who has had a heart attack or angina, 11 people need to take the powerful cholesterol lowering drug, simvastatin, to prevent one new cardiovascular event. This a medication that most doctors think is a fantastic drug, yet most individuals taking it will not benefit from it. The same goes for antidepressant medication, for which about eight patients need to take an antidepressant for 8-12 weeks to get a remission, so again, most people will not benefit. About half of those eight will get better as a result of the placebo and, potentially, there are another three who may not benefit from this treatment but may get better from something else. There are some who may not get better from any treatment.

I like to mention to students that most people don’t get better from most medications, as I feel there is an impression among students and colleagues that ‘everyone’ gets better with antidepressants or antibiotics. I will present an example of some work Dr Tim Kenealy and I have done on acute purulent rhinitis. You may wonder why we are interested in coloured mucus coming from the nose. It is because this is the major predictor of antibiotic use in the United States. We recently published a systematic review on antibiotics for this condition in the BMJ. The numbers needed to treat to improve this condition is between seven and 15. Our recommendation was that as this is not usually a serious condition we would suggest not using antibiotics initially. As with the other medications most people do not benefit from treatment.

Summary

When seeing a doctor for whatever condition you have always ask about the benefits and harms of treatment and the benefits and harms of no treatment. If necessary ask what the randomised trials show and be sceptical.

References available from the editor.