Surprising results from a US study of the effectiveness of counselling on reducing juvenile crime.
In the March NZ Skeptic, Dr John Welch’s excellent column mentioned an article in the British Medical Journal (BMJ) about a social experiment which started in 1939. I have not seen the BMJ article but it can only refer to the Cambridge-Somerville experiment. Not just because this was the only such study started in 1939, but it is still (to the best of my knowledge) the only large-scale, long-term study on the effects of counselling which can reasonably be regarded as good science.
It is worth looking at this famous experiment in a little more detail. The instigator was the Harvard Professor of Medicine and Social Ethics. The subjects were boys between the age of five and 13 thought to be “at risk” of juvenile delinquency. It was proposed that a programme be started to prevent these boys becoming delinquent. It would involve “all the aid that a resourceful counsellor could possibly give, backed by the school and community agencies”.
In fact it eventually involved churches, scouts, YMCA, and summer camps plus, where necessary, medical and psychiatric treatment. The counsellors were particularly concerned to involve the families of the boys and this was done whenever possible. The treatment programme was intensive and lengthy; on average it lasted five years — a considerable time in the life of a child.
Professor Cabot (who died in the year the project started), while convinced the programme would be valuable, was concerned that it should be properly assessed. Thus the boys were grouped into 325 matched pairs, each pair being similar in age, background, etc. One of each pair was randomly assigned to the treatment group, the other to the control. It is because of this that it was possible to decide “Did the treatment help?”.
Major papers on this study were published in 1949 and 1951 and the final paper, by Dr Joan McCord, was published in 1978. Some 253 of the matched pairs had completed the programme and 30 years after the project started, Dr McCord was able to locate 480 of the men involved.
About half of these were from the treatment group, and about two thirds of them felt the project had been helpful and improved their lives. Most had fond memories of their counsellors.
Dr Welch writes that the BMJ article found the treatment group to be “sicker, drunker, poorer and more criminal”. This is true but I think it important to note the individual differences were very small.
The project was started to prevent juvenile crime. Of the treatment group, 72 had a juvenile criminal record, compared with 67 from the control group. This is a very slight difference, but clearly the project failed in its main aim which was to prevent juvenile delinquency.
Similarly, 49 of the treatment group had been involved in serious adult crime, compared with 42 of the control group. Again a very slight difference. For factors such as recidivism, alcoholism, stress-related illness, and job satisfaction the pattern was similar. That is, the control group did better than those who were treated — but only by a very small amount.
In only one important way was the treatment group better — minor adult crime. But again the difference was very slight: 119 of the treatment group had minor criminal records, compared with 126 for the control group.
It is true, however, that taken together the differences between the two groups were found to be statistically significant. The treated group had been harmed by the treatment, although the harm was minimal and would not have been revealed by a small-scale study.
There are several major lessons for skeptics here. Firstly, all treatments should be properly assessed and that means using a control group (obviously in this kind of treatment “blind” studies are not possible). How much money (taxpayers’ money) is being spent in New Zealand on counselling? Is the money well spent? Is any attempt being made to assess the value of the treatment?
Secondly, the natural and powerful objections to such assessments must be resisted. The idea of using a control group horrifies many people — “But these people are being used in an experiment! They are not being treated!” Such objections assume we already know the treatment works. But we do not know this and our intuition may be completely wrong.
Thirdly, people are incapable of objectively assessing their own treatment. That is why testimonials to the healing power of any treatment are completely worthless.
Fourthly, non-intervention may be the best treatment. The problem is that it is the hardest to apply because there are powerful forces mobilised against it. The patient welcomes treatment (just how neglected did those boys in the control group feel?) and the professional wants to help.
Counselling is getting to be a major industry in New Zealand but its value should be questioned. All such professionals should adopt the motto “First do no harm”, but until proper assessments are made, how do they know whether they are doing harm or good?
