Evidence-based decisions

This from today's New Scientist: We expect medical therapies to undergo rigorous trials to ensure they are safe and effective. Yet we seem content to let our leaders conjure up policies based on what sounds good, rather than on what has been proved to work. The effectiveness of policies in many areas, from education and crime to reducing greenhouse gas emissions, can be empirically determined. As in medicine, the best evidence comes from randomised controlled trials; better still, a systematic review of multiple randomised trials.

As usual the doctors are decades ahead of the politicians. Over the years we have suffered form adopting policies in education, crime, public order, alcohol and tobacco use as well as other drugs' policy, and even military decisions that are based on people's opinions rather than evidence. Even worse is when public policy is based on a single anecdote that has been trumpeted in the Tabloid newspapers.

The nature of news is that it is so unusual as to catch the public eye. For example a few years ago a picture appeared in the pages of the BMJ of an X-ray showing a huge butcher's knife embedded in a person's skull; despite the horror of the picture, the patients survived unscathed. It was such a startling picture that it was incorporated into the story line of ER. Supposing public policy was based on that news story. An exception should be made to knife crime legislation because butchers knives inserted into skulls cause no lasting damage. Ridiculous, isn't it. Yet the plural of anecdote is not evidence.

Some years ago two little girls were murdered at a small Cambridgeshire village called Soham. Ian Huntly, their schools janitor was convicted of the murder and it was later established that Huntly had previously been investigated for but not charged in connection with one act of indecent assault, four acts of underage sex and three rapes. This information was unavailable to the school authorities when they appointed Huntley.

This horrific and obviously quite unusual case, triggered legislation that insisted that anyone appointed to a post which involved working with children must have a criminal records check which would uncover, not only crimes for which they were convicted, but also for which they had been arrested but not charged. For example, even though I do not treat children, the private hospital at which I did the occasional session insisted that I have a check on the off chance that I might encounter a child there. Scoutmasters, coach drivers, and Sunday school teachers all have to be checked and the government has recently extended the scope of the legislation so that even moms giving their kid's friends a lift to soccer training need to be checked. These checks cost about $50 and someone has to pay - I had to pay for my own check for the hospital. For every new activity a new check is required so for teaching Sunday School I had to be checked again. It has been estimated that one adult in 4 will need checking at a cost of about $250 million a year.

Had this been medicine NICE would have ruled it as unaffordable because 1 QALY would cost more than £30,000. The same would be true for crash barriers between carriageways on motorways (£1 million for every life saved) or the automated train warning system which stops trains passing through red lights (£3 million for every life saved). These systems may seem like a good idea, but they certainly aren't evidence based.