What should we make of the Lancet paper showing 655,000 excess deaths in Iraq? My immediate reaction is that it is poppycock. To say why it is nonsense requires statistical expertise beyond my own. Readers can make their own minds up by going to websites here or here or here or here or here or here.
Statistical evidence is beyond the understanding of most people. Some people believe that statistics don't lie, other believe that there are lies, damned lies and statistics. I rather take the view that things that are obviously true don't need statistics to prove them, but statistical evidence is easlily abused by those with a strong point of view such as pharmaceutical companies who want to sell their wares, politicians who want to mislead the public and scientists who want to get published for the sake of career advancement. A short while ago a paper published in the BMJ claimed that only 30% of scientific papers used statistics correctly.
So I like to subject statistical claims to a reality check. Here is a quote from the first of the links:
If you finished a cohort study which concluded that the forest, by extrapolation was denuded yet aerial photography did not show any logged over areas then you would be exactly in the position of accepting Burnham's proposition that 655,000 people died without observing the masses of widows, orphans, mass graves and bomb craters that one would expect to accompany such an enormous loss. My parents were unfortunate enough to go through a major urban battle. It only killed 100K people but what it did was produce a flood of refugees and displaced persons whose existence was palpable. One quarter of Burnham's sample is from Baghdad. Where are the refugees? It is not widely appreciated but Iraq is host to millions of Shi'ite pilgrims every year. There is not a single historical instance I can think of where tourists continued to visit a country beset on the scale claimed.
Now let's consider Mr. Burnham's claim that "Overall, 13% of deaths were attributed to airstrikes" because this gives us a window into his data. A sample of his sample, so to speak. That's 85,000 deaths from airstrikes. The problem with this is that airstrikes virtually ceased since 2004. Now maybe "airstrikes" is really understood to be all kinds of fire. Could this be true?
Where would these airstrikes have happened? Where Burnham sampled, of course because he claims these deaths are backed by death certificates. Now recall that of his 47 clusters, 12 were in Baghdad, 2 were in Basra, and 3 were in Anbar. But wait! Basra is in the British sector and there have been no air missions in Basra to speak of. As for Baghdad, if which has 1/4 of the clusters, at least 20,000 people would have died in airstrikes within full view of the media since 2004. Anbar is the place where most airstrikes would probably have occured. But it's a sparsely populated area and contains 3/47 clusters. Could it be that most of the airstrikes were there?
So we come to the question of what his snapshots really look like. And as I said, I am disappointed.
Finally, it is worth looking at where the information is coming from. The Lancet under Rchard Horton has assumed a stance to the left of the Independent. It has always had a reputation for playing fast and loose with the peerr reviewing process, preferring to publish what is 'newsy' rather than what is scientifically correct, witness the MMR scandal. Richard Horton, himself, goes on 'Stop the War' marches. Under his editorship it has sunk from being the permier medical journal to perhaps the 3rd or 4th.
So, for many reasons, even without statistical expertise I think the Lancet article is poppycock.