I met Eli Estey on a trip to MD Anderson in Houston. I found him to be a charming and thoughtful guy. He has now moved to the 'Hutch' in Seattle and has penned an interesting article entitled "Do commonly used clinical trial designs reflect clinical reality?" in the journal Haematologica.
We normally rely on randomized phase III clinical trials which compare two treatments using endpoints like overall survival, progression-free survival or time to next treatment. We rely heavily on p<0.05 which means that there is less than a 1 in 20 chance that any difference between the two could have occurred accidentally. In other words we are pretty sure that any positive result is not a false positive. In leukemia trials there is a tradition, which comes from St Jude's Memphis, that we inch forwards in our improvements. Childhood ALL is curable in the majority of cases. Treatment involves the use of several drugs in a particular order over a couple of years. Successive trials made small changes to the protocol that gave small improvements in survival. In order to be sure that these small improvements were not false positives the trials needed to utilize large numbers of patients so that a 'p' value of <0.05 could be obtained. Childhood ALL trials started with drugs that gave a high rate of complete remissions (vincristine and prednisone) so there was a good chance that adding more chemotherapy would improve things, but not by much. Hence, it was necessary to be certain that adding these new drugs did make a difference and were not false positives.
With adults, the problem is not that we are doing pretty well already, but that we don't cure very many at all. Therefore, our problem is not that we have too many false positives; it is that there may be drugs out there that help, but we reject them because we have too many false negatives.
The cure rate for CLL and adult AML is so poor that we are not really interested in drugs that make a small difference - we want agents that make a big difference - we should be willing to accept a higher risk of a false positive.
When trials are designed the numbers needed in each arm are partially determined by the power calculation. Traditionally an 80% power is required. What does this mean? It is actually the false negative rate. It means that one trial in five is discarding a treatment incorrectly because the negative answer was a false one.
Estey argues that this way of doing things is not clinically relevant. On the contrary there have been quite massive improvements in adult leukemia that did not come from these sorts of trials at all. He cites the value of imatinib in CML, CDA in hairy cell leukemia, arsenic trioxide and ATRA in acute promyelocytic leukemia and high-dose ara-C for core-binding factor AML as examples where the effect was so large that it didn't need an enormous trial to detect its value.
I raise this problem for CLL recently in this way. John Byrd recently reported that there are 107 different agents being studied at teh preclinical and clinical level for teh treatment of CLL. The typical phase III trial in CLL requires 800+ patients. There are simply not enough patients to test all those agent singly let alone in combination. WE must find a better way of doing things.
The better way that Estey proposes is the use of Bayesian statistics. Thomas Bayes (c. 1702 – 17 April 1761) was a British mathematician and Presbyterian minister, known for having formulated a specific case of the theorem that bears his name: Bayes' theorem, which was published posthumously. I have read the paper several times but I am none the wiser. Would someone please explain Baysian statistics to me.