The objective of non-inferiority trials is to compare a novel treatment to an active treatment, with a view of demonstrating that it isn’t clinically worse with regard to a specified endpoint. As treatments improve, showing superiority of a new therapy becomes more and more difficult because incremental improvements are ever smaller, whereas showing non-inferiority becomes ever easier. Going further down this wormhole, in many conditions (as in oncology), a treatment may be very effective in a minority but doesn’t outperform a placebo in all comers. Imagine how easy it would be to produce a study of a novel medication that doesn’t have to prove it’s better in that minority but instead shows itself to be non-inferior to the older treatment for all comers—non-inferior to placebo overall?
It’s important to remember that the non-inferiority trial does not have to show it’s as good as the old treatment (ie, equivalence)—it just has to show that the new intervention is “not unacceptably worse” than the intervention used as the control.1 As editor in chief of the Canadian Journal of Emergency Medicine, I believe that non-inferiority trials are of a quality that does not produce valid enough data to warrant publication.
A CLOSER LOOK AT NON-INFERIORITY
Given the above, why would we ever want to see a non-inferiority trial? Some theoretical reasons to show non-inferiority are:
- Marketing: It’s no worse but more “convenient” to take (once a day instead of twice a day, for example).
- Marketing: The adverse-effect profile is superior so that quality of life is potentially better.
- Safety: Its benefit isn’t unacceptably worse, but fewer people suffer harm.
There are inherent problems to this approach. The authors themselves define what they mean by “not unacceptably worse” (non-inferior). Since that margin of difference is essentially based on the opinion of the biased authors rather than on any consensus clinical outcome, it leads us to a slippery slope of sample-size calculation. If the authors theorize that a 10 percent worse outcome is acceptable (as compared to 5 percent), they need to recruit fewer patients. When calculating sample size for a study, the smaller the difference between the two groups, the greater the number of patients required to detect that difference. In a non-inferiority study, making the assumption that there is a larger difference means you need to recruit fewer patients. If the authors choose less restrictive inclusion criteria (eg, all people with chronic heart failure [CHF] instead of only those with CHF from coronary artery disease), recruitment is easier and requires fewer patients screened. Since the goal is to prove non-inferiority, fewer patients means it is easier to not find a difference! That is, it must be non-inferior.