The researchers doing non-inferiority trials are therefore “encouraged” to be less rigorous in their methodology and assumptions!

In general, non-inferiority trials require fewer patients than a standardized superiority trial. So hold on then: The vast majority of superiority trials aren’t large enough to assess the number needed to harm (NNH)—only the number needed to treat (NNT). Superiority trials can successfully reject the null hypothesis (stating that there is no relationship between specified groups, with any observed differences due to sampling or experiment errors) and demonstrate superiority.

In a 2014 The BMJ article, 76 percent of studies in a Cochrane cohort failed to report adequately on harm. Even in studies reporting adverse events, 41 percent failed to adequately report outcomes related to harm.2 Non-inferiority studies, which on average recruit fewer patients into a trial than a superiority trial, consistently fail to report NNH for three reasons:

1. It isn’t in the interest of the pharmaceutical industry to publish data about harm when they cannot even show equivalency.
2. The sample sizes used are insufficient to accrue enough data to calculate NNH.
3. The number to be recruited would increase the cost of doing the study to the point of being outside the budget of most researchers.

To confuse the reader more, given that the researcher chooses the confidence interval for the margin of efficacy (it isn’t as efficacious but within our assumed limits to be non-inferior), a result of such a study may simultaneously demonstrate that the novel treatment is both non-inferior and inferior at the same time. (The opposite of non-inferior is “not non-inferior” rather than “inferior.”)1 Such results are never reported by the authors of non-inferiority trials.

A confidence interval basically is used like in a poll: We chose 1,000 people for the poll, which makes our result valid ±0.3 percent. (Confidence interval is roughly the equivalent of 3/n, where “n” is the number of people recruited). If you assume a difference required to make it non-inferior as being larger than what the difference actually is (wide confidence intervals), the normal study calculation may find that your arm is indeed inferior but still within the range of what you chose as being non-inferior. Remember: You choose a difference of performance that you declare as acceptable and therefore non-inferior. That does not mean analysis may not actually find it inferior but still within your assumed range that you declared at the start of the study.