Table 2 'Statistical non-significance'

Figure 1 illustrates two problems that arise when results are classified as 'statistically non-significant' or 'negative':

1. The classification is based on an arbitrary cut-off. The results of Study 1, for example, are marginally different from the results of Study 2. But by using the conventional cut-off of P < 0.05, the results of Study 1 are ranked as 'statistically significant' and the results of Study 2 as 'statistically non-significant'.

2. 'Statistically non-significant' results may or may not be inconclusive. If the short green vertical line in the figure below indicates the smallest effect considered important, the results for Study 3 would be conclusive, since an important impact is highly unlikely. The results for Study 4 would be categorised as 'inconclusive' since it is not unlikely that there would be an important impact (the 95% confidence interval crosses the threshold for what is considered to be an important effect). Both results, however, might be regarded as 'statistically non-significant' or 'negative'.