Factors Which Influence Sample Size Calculation

Created on Wed, 04/19/2017 - 04:05
Last updated on Wed, 04/19/2017 - 04:05

Next Chapter:

This chapter answers parts from Section A(d) of the Primary Syllabus, "Describe bias, types of error, confounding factors and sample size calculations, and the factors that influence them )".  This topic was examined in Question 2 (p.2) from the first paper of 2009.  It is expanded upon in the Required Reading chapter for the Part II exam ("Study power, population and sample size").

In summary, calculation of sample size involves the following factors:

  • Alpha value: the level of significance (normally 0.05); i.e. the level of probability you accept as "real", i.e. not due to chance.
  • Beta-value: the power (normally 0.2), i.e. the percentage chance of detecting a treatment effect if there actually is one. 
  • The statistical test you plan to use
  • The variance of the population (the greater the variance, the larger the sample size)
  • The effect size (the smaller the effect size, the larger the required sample)
  • The control group outcome rate: How many of the control group are expected to develop the treatment effect.
  • Study design, i.e. is it an RCT? In randomised controlled trials, there is an additional benefit to randomisation which develops above a certain sample size (N=200).

How many patients does my trial need?

That depends on several factors.

The magnitude of the treatment effect: The larger the effect, the smaller the required sample size. For a truly tiny treatment effect, one would require truly massive numbers.

The control group outcome rate: How many of the control group are expected to develop the treatment effect.

The agreed-upon significance level (alpha): The level of probability you accept as "real", i.e. not due to chance. The greater your demands for significance, the larger the number of patients needs to be enrolled.

The power (beta): the percentage chance of detecting a treatment effect if there actually is one. This is something you decide upon before commencing the trial; the higher the power value, the more patients you will need. Typically, beta is 0.8, so there is a 20% chance (1-beta) of commiting a Type 2 error , or a "false negative".

Obviously, if your trial has too few patients, you are more likely to commit a Type 2 error. The negative results of this trial will force you to discard a treatment which does actually have a beneficial effect, an effect which you and your tiny useless trial have failed to reveal.

The concept of statistical efficiency demands that the randomised controlled trial achieve its goal (discriminating the treatment effect) with the smallest possible number of patients. However, there is probably a minimum.

In randomised controlled trials, there is an additional benefit to randomisation which develops above a certain sample size (N=200). This is the benefit of randomization, which ensures an approximately equal distribution of unknown confounding factors (such as weird genetic variations and other such unpredictable things). In trials smaller than N=200, this effect of randomisation can no longer be relied upon- one simply cannot guarantee that one group is sufficiently similar to the other group in its incidence of unpredictable features.

 

References

Cohen, Jacob. "Statistical power analysis." Current directions in psychological science 1.3 (1992): 98-101.

Cook, Richard J., and David L. Sackett. "The number needed to treat: a clinically useful measure of treatment effect." Bmj 310.6977 (1995): 452-454.