Levels of Evidence Table

Clinical Nurse SpecialistA Copyright © 2024 Wolters Kluwer Health, Inc. All rights reserved.

Using Research to Advance Nursing Practice

Column Editor: Sandra L. Siedlecki, PhD, RN, APRN-CNS, FAAN

How to Determine When Your Sample Size Is Big Enough But Not Too Big

Autho search The au Corres Depart Euclid A DOI: 1

Clinic

Sandra L. Siedlecki, PhD, RN, APRN-CNS, FAAN

KEY WORDS: research, sample, sampling

Perhaps one of the most daunting decisions to make when developing a protocol for a study is justifica- tion of the sample size. If you are lucky enough to

have access to a statistician, they can calculate a sample size for you. However, not everyone has this resource. Be- cause the clinical nurse specialist is involved inmany of the research projects conducted in an organization and is often the mentor or the lead researcher on these projects, it is helpful to have a good understanding of sample size. The purpose of this article is to define key terms (Table 1), explain important is- sues related to sample size, and to offer suggestions and re- sources for those nurseswhodonot have access to a statistician.

WHY IS SAMPLE SIZE IMPORTANT? The size of a sample will impact the reliability and validity of the findings of a study. A sample that is too small may re- sult in not finding a significant difference between treat- ment groups, when in fact a difference does exist (type II error).3 Likewise, a sample that is too big can make very small differences look statistically significant, when they are likely to be clinically insignificant (type I error).3 The trick is to arrive at the ideal sample size; like Goldilocks, the researcher wants a sample that is neither too big nor too small, but just right.

The correct sample size will optimize statistical power and have a smaller margin of error, making estimates valid and reliable. For those without statistical consultation ser- vices, it is important to note that many statisticians suggest

r Affiliations: Senior Nurse Scientist, Department of Nursing Re- and Innovation, Cleveland Clinic, Ohio. thor reports no conflicts of interest. pondence: Sandra L. Siedlecki, PhD, RN, APRN-CNS, FAAN, ment of Nursing Research and Innovation, Cleveland Clinic, 9500 ve, Cleveland, OH 44195 ([email protected]; [email protected]). 0.1097/NUR.0000000000000844

al Nurse SpecialistA

Copyright © 2024 Wolters Kluwer H

that for any study a sample of at least 100 is needed to pro- duce results that can be meaningful or valid. If the popula- tion of interest consists of fewer than 100 individuals, then every attempt should be made to survey all members of that population. On the opposite side of the spectrum, stat- isticians warn that sample sizes that exceed 1000 may re- sult in spurious findings.3

The best sample size may lay between 200 and 400, but this is arbitrary and depends on the design, the number of groups, the number of variables, and the level of measure- ment for the variables. The precision of study estimates is related to the square root of the sample size. Thus, to dou- ble precision, the sample size would be quadrupled.1

However, after the sample reaches 300 or more, there are diminishing returns for continuing to increase the sample size. So, considerations related to time, use of resources, and burden on subjects need to be addressed when deter- mining the best sample size.

SAMPLE SIZE CONSIDERATIONS Knowledge of the available sample is critical to determin- ing the best sampling technique and sample size required to answer the research question(s). Table 2 provides some information formaking sampling decisions. For example, if the incidence of an event occurring is low, then the sample size will need to be large. A good example of this phenom- enon is when trying to compare characteristics of people who do and do not fall. Because fewer than 2% of hospital- ized patients fall, if I needed 100 patients with falls, I would have to survey 4000 people. Another issue that would im- pact sample size is called treatment effect. For example, if groups are very different (one gets a treatment, and the other gets nothing or even a placebo), then it is expected that if there are differences in terminal outcomes, they would be large. In contrast, if there are minimal differences between treatments, then a very large sample would be needed. A good example of this would be testing the dif- ference in outcomes between 2 medications that both lower blood pressure.

www.cns-journal.com 205

ealth, Inc. All rights reserved.

Table 1. Key Terms

Terminology Definitions Association With Sample Size

α level Also referred to as significance level is the probability of making the wrong decisions when there is no difference between groups (null hypothesis)1,2

Typically, .05 is used in most research, but when the sample is very large, it may be .01 to prevent a type I error

β level Probability of not rejecting the null hypothesis (there is no difference), when there is a difference1,2

This number is typically set at .20 and is used to calculate power (1 – β)

Confidence level

Number of times you expect to get the same results when repeating the study with a similar but different sample over and over1,2

Researchers typically aim for a 95% confidence level

Effect size Difference between 2 variables or 2 groups1,2 Small effect size requires a larger sample than a large effect size

Margin of error

Explains the difference between the findings and what could be expected if the whole population had been sampled1,2

Careful design and collection of data as well as a sufficiently large sample will minimize this error

Power Probability of rejecting the null hypothesis and finding a difference when there is none—calculated as 1− β1,2

Since β is usually set at .20, power is usually set at 0.1- 0.20 or 0.80

Precision Degree of accuracy for an estimate, a measure of random error1,2 Bigger samples will minimize random error and increase precision

Random error Chance error in measurement between the true value and the value observed1,2

Random error is always present, but larger samples and repeated measurement can minimize its impact

Treatment effect

Difference between the means (or medians) of the treatment and control group1,2

The bigger the difference, the larger the treatment effect; bigger effects require smaller samples

Type I error A false-positive occurs if an investigator rejects a null hypothesis (there is no difference)—finding a difference when none exists1,2

Can be associated with overly large samples, where everything seems to be significant

Type II error A false-negative occurs if an investigator fails to reject a null hypothesis (there is no difference)—finding no difference when one does exist1,2

Usually associated with sample that are too small resulting in low power to detect differences

Using Research to Advance Nursing Practice

The researcher also needs to consider if making a type I (α) or type II (β) error is worse. A type I error might mean that the study finds a treatment to be effective when it is

Table 2. Factors That Will Impact the Sample Size

Baseline incidence of outcome of interest in the population the sample

• Low incidence

• High incidence

SD (variance) of outcome

• Small SD (1) means 68% of data points are within 1 SD of the me

• Large SD (3) means 99% of data points are within 3 SDs from the

Treatment effect—depends on the difference between 2 or more treat

• Treatment groups are more alike than different (2 different types o

• Treatment groups are very different (one group gets music and on

Probability of a type I error (α)—typically, researchers are willing to acc chance of making a type I error

• α = .05

• α = .10

Probability of a type II error (β)—typically, researchers are willing to ta (power− 1 − β)

• β of .02 = power of 80% to detect a difference if one exists

• β of .01 = power of 90% to detect a difference if one exists

206 www.cns-journal.com

Copyright © 2024 Wolters Kluwer

not. In contrast, a type II error means there was an effect, but the researcher failed to find it. Finally, the researcher needs to consider feasibility. If the cost in time, money,

Requirements

is drawn from (falls, deaths, suicide, infections)

an—very little variance

mean—a lot of variances

ments that are being compared

f music—but both groups get music)

e group does not get music)

ept a 5% (.05) chance of making a type I error—a larger α increases

ke a 20% (0.2) chance that they will miss a significant difference

September/October 2024

Health, Inc. All rights reserved.

Table 3. Free Online Sample Size Calculators

Access Site Description Authors

https://www.sample-size.net/ Useful for multiple design types and for different levels of data (nominal, ordinal, or continuous). Provides additional tutorials

Kohn and Senyak2 (2024)

https://clincalc.com/stats/ samplesize.aspx

Useful for 1- and 2-group designs and different levels of data (nominal, ordinal, or continuous). Excellent graphics, easy to follow

Kane4 (2024)

https://www.checkmarket.com/ sample-size-calculator/

Especially useful for survey research, as it takes into consideration your expected survey return rate

Check Market

https://riskcalc.org/samplesize/ Provides a wide range of design options and sample size calculations. Includes nice graphics and tutorials

Cleveland Clinic5

https://www.questionpro.com/ sample-size-calculator/

A very basic calculator, but the website provides a nice tutorial, and this will do most of what needs to be done

Question Pro

Sample size calculator (univie.ac. at)

This excellent website allows the user to select the design and design options as well as the level of measurement to get a more accurate calculation

Contact information: robin. [email protected]

or resources of obtaining an adequate sample is too high, then both the sampling and design may need to be revisited and revised. Above all else, a study must be feasi- ble, and the time to identify this issue is before a study begins.

RESOURCES There are many online resources the clinical nurse special- ist can use to estimate sample size (Table 3). Note that the calculations are different, based on study design (random- ized trial vs a survey) and number of comparisons or groups (more groups = more subjects).2,4 Almost all calcu- lators need a population proportion, and this is typically not known, but for most purposes, once your population exceeds 1000, there is little difference in the sample size es- timates. Calculators also need a confidence level (use 95%) and a margin of error (use 5%).2 The easiest way to gain expertise on calculating sample size and to get a better un- derstanding of how and why the numbers change is to prac- tice inserting information into the calculators provided.

CONCLUSION Sample size considerations are best made at the beginning of your research project, right after writing your research questions. Even if you have access to a statistician, it will

Clinical Nurse SpecialistA

Copyright © 2024 Wolters Kluwer H

be helpful to have at least a likely estimate of sample size before starting. The clinical nurse specialist can use this in- formation to guide nurses as they develop their research ideas and can help them identify issues related to feasibil- ity, design selection, and level of measurement. An under- powered study is not worth doing because much effort, time, and resources would be wasted. Instead, use the in- formation in this article to enhance the likelihood of conducting a valid study with meaningful results.

References 1. Julious SA. (2023). Sample Sizes for Clinical Trials (2nd ed.). Chap-

man and Hall/CRC: 2023 https://doi.org/10.1201/9780429503658 2. Kohn MA, Senyak J. Sample size calculators [website]. UCSF CTSI.

11 January 2024. Available at https://www.sample-size.net/ [Accessed 04 June 2024] https://www.sample-size.net/

3. Serdar CC, Cihan M, Yücel D, Serdar MA. Sample size, power and effect size revisited: simplified and practical approaches in pre- clinical, clinical and laboratory studies. Biochem Med (Zagreb). 2021;31(1):010502. doi:10.11613/BM.2021.010502 Epub 2020 Dec 15. PMID: 33380887; PMCID: PMC7745163.

4. Kane SP. Sample size calculator. ClinCalc: https://clincalc.com/ stats/samplesize.aspx. Updated July 24, 2019. Accessed June 4, 2024.

5. Wang X, Ji X. Sample size estimation in clinical research: from ran- domized controlled trials to observational studies. Chest. 2020; 158(1S):S12–S20. doi:10.1016/j.chest.2020.03.010 PMID: 32658647.

www.cns-journal.com 207

ealth, Inc. All rights reserved.