THE NULL HYPOTHESIS AND YOU 1
The Null Hypothesis and You
Hannah K. Smith
School of Behavioral Sciences, Liberty University
EDCO 735: Statistics
Dr. Robin Henson
September 7, 2025
The Null Hypothesis and You
Prompt 1
When there is evidence to justify the direction of an effect, then a directional
significance test is appropriate. For example, a school counselor is trying to evaluate the
effectiveness of a new resilience skill intervention to her group of students. The intervention has
evidence to support higher post-intervention scores, meaning the students should show growth
in their level of resiliency. In this given situation, the directional hypothesis can be defended
because only improvement is probable given the mechanism and previous data supports. Harm
is not expected in the intervention, but it will be ethically monitored to ensure it will not occur.
By using a onetailed test, the concentration of alpha in one tail should be slightly increasing in
THE NULL HYPOTHESIS AND YOU 2
power to detect improvement in the predicted direction. However, this does reduce the ability
to detect effects that occur in the opposite direction and could be important to acknowledge
and bring awareness to.
When looking at the alpha value (Type 1 error), it is important to acknowledge
contributing factors to it. Therefore, when looking at the example, one should consider the
field’s conventions, meaning what is an acceptable standard for a significant result for adopting
the intervention. They should also look at the stakes versus the tradeoff, meaning when false
positives costly and how it influences the adoption of the intervention for a district-wide
program. They should also look at the sample size of the design. Were the measure and
numbers reliable? Is it a good representation of the whole? Each factor influences the alpha
value and emphasize the need for alpha and beta (Type 2 error) to be balanced thus leading us
to see that researchers should maintain this balance throughout the plan and study.
When a researcher reports a p value, they are reporting the probability under the
assumption that the null hypothesis is true. When a small p value is obtained, we have indicated
that the data is unusual if the null is true. It does not quantify the probability that the null is true
nor the size of effect or importance (Wasserstein & Lazar, 2016). When we look at p values, we
usually want a smaller p value because they provide evidence against the null hypothesis. When
looking at p values, they should be viewed alongside effect sizes, confidence intervals, and the
study design. When researchers look at data for depression, a small change in the scores could
show up as having statistical significance. In reality, it may not matter much. If the study does
not reach statistical significance but it does show improvement, it could be worth noting as a
resource when planning for future programs.
Prompt 2
Type 1 Error
THE NULL HYPOTHESIS AND YOU 3
A school district is testing a new online reading program and is comparing the results
with traditional classroom instruction. When looking at the data, the researcher finds that
students who participated in the online reading program have slightly higher scores on one
standardized reading test. Based on the results, the district comes to the conclusion that the
program is overall effective and better. They decide to make it a requirement district wide. With
little difference, there is no true data to support the difference. The effect that was received was
simply a fluke of random variation thus giving a false positive result. Since the district went off
the small sample results received, they have now wasted funding on the software and training
and have taken away from traditional classroom reading instruction that has proven to be
effective over time.
Type 2 Error
A pharmaceutical company tests a new anxiety medicine on a small group of people. The
medicine does actually work, but due to the small sample size of the test group and the short
test run, the results come back as not significant. The doctor then decides to conclude the study
and say the medicine does not work. The small sample size and short test run give a false
negative and keeps the medicine off the market.
For a type 1 error, the number of tests, selective reporting, and violations of assumptions
influence the magnitude of risk. For a type 2 error, unreliable or weak measure, low power from
a small sample, and high outcome variance influence the magnitude of risk. The statistical
power can be increased with a larger sample size and precise designs (Cohen, 1988).
Statistical errors can lead to unclear conclusions which could result in unsafe use of an
intervention or practice (Dhulkhed et al., 2021). Appropriate planning with corrections as
needed can help reduce the risk. Often sampling techniques and the methods are not
mentioned when discussing errors occurring when selecting a sample that does not represent
the entire population (Dhulkhed et al., 2021). Therefore, it is important to select your sample
THE NULL HYPOTHESIS AND YOU 4
based on the whole and take the true results and not just those that support your claim.
Researchers should also take note that the study should be verifiable and the results should be
reproducible (Dhulkhed et al., 2021, because if they cannot be, how accurate is the study?
If the null is incorrectly reported as significant, that would be considered a Type 1 error.
Subsequent implications could be overestimating the program, the opportunity costs, the
potential harm if the effective treatment overshadows better options, and the credibility that is
lost when the results cannot be replicated. The conclusions from such results should tell
researchers to be cautious. There is a need for replication, effect size scrutiny, and sensitive
analyses rather than confident practice changes based on a single significant finding
(Wasserstein & Lazar, 2016). When results cannot be replicated, then there is a lack of validity in
the study that must be addressed (Malin et al., 2019), thus impacting the future results and
efficacy of the study and could have the fidelity questioned each time it is conducted.
THE NULL HYPOTHESIS AND YOU 5
References
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). Lawrence
Erlbaum.
Dhulkhed, V., Tantry, T., & Kurdi, M. (2021). Minimising statistical errors in the research domain:
Time to work harder and dig deeper. Indian Journal of Anaesthesia, 65(8), 567-
571. https://doi.org/10.4103/ija.IJA_720_21
Malin, D. H., Hetherington, S. A., Nghiem, D. M., Ward, C. P., Kelling, N., Izygon, J. J., Harrison‐
Walker, L. J., Campbell, J. R., Hughes, R., & Buras, W. R. (2019). Software for designing
rigorous and replicable preclinical research: The experimental design accelerator. Journal
of Neuroscience Research, 97(9), 1043-1050.
https://doi.org/10.1002/jnr.24440
Wasserstein, R. L., & Lazar, N. A. (2016). The ASA's statement on p-values: Context, process, and
purpose. The American Statistician, 70(2), 129-133.
Powered by TCPDF (www.tcpdf.org)
