COMPARING TWO INDEPENDENT GROUPS 1
Comparing Two Independent Groups
Hannah K. Smith
School of Behavioral Sciences, Liberty University
EDCO 735: Statistics
Dr. Robin Henson
September 21, 2025
Comparing Two Independent Groups
Prompt 1
An independent samples t test is used when a researcher wants to find out the means
for a dependent variable having significant differences across two groups, and it is often used in
research more than the one-sample t test (Warner, 2021). With several assumptions for the use
of independent samples t test, we will explore them further. The first one is that Y scores must
be quantitative because it would not make sense to figure out the mean if they were categorical
(Warner, 2021). Another is Y scores are independent of each other both between and within
groups. If they are not, meaning is they are paired, matched, or repeated, a different test must
be used. That test is the paired-samples t test. The Y scores should be independent because
COMPARING TWO INDEPENDENT GROUPS 2
they could influence the other scores if not. To ensure they are independent, researchers could
assess participants separately. Another assumption that can be made is homogeneity of
variance. This means the variances in Y scores that are used in comparison to the samples are
similar or equal. Warner (2021) notes that if sample sizes are equal then there is very little
distortion in the Type I error rate unless the sample size is very small and the ratio of variance is
large. Another assumption is that there are no outliers within groups. When there are extreme
outliers, they violate the assumption of normality in samples and cause problems with the data
analysis (Warner, 2021).
Violations may not be a big issue for other tests, but they can cause some big problems
when conducting independent-samples t test (Warner, 2021). Some violations are the following.
The independent-samples t test cannot be used if the dependent variable is categorical or if the
scores are ranks within groups. The test cannot be used if assumptions of independence of
observations are violated (e.g., between or within groups). If outliers cause p values over or
underestimate the true risk for Type I error. You should document the outliers and explain if you
dropped or retained them. One assumption is that for all null hypothesis significance tests, you
only do one test. If more are run, the risk for getting at least one Type I error increases.
The p value is the probability, and it estimates the risk for a Type I error (Warner, 2021).
The p value can never be 0, but it can get very close to it. If researchers are using a software that
rounds, it may be represented as .000, but the correct way to write it would be p<.001. this tells
us that the results are highly unlikely, but they are not truly impossible.
Prompt 2
Effect sizes give us a good idea of the practical importance and application of findings
that go beyond whether or not they reach statistical significance. When looking at
independentsamples t tests, three widely used effect sizes are Cohen’s d, Point Biserial r, and Eta
Squared.
COMPARING TWO INDEPENDENT GROUPS 3
I would say Cohen’s d is the most common, or at least it is the one that I had heard about
before taking this course. It is different than the next two I am going to cover. Cohen’s d is
standardized and does not depend on N (Warner, 2021). While there is not a fixed range for
possible values when running using Cohen’s d, any value lower than -2 or higher than +2 are
considered uncommon (Warner, 2021). Cohen’s d is the distance between two group means of
with-in group standard deviations. Basically, it tells us how large the overlap is between two sets
of scores. When looking at it on a bell curve, you would see it, but it may be hard to make that
comparison when looking at real life situations as some things cannot be accurately measured.
Point Biserial also shows us the relationship between group membership and the
dependent variable (Warner, 2021). It is just the square root of η2 (Warner, 2021). Point Biserial
is often the preferred effect size index when combined across studies by using meta-analytic
procedures (Warner, 2021). It is fixed between -1 to +1 and does not depend on sample size or
original units of measurement (Warner, 2021). This method is preferred over the other two
because Cohen’s d expresses effect size in terms of standardized mean differences which
provide less intuitive interpretations, and eta squared works well for ANOVA, but the point
biserial is often a more simple and interpretable means.
Eta squared is also an effect size that tells us the proportion of variance in Y dependent
variable scores in group membership (Warner, 2021). It has a fixed range 0 to 1. Warner (2021)
notes that 0=no association and 1=perfect and are not related to sample size. The statistical
significance of eta squared depends on sample size and effect size (Warner, 2021).
When a researcher reports a large eta squared of .64, that tells us the group
membership explains the differences in scores. Eta squared and the t value are related, and if
the eta squared is large (.64), then it should usually be paired with a comparable large t value.
Therefore, if a researcher reports a large eta squared of .64 but shows us a nonsignificant t
value, that could be concerning. Some reasons would be a really small sample size, or low
COMPARING TWO INDEPENDENT GROUPS 4
degrees of freedom. When there are few participants per group, the degrees of freedom are
limited. This, in turn, inflates the standard error causing the t value to be small even if the group
means are far apart. If there are inconsistencies in the reporting, that could also raise concerns.
When calculating eta squared, mismatched values (e.g., big eta squared but small t value), could
mean the effect size was miscalculated, or there was something wrong in the reporting. There
are also violations of assumptions. If the groups are unequal or have some outliers that distort
the mean, it could inflate the variance of eta squared and weaken the reliability of the t test. As
one uses eta squared, they can also use the partial of it. They should be cautious by the partial
use though, because it may give misleading data when the design includes a blocking factor
(Olejnik & Algina, 2003). A blocking factor is something that researchers include in the design to
account for variations that could affect the outcome (Design of Engineering Experiments Design
of engineering experiments, n.d.).
Three examples that influence the size of t are effect size and N, dosage levels for
treatment, and control of within-group error variance (Warner, 2021). Effect size can influence
the size of t because if it is consistent t and N are expected to increase. When looking at dosage
levels for treatment, you need to look at it in terms of an example. The caffeine example used in
the text mentions caffeine milligrams given to participants, and we can only assume that
caffeine does effect heart rate (Warner, 2021). That would give an explanation in why the heart
rate is larger in one study over the others. Control of within-group error variance, or
experimental error
(Warner, 2021), is another example that can influence the size of t. Largely found in drug studies
(Warner, 2021), experimental error can be large if the participants receive the same treatment
but in different ways. Based on the example in the test, if participants are all very similar in all
aspects of their health, age, and past caffeine consumption, there will likely be a small t value
obtained, but if the participants all vary differently and largely, then the t value will be bigger.
COMPARING TWO INDEPENDENT GROUPS 5
One can make the inference that with a large eta squared of .64 reported that it is a large
effect size and represents a large percentage of the group membership variation. This number is
far above normal and what is already considered large. If the inference is correct, then it
suggests that the difference between the groups is strong and meaningful. In real world terms,
the group membership is accounting for the differences in scores.
References
Design of Engineering Experiments Design of engineering experiments. (n.d.).
https://sites.stat.washington.edu/pds/stat502/LectureNotes/RCBD.pdf
Olejnik, S., & Algina, J. (2003). Generalized eta and omega squared statistics: Measures of effect
size for some common research designs. Psychological Methods., 8(4), 434–447.
https://doi.org/10.1037/1082-989X.8.4.434
Warner, R. M. (2021). Applied statistics I: Basic bivariate techniques (3rd ed.). Sage Publications.
ISBN: 978-1-5063-5280-0
