minitab
Lecture 5
From Sample Means to Experiments
Statistical Decision Theory: Hypothesis testing
Student’s t-Distribution
Different Types of T Tests
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
02-25-2021
From sample means to experiments (and from
statistical estimation theory to statistical decision theory)
Questions in statistical estimation theory:
How precise is value of the variable that I measured? (sample mean, standard deviation, standard error, margin of error)
How confident am I in that precision? (confidence level, interval)
Note: Statistical estimation theory does not answer an experimental question.
Questions a scientist has about a value measured in an experiment:
Is there a difference (and if so, how large?) when a variable in an experiment is changed? (for example: comparing treatment to control)
What is the precision of the difference between the two values?
What is the probability that an observed difference is real and not due to random errors (such as sampling or measurement errors)?
Bottom line: A scientist needs to make a decision:
Is a difference measured between two values (such as sample means) a “real” result or is it due to the statistical distribution of random errors?
In other words: Is an experimental result statistically significant or not?
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Most often we are not just measuring something and getting its statistics (mean, standard deviation, standard error, margin of error, confidence interval) for the purpose of getting that value.
2
How statistical decision theory operates: Statistical hypothesis testing
The statistical decision that is to be made: Can a scientific result be explained by random errors alone or not?
To make this decision statisticians create two hypothesis:
A. The result of the experiment can be explained by random errors alone. In other words: Any difference observed in an experiment is not a “real” difference. Statisticians call this the “null” hypothesis, or Ho.
B. The result of the experiment cannot be explained by random errors alone. There must be a real effect responsible for the difference. Statisticians call this the “alternative” hypothesis, or HAlt, sometimes also denoted H1 or HA.
As you can already guess, dealing with statistics never produces a “yes” or “no” answer, a clear “true” or “false”. So neither the null hypothesis nor the alternative hypothesis will ever be proven in hypothesis testing.
All statistics does is calculate the probability p that a statistic (such as a mean value calculated from the data) may be due to random errors.
Based on that probability, Ho will either be rejected or not rejected (note that the word “accepted” is not used here, since it would imply that is was proven that Ho is true).
That is the decision being made!
Statistical hypothesis testing is solely the process of deciding whether to reject or not reject the null hypothesis H0.
Scientific versus statistical hypotheses
For a scientist a hypothesis is a prediction of the outcome of an experiment, that is, a prediction of the answer to a scientific question. (Remember learning about the scientific method?)
In statistics the null hypothesis states that any outcome of an experiment can be explained by random errors (the exact opposite of the scientist’s hypothesis).
Only if there is enough evidence against the null hypothesis, that is, the p value - the probability that a result is due to random errors - is very small, will the null hypothesis be rejected.
So statistics operates kind of like a jury, assuming innocence (or “nothing new happened in the experiment”) until there is enough evidence to prove guilt (or “something new happened in the experiment).
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Once we calculated p (the probability that the data are due to random errors) are we ready for the decision to reject or not reject Ho?
No! Why not?
Answer: We haven’t decided on the criteria for our decision:
At what cutoff value of p do we reject Ho?
(Above that cutoff we will not reject Ho.)
That value is called the significance level (or α) of the test.
It determines the “power” of the test.
Some more things needed to help decide
The significance level α (or power) of a statistical test needs to be declared ahead of conducting the test!
Note: This is done in order to maintain some objectivity and prevent us from changing our minds. Otherwise, we might be tempted and change α once we see our results and engage in wishful thinking to prove our scientific hypothesis.
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Decision errors – type I and type II
As we saw, the significance level (power of the test) is the cutoff p value for whether Ho is rejected or not rejected. However, that p value is just a probability and thus there is a small chance that Ho is actually true when you reject it.
(Remember statistics can’t determine “true or false”, “yes or no”, just probabilities.)
So when you set α to 5% you are accepting a 5% chance to make an error.
What error would that be?
The two main types of hypothetical decision errors in hypothesis testing:
The scientist rejects Ho, and accepts HAlt , claiming the data show a true effect. However, in reality (which we can’t know) there is no effect.
In statistics that is called a Type I error.
We can also call it a false positive or false alarm: you think there is an effect when there is none. The probability for a type I error is equal to α.
The scientist accepts Ho, and thus claims the data show no true effect. However, in reality there is an effect.
In statistics that is called a Type II error. We can also call it a false negative: you think there is no effect when there is one. So you missed out on a chance to discover something. The probability for a type II error is called β.
A decision error: making the wrong decision!
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Let’s put these explanations into a diagram:
6
| DECISION | INTERPRETATION | OBJECTIVE TRUTH OUT THERE (remains unknown to us) | |
| No effect, Ho is true* | Real effect, Ho is false* | ||
| Do not reject Ho | Result is non- significant, can’t show an effect | Correct! | Wrong! (Type II error) (or false negative) |
| Reject Ho | Result is significant, there is a real effect | Wrong! (Type I error) (or false positive) | Correct! |
Type I and type II decision errors in statistical hypothesis testing
* Since we will never know the actual truth, we don’t use the terms true and false for Ho or HA, except in this context: when we make a hypothetical assumption about the real truth.
2 possible decisions
2 possible realities
4 possible outcomes
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Statistical hypothesis testing in graphical form
α: Significance level / Probability of making a type I error: set by researcher
β: Probability of making a type II error; depends on:
- Difference between μ0 and μ1
- Width of sampling distributions
- Significance level
Example: Comparing two experimental means: the mean of the control experiment ( µ0) and the mean of the “treatment” ( µ1)
8
Before we jump into action calculating statistics:
I have to tell you that few of the statistical tests and none of the published results using such tests are based on the normal distribution.
We could do a lot of statistical tests using the normal distribution, but that would be a waste of time. So let’s go to a distribution that is widely used in statistics to compare mean values: the student’s t distribution.
Later on in the course I will introduce other distributions that play a role when comparing categorical data or analyze a correlation between x and y.
9
The Student’s t distribution – biotech’s contribution to statistics
The Student’s t-distribution is a series of probability distributions that have been developed for small sample sizes in situations where the population mean and standard deviation are unknown.
It was discovered by William Sealy Gosset, working on scientific methods to improve the quality of beer, hops and barley for the Guinness Brewery. At the time, Guinness didn’t allow its employees to publish papers so he used the pseudonym “Student”.
While the statistics established at the time (using the normal distribution) required sample sizes of > 30, his experiments brewing beer in batches and growing barley in fields couldn’t afford sample sizes that large.
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
DF= degrees of freedom (DF=N-1)
Properties of the Student’s t-distribution
Bell-shaped, symmetrical, mean=median=0
Different t distributions for different “degrees of freedom” (DF = sample size minus one)
For increasing sample size the t distribution converges towards the standard normal distribution.
With decreasing DF the tails become “fatter”, indicating a higher probability of values further from the mean. This reflects the greater uncertainty when working with small sample sizes and unknown population parameters.
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Probability Density Curves For A Few Student’s t-distributions
Z distribution -4 -3.9 -3.8 -3.7 -3.6 -3.5 -3.4 -3.3 -3.2 -3.1 -3 -2.9 -2.8 -2.7 -2.6 -2.5 -2.4 -2.2999999999999998 -2.2000000000000002 -2.1 -2 -1.9 -1.8 -1.7 -1.6 -1.5 -1.4 -1.3 -1.2 -1.1000000000000001 -1 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1000000000000001 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 2.1 2.2000000000000002 2.2999999999999998 2.4 2.5 2.6 2.7 2.8 2.9 3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 4 1.3383022576488537E-4 1.9865547139277272E-4 2.9194692579146027E-4 4.2478027055075143E-4 6.119019301137719E-4 8.7268269504576015E-4 1.2322191684730199E-3 1.7225689390536812E-3 2.3840882014648404E-3 3.2668190561999182E-3 4.4318484119380075E-3 5.9525324197758538E-3 7.9154515829799686E-3 1.0420934814422592E-2 1.3582969233685613E-2 1.752830049356854E-2 2.2394530294842899E-2 2.8327037741601186E-2 3.5474592846231424E-2 4.3983595980427191E-2 5.3990966513188063E-2 6.5615814774676595E-2 7.8950158300894149E-2 9.4049077376886947E-2 0.11092083467945554 0.12951759566589174 0.14972746563574488 0.17136859204780736 0.19418605498321295 0.21785217703255053 0.24197072451914337 0.26608524989875482 0.28969155276148273 0.31225393336676127 0.33322460289179967 0.35206532676429952 0.36827014030332333 0.38138781546052414 0.39104269397545588 0.39695254747701181 0.3989422804014327 0.39695254747701181 0.39104269397545588 0.38138781546052414 0.36827014030332333 0.35206532676429952 0.33322460289179967 0.31225393336676127 0.2896915527614 8273 0.26608524989875482 0.24197072451914337 0.21785217703255053 0.19418605498321295 0.17136859204780736 0.14972746563574488 0.12951759566589174 0.11092083467945554 9.4049077376886947E-2 7.8950158300894149E-2 6.5615814774676595E-2 5.3990966513188063E-2 4.3983595980427191E-2 3.5474592846231424E-2 2.8327037741601186E-2 2.2394530294842899E-2 1.752830049356854E-2 1.3582969233685613E-2 1.0420934814422592E-2 7.9154515829799686E-3 5.9525324197758538E-3 4.4318484119380075E-3 3.2668190561999182E-3 2.3840882014648404E-3 1.7225689390536812E-3 1.2322191684730199E-3 8.7268269504576015E-4 6.119019301137719E-4 4.2478027055075143E-4 2.9194692579146027E-4 1.9865547139277272E-4 1.3383022576488537E-4 T.DIST DF= 3 -4 -3.9 -3.8 -3.7 -3.6 -3.5 -3.4 -3.3 -3.2 -3.1 -3 -2.9 -2.8 -2.7 -2.6 -2.5 -2.4 -2.2999999999999998 -2.2000000000000002 -2.1 -2 -1.9 -1.8 -1.7 -1.6 -1.5 -1.4 -1.3 -1.2 -1.1000000000000001 -1 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1000000000000001 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 2.1 2.2000000000000002 2.2999999999999998 2.4 2.5 2.6 2.7 2.8 2.9 3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 4 9.1633611427444726E-3 9.9756709055489768E-3 1.0875996116865797E-2 1.187542966221437E-2 1.2986622934728548E-2 1.422401880152971E-2 1.5604119051380573E-2 1.7145790526982049E-2 1.887061415861228E-2 2.0803280835 425438E-2 2.2972037309241342E-2 2.5409183884938433E-2 2.81516231782209E-2 3.1241455256556489E-2 3.4726608402172142E-2 3.8661485727167301E-2 4.3107594875663999E-2 4.8134109759614963E-2 5.3818288156802389E-2 6.0245635389509999E-2 6.7509660663892967E-2 7.571101806804327E-2 8.4955759279738682E-2 9.5352353202335802E-2 0.10700705749349003 0.1200171745135874 0.13446171682048136 0.15038908590753605 0.16780158735749706 0.18663702938545559 0.20674833578317209 0.22788306587380588 0.2496659048220892 0.27158835908824669 0.29301067996481306 0.31318091100882872 0.33127437234925833 0.34645357427454188 0.35794379463845583 0.36511444382851777 0.36755259694786152 0.36511444382851777 0.35794379463845583 0.34645357427454188 0.33127437234925833 0.31318091100882872 0.29301067996481306 0.27158835908824669 0.24966 59048220892 0.22788306587380588 0.20674833578317209 0.18663702938545559 0.16780158735749706 0.15038908590753605 0.13446171682048136 0.1200171745135874 0.10700705749349003 9.5352353202335802E-2 8.4955759279738682E-2 7.571101806804327E-2 6.7509660663892967E-2 6.0245635389509999E-2 5.3818288156802389E-2 4.8134109759614963E-2 4.3107594875663999E-2 3.8661485727167301E-2 3.4726608402172142E-2 3.1241455256556489E-2 2.81516231782209E-2 2.5409183884938433E-2 2.2972037309241342E-2 2.0803280835425438E-2 1.887061415861228E-2 1.7145790526982049E-2 1.5604119051380573E-2 1.422401880152971E-2 1.2986622934728548E-2 1.187542966221437E-2 1.0875996116865797E-2 9.9756709055489768E-3 9.1633611427444726E-3 T.DIST DF= 5 -4 -3.9 -3.8 -3.7 -3.6 -3.5 -3.4 -3.3 -3.2 -3.1 -3 -2.9 -2.8 -2.7 -2.6 -2.5 -2.4 -2.2999999999999998 -2.2000000000000002 -2.1 -2 -1.9 -1.8 -1.7 -1.6 -1.5 -1.4 -1.3 -1.2 -1.1000000000000001 -1 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1000000000000001 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 2.1 2.2000000000000002 2.2999999999999998 2.4 2.5 2.6 2.7 2.8 2.9 3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 4 5.1237270519179116E-3 5.7483728547694009E-3 6.458848364369843E-3 7.2680175325693956E-3 8.1907726871290592E-3 9.244354092520923E-3 1.0448714749395219E-2 1.1826934151171167E-2 1.3405683736328885E-2 1.521574504 4952824E-2 1.7292578800222964E-2 1.9676938890598517E-2 2.2415519021677269E-2 2.5561611020544554E-2 2.9175741685939279E-2 3.3326238887022831E-2 3.8089656526431967E-2 4.355096135044003E-2 4.9803352151145085E-2 5.6947544172170565E-2 6.5090310326216497E-2 7.4342030033196185E-2 8.4812962896903751E-2 9.6607948713911859E-2 0.10981925265599095 0.12451734464635514 0.14073954789491464 0.15847673572898244 0.17765861346493556 0.19813859080334625 0.2196797973509807 0.24194434361358991 0.26448835680795757 0.28676545757669797 0.30814100972341996 0.32791853132274656 0.34537807575273344 0.35982432834900979 0.37063997771396962 0.37733812996643123 0.37960668982249451 0.37733812996643123 0.37063997771396962 0.35982432834900979 0.34537807575273344 0.32791853132274656 0.30814100972341996 0.28676545757669797 0.2 6448835680795757 0.24194434361358991 0.2196797973509807 0.19813859080334625 0.17765861346493556 0.15847673572898244 0.14073954789491464 0.12451734464635514 0.10981925265599095 9.6607948713911859E-2 8.4812962896903751E-2 7.4342030033196185E-2 6.5090310326216497E-2 5.6947544172170565E-2 4.9803352151145085E-2 4.355096135044003E-2 3.8089656526431967E-2 3.3326238887022831E-2 2.9175741685939279E-2 2.5561611020544554E-2 2.2415519021677269E-2 1.9676938890598517E-2 1.7292578800222964E-2 1.5215745044952824E-2 1.3405683736328885E-2 1.1826934151171167E-2 1.0448714749395219E-2 9.244354092520923E-3 8.1907726871290592E-3 7.2680175325693956E-3 6.458848364369843E-3 5.7483728547694009E-3 5.1237270519179116E-3 T.DIST DF= 10 -4 -3.9 -3.8 -3.7 -3.6 -3.5 -3.4 -3.3 -3.2 -3.1 -3 -2.9 -2.8 -2.7 -2.6 -2.5 -2.4 -2.2999999999999998 -2.2000000000000002 -2.1 -2 -1.9 -1.8 -1.7 -1.6 -1.5 -1.4 -1.3 -1.2 -1.1000000000000001 -1 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1000000000000001 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 2.1 2.2000000000000002 2.2999999999999998 2.4 2.5 2.6 2.7 2.8 2.9 3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 4 2.0310339110412167E-3 2.4066888019954914E-3 2.854394394609606E-3 3.3881509779623989E-3 4.0246232150294671E-3 4.7836071267013227E-3 5.6885611066299349E-3 6.7672024406869391E-3 8.052167372342163E-3 9. 5817276708977175E-3 1.1400549464542524E-2 1.3560470295244924E-2 1.6121257439422144E-2 1.9151294092490986E-2 2.2728119798464959E-2 2.6938727628244463E-2 3.1879493750030567E-2 3.7655586709753393E-2 4.4379676614245689E-2 5.2169742604355016E-2 6.1145766321218181E-2 7.1425107032802512E-2 8.3116389653879602E-2 9.631180963322937E-2 0.11107787729698333 0.12744479428709171 0.14539487566000614 0.16485069296801935 0.18566389362670319 0.20760591316421406 0.23036198922913867 0.25352995055982758 0.27662513233825647 0.29909241773685274 0.32032581052912462 0.33969513635207788 0.35657853369790399 0.37039846155274558 0.38065818105444926 0.38697522581518051 0.38910838396603115 0.38697522581518051 0.38065818105444926 0.37039846155274558 0.35657853369790399 0.33969513635207788 0.32032581052912462 0.29909241773685274 0.27662513233825647 0.25352995055982758 0.23036198922913867 0.20760591316421406 0.18566389362670319 0.16485069296801935 0.14539487566000614 0.12744479428709171 0.11107787729698333 9.631180963322937E-2 8.3116389653879602E-2 7.1425107032802512E-2 6.1145766321218181E-2 5.2169742604355016E-2 4.4379676614245689E-2 3.7655586709753393E-2 3.1879493750030567E-2 2.6938727628244463E-2 2.2728119798464959E-2 1.9151294092490986E-2 1.6121257439422144E-2 1.3560470295244924E-2 1.1400549464542524E-2 9.5817276708977175E-3 8.052167372342163E-3 6.7672024406869391E-3 5.6885611066299349E-3 4.7836071267013227E-3 4.0246232150294671E-3 3.3881509779623989E-3 2.854394394609606E-3 2.4066888019954914E-3 2.0310339110412167E-3 T.DIST DF= 30 -4 -3.9 -3.8 -3.7 -3.6 -3.5 -3.4 -3.3 -3.2 -3.1 -3 -2.9 -2.8 -2.7 -2.6 -2.5 -2.4 -2.2999999999999998 -2.2000000000000002 -2.1 -2 -1.9 -1.8 -1.7 -1.6 -1.5 -1.4 -1.3 -1.2 -1.1000000000000001 -1 -0.9 -0.8 -0.7 -0.6 -0.5 -0.4 -0.3 -0.2 -0.1 0 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1.1000000000000001 1.2 1.3 1.4 1.5 1.6 1.7 1.8 1.9 2 2.1 2.2000000000000002 2.2999999999999998 2.4 2.5 2.6 2.7 2.8 2.9 3 3.1 3.2 3.3 3.4 3.5 3.6 3.7 3.8 3.9 4 5.2471644019740798E-4 6.8633509016922955E-4 8.957220872584876E-4 1.1661364480637616E-3 1.5141706922918103E-3 1.9604555808020938E-3 2.5304642724106696E-3 3.2554111678475802E-3 4.1732316830005159E-3 5.3296170578658016E-3 6.7790627460931003E-3 8.585869863140699E-3 1.0825016893845324E-2 1.3582794199685938E-2 1.6957068343387546E-2 2.1057019220621639E-2 2.6002173732796896E-2 3.1920549432000164E-2 3.8945725132978719E-2 4.7212678193653275E-2 5.6852275047197962E-2 6.7984376569213315E-2 8.0709624798490698E-2 9.5100110953271658E-2 0.11118928083310883 0.12896160173967178 0.14834267891674144 0.16919064854071297 0.19128976490598698 0.21434711785664057 0.23799334232287983 0.26178800210082609 0.2852300432111225 0.30777333104241611 0.3288468382685204 0.34787857969720454 0.3643219485288568 0.37768275260924278 0.38754503154481451 0.39359369563267937 0.39563218489409779 0.39359369563267937 0.38754503154481451 0.37768275260924278 0.3643219485288568 0.34787857969720454 0.3288468382685204 0.30777333104241611 0.2852300432111225 0.26178800210082609 0.23799334232287983 0.21434711785664057 0.19128976490598698 0.16919064854071297 0.14834267891674144 0.12896160173967178 0.11118928083310883 9.5100110953271658E-2 8.0709624798490698E-2 6.7984376569213315E-2 5.6852275047197962E-2 4.7212678193653275E-2 3.8945725132978719E-2 3.1920549432000164E-2 2.6002173732796896E-2 2.1057019220621639E-2 1.6957068343387546E-2 1.3582794199685938E-2 1.0825016893845324E-2 8.585869863140699E-3 6.7790627460931003E-3 5.3296170578658016E-3 4.1732316830005159E-3 3.2554111678475802E-3 2.5304642724106696E-3 1.9604555808020938E-3 1.5141706922918103E-3 1.1661364480637616E-3 8.957220872584876E-4 6.8633509016922955E-4 5.2471644019740798E-4
t-score
Probability density
(Normal distribution)
Integral of the Student’s t distribution
(used to determine the p values found in t tables)
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
T table contains values for many different t distributions, having different sample sizes and degrees of freedom
Only has selected values for t and p because it would fill too many pages otherwise.
13
The t distribution is used in the place of the z distribution (standard normal distribution) to compare sample means.
Use the t distribution in any situation where the sample size is smaller than 30 and where the population mean and standard deviation are unknown. So this covers a lot of situations in the real life of a scientist.
Use the t distribution in any situation where you would otherwise use z distribution. This is because for sample sizes larger than 30 the t distribution becomes very close to the normal distribution, so you will get very similar results for probabilities and confidence intervals.
Taking the first two points together, many statisticians argue that you should ALWAYS use the t distribution instead of the z distribution. In doing so, you are just erring a bit more on the side of caution (slightly lower confidence levels, slightly wider margins of error, slightly larger p values so your data need to be a bit better to reject the null hypothesis).
When to use the t distribution and when the z distribution to compare sample means?
Short answer: Always use t instead of z
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
What was the point then to teach you about the normal distribution?
The normal distribution is a much easier to explain example of a probability distribution function. In particular it is easier to explain how to obtain probabilities from such a function.
The normal distribution makes it easier to explain statistical concepts such as standard error of the mean, confidence intervals, or probabilities (as compared to the t distributions).
It is easier to explain how to use the z table as opposed to the t table (which in the past people needed for the t distribution – now we have software).
It is hard to understand the t distribution without first understanding the normal distribution.
Short answer: it was for didactical reasons.
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Different t tests for different questions or comparisons
General question all t tests have in common:
Is an observed difference in mean values larger than what you would expect from random fluctuations alone?
Or, in statistics lingo: Is an observed difference in mean values significant?
| Specific question | t test type |
| Is a sample mean significantly different from a known population mean? | One-sample t test |
| Are two sample means significantly different from each other? | 2 versions of two-sample t test: |
| Case 1: The standard deviations of both samples are similar | Unpaired t test - equal variance |
| Case 2: The standard deviations of both samples are very different | Unpaired t test - unequal variance (also called Welch test) |
| Are paired means significantly different from each other?* | Paired t test |
* Example of paired means:
Before and after values of groups of patients treated with placebo or drug
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
One-tailed and two-tailed t tests
Note: Each of the 4 t test variations can be run as a one-tailed or two-tailed test, depending on your hypothesis HAlt .
As we just saw, each of the four types of t tests (one-sample, unpaired-equal var., unpaired-unequal var., paired) answers the same question: are 2 sample means different? But each test does so in a slightly different way.
To make the situation even more complicated, “different” can mean 3 things for the Ho and HAlt hypotheses:
| Meaning of “different” | t test variant to apply |
| A sample mean is larger than the one defined in Ho. | One-tailed t test (right tail) |
| A sample mean is smaller than the one defined in Ho. | One-tailed t test (left tail) |
| A sample mean is either larger or smaller than the one defined in Ho. | Two-tailed t test (both tails) |
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Formulate Ho and HAlt for your planned experiment.
Chose the significance level α (power) of the test that you want your conclusion to have. Most commonly that level is set to 5%, (p=0.05).
(This is sort of analogous to a 95% confidence interval, even though confidence intervals are a different concept and don’t appear in hypothesis testing.)
Select the test appropriate for your experiment (depending on whether your data are numerical or categorical, whether you want to compare an experimental mean value to a known mean value or whether you want to compare two mean values or more, etc.).
Conduct your experiment.
Enter your raw data into the respective test program.
Procedure for statistical hypothesis testing
(testing for statistical significance)
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
The test program will calculate the test statistic for your experiment.
The test statistic represents your raw data as a single number. For t tests it is called t statistic, for z tests, z statistic (other test will be addressed later in the course). That number is obtained through a standardization of your data, analogous to converting a result (mean, standard deviation) into a z score, just that the formulas for comparing different values are more complicated.
The test program will calculate the p value for the test statistic, which is the probability that the test statistic is only due to random errors and fluctuations, based on the underlying distribution ( t distribution, z distribution, etc.).
Make your decision to reject or not reject Ho (that is, draw your conclusion):
If the p value is larger than the significance level that you selected previously, you fail to reject Ho, stating that the result from your data can be explained by random errors and fluctuations alone.
If the p value is smaller than the significance level you reject Ho, stating that the result is statistically significant, and listing the p value obtained.
Procedure for statistical hypothesis testing (cont’d)
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924