hw 6 minitab and excel
pbiner
LectureClass604MAR2021.pptxLecture 6
Statistics with categorical variables
Summary statistics
Binomial Distribution
Sampling distribution of a proportion
Errors & confidence intervals for a proportion
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
03-04-2021
A quick refresher about categorical variables
What are they?
How are they defined?
What are some of their properties?
How can we organize and display them?
What do their summary statistics look like?
Examples of categorical variables
Examples of “values” or outcomes for categorical variables
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Organizing two binary categorical variables in two-way tables
(also called fourfold tables, cross-tabulations, or contingency tables)*
| Smoker | Non-smoker | Marginal row totals | |
| Male | Σ R1 | ||
| Female | Σ R2 | ||
| Marginal column totals | Σ C1 | Σ C2 | Grand total* |
*Grand total: Either add up the columns or the rows!
Either one will add up to the total number of subjects in the study.
*The two-way/fourfold table is a special type of a cross-tabulation or contingency table, which can have many more categorical variables and not just binary variables .
2. Summarize frequency data by calculating proportions
Frequency data
1. Enter counts from raw data or
“case-level data“ (i.e. individual observations)
Variable 2: smoking status
2 possible values
Event counts (frequency of outcomes)
Variable 1: Gender
2 possible values
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
| Smoker | Non-smoker | Marginal row totals | |
| Male | Σ R1 | ||
| Female | Σ R2 | ||
| Marginal column totals | Σ C1 | Σ C2 | Grand total=1* |
2. (cont’d): Divide each event count by grand total to obtain the respective proportion.*
Optional: Multiply result by 100 to display values as percentages.
Converting event counts to proportions
* Note: The proportions in a 2-way table are always based on the total number of subjects (the grand total).
Such proportions are also refered to as “joint distributions”
*Grand total: Either add up the columns or the rows!
Either one will add up to 1 (or 100, if you display the values in percent)
Summary statistics for categorical variables
Proportions of grand total*
Variable 1: 2 possible values
Variable 2:
2 possible values
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Anything else to add for the summary statistic?
So what kind of statistical errors can occur in these numbers?
No
How about a sample mean? A sample standard deviation?
There is none. The categories are clearly defined, and so are the values.
There is no doubt in the counts (unless someone screwed up).
None in the numbers themselves.
What if we want to use the data from this study to draw conclusions about the larger population?
If we see a difference in the smoker/non-smoker proportion in a sample (lets say between 100 males and females), how can we tell that
a) this difference is present in the population as well?
b) this difference is not just due to random chance, that means, it is significant?
Hmmm…. Now it gets interesting. How would we do this?
We hear all the time of margins of errors and confidence intervals for these kinds of surveys. So there must be a way to address randomness….
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
How to create a distribution for random answers?
Such a distribution would be needed to answer the question:
What is the probability that the proportions found in the table (calculated from the counts) occurred randomly?
(This would be kind of looking for a p value to determine the significance of the study result.)
To create such a random distribution, we assume that all observations (such as answers by people to a question) randomly fall into the 2 possible outcomes, like flipping a coin. That means that there is a 50% chance (or a probability of 0.5) that the answer is yes or no, smoker or non-smoker, hot or cold, sick or healthy, etc..
The good news: that distribution already exists!
It is the binomial distribution.
It predicts the likelihood of different outcomes for flipping coins (such as: What is the chance to get 0, 1, 2, 3, 4….98, 99, 100 heads with 100 tosses?)
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
The bad news: you don’t want to do the math for that distribution!
Another good news: You don’t have to: people with math brains have already done it!
If we do 100 coin tosses (or we ask 100 people in a survey a yes or no question), we can calculate the probability of getting a certain number (let’s say 30) of heads (or yes answers, or “successes”*), just by chance.
Where
n: fixed number of trials
x: specified number of successes
n-x: number of failures
p: probability of success in each trial
1-p: probability of failure in each trial
For the above example (n=100, p=0.5, x=30) the formula comes out to:
(which results in pr = 2.317 x 10e-5)
* The researcher needs to define which of the two possible outcomes is labeled as success and which one as failure. It doesn’t matter but it has to be consistent.
In this expression stands for “n chose x”, the number of ways to rearrange x successes among n trials.
Formula for binomial probabilities
The general formula for the probability (pr) of x successes in n trials is:
Watch out for the two different probabilities in the binomial!
( I call them pr and p to keep them apart)
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
7
If we calculate the probability not just for one number of successes as we did in the last slide (probability of getting 30 heads with 100 coin tosses), but for all numbers (1 to 100 heads) we get the frequency distribution for 100 coin tosses, which is the binomial distribution.
The probability function for the binomial distribution(s)*
*As you can see, there is an infinite number of them, (not just this one) for each number of trials and for each probability p for each trial (p doesn’t have to be 0.5).
| X | pr(X) |
| 45 | 4.847E-02 |
| 46 | 5.796E-02 |
| 47 | 6.659E-02 |
| 48 | 7.353E-02 |
| 49 | 7.803E-02 |
| 50 | 7.959E-02 |
| 51 | 7.803E-02 |
| 52 | 7.353E-02 |
| 53 | 6.659E-02 |
| 54 | 5.796E-02 |
| 55 | 4.847E-02 |
EXCEL function used: BINOM.DIST(number of successes X, number of trials n, probability of success p, FALSE) (for distribution function)
Some of the values for the graph
pr(X)
Here are the two different probabilities again (pr and p)!
X
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Binomial distribution
(Probability pr of X successes in 100 trials, assuming p=0.5 for each trial)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 7.8886090522101049E-31 7.8886090522101158E-29 3.9048614808440493E-27 1.2755880837423889E-25 3.0933011030752918E-24 5.9391381179045101E-23 9.4036353533488793E-22 1.2627738903068301E-20 1.4679746474816979E-19 1.5005963063146118E-18 1.3655426387462979E-17 1.1172621589742489E-16 8.2863610123923984E-16 5.6092289930040794E-15 3.4857351599382189E-14 1.9984881583645948E-13 1.0616968341311918E-12 5.2460314157070423E-12 2.419003375020489E-11 1.0439909302719939E-10 4.2281632676015464E-10 1.6107288638482146E-9 5.7839809201822048E-9 1.9615239642357071E-8 6.2932227185896111E-8 1.9131397064512392E-7 5.5186722301477995E-7 1.5125249815960639E-6 3.9433687020183031E-6 9.7904326394937912E-6 2.3170690580135296E-5 5.2320914213208622E-5 1.1281697127223065E-4 2.3247133474277876E-4 4.5810527728724036E-4 8.6385566574165252E-4 1.5597393964779853E-3 2.6979276047186741E-3 4.4728799762441106E-3 7.1107322699265549E-3 1.0843866711637992E-2 1.5869073236543376E-2 2.2292269546572856E-2 3.0068642644214549E-2 3.8952559789096154E-2 4.8474296626430782E-2 5.7958398140297657E-2 6.659049999098024E-2 7.3527010406707352E-2 7.802866410507725E-2 7.9589237387178782E-2 7.802866410507725E-2 7.3527010406707352E-2 6.659049999098024E-2 5.7958398140297643E-2 4.8474296626430782E-2 3.8952559789096154E-2 3.0068642644214549E-2 2.2292269546572856E-2 1.5869073236543376E-2 1.0843866711637992E-2 7.1107322699265549E-3 4.4728799762441098E-3 2.6979276047186741E-3 1.5597393964779846E-3 8.6385566574165252E-4 4.5810527728724047E-4 2.3247133474277876E-4 1.1281697127223065E-4 5.2320914213208622E-5 2.3170690580135296E-5 9.7904326394937895E-6 3.9433687020183031E-6 1.5125249815960639E-6 5.518672230147791E-7 1.9131397064512423E-7 6.2932227185896124E-8 1.9615239642357035E-8 5.7839809201822048E-9 1.6107288638482146E-9 4.2281632676015614E-10 1.0439909302719901E-10 2.419003375020489E-11 5.2460314157070423E-12 1.0616968341311918E-12 1.9984881583645948E-13 3.4857351599382189E-14 5.6092289930040794E-15 8.2863610123923984E-16 1.1172621589742489E-16 1.3655426387462979E-17 1.5005963063146224E-18 1.4679746474816979E-19 1.2627738903068301E-20 9.4036353533488793E-22 5.9391381179045101E-23 3.0933011030752697E-24 1.2755880837423889E-25 3.9048614808440493E-27 7.8886090522101158E-29 7.8886090522101049E-31
If the random variable (representing the number of successes in the counted data) has a binomial distribution with n trials , with a probability p of success for each trial, then:
* Contrary to what I said before we can calculate the mean and standard deviation of X according to the above formulas, but ONLY if we treat our sample data in the 2-way table as random variables, following the binomial distribution.
However, unlike for numerical variables, we still can’t calculate a mean and standard deviation just from the sample data alone.
Mean, variance, and standard deviation of the binomial distribution*
Conditions to be met for a random variable X (=number of successes) to have a binomial distribution:
The experiment consists of a fixed number n of identical trials (observations).
There are only two possible outcomes for each trial: success or failure.
The probability p for success is the same for each trial (observation).
Each trial is independent of the others.
As I said before: people with exceptional math brains have already worked this out for us!
The mean of X is:
The variance of X is:
The standard deviation of X is:
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Graphical display of the mean, variance, and standard deviation
Mean of X :
Variance of X :
Standard deviation of X :
For n=100, p=0.5:
µ= 50
σ2=25
σ=5
X
pr(X)
µ=50
σ=5
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Binomial distribution
(probability of X successes in 100 trials, p=0.5)
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 7.8886090522101049E-31 7.8886090522101158E-29 3.9048614808440493E-27 1.2755880837423889E-25 3.0933011030752918E-24 5.9391381179045101E-23 9.4036353533488793E-22 1.2627738903068301E-20 1.4679746474816979E-19 1.5005963063146118E-18 1.3655426387462979E-17 1.1172621589742489E-16 8.2863610123923984E-16 5.6092289930040794E-15 3.4857351599382189E-14 1.9984881583645948E-13 1.0616968341311918E-12 5.2460314157070423E-12 2.419003375020489E-11 1.0439909302719939E-10 4.2281632676015464E-10 1.6107288638482146E-9 5.7839809201822048E-9 1.9615239642357071E-8 6.2932227185896111E-8 1.9131397064512392E-7 5.5186722301477995E-7 1.5125249815960639E-6 3.9433687020183031E-6 9.7904326394937912E-6 2.3170690580135296E-5 5.2320914213208622E-5 1.1281697127223065E-4 2.3247133474277876E-4 4.5810527728724036E-4 8.638556657 4165252E-4 1.5597393964779853E-3 2.6979276047186741E-3 4.4728799762441106E-3 7.1107322699265549E-3 1.0843866711637992E-2 1.5869073236543376E-2 2.2292269546572856E-2 3.0068642644214549E-2 3.8952559789096154E-2 4.8474296626430782E-2 5.7958398140297657E-2 6.659049999098024E-2 7.3527010406707352E-2 7.802866410507725E-2 7.9589237387178782E-2 7.802866410507725E-2 7.3527010406707352E-2 6.659049999098024E-2 5.7958398140297643E-2 4.8474296626430782E-2 3.8952559789096154E-2 3.0068642644214549E-2 2.2292269546572856E-2 1.5869073236543376E-2 1.0843866711637992E-2 7.1107322699265549E-3 4.4728799762441098E-3 2.6979276047186741E-3 1.5597393964779846E-3 8.6385566574165252E-4 4.5810527728724047E-4 2.3247133474277876E-4 1.1281697127223065E-4 5.2320914213208622E-5 2.3170690580135296E-5 9.7904326394937895E-6 3.9433687020183031E-6 1.512524981 5960639E-6 5.518672230147791E-7 1.9131397064512423E-7 6.2932227185896124E-8 1.9615239642357035E-8 5.7839809201822048E-9 1.6107288638482146E-9 4.2281632676015614E-10 1.0439909302719901E-10 2.419003375020489E-11 5.2460314157070423E-12 1.0616968341311918E-12 1.9984881583645948E-13 3.4857351599382189E-14 5.6092289930040794E-15 8.2863610123923984E-16 1.1172621589742489E-16 1.3655426387462979E-17 1.5005963063146224E-18 1.4679746474816979E-19 1.2627738903068301E-20 9.4036353533488793E-22 5.9391381179045101E-23 3.0933011030752697E-24 1.2755880837423889E-25 3.9048614808440493E-27 7.8886090522101158E-29 7.8886090522101049E-31
It gets even better:
So we can use a z table to look up the p values (note this is yet another p than the two in the binomial formulas!) and calculate confidence levels and confidence intervals!
If the number of trials is large enough, the binomial distribution converges towards the normal distribution (because of the Central Limit Theorem).
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Binomial distributions for increasing numbers of trials
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 9.3132257461547934E-10 2.7939677238464359E-8 4.0512531995773342E-7 3.781169652938836E-6 2.5522895157337233E-5 1.3271905481815344E-4 5.5299606174230467E-4 1.8959864974021896E-3 5.4509611800313048E-3 1.3324571773409849E-2 2.7981600724160692E-2 5.0875637680292116E-2 8.0553092993795886E-2 0.1115350518375635 0.13543542008847004 0.14446444809436798 0.13543542008847004 0.1115350518375635 8.0553092993795886E-2 5.0875637680292116E-2 2.7981600724160689E-2 1.3324571773409843E-2 5.4509611800313022E-3 1.8959864974021896E-3 5.5299606174230467E-4 1.3271905481815344E-4 2.5522895157337233E-5 3.781169652938836E-6 4.0512531995773411E-7 2.7939677238464359E-8 9.3132257461547934E-10
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 9.5367431640625E-7 1.90734863 28125034E-5 1.8119812011718755E-4 1.0871887207031263E-3 4.6205520629882752E-3 1.4785766601562502E-2 3.6964416503906257E-2 7.3928833007812458E-2 0.12013435363769531 0.16017913818359369 0.17619705200195307 0.16017913818359369 0.12013435363769531 7.3928833007812472E-2 3.6964416503906257E-2 1.4785766601562502E-2 4.6205520629882752E-3 1.0871887207031261E-3 1.8119812011718753E-4 1.9073486328125E-5 9.5367431640625E-7
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 3.0517578125000014E-5 4.5776367187500022E-4 3.2043457031250035E-3 1.3885498046874986E-2 4.1656494140625021E-2 9.1644287109375042E-2 0.152740478515625 0.19638061523437506 0.19638061523437506 0.152740478515625 9.1644287109375042E-2 4.1656494140625021E-2 1.3885498046874986E-2 3.2043457031250035E-3 4.5776367187500022E-4 3.0517578125000014E-5
0 1 2 3 4 5 6 7 8 9 10 9.765625E-4 9.7656250000000017E-3 4.3945312499999972E-2 0.11718750000000003 0.20507812500000006 0.24609375000000008 0.20507812500000006 0.11718750000000003 4.39453124 99999986E-2 9.7656250000000017E-3 9.765625E-4
0 1 2 3 4 5 3.125E-2 0.15624999999999992 0.3125 0.3125 0.15624999999999992 3.125E-2
n=5
n=10
n=20
n=30
Overlay of normal and binomial distributions
Norm.Dist 10 0 1 2 3 4 5 6 7 8 9 10 1.7000733205040698E-3 1.0284844252703535E-2 4.1707100072566027E-2 0.11337165224497912 0.20657661898691132 0.252313252202016 0.20657661898691132 0.11337165224497912 4.1707100072566027E-2 1.0284844252703535E-2 1.7000733205040698E-3
0 1 2 3 4 5 6 7 8 9 10 9.765625E-4 9.7656250000000017E-3 4.3945312499999972E-2 0.11718750000000003 0.20507812500000006 0.24609375000000008 0.20507812500000006 0.11718750000000003 4.39453124 99999986E-2 9.7656250000000017E-3 9.765625E-4
Norm.Dist 5 0 1 2 3 4 5 2.928996512385296E-2 0.14507414696784585 0.32286845174307244 0.32286845174307244 0.14507414696784585 2.928996512385296E-2
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 9.5367431640625E-7 1.90734863 28125034E-5 1.8119812011718755E-4 1.0871887207031263E-3 4.6205520629882752E-3 1.4785766601562502E-2 3.6964416503906257E-2 7.3928833007812458E-2 0.12013435363769531 0.16017913818359369 0.17619705200195307 0.16017913818359369 0.12013435363769531 7.3928833007812472E-2 3.6964416503906257E-2 1.4785766601562502E-2 4.6205520629882752E-3 1.0871887207031261E-3 1.8119812011718753E-4 1.9073486328125E-5 9.5367431640625E-7
Norm.Dist 30 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 4.4561746690499324E-8 3.0803367974105546E-7 1.8634951695992601E-6 9.8662566345422729E-6 4.5716251520361302E-5 1.8538854164963184E-4 6.5794445606580886E-4 2.0435705935538165E-3 5.5550008096306013E-3 1.321516766571643E-2 2.7514099127641471E-2 5.0133957296283636E-2 7.9947105560693316E-2 0.11157517387017278 0.13627822510722956 0.14567312407894387 0.13627822510722956 0.11157517387017278 7.9947105560693316E-2 5.0133957296283636E-2 2.7514099127641471E-2 1.321516766571643E-2 5.5550008096306013E-3 2.0435705935538165E-3 6.5794445606580886E-4 1.8538854164963184E-4 4.5716251520361302E-5 9.8662566345422729E-6 1.8634951695992601E-6 3.0803367974105546E-7 4.4561746690499324E-8
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 9.3132257461547934E-10 2.7939677238464359E-8 4.0512531995773342E-7 3.781169652938836E-6 2.5522895157337233E-5 1.3271905481815344E-4 5.5299606174230467E-4 1.8959864974021896E-3 5.4509611800313048E-3 1.3324571773409849E-2 2.7981600724160692E-2 5.0875637680292116E-2 8.0553092993795886E-2 0.1115350518375635 0.13543542008847004 0.14446444809436798 0.13543542008847004 0.1115350518375635 8.0553092993795886E-2 5.0875637680292116E-2 2.7981600724160689E-2 1.3324571773409843E-2 5.4509611800313022E-3 1.8959864974021896E-3 5.5299606174230467E-4 1.3271905481815344E-4 2.5522895157337233E-5 3.781169652938836E-6 4.0512531995773411E-7 2.7939677238464359E-8 9.3132257461547934E-10
Norm.Dist 20 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 8.0999109560891024E-6 5.4155149644273216E-5 2.9644244014386559E-4 1.3285628439771073E-3 4.8748912161279473E-3 1.464498256192648E-2 3.6020844672153669E-2 7.2537073483922923E-2 0.11959341596728197 0.16143422587153622 0.17841241161527713 0.16143422587153622 0.11959341596728197 7.2537073483922923E-2 3.6020844672153669E-2 1.464498256192648E-2 4.8748912161279473E-3 1.3285628439771073E-3 2.9644244014386559E-4 5.4155149644273216E-5 8.0999109560891024E-6
The sampling distribution of a proportion (for a categorical variable)
Experiments or studies, whether they involve numerical or categorical variables, are often performed on a statistical sample (group of objects from a population).
If we took another sample, how likely would it be that it gives the same result?
If we took a lot of samples, what would the distribution of the data look like?
As we learned with numerical variables, such a distribution is called the sampling distribution.
As we saw with the sampling distribution of mean values, it allows us to calculate the standard error of the mean, the margin of error and the corresponding confidence interval.
These parameters help us to establish the precision of a mean value from just one experiment and tell us how likely the confidence interval of that mean value reflects the mean value of the population.
One can also construct a sampling distribution for a categorical variable.
The only numbers we get for categorical variables are the counts for each category. We could do sampling distributions for these counts.
However, it is more useful to use the proportions derived from these counts. (remember they are the summary statistic from the counts?; see the 2-ways tables earlier in the presentation).
Going from counts to proportions can be likened to standardizing the data.
Therefore, the sampling distribution for a categorical variable is the sampling distribution of its proportion.
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Assuming that the sample size is reasonably large, we can apply the normal distribution (so we don’t need to deal with the binomial distribution, its formulas and its tables).
Thus we can forget everything about the binomial distribution we just learned, except for the formulas for mean and standard deviation (and one more, the standard error)!
The sampling distribution of a proportion is NORMAL!
(if certain conditions are met)
What is a large enough sample size?
Remember, the Central Limit Theorem can be applied to the sampling distribution of mean values if the sample size is 30 or above.
For a proportion the guidelines are a bit more complicated, as they not only depend on the sample size but also the value of the proportion.
A sampling distribution of a proportion can be assumed normal
If n x p is > 10 and n x (1-p) is > 10
(n times success and n times failure must be greater than 10)
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Constructing the sampling distribution of a proportion
Example:
Smoking rate of males ages 18-60 in country X: proportion of smokers p(pop) =0.45
This is called the population proportion (in analogy to the population mean).
It’s usually just called p, but with so many other p’s around, I’ll call it p(pop)
After a 1-year anti-smoking campaign a new survey is conducted.
Sample size: 500 men
Yes answers: 190 (convert to sample proportion: p(s) =0.38).
Question: What is the probability that that result occurred as a sampling error?
Solution:
Create a sampling distribution of the original data.
Find where the sample proportion p=0.38 lies on that distribution.
Determine the area under the curve (its p-value, unrelated to the proportion p).
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
16
In order to draw a normal distribution curve for a sampling distribution of a proportion we need:
A mean value. The population proportion has not been calculated as a mean, but it becomes the mean of the sampling distribution. Thus: mean = p(pop) = 0.45 (enter into Excel NORM.DIST)
The standard error S.E. of the sampling proportion
Constructing the sampling distribution of a proportion (cont’d)
Using n=500 and p(pop) =0.45 as inputs, that comes out to S.E. = 0.0222.
(enter this as the standard deviation into NORM.DIST)
A range of x values (let’s go from 0.36 to 0.54 around the population proportion (enter separately into NORM.DIST to get f(x) for each x)
Put Excel to work to calculate the probability function and create the graph.
S.E. is calculated using the formula:
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Area under the curve to the left:
The p-value: probability that a proportion p has the value 0.38 or smaller.
Since the area looks negligible we can predict that the drop in the smoking rate will be statistically significant.
To calculate the area we need to convert the normal distribution to a z distribution and calculate the z score for p=0.38. Then we can look up the p-value for that z score.
Enter: p #4 (shown here in green to try to reduce the confusion). This is the p we know from before: the p-value for significance, the area under the tail of a probability curve.
f(X)
Probability density function
Proportion p
P(pop)=0.45
S.E =0.022
p=0.38
Constructing the sampling distribution of a proportion (cont’d)
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Normal sampling distribution of a proportion;
p(pop)=0.45, sample size 500
0.36 0.37 0.38 0.39 0.4 0.41 0.42 0.43 0.44 0.45 0.46 0.47 0.48 0.49 0.5 0.51 0.52 0.53 0.54 5.0152083286464722E-3 2.7928472239272133E-2 0.12707763437431671 0.47244885945179271 1.435172913608604 3.5621925792645803 7.2242755234671767 11.971124525186125 16.208369805568921 17.931122038494539 16.208369805568921 11.971124525186152 7.2242755234671989 3.562192579264595 1.435172913608604 0.47244885945179271 0.12707763437431671 2.7928472239272133E-2 5.0152083286464722E-3
Converting proportions to z values
The general rule for z value conversion is to
Shift the curve so zero is in the middle.
This is done by subtracting the mean (or in this case the population proportion p (pop) =0.45) from each value for the proportion (like p=0.38) to get its z-score
2) Squeeze or stretch the x axis so its unit becomes the standard deviation. This is done by dividing by the standard deviation (or, for a sampling distribution, by the standard error) Remember: the standard error is the standard deviation of a sampling distribution.
The formula for converting proportions to a z-score is:
Creating a small calculator in Excel and using p=0.38, p(pop)=0.45, n=500
results in a z-score for p=0.38 of z=-3.14 and a p (left tail) probability of = 0.000845 (or 0.8%).
Thus it can be concluded that there is only a 0.08% chance that the more recent proportion found for men smoking is part of the previous population distribution. Therefore the anti-smoking program is working.
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
If the researchers want to publish their findings about the smoking program, they need to include a confidence interval (or margin of error) and corresponding confidence level along with the proportion of men smoking.
Calculating the confidence interval for one proportion
C.I. = value +/- MOE -> C.I. (95% confidence) = 0.38 +/- 0.044
which can also be written as: C.I. ranges from (variable – MOE) to (variable + MOE)
-> C.I. (95% confidence) ranges from 0.336 to 0.424
This should now be super easy for you! You know from dealing with mean values that:
Margin of error MOE = z * S.E.
with z being the score belonging to the desired confidence level.
You may remember that z=1.96 for a 95% confidence level.*
The formula for the standard error (S.E.) of a proportion is shown in slide 17 and it is already calculated there for the smoking study to be 0.022.
Rounding z for 95% confidence off to 2, the margin of error becomes 2 x 0.022 = 0.044
Lastly, the confidence interval for the value of a variable is defined as
(* Don’t confuse the z score belonging to a confidence level with the z score obtained through standardization of a proportion. These are two different processes for two different purposes, even though it’s the same z from the standard normal distribution.)
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
Appendix Lecture 5 (for those interested): Linking parameters of the sampling distribution for a proportion to the binomial distribution
We also could have created a binomial distribution instead of a normal distribution.
This would require a little bit of mental gymnastics, in order to equate the particular terms and logic of the binomial distribution to the counting of a categorical variable in a sample.
| Binomial distribution Terms and input values | Sampling distribution of a proportion Terms and input values | Comments |
| Statistical experiment | Survey or scientific experiment | |
| Number of trials | Sample size | Each individual or subject tested represents a trial |
| Probability p of success for each trial | Proportion p of one type of outcomes | Note the switch from probability p to proportion p !* |
| Probability of failure for each trial (=1-p) | Proportion of other type of outcomes (=1-p) | Success + failure =1, as we have binary variables here. |
| X values (numbers of success) for all possible outcomes | X values (possible proportions) for range of possible outcomes | Proportions range from 0 to 1. X values for binomials depend on number of trials. |
* Yet another p to deal with! The proportion or %age of people responding to a question in a certain way is mathematically linked to the probability of success for each trial:
If x % of people answered yes, that is equivalent to an x% probability or chance for each person to answer yes.
Dr. Doerre Data Analyses and Statistical Concepts in Biotechnology FSU Math 924
