SPSS questions
Richard G. Lomax The Ohio State University
Debbie L. Hahs-Vaughn University of Central Florida
Routledge Taylor & Francis Group 711 Third Avenue New York, NY 10017
Routledge Taylor & Francis Group 27 Church Road Hove, East Sussex BN3 2FA
© 2012 by Taylor & Francis Group, LLC Routledge is an imprint of Taylor & Francis Group, an Informa business
Printed in the United States of America on acid-free paper Version Date: 20111003
International Standard Book Number: 978-0-415-88005-3 (Hardback)
For permission to photocopy or use material electronically from this work, please access www.copyright.com (http:// www.copyright.com/) or contact the Copyright Clearance Center, Inc. (CCC), 222 Rosewood Drive, Danvers, MA 01923, 978-750-8400. CCC is a not-for-profit organization that provides licenses and registration for a variety of users. For organizations that have been granted a photocopy license by the CCC, a separate system of payment has been arranged.
Trademark Notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe.
Library of Congress Cataloging‑in‑Publication Data
Lomax, Richard G. An introduction to statistical concepts / Richard G. Lomax, Debbie L. Hahs-Vaughn. -- 3rd ed.
p. cm. Includes bibliographical references and index. ISBN 978-0-415-88005-3 1. Statistics. 2. Mathematical statistics. I. Hahs-Vaughn, Debbie L. II. Title.
QA276.12.L67 2012 519.5--dc23 2011035052
Visit the Taylor & Francis Web site at http://www.taylorandfrancis.com
and the Psychology Press Web site at http://www.psypress.com
This book is dedicated to our families
and to all of our former students.
vii
Contents
Preface�������������������������������������������������������������������������������������������������������������������������������������������� xiii Acknowledgments���������������������������������������������������������������������������������������������������������������������� xvii
1. Introduction������������������������������������������������������������������������������������������������������������������������������ 1 1�1� What�Is�the�Value�of�Statistics?������������������������������������������������������������������������������������ 3 1�2� Brief�Introduction�to�History�of�Statistics������������������������������������������������������������������� 4 1�3� General�Statistical�Definitions�������������������������������������������������������������������������������������� 5 1�4� Types�of�Variables���������������������������������������������������������������������������������������������������������� 7 1�5� Scales�of�Measurement�������������������������������������������������������������������������������������������������� 8 1�6� Summary����������������������������������������������������������������������������������������������������������������������� 13 Problems����������������������������������������������������������������������������������������������������������������������������������� 14
2. Data Representation�������������������������������������������������������������������������������������������������������������� 17 2�1� �Tabular�Display�of�Distributions������������������������������������������������������������������������������� 18 2�2� �Graphical�Display�of�Distributions��������������������������������������������������������������������������� 23 2�3� �Percentiles��������������������������������������������������������������������������������������������������������������������� 29 2�4� �SPSS�������������������������������������������������������������������������������������������������������������������������������� 33 2�5� �Templates�for�Research�Questions�and�APA-Style�Paragraph������������������������������ 41 2�6� �Summary����������������������������������������������������������������������������������������������������������������������� 42 Problems����������������������������������������������������������������������������������������������������������������������������������� 43
3. Univariate Population Parameters and Sample Statistics��������������������������������������������� 49 3�1� �Summation�Notation��������������������������������������������������������������������������������������������������� 50 3�2� Measures�of�Central�Tendency����������������������������������������������������������������������������������� 51 3�3� �Measures�of�Dispersion����������������������������������������������������������������������������������������������� 56 3�4� �SPSS�������������������������������������������������������������������������������������������������������������������������������� 65 3�5� �Templates�for�Research�Questions�and�APA-Style�Paragraph������������������������������ 69 3�6� �Summary����������������������������������������������������������������������������������������������������������������������� 70 Problems����������������������������������������������������������������������������������������������������������������������������������� 71
4. Normal Distribution and Standard Scores���������������������������������������������������������������������� 77 4�1� �Normal�Distribution���������������������������������������������������������������������������������������������������� 78 4�2� �Standard�Scores������������������������������������������������������������������������������������������������������������ 84 4�3� �Skewness�and�Kurtosis�Statistics������������������������������������������������������������������������������� 87 4�4� �SPSS�������������������������������������������������������������������������������������������������������������������������������� 91 4�5� �Templates�for�Research�Questions�and�APA-Style�Paragraph������������������������������ 98 4�6� �Summary����������������������������������������������������������������������������������������������������������������������� 99 Problems����������������������������������������������������������������������������������������������������������������������������������� 99
5. Introduction to Probability and Sample Statistics������������������������������������������������������� 105 5�1� �Brief�Introduction�to�Probability������������������������������������������������������������������������������ 106 5�2� �Sampling�and�Estimation����������������������������������������������������������������������������������������� 109 5�3� �Summary��������������������������������������������������������������������������������������������������������������������� 117
� Appendix:�Probability�That�at�Least�Two Individuals�Have�the�Same�Birthday�������� 117 Problems��������������������������������������������������������������������������������������������������������������������������������� 118
viii Contents
6. Introduction to Hypothesis Testing: Inferences About a Single Mean������������������� 121 6�1� Types�of�Hypotheses������������������������������������������������������������������������������������������������� 122 6�2� Types�of�Decision�Errors������������������������������������������������������������������������������������������� 124 6�3� Level�of�Significance�(α)��������������������������������������������������������������������������������������������� 127 6�4� Overview�of�Steps�in�Decision-Making�Process��������������������������������������������������� 129 6�5� Inferences�About�μ�When�σ�Is�Known�������������������������������������������������������������������� 130 6�6� Type�II�Error�(β)�and�Power�(1�−�β)��������������������������������������������������������������������������� 134 6�7� Statistical�Versus�Practical�Significance������������������������������������������������������������������ 138 6�8� Inferences�About�μ�When�σ�Is�Unknown��������������������������������������������������������������� 139 6�9� SPSS������������������������������������������������������������������������������������������������������������������������������ 145 6�10� G*Power����������������������������������������������������������������������������������������������������������������������� 149 6�11� Template�and�APA-Style�Write-Up�������������������������������������������������������������������������� 155 6�12� Summary��������������������������������������������������������������������������������������������������������������������� 156 Problems��������������������������������������������������������������������������������������������������������������������������������� 157
7. Inferences About the Difference Between Two Means����������������������������������������������� 163 7�1� �New�Concepts������������������������������������������������������������������������������������������������������������� 164 7�2� �Inferences�About�Two�Independent�Means����������������������������������������������������������� 166 7�3� �Inferences�About�Two�Dependent�Means�������������������������������������������������������������� 176 7�4� �SPSS������������������������������������������������������������������������������������������������������������������������������ 180 7�5� �G*Power����������������������������������������������������������������������������������������������������������������������� 192 7�6� �Template�and�APA-Style�Write-Up�������������������������������������������������������������������������� 195 7�7� �Summary��������������������������������������������������������������������������������������������������������������������� 198 Problems��������������������������������������������������������������������������������������������������������������������������������� 198
8. Inferences About Proportions������������������������������������������������������������������������������������������ 205 8�1� �Inferences�About�Proportions�Involving�Normal�Distribution�������������������������� 206 8�2� �Inferences�About�Proportions�Involving�Chi-Square�Distribution�������������������� 217 8�3� �SPSS������������������������������������������������������������������������������������������������������������������������������ 224 8�4� �G*Power����������������������������������������������������������������������������������������������������������������������� 231 8�5� �Template�and�APA-Style�Write-Up�������������������������������������������������������������������������� 234 8�6� �Summary��������������������������������������������������������������������������������������������������������������������� 236 Problems��������������������������������������������������������������������������������������������������������������������������������� 237
9. Inferences About Variances���������������������������������������������������������������������������������������������� 241 9�1� �New�Concepts������������������������������������������������������������������������������������������������������������� 242 9�2� �Inferences�About�Single�Variance���������������������������������������������������������������������������� 244 9�3� �Inferences�About�Two�Dependent�Variances��������������������������������������������������������� 246 9�4� Inferences�About�Two�or�More�Independent�Variances�(Homogeneity�
of Variance�Tests)�������������������������������������������������������������������������������������������������������� 248 9�5� �SPSS������������������������������������������������������������������������������������������������������������������������������ 252 9�6� �Template�and�APA-Style�Write-Up�������������������������������������������������������������������������� 253 9�7� �Summary��������������������������������������������������������������������������������������������������������������������� 253 Problems��������������������������������������������������������������������������������������������������������������������������������� 254
ixContents
10. Bivariate Measures of Association����������������������������������������������������������������������������������� 259 10�1� �Scatterplot������������������������������������������������������������������������������������������������������������������� 260 10�2� �Covariance������������������������������������������������������������������������������������������������������������������ 263 10�3� �Pearson�Product–Moment�Correlation�Coefficient����������������������������������������������� 265 10�4� �Inferences�About�Pearson�Product–Moment�Correlation�Coefficient���������������� 266 10�5� �Assumptions�and�Issues�Regarding�Correlations������������������������������������������������� 269 10�6� �Other�Measures�of�Association�������������������������������������������������������������������������������� 272 10�7� �SPSS������������������������������������������������������������������������������������������������������������������������������ 276 10�8� �G*Power����������������������������������������������������������������������������������������������������������������������� 283 10�9� �Template�and�APA-Style�Write-Up�������������������������������������������������������������������������� 286 10�10� �Summary��������������������������������������������������������������������������������������������������������������������� 287 Problems��������������������������������������������������������������������������������������������������������������������������������� 287
11. One-Factor Analysis of Variance: Fixed-Effects Model��������������������������������������������� 291 11�1� �Characteristics�of�One-Factor�ANOVA�Model������������������������������������������������������� 292 11�2� �Layout�of�Data������������������������������������������������������������������������������������������������������������ 296 11�3� �ANOVA�Theory���������������������������������������������������������������������������������������������������������� 296 11�4� �ANOVA�Model����������������������������������������������������������������������������������������������������������� 302 11�5� �Assumptions�and�Violation�of�Assumptions��������������������������������������������������������� 309 11�6� �Unequal�n’s�or�Unbalanced�Procedure������������������������������������������������������������������� 312 11�7� �Alternative�ANOVA�Procedures������������������������������������������������������������������������������ 312 11�8� �SPSS�and�G*Power������������������������������������������������������������������������������������������������������ 313 11�9� �Template�and�APA-Style�Write-Up�������������������������������������������������������������������������� 334 11�10� �Summary��������������������������������������������������������������������������������������������������������������������� 336 Problems��������������������������������������������������������������������������������������������������������������������������������� 336
12. Multiple Comparison Procedures������������������������������������������������������������������������������������ 341 12�1� �Concepts�of�Multiple�Comparison�Procedures������������������������������������������������������ 342 12�2� �Selected�Multiple�Comparison�Procedures������������������������������������������������������������ 348 12�3� �SPSS������������������������������������������������������������������������������������������������������������������������������ 362 12�4� �Template�and�APA-Style�Write-Up�������������������������������������������������������������������������� 366 12�5� �Summary��������������������������������������������������������������������������������������������������������������������� 366 Problems��������������������������������������������������������������������������������������������������������������������������������� 367
13. Factorial Analysis of Variance: Fixed-Effects Model��������������������������������������������������� 371 13�1� �Two-Factor�ANOVA�Model��������������������������������������������������������������������������������������� 372 13�2� �Three-Factor�and�Higher-Order�ANOVA��������������������������������������������������������������� 390 13�3� �Factorial�ANOVA�With�Unequal�n’s������������������������������������������������������������������������ 393 13�4� �SPSS�and�G*Power������������������������������������������������������������������������������������������������������ 395 13�5� �Template�and�APA-Style�Write-Up�������������������������������������������������������������������������� 417 13�6� �Summary��������������������������������������������������������������������������������������������������������������������� 419 Problems��������������������������������������������������������������������������������������������������������������������������������� 420
14. Introduction to Analysis of Covariance: One- Factor Fixed-Effects Model With Single Covariate��������������������������������������������������������������������������������������������������������� 427 14�1� �Characteristics�of�the�Model������������������������������������������������������������������������������������� 428 14�2� �Layout�of�Data������������������������������������������������������������������������������������������������������������ 431 14�3� �ANCOVA�Model��������������������������������������������������������������������������������������������������������� 431
x Contents
14�4� �ANCOVA�Summary�Table���������������������������������������������������������������������������������������� 432 14�5� �Partitioning�the�Sums�of�Squares���������������������������������������������������������������������������� 433 14�6� �Adjusted�Means�and�Related�Procedures�������������������������������������������������������������� 434 14�7� �Assumptions�and�Violation�of�Assumptions��������������������������������������������������������� 436 14�8� �Example����������������������������������������������������������������������������������������������������������������������� 441 14�9� �ANCOVA�Without�Randomization������������������������������������������������������������������������� 443 14�10� �More�Complex�ANCOVA�Models���������������������������������������������������������������������������� 444 14�11� �Nonparametric�ANCOVA�Procedures�������������������������������������������������������������������� 444 14�12� �SPSS�and�G*Power������������������������������������������������������������������������������������������������������ 445 14�13� �Template�and�APA-Style�Paragraph������������������������������������������������������������������������ 469 14�14� �Summary��������������������������������������������������������������������������������������������������������������������� 471 Problems��������������������������������������������������������������������������������������������������������������������������������� 471
15. Random- and Mixed-Effects Analysis of Variance Models��������������������������������������� 477 15�1� �One-Factor�Random-Effects�Model������������������������������������������������������������������������� 478 15�2� �Two-Factor�Random-Effects�Model������������������������������������������������������������������������� 483 15�3� �Two-Factor�Mixed-Effects�Model����������������������������������������������������������������������������� 488 15�4� �One-Factor�Repeated�Measures�Design������������������������������������������������������������������ 493 15�5� �Two-Factor�Split-Plot�or�Mixed�Design������������������������������������������������������������������� 500 15�6� �SPSS�and�G*Power������������������������������������������������������������������������������������������������������ 508 15�7� �Template�and�APA-Style�Write-Up�������������������������������������������������������������������������� 548 15�8� �Summary��������������������������������������������������������������������������������������������������������������������� 551 Problems��������������������������������������������������������������������������������������������������������������������������������� 551
16. Hierarchical and Randomized Block Analysis of Variance Models������������������������ 557 16�1� �Two-Factor�Hierarchical�Model������������������������������������������������������������������������������� 558 16�2� �Two-Factor�Randomized�Block�Design�for�n�=�1��������������������������������������������������� 566 16�3� �Two-Factor�Randomized�Block�Design�for�n�>�1��������������������������������������������������� 574 16�4� �Friedman�Test������������������������������������������������������������������������������������������������������������� 574 16�5� �Comparison�of�Various�ANOVA�Models��������������������������������������������������������������� 575 16�6� �SPSS������������������������������������������������������������������������������������������������������������������������������ 576 16�7� �Template�and�APA-Style�Write-Up�������������������������������������������������������������������������� 603 16�8� �Summary��������������������������������������������������������������������������������������������������������������������� 605 Problems��������������������������������������������������������������������������������������������������������������������������������� 605
17. Simple Linear Regression�������������������������������������������������������������������������������������������������� 611 17�1� �Concepts�of�Simple�Linear�Regression������������������������������������������������������������������� 612 17�2� �Population�Simple�Linear�Regression�Model��������������������������������������������������������� 614 17�3� �Sample�Simple�Linear�Regression�Model��������������������������������������������������������������� 615 17�4� �SPSS������������������������������������������������������������������������������������������������������������������������������ 634 17�5� �G*Power����������������������������������������������������������������������������������������������������������������������� 647 17�6� �Template�and�APA-Style�Write-Up�������������������������������������������������������������������������� 650 17�7� �Summary��������������������������������������������������������������������������������������������������������������������� 652 Problems��������������������������������������������������������������������������������������������������������������������������������� 652
xiContents
18. Multiple Regression������������������������������������������������������������������������������������������������������������ 657 18�1� Partial�and�Semipartial�Correlations���������������������������������������������������������������������� 658 18�2� Multiple�Linear�Regression�������������������������������������������������������������������������������������� 661 18�3� Methods�of�Entering�Predictors������������������������������������������������������������������������������� 676 18�4� Nonlinear�Relationships������������������������������������������������������������������������������������������� 679 18�5� Interactions����������������������������������������������������������������������������������������������������������������� 680 18�6� Categorical�Predictors����������������������������������������������������������������������������������������������� 680 18�7� SPSS������������������������������������������������������������������������������������������������������������������������������ 682 18�8� G*Power����������������������������������������������������������������������������������������������������������������������� 698 18�9� Template�and�APA-Style�Write-Up�������������������������������������������������������������������������� 701 18�10� Summary��������������������������������������������������������������������������������������������������������������������� 703 Problems��������������������������������������������������������������������������������������������������������������������������������� 704
19. Logistic Regression������������������������������������������������������������������������������������������������������������� 709 19�1� �How�Logistic�Regression�Works������������������������������������������������������������������������������ 710 19�2� �Logistic�Regression�Equation����������������������������������������������������������������������������������� 711 19�3� �Estimation�and�Model�Fit����������������������������������������������������������������������������������������� 715 19�4� �Significance�Tests������������������������������������������������������������������������������������������������������� 716 19�5� �Assumptions�and�Conditions���������������������������������������������������������������������������������� 721 19�6� �Effect�Size�������������������������������������������������������������������������������������������������������������������� 725 19�7� �Methods�of�Predictor�Entry�������������������������������������������������������������������������������������� 726 19�8� �SPSS������������������������������������������������������������������������������������������������������������������������������ 727 19�9� �G*Power����������������������������������������������������������������������������������������������������������������������� 746 19�10� �Template�and�APA-Style�Write-Up�������������������������������������������������������������������������� 749 19�11� �What�Is�Next?�������������������������������������������������������������������������������������������������������������� 751 19�12� �Summary��������������������������������������������������������������������������������������������������������������������� 752 Problems��������������������������������������������������������������������������������������������������������������������������������� 752
Appendix: Tables������������������������������������������������������������������������������������������������������������������������ 757
References������������������������������������������������������������������������������������������������������������������������������������ 783
Odd-Numbered Answers to Problems����������������������������������������������������������������������������������� 793
Author Index�������������������������������������������������������������������������������������������������������������������������������� 809
Subject Index������������������������������������������������������������������������������������������������������������������������������� 813
xiii
Preface
Approach
We�know,�we�know!�We’ve�heard�it�a�million�times�before��When�you�hear�someone�at�a� party�mention�the�word�statistics�or�statistician,�you�probably�say�“I�hate�statistics”�and�turn� the�other�cheek��In�the�many�years�that�we�have�been�in�the�field�of�statistics,�it�is�extremely� rare� when� someone� did� not� have� that� reaction�� Enough� is� enough�� With� the� help� of� this� text,�we�hope�that�“statistics�hating”�will�become�a�distant�figment�of�your�imagination�
As�the�title�suggests,�this�text�is�designed�for�a�course�in�statistics�for�students�in�educa- tion� and� the� behavioral� sciences�� We� begin� with� the� most� basic� introduction� to�statistics� in� the� first� chapter� and� proceed� through� intermediate� statistics�� The� text� is� designed� for� you�to�become�a�better�prepared�researcher�and�a�more�intelligent�consumer�of�research�� We�do�not�assume�that�you�have�extensive�or�recent�training�in�mathematics��Many�of�you� have�only�had�algebra,�perhaps�some�time�ago��We�also�do�not�assume�that�you�have�ever� had�a�statistics�course��Rest�assured;�you�will�do�fine�
We�believe�that�a�text�should�serve�as�an�effective�instructional�tool��You�should�find�this� text�to�be�more�than�a�reference�book;�you�might�actually�use�it�to�learn�statistics��(What�an� oxymoron�that�a�statistics�book�can�actually�teach�you�something�)�This�text�is�not�a�theo- retical�statistics�book,�nor�is�it�a�cookbook�on�computing�statistics�or�a�statistical�software� manual�� Recipes� have� to� be� memorized;� consequently,� you� tend� not� to� understand� how� or�why�you�obtain�the�desired�product��As�well,�knowing�how�to�run�a�statistics�package� without� understanding� the� concepts� or� the� output� is� not� particularly� useful�� Thus,� con- cepts�drive�the�field�of�statistics�
Goals and Content Coverage
Our�goals�for�this�text�are�lofty,�but�the�effort�and�its�effects�will�be�worthwhile��First,�the� text�provides�a�comprehensive�coverage�of�topics�that�could�be�included�in�an�undergradu- ate�or�graduate�one-�or�two-course�sequence�in�statistics��The�text�is�flexible�enough�so�that� instructors�can�select�those�topics�that�they�desire�to�cover�as�they�deem�relevant�in�their� particular�discipline��In�other�words,�chapters�and�sections�of�chapters�from�this�text�can� be�included�in�a�statistics�course�as�the�instructor�sees�fit��Most�of�the�popular�as�well�as� many�of�the�lesser-known�procedures�and�models�are�described�in�the�text��A�particular� feature�is�a�thorough�and�up-to-date�discussion�of�assumptions,�the�effects�of�their�viola- tion,�and�how�to�deal�with�their�violation�
The�first�five�chapters�of�the�text�cover�basic�descriptive�statistics,�including�ways�of�repre- senting�data�graphically,�statistical�measures�which�describe�a�set�of�data,�the�normal�distri- bution�and�other�types�of�standard�scores,�and�an�introduction�to�probability�and�sampling��
xiv Preface
The�remainder�of�the�text�covers�different�inferential�statistics��In�Chapters�6�through�10,�we� deal�with�different�inferential�tests�involving�means�(e�g�,�t�tests),�proportions,�variances,�and� correlations��In�Chapters�11�through�16,�all�of�the�basic�analysis�of�variance�(ANOVA)�models� are�considered��Finally,�in�Chapters�17�through�19�we�examine�various�regression�models�
Second,�the�text�communicates�a�conceptual,�intuitive�understanding�of�statistics,�which� requires� only� a� rudimentary� knowledge� of� basic� algebra� and� emphasizes� the� important� concepts�in�statistics��The�most�effective�way�to�learn�statistics�is�through�the�conceptual� approach��Statistical�concepts�tend�to�be�easy�to�learn�because�(a)�concepts�can�be�simply� stated,�(b)�concepts�can�be�made�relevant�through�the�use�of�real-life�examples,�(c)�the�same� concepts�are�shared�by�many�procedures,�and�(d)�concepts�can�be�related�to�one�another�
This�text�will�help�you�to�reach�these�goals��The�following�indicators�will�provide�some� feedback�as�to�how�you�are�doing��First,�there�will�be�a�noticeable�change�in�your�attitude� toward� statistics�� Thus,� one� outcome� is� for� you� to� feel� that� “statistics� is� not� half� bad,”� or� “this� stuff� is� OK�”� Second,� you� will� feel� comfortable� using� statistics� in� your� own� work�� Finally,�you�will�begin�to�“see�the�light�”�You�will�know�when�you�have�reached�this�high- est�stage�of�statistics�development�when�suddenly,�in�the�middle�of�the�night,�you�wake�up� from�a�dream�and�say,�“now�I�get�it!”�In�other�words,�you�will�begin�to�think�statistics�rather� than�think�of�ways�to�get�out�of�doing�statistics�
Pedagogical Tools
The�text�contains�several�important�pedagogical�features�to�allow�you�to�attain�these�goals�� First,�each�chapter�begins�with�an�outline�(so�you�can�anticipate�what�will�be�covered),�and� a�list�of�key�concepts�(which�you�will�need�in�order�to�understand�what�you�are�doing)�� Second,�realistic�examples�from�education�and�the�behavioral�sciences�are�used�to�illustrate� the�concepts�and�procedures�covered�in�each�chapter��Each�of�these�examples�includes�an� initial� vignette,� an� examination� of� the� relevant� procedures� and� necessary� assumptions,� how�to�run�SPSS�and�develop�an�APA�style�write-up,�as�well�as�tables,�figures,�and�anno- tated�SPSS�output�to�assist�you��Third,�the�text�is�based�on�the�conceptual�approach��That� is,�material�is�covered�so�that�you�obtain�a�good�understanding�of�statistical�concepts��If� you� know� the� concepts,� then� you� know� statistics�� Finally,� each� chapter� ends� with� three� sets�of�problems,�computational,�conceptual,�and�interpretive��Pay�particular�attention�to� the� conceptual� problems� as� they� provide� the� best� assessment� of� your� understanding� of� the�concepts�in�the�chapter��We�strongly�suggest�using�the�example�data�sets�and�the�com- putational� and� interpretive� problems� for� additional� practice� through� available� statistics� software��This�will�serve�to�reinforce�the�concepts�covered��Answers�to�the�odd-numbered� problems�are�given�at�the�end�of�the�text�
New to This Edition
A� number� of� changes� have� been� made� in� the� third� edition� based� on� the� suggestions� of�reviewers,�instructors,�teaching�assistants,�and�students��These�improvements�have� been� made� in� order� to� better� achieve� the� goals� of� the� text�� You� will� note� the� addition� of� a� coauthor� to� this� edition,� Debbie� Hahs-Vaughn,� who� has� contributed� greatly� to�
xvPreface
the�further�development�of�this�text��The�changes�include�the�following:�(a)�additional� end�of�chapter�problems�have�been�included;�(b)�more�information�on�power�has�been� added,�particularly�use�of�the�G*Power�software�with�screenshots;�(c)�content�has�been� updated�and�numerous�additional�references�have�been�provided;�(d)�the�final�chapter� on� logistic� regression� has� been� added� for� a� more� complete� presentation� of� regression� models;�(e)�numerous�SPSS�(version�19)�screenshots�on�statistical�techniques�and�their� assumptions�have�been�included�to�assist�in�the�generation�and�interpretation�of�output;� (f)�more�information�has�been�added�to�most�chapters�on�SPSS;�(g)�research�vignettes� and�templates�have�been�added�to�the�beginning�and�end�of�each�chapter,�respectively;� (h)�a�discussion�of�expected�mean�squares�has�been�folded�into�the�analysis�of�variance� chapters�to�provide�a�rationale�for�the�formation�of�proper�F�ratios;�and�(i)�a�website�for� the�text�that�provides�students�and�instructors�access�to�detailed�solutions�to�the�book’s� odd-numbered�problems;�chapter�outlines;�lists�of�key�terms�for�each�chapter;�and�SPSS� datasets� that� correspond� to� the� chapter� examples� and� end-of-chapter� problems� that� can�be�used�in�SPSS�and�other�packages�such�as�SAS,�HLM,�STATA,�and�LISREL��Only� instructors� are� granted� access� to� the� PowerPoint� slides� for� each� chapter� that� include� examples� and� APA� style� write� ups,� chapter� outlines,� and� key� terms;� multiple-choice� (approximately�25�for�each�chapter)�and�short�answer�(approximately�5�for�each�chapter)� test� questions;� and� answers� to� the� even-numbered� problems�� This� material� is� available� at:� http://www�psypress�com/an-introduction-to-statistical-concepts-9780415880053�
xvii
Acknowledgments
There� are� many� individuals� whose� assistance� enabled� the� completion� of� this� book�� We� would� like� to� thank� the� following� individuals� whom� we� studied� with� in� school:� Jamie� Algina,� Lloyd� Bond,� Amy� Broeseker,� Jim� Carlson,� Bill� Cooley,� Judy� Giesen,� Brian� Gray,� Harry�Hsu,�Mary�Nell�McNeese,�Camille�Ogden,�Lou�Pingel,�Rod�Roth,�Charles�Stegman,� and�Neil�Timm��Next,�numerous�colleagues�have�played�an�important�role�in�our�personal� and� professional� lives� as� statisticians�� Rather� than� include� an� admittedly� incomplete� listing,�we�just�say�“thank�you”�to�all�of�you��You�know�who�you�are�
Thanks� also� to� all� of� the� wonderful� people� at� Lawrence� Erlbaum� Associates� (LEA),� in� particular,�to�Ray�O’Connell�for�inspiring�this�project�back�in�1986,�and�to�Debra�Riegert� (formerly� at� LEA� and� now� at� Routledge)� for� supporting� the� development� of� subsequent� texts� and� editions�� We� are� most� appreciative� of� the� insightful� suggestions� provided� by� the� reviewers� of� this� text� over� the� years,� and� in� particular� the� reviewers� of� this� edition:� Robert�P��Conti,�Sr��(Mount�Saint�Mary�College),�Feifei�Ye�(University�of�Pittsburgh),�Nan� Thornton� (Capella� University),� and� one� anonymous� reviewer�� A� special� thank� you� to� all�of�the�terrific�students�that�we�have�had�the�pleasure�of�teaching�at�the�University�of� Pittsburgh,�the�University�of�Illinois–Chicago,�Louisiana�State�University,�Boston�College,� Northern� Illinois� University,� the� University� of� Alabama,� The� Ohio� State� University,� and� the� University� of� Central� Florida�� For� all� of� your� efforts,� and� the� many� lights� that� you� have�seen�and�shared�with�us,�this�book�is�for�you��We�are�most�grateful�to�our�families,� in�particular�to�Lea�and�Kristen,�and�to�Mark�and�Malani��It�is�because�of�your�love�and� understanding�that�we�were�able�to�cope�with�such�a�major�project��Thank�you�one�and�all�
Richard G. Lomax Debbie L. Hahs-Vaughn
1
1 Introduction
Chapter Outline
1�1� What�Is�the�Value�of�Statistics? 1�2� Brief�Introduction�to�History�of�Statistics 1�3� General�Statistical�Definitions 1�4� Types�of�Variables 1�5� Scales�of�Measurement
1�5�1� Nominal�Measurement�Scale 1�5�2� Ordinal�Measurement�Scale 1�5�3� Interval�Measurement�Scale 1�5�4� Ratio�Measurement�Scale
Key Concepts
� 1�� General�statistical�concepts Population Parameter Sample Statistic Descriptive�statistics Inferential�statistics
� 2�� Variable-related�concepts Variable Constant Categorical Dichotomous�variables Numerical Discrete�variables Continuous�variables
2 An Introduction to Statistical Concepts
� 3�� Measurement�scale�concepts Measurement Nominal Ordinal Interval Ratio
We�want�to�welcome�you�to�the�wonderful�world�of�statistics��More�than�ever,�statistics�are� everywhere��Listen�to�the�weather�report�and�you�hear�about�the�measurement�of�variables� such�as�temperature,�rainfall,�barometric�pressure,�and�humidity��Watch�a�sporting�event� and�you�hear�about�batting�averages,�percentage�of�free�throws�completed,�and�total�rush- ing�yardage��Read�the�financial�page�and�you�can�track�the�Dow�Jones�average,�the�gross� national�product,�and�bank�interest�rates��Turn�to�the�entertainment�section�to�see�movie� ratings,�movie�revenue,�or�the�top�10�best-selling�novels��These�are�just�a�few�examples�of� statistics�that�surround�you�in�every�aspect�of�your�life�
Although�you�may�be�thinking�that�statistics�is�not�the�most�enjoyable�subject�on�the�planet,� by�the�end�of�this�text,�you�will�(a)�have�a�more�positive�attitude�about�statistics,�(b)�feel�more� comfortable�using�statistics,�and�thus�be�more�likely�to�perform�your�own�quantitative�data� analyses,�and�(c)�certainly�know�much�more�about�statistics�than�you�do�now��In�other�words,� our�goal�is�to�equip�you�with�the�skills�you�need�to�be�both�a�better�consumer�and�producer�of� research��But�be�forewarned;�the�road�to�statistical�independence�is�not�easy��However,�we�will� serve�as�your�guides�along�the�way��When�the�going�gets�tough,�we�will�be�there�to�help�you� with�advice�and�numerous�examples�and�problems��Using�the�powers�of�logic,�mathematical� reasoning,�and�statistical�concept�knowledge,�we�will�help�you�arrive�at�an�appropriate�solu- tion�to�the�statistical�problem�at�hand�
Some�students�begin�their�first�statistics�class�with�some�anxiety��This�could�be�caused� by�not�having�had�a�quantitative�course�for�some�time,�apprehension�built�up�by�delaying� taking�statistics,�a�poor�past�instructor�or�course,�or�less�than�adequate�past�success��Let� us�offer�a�few�suggestions�along�these�lines��First,�this�is�not�a�math�class�or�text��If�you� want�one�of�those,�then�you�need�to�walk�over�to�the�math�department��This�is�a�course� and�text�on�the�application�of�statistics�to�education�and�the�behavioral�sciences��Second,� the�philosophy�of�the�text�is�on�the�understanding�of�concepts�rather�than�on�the�deriva- tion�of�statistical�formulas��It�is�more�important�to�understand�concepts�than�to�derive�or� memorize�various�and�sundry�formulas��If�you�understand�the�concepts,�you�can�always� look�up�the�formulas�if�need�be��If�you�do�not�understand�the�concepts,�then�knowing�the� formulas�will�only�allow�you�to�operate�in�a�cookbook�mode�without�really�understanding� what� you� are� doing�� Third,� the� calculator� and� computer� are� your� friends�� These� devices� are�tools�that�allow�you�to�complete�the�necessary�computations�and�obtain�the�results�of� interest��If�you�are�performing�hand�computations,�find�a�calculator�that�you�are�comfort- able�with;�it�need�not�have�800�functions,�as�the�four�basic�operations�and�sum�and�square� root� functions� are� sufficient� (one� of� our� personal� calculators� is� one� of� those� little� credit� card�calculators,�although�we�often�use�the�calculator�on�our�computers)��If�you�are�using� a� statistical� software� program,� find� one� that� you� are� comfortable� with� (most� instructors� will�have�you�using�a�program�such�as�SPSS,�SAS,�or�Statistica)��In�this�text,�we�use�SPSS� to�illustrate�statistical�applications��Finally,�this�text�will�take�you�from�raw�data�to�results� using�realistic�examples��These�can�then�be�followed�up�using�the�problems�at�the�end�of� each�chapter��Thus,�you�will�not�be�on�your�own�but�will�have�the�text,�a�computer/calculator,� as�well�as�your�course�and�instructor,�to�help�guide�you�
3Introduction
The�intent�and�philosophy�of�this�text�is�to�be�conceptual�and�intuitive�in�nature��Thus,�the� text�does�not�require�a�high�level�of�mathematics�but�rather�emphasizes�the�important�con- cepts�in�statistics��Most�statistical�concepts�really�are�fairly�easy�to�learn�because�(a)�concepts� can�be�simply�stated,�(b)�concepts�can�be�related�to�real-life�examples,�(c)�many�of�the�same� concepts�run�through�much�of�statistics,�and�therefore,�(d)�many�concepts�can�be�related�
In� this� introductory� chapter,� we� describe� the� most� basic� statistical� concepts�� We� begin� with� the� question,� “What� is� the� value� of� statistics?”� We� then� look� at� a� brief� history� of� statistics� by� mentioning� a� few� of� the� more� important� and� interesting� statisticians�� Then� we�consider�the�concepts�of�population,�parameter,�sample�and�statistic,�descriptive�and� inferential�statistics,�types�of�variables,�and�scales�of�measurement��Our�objectives�are�that� by�the�end�of�this�chapter,�you�will�(a)�have�a�better�sense�of�why�statistics�are�necessary,� (b)�see�that�statisticians�are�an�interesting�group�of�people,�and�(c)�have�an�understanding� of�several�basic�statistical�concepts�
1.1 What Is the Value of Statistics?
Let�us�start�off�with�a�reasonable�rhetorical�question:�why�do�we�need�statistics?�In�other� words,�what�is�the�value�of�statistics,�either�in�your�research�or�in�your�everyday�life?�As�a� way�of�thinking�about�these�questions,�consider�the�following�headlines,�which�have�prob- ably�appeared�in�your�local�newspaper�
Cigarette Smoking Causes Cancer—Tobacco Industry Denies Charges
A� study� conducted� at� Ivy-Covered� University� Medical� School,� recently� published� in� the� New England Journal of Medicine,� has� definitively� shown� that� cigarette� smoking� causes�cancer��In�interviews�with�100�randomly�selected�smokers�and�nonsmokers�over� 50 years�of�age,�30%�of�the�smokers�have�developed�some�form�of�cancer,�while�only� 10%� of� the� nonsmokers� have� cancer�� “The� higher� percentage� of� smokers� with� cancer� in� our� study� clearly� indicates� that� cigarettes� cause� cancer,”� said� Dr�� Jason� P�� Smythe�� On� the� contrary,� “this� study� doesn’t� even� suggest� that� cigarettes� cause� cancer,”� said� tobacco�lobbyist�Cecil�B��Hacker��“Who�knows�how�these�folks�got�cancer;�maybe�it�is� caused�by�the�aging�process�or�by�the�method�in�which�individuals�were�selected�for� the�interviews,”�Mr��Hacker�went�on�to�say�
North Carolina Congressional Districts Gerrymandered—African-Americans Slighted
A�study�conducted�at�the�National�Center�for�Legal�Research�indicates�that�congressio- nal�districts�in�the�state�of�North�Carolina�have�been�gerrymandered�to�minimize�the� impact�of�the�African-American�vote��“From�our�research,�it�is�clear�that�the�districts� are�apportioned�in�a�racially�biased�fashion��Otherwise,�how�could�there�be�no�single� district� in� the� entire� state� which� has� a� majority� of� African-American� citizens� when� over� 50%� of� the� state’s� population� is� African-American�� The� districting� system� abso- lutely�has�to�be�changed,”�said�Dr��I��M��Researcher��A�spokesman�for�The�American� Bar�Association�countered�with�the�statement�“according�to�a�decision�rendered�by�the� United�States�Supreme�Court�in�1999�(No��98-85),�intent�or�motive�must�be�shown�for� racial�bias�to�be�shown�in�the�creation�of�congressional�districts��The�decision�states�a�
4 An Introduction to Statistical Concepts
‘facially�neutral�law�…�warrants�strict�scrutiny�only�if�it�can�be�proved�that�the�law�was� motivated�by�a�racial�purpose�or�object�’�The�data�in�this�study�do�not�show�intent�or� motive��To�imply�that�these�data�indicate�racial�bias�is�preposterous�”
Global Warming—Myth According to the President
Research�conducted�at�the�National�Center�for�Global�Warming�(NCGW)�has�shown� the�negative�consequences�of�global�warming�on�the�planet�Earth��As�summarized�by� Dr��Noble�Pryze,�“our�studies�at�NCGW�clearly�demonstrate�that�if�global�warming�is� not�halted�in�the�next�20�years,�the�effects�on�all�aspects�of�our�environment�and�cli- matology�will�be�catastrophic�”�A�different�view�is�held�by�U�S��President�Harold�W�� Tree��He�stated�in�a�recent�address�that�“the�scientific�community�has�not�convinced� him�that�global�warming�even�exists��Why�should�our�administration�spend�millions� of�dollars�on�an�issue�that�has�not�been�shown�to�be�a�real�concern?”
How� is� one� to� make� sense� of� the� studies� described� by� these� headlines?� How� is� one� to� decide�which� side�of�the�issue�these�data�support,�so�as�to�take�an�intellectual� stand?�In� other�words,�do�the�interview�data�clearly�indicate�that�cigarette�smoking�causes�cancer?� Do� the� congressional� district� percentages� of� African-Americans� necessarily� imply� that� there�is�racial�bias?�Have�scientists�convinced�us�that�global�warming�is�a�problem?�These� studies�are�examples�of�situations�where�the�appropriate�use�of�statistics�is�clearly�neces- sary��Statistics�will�provide�us�with�an�intellectually�acceptable�method�for�making�deci- sions�in�such�matters��For�instance,�a�certain�type�of�research,�statistical�analysis,�and�set� of� results� are� all� necessary� to� make� causal� inferences� about� cigarette� smoking�� Another� type�of�research,�statistical�analysis,�and�set�of�results�are�all�necessary�to�lead�one�to�con- fidently�state�that�the�districting�system�is�racially�biased�or�not,�or�that�global�warming� needs�to�be�dealt�with��The�bottom�line�is�that�the�purpose�of�statistics,�and�thus�of�this� text,�is�to�provide�you�with�the�tools�to�make�important�decisions�in�an�appropriate�and� confident�manner��You�will�not�have�to�trust�a�statement�made�by�some�so-called�expert�on� an�issue,�which�may�or�may�not�have�any�empirical�basis�or�validity;�you�can�make�your� own�judgments�based�on�the�statistical�analyses�of�data��For�you,�the�value�of�statistics�can� include�(a)�the�ability�to�read�and�critique�articles�in�both�professional�journals�and�in�the� popular� press� and� (b)� the� ability� to� conduct� statistical� analyses� for� your� own� research� (e�g�,�thesis�or�dissertation)�
1.2 Brief Introduction to History of Statistics
As�a�way�of�getting�to�know�the�topic�of�statistics,�we�want�to�briefly�introduce�you�to�a� few� famous� statisticians�� The� purpose� of� this� section� is� not� to� provide� a� comprehensive� history�of�statistics,�as�those�already�exist�(e�g�,�Heyde,�Seneta,�Crepel,�Fienberg,�&�Gani,� 2001;�Pearson,�1978;�Stigler,�1986)��Rather,�the�purpose�of�this�section�is�to�show�that�famous� statisticians�not�only�are�interesting�but�are�human�beings�just�like�you�and�me�
One� of� the� fathers� of� probability� (see� Chapter� 5)� is� acknowledged� to� be� Blaise� Pascal� from� the� late� 1600s�� One� of� Pascal’s� contributions� was� that� he� worked� out� the� probabili- ties� for� each� dice� roll� in� the� game� of� craps,� enabling� his� friend,� a� member� of� royalty,� to� become�a�consistent�winner��He�also�developed�Pascal’s�triangle�which�you�may�remember�
5Introduction
from�your�early�mathematics�education��The�statistical�development�of�the�normal�or�bell- shaped�curve�(see�Chapter�4)�is�interesting��For�many�years,�this�development�was�attrib- uted�to�Karl�Friedrich�Gauss�(early�1800s)�and�was�actually�known�for�some�time�as�the� Gaussian� curve�� Later� historians� found� that� Abraham� DeMoivre� actually� developed� the� normal�curve�in�the�1730s��As�statistics�was�not�thought�of�as�a�true�academic�discipline� until� the� late� 1800s,� people� like� Pascal� and� DeMoivre� were� consulted� by� the� wealthy� on� odds�about�games�of�chance�and�by�insurance�underwriters�to�determine�mortality�rates�
Karl� Pearson� is� one� of� the� most� famous� statisticians� to� date� (late� 1800s� to� early� 1900s)�� Among�his�many�accomplishments�is�the�Pearson�product–moment�correlation�coefficient� still�in�use�today�(see�Chapter�10)��You�may�know�of�Florence�Nightingale�(1820–1910)�as�an� important�figure�in�the�field�of�nursing��However,�you�may�not�know�of�her�importance�in� the�field�of�statistics��Nightingale�believed�that�statistics�and�theology�were�linked�and�that� by�studying�statistics�we�might�come�to�understand�God’s�laws�
A�quite�interesting�statistical�personality�is�William�Sealy�Gossett,�who�was�employed� by� the� Guinness� Brewery� in� Ireland�� The� brewery� wanted� to� select� a� sample� of� people� from�Dublin�in�1906�for�purposes�of�taste�testing��Gossett�was�asked�how�large�a�sample� was�needed�in�order�to�make�an�accurate�inference�about�the�entire�population�(see�next� section)�� The� brewery� would� not� let� Gossett� publish� any� of� his� findings� under� his� own� name,� so� he� used� the� pseudonym� of� Student�� Today,� the� t� distribution� is� still� known� as� Student’s�t�distribution��Sir�Ronald�A��Fisher�is�another�of�the�most�famous�statisticians�of� all�time��Working�in�the�early�1900s,�Fisher�introduced�the�analysis�of�variance�(ANOVA)� (see�Chapters�11�through�16)�and�Fisher’s�z�transformation�for�correlations�(see�Chapter�10)�� In� fact,� the� major� statistic� in� the� ANOVA� is� referred� to� as� the� F� ratio� in� honor� of� Fisher�� These� individuals� represent� only� a� fraction� of� the� many� famous� and� interesting� statisti- cians�over�the�years��For�further�information�about�these�and�other�statisticians,�we�sug- gest�you�consult�references�such�as�Pearson�(1978),�Stigler�(1986),�and�Heyde�et�al��(2001),� which�consist�of�many�interesting�stories�about�statisticians�
1.3 General Statistical Definitions
In�this�section,�we�define�some�of�the�most�basic�concepts�in�statistics��Included�here�are� definitions�and�examples�of�the�following�concepts:�population,�parameter,�sample,�statis- tic,�descriptive�statistics,�and�inferential�statistics�
The�first�four�concepts�are�tied�together,�so�we�discuss�them�together��A�population�is� defined�as�consisting�of�all�members�of�a�well-defined�group��A�population�may�be�large� in�scope,�such�as�when�a�population�is�defined�as�all�of�the�employees�of�IBM�worldwide�� A� population� may� be� small� in� scope,� such� as� when� a� population� is� defined� as� all� of� the� IBM� employees� at� the� building� on� Main� Street� in� Atlanta�� Thus,� a� population� could� be� large�or�small�in�scope��The�key�is�that�the�population�is�well�defined�such�that�one�could� determine�specifically�who�all�of�the�members�of�the�group�are�and�then�information�or� data�could�be�collected�from�all�such�members��Thus,�if�our� population�is�defined�as�all� members�working�in�a�particular�office�building,�then�our�study�would�consist�of�collect- ing�data�from�all�employees�in�that�building��It�is�also�important�to�remember�that�you,�the� researcher,�define�the�population�
A� parameter� is� defined� as� a� characteristic� of� a� population�� For� instance,� parameters� of� our� office� building� example� might� be� the� number� of� individuals� who� work� in� that�
6 An Introduction to Statistical Concepts
building�(e�g�,�154),�the�average�salary�of�those�individuals�(e�g�,�$49,569),�and�the�range�of� ages�of�those�individuals�(e�g�,�21–68�years�of�age)��When�we�think�about�characteristics�of� a�population,�we�are�thinking�about�population parameters��Those�two�terms�are�often� linked�together�
A� sample� is� defined� as� consisting� of� a� subset� of� a� population�� A� sample� may� be� large� in�scope,�such�as�when�a�population�is�defined�as�all�of�the�employees�of�IBM�worldwide� and�20%�of�those�individuals�are�included�in�the�sample��A�sample�may�be�small�in�scope,� such�as�when�a�population�is�defined�as�all�of�the�IBM�employees�at�the�building�on�Main� Street�in�Atlanta�and�10%�of�those�individuals�are�included�in�the�sample��Thus,�a�sample� could�be�large�or�small�in�scope�and�consist�of�any�portion�of�the�population��The�key�is� that� the� sample� consists� of� some,� but� not� all,� of� the� members� of� the� population;� that� is,� anywhere�from�one�individual�to�all�but�one�individual�from�the�population�is�included�in� the�sample��Thus,�if�our�population�is�defined�as�all�members�working�in�the�IBM�building� on�Main�Street�in�Atlanta,�then�our�study�would�consist�of�collecting�data�from�a�sample� of�some�of�the�employees�in�that�building��It�follows�that�if�we,�the�researcher,�define�the� population,�then�we�also�determine�what�the�sample�will�be�
A�statistic�is�defined�as�a�characteristic�of�a�sample��For�instance,�statistics�of�our�office� building�example�might�be�the�number�of�individuals�who�work�in�the�building�that�we� sampled�(e�g�,�77),�the�average�salary�of�those�individuals�(e�g�,�$54,090),�and�the�range�of� ages� of� those� individuals� (e�g�,� 25–62� years� of� age)�� Notice� that� the� statistics� of� a� sample� need�not�be�equal�to�the�parameters�of�a�population�(more�about�this�in�Chapter�5)��When� we�think�about�characteristics�of�a�sample,�we�are�thinking�about�sample statistics��Those� two� terms� are� often� linked� together�� Thus,� we� have� population� parameters� and� sample� statistics,� but� no� other� combinations� of� those� terms� exist�� The� field� has� become� known� as�statistics�simply�because�we�are�almost�always�dealing�with�sample�statistics�because� population�data�are�rarely�obtained�
The�final�two�concepts�are�also�tied�together�and�thus�considered�together��The�field�of� statistics�is�generally�divided�into�two�types�of�statistics,�descriptive�statistics�and�inferen- tial�statistics��Descriptive statistics�are�defined�as�techniques�which�allow�us�to�tabulate,� summarize,�and�depict�a�collection�of�data�in�an�abbreviated�fashion��In�other�words,�the� purpose�of�descriptive�statistics�is�to�allow�us�to�talk�about�(or�describe)�a�collection�of�data� without�having�to�look�at�the�entire�collection��For�example,�say�we�have�just�collected�a� set�of�data�from�100,000�graduate�students�on�various�characteristics�(e�g�,�height,�weight,� gender,�grade�point�average,�aptitude�test�scores)��If�you�were�to�ask�us�about�the�data,�we� could�do�one�of�two�things��On�the�one�hand,�we�could�carry�around�the�entire�collection� of�data�everywhere�we�go,�and�when�someone�asks�us�about�the�data,�simply�say�“Here�is� the�data;�take�a�look�at�them�yourself�”�On�the�other�hand,�we�could�summarize�the�data� in�an�abbreviated�fashion,�and�when�someone�asks�us�about�the�data,�simply�say�“Here�is� a�table�and�a�graph�about�the�data;�they�summarize�the�entire�collection�”�So,�rather�than� viewing�100,000�sheets�of�paper,�perhaps�we�would�only�have�to�view�two�sheets�of�paper�� Since� statistics� is� largely� a� system� of� communicating� information,� descriptive� statistics� are�considerably�more�useful�to�a�consumer�than�an�entire�collection�of�data��Descriptive� statistics�are�discussed�in�Chapters�2�through�4�
Inferential statistics�are�defined�as�techniques�which�allow�us�to�employ�inductive�rea- soning�to�infer�the�properties�of�an�entire�group�or�collection�of�individuals,�a�population,� from�a�small�number�of�those�individuals,�a�sample��In�other�words,�the�purpose�of�infer- ential�statistics�is�to�allow�us�to�collect�data�from�a�sample�of�individuals�and�then�infer�the� properties�of�that�sample�back�to�the�population�of�individuals��In�case�you�have�forgotten� about�logic,�inductive�reasoning�is�where�you�infer�from�the�specific�(here�the�sample)�to�
7Introduction
the�general�(here�the�population)��For�example,�say�we�have�just�collected�a�set�of�sample� data�from�5,000�of�the�population�of�100,000�graduate�students�on�various�characteristics� (e�g�,�height,�weight,�gender,�grade�point�average,�aptitude�test�scores)��If�you�were�to�ask� us�about�the�data,�we�could�compute�various�sample�statistics�and�then�infer�with�some� confidence�that�these�would�be�similar�to�the�population�parameters��In�other�words,�this� allows� us� to� collect� data� from� a� subset� of� the� population� yet� still� make� inferential� state- ments�about�the�population�without�collecting�data�from�the�entire�population��So,�rather� than�collecting�data�from�all�100,000�graduate�students�in�the�population,�we�could�collect� data�on�a�sample�of�say�5,000�students�
As�another�example,�Gossett�(aka�Student)�was�asked�to�conduct�a�taste�test�of�Guinness� beer� for� a� sample� of� Dublin� residents�� Because� the� brewery� could� not� afford� to� do� this� with�the�entire�population�of�Dublin,�Gossett�collected�data�from�a�sample�of�Dublin�resi- dents�and�was�able�to�make�an�inference�from�these�sample�results�back�to�the�population�� A discussion�of�inferential�statistics�begins�in�Chapter�5��In�summary,�the�field�of�statistics� is�roughly�divided�into�descriptive�statistics�and�inferential�statistics��Note,�however,�that� many�further�distinctions�are�made�among�the�types�of�statistics,�but�more�about�that�later�
1.4 Types of Variables
There�are�several�terms�we�need�to�define�about�variables��First,�it�might�be�useful�to�define� the�term�variable��A�variable�is�defined�as�any�characteristic�of�persons�or�things�that�is� observed�to�take�on�different�values��In�other�words,�the�values�for�a�particular�character- istic�vary�across�the�individuals�observed��For�example,�the�annual�salary�of�the�families� in�your�neighborhood�varies�because�not�every�family�earns�the�same�annual�salary��One� family�might�earn�$50,000�while�the�family�right�next�door�might�earn�$65,000��Thus,�the� annual�family�salary�is�a�variable�because�it�varies�across�families�
In� contrast,� a� constant� is� defined� as� any� characteristic� of� persons� or� things� that� is� observed�to�take�on�only�a�single�value��In�other�words,�the�values�for�a�particular�char- acteristic� are� the� same� for� all� individuals� observed�� For� example,� say� every� family� in� your� neighborhood� has� a� lawn�� Although� the� nature� of� the� lawns� may� vary,� everyone� has�a�lawn��Thus,�whether�a�family�has�a�lawn�in�your�neighborhood�is�a�constant�and� therefore�would�not�be�a�very�interesting�characteristic�to�study��When�designing�a�study,� you�(i�e�,�the�researcher)�can�determine�what�is�a�constant��This�is�part�of�the�process�of� delimiting,�or�narrowing�the�scope�of,�your�study��As�an�example,�you�may�be�interested� in� studying� career� paths� of� girls� who� complete� AP� science� courses�� In� designing� your� study,�you�are�only�interested�in�girls,�and�thus,�gender�would�be�a�constant��This�is�not� to� say� that� the� researcher� wholly� determines� when� a� characteristic� is� a� constant�� It� is� sometimes�the�case�that�we�find�that�a�characteristic�is�a�constant�after�we�conduct�the� study�� In� other� words,� one� of� the� measures� has� no� variation—everyone� or� everything� scored�or�remained�the�same�on�that�particular�characteristic�
There�are�different�typologies�for�describing�variables��One�typology�is�categorical�(or� qualitative)� versus� numerical� (or� quantitative),� and� within� numerical,� discrete,� and� con- tinuous��A�categorical�variable�is�a�qualitative�variable�that�describes�categories�of�a�char- acteristic�or�attribute��Examples�of�categorical�variables�include�political�party�affiliation� (Republican�=�1,�Democrat�=�2,�Independent�=�3),�religious�affiliation�(e�g�,�Methodist�=�1,� Baptist�=�2,�Roman�Catholic�=�3),�and�course�letter�grade�(A�=�4,�B�=�3,�C�=�2,�D�=�1,�F�=�0)��
8 An Introduction to Statistical Concepts
A�dichotomous variable�is�a�special,�restricted�type�of�categorical�variable�and�is�defined� as� a� variable� that� can� take� on� only� one� of� two� values�� For� example,� biologically� deter- mined�gender�is�a�variable�that�can�only�take�on�the�values�of�male�or�female�and�is�often� coded�numerically�as�0�(e�g�,�for�males)�or�1�(e�g�,�for�females)��Other�dichotomous�variables� include�pass/fail,�true/false,�living/dead,�and�smoker/nonsmoker��Dichotomous�variables� will�take�on�special�importance�as�we�study�binary�logistic�regression�(Chapter�19)�
A�numerical�variable�is�a�quantitative�variable��Numerical�variables�can�further�be�clas- sified�as�either�discrete�or�continuous��A�discrete variable�is�defined�as�a�variable�that�can� only�take�on�certain�values��For�example,�the�number�of�children�in�a�family�can�only�take�on� certain�values��Many�values�are�not�possible,�such�as�negative�values�(e�g�,�the�Joneses�cannot� have�−2�children)�or�decimal�values�(e�g�,�the�Smiths�cannot�have�2�2�children)��In�contrast,� a�continuous variable�is�defined�as�a�variable�that�can�take�on�any�value�within�a�certain� range�given�a�precise�enough�measurement�instrument��For�example,�the�distance�between� two� cities� can� be� measured� in� miles,� with� miles� estimated� in� whole� numbers�� However,� given� a� more� precise� instrument� with� which� to� measure,� distance� can� even� be� measured� down� to� the� inch� or� millimeter�� When� considering� the� difference� between� a� discrete� and� continuous� variable,� keep� in� mind� that� discrete variables arise from the counting process� and� continuous variables arise from the measuring process�� For� example,� the� number� of� students� enrolled�in�your�statistics�class�is�a�discrete�variable��If�we�were�to�measure�(i�e�,�count)�the� number�of�students�in�the�class,�it�would�not�matter�if�we�counted�first�names�alphabetically� from�A�to�Z�or�if�we�counted�beginning�with�who�sat�in�the�front�row�to�the�last�person�in� the�back�row—either�way,�we�would�arrive�at�the�same�value��In�other�words,�how�we�“mea- sure”�(again,�count)�the�students�in�the�class�does�not�matter—we�will�always�arrive�at�the� same�result��In�comparison,�the�value�of�a�continuous�variable�is�dependent�on�how�precise� the�measuring�instrument�is��Weighing�yourself�on�a�scale�that�rounds�to�whole�numbers� will�give�us�one�measure�of�weight��However,�weighing�on�another,�more�precise,�scale�that� rounds�to�three�decimal�places�will�provide�a�more�precise�measure�of�weight�
Here�are�a�few�additional�examples�of�the�discrete�and�continuous�variables��Other�dis- crete� variables� include� a� number� of� CDs� owned,� number� of� credit� hours� enrolled,� and� number�of�teachers�employed�at�a�school��Other�continuous�variables�include�salary�(from� zero�to�billions�in�dollars�and�cents),�age�(from�zero�up,�in�millisecond�increments),�height� (from� zero� up,�in�increments�of�fractions�of�millimeters),�weight� (from� zero�up,� in�incre- ments�of�fractions�of�ounces),�and�time�(from�zero�up,�in�millisecond�increments)��Variable� type�is�often�important�in�terms�of�selecting�an�appropriate�statistic,�as�shown�later�
1.5 Scales of Measurement
Another�concept�useful�for�selecting�an�appropriate�statistic�is�the�scale�of�measurement� of�the�variables��First,�however,�we�define�measurement�as�the�assignment�of�numerical� values�to�persons�or�things�according�to�explicit�rules��For�example,�how�do�we�measure�a� person’s�weight?�Well,�there�are�rules�that�individuals�commonly�follow��Currently,�weight� is�measured�on�some�sort�of�balance�or�scale�in�pounds�or�grams��In�the�old�days,�weight� was�measured�by�different�rules,�such�as�the�number�of�stones�or�gold�coins��These�explicit� rules�were�developed�so�that�there�was�a�standardized�and�generally�agreed�upon�method� of� measuring� weight�� Thus,� if� you� weighted� 10� stones� in� Coventry,� England,� then� that� meant�the�same�as�10�stones�in�Liverpool,�England�
9Introduction
In�1951,�the�psychologist�S�S��Stevens�developed�four�types�of�measurement�scales�that� could�be�used�for�assigning�these�numerical�values��In�other�words,�the�type�of�rule�used� was�related�to�the�measurement�scale��The�four�types�of�measurement�scales�are�the�nomi- nal,�ordinal,�interval,�and�ratio�scales��They�are�presented�in�order�of�increasing�complex- ity�and�of�increasing�information�(remembering�the�acronym�NOIR�might�be�helpful)�� It�is�worth�restating�the�importance�of�understanding�the�measurement�scales�of�variables� as�the�measurement�scales�will�dictate�what�statistical�procedures�can�be�performed�with� the�data�
1.5.1 Nominal Measurement Scale
The� simplest� scale� of� measurement� is� the� nominal scale�� Here� individuals� or� objects� are�classified�into�categories�so�that�all�of�those�in�a�single�category�are�equivalent�with� respect� to� the� characteristic� being� measured�� For� example,� the� country� of� birth� of� an� individual� is� a� nominally� scaled� variable�� Everyone� born� in� France� is� equivalent� with� respect�to�this�variable,�whereas�two�people�born�in�different�countries�(e�g�,�France�and� Australia)�are�not�equivalent�with�respect�to�this�variable��The�categories�are�truly�quali- tative�in�nature,�not�quantitative��Categories�are�typically�given�names�or�numbers��For� our� example,� the� country� name� would� be� an� obvious� choice� for� categories,� although� numbers�could�also�be�assigned�to�each�country�(e�g�,�Brazil�=�5,�India�=�34)��The�numbers� do�not�represent�the�amount�of�the�attribute�possessed��An�individual�born�in�India�does� not�possess�any�more�of�the�“country�of�birth�origin”�attribute�than�an�individual�born� in�Brazil�(which�would�not�make�sense�anyway)��The�numbers�merely�identify�to�which� category� an� individual� or� object� belongs�� The� categories� are� also� mutually� exclusive�� That�is,�an�individual�can�belong�to�one�and�only�one�category,�such�as�a�person�being� born�in�only�one�country�
The� statistics� of� a� nominal� scale� variable� are� quite� simple� as� they� can� only� be� based� on� the� frequencies� that� occur� within� each� of� the� categories�� For� example,� we� may� be� studying�characteristics�of�various�countries�in�the�world��A�nominally�scaled�variable� could� be� the� hemisphere� in� which� the� country� is� located� (northern,� southern,� eastern,� and�western)��While�it�is�possible�to�count�the�number�of�countries�that�belong�to�each� hemisphere,�that�is�all�that�we�can�do��The�only�mathematical�property�that�the�nominal� scale�possesses�is�that�of�equality�versus�inequality��In�other�words,�two�individuals�or� objects�are�either�in�the�same�category�(equal)�or�in�different�categories�(unequal)��For�the� hemisphere�variable,�we�can�either�use�the�country�name�or�assign�numerical�values� to�each�country��We�might�perhaps�assign�each�hemisphere�a�number�alphabetically�from� 1�to�4��Countries�that�are�in�the�same�hemisphere�are�equal�with�respect�to�this�character- istic��Countries�that�are�in�different�hemispheres�are�unequal�with�respect�to�this�charac- teristic��Again,�these�particular�numerical�values�are�meaningless�and�could�arbitrarily� be�any�values��The�numerical�values�assigned�only�serve�to�keep�the�categories�distinct� from�one�another��Many�other�numerical�values�could�be�assigned�for�the�hemispheres� and� still� maintain� the� equality� versus� inequality� property�� For� example,� the� northern� hemisphere�could�easily�be�categorized�as�1000�and�the�southern�hemisphere�as�2000�with� no�change�in�information��Other�examples�of�nominal�scale�variables�include�hair�color,� eye�color,�neighborhood,�gender,�ethnic�background,�religious�affiliation,�political�party� affiliation,�type�of�life�insurance�owned�(e�g�,�term,�whole�life),�blood�type,�psychological� clinical�diagnosis,�Social�Security�number,�and�type�of�headache�medication�prescribed�� The� term� nominal� is� derived� from� “giving� a� name�”� Nominal� variables� are� considered� categorical�or�qualitative�
10 An Introduction to Statistical Concepts
1.5.2 Ordinal Measurement Scale
The�next�most�complex�scale�of�measurement�is�the�ordinal scale��Ordinal�measurement� is�determined�by�the�relative�size�or�position�of�individuals�or�objects�with�respect�to�the� characteristic�being�measured��That�is,�the�individuals�or�objects�are�rank-ordered�accord- ing�to�the�amount�of�the�characteristic�that�they�possess��For�example,�say�a�high�school� graduating� class� had� 250� students�� Students� could� then� be� assigned� class� ranks� accord- ing�to�their�academic�performance�(e�g�,�grade�point�average)�in�high�school��The�student� ranked�1�in�the�class�had�the�highest�relative�performance,�and�the�student�ranked�250�had� the�lowest�relative�performance�
However,�equal�differences�between�the�ranks�do�not�imply�equal�distance�in�terms�of� the�characteristic�being�measured��For�example,�the�students�ranked�1�and�2�in�the�class� may�have�a�different�distance�in�terms�of�actual�academic�performance�than�the�students� ranked� 249� and� 250,� even� though� both� pairs� of� students� differ� by� a� rank� of� 1�� In� other� words,�here�a�rank�difference�of�1�does�not�imply�the�same�actual�performance�distance�� The�pairs�of�students�may�be�very,�very�close�or�be�quite�distant�from�one�another��As� a�result�of�equal�differences�not�implying�equal�distances,�the�statistics�that�we�can�use� are�limited�due�to�these�unequal�intervals��The�ordinal�scale�then�consists�of�two�math- ematical�properties:�equality�versus�inequality�again;�and�if�two�individuals�or�objects� are�unequal,�then�we�can�determine�greater�than�or�less�than��That�is,�if�two�individuals� have�different�class�ranks,�then�we�can�determine�which�student�had�a�greater�or�lesser� class�rank��Although�the�greater�than�or�less�than�property�is�evident,�an�ordinal�scale� cannot�tell�us�how�much�greater�than�or�less�than�because�of�the�unequal�intervals��Thus,� the�student�ranked�250�could�be�farther�away�from�student�249�than�the�student�ranked�2� from�student�1�
When� we� have� untied� ranks,� as� shown� on� the� left� side� of� Table� 1�1,� assigning� ranks� is� straightforward�� What� do� we� do� if� there� are� tied� ranks?� For� example,� suppose� there� are� two�students�with�the�same�grade�point�average�of�3�8�as�given�on�the�right�side�of�Table�1�1�� How�do�we�assign�them�into�class�ranks?�It�is�clear�that�they�have�to�be�assigned�the�same� rank,�as�that�would�be�the�only�fair�method��However,�there�are�at�least�two�methods�for� dealing�with�tied�ranks��One�method�would�be�to�assign�each�of�them�a�rank�of�2�as�that�is� the�next�available�rank��However,�there�are�two�problems�with�that�method��First,�the�sum� of�the�ranks�for�the�same�number�of�scores�would�be�different�depending�on�whether�there�
Table 1.1
Untied�Ranks�and�Tied�Ranks�for�Ordinal�Data
Untied Ranks Tied Ranks
Grade Point Average Rank
Grade Point Average Rank
4�0 1 4�0 1 3�9 2 3�8 2�5 3�8 3 3�8 2�5 3�6 4 3�6 4 3�2 5 3�0 6 3�0 6 3�0 6 2�7 7 3�0 6
Sum�=�28 Sum�=�28
11Introduction
were�ties�or�not��Statistically,�this�is�not�a�satisfactory�solution��Second,�what�rank�would� the�next�student�having�the�3�6�grade�point�average�be�given,�a�rank�of�3�or�4?
The� second� and� preferred� method� is� to� take� the� average� of� the� available� ranks� and� assign�that�value�to�each�of�the�tied�individuals��Thus,�the�two�persons�tied�at�a�grade� point�average�of�3�8�have�as�available�ranks�2�and�3��Both�would�then�be�assigned�the� average�rank�of�2�5��Also,�the�three�persons�tied�at�a�grade�point�average�of�3�0�have�as� available�ranks�5,�6,�and�7��These�all�would�be�assigned�the�average�rank�of�6��You�also� see�in�the�table�that�with�this�method�the�sum�of�the�ranks�for�7�scores�is�always�equal� to�28,�regardless�of�the�number�of�ties��Statistically,�this�is�a�satisfactory�solution�and�the� one� we� prefer,� whether� we� are� using� a� statistical� software� package� or� hand� computa- tions�� Other� examples� of� ordinal� scale� variables� include� course� letter� grades,� order� of� finish� in� the� Boston� Marathon,� socioeconomic� status,� hardness� of� minerals� (1� =� soft- est�to�10�=�hardest),�faculty�rank�(assistant,�associate,�and�full�professor),�student�class� (freshman,�sophomore,�junior,�senior,�graduate�student),�ranking�on�a�personality�trait� (e�g�,� extreme� intrinsic� to� extreme� extrinsic� motivation),� and� military� rank�� The� term� ordinal� is� derived� from� “ordering”� individuals� or� objects�� Ordinal� variables� are� most� often�considered�categorical�or�qualitative�
1.5.3 Interval Measurement Scale
The�next�most�complex�scale�of�measurement�is�the�interval scale��An�interval�scale�is�one� where� individuals� or� objects� can� be� ordered,� and� equal� differences� between� the� values� do�imply�equal�distance�in�terms�of�the�characteristic�being�measured��That�is,�order�and� distance�relationships�are�meaningful��However,�there�is�no�absolute�zero�point��Absolute� zero,�if�it�exists,�implies�the�total�absence�of�the�property�being�measured��The�zero�point�of� an�interval�scale,�if�it�exists,�is�arbitrary�and�does�not�reflect�the�total�absence�of�the�prop- erty�being�measured��Here�the�zero�point�merely�serves�as�a�placeholder��For�example,�sup- pose�that�we�gave�you�the�final�exam�in�advanced�statistics�right�now��If�you�were�to�be�so� unlucky�as�to�obtain�a�score�of�0,�this�score�does�not�imply�a�total�lack�of�knowledge�of�sta- tistics��It�would�merely�reflect�the�fact�that�your�statistics�knowledge�is�not�that�advanced� yet�(or�perhaps�the�questions�posed�on�the�exam�just�did�not�capture�those�concepts�that� you�do�understand)��You�do�have�some�knowledge�of�statistics�but�just�at�an�introductory� level�in�terms�of�the�topics�covered�so�far�
Take�as�an�example�the�Fahrenheit�temperature�scale,�which�has�a�freezing�point�of� 32�degrees��A�temperature�of�zero�is�not�the�total�absence�of�heat,�just�a�point�slightly� colder� than� 1� degree� and� slightly� warmer� than� −1� degree�� In� terms� of� the� equal� dis- tance�notion,�consider�the�following�example��Say�that�we�have�two�pairs�of�Fahrenheit� temperatures,�the�first�pair�being�55�and�60�degrees�and�the�second�pair�being�25�and� 30�degrees��The�difference�of�5�degrees�is�the�same�for�both�pairs�and�is�also�the�same� everywhere�along�the�Fahrenheit�scale��Thus,�every�5�degree�interval�is�an�equal�interval�� However,�we�cannot�say�that�60�degrees�is�twice�as�warm�as�30�degrees,�as�there�is�no� absolute�zero��In�other�words,�we�cannot�form�true�ratios�of�values�(i�e�,�60/30�=�2)��This� property�only�exists�for�the�ratio�scale�of�measurement��The�interval�scale�has�as�math- ematical� properties� equality� versus� inequality,� greater� than� or� less� than� if� unequal,� and�equal�intervals��Other�examples�of�interval�scale�variables�include�the�Centigrade� temperature� scale,� calendar� time,� restaurant� ratings� by� the� health� department� (on� a� 100-point�scale),�year�(since�1�AD),�and�arguably,�many�educational�and�psychological� assessment�devices�(although�statisticians�have�been�debating�this�one�for�many�years;�
12 An Introduction to Statistical Concepts
e�g�,�on�occasion�there�is�a�fine�line�between�whether�an�assessment�is�measured�along� the�ordinal�or�the�interval�scale)��Interval�variables�are�considered�numerical�and�pri- marily�continuous�
1.5.4 Ratio Measurement Scale
The�most�complex�scale�of�measurement�is�the�ratio scale��A�ratio�scale�has�all�of�the�proper- ties�of�the�interval�scale,�plus�an�absolute�zero�point�exists��Here�a�measurement�of�0�indi- cates�a�total�absence�of�the�property�being�measured��Due�to�an�absolute�zero�point�existing,� true�ratios�of�values�can�be�formed�which�actually�reflect�ratios�in�the�amounts�of�the�charac- teristic�being�measured��Thus,�if�concepts�such�as�“one-half�as�big”�or�“twice�as�large”�make� sense,�then�that�may�be�a�good�indication�that�the�variable�is�ratio�in�scale�
For�example,�the�height�of�individuals�is�a�ratio�scale�variable��There�is�an�absolute�zero� point� of� zero� height�� We� can� also� form� ratios� such� that� 6′0″� Sam� is� twice� as� tall� as� his� 3′0″� daughter� Samantha�� The� ratio� scale� of� measurement� is� not� observed� frequently� in� education�and�the�behavioral�sciences,�with�certain�exceptions��Motor�performance�vari- ables�(e�g�,�speed�in�the�100�meter�dash,�distance�driven�in�24�hours),�elapsed�time,�calorie� consumption,�and�physiological�characteristics�(e�g�,�weight,�height,�age,�pulse�rate,�blood� pressure)� are� ratio� scale� measures� (and� are� all� also� examples� of� continuous� variables)�� Discrete�variables,�those�that�arise�from�the�counting�process,�are�also�examples�of�ratio� variables�since�zero�indicates�an�absence�of�what�is�measured�(e�g�,�the�number�of�children� in�a�family�or�the�number�of�trees�in�a�park)��A�summary�of�the�measurement�scales,�their� characteristics,� and� some� examples� is� given� in� Table� 1�2�� Ratio� variables� are� considered� numerical�and�can�be�either�discrete�or�continuous�
Table 1.2
Summary�of�the�Scales�of�Measurement
Scale Characteristics Examples
Nominal Classify�into�categories;�categories�are�given� names�or�numbers,�but�the�numbers�are� arbitrary;�mathematical�property:
1��Equal�versus�unequal
Hair�or�eye�color,�ethnic�background,� neighborhood,�gender,�country�of�birth,�social� security�number,�type�of�life�insurance,�religious� or�political�affiliation,�blood�type,�clinical� diagnosis
Ordinal Rank-ordered�according�to�relative�size� or position;�mathematical�properties:
1��Equal�versus�unequal 2��If�unequal,�then�greater�than�or�less�than
Letter�grades,�order�of�finish�in�race,�class�rank,� SES,�hardness�of�minerals,�faculty�rank,�student� class,�military�rank,�rank�on�personality�trait
Interval Rank-ordered�and�equal�differences�between� values�imply�equal�distances�in�the�attribute;� mathematical�properties:
1��Equal�versus�unequal 2��If�unequal,�then�greater�than�or�less�than 3��Equal�intervals
Temperature,�calendar�time,�most�assessment� devices,�year,�restaurant�ratings
Ratio Rank-ordered,�equal�intervals,�absolute�zero� allows�ratios�to�be�formed;�mathematical� properties:
1��Equal�versus�unequal 2��If�unequal,�then�greater�than�or�less�than 3��Equal�intervals 4��Absolute�zero
Speed�in�100�meter�dash,�height,�weight,�age,� distance�driven,�elapsed�time,�pulse�rate,�blood� pressure,�calorie�consumption
13Introduction
1.6 Summary
In� this� chapter,� an� introduction� to� statistics� was� given�� First,� we� discussed� the� value� and� need�for�knowledge�about�statistics�and�how�it�assists�in�decision�making��Next,�a�few�of� the�more�colorful�and�interesting�statisticians�of�the�past�were�mentioned��Then,�we�defined� the�following�general�statistical�terms:�population,�parameter,�sample,�statistic,�descriptive� statistics,�and�inferential�statistics��We�then�defined�variable-related�terms�including�vari- ables,� constants,� categorical� variables,� and� continuous� variables�� For� a� summary� of� these� definitions,�see�Box�1�1��Finally,�we�examined�the�four�classic�types�of�measurement�scales,� nominal,� ordinal,� interval,� and� ratio�� By� now,� you� should� have� met� the� following� objec- tives:�(a) have�a�better�sense�of�why�statistics�are�necessary;�(b)�see�that�statisticians�are�an� interesting�group�of�people;�and�(c)�have�an�understanding�of�the�basic�statistical�concepts� of�population,�parameter,�sample,�and�statistic,�descriptive�and�inferential�statistics,�types� of� variables,� and� scales� of� measurement�� The� next� chapter� begins� to� address� some� of� the� details�of�descriptive�statistics�when�we�consider�how�to�represent�data�in�terms�of�tables� and�graphs��In�other�words,�rather�than�carrying�our�data�around�with�us�everywhere�we�go,� we�examine�ways�to�display�data�in�tabular�and�graphical�forms�to�foster�communication�
STOp aNd ThINk bOx 1.1
Summary�of�Definitions
Term Definition Example(s)
Population All�members�of�a�well-defined�group All�employees�of�IBM�Atlanta
Parameter A�characteristic�of�a�population Average�salary�of�a�population
Sample A�subset�of�a�population Some�employees�of�IBM�Atlanta
Statistic A�characteristic�of�a�sample Average�salary�of�a�sample
Descriptive�statistics Techniques�which�allow�us�to�tabulate,� summarize,�and�depict�a�collection�of�data� in an�abbreviated�fashion
Table�or�graph�summarizing�data
Inferential�statistics Techniques�which�allow�us�to�employ�inductive� reasoning�to�infer�the�properties�of�a� population�from�a�sample
Taste�test�statistics�from�sample� of Dublin�residents
Variable Any�characteristic�of�persons�or�things�that� is observed�to�take�on�different�values
Salary�of�the�families�in�your� neighborhood
Constant Any�characteristic�of�persons�or�things�that� is observed�to�take�on�only�a�single�value
Every�family�has�a�lawn�in�your� neighborhood
Categorical�variable A�qualitative�variable Political�party�affiliation
Dichotomous�variable A�categorical�variable�that�can�take�on�only� one of�two�values
Biologically�determined�gender
Numerical�variable A�quantitative�variable�that�is�either�discrete� or continuous
Number�of�children�in�a�family;� the�distance�between�two�cities
Discrete�variable A�numerical�variable�that�arises�from�the� counting�process�that�can�take�on�only�certain� values
Number�of�children�in�a�family
Continuous�variable A�numerical�variable�that�can�take�on�any�value� within�a�certain�range�given�a�precise�enough� measurement�instrument
Distance�between�two�cities
14 An Introduction to Statistical Concepts
Problems
Conceptual problems
1.1� �A�mental�health�counselor�is�conducting�a�research�study�on�satisfaction�that�married� couples� have� with� their� marriage�� “Marital� status”� (e�g�,� single,� married,� divorced,� widowed),�in�this�scenario,�is�which�one�of�the�following?
� a�� Constant � b�� Variable 1.2� �Belle� randomly� samples� 100� library� patrons� and� gathers� data� on� the� genre� of� the�
“first�book”�that�they�checked�out�from�the�library��She�finds�that�85�library�patrons� checked� out� a� fiction� book� and� 15� library� patrons� checked� out� a� nonfiction� book�� Which�of�the�following�best�characterizes�the�type�of�“first�book”�checked�out�in�this� study?
� a�� Constant � b�� Variable 1.3� For�interval�level�variables,�which�of�the�following�properties�does�not�apply? � a�� Jim�is�two�units�greater�than�Sally� � b�� Jim�is�greater�than�Sally� � c�� Jim�is�twice�as�good�as�Sally� � d�� Jim�differs�from�Sally� 1.4� �Which�of�the�following�properties�is�appropriate�for�ordinal�but�not�for�nominal�variables? � a�� Sue�differs�from�John� � b�� Sue�is�greater�than�John� � c�� Sue�is�10�units�greater�than�John� � d�� Sue�is�twice�as�good�as�John� 1.5� �Which� scale� of� measurement� is� implied� by� the� following� statement:� “Jill’s� score� is�
three�times�greater�than�Eric’s�score”? � a�� Nominal � b�� Ordinal � c�� Interval � d�� Ratio 1.6� �Which�scale�of�measurement�is�implied�by�the�following�statement:�“Bubba�had�the�
highest�score”? � a�� Nominal � b�� Ordinal � c�� Interval � d�� Ratio 1.7� �A�band�director�collects�data�on�the�number�of�years�in�which�students�in�the�band�
have� played� a� musical� instrument�� Which� scale� of� measurement� is� implied� by� this� scenario?
15Introduction
� a�� Nominal � b�� Ordinal � c�� Interval � d�� Ratio 1.8� �Kristen�has�an�IQ�of�120��I�assert�that�Kristen�is�20%�more�intelligent�than�the�average�
person�having�an�IQ�of�100��Am�I�correct? 1.9� Population�is�to�parameter�as�sample�is�to�statistic��True�or�false? 1.10� Every�characteristic�of�a�sample�of�100�persons�constitutes�a�variable��True�or�false? 1.11� A�dichotomous�variable�is�also�a�categorical�variable��True�or�false? 1.12� �The� amount� of� time� spent� studying� in� 1� week� for� a� population� of� students� is� an�
inferential�statistic��True�or�false? 1.13� For�ordinal�level�variables,�which�of�the�following�properties�does�not�apply? � a�� IBM�differs�from�Apple� � b�� IBM�is�greater�than�Apple� � c�� IBM�is�two�units�greater�than�Apple� � d�� All�of�the�aforementioned�properties�apply� 1.14� �A�sample�of�50�students�take�an�exam,�and�the�instructor�decides�to�give�the�top�5�
scores�a�bonus�of�5�points��Compared�to�the�original�set�of�scores�(no�bonus),�I�assert� that�the�ranks�of�the�new�set�of�scores�(including�bonus)�will�be�exactly�the�same��Am� I�correct?
1.15� �Johnny�and�Buffy�have�class�ranks�of�5�and�6��Ingrid�and�Toomas�have�class�ranks�of� 55�and�56��I�assert�that�the�GPAs�of�Johnny�and�Buffy�are�the�same�distance�apart�as� are�the�GPAs�of�Ingrid�and�Toomas��Am�I�correct?
Computational problems
1.1� �Rank� the� following� values� of� the� number� of� CDs� owned,� assigning� rank� 1� to� the� largest�value: 10 15 12 8 20 17 5 21 3 19
1.2� �Rank�the�following�values�of�the�number�of�credits�earned,�assigning�rank�1�to�the� largest�value: 10 16 10 8 19 16 5 21 3 19
1.3� �Rank�the�following�values�of�the�number�of�pairs�of�shoes�owned,�assigning�rank�1� to�the�largest�value:
8 6 3 12 19 7 10 25 4 42
Interpretive problems
Consider�the�following�class�survey:
1.1� What�is�your�gender? 1.2� What�is�your�height�in�inches? 1.3� What�is�your�shoe�size�(length)?
16 An Introduction to Statistical Concepts
1.4� Do�you�smoke? 1.5� Are�you�left-�or�right-handed?�Your�mother?�Your�father? 1.6� How�much�did�you�spend�at�your�last�hair�appointment�(including�tip)? 1.7� How�many�CDs�do�you�own? 1.8� What�was�your�quantitative�GRE�score? 1.9� What�is�your�current�GPA? 1.10� On�average,�how�much�exercise�do�you�get�per�week�(in�hours)? 1.11� �On�a�5-point�scale,�what�is�your�political�view�(1�=�very�liberal,�3�=�moderate,�5�=�very�
conservative)? 1.12� On�average,�how�many�hours�of�TV�do�you�watch�per�week? 1.13� How�many�cups�of�coffee�did�you�drink�yesterday? 1.14� How�many�hours�did�you�sleep�last�night? 1.15� On�average,�how�many�alcoholic�drinks�do�you�have�per�week? 1.16� Can�you�tell�the�difference�between�Pepsi�and�Coke? 1.17� What�is�the�natural�color�of�your�hair�(black,�blonde,�brown,�red,�other)? 1.18� What�is�the�natural�color�of�your�eyes�(black,�blue,�brown,�green,�other)? 1.19� How�far�do�you�live�from�this�campus�(in�miles)? 1.20� On�average,�how�many�books�do�you�read�for�pleasure�each�month? 1.21� On�average,�how�many�hours�do�you�study�per�week? 1.22� �Which�question�on�this�survey�is�the�most�interesting�to�you?�The�least�interesting?
Possible Activities
1�� �For�each�item,�determine�the�most�likely�scale�of�measurement�(nominal,�ordinal,�inter- val,�or�ratio)�and�the�type�of�variable�[categorical�or�numerical�(if�numerical,�discrete�or� continuous)]�
2�� �Create� scenarios� in� which� one� or� more� of� the� variables� in� this� survey� would� be� a� constant,�given�the�delimitations�that�you�define�for�your�study��For�example,�we�are� designing�a�study�to�measure�study�habits�(as�measured�by�Question�1�21)�for�students� who�do�not�exercise�(Question�1�10)��In�this�sample�study,�our�constant�is�the�number�of� hours�per�week�that�a�student�studies�(in�this�case,�we�are�delimiting�that�to�be�zero— and�thus,�Question�1�10�will�be�a�constant;�all�students�in�our�study�will�have�answered� Question�1�10�as�“zero”)�
3�� �Collect�data�from�a�sample�of�individuals��In�subsequent�chapters,�you�will�be�asked�to� analyze�these�data�for�different�procedures�
N O T E : � An�actual�sample�dataset�using�this�survey�is�contained�on�the�website�(SPSS�file:� survey1)�and�is�utilized�in�later�chapters�
17
2 Data Representation
Chapter Outline
2�1� Tabular�Display�of�Distributions 2�1�1� Frequency�Distributions 2�1�2� Cumulative�Frequency�Distributions 2�1�3� Relative�Frequency�Distributions 2�1�4� Cumulative�Relative�Frequency�Distributions
2�2� Graphical�Display�of�Distributions 2�2�1� Bar�Graph 2�2�2� Histogram 2�2�3� Frequency�Polygon 2�2�4� Cumulative�Frequency�Polygon 2�2�5� Shapes�of�Frequency�Distributions 2�2�6� Stem-and-Leaf�Display
2�3� Percentiles 2�3�1� Percentiles 2�3�2� Quartiles 2�3�3� Percentile�Ranks 2�3�4� Box-and-Whisker�Plot
2�4� SPSS 2�5� Templates�for�Research�Questions�and�APA-Style�Paragraph
Key Concepts
� 1�� Frequencies,�cumulative�frequencies,�relative�frequencies,�and�cumulative�relative� frequencies
� 2�� Ungrouped�and�grouped�frequency�distributions � 3�� Sample�size � 4�� Real�limits�and�intervals � 5�� Frequency�polygons � 6�� Normal,�symmetric,�and�skewed�frequency�distributions � 7�� Percentiles,�quartiles,�and�percentile�ranks
18 An Introduction to Statistical Concepts
In� Chapter� 1,� we� introduced� the� wonderful� world� of� statistics�� There,� we� discussed� the� value�of�statistics,�met�a�few�of�the�more�interesting�statisticians,�and�defined�several�basic� statistical� concepts�� The� concepts� included� population,� parameter,� sample� and� statistic,� descriptive�and�inferential�statistics,�types�of�variables,�and�scales�of�measurement��In�this� chapter,�we�begin�our�examination�of�descriptive�statistics,�which�we�previously�defined� as�techniques�that�allow�us�to�tabulate,�summarize,�and�depict�a�collection�of�data�in�an� abbreviated� fashion�� We� used� the� example� of� collecting� data� from� 100,000� graduate� stu- dents�on�various�characteristics�(e�g�,�height,�weight,�gender,�grade�point�average,�aptitude� test� scores)�� Rather� than� having� to� carry� around� the� entire� collection� of� data� in� order� to� respond�to�questions,�we�mentioned�that�you�could�summarize�the�data�in�an�abbreviated� fashion�through�the�use�of�tables�and�graphs��This�way,�we�could�communicate�features�of� the�data�through�a�few�tables�or�figures�without�having�to�carry�around�the�entire�dataset�
This�chapter�deals�with�the�details�of�the�construction�of�tables�and�figures�for�purposes� of�describing�data��Specifically,�we�first�consider�the�following�types�of�tables:�frequency�dis- tributions�(ungrouped�and�grouped),�cumulative�frequency�distributions,�relative�frequency� distributions,�and�cumulative�relative�frequency�distributions��Next�we�look�at�the�following� types�of�figures:�bar�graph,�histogram,�frequency�polygon,�cumulative�frequency�polygon,� and�stem-and-leaf�display��We�also�discuss�common�shapes�of�frequency�distributions��Then� we� examine� the� use� of� percentiles,� quartiles,� percentile� ranks,� and� box-and-whisker� plots�� Finally,�we�look�at�the�use�of�SPSS�and�develop�an�APA-style�paragraph�of�results��Concepts� to�be�discussed�include�frequencies,�cumulative�frequencies,�relative�frequencies,�and�cumu- lative�relative�frequencies;�ungrouped�and�grouped�frequency�distributions;�sample�size;�real� limits�and�intervals;�frequency�polygons;�normal,�symmetric,�and�skewed�frequency�distri- butions;�and�percentiles,�quartiles,�and�percentile�ranks��Our�objectives�are�that�by�the�end�of� this�chapter,�you�will�be�able�to�(1)�construct�and�interpret�statistical�tables,�(2)�construct�and� interpret�statistical�graphs,�and�(3)�determine�and�interpret�percentile-related�information�
2.1 Tabular Display of Distributions
Consider�the�following�research�scenario:
Marie,�a�graduate�student�pursuing�a�master’s�degree�in�educational�research,�has�been� assigned�to�her�first�task�as�a�research�assistant��Her�faculty�mentor�has�given�Marie� quiz�data�collected�from�25�students�enrolled�in�an�introductory�statistics�course�and� has�asked�Marie�to�summarize�the�data��In�addition�to�the�data,�the�faculty�mentor�has� shared�the�following�research�question�that�should�guide�Marie�in�her�analysis�of�the� data:�How can the quiz scores of students enrolled in an introductory statistics class be graphi- cally represented in a table? In a figure? What is the distributional shape of the statistics quiz score? What is the 50th�percentile of the quiz scores?
In�this�section,�we�consider�ways�in�which�data�can�be�represented�in�the�form�of�tables�� More�specifically,�we�are�interested�in�how�the�data�for�a�single�variable�can�be�represented� (the�representation�of�data�for�multiple�variables�is�covered�in�later�chapters)��The�methods� described� here� include� frequency� distributions� (both� ungrouped� and� grouped),� cumu- lative� frequency� distributions,� relative� frequency� distributions,� and� cumulative� relative� frequency�distributions�
19Data Representation
2.1.1 Frequency distributions
Let�us�use�an�example�set�of�data�in�this�chapter�to�illustrate�ways�in�which�data�can�be� represented��We�have�selected�a�small�dataset�for�purposes�of�simplicity,�although�datasets� are�typically�larger�in�size��Note�that�there�is�a�larger�dataset�(based�on�the�survey�from� Chapter�1�interpretive�problem)�utilized�in�the�end�of�chapter�problems�and�available�on� our�website�as�“survey1�”�As�shown�in�Table�2�1,�the�smaller�dataset�consists�of�a�sample� of�25�student�scores�on�a�statistics�quiz,�where�the�maximum�score�is�20�points��If�a�col- league� asked� a� question� about� these� data,� again� a� response� could� be,� “take� a� look� at� the� data�yourself�”�This�would�not�be�very�satisfactory�to�the�colleague,�as�the�person�would� have�to�eyeball�the�data�to�answer�his�or�her�question��Alternatively,�one�could�present�the� data�in�the�form�of�a�table�so�that�questions�could�be�more�easily�answered��One�question� might�be�which�score�occurred�most�frequently?�In�other�words,�what�score�occurred�more� than�any�other�score?�Other�questions�might�be�which�scores�were�the�highest�and�lowest� scores�in�the�class?�and�where�do�most�of�the�scores�tend�to�fall?�In�other�words,�how�well� did�the�students�tend�to�do�as�a�class?�These�and�other�questions�can�be�easily�answered� by�looking�at�a�frequency distribution�
Let�us�first�look�at�how�an�ungrouped frequency distribution�can�be�constructed�for� these� and� other� data�� By� following� these� steps,� we� develop� the� ungrouped� frequency� distribution�as�shown�in�Table�2�2��The�first�step�is�to�arrange�the�unique�scores�on�a�list� from�the�lowest�score�to�the�highest�score��The�lowest�score�is�9�and�the�highest�score�is�20��Even� though�scores�such�as�15�were�observed�more�than�once,�the�value�of�15�is�only�entered� in�this�column�once��This�is�what�we�mean�by�unique��Note�that�if�the�score�of�15�was�not� observed,�it�could�still�be�entered�as�a�value�in�the�table�to�serve�as�a�placeholder�within�
Table 2.1
Statistics�Quiz�Data
9 11 20 15 19 10 19 18 14 12 17 11 13 16 17 19 18 17 13 17 15 18 17 19 15
Table 2.2
Ungrouped�Frequency�Distribution� of Statistics�Quiz�Data
X f cf rf crf
9 1 1 f/n�=�1/25�=��04 �04 10 1 2 �04 �08 11 2 4 �08 �16 12 1 5 �04 �20 13 2 7 �08 �28 14 1 8 �04 �32 15 3 11 �12 �44 16 1 12 �04 �48 17 5 17 �20 �68 18 3 20 �12 �80 19 4 24 �16 �96 20 1 25 �04 1�00
n�=�25 1�00
20 An Introduction to Statistical Concepts
the�distribution�of�scores�observed��We�label�this�column�as�“raw�score”�or�“X,”�as�shown�by� the�first�column�in�the�table��Raw scores�are�a�set�of�scores�in�their�original�form;�that�is,�the� scores�have�not�been�altered�or�transformed�in�any�way��X�is�often�used�in�statistics�to�denote� a�variable,�so�you�see�X�quite�a�bit�in�this�text��(As�a�side�note,�whenever�upper�or�lowercase� letters�are�used�to�denote�statistical�notation,�the�letter�is�always�italicized�)
The� second� step� is� to� determine� for� each� unique� score� the� number� of� times� it� was� observed�� We� label� this� second� column� as� “frequency”� or� by� the� abbreviation� “f�”� The� frequency� column� tells� us� how� many� times� or� how� frequently� each� unique� score� was� observed��For�instance,�the�score�of�20�was�only�observed�one�time�whereas�the�score�of�17� was�observed�five�times��Now�we�have�some�information�with�which�to�answer�the�ques- tions�of�our�colleague��The�most�frequently�observed�score�is�17,�the�lowest�score�is�9,�and� the�highest�score�is�20��We�can�also�see�that�scores�tended�to�be�closer�to�20�(the�highest� score)�than�to�9�(the�lowest�score)�
Two�other�concepts�need�to�be�introduced�that�are�included�in�Table�2�2��The�first�concept� is�sample size��At�the�bottom�of�the�second�column,�you�see�n�=�25��From�now�on,�n�will� be�used�to�denote�sample�size,�that�is,�the�total�number�of�scores�obtained�for�the�sample�� Thus,�because�25�scores�were�obtained�here,�then�n�=�25�
The�second�concept�is�related�to�real limits�and�intervals��Although�the�scores�obtained� for� this� dataset� happened� to� be� whole� numbers,� not� fractions� or� decimals,� we� still� need� a� system�that�will�cover�that�possibility��For�example,�what�would�we�do�if�a�student�obtained� a�score�of�18�25?�One�option�would�be�to�list�that�as�another�unique�score,�which�would�prob- ably�be�more�confusing�than�useful��A�second�option�would�be�to�include�it�with�one�of�the� other�unique�scores�somehow;�this�is�our�option�of�choice��The�system�that�all�researchers� use�to�cover�the�possibility�of�any�score�being�obtained�is�through�the�concepts�of�real�limits� and� intervals�� Each� value� of� X� in� Table� 2�2� can� be� thought� of� as� being� the� midpoint� of� an� interval��Each�interval�has�an�upper�and�a�lower�real�limit��The�upper�real�limit�of�an�interval� is�halfway�between�the�midpoint�of�the�interval�under�consideration�and�the�midpoint�of� the�next�larger�interval��For�example,�the�value�of�18�represents�the�midpoint�of�an�interval�� The�next�larger�interval�has�a�midpoint�of�19��Therefore,�the�upper�real�limit�of�the�interval� containing�18�would�be�18�5,�halfway�between�18�and�19��The�lower�real�limit�of�an�interval� is�halfway�between�the�midpoint�of�the�interval�under�consideration�and�the�midpoint�of�the� next�smaller�interval��Following�the�example�interval�of�18�again,�the�next�smaller�interval� has�a�midpoint�of�17��Therefore,�the�lower�real�limit�of�the�interval�containing�18�would�be� 17�5,�halfway�between�18�and�17��Thus,�the�interval�of�18�has�18�5�as�an�upper�real�limit�and� 17�5�as�a�lower�real�limit��Other�intervals�have�their�upper�and�lower�real�limits�as�well�
Notice� that� adjacent� intervals� (i�e�,� those� next� to� one� another)� touch� at� their� respective� real�limits��For�example,�the�18�interval�has�18�5�as�its�upper�real�limit�and�the�19�interval� has� 18�5� as� its� lower� real� limit�� This� implies� that� any� possible� score� that� occurs� can� be� placed�into�some�interval�and�no�score�can�fall�between�two�intervals��If�someone�obtains� a�score�of�18�25,�that�will�be�covered�in�the�18�interval��The�only�limitation�to�this�procedure� is� that� because� adjacent� intervals� must� touch� in� order� to� deal� with� every� possible� score,� what�do�we�do�when�a�score�falls�precisely�where�two�intervals�touch�at�their�real�limits� (e�g�,�at�18�5)?�There�are�two�possible�solutions��The�first�solution�is�to�assign�the�score�to� one�interval�or�another�based�on�some�rule��For�instance,�we�could�randomly�assign�such� scores� to� one� interval� or� the� other� by� flipping� a� coin�� Alternatively,� we� could� arbitrarily� assign�such�scores�always�into�either�the�larger�or�smaller�of�the�two�intervals��The�second� solution�is�to�construct�intervals�such�that�the�number�of�values�falling�at�the�real�limits� is�minimized��For�example,�say�that�most�of�the�scores�occur�at��5�(e�g�,�15�5,�16�5,�17�5)��We� could�construct�the�intervals�with��5�as�the�midpoint�and��0�as�the�real�limits��Thus,�the�15�5�
21Data Representation
interval�would�have�15�5�as�the�midpoint,�16�0�as�the�upper�real�limit,�and�15�0�as�the�lower� real�limit��It�should�also�be�noted�that,�strictly�speaking,�real�limits�are�only�appropriate� for�continuous�variables�but�not�for�discrete�variables��That�is,�since�discrete�variables�can� only�have�limited�values,�we�probably�don’t�need�to�worry�about�real�limits�(e�g�,�there�is� not�really�an�interval�for�two�children)�
Finally,� the� width� of� an� interval� is� defined� as� the� difference� between� the� upper� and� lower�real�limits�of�an�interval��We�can�denote�this�as�w = URL − LRL,�where�w�is�interval� width,�and�URL�and�LRL�are�the�upper�and�lower�real�limits,�respectively��In�the�case�of� our�example�interval�again,�we�see�that�w = URL − LRL�=�18�5�−�17�5�=�1�0��For�Table�2�2,� then,�all�intervals�have�the�same�interval�width�of�1�0��For�each�interval,�we�have�a�mid- point,�a�lower�real�limit�that�is�one-half�unit�below�the�midpoint,�and�an�upper�real�limit� that�is�one-half�unit�above�the�midpoint��In�general,�we�want�all�of�the�intervals�to�have�the� same�width�for�consistency�as�well�as�for�equal�interval�reasons��The�only�exception�might� be�if�the�largest�or�smallest�intervals�were�above�a�certain�value�(e�g�,�greater�than�20)�or� below�a�certain�value�(e�g�,�less�than�9),�respectively�
A�frequency�distribution�with�an�interval�width�of�1�0�is�often�referred�to�as�an�ungrouped frequency distribution,�as�the�intervals�have�not�been�grouped�together��Does�the�interval� width�always�have�to�be�equal�to�1�0?�The�answer,�of�course,�is�no��We�could�group�intervals� together�and�form�what�is�often�referred�to�as�a�grouped frequency distribution��For�our� example�data,�we�can�construct�a�grouped�frequency�distribution�with�an�interval�width� of�2�0,�as�shown�in�Table�2�3��The�largest�interval�now�contains�the�scores�of�19�and�20,�the� second� largest� interval� the� scores� of� 17� and� 18,� and� so� on� down� to� the� smallest� interval� with�the�scores�of�9�and�10��Correspondingly,�the�largest�interval�contains�a�frequency�of�5,� the�second�largest�interval�a�frequency�of�8,�and�the�smallest�interval�a�frequency�of�2��All� we�have�really�done�is�collapse�the�intervals�from�Table�2�2,�where�interval�width�was�1�0,� into�the�intervals�of�width�2�0,�as�shown�in�Table�2�3��If�we�take,�for�example,�the�interval� containing�the�scores�of�17�and�18,�then�the�midpoint�of�the�interval�is�17�5,�the�URL�is�18�5,� the�LRL�is�16�5,�and�thus�w�=�2�0��The�interval�width�could�actually�be�any�value,�including� �20�or�100,�depending�on�what�best�suits�the�data�
How�does�one�determine�what�the�proper�interval�width�should�be?�If�there�are�many� frequencies�for�each�score�and�less�than�15�or�20�intervals,�then�an�ungrouped�frequency� distribution�with�an�interval�width�of�1�is�appropriate�(and�this�is�the�default�in�SPSS�for� determining� frequency� distributions)�� If� there� are� either� minimal� frequencies� per� score� (say� 1� or� 2)� or� a� large� number� of� unique� scores� (say� more� than� 20),� then� a� grouped� fre- quency�distribution�with�some�other�interval�width�is�appropriate��For�a�first�example,�say�
Table 2.3
Grouped�Frequency�Distribution� of�Statistics�Quiz�Data
X f
9–10 2 11–12 3 13–14 3 15–16 4 17–18 8 19–20 5
n�=�25
22 An Introduction to Statistical Concepts
that�there�are�100�unique�scores�ranging�from�0�to�200��An�ungrouped�frequency�distri- bution�would�not�really�summarize�the�data�very�well,�as�the�table�would�be�quite�large�� The�reader�would�have�to�eyeball�the�table�and�actually�do�some�quick�grouping�in�his�or� her�head�so�as�to�gain�any�information�about�the�data��An�interval�width�of�perhaps�10–15� would�be�more�useful��In�a�second�example,�say�that�there�are�only�20�unique�scores�rang- ing�from�0�to�30,�but�each�score�occurs�only�once�or�twice��An�ungrouped�frequency�dis- tribution�would�not�be�very�useful�here�either,�as�the�reader�would�again�have�to�collapse� intervals�in�his�or�her�head��Here�an�interval�width�of�perhaps�2–5�would�be�appropriate�
Ultimately,�deciding�on�the�interval�width,�and�thus,�the�number�of�intervals,�becomes�a� trade-off�between�good�communication�of�the�data�and�the�amount�of�information�contained� in�the�table��As�interval�width�increases,�more�and�more�information�is�lost�from�the�original� data�� For� the� example� where� scores� range� from� 0� to� 200� and� using� an� interval� width� of� 10,� some� precision� in� the� 15� scores� contained� in� the� 30–39� interval� is� lost�� In� other� words,� the� reader�would�not�know�from�the�frequency�distribution�where�in�that�interval�the�15�scores� actually� fall�� If� you� want� that� information� (you� may� not),� you� would� need� to� return� to� the� original�data��At�the�same�time,�an�ungrouped�frequency�distribution�for�those�data�would� not�have�much�of�a�message�for�the�reader��Ultimately,�the�decisive�factor�is�the�adequacy�with� which�information�is�communicated�to�the�reader��The�nature�of�the�interval�grouping�comes� down�to�whatever�form�best�represents�the�data��With�today’s�powerful�statistical�computer� software,�it�is�easy�for�the�researcher�to�try�several�different�interval�widths�before�deciding� which�one�works�best�for�a�particular�set�of�data��Note�also�that�the�frequency�distribution�can� be�used�with�variables�of�any�measurement�scale,�from�nominal�(e�g�,�the�frequencies�for�eye� color�of�a�group�of�children)�to�ratio�(e�g�,�the�frequencies�for�the�height�of�a�group�of�adults)�
2.1.2 Cumulative Frequency distributions
A�second�type�of�frequency�distribution�is�known�as�the�cumulative frequency distribution�� For�the�example�data,�this�is�depicted�in�the�third�column�of�Table�2�2�and�labeled�as�“cf�”�To� put�it�simply,�the�number�of�cumulative�frequencies�for�a�particular�interval�is�the�number�of� scores�contained�in�that�interval�and�all�of�the�smaller�intervals��Thus,�the�nine�interval�con- tains�one�frequency,�and�there�are�no�frequencies�smaller�than�that�interval,�so�the�cumulative� frequency�is�simply�1��The�10�interval�contains�one�frequency,�and�there�is�one�frequency�in� a�smaller�interval,�so�the�cumulative�frequency�is�2��The�11�interval�contains�two�frequencies,� and�there�are�two�frequencies�in�smaller�intervals;�thus,�the�cumulative�frequency�is�4��Then� four�people�had�scores�in�the�11�interval�and�smaller�intervals��One�way�to�think�about�deter- mining�the�cumulative�frequency�column�is�to�take�the�frequency�column�and�accumulate� downward�(i�e�,�from�the�top�down,�yielding�1;�1�+�1�=�2;�1�+�1�+�2�=�4;�etc�)��Just�as�a�check,�the� cf�in�the�largest�interval�(i�e�,�the�interval�largest�in�value)�should�be�equal�to�n,�the�number� of�scores�in�the�sample,�25�in�this�case��Note�also�that�the�cumulative�frequency�distribution� can�be�used�with�variables�of�measurement�scales�from�ordinal�(e�g�,�the�number�of�students� receiving�a�B�or�less)�to�ratio�(e�g�,�the�number�of�adults�that�are�5′7″�or�less),�but�cannot�be� used�with�nominal�as�there�is�not�at�least�rank�order�to�nominal�data�(and�thus�accumulating� information�from�one�nominal�category�to�another�does�not�make�sense)�
2.1.3 Relative Frequency distributions
A�third�type�of�frequency�distribution�is�known�as�the�relative frequency distribution��For� the�example�data,�this�is�shown�in�the�fourth�column�of�Table�2�2�and�labeled�as�“rf�”�Relative� frequency�is�simply�the�percentage�of�scores�contained�in�an�interval��Computationally,�
23Data Representation
rf = f/n�� For� example,� the� percentage� of� scores� occurring� in� the� 17� interval� is� computed� as� rf� =� 5/25� =� �20�� Relative� frequencies� take� sample� size� into� account� allowing� us� to� make� statements�about�the�number�of�individuals�in�an�interval�relative�to�the�total�sample��Thus,� rather�than�stating�that�5�individuals�had�scores�in�the�17�interval,�we�could�say�that�20%�of� the�scores�were�in�that�interval��In�the�popular�press,�relative�frequencies�(which�they�call� percentages)�are�quite�often�reported�in�tables�without�the�frequencies��Note�that�the�sum�of� the�relative�frequencies�should�be�1�00�(or�100%)�within�rounding�error��Also�note�that�the� relative�frequency�distribution�can�be�used�with�variables�of�any�measurement�scale,�from� nominal�(e�g�,�the�percent�of�children�with�blue�eye�color)�to�ratio�(e�g�,�the�percent�of�adults� that�are�5′7″)�
2.1.4 Cumulative Relative Frequency distributions
A�fourth�and�final�type�of�frequency�distribution�is�known�as�the�cumulative relative fre- quency distribution��For�the�example�data,�this�is�depicted�in�the�fifth�column�of�Table�2�2� and�labeled�as�“crf�”�The�number�of�cumulative�relative�frequencies�for�a�particular�interval� is�the�percentage�of�scores�in�that�interval�and�smaller��Thus,�the�nine�interval�has�a�rela- tive�frequency�of��04,�and�there�are�no�relative�frequencies�smaller�than�that�interval,�so�the� cumulative�relative�frequency�is�simply��04��The�10�interval�has�a�relative�frequency�of��04,� and� the� relative� frequencies� less� than� that� interval� are� �04,� so� the� cumulative� relative� fre- quency�is��08��The�11�interval�has�a�relative�frequency�of��08,�and�the�relative�frequencies�less� than�that�interval�total��08,�so�the�cumulative�relative�frequency�is��16��Thus,�16%�of�the�peo- ple�had�scores�in�the�11�interval�and�smaller��In�other�words,�16%�of�people�scored�11�or�less�� One�way�to�think�about�determining�the�cumulative�relative�frequency�column�is�to�take� the�relative�frequency�column�and�accumulate�downward�(i�e�,�from�the�top�down,�yield- ing��04;��04�+��04�=��08;��04�+��04�+��08�=��16;�etc�)��Just�as�a�check,�the�crf�in�the�largest�interval� should�be�equal�to�1�0,�within�rounding�error,�just�as�the�sum�of�the�relative�frequencies�is� equal�to�1�0��Also�note�that�the�cumulative�relative�frequency�distribution�can�be�used�with� variables�of�measurement�scales�from�ordinal�(e�g�,�the�percent�of�students�receiving�a�B�or� less)�to�ratio�(e�g�,�the�percent�of�adults�that�are�5′7″�or�less)��As�with�relative�frequency�dis- tributions,�cumulative�relative�frequency�distributions�cannot�be�used�with�nominal�data�
2.2 Graphical Display of Distributions
In� this� section,� we� consider� several� types� of� graphs�for� viewing� a� distribution� of� scores�� Again,�we�are�still�interested�in�how�the�data�for�a�single�variable�can�be�represented,�but� now� in� a� graphical� display� rather� than� a� tabular� display�� The� methods� described� here� include�the�bar�graph,�histogram,�frequency,�relative�frequency,�cumulative�frequency�and� cumulative� relative� frequency� polygons,� and� stem-and-leaf� display�� Common� shapes� of� distributions�will�also�be�discussed�
2.2.1 bar Graph
A�popular�method�used�for�displaying�nominal�scale�data�in�graphical�form�is�the�bar graph�� As� an� example,� say� that� we� have� data� on� the� eye� color� of� a� sample� of� 20� chil- dren�� Ten� children� are� blue� eyed,� six� are� brown� eyed,� three� are� green� eyed,� and� one�
24 An Introduction to Statistical Concepts
is�black�eyed��A�bar�graph�for�these�data�is�shown�in�Figure�2�1�(SPSS�generated)��The� horizontal� axis,� going� from� left� to� right� on� the� page,� is� often� referred� to� in� statistics� as�the�X�axis�(for�variable�X,�in�this�example�our�variable�is�eye color)��On�the�X�axis�of� Figure�2�1,�we�have�labeled�the�different�eye�colors�that�were�observed�from�individu- als� in� our� sample�� The� order� of� the� colors� is� not� relevant� (remember,� this� is� nominal� data,�so�order�or�rank�is�irrelevant)��The�vertical�axis,�going�from�bottom�to�top�on�the� page,�is�often�referred�to�in�statistics�as�the�Y�axis�(the�Y�label�will�be�more�relevant�in� later�chapters�when�we�have�a�second�variable�Y)��On�the�Y�axis�of�Figure�2�1,�we�have� labeled�the�frequencies��Finally,�a�bar�is�drawn�for�each�eye�color�where�the�height�of� the�bar�denotes�the�number�of�frequencies�for�that�particular�eye�color�(i�e�,�the�number� of�times�that�particular�eye�color�was�observed�in�our�sample)��For�example,�the�height� of�the�bar�for�the�blue-eyed�category�is�10�frequencies��Thus,�we�see�in�the�graph�which� eye� color� is� most� popular� in� this� sample� (i�e�,� blue)� and� which� eye� color� occurs� least� (i�e�,�black)�
Note�that�the�bars�are�separated�by�some�space�and�do�not�touch�one�another,�reflect- ing�the�nature�of�nominal�data��As�there�are�no�intervals�or�real�limits�here,�we�do�not� want� the� bars� to� touch� one� another�� One� could� also� plot� relative� frequencies� on� the� Y� axis�to�reflect�the�percentage�of�children�in�the�sample�who�belong�to�each�category�of� eye�color��Here�we�would�see�that�50%�of�the�children�had�blue�eyes,�30%�brown�eyes,� 15%�green�eyes,�and�5%�black�eyes��Another�method�for�displaying�nominal�data�graphi- cally�is�the�pie�chart,�where�the�pie�is�divided�into�slices�whose�sizes�correspond�to�the� frequencies� or� relative� frequencies� of� each� category�� However,� for� numerous� reasons� (e�g�,�contains�little�information�when�there�are�few�categories;�is�unreadable�when�there� are�many�categories;�visually�assessing�the�sizes�of�each�slice�is�difficult�at�best),�the�pie� chart� is� statistically� problematic� such� that� Tufte� (1992)� states,� “the� only� worse� design� than�a�pie�chart�is�several�of�them”�(p��178)��The�bar�graph�is�the�recommended�graphic� for�nominal�data�
FIGuRe 2.1 Bar�graph�of�eye-color�data�
10
8
6
4
Fr eq
ue nc
y
2
Black Blue Brown Eye color
Green
25Data Representation
2.2.2 histogram
A�method�somewhat�similar�to�the�bar�graph�that�is�appropriate�for�data�that�are�at�least� ordinal�(i�e�,�ordinal,�interval,�or�ratio)�is�the�histogram��Because�the�data�are�at�least�theo- retically�continuous�(even�though�they�may�be�measured�in�whole�numbers),�the�main�dif- ference�in�the�histogram�(as�compared�to�the�bar�graph)�is�that�the�bars�touch�one�another,� much�like�intervals�touching�one�another�as�real�limits��An�example�of�a�histogram�for�the� statistics�quiz�data�is�shown�in�Figure�2�2�(SPSS�generated)��As�you�can�see,�along�the�X�axis� we�plot�the�values�of�the�variable�X�and�along�the�Y�axis�the�frequencies�for�each�interval�� The�height�of�the�bar�again�corresponds�to�the�number�of�frequencies�for�a�particular�value� of�X��This�figure�represents�an�ungrouped�histogram�as�the�interval�size�is�1��That�is,�along� the�X�axis�the�midpoint�of�each�bar�is�the�midpoint�of�the�interval,�the�bar�begins�on�the�left� at�the�lower�real�limit�of�the�interval,�the�bar�ends�on�the�right�at�the�upper�real�limit,�and� the�bar�is�one�unit�wide��If�we�wanted�to�use�an�interval�size�of�2,�for�example,�using�the� grouped�frequency�distribution�in�Table�2�3,�then�we�could�construct�a�grouped�histogram� in�the�same�way;�the�differences�would�be�that�the�bars�would�be�two�units�wide,�and�the� height�of�the�bars�would�obviously�change��Try�this�one�on�your�own�for�practice�
One� could� also� plot� relative� frequencies� on� the� Y� axis� to� reflect� the� percentage� of� stu- dents�in�the�sample�whose�scores�fell�into�a�particular�interval��In�reality,�all�that�we�have� to�change�is�the�scale�of�the�Y�axis��The�height�of�the�bars�would�remain�the�same��For�this� particular�dataset,�each�frequency�corresponds�to�a�relative�frequency�of��04�
2.2.3 Frequency polygon
Another� graphical� method� appropriate� for� data� that� have� at� least� some� rank� order� (i�e�,� ordinal,�interval,�or�ratio)�is�the�frequency polygon�(line�graph�in�SPSS�terminology)��A�poly- gon�is�defined�simply�as�a�many-sided�figure��The�frequency�polygon�is�set�up�in�a�fashion�
5
4
3
2
Fr eq
ue nc
y
1
9 10 11 12 13 14 15 Quiz
16 17 18 19 20 FIGuRe 2.2 Histogram�of�statistics�quiz�data�
26 An Introduction to Statistical Concepts
similar�to�the�histogram��However,�rather�than�plotting�a�bar�for�each�interval,�points�are� plotted�for�each�interval�and�then�connected�together�as�shown�in�Figure�2�3�(SPSS�gener- ated)��The�axes�are�the�same�as�with�the�histogram��A�point�is�plotted�at�the�intersection�(or� coordinates)�of�the�midpoint�of�each�interval�along�the�X�axis�and�the�frequency�for�that� interval�along�the�Y�axis��Thus,�for�the�15�interval,�a�point�is�plotted�at�the�midpoint�of�the� interval� 15�0�and�for�three� frequencies��Once� the�points� are� plotted� for� each� interval,� we� “connect�the�dots�”
One�could�also�plot�relative�frequencies�on�the�Y�axis�to�reflect�the�percentage�of�students� in�the�sample�whose�scores�fell�into�a�particular�interval��This�is�known�as�the�relative fre- quency polygon��As�with�the�histogram,�all�we�have�to�change�is�the�scale�of�the�Y�axis��The� position�of�the�polygon�would�remain�the�same��For�this�particular�dataset,�each�frequency� corresponds�to�a�relative�frequency�of��04�
Note�also�that�because�the�histogram�and�frequency�polygon�each�contain�the�exact�same� information,�Figures�2�2�and�2�3�can�be�superimposed�on�one�another��If�you�did�this,�you� would�see�that�the�points�of�the�frequency�polygon�are�plotted�at�the�top�of�each�bar�of�the� histogram��There�is�no�advantage�of�the�histogram�or�frequency�polygon�over�the�other;�how- ever,�the�histogram�is�more�frequently�used�due�to�its�availability�in�all�statistical�software�
2.2.4 Cumulative Frequency polygon
Cumulative�frequencies�of�data�that�have�at�least�some�rank�order�(i�e�,�ordinal,�interval,� or�ratio)�can�be�displayed�as�a�cumulative frequency polygon�(sometimes�referred�to�as� the�ogive curve)��As�shown�in�Figure�2�4�(SPSS�generated),�the�differences�between�the� frequency� polygon� and� the� cumulative� frequency� polygon� are� that� (a)� the� cumulative� frequency� polygon� involves� plotting� cumulative� frequencies� along� the� Y� axis,� (b)� the� points� should� be� plotted� at the upper real limit� of� each� interval� (although� SPSS� plots� the points�at�the�interval�midpoints�by�default),�and�(c)�the�polygon�cannot�be�closed�on� the�right-hand�side�
FIGuRe 2.3 Frequency�polygon�of�statistics�quiz�data�
5 Markers/lines show count
4
3
Fr eq
ue nc
y
2
1
0 9 10 11 12 13 14 15
Quiz 16 17 18 19 20
27Data Representation
Let�us�discuss�each�of�these�differences��First,�the�Y�axis�represents�the�cumulative�frequen- cies�from�the�cumulative�frequency�distribution��The�X�axis�is�the�usual�set�of�raw�scores�� Second,�to�reflect�the�cumulative�nature�of�this�type�frequency,�the�points�must�be�plotted�at� the�upper�real�limit�of�each�interval��For�example,�the�cumulative�frequency�for�the�16�inter- val�is�12,�indicating�that�there�are�12�scores�in�that�interval�and�smaller��Finally,�the�polygon� cannot� be� closed� on� the� right-hand� side�� Notice� that� as� you� move� from� left� to� right� in� the� cumulative�frequency�polygon,�the�height�of�the�points�always�increases�or�stays�the�same�� Because� of� the� nature� of� accumulating� information,� there� will� never� be� a� decrease� in� the� accumulation�of�the�frequencies��For�example,�there�is�an�increase�in�cumulative�frequency� from�the�16�to�the�17�interval�as�five�new�frequencies�are�included��Beyond�the�20�interval,�the� number�of�cumulative�frequencies�remains�at�25�as�no�new�frequencies�are�included�
One�could�also�plot�cumulative�relative�frequencies�on�the�Y�axis�to�reflect�the�percent- age�of�students�in�the�sample�whose�scores�fell�into�a�particular�interval�and�smaller��This� is� known� as� the� cumulative relative frequency polygon�� All� we� have� to� change� is� the� scale� of� the� Y� axis� to� cumulative� relative� frequency�� The� position� of� the� polygon� would� remain�the�same��For�this�particular�dataset,�each�cumulative�frequency�corresponds�to�a� cumulative�relative�frequency�of��04��Thus,�a�cumulative�relative�frequency�polygon�of�the� example�data�would�look�exactly�like�Figure�2�4;�except�on�the�Y�axis�we�plot�cumulative� relative�frequencies�ranging�from�0�to�1�
2.2.5 Shapes of Frequency distributions
There�are�several�common�shapes�of�frequency�distributions�that�you�are�likely�to�encoun- ter,� as� shown� in� Figure� 2�5�� These� are� briefly� described� here� and� more� fully� in� later� chapters�� Figure� 2�5a� is� a� normal distribution� (or� bell-shaped� curve)� where� most� of� the� scores�are�in�the�center�of�the�distribution�with�fewer�higher�and�lower�scores��The�normal� distribution�plays�a�large�role�in�statistics,�both�for�descriptive�statistics�(as�we�show�begin- ning�in�Chapter�4)�and�particularly�as�an�assumption�for�many�inferential�statistics�(as�we� show�beginning�in�Chapter�6)��This�distribution�is�also�known�as�symmetric�because�if�we� divide�the�distribution�into�two�equal�halves�vertically,�the�left�half�is�a�mirror�image�of� the�right�half�(see�Chapter�4)��Figure�2�5b�is�a�positively skewed�distribution�where�most� of�the�scores�are�fairly�low�and�there�are�a�few�higher�scores�(see�Chapter�4)��Figure�2�5c�is�
25
20
15
10
C um
ul at
iv e
fr eq
ue nc
y
5
0
9 10 11 12 13 14 15 Quiz
16 17 18 19 20 FIGuRe 2.4 Cumulative� frequency� polygon� of� statistics�quiz�data�
28 An Introduction to Statistical Concepts
a�negatively skewed�distribution�where�most�of�the�scores�are�fairly�high�and�there�are�a� few�lower�scores�(see�Chapter�4)��Skewed�distributions�are�not�symmetric�as�the�left�half�is� not�a�mirror�image�of�the�right�half�
2.2.6 Stem-and-leaf display
A�refined�form�of�the�grouped�frequency�distribution�is�the�stem-and-leaf display,�devel- oped�by�John�Tukey�(1977)��This�is�shown�in�Figure�2�6�(SPSS�generated)�for�the�example� statistics�quiz�data��The�stem-and-leaf�display�was�originally�developed�to�be�constructed� on� a� typewriter� using� lines� and� numbers� in� a� minimal� amount� of� space�� In� a� way,� the�
f
x(a)
f
x(b)
f
x(c)
FIGuRe 2.5 Common�shapes�of�frequency�distributions:�(a)�normal,�(b)�positively�skewed,�and�(c)�negatively�skewed�
FIGuRe 2.6 Stem-and-leaf�display�of�statistics�quiz�data�
Quiz Stem-and-Leaf Plot
Frequency Stem and Leaf
1.00 0 . 9 7.00 1 . 0112334 16.00 1 . 5556777778889999 1.00 2 . 0
Stem width: 10.0 Each leaf: 1 case(s)
29Data Representation
stem-and-leaf�display�looks�like�a�grouped�type�of�histogram�on�its�side��The�vertical�value� on�the�left�is�the�stem�and,�in�this�example,�represents�all�but�the�last�digit�(i�e�,�the�tens�digit)�� The� leaf� represents,� in� this� example,� the� remaining� digit� of� each� score� (i�e�,� the� unit’s� digit)��Note�that�SPSS�has�grouped�values�in�increments�of�five��For�example,�the�second� line�(“1�0112334”)�indicates�that�there�are�7�scores�from�10�to�14;�thus,�“1�0”�means�that�there� is�one�frequency�for�the�score�of�10��The�fact�that�there�are�two�values�of�“1”�that�occur�in� that�stem�indicates�that�the�score�of�11�occurred�twice��Interpreting�the�rest�of�this�stem,�we� see�that�12�occurred�once�(i�e�,�there�is�only�one�2�in�the�stem),�13�occurred�twice�(i�e�,�there� are�two�3s�in�the�stem),�and�14�occurred�once�(i�e�,�only�one�4�in�the�stem)��From�the�stem- and-leaf�display,�one�can�determine�every�one�of�the�raw�scores;�this�is�not�possible�with� a� typical� grouped� frequency� distribution� (i�e�,� no� information� is� lost� in� a� stem-and-leaf� display)��However,�with�a�large�sample�the�display�can�become�rather�unwieldy��Consider� what�a�stem-and-leaf�display�would�look�like�for�100,000�GRE�scores!
In�summary,�this�section�included�the�most�basic�types�of�statistical�graphics,�although� more� advanced� graphics� are� described� in� later� chapters�� Note,� however,� that� there� are� a� number�of�publications�on�how�to�properly�display�graphics,�that�is,�“how�to�do�graphics� right�”�While�a�detailed�discussion�of�statistical�graphics�is�beyond�the�scope�of�this�text,� the� following� publications� are� recommended:� Chambers,� Cleveland,� Kleiner,� and� Tukey� (1983),�Schmid�(1983),�Wainer�(e�g�,�1984,�1992,�2000),�Tufte�(1992),�Cleveland�(1993),�Wallgren,� Wallgren,�Persson,�Jorner,�and�Haaland�(1996),�Robbins�(2004),�and�Wilkinson�(2005)�
2.3 Percentiles
In�this�section,�we�consider�several�concepts�and�the�necessary�computations�for�the�area� of�percentiles,�including�percentiles,�quartiles,�percentile�ranks,�and�the�box-and-whisker� plot��For�instance,�you�might�be�interested�in�determining�what�percentage�of�the�distribu- tion�of�the�GRE-Quantitative�subtest�fell�below�a�score�of�600�or�in�what�score�divides�the� distribution�of�the�GRE-Quantitative�subtest�into�two�equal�halves�
2.3.1 percentiles
Let�us�define�a�percentile�as�that�score�below�which�a�certain�percentage�of�the�distribu- tion�lies��For�instance,�you�may�be�interested�in�that�score�below�which�50%�of�the�distri- bution�of�the�GRE-Quantitative�subscale�lies��Say�that�this�score�is�computed�as�480;�then� this�would�mean�that�50%�of�the�scores�fell�below�a�score�of�480��Because�percentiles�are� scores,�they�are�continuous�values�and�can�take�on�any�value�of�those�possible��The�30th� percentile�could�be,�for�example,�the�score�of�387�6750��For�notational�purposes,�a�percen- tile�will�be�known�as�Pi,�where�the�i�subscript�denotes�the�particular�percentile�of�interest,� between�0�and�100��Thus,�the�30th�percentile�for�the�previous�example�would�be�denoted� as�P30�=�387�6750�
Let�us�now�consider�how�percentiles�are�computed��The�formula�for�computing�the�Pi� percentile�is
� P LRL i n cf
f wi = +
−
%( ) � (2�1)
30 An Introduction to Statistical Concepts
where LRL�is�the�lower�real�limit�of�the�interval�containing�Pi i%�is�the�percentile�desired�(expressed�as�a�proportion�from�0�to�1) n�is�the�sample�size cf� is� the� cumulative� frequency� less� than� but� not� including� the� interval� containing� Pi�
(known�as�cf�below) f�is�the�frequency�of�the�interval�containing�Pi w�is�the�interval�width
As� an� example,� consider� computing� the� 25th� percentile� of� our� statistics� quiz� data�� This� would�correspond�to�that�score�below�which�25%�of�the�distribution�falls��For�the�example� data�in�the�form�presented�in�Table�2�2,�using�Equation�2�1,�we�compute�P25�as�follows:
� P LRL
i n cf f
w25 12 5 25 25 5
2 1 12 5 0 625= +
−
= + −
= + %( )
. %( )
. . == 13 125.
Conceptually,� let� us� discuss� how� the� equation� works�� First,� we� have� to� determine� what� interval�contains�the�percentile�of�interest��This�is�easily�done�by�looking�in�the�crf�column� of�the�frequency�distribution�for�the�interval�that�contains�a�crf�of��25�somewhere�within� the� interval�� We� see� that� for� the� 13� interval� the� crf� =� �28,� which� means� that� the� interval� spans�a�crf�of��20�(the�URL�of�the�12�interval)�up�to��28�(the�URL�of�the�13�interval)�and�thus� contains��25��The�next�largest�interval�of�14�takes�us�from�a�crf�of��28�up�to�a�crf�of��32�and� thus�is�too�large�for�this�particular�percentile��The�next�smallest�interval�of�12�takes�us�from� a�crf�of��16�up�to�a�crf�of��20�and�thus�is�too�small��The�LRL�of�12�5�indicates�that�P25�is�at�least� 12�5��The�rest�of�the�equation�adds�some�positive�amount�to�the�LRL�
Next�we�have�to�determine�how�far�into�that�interval�we�need�to�go�in�order�to�reach�the� desired�percentile��We�take�i�percent�of�n,�or�in�this�case�25%�of�the�sample�size�of�25,�which�is� 6�25��So�we�need�to�go�one-fourth�of�the�way�into�the�distribution,�or�6�25�scores,�to�reach�the� 25th�percentile��Another�way�to�think�about�this�is,�because�the�scores�have�been�rank-ordered� from�lowest�or�smallest�(top�of�the�frequency�distribution)�to�highest�or�largest�(bottom�of�the� frequency�distribution),�we�need�to�go�25%,�or�6�25�scores,�into�the�distribution�from�the�top� (or�smallest�value)�to�reach�the�25th�percentile��We�then�subtract�out�all�cumulative�frequen- cies�smaller�than�(or�below)�the�interval�we�are�looking�in,�where�cf�below�=�5��Again�we�just� want�to�determine�how�far�into�this�interval�we�need�to�go,�and�thus,�we�subtract�out�all�of� the�frequencies�smaller�than�this�interval,�or�cf�below��The�numerator�then�becomes�6�25�−�5�=� 1�25��Then�we�divide�by�the�number�of�frequencies�in�the�interval�containing�the�percentile� we�are�looking�for��This�forms�the�ratio�of�how�far�into�the�interval�we�go��In�this�case,�we� needed�to�go�1�25�scores�into�the�interval�and�the�interval�contains�2�scores;�thus,�the�ratio�is� 1�25/2�=��625��In�other�words,�we�need�to�go��625�unit�into�the�interval�to�reach�the�desired� percentile��Now�that�we�know�how�far�into�the�interval�to�go,�we�need�to�weigh�this�by�the� width�of�the�interval��Here�we�need�to�go�1�25�scores�into�an�interval�containing�2�scores�that� is�1�unit�wide,�and�thus,�we�go��625�unit�into�the�interval�[(1�25/2)�1�=��625]��If�the�interval�width� was�instead�10,�then�1�25�scores�into�the�interval�would�be�equal�to�6�25�units�
Consider�two�more�worked�examples�to�try�on�your�own,�either�through�statistical�software� or�by�hand��The�50th�percentile,�P50,�is
� P50 16 5
50 25 12 5
1 16 5 0 100 16 600= + −
= + =. %( )
. . .
31Data Representation
while�the�75th�percentile,�P75,�is
� P75 17 5
75 25 17 3
1 17 5 0 583 18 083= + −
= + =. %( )
. . .
We�have�only�examined�a�few�example�percentiles�of�the�many�possibilities�that�exist��For� example,�we�could�also�have�determined�P55�5�or�even�P99�5��Thus,�we�could�determine�any� percentile,�in�whole�numbers�or�decimals,�between�0�and�100��Next�we�examine�three�par- ticular�percentiles�that�are�often�of�interest,�the�quartiles�
2.3.2 Quartiles
One�common�way�of�dividing�a�distribution�of�scores�into�equal�groups�of�scores�is�known� as�quartiles��This�is�done�by�dividing�a�distribution�into�fourths�or�quartiles�where�there�are� four�equal�groups,�each�containing�25%�of�the�scores��In�the�previous�examples,�we�deter- mined�P25,�P50,�and�P75,�which�divided�the�distribution�into�four�equal�groups,�from�0�to�25,� from�25�to�50,�from�50�to�75,�and�from�75�to�100��Thus,�the�quartiles�are�special�cases�of�per- centiles��A�different�notation,�however,�is�often�used�for�these�particular�percentiles�where� we�denote�P25�as�Q1,�P50�as�Q2,�and�P75�as�Q3��Thus,�the�Qs�represent�the�quartiles�
An�interesting�aspect�of�quartiles�is�that�they�can�be�used�to�determine�whether�a�distri- bution�of�scores�is�positively�or�negatively�skewed��This�is�done�by�comparing�the�values�of� the�quartiles�as�follows��If�(Q3�−�Q2)�>�(Q2�−�Q1),�then�the�distribution�of�scores�is�positively� skewed� as� the� scores� are� more� spread� out� at� the� high� end� of� the� distribution� and� more� bunched�up�at�the�low�end�of�the�distribution�(remember�the�shapes�of�the�distributions� from�Figure�2�5)��If�(Q3�−�Q2)�<�(Q2�−�Q1),�then�the�distribution�of�scores�is�negatively�skewed� as�the�scores�are�more�spread�out�at�the�low�end�of�the�distribution�and�more�bunched�up� at�the�high�end�of�the�distribution��If�(Q3�−�Q2)�=�(Q2�−�Q1),�then�the�distribution�of�scores� is�obviously�not�skewed,�but�is�symmetric�(see�Chapter�4)��For�the�example�statistics�quiz� data,�(Q3�−�Q2)�=�1�4833�and�(Q2�−�Q1)�=�3�4750;�thus,�(Q3�−�Q2)�<�(Q2�−�Q1)�and�we�know� that� the� distribution� is� negatively� skewed�� This� should� already� have� been� evident� from� examining� the� frequency� distribution� in� Figure� 2�3� as� scores� are� more� spread� out� at� the� low�end�of�the�distribution�and�more�bunched�up�at�the�high�end��Examining�the�quartiles� is�a�simple�method�for�getting�a�general�sense�of�the�skewness�of�a�distribution�of�scores�
2.3.3 percentile Ranks
Let�us�define�a�percentile rank�as�the�percentage�of�a�distribution�of�scores�that�falls�below� (or�is�less�than)�a�certain� score��For�instance,�you�may�be�interested� in�the�percentage� of� scores�of�the�GRE-Quantitative�subscale�that�falls�below�the�score�of�480��Say�that�the�per- centile�rank�for�the�score�of�480�is�computed�to�be�50;�then�this�would�mean�that�50%�of� the�scores�fell�below�a�score�of�480��If�this�sounds�familiar,�it�should��The�50th�percentile� was� previously� stated� to� be� 480�� Thus,� we� have� logically� determined� that� the� percentile� rank�of�480�is�50��This�is�because�percentile�and�percentile�rank�are�actually�opposite�sides� of�the�same�coin��Many�are�confused�by�this�and�equate�percentiles�and�percentile�ranks;� however,� they� are� related� but� different� concepts�� Recall� earlier� we� said� that� percentiles� were�scores��Percentile�ranks�are�percentages,�as�they�are�continuous�values�and�can�take� on�any�value�from�0�to�100��The�score�of�400�can�have�a�percentile�rank�of�42�6750��For�nota- tional�purposes,�a�percentile�rank�will�be�known�as�PR(Pi),�where�Pi�is�the�particular�score�
32 An Introduction to Statistical Concepts
whose�percentile�rank,�PR,�you�wish�to�determine��Thus,�the�percentile�rank�of�the�score� 400�would�be�denoted�as�PR(400)�=�42�6750��In�other�words,�about�43%�of�the�distribution� falls�below�the�score�of�400�
Let�us�now�consider�how�percentile�ranks�are�computed��The�formula�for�computing�the� PR(Pi)�percentile�rank�is
� PR P cf
f P LRL w
n i
i
( )
( )
%= +
−
100 � (2�2)
where PR(Pi)�indicates�that�we�are�looking�for�the�percentile�rank�PR�of�the�score�Pi cf� is� the� cumulative� frequency� up� to� but� not� including� the� interval� containing� PR(Pi)�
(again�known�as�cf�below) f�is�the�frequency�of�the�interval�containing�PR(Pi) LRL�is�the�lower�real�limit�of�the�interval�containing�PR(Pi) w�is�the�interval�width n�is�the�sample�size,�and�finally�we�multiply�by�100%�to�place�the�percentile�rank�on�a�
scale�from�0�to�100�(and�also�to�remind�us�that�the�percentile�rank�is�a�percentage)
As�an�example,�consider�computing�the�percentile�rank�for�the�score�of�17��This�would�cor- respond�to�the�percentage�of�the�distribution�that�falls�below�a�score�of�17��For�the�example� data�again,�using�Equation�2�2,�we�compute�PR(17)�as�follows:
�
PR( )
( . )
% .
%17 12
5 17 16 5 1
25 100
12 2 5 25
100 5= +
−
= +
= 88 00. %
Conceptually,�let�us�discuss�how�the�equation�works��First,�we�have�to�determine�what�inter- val�contains�the�percentile�rank�of�interest��This�is�easily�done�because�we�already�know�the� score�is�17�and�we�simply�look�in�the�interval�containing�17��The�cf�below�the�17�interval�is� 12�and�n�is�25��Thus,�we�know�that�we�need�to�go�at�least�12/25,�or�48%,�of�the�way�into�the� distribution�to�obtain�the�desired�percentile�rank��We�know�that�Pi�=�17�and�the�LRL�of�that� interval�is�16�5��There�are�5�frequencies�in�that�interval,�so�we�need�to�go�2�5�scores�into� the�interval�to�obtain�the�proper�percentile�rank��In�other�words,�because�17�is�the�midpoint� of�an�interval�with�width�of�1,�we�need�to�go�halfway�or�2�5/5�of�the�way�into�the�interval� to�obtain�the�percentile�rank��In�the�end,�we�need�to�go�14�5/25�(or��58)�of�the�way�into�the� distribution�to�obtain�our�percentile�rank,�which�translates�to�58%�
As� another� example,� we� have� already� determined� that� P50� =� 16�6000�� Therefore,� you� should�be�able�to�determine�on�your�own�that�PR(16�6000)�=�50%��This�verifies�that�percen- tiles�and�percentile�ranks�are�two�sides�of�the�same�coin��The�computation�of�percentiles� identifies� a� specific� score,� and� you� start� with� the� score� to� determine� the� score’s� percen- tile� rank�� You� can� further� verify� this� by� determining� that� PR(13�1250)� =� 25�00%� and� PR(18�0833)� =� 75�00%�� Next� we� consider� the� box-and-whisker� plot,� where� quartiles� and� percentiles�are�used�graphically�to�depict�a�distribution�of�scores�
33Data Representation
2.3.4 box-and-Whisker plot
A�simplified�form�of�the�frequency�distribution�is�the�box-and-whisker plot�(often�referred� to� simply� as� a� “box� plot”),� developed� by� John� Tukey� (1977)�� This� is� shown� in� Figure� 2�7� (SPSS�generated)�for�the�example�data��The�box-and-whisker�plot�was�originally�developed� to�be�constructed�on�a�typewriter�using�lines�in�a�minimal�amount�of�space��The�box�in� the� center� of� the� figure� displays� the� middle� 50%� of� the� distribution� of� scores�� The� left- hand�edge�or�hinge�of�the�box�represents�the�25th�percentile�(or�Q1)��The�right-hand�edge� or�hinge�of�the�box�represents�the�75th�percentile�(or�Q3)��The�middle�vertical�line�in�the� box�represents�the�50th�percentile�(or�Q2)��The�lines�extending�from�the�box�are�known�as� the�whiskers��The�purpose�of�the�whiskers�is�to�display�data�outside�of�the�middle�50%�� The�left-hand�whisker�can�extend�down�to�the�lowest�score�(as�is�the�case�with�SPSS),�or� to�the�5th�or�the�10th�percentile�(by�other�means),�to�display�more�extreme�low�scores,�and� the�right-hand� whisker� correspondingly� can� extend� up� to�the�highest� score�(SPSS),� or�to� the� 95th� or� 90th� percentile� (elsewhere),� to� display� more� extreme� high� scores�� The� choice� of�where�to�extend�the�whiskers�is�the�preference�of�the�researcher�and/or�the�software�� Scores�that�fall�beyond�the�end�of�the�whiskers,�known�as�outliers�due�to�their�extreme- ness�relative�to�the�bulk�of�the�distribution,�are�often�displayed�by�dots�and/or�asterisks�� Box-and-whisker�plots�can�be�used�to�examine�such�things�as�skewness�(through�the�quar- tiles),�outliers,�and�where�most�of�the�scores�tend�to�fall�
2.4 SPSS
The�purpose�of�this�section�is�to�briefly�consider�applications�of�SPSS�for�the�topics�covered� in�this�chapter�(including�important�screenshots)��The�following�SPSS�procedures�will�be� illustrated:�“Frequencies”�and�“Graphs.”
8 10 12 14 16 18 20
Q ui
z
FIGuRe 2.7 Box-and-whisker�plot�of�statistics�quiz�data�
34 An Introduction to Statistical Concepts
Frequencies
Frequencies: Step 1.�For�the�types�of�tables�discussed�in�this�chapter,�in�SPSS�go�to� “Analyze”� in� the� top� pulldown� menu,� then�“Descriptive Statistics,”� and� then� select� “Frequencies.”� Following� the� screenshot� for� “Frequencies: Step 1”� will� produce�the�“Frequencies”�dialog�box�
�
A
B C
Frequencies: Step 1
Stem and leaf plots (and many other statistics)
can be generated using the
“Explore” program.
Frequencies: Step 2.�The�“Frequencies”�dialog�box�will�open�(see�screenshot�for� “Frequencies: Step 2”)��From�this�main�“Frequencies”�dialog�box,�click�the�vari- able�of�interest�from�the�list�on�the�left�(e�g�,�quiz)�and�move�it�into�the�“Variables”�box� by�clicking�on�the�arrow�button��By�default,�there�is�a�checkmark�in�the�box�for�“Display frequency tables,”�and�we�will�keep�this�checked��This�(i�e�,�selecting�“Display fre- quency tables”)�will�generate�a�table�of�frequencies,�relative�frequencies,�and�cumula- tive�relative�frequencies��There�are�three�buttons�on�the�right�side�of�the�“Frequencies”� dialog�box�(“Statistics,” “Charts,” and “Format”)��Let�us�first�cover�the�options� available�through�“Statistics.”
Select the variable of interest from
the list on the left and use the arrow
to move to the “Variable” box
on the right.
�is is checked by default and will produce a frequency distribution
table in the output.
Clicking on these options will allow
you to select various statistics
and graphs.
Frequencies: Step 2
35Data Representation
Frequencies: Step 3a. If� you� click� on� the� “Statistics”� button� from� the� main� “Frequencies”�dialog�box�(see�“Frequencies: Step 2”),�a�new�box�labeled�“Frequencies: Statistics”�will�appear�(see�screenshot�for�“Frequencies: Step 3a”)��From�here,�you�can� obtain�quartiles�and�selected�percentiles�as�well�as�numerous�other�descriptive�statistics�simply� by�placing�a�checkmark�in�the�boxes�for�the�statistics�that�you�want�to�generate��For�better�accu- racy�when�generating�the�median,�quartiles,�and�percentiles,�check�the�box�for�“Values are group midpoints.”�However,�it�should�be�noted�that�these�values�are�not�always�as�precise� as�those�from�the�formula�given�earlier�in�this�chapter�
Check this for better
accuracy with the median,
quartiles and percentiles.
Options available when clicking on “Statistics” from the main dialog box for Frequencies. Placing a
checkmark will generate the respective statistic in the output.
Frequencies: Step 3a
Frequencies: Step 3b.�If�you�click�on�the�“Charts”�button�from�the�main�“Frequencies”� dialog�box�(see�screenshot�for�“Frequencies: Step 2”),�a�new�box�labeled�“Frequencies: Charts”�will�appear�(see�screenshot�for�“Frequencies: Step 3b”)��From�here,�you�can� select� options� to� generate� bar� graphs,� pie� charts,� or� histograms�� If� you� select� bar� graphs� or� pie� charts,� you� can� plot� either� frequencies� or� percentages� (relative� frequencies)�� Thus,� the� “Frequencies”�program�enables�you�to�do�much�of�what�this�chapter�has�covered��In�addi- tion,� stem-and-leaf� plots� are� available� in� the�“Explore”� program� (see�“Frequencies: Step 1”�for�a�screenshot�on�where�the�“Explore”�program�can�be�accessed)�
Options available when clicking on
“Charts” from the main dialog box for
frequencies.
Frequencies: Step 3b
36 An Introduction to Statistical Concepts
Graphs
There�are�multiple�graphs�that�can�be�generated�in�SPSS��We�will�examine�how�to�generate� histograms,�boxplots,�bar�graphs,�and�more�using�the�“Graphs”�procedure�in�SPSS�
Histograms
Histograms: Step 1.�For�other�ways�to�generate�the�types�of�graphical�displays�covered� in�this�chapter,�go�to�“Graphs”�in�the�top�pulldown� menu��From� there,�select�“Legacy Dialogs,”�then�“Histogram”�(see�screenshot�for�“Graphs: Step 1”)��Another�option� for�creating�a�histogram,�although�not�shown�here,�starts�again�from�the�“Graphs”�option� in� the� top� pulldown� menu,� where� you� select�“Legacy Dialogs,”� then�“Graphboard Template Chooser,”�and�finally�“Histogram.”
Options available when clicking on
“Legacy Dialogs” from the main
pulldown menu for graphs.
Graphs: Step 1
A
B
C
Histograms: Step 2.�This�will�bring�up�the�“Histogram”�dialog�box�(see�screenshot� for�“Histograms: Step 2”)��Click�the�variable�of�interest�(e�g�,�quiz)�and�move�it�into�the� “Variable(s)”�box�by�clicking�on�the�arrow��Place�a�checkmark�in�“Display normal curve,”� and� then� click�“OK.”� This� will� generate� the� same� histogram� as� was� produced� through�the�“Frequencies”�program�already�mentioned�
Histograms: Step 2
37Data Representation
Boxplots
Boxplots: Step 1.�To�produce�a�boxplot�for�individual�variables,�click�on�“Graphs”� in�the�top�pulldown�menu��From�there,�select�“Legacy Dialogs,”�then�“Boxplot”� (see� “GRAPHS: Step 1”� for� screenshot� of� this� step)�� Another� option� for� creating� a� boxplot� (although� not� shown� here)� starts� again� from� the� “Graphs”� option� in� the� top� pulldown� menu,� where� you� select� “Graphboard Template chooser,”� then� “Boxplots.”
Boxplots: Step 2.� This� will� bring� up� the� “Boxplot”� dialog� box� (see� screenshot� for�“Boxplots: Step 2”)��Select�the�“Simple”�option�(by�default,�this�will�already�be� selected)��To�generate�a�separate�boxplot�for�individual�variables,�click�on�the�“Summaries of separate variables”�radio�button��Then�click�“Define.”
Boxplots: Step 2
Boxplots: Step 3.�This�will�bring�up�the�“Define Simple Boxplot: Summaries of Separate Variables”�dialog�box�(see�screenshot�for�“Boxplots: Step 3”)��Click� the�variable�of�interest�(e�g�,�quiz)�into�the�“Variable(s)”�box��Then�click�“OK.”�This�will� generate�a�boxplot�
Boxplots: Step 3
38 An Introduction to Statistical Concepts
Bar Graphs
Bar Graphs: Step 1.� To� produce� a� bar� graph� for� individual� variables,� click� on� “Graphs”�in�the�top�pulldown�menu��From�there,�select�“Legacy Dialogs,”�then�“Bar”� (see�“Graphs: Step 1”�for�screenshot�of�this�step)�
Bar Graphs: Step 2.�From�the�main�“Bar Chart”�dialog�box,�select�“Simple”�(which� will�be�selected�by�default)�and�click�on�the�“Summaries for groups of cases”�radio� button�(see�screenshot�for�“Bar Graphs: Step 2”)�
Bar graphs: Step 2
Bar Graphs: Step 3.�A�new�box�labeled�“Define Simple Bar: Summaries for Groups of Cases”�will�appear��Click�the�variable�of�interest�(e�g�,�eye�color)�and�move� it�into�the�“Variable”�box�by�clicking�the�arrow�button��Then�a�decision�must�be�made� for�how�the�bars�will�be�displayed��Several�types�of�displays�for�bar�graph�data�are�avail- able,�including�“N of cases”�for�frequencies,�“cum. N”�for�cumulative�frequencies,� “% of cases”�for�relative�frequencies,�and�“cum. %”�for�cumulative�relative�frequen- cies�(see�screenshot�for�“Bar Graphs: Step 3”)��Additionally,�other�statistics�can�be� selected�through�the�“Other statistic (e.g., mean)”�option��The�most�common� bar�graph�is�one�which�simply�displays�the�frequencies�(i�e�,�selecting�the�radio�button� for�“N of cases”)��Once�your�selections�are�made,�click�“OK.”�This�will�generate�a� bar�graph�
39Data Representation
When “Other statistic (e.g., mean)” is selected, a dialog box (shown here as “Statistic”) will
appear. All other statistics
that can be represented by the bars in the graph
are listed. Clicking on the radio button will
select the statistic. Once the selection is made, click on
“Continue” to return to the “Define
Simple:Summaries for Groups of Cases” dialog box.
Bar graphs: Step 3
Frequency Polygons
Frequency Polygons: Step 1.� Frequency� polygons� can� be� generated� by� clicking� on�“Graphs”� in� the� top� pulldown� menu�� From� there,� select�“Legacy Dialogs,”� then� “Line”�(see�“Graphs: Step 1”�for�a�screenshot�of�this�step)�
Frequency Polygons: Step 2.�From�the�main�“Line Charts”�dialog�box,�select� “Simple”�(which�will�be�selected�by�default)�and�click�on�the�“Summaries for groups of cases”�(which�will�be�selected�by�default)�radio�button�(see�screenshot�for�“Frequency Polygons: Step 2”)�
40 An Introduction to Statistical Concepts
Frequency polygons: Step 2
Frequency Polygons: Step 3.�A�new�box�labeled�“Define Simple Line: Summaries for Groups of Cases”� will� appear�� Click� the� variable� of� interest� (e�g�,� quiz)�and�move�it�into�the�“Variable”�box�by�clicking�the�arrow�button��Then�a�decision� must�be�made�for�how�the�lines�will�be�displayed��Several�types�of�displays�for�line�graph� (i�e�,� frequency� polygon)� data� are� available,� including� “N of cases”� for� frequencies,� “cum. N”�for�cumulative�frequencies,�“% of cases”�for�relative�frequencies,�and�“cum. %”�for�cumulative�relative�frequencies�(see�screenshot�for�“Frequency Polygons: Step 3”)��Additionally,�other�statistics�can�be�selected�through�the�“Other statistic (e.g., mean)”� option�� The� most� common� frequency� polygon� is� one� which� simply� displays� the� frequencies�(i�e�,�selecting�the�radio�button�for�“N of cases”)��Once�your�selections�are� made,�click�“OK.”�This�will�generate�a�frequency�polygon�
When “Other statistic (e.g., mean)” is selected, a dialog box (shown here
as “Statistic”) will appear.
All other statistics that can be
represented by the bars in the graph
are listed. Clicking on the radio button will
select the statistic. Once the selection is made, click on “Continue” to
return to the “Define Simple: Summaries for
Groups of Cases” dialog box.
Frequency polygons: Step 3
41Data Representation
Editing Graphs
Once�a�graph�or�table�is�created,�double�clicking�on�the�table�or�graph�produced�in�the�out- put�will�allow�the�user�to�make�changes�such�as�changing�the�X�and/or�Y�axis,�colors,�and� more��An�illustration�of�the�options�available�in�chart�editor�is�presented�here�
5
4
3
Fr eq
ue nc
y
2
1
0 9.0 12.0 15.0
Quiz 18.0 21.0
Mean = 15.56 Std. Dev. = 3.163 N = 25
Chart editor
2.5 Templates for Research Questions and APA-Style Paragraph
Depending�on�the�purpose�of�your�research�study,�you�may�or�may�not�write�a�research� question�that�corresponds�to�your�descriptive�statistics��If�the�end�result�of�your�research�paper� is� to� present� results� from� inferential� statistics,� it� may� be� that� your� research� questions� correspond�only�to�those�inferential�questions�and�thus�no�question�is�presented�to�rep- resent�the�descriptive�statistics��That�is�quite�common��On�the�other�hand,�if�the�ultimate� purpose�of�your�research�study�is�purely�descriptive�in�nature,�then�writing�one�or�more� research�questions�that�correspond�to�the�descriptive�statistics�is�not�only�entirely�appro- priate� but� (in� most� cases)� absolutely� necessary�� At� this� time,� let� us� revisit� our� gradu- ate� research� assistant,� Marie,� who� was� introduced� at� the� beginning� of� the� chapter�� As� you� may� recall,� her� task� was� to� summarize� data� from� 25� students� enrolled� in� a� statis- tics�course��The�questions�that�Marie’s�faculty�mentor�shared�with�her�were�as�follows:� How can the quiz scores of students enrolled in an introductory
42 An Introduction to Statistical Concepts
statistics class be graphically represented in a table? In a figure? What is the distributional shape of the statistics quiz score? What is the 50th percentile of the quiz scores?�A�template�for�writing�descriptive� research�questions�for�summarizing�data�may�be�as�follows��Please�note�that�these�are� just�a�few�examples��Given�the�multitude�of�descriptive�statistics�that�can�be�generated,� these�are�not�meant�to�be�exhaustive�
How can [variable] be graphically represented in a table? In a figure? What is the distributional shape of the [variable]? What is the 50th percentile of [variable]?
Next,�we�present�an�APA-like�paragraph�summarizing�the�results�of�the�statistics�quiz�data� example�
As shown in Table 2.2 and Figure 2.2, scores ranged from 9 to 20, with more students achieving a score of 17 than any other score (20%). From Figure 2.2, we also know that the distribution of scores was negatively skewed, with the bulk of the scores being at the high end of the distribution. Skewness was also evident as the quartiles were not equally spaced, as shown in Figure 2.7. Thus, overall the sample of students tended to do rather well on this particular quiz (must have been the awesome teach- ing), although a few low scores should be troubling (as 20% did not pass the quiz and need some remediation).
2.6 Summary
In�this�chapter,�we�considered�both�tabular�and�graphical�methods�for�representing�data�� First,� we� discussed� the� tabular� display� of� distributions� in� terms� of� frequency� distribu- tions� (ungrouped� and� grouped),� cumulative� frequency� distributions,� relative� frequency� distributions,�and�cumulative�relative�frequency�distributions��Next,�we�examined�various� methods�for�depicting�data�graphically,�including�bar�graphs,�histograms�(ungrouped�and� grouped),� frequency� polygons,� cumulative� frequency� polygons,� shapes� of� distributions,� and� stem-and-leaf� displays�� Then,� concepts� and� procedures� related� to� percentiles� were� covered,� including� percentiles,� quartiles,� percentile� ranks,� and� box-and-whisker� plots�� Finally,� an� overview� of� SPSS� for� these� procedures� was� included,� as� well� as� a� summary� APA-style�paragraph�of�the�quiz�dataset��We�include�Box�2�1�as�a�summary�of�which�data� representation� techniques� are� most� appropriate� for� each� type� of� measurement� scale�� At� this�point,�you�should�have�met�the�following�objectives:�(a)�be�able�to�construct�and�inter- pret�statistical�tables,�(b)�be�able�to�construct�and�interpret�statistical�graphs,�and�(c)�be�able� to�determine�and�interpret�percentile-related�information��In�the�next�chapter,�we�address� the�major�population�parameters�and�sample�statistics�useful�for�looking�at�a�single�vari- able��In�particular,�we�are�concerned�with�measures�of�central�tendency�and�measures�of� dispersion�
43Data Representation
STOp aNd ThINk bOx 2.1
Appropriate�Data�Representation�Techniques
Measurement Scale Tables Figures
Nominal •�Frequency�distribution •�Bar�graph •�Relative�frequency�distribution
Ordinal,�interval,�or�ratio •�Frequency�distribution •�Histogram •��Cumulative�frequency�
distribution •�Relative�frequency�distribution •��Cumulative�relative�frequency�
distribution
•�Frequency�polygon •�Relative�frequency�polygon •�Cumulative�frequency�polygon •��Cumulative�relative�frequency�
polygon •�Stem-and-leaf�display �•�Box-and-whisker�plot
Problems
Conceptual problems
2.1� For�a�distribution�where�the�50th�percentile�is�100,�what�is�the�percentile�rank�of�100? � a�� 0 � b�� �50 � c�� 50 � d�� 100 2.2� Which�of�the�following�frequency�distributions�will�generate�the�same�relative�fre-
quency�distribution?
X f Y f Z f
100 2 100 6 100 8 99 5 99 15 99 18 98 8 98 24 98 28 97 5 97 15 97 18 96 2 96 6 96 8
� a�� X�and�Y�only � b�� X�and�Z�only � c�� Y�and�Z�only � d�� X,�Y,�and�Z � e�� None�of�the�above
44 An Introduction to Statistical Concepts
2.3� Which� of� the� following� frequency� distributions� will� generate� the� same� cumulative� relative�frequency�distribution?
X f Y f Z f
100 2 100 6 100 8 99 5 99 15 99 18 98 8 98 24 98 28 97 5 97 15 97 18 96 2 96 6 96 8
� a�� X�and�Y�only � b�� X�and�Z�only � c�� Y�and�Z�only � d�� X,�Y,�and�Z � e�� None�of�the�above 2.4� In�a�histogram,�48%�of�the�area�lies�below�the�score�whose�percentile�rank�is�52��True�
or�false? 2.5� Among�the�following,�the�preferred�method�of�graphing�data�pertaining�to�the�eth-
nicity�of�a�sample�would�be � a�� A�histogram � b�� A�frequency�polygon � c�� A�cumulative�frequency�polygon � d�� A�bar�graph 2.6� The�proportion�of�scores�between�Q1�and�Q3�may�be�less�than��50��True�or�false? 2.7� The�values�of�Q1,�Q2,�and�Q3�in�a�positively�skewed�population�distribution�are�calcu-
lated��What�is�the�expected�relationship�between�(Q2�−�Q1)�and�(Q3�−�Q2)? � a�� (Q2�−�Q1)�is�greater�than�(Q3�−�Q2)� � b�� (Q2�−�Q1)�is�equal�to�(Q3�−�Q2)� � c�� (Q2�−�Q1)�is�less�than�(Q3�−�Q2)� � d�� Cannot�be�determined�without�examining�the�data� 2.8� If�the�percentile�rank�of�a�score�of�72�is�65,�we�may�say�that�35%�of�the�scores�exceed�
72��True�or�false? 2.9� In�a�negatively�skewed�distribution,�the�proportion�of�scores�between�Q1�and�Q2�is�
less�than��25��True�or�false? 2.10� A� group� of� 200� sixth-grade� students� was� given� a� standardized� test� and� obtained�
scores�ranging�from�42�to�88��If�the�scores�tended�to�“bunch�up”�in�the�low�80s,�the� shape�of�the�distribution�would�be�which�one�of�the�following:
� a�� Symmetrical � b�� Positively�skewed � c�� Negatively�skewed � d�� Normal
45Data Representation
2.11� The�preferred�method�of�graphing�data�on�the�eye�color�of�a�sample�is�which�one�of� the�following?
� a�� Bar�graph � b�� Frequency�polygon � c�� Cumulative�frequency�polygon � d�� Relative�frequency�polygon 2.12� If�Q2�=�60,�then�what�is�P50? � a�� 50 � b�� 60 � c�� 95 � d�� Cannot�be�determined�with�the�information�provided 2.13� With�the�same�data�and�using�an�interval�width�of�1,�the�frequency�polygon�and�his-
togram�will�display�the�same�information��True�or�false? 2.14� A�researcher�develops�a�histogram�based�on�an�interval�width�of�2��Can�she�recon-
struct�the�raw�scores�using�only�this�histogram?�Yes�or�no? 2.15� Q2�=�50�for�a�positively�skewed�variable,�and�Q2�=�50�for�a�negatively�skewed�variable��
I�assert�that�Q1�will�not�necessarily�be�the�same�for�both�variables��Am�I�correct?�True� or�false?
2.16� Which�of�the�following�statements�is�correct�for�a�continuous�variable? � a�� The�proportion�of�the�distribution�below�the�25th�percentile�is�75%� � b�� The�proportion�of�the�distribution�below�the�50th�percentile�is�25%� � c�� The�proportion�of�the�distribution�above�the�third�quartile�is�25%� � d�� The�proportion�of�the�distribution�between�the�25th�and�75th�percentiles�is 25%� 2.17� For�a�dataset�with�four�unique�values�(55,�70,�80,�and�90),�the�relative�frequency�for�the�
value�55�is�20%,�the�relative�frequency�for�70�is�30%,�the�relative�frequency�for�80�is�20%,� and�the�relative�frequency�for�90�is�30%��What�is�the�cumulative�relative�frequency�for� the�value�70?
� a�� 20% � b�� 30% � c�� 50% � d�� 100% 2.18� In�examining�data�collected�over�the�past�10�years,�researchers�at�a�theme�park�find�
the�following�for�5000�first-time�guests:�2250�visited�during�the�summer�months;� 675� visited� during� the� fall;� 1300� visited� during� the� winter;� and� 775� visited� dur- ing� the� spring�� What� is� the� relative� frequency� for� guests� who� visited� during� the� spring?
� a�� �135 � b�� �155 � c�� �260 � d�� �450
46 An Introduction to Statistical Concepts
Computational problems
2.1� The�following�scores�were�obtained�from�a�statistics�exam:
47 50 47 49 46 41 47 46 48 44 46 47 45 48 45 46 50 47 43 48 47 45 43 46 47 47 43 46 42 47 49 44 44 50 41 45 47 44 46 45 42 47 44 48 49 43 45 49 49 46
Using�an�interval�size�of�1,�construct�or�compute�each�of�the�following: � a�� Frequency�distribution � b�� Cumulative�frequency�distribution � c�� Relative�frequency�distribution � d�� Cumulative�relative�frequency�distribution � e� Histogram�and�frequency�polygon � f� Cumulative�frequency�polygon � g�� Quartiles � h�� P10�and�P90 � i�� PR(41)�and�PR(49�5) � j�� Box-and-whisker�plot � k�� Stem-and-leaf�display 2.2� The�following�data�were�obtained�from�classroom�observations�and�reflect�the�num-
ber�of�incidences�that�preschool�children�shared�during�an�8�hour�period�
4 8 10 5 12 10 14 5 10 14 12 14 8 5 0 8 12 8 12 5 4 10 8 5
Using�an�interval�size�of�1,�construct�or�compute�each�of�the�following: � a�� Frequency�distribution � b�� Cumulative�frequency�distribution � c�� Relative�frequency�distribution � d�� Cumulative�relative�frequency�distribution � e�� Histogram�and�frequency�polygon � f�� Cumulative�frequency�polygon � g�� Quartiles � h�� P10�and�P90 � i�� PR(10) � j�� Box-and-whisker�plot � k�� Stem-and-leaf�display
47Data Representation
2.3� A�sample�distribution�of�variable�X�is�as�follows:
X f
2 1 3 2 4 5 5 8 6 4 7 3 8 4 9 1
10 2
Calculate�or�draw�each�of�the�following�for�the�sample�distribution�of�X: � a�� Q1 � b�� Q2 � c�� Q3 � d�� P44�5 � e�� PR(7�0) � f�� Box-and-whisker�plot � g�� Histogram�(ungrouped) 2.4� A�sample�distribution�of�classroom�test�scores�is�as�follows:
X f
70 1 75 2 77 3 79 2 80 6 82 5 85 4 90 4 96 3
Calculate�or�draw�each�of�the�following�for�the�sample�distribution�of�X: � a�� Q1 � b�� Q2 � c�� Q3 � d�� P44�5 � e�� PR(82) � f�� Box-and-whisker�plot � g�� Histogram�(ungrouped)
48 An Introduction to Statistical Concepts
Interpretive problems
Select�two�variables�from�the�survey1�dataset�on�the�website,�one�that�is�nominal�and�one� that�is�not�
2.1� �Write� research� questions� that� will� be� answered� from� these� data� using� descriptive� statistics�(you�may�want�to�review�the�research�question�template�in�this�chapter)�
2.2� �Construct�the�relevant�tables�and�figures�to�answer�the�questions�you�posed� 2.3� �Write�a�paragraph�which�summarizes�the�findings�for�each�variable�(you�may�want�
to�review�the�writing�template�in�this�chapter)�
49
3 Univariate Population Parameters and Sample Statistics
Chapter Outline
3�1� Summation�Notation 3�2� Measures�of�Central�Tendency
3�2�1� Mode 3�2�2� Median 3�2�3� Mean 3�2�4� Summary�of�Measures�of�Central�Tendency
3�3� Measures�of�Dispersion 3�3�1� Range 3�3�2� H�Spread 3�3�3� Deviational�Measures 3�3�4� Summary�of�Measures�of�Dispersion
3�4� SPSS 3�5� Templates�for�Research�Questions�and�APA-Style�Paragraph
Key Concepts
� 1�� Summation � 2�� Central�tendency � 3�� Outliers � 4�� Dispersion � 5�� Exclusive�versus�inclusive�range � 6�� Deviation�scores � 7�� Bias
In�the�second�chapter,�we�began�our�discussion�of�descriptive�statistics�previously�defined�as� techniques�which�allow�us�to�tabulate,�summarize,�and�depict�a�collection�of�data�in�an�abbre- viated�fashion��There�we�considered�various�methods�for�representing�data�for�purposes�of� communicating�something�to�the�reader�or�audience��In�particular,�we�were�concerned�with� ways�of�representing�data�in�an�abbreviated�fashion�through�both�tables�and�figures�
50 An Introduction to Statistical Concepts
In� this� chapter,� we� delve� more� into� the� field� of� descriptive� statistics� in� terms� of� three� general� topics�� First,� we� examine� summation� notation,� which� is� important� for� much� of� the� chapter� and,� to� some� extent,� the� remainder� of� the� text�� Second,� measures� of� central� tendency�allow�us�to�boil�down�a�set�of�scores�into�a�single�value,�a�point�estimate,�which� somehow� represents� the� entire� set�� The� most� commonly� used� measures� of� central� ten- dency�are�the�mode,�median,�and�mean��Finally,�measures�of�dispersion�provide�us�with� information�about� the�extent�to� which� the� set�of�scores� varies—in� other�words,� whether� the�scores�are�spread�out�quite�a�bit�or�are�pretty�much�the�same��The�most�commonly�used� measures�of�dispersion�are�the�range�(exclusive�and�inclusive�ranges),�H�spread,�and�vari- ance�and�standard�deviation��In�summary,�concepts�to�be�discussed�in�this�chapter�include� summation,�central�tendency,�and�dispersion��Within�this�discussion,�we�also�address�out- liers�and�bias��Our�objectives�are�that�by�the�end�of�this�chapter,�you�will�be�able�to�do� the�following:�(a)�understand�and�utilize�summation�notation,�(b)�determine�and�interpret� the�three�commonly�used�measures�of�central�tendency,�and�(c)�determine�and�interpret�dif- ferent�measures�of�dispersion�
3.1 Summation Notation
We�were�introduced�to�the�following�research�scenario�in�Chapter�2�and�revisit�Marie�in� this�chapter�
Marie,� a� graduate� student� pursuing� a� master’s� degree� in� educational� research,� has� been� assigned� to� her� first� task� as� a� research� assistant�� Her� faculty� mentor� has� given� Marie�quiz�data�collected�from�25�students�enrolled�in�an�introductory�statistics�course� and� has� asked� Marie� to� summarize� the� data�� The� faculty� member� was� pleased� with� the�descriptive�analysis�and�presentation�of�results�previously�shared,�and�has�asked� Marie�to�conduct�additional�analysis�related�to�the�following�research�questions:�How can quiz scores of students enrolled in an introductory statistics class be summarized using measures of central tendency? Measures of dispersion?
Many� areas� of� statistics,� including� many� methods� of� descriptive� and� inferential� statis- tics,�require�the�use�of�summation�notation��Say�we�have�collected�heart�rate�scores�from� 100�students��Many�statistics�require�us�to�develop�“sums”�or�“totals”�in�different�ways�� For� example,� what� is� the� simple� sum� or� total� of� all� 100� heart� rate� scores?� Summation� (i�e�,�addition)�is�not�only�quite�tedious�to�do�computationally�by�hand,�but�we�also�need� a�system�of�notation�to�communicate�how�we�have�conducted�this�summation�process�� This�section�describes�such�a�notational�system�
For�simplicity,�let�us�utilize�a�small�set�of�scores,�keeping�in�mind�that�this�system�can� be�used�for�a�set�of�numerical�values�of�any�size��In�other�words,�while�we�speak�in�terms� of�“scores,”�this�could�just�as�easily�be�a�set�of�heights,�distances,�ages,�or�other�measures�� Specifically� in� this� example,� we� have� a� set� of� five� ages:� 7,� 11,� 18,� 20,� and� 24�� Recall� from� Chapter�2�the�use�of�X�to�denote�a�variable��Here�we�define�Xi�as�the�score�for�variable�X�(in� this�example,�age)�for�a�particular�individual�or�object�i��The�subscript�i�serves�to�identify� one�individual�or�object�from�another��These�scores�would�then�be�denoted�as�follows:� X1�=�7,�X2�=�11,�X3�=�18,�X4�=�20,�and�X5�=�24��To�interpret�X1�=�7�means�that�for�variable�X� and�individual�1,�the�value�of�the�variable�age�is�7��In�other�words,�individual�1�is�7�years�of�age��
51Univariate Population Parameters and Sample Statistics
With�five�individuals�measured�on�age,�then�i�=�1,�2,�3,�4,�5��However,�with�a�large�set�of� values,�this�notation�can�become�quite�unwieldy,�so�as�shorthand�we�abbreviate�this�as� i�=�1,…,�5,�meaning�that�X�ranges�or�goes�from�i�=�1�to�i�=�5�
Next�we�need�a�system�of�notation�to�denote�the�summation�or�total�of�a�set�of�scores��
The�standard�notation�used�is� Xi i a
b
= ∑ ,�where�Σ�is�the�Greek�capital�letter�sigma�and�merely�
means�“the�sum�of,”�Xi�is�the�variable�we�are�summing�across�for�each�of�the�i�individuals,� i = a�indicates�that�a�is�the�lower�limit�(or�beginning)�of�the�summation�(i�e�,�the�first�value� with� which� we� begin� our� addition),� and� b� indicates� the� upper� limit� (or� end)� of� the� sum- mation�(i�e�,�the�last�value�added)��For�our�example�set�of�ages,�the�sum�of�all�of�the�ages�
would�be�denoted�as� Xi i= ∑
1
5
�in�shorthand�version�and�as� X X X X X Xi i= ∑ = + + + +
1
5
1 2 3 4 5�in�
longhand�version��For�the�example�data,�the�sum�of�all�of�the�ages�is�computed�as�follows:
X X X X X Xi i= ∑ = + + + + = + + + + =
1
5
1 2 3 4 5 7 11 18 20 24 80
Thus,�the�sum�of�the�age�variable�across�all�five�individuals�is�80� For�large�sets�of�values,�the�longhand�version�is�rather�tedious,�and,�thus,�the�shorthand�
version�is�almost�exclusively�used��A�general�form�of�the�longhand�version�is�as�follows:
X X X X Xi i a
b
a a b b
=
+ −∑ = + + + +1 1…
The�ellipse�notation�(i�e�,�…)�indicates�that�there�are�as�many�values�in�between�the�two� values� on� either� side� of� the� ellipse� as� are� necessary�� The� ellipse� notation� is� then� just� shorthand�for�“there�are�some�values�in�between�here�”�The�most�frequently�used�values� for�a�and�b�with�sample�data�are�a�=�1�and�b = n�(as�you�may�recall,�n�is�the�notation�used� to� represent� our� sample� size)�� Thus,� the� most� frequently� used� summation� notation� for�
sample�data�is� Xi i
n
= ∑
1
.
3.2 Measures of Central Tendency
One�method�for�summarizing�a�set�of�scores�is�to�construct�a�single�index�or�value�that�can� somehow�be�used�to�represent�the�entire�collection�of�scores��In�this�section,�we�consider� the�three�most�popular�indices,�known�as�measures of central tendency��Although�other� indices�exist,�the�most�popular�ones�are�the�mode,�the�median,�and�the�mean�
3.2.1 Mode
The� simplest� method� to� use� for� measuring� central� tendency� is� the� mode�� The� mode� is� defined� as� that� value� in� a� distribution� of� scores� that� occurs� most� frequently�� Consider� the�example�frequency�distributions�of�the�number�of�hours�of�TV�watched�per�week,�as�
52 An Introduction to Statistical Concepts
shown� in� Table� 3�1�� In� distribution� (a),� the� mode� is� easy� to� determine,� as� the� interval� for� value�8�contains�the�most�scores,�3�(i�e�,�the�mode�number�of�hours�of�TV�watched�is�8)��In� distribution�(b),�the�mode�is�a�bit�more�complicated�as�two�adjacent�intervals�each�contain� the�most�scores;�that�is,�the�8�and�9�hour�intervals�each�contain�three�scores��Strictly�speak- ing,�this�distribution�is�bimodal,�that�is,�containing�two�modes,�one�at�8�and�one�at�9��This� is� our� personal� preference� for� reporting� this� particular� situation�� However,� because� the� two�modes�are�in�adjacent�intervals,�some�individuals�make�an�arbitrary�decision�to�aver- age�these�intervals�and�report�the�mode�as�8�5�
Distribution�(c)�is�also�bimodal;�however,�here�the�two�modes�at�7�and�11�hours�are�not� in�adjacent�intervals��Thus,�one�cannot�justify�taking�the�average�of�these�intervals,�as�the� average� of�9�hours�[i�e�,�(7�+�11)/2]�is�not�representative�of� the�most�frequently�occurring� score��The�score�of�9�occurs�less�than�any�other�score�observed��We�recommend�reporting� both� modes� here� as� well�� Obviously,� there� are� other� possible� situations� for� the� mode� (e�g�,�trimodal�distribution),�but�these�examples�cover�the�basics��As�one�further�example,� the�example�data�on�the�statistics�quiz�from�Chapter�2�are�shown�in�Table�3�2�and�are�used� to� illustrate� the� methods� in� this� chapter�� The� mode� is� equal� to� 17� because� that� interval� contains�more�scores�(5)�than�any�other�interval��Note�also�that�the�mode�is�determined�in�
Table 3.2
Frequency�Distribution� of Statistics�Quiz�Data
X f cf rf crf
9 1 1 �04 �04 10 1 2 �04 �08 11 2 4 �08 �16 12 1 5 �04 �20 13 2 7 �08 �28
14 1 8 �04 �32 15 3 11 �12 �44 16 1 12 �04 �48 17 5 17 �20 �68 18 3 20 �12 �80 19 4 24 �16 �96 20 1 25 �04 1�00
n�=�25 1�00
Table 3.1
Example�Frequency�Distributions
X f(a) f(b) f(c)
6 1 1 2 7 2 2 3 8 3 3 2 9 2 3 1
10 1 2 2 11 0 1 3 12 0 0 2
53Univariate Population Parameters and Sample Statistics
precisely�the�same�way�whether�we�are�talking�about�the�population�mode�(i�e�,�the�popu- lation�parameter)�or�the�sample�mode�(i�e�,�the�sample�statistic)�
Let�us�turn�to�a�discussion�of�the�general�characteristics�of�the�mode,�as�well�as�whether� a�particular�characteristic�is�an�advantage�or�a�disadvantage�in�a�statistical�sense��The�first� characteristic�of�the�mode�is�it�is�simple�to�obtain��The�mode�is�often�used�as�a�quick-and- dirty� method� for� reporting� central� tendency�� This� is� an� obvious� advantage�� The� second� characteristic�is�the�mode�does�not�always�have�a�unique�value��We�saw�this�in�distribu- tions� (b)� and� (c)� of� Table� 3�1�� This� is� generally� a� disadvantage,� as� we� initially� stated� we� wanted�a�single�index�that�could�be�used�to�represent�the�collection�of�scores��The�mode� cannot�guarantee�a�single�index�
Third,�the�mode�is�not�a�function�of�all�of�the�scores�in�the�distribution,�and�this�is�generally� a�disadvantage��The�mode�is�strictly�determined�by�which�score�or�interval�contains�the�most� frequencies��In�distribution�(a),�as�long�as�the�other�intervals�have�fewer�frequencies�than�the� interval�for�value�8,�then�the�mode�will�always�be�8��That�is,�if�the�interval�for�value�8�contains� three�scores�and�all�of�the�other�intervals�contain�less�than�three�scores,�then�the�mode�will� be�8��The�number�of�frequencies�for�the�remaining�intervals�is�not�relevant�as�long�as�it�is�less� than�3��Also,�the�location�or�value�of�the�other�scores�is�not�taken�into�account�
The�fourth�characteristic�of�the�mode�is�that�it�is�difficult�to�deal�with�mathematically��For� example,�the�mode�is�not�very�stable�from�one�sample�to�another,�especially�with�small�sam- ples��We�could�have�two�nearly�identical�samples�except�for�one�score,�which�can�alter�the� mode��For�example,�in�distribution�(a),�if�a�second�similar�sample�contains�the�same�scores� except�that�an�8�is�replaced�with�a�7,�then�the�mode�is�changed�from�8�to�7��Thus,�changing� a� single� score� can� change� the� mode,� and� this� is� considered� to� be� a� disadvantage�� A� fifth� and�final�characteristic�is�the�mode�can�be�used�with�any�type�of�measurement�scale,�from� nominal�to�ratio,�and�is�the�only�measure�of�central�tendency�appropriate�for�nominal�data�
3.2.2 Median
A�second�measure�of�central�tendency�represents�a�concept�that�you�are�already�familiar� with��The�median�is�that�score�which�divides�a�distribution�of�scores�into�two�equal�parts�� In� other� words,� one-half� of� the� scores� fall� below� the� median,� and� one-half� of� the� scores� fall�above�the�median��We�already�know�this�from�Chapter�2�as�the�50th�percentile�or�Q2�� In� other� words,� the� 50th� percentile,� or� Q2,� represents� the� median� value�� The� formula�for� computing�the�median�is
� Median LRL
n cf f
w= + −50% ( )
� (3�1)
where�the�notation�is�the�same�as�previously�described�in�Chapter�2��Just�as�a�reminder,� LRL� is� the� lower� real� limit� of� the� interval� containing� the� median,� 50%� is� the� percentile� desired,�n�is�the�sample�size,�cf�is�the�cumulative�frequency�of�all�intervals�less�than�but� not�including�the�interval�containing�the�median�(cf�below),�f�is�the�frequency�of�the�interval� containing�the�median,�and�w�is�the�interval�width��For�the�example�quiz�data,�the�median� is�computed�as�follows:
Median = + −
= + =16 5
50 25 12 5
1 16 5 0 1000 16 6000. % ( )
( ) . . .
54 An Introduction to Statistical Concepts
Occasionally,� you� will� run� into� simple� distributions� of� scores� where� the� median� is� easy� to�identify��If�you�have�an�odd�number�of�untied�scores,�then�the�median�is�the�middle- ranked�score��For�an�example,�say�we�have�measured�individuals�on�the�number�of�CDs� owned�and�find�values�of�1,�3,�7,�11,�and�21��For�these�data,�the�median�is�7�(e�g�,�7�CDs�is� the�middle-ranked�value�or�score)��If�you�have�an�even�number�of�untied�scores,�then�the� median�is�the�average�of�the�two�middle-ranked�scores��For�example,�a�different�sample� reveals�the�following�number�of�CDs�owned:�1,�3,�5,�11,�21,�and�32��The�two�middle�scores� are�5�and�11,�and,�thus,�the�median�is�the�average�of�8�CDs�owned�(i�e�,�(5�+�11)/2)��In�most� other� situations� where� there� are� tied� scores,� the� median� is� not� as� simple� to� locate� and� Equation�3�1�is�necessary��Note�also�that�the�median�is�computed�in�precisely�the�same�way� whether�we�are�talking�about�the�population�median�(i�e�,�the�population�parameter)�or�the� sample�median�(i�e�,�the�sample�statistic)�
The�general�characteristics�of�the�median�are�as�follows��First,�the�median�is�not�influenced� by�extreme�scores�(scores�far�away�from�the�middle�of�the�distribution�are�known�as�outliers)�� Because�the�median�is�defined�conceptually�as�the�middle�score,�the�actual�size�of�an�extreme� score�is�not�relevant��For�the�example�statistics�quiz�data,�imagine�that�the�extreme�score�of�9� was�somehow�actually�0�(e�g�,�incorrectly�scored)��The�median�would�still�be�16�6,�as�half�of�the� scores�are�still�above�this�value�and�half�below��Because�the�extreme�score�under�consideration� here�still�remained�below�the�50th�percentile,�the�median�was�not�altered��This�characteristic� is�an�advantage,�particularly�when�extreme�scores�are�observed��As�another�example�using� salary�data,�say�that�all�but�one�of�the�individual�salaries�are�below�$100,000�and�the�median� is�$50,000��The�remaining�extreme�observation�has�a�salary�of�$5,000,000��The�median�is�not� affected�by�this�millionaire—the�extreme�individual�is�simply�treated�as�every�other�observa- tion�above�the�median,�no�more�or�no�less�than,�say,�the�salary�of�$65,000�
A� second� characteristic� is� the� median� is� not� a� function� of� all� of� the� scores�� Because� we� already�know�that�the�median�is�not�influenced�by�extreme�scores,�we�know�that�the�median� does� not� take� such� scores� into� account�� Another� way� to� think� about� this� is� to� examine� Equation�3�1�for�the�median��The�equation�only�deals�with�information�for�the�interval�con- taining�the�median��The�specific�information�for�the�remaining�intervals�is�not�relevant�so� long�as�we�are�looking�in�the�median-contained�interval��We�could,�for�instance,�take�the�top� 25%�of�the�scores�and�make�them�even�more�extreme�(say�we�add�10�bonus�points�to�the�top� quiz�scores)��The�median�would�remain�unchanged��As�you�probably�surmised,�this�charac- teristic�is�generally�thought�to�be�a�disadvantage��If�you�really�think�about�the�first�two�char- acteristics,�no�measure�could�possibly�possess�both��That�is,�if�a�measure�is�a�function�of�all� of�the�scores,�then�extreme�scores�must�also�be�taken�into�account��If�a�measure�does�not�take� extreme�scores�into�account,�like�the�median,�then�it�cannot�be�a�function�of�all�of�the�scores�
A� third� characteristic� is� the� median� is� difficult� to� deal� with� mathematically,� a� disad- vantage� as� with� the� mode�� The� median� is� somewhat� unstable� from� sample� to� sample,� especially�with�small�samples��As�a�fourth�characteristic,�the�median�always�has�a�unique� value,�another�advantage��This�is�unlike�the�mode,�which�does�not�always�have�a�unique� value��Finally,�the�fifth�characteristic�of�the�median�is�that�it�can�be�used�with�all�types�of� measurement�scales�except�the�nominal��Nominal�data�cannot�be�ranked,�and,�thus,�per- centiles�and�the�median�are�inappropriate�
3.2.3 Mean
The� final� measure� of� central� tendency� to� be� considered� is� the� mean,� sometimes� known� as�the�arithmetic�mean�or�“average”�(although�the�term�average�is�used�rather�loosely�by� laypeople)��Statistically,�we�define�the�mean�as�the�sum�of�all�of�the�scores�divided�by�the�
55Univariate Population Parameters and Sample Statistics
number�of�scores��Thought�of�in�those�terms,�you�may�have�been�computing�the�mean�for� many�years,�and�may�not�have�even�known�it�
The�population�mean�is�denoted�by�μ�(Greek�letter�mu)�and�computed�as�follows:
µ = = ∑X N
i
i
N
1
For�sample�data,�the�sample�mean�is�denoted�by�X – �(read�“X�bar”)�and�computed�as�follows:
X
X
n
i
i
n
= = ∑
1
For�the�example�quiz�data,�the�sample�mean�is�computed�as�follows:
X
X
n
i
i
n
= = == ∑
1 389 25
15 5600.
Here�are�the�general�characteristics�of�the�mean��First,�the�mean�is�a�function�of�every�score,� a�definite�advantage�in�terms�of�a�measure�of�central�tendency�representing�all�of�the�data�� If�you�look�at�the�numerator�of�the�mean,�you�see�that�all�of�the�scores�are�clearly�taken�into� account�in�the�sum��The�second�characteristic�of�the�mean�is�that�it�is�influenced�by�extreme� scores��Because�the�numerator�sum�takes�all�of�the�scores�into�account,�it�also�includes�the� extreme�scores,�which�is�a�disadvantage��Let�us�return�for�a�moment�to�a�previous�example� of�salary�data�where�all�but�one�of�the�individuals�have�an�annual�salary�under�$100,000,�and� the�one�outlier�is�making�$5,000,000��Because�this�one�outlying�value�is�so�extreme,�the�mean� will�be�greatly�influenced��In�fact,�the�mean�could�easily�fall�somewhere�between�the�second� highest�salary�and�the�millionaire,�which�does�not�represent�well�the�collection�of�scores�
Third,�the�mean�always�has�a�unique�value,�another�advantage��Fourth,�the�mean�is�easy� to�deal�with�mathematically��The�mean�is�the�most�stable�measure�of�central�tendency�from� sample�to�sample,�and�because�of�that�is�the�measure�most�often�used�in�inferential�statistics� (as�we�show�in�later�chapters)��Finally,�the�fifth�characteristic�of�the�mean�is�that�it�is�only� appropriate�for�interval�and�ratio�measurement�scales��This�is�because�the�mean�implicitly� assumes�equal�intervals,�which�of�course�the�nominal�and�ordinal�scales�do�not�possess�
3.2.4 Summary of Measures of Central Tendency
To�summarize�the�measures�of�central�tendency�then,
� 1�� The�mode�is�the�only�appropriate�measure�for�nominal�data� � 2�� The�median�and�mode�are�both�appropriate�for�ordinal�data�(and�conceptually�the�
median�fits�the�ordinal�scale�as�both�deal�with�ranked�scores)� � 3�� All�three�measures�are�appropriate�for�interval�and�ratio�data�
A�summary�of�the�advantages�and�disadvantages�of�each�measure�is�presented�in�Box�3�1�
56 An Introduction to Statistical Concepts
STOp aNd ThINk bOx 3.1
Advantages�and�Disadvantages�of�Measures�of�Central�Tendency
Measure of Central Tendency Advantages Disadvantages
Mode •��Quick�and�easy�method�for�reporting� central�tendency
•��Can�be�used�with�any�measurement�scale� of variable
•�Does�not�always�have�a�unique�value •��Not�a�function�of�all�scores�in�the�
distribution •��Difficult�to�deal�with�mathematically�
due�to�its�instability
Median •�Not�influenced�by�extreme�scores •�Has�a�unique�value •��Can�be�used�with�ordinal,�interval,�and�
ratio�measurement�scales�of�variables
•��Not�a�function�of�all�scores�in�the� distribution
•��Difficult�to�deal�with�mathematically� due�to�its�instability
•�Cannot�be�used�with�nominal�data
Mean •�Function�of�all�scores�in�the�distribution •�Has�a�unique�value •�Easy�to�deal�with�mathematically •��Can�be�used�with�interval�and�ratio�
measurement�scales�of�variables
•�Influenced�by�extreme�scores •��Cannot�be�used�with�nominal�or�
ordinal�variables
3.3 Measures of Dispersion
In�the�previous�section,�we�discussed�one�method�for�summarizing�a�collection�of�scores,� the�measures�of�central�tendency��Central�tendency�measures�are�useful�for�describing�a� collection�of�scores�in�terms�of�a�single�index�or�value�(with�one�exception:�the�mode�for� distributions�that�are�not�unimodal)��However,�what�do�they�tell�us�about�the�distribution� of�scores?�Consider�the�following�example��If�we�know�that�a�sample�has�a�mean�of�50,�what� do�we�know�about�the�distribution�of�scores?�Can�we�infer�from�the�mean�what�the�distri- bution�looks�like?�Are�most�of�the�scores�fairly�close�to�the�mean�of�50,�or�are�they�spread� out�quite�a�bit?�Perhaps�most�of�the�scores�are�within�two�points�of�the�mean��Perhaps�most� are�within�10�points�of�the�mean��Perhaps�most�are�within�50�points�of�the�mean��Do�we� know?�The�answer,�of�course,�is�that�the�mean�provides�us�with�no�information�about�what� the�distribution�of�scores�looks�like,�and�any�of�the�possibilities�mentioned,�and�many�oth- ers,�can�occur��The�same�goes�if�we�only�know�the�mode�or�the�median�
Another�method�for�summarizing�a�set�of�scores�is�to�construct�an�index�or�value�that� can� be� used� to� describe� the� amount� of� spread� among� the� collection� of� scores�� In� other� words,� we� need� measures� that� can� be� used� to� determine� whether� the� scores� fall� fairly� close� to� the� central� tendency� measure,� are� fairly� well� spread� out,� or� are� somewhere� in� between��In�this�section,�we�consider�the�four�most�popular�such�indices,�which�are�known� as�measures of dispersion�(i�e�,�the�extent�to�which�the�scores�are�dispersed�or�spread�out)�� Although�other�indices�exist,�the�most�popular�ones�are�the�range�(exclusive�and�inclusive),� H�spread,�the�variance,�and�the�standard�deviation�
3.3.1 Range
The�simplest�measure�of�dispersion�is�the�range��The�term�range�is�one�that�is�in�common� use�outside�of�statistical�circles,�so�you�have�some�familiarity�with�it�already��For�instance,�
57Univariate Population Parameters and Sample Statistics
you�are�at�the�mall�shopping�for�a�new�pair�of�shoes��You�find�six�stores�have�the�same�pair� of�shoes�that�you�really�like,�but�the�prices�vary�somewhat��At�this�point,�you�might�actu- ally�make�the�statement�“the�price�for�these�shoes�ranges�from�$59�to�$75�”�In�a�way,�you� are�talking�about�the�range�
Let�us�be�more�specific�as�to�how�the�range�is�measured��In�fact,�there�are�actually�two� different� definitions� of� the� range,� exclusive� and� inclusive,� which� we� consider� now�� The� exclusive range�is�defined�as�the�difference�between�the�largest�and�smallest�scores�in�a� collection� of� scores�� For� notational� purposes,� the� exclusive� range� (ER)� is� shown� as� ER = Xmax�−�Xmin,�where�Xmax�is�the�largest�or�maximum�score�obtained,�and�Xmin�is�the�smallest� or�minimum�score�obtained��For�the�shoe�example�then,�ER = Xmax�−�Xmin�=�75�−�59�=�16��In� other�words,�the�actual�exclusive�range�of�the�scores�is�16�because�the�price�varies�from�59� to�75�(in�dollar�units)�
A�limitation�of�the�exclusive�range�is�that�it�fails�to�account�for�the�width�of�the�intervals� being�used��For�example,�if�we�use�an�interval�width�of�1�dollar,�then�the�59�interval�really� has�59�5�as�the�upper�real�limit�and�58�5�as�the�lower�real�limit��If�the�least�expensive�shoe� is� $58�95,� then� the� exclusive� range� covering� from� $59� to� $75� actually� excludes� the� least� expensive�shoe��Hence�the�term�exclusive range�means�that�scores�can�be�excluded�from� this�range��The�same�would�go�for�a�shoe�priced�at�$75�25,�as�it�would�fall�outside�of�the� exclusive�range�at�the�high�end�of�the�distribution�
Because�of�this�limitation,�a�second�definition�of�the�range�was�developed,�known�as�the� inclusive range��As�you�might�surmise,�the�inclusive�range�takes�into�account�the�interval� width�so�that�all�scores�are�included�in�the�range��The�inclusive�range�is�defined�as�the�differ- ence�between�the�upper�real�limit�of�the�interval�containing�the�largest�score�and�the�lower� real�limit�of�the�interval�containing�the�smallest�score�in�a�collection�of�scores��For�notational� purposes,�the�inclusive�range�(IR)�is�shown�as�IR = URL�of�Xmax�−�LRL�of�Xmin��If�you�think� about�it,�what�we�are�actually�doing�is�extending�the�range�by�one-half�of�an�interval�at�each� extreme,�one-half�an�interval�width�at�the�maximum�value,�and�one-half�an�interval�width�at� the�minimum�value��In�notational�form,�IR = ER + w��For�the�shoe�example,�using�an�interval� width�of�1,�then�IR = URL�of�Xmax�−�LRL�of�Xmin�=�75�5�−�58�5�=�17��In�other�words,�the�actual� inclusive�range�of�the�scores�is�17�(in�dollar�units)��If�the�interval�width�was�instead�2,�then� we�would�add�1�unit�to�each�extreme�rather�than�the��5�unit�that�we�previously�added�to�each� extreme��The�inclusive�range�would�instead�be�18��For�the�example�quiz�data�(presented� in�Table�3�2),�note�that�the�exclusive�range�is�11�and�the�inclusive�range�is�12�(as�interval� width�is�1)�
Finally,�we�need�to�examine�the�general�characteristics�of�the�range�(they�are�the�same� for�both�definitions�of�the�range)��First,�the�range�is�simple�to�compute,�which�is�a�definite� advantage�� One� can� look� at� a� collection� of� data� and� almost� immediately,� even� without� a� computer�or�calculator,�determine�the�range�
The�second�characteristic�is�the�range�is�influenced�by�extreme�scores,�a�disadvantage�� Because� the� range� is� computed� from� the� two� most� extreme� scores,� this� characteristic� is� quite�obvious��This�might�be�a�problem,�for�instance,�if�all�of�the�salary�data�range�from� $10,000�to�$95,000�except�for�one�individual�with�a�salary�of�$5,000,000��Without�this�out- lier,�the�exclusive�range�is�$85,000��With�the�outlier,�the�exclusive�range�is�$4,990,000��Thus,� the�millionaire’s�salary�has�a�drastic�impact�on�the�range�
Third,� the� range� is� only� a� function� of� two� scores,� another� disadvantage�� Obviously,� the� range�is�computed�from�the�largest�and�smallest�scores�and�thus�is�only�a�function�of�those� two�scores��The�spread�of�the�distribution�of�scores�between�those�two�extreme�scores�is�not� at�all�taken�into�account��In�other�words,�for�the�same�maximum�($5,000,000)�and�minimum� ($10,000)�salaries,�the�range�is�the�same�whether�the�salaries�are�mostly�near�the�maximum�
58 An Introduction to Statistical Concepts
salary,�mostly�near�the�minimum�salary,�or�spread�out�evenly��The�fourth�characteristic�is� the� range� is� unstable� from� sample� to� sample,� another� disadvantage�� Say� a� second� sample� of�salary�data�yielded�the�exact�same�data�except�for�the�maximum�salary�now�being�a�less� extreme� $100,000�� The� range� is� now� dramatically� different�� Also,� in� statistics� we� tend� to� worry�about�measures�that�are�not�stable�from�sample�to�sample,�as�that�implies�the�results� are�not�very�reliable��Finally,�the�range�is�appropriate�for�data�that�are�ordinal,�interval,�or� ratio�in�measurement�scale�
3.3.2 H Spread
The�next�measure�of�dispersion�is�H�spread,�a�variation�on�the�range�measure�with�one� major� exception�� Although� the� range� relies� upon� the� two� extreme� scores,� resulting� in� certain� disadvantages,� H� spread� relies� upon� the� difference� between� the� third� and� first� quartiles�� To� be� more� specific,� H� spread� is� defined� as� Q3� −� Q1,� the� simple� difference� between�the�third�and�first�quartiles��The�term�H�spread�was�developed�by�Tukey�(1977),� H�being�short�for�hinge�from�the�box-and-whisker�plot,�and�is�also�known�as�the�inter- quartile�range�
For�the�example�statistics�quiz�data�(presented�in�Table�3�2),�we�already�determined�in� Chapter�2�that�Q3�=�18�0833�and�Q1�=�13�1250��Therefore,�H = Q3�−�Q1�=�18�0833�−�13�1250�=� 4�9583��H�measures�the�range�of�the�middle�50%�of�the�distribution��The�larger�the�value,� the�greater�is�the�spread�in�the�middle�of�the�distribution��The�size�or�magnitude�of�any�of� the� range� measures� takes� on� more� meaning� when� making� comparisons� across� samples�� For�example,�you�might�find�with�salary�data�that�the�range�of�salaries�for�middle�manage- ment�is�smaller�than�the�range�of�salaries�for�upper�management��As�another�example,�we� might�expect�the�salary�range�to�increase�over�time�
What� are� the� characteristics� of� H� spread?� The� first� characteristic� is� H� is� unaffected� by� extreme�scores,�an�advantage��Because�we�are�looking�at�the�difference�between�the�third� and�first�quartiles,�extreme�observations�will�be�outside�of�this�range��Second,�H is�not�a� function�of�every�score,�a�disadvantage��The�precise�placement�of�where�scores�fall�above� Q3,�below�Q1,�and�between�Q3�and�Q1�is�not�relevant��All�that�matters�is�that�25%�of�the� scores�fall�above�Q3,�25%�fall�below�Q1,�and�50%�fall�between�Q3�and Q1��Thus,�H�is�not�a� function�of�very�many�of�the�scores�at�all,�just�those�around�Q3 and Q1��Third,�H�is�not�very� stable�from�sample�to�sample,�another�disadvantage�especially�in�terms�of�inferential�sta- tistics�and�one’s�ability�to�be�confident�about�a�sample�estimate�of�a�population�parameter�� Finally,�H�is�appropriate�for�all�scales�of�measurement�except�for�nominal�
3.3.3 deviational Measures
In�this�section,�we�examine�deviation�scores,�population�variance�and�standard�deviation,� and�sample�variance�and�standard�deviation,�all�methods�that�deal�with�deviations�from� the�mean�
3.3.3.1 Deviation Scores
In� the� last� category� of� measures� of� dispersion� are� those� that� utilize� deviations� from� the� mean��Let�us�define�a�deviation score�as�the�difference�between�a�particular�raw�score�and� the�mean�of�the�collection�of�scores�(population�or�sample,�either�will�work)��For�popula- tion�data,�we�define�a�deviation�as�di�=�Xi�−�μ��In�other�words,�we�can�compute�the�deviation�
59Univariate Population Parameters and Sample Statistics
from�the�mean�for�each�individual�or�object��Consider�the�credit�card�dataset�as�shown�in� Table�3�3��To�make�matters�simple,�we�only�have�a�small�population�of�data,�five�values�to� be�exact��The�first�column�lists�the�raw�scores,�which�are�in�this�example�the�number�of� credit�cards�owned�for�five�individuals�and,�at�the�bottom�of�the�first�column,�indicates�the� sum�(Σ�=�30),�population�size�(N�=�5),�and�population�mean�(μ�=�6�0)��The�second�column� provides�the�deviation�scores�for�each�observation�from�the�population�mean�and,�at�the� bottom�of�the�second�column,�indicates�the�sum�of�the�deviation�scores,�denoted�by
( )Xi i
N
− =
∑ µ 1
From�the�second�column,�we�see�that�two�of�the�observations�have�positive�deviation�scores� as�their�raw�score�is�above�the�mean,�one�observation�has�a�zero�deviation�score�as�that�raw� score�is�at�the�mean,�and�two�other�observations� have�negative�deviation�scores�as�their� raw� score� is� below� the� mean�� However,� when� we� sum� the� deviation� scores,� we� obtain� a� value�of�zero��This�will�always�be�the�case�as�follows:
( )Xi i
N
− = =
∑ µ 0 1
The� positive� deviation� scores� will� exactly� offset� the� negative� deviation� scores�� Thus� any� measure�involving�simple�deviation�scores�will�be�useless�in�that�the�sum�of�the�deviation� scores�will�always�be�zero,�regardless�of�the�spread�of�the�scores�
What�other�alternatives�are�there�for�developing�a�deviational�measure�that�will�yield�a� sum�other�than�zero?�One�alternative�is�to�take�the�absolute�value�of�the�deviation�scores� (i�e�,�where�the�sign�is�ignored)��Unfortunately,�however,�this�is�not�very�useful�mathematically� in� terms� of�deriving� other�statistics,�such� as�inferential� statistics��As�a�result,� this� devia- tional�measure�is�rarely�used�in�statistics�
3.3.3.2 Population Variance and Standard Deviation
So�far,�we�found�the�sum�of�the�deviations�and�the�sum�of�the�absolute�deviations�not�to�be� very�useful�in�describing�the�spread�of�the�scores�from�the�mean��What�other�alternative�
Table 3.3
Credit�Card�Data
X X − μ (X − μ)2
1 −5 25 5 −1 1 6 0 0 8 2 4
10 4 16
=∑ 30 =∑ 0 =∑ 46 N�=�5 μ�=�6
60 An Introduction to Statistical Concepts
might�be�useful?�As�shown�in�the�third�column�of�Table�3�3,�one�could�square�the�devia- tion�scores�to�remove�the�sign�problem��The�sum�of�the�squared�deviations�is�shown�at�the� bottom�of�the�column�as��=�46�and�denoted�as
( )Xi i
N
− =
∑ µ 2 1
As�you�might�suspect,�with�more�scores,�the�sum�of�the�squared�deviations�will�increase�� So�we�have�to�weigh�the�sum�by�the�number�of�observations�in�the�population��This�yields� a�deviational�measure�known�as�the�population variance,�which�is�denoted�as�σ2�(lower- case�Greek�letter�sigma)�and�computed�by�the�following�formula:
σ µ
2
2
1= −
= ∑( )X
N
i
i
N
For�the�credit�card�example,�the�population�variance�σ2�=�(46/5)�=�9�2��We�refer�to�this�par- ticular�formula�for�the�population�variance�as�the�definitional formula,�as�conceptually� that�is�how�we�define�the�variance��Conceptually,�the�variance�is�a�measure�of�the�area�of�a� distribution��That�is,�the�more�spread�out�the�scores,�the�more�area�or�space�the�distribution� takes�up�and�the�larger�is�the�variance��The�variance�may�also�be�thought�of�as�an�average� distance�from�the�mean��The�variance�has�nice�mathematical�properties�and�is�useful�for� deriving�other�statistics,�such�as�inferential�statistics�
The�computational formula�for�the�population�variance�is
σ2
2
1 1
2
2=
−
= = ∑ ∑N X X
N
i
i
N
i
i
N
This�method�is�computationally�easier�to�deal�with�than�the�definitional�formula��Imagine� if�you�had�a�population�of�100�scores��Using�hand�computations,�the�definitional�formula� would�take�considerably�more�time�than�the�computational�formula��With�the�computer,� this�is�a�moot�point,�obviously��But�if�you�do�have�to�compute�the�population�variance�by� hand,�then�the�easiest�formula�to�use�is�the�computational�one�
Exactly� how� does� this� formula� work?� For� the� first� summation� in� the� numerator,� we� square�each�score�first,�then�sum�all�the�squared�scores��This�value�is�then�multiplied�by� the� population� size�� For� the� second� summation� in� the� numerator,� we� sum� all� the� scores� first,�then�square�the�summed�scores��After�subtracting�the�values�computed�in�the�numer- ator,�we�divide�by�the�squared�population�size�
For the first summation in the numerator, we square each score first, then sum across
the squared scores.
For the second summation in the numerator, we sum
across the scores �rst, then square the summed scores.N
2 σ 2 =
Σ N
X2i i=1
Σ N
i=1
2 XiN
61Univariate Population Parameters and Sample Statistics
The�two�quantities�derived�by�the�summation�operations�in�the�numerator�are�computed� in�much�different�ways�and�generally�yield�different�values�
Let� us� return� to� the� credit� card� dataset� and� see� if� the� computational� formula� actually� yields�the�same�value�for�σ2�as�the�definitional�formula�did�earlier�(σ2�=�9�2)��The�computa- tional�formula�shows�σ2�to�be�as�follows:
σ2 ( ) ( )
( ) =
−
= −
= −
== = ∑ ∑N X X
N
i
i
N
i
i
N 2
1 1
2
2
2
2
5 226 30 5
1130 900 25
99 2000.
which�is�precisely�the�value�we�computed�previously� A�few�individuals�(none�of�us,�of�course)�are�a�bit�bothered�about�the�variance�for�the�
following�reason��Say�you�are�measuring�the�height�of�children�in�inches��The�raw�scores� are�measured�in�terms�of�inches,�the�mean�is�measured�in�terms�of�inches,�but�the�vari- ance�is�measured�in�terms�of�inches�squared��Squaring�the�scale�is�bothersome�to�some� as� the� scale� is� no� longer� in� the� original� units� of� measure,� but� rather� a� squared� unit� of� measure—making�interpretation�a�bit�difficult��To�generate�a�deviational�measure�in�the� original�scale�of�inches,�we�can�take�the�square�root�of�the�variance��This�is�known�as�the� standard deviation� and� is� the� final� measure� of� dispersion� we� discuss�� The� population� standard�deviation�is�defined�as�the�positive�square�root�of�the�population�variance�and� is�denoted�by�σ�(i�e�,�σ σ= + 2 )��The�standard�deviation,�then,�is�measured�in�the�original� scale�of�inches��For�the�credit�card�data,�the�standard�deviation�is�computed�as�follows:
σ σ= + = + =2 9 2 3 0332. .
What�are�the�major�characteristics�of�the�population�variance�and�standard�deviation?� First,�the�variance�and�standard�deviation�are�a�function�of�every�score,�an�advantage�� An� examination� of� either� the� definitional� or� computational� formula� for� the� variance� (and�standard�deviation�as�well)�indicates�that�all�of�the�scores�are�taken�into�account,� unlike�the�range�or�H�spread��Second,�therefore,�the�variance�and�standard�deviation�are� affected�by�extreme�scores,�a�disadvantage��As�we�said�earlier,�if�a�measure�takes�all�of� the�scores�into�account,�then�it�must�take�into�account�the�extreme�scores�as�well��Thus,�a� child�much�taller�than�all�of�the�rest�of�the�children�will�dramatically�increase�the�vari- ance,�as�the�area�or�size�of�the�distribution�will�be�much�more�spread�out��Another�way� to�think�about�this�is�the�size�of�the�deviation�score�for�such�an�outlier�will�be�large,�and� then�it�will�be�squared,�and�then�summed�with�the�rest�of�the�deviation�scores��Thus,�an� outlier�can�really�increase�the�variance��Also,�it�goes�without�saying�that�it�is�always�a� good�idea�when�using�the�computer�to�verify�your�data��A�data�entry�error�can�cause�an� outlier�and�therefore�a�larger�variance�(e�g�,�that�child�coded�as�700�inches�tall�instead�of� 70�will�surely�inflate�your�variance)�
Third,� the� variance� and� standard� deviation� are� only� appropriate� for� interval� and� ratio� measurement�scales��Like�the�mean,�this�is�due�to�the�implicit�requirement�of�equal�intervals�� A� fourth� and� final� characteristic� of� the� variance� and� standard� deviation� is� they� are� quite� useful�for�deriving�other�statistics,�particularly�in�inferential�statistics,�another�advantage�� In�fact,�Chapter�9�is�all�about�making�inferences�about�variances,�and�many�other�inferential� statistics�make�assumptions�about�the�variance��Thus,�the�variance�is�quite�important�as�a� measure� of� dispersion�� It� is� also� interesting� to� compare� the� measures� of� central� tendency� with�the�measures�of�dispersion,�as�they�do�share�some�important�characteristics��The�mode�
62 An Introduction to Statistical Concepts
and� the� range� share� certain� characteristics�� Both� only� take� some� of� the� data� into� account,� are�simple�to�compute,�and�are�unstable�from�sample�to�sample��The�median�shares�certain� characteristics�with�H�spread��These�are�not�influenced�by�extreme�scores,�are�not�a�function� of�every�score,�are�difficult�to�deal�with�mathematically�due�to�their�instability�from�sample� to�sample,�and�can�be�used�with�all�measurement�scales�except�the�nominal�scale��The�mean� shares�many�characteristics�with�the�variance�and�standard�deviation��These�all�are�a�func- tion�of�every�score,�are�influenced�by�extreme�scores,�are�useful�for�deriving�other�statistics,� and�are�only�appropriate�for�interval�and�ratio�measurement�scales�
To�complete�this�section�of�the�chapter,�we�take�a�look�at�the�sample�variance�and�stan- dard�deviation�and�how�they�are�computed�for�large�samples�of�data�(i�e�,�larger�than�our� credit�card�dataset)�
3.3.3.3 Sample Variance and Standard Deviation
Most�of�the�time,�we�are�interested�in�computing�the�sample�variance�and�standard�devia- tion;�we�also�often�have�large�samples�of�data�with�multiple�frequencies�for�many�of�the� scores��Here�we�consider�these�last�aspects�of�the�measures�of�dispersion��Recall�when�we� computed� the� sample� statistics� of� central� tendency�� The� computations� were� exactly� the� same� as� with� the� population� parameters� (although� the� notation� for� the� population� and� sample�means�was�different)��There�are�also�no�differences�between�the�sample�and�popu- lation�values�for�the�range,�or�H�spread��However,�there�is�a�difference�between�the�sample� and�population�values�for�the�variance�and�standard�deviation,�as�we�see�next�
Recall�the�definitional�formula�for�the�population�variance�as�follows:
σ µ
2
2
1= −
= ∑( )X
N
i
i
N
Why� not� just� take� this� equation� and� convert� everything� to� sample� statistics?� In� other� words,�we�could�simply�change�N�to�n�and�μ�to�X
– ��What�could�be�wrong�with�that?�The�
answer�is�that�there�is�a�problem�which�prevents�us�from�simply�changing�the�notation�in� the�formula�from�population�notation�to�sample�notation�
Here�is�the�problem��First,�the�sample�mean,�X – ,�may�not�be�exactly�equal�to�the�popu-
lation� mean,� � In� fact,� for� most� samples,� the� sample� mean� will� be� somewhat� different� from� the� population� mean�� Second,� we� cannot� use� the� population� mean� anyway� as� it� is� unknown� (in� most� instances� anyway)�� Instead,� we� have� to� substitute� the� sample� mean� into�the�equation�(i�e�,�the�sample�mean,�X
– ,�is�the�sample�estimate�for�the�population�mean,�μ)��
Because� the� sample� mean� is� different� from� the� population� mean,� the� deviations� will� all� be� affected�� Also,� the� sample� variance� that� would� be� obtained� in� this� fashion� would� be� a� biased� estimate� of� the� population� variance�� In� statistics,� bias� means� that� something� is� systematically� off�� In� this� case,� the� sample� variance� obtained� in� this� manner� would� be� systematically�too�small�
In�order�to�obtain�an�unbiased�sample�estimate�of�the�population�variance,�the�following� adjustments�have�to�be�made�in�the�definitional�and�computational�formulas,�respectively:
s
X X
n
i
i
n
2
2
1
1 =
−
− =
∑( )
63Univariate Population Parameters and Sample Statistics
s
n X X
n n
i
i
n
i
i
n
2
2
1 1
2
1 =
−
− = =
∑ ∑ ( )
In�terms�of�the�notation, s2�is�the�sample�variance n�has�been�substituted�for�N X – �has�been�substituted�for�μ
These�changes�are�relatively�minor�and�expected��The�major�change�is�in�the�denominator,� where�instead�of�N�for�the�definitional�formula�we�have�n −�1,�and�instead�of�N 2�for�the�com- putational�formula�we�have�n(n�−�1)��This�turns�out�to�be�the�correction�that�early�statisticians� discovered�was�necessary�to�obtain�an�unbiased�estimate�of�the�population�variance�
It�should�be�noted�that�(a)�when�sample�size�is�relatively�large�(e�g�,�n�=�1000),�the�correc- tion�will�be�quite�small,�and�(b)�when�sample�size�is�relatively�small�(e�g�,�n�=�5),�the�cor- rection�will�be�quite�a�bit�larger��One�suggestion�is�that�when�computing�the�variance�on�a� calculator�or�computer,�you�might�want�to�be�aware�of�whether�the�sample�or�population� variance�is�being�computed�as�it�can�make�a�difference�(typically�the�sample�variance�is� computed)��The�sample�standard�deviation�is�denoted�by�s�and�computed�as�the�positive� square�root�of�the�sample�variance�s2�(i�e�,�s s= + 2 )�
For�our�example�statistics�quiz�data�(presented�in�Table�3�2),�we�have�multiple�frequen- cies�for�many�of�the�raw�scores�which�need�to�be�taken�into�account��A�simple�procedure� for�dealing�with�this�situation�when�using�hand�computations�is�shown�in�Table�3�4��Here� we�see�that�in�the�third�and�fifth�columns,�the�scores�and�squared�scores�are�multiplied�by� their�respective�frequencies��This�allows�us�to�take�into�account,�for�example,�that�the�score� of�19�occurred�four�times��Note�for�the�fifth�column�that�the�frequencies�are�not�squared;� only�the�scores�are�squared��At�the�bottom�of�the�third�and�fifth�columns�are�the�sums�we� need�to�compute�the�parameters�of�interest�
Table 3.4
Sums�for�Statistics�Quiz�Data
X f fX X2 fX2
9 1 9 81 81 10 1 10 100 100 11 2 22 121 242 12 1 12 144 144 13 2 26 169 338 14 1 14 196 196 15 3 45 225 675 16 1 16 256 256 17 5 85 289 1445 18 3 54 324 972 19 4 76 361 1444 20 1 20 400 400
n�=�25 =∑ 389 =∑ 6293
64 An Introduction to Statistical Concepts
The�computations�are�as�follows��We�compute�the�sample�mean�to�be
X
fX
n
i
i
n
= = == ∑
1 389 25
15 5600.
The�sample�variance�is�computed�to�be�as�follows:
s
n fX fX
n n
i
i
n
i
i
n
2
2
1 1
2
2
1 25 6 293 389
25 24 =
−
− =
−= = ∑ ∑
( ) ( , ) ( )
( ) ==
− = =
157 325 151 321 600
6 004 600
10 0067 , , ,
.
Therefore,�the�sample�standard�deviation�is
s s= + = + =2 10 0067 3 1633. .
3.3.4 Summary of Measures of dispersion
To�summarize�the�measures�of�dispersion�then,
� 1�� The� range� is� the� only� appropriate� measure� for� ordinal� data�� The� H� spread,� vari- ance,� and� standard� deviation� can� be� used� with� interval� or� ratio� measurement� scales�
� 2�� There�are�no�measures�of�dispersion�appropriate�for�nominal�data�
A� summary� of� the� advantages� and� disadvantages� of� each� measure� is� presented� in� Box�3�2�
STOp aNd ThINk bOx 3.2
Advantages�and�Disadvantages�of�Measures�of�Dispersion
Measure of Dispersion Advantages Disadvantages
Range •�Simple�to�compute •��Can�be�used�with�ordinal,�interval,�and�
ratio�measurement�scales�of�variables
•�Influenced�by�extreme�scores •�Function�of�only�two�scores •�Unstable�from�sample�to�sample •�Cannot�be�used�with�nominal�data
H�spread •�Unaffected�by�extreme�scores •��Can�be�used�with�ordinal,�interval,�and�
ratio�measurement�scales�of�variables
•��Not�a�function�of�all�scores�in�the�distribution •��Difficult�to�deal�with�mathematically�due�to�
its�instability •�Cannot�be�used�with�nominal�data
Variance�and� standard� deviation
•�Function�of�all�scores�in�the�distribution •�Useful�for�deriving�other�statistics •��Can�be�used�with�interval�and�ratio�
measurement�scales�of�variables
•�Influenced�by�extreme�scores •��Cannot�be�used�with�nominal�or�ordinal�
variables
65Univariate Population Parameters and Sample Statistics
3.4 SPSS
The�purpose�of�this�section�is�to�see�what�SPSS�has�to�offer�in�terms�of�computing�mea- sures� of� central� tendency� and� dispersion�� In� fact,� SPSS� provides� us� with� many� differ- ent�ways�to�obtain�such�measures��The�three�programs� that�we�have�found�to�be�most� useful� for� generating� descriptive� statistics� covered� in� this� chapter� are� “Explore,” “Descriptives,” and “Frequencies.”�Instructions�for�using�each�are�provided�as� follows�
Explore
Explore: Step 1.� The� first� program,�“Explore,”� can� be� invoked� by� clicking� on� “Analyze”�in�the�top�pulldown�menu,�then�“Descriptive Statistics,”�and�then� “Explore.”� Following� the� screenshot,� as� follows,� will� produce� the�Explore� dialog� box�� For� brevity,� we� have� not� reproduced� this� initial� screenshot� when� we� discuss� the� “Descriptives”� and�“Frequencies”� programs;� however,� you� can� see� here� where� they�can�be�found�from�the�pulldown�menus�
A
B
C
Descriptives and frequencies can also be invoked from this menu.
Explore: Step 1
Explore: Step 2.�Next,�from�the�main�“Explore”�dialog�box,�click�the�variable�of� interest�from�the�list�on�the�left�(e�g�,�quiz),�and�move�it�into�the�“Dependent List”� box�by�clicking�on�the�arrow�button�(see�screenshot�for�“Explore: Step 2”)��Then� click�on�the�“OK”�button�
66 An Introduction to Statistical Concepts
Explore: Step 2
Select the variable of interest from
the list on the left and use the arrow
to move to the “Dependent List”
box on the right.
This� will� automatically� generate� the� mean,� median� (approximate),� variance,� standard� deviation,�minimum,�maximum,�exclusive�range,�and�interquartile�range�(H)�(plus�skew- ness�and�kurtosis�to�be�covered�in�Chapter�4)��The�SPSS�output�from�“Explore”�is�shown� in�the�top�panel�of�Table�3�5�
Table 3.5
Select�SPSS�Output�for�Statistics�Quiz�Data�Using�“Explore,”�“Descriptives,”� and “Frequencies”
Descriptives
Statistic Std. Error
Mean .63267 Lower bound95% Confidence interval
for mean Upper bound 5% Trimmed mean Median Variance Std. deviation Minimum Maximum Range Interquartile range Skewness .464
Quiz
Kurtosis
15.5600 14.2542 16.8658 15.6778 17.0000
10.007 3.16333
9.00 20.00 11.00
5.00 –.598 –.741 .902
Descriptive Statistics
N Range Minimum Maximum Mean Std. Deviation Variance Quiz 25 11.00 9.00 20.00 15.5600 3.16333 10.007 Valid N (listwise) 25
�is is an example of the output generated using the“Descriptives” procedure in SPSS.
This is an example of the output generated using the “Explore” procedure in
SPSS. By default, a stem-and- leaf plot and boxplot are also generated from “Explore” (but are not presented here).
67Univariate Population Parameters and Sample Statistics
Table 3.5 (continued)
Select�SPSS�Output�for�Statistics�Quiz�Data�Using�“Explore,”�“Descriptives,”� and�“Frequencies”
Statistics Quiz
Valid 25N
Missing 0 Mean 15.5600 Median 16.3333a
Mode 17.00 Std. deviation 3.16333 Variance 10.007 Range 11.00 Minimum 9.00 Maximum 20.00
a Calculated from grouped data.
�is is an example of the output generated using the “Frequencies” procedure in
SPSS. By default, a frequency table is generated from “Frequencies”
(but is not presented here).
Descriptives
Descriptives: Step 1.� The� second� program� to� consider� is� “Descriptives.”� It� can� also� be� accessed� by� going� to�“Analyze”� in� the� top� pulldown� menu,� then� selecting� “Descriptive Statistics,”� and� then�“Descriptives”� (see�“Explore: Step 1”� for�a�screenshot�of�this�step)�
Descriptives: Step 2.� This� will� bring� up� the� “Descriptives”� dialog� box� (see� “Descriptives: Step 2”� screenshot)�� From� the� main�“Descriptives”� dialog� box,� click�the�variable�of�interest�(e�g�,�quiz)�and�move�into�the�“Variable(s)”�box�by�clicking� on�the�arrow��Next,�click�on�the�“Options”�button�
Select the variable of interest from the list on the left and use the arrow to
move to the “Variable” box on
the right.
Clicking on “Options” will allow
you to select various statistics to
be generated.
Descriptives: Step 2
Descriptives: Step 3.�A�new�box�called�“Descriptives: Options”�will�appear� (see�“Descriptives: Step 3”�screenshot),�and�you�can�simply�place�a�checkmark�in� the�boxes�for�the�statistics�that�you�want�to�generate��From�here,�you�can�obtain�the�mean,� variance,� standard� deviation,� minimum,� maximum,� and� exclusive� range� (among� oth- ers�available)��The�SPSS�output�from�“Descriptives”�is�shown�in�the�middle�panel�of�
68 An Introduction to Statistical Concepts
Table�3�5��After�making�your�selections,�click�on�“Continue.”�You�will�then�be�returned� to�the�main�“Descriptives”�dialog�box��From�there,�click�“OK.”
Descriptives: Step 3
Statistics available when clicking on “Options”
from the main dialog box for Descriptives.
Placing a checkmark will generate the respective statistic in the output.
Frequencies
Frequencies: Step 1.� The� final� program� to� consider� is� “Frequencies.”� Go� to� “Analyze”� in� the� top� pulldown� menu,� then�“Descriptive Statistics,”� and� then� select�“Frequencies.”�See�“Explore: Step 1”�for�a�screenshot�of�this�step�
Frequencies: Step 2.�The�“Frequencies”�dialog�box�will�open�(see�screenshot�for� “Frequencies: Step 2”)�� From� this� main� “Frequencies”� dialog� box,� click� the� variable� of� interest� from� the� list� on� the� left� (e�g�,� quiz)� and� move� it� into� the�“Variables”� box� by� clicking� on� the� arrow� button�� By� default,� there� is� a� checkmark� in� the� box� for� “Display frequency tables,”�and�we�will�keep�this�checked��This�(i�e�,�selecting�“Display fre- quency tables”)�will�generate�a�table�of�frequencies,�relative�frequencies,�and�cumulative� relative�frequencies��Then�click�on�“Statistics”�located�in�the�top�right�corner�
Select the variable of interest from
the list on the left and use the arrow
to move to the “Variable” box on
the right.
Clicking on “Statistics” will
allow you to select various statistics to
be generated.
Frequencies: Step 2
69Univariate Population Parameters and Sample Statistics
Frequencies: Step 3.�A�new�dialog�box�labeled�“Frequencies: Statistics”�will� appear� (see� screenshot� for�“Frequencies: Step 3”)�� Here� you� can� obtain� the� mean,� median� (approximate),� mode,� variance,� standard� deviation,� minimum,� maximum,� and� exclusive�range�(among�others)��In�order�to�obtain�the�closest�approximation�to�the�median,� check�the�“Values are group midpoints”�box,�as�shown��However,�it�should�be�noted� that�these�values�are�not�always�as�precise�as�those�from�the�formula�given�earlier�in�this� chapter��The�SPSS�output�from�“Frequencies”�is�shown�in�the�bottom�panel�of�Table�3�5�� After� making� your� selections,� click� on� “Continue.”� You� will� then� be� returned� to� the� main�“Frequencies”�dialog�box��From�there,�click�“OK.”
Options available when clicking on “Statistics” from the main dialog
box for Frequencies. Placing a checkmark will generate the
respective statistic in the output.
Check this for better
accuracy with quartiles and percentiles
(e.g., the median).
Frequencies: Step 3
3.5 Templates for Research Questions and APA-Style Paragraph
As�we�stated�in�Chapter�2,�depending�on�the�purpose�of�your�research�study,�you�may� or�may�not�write�a�research�question�that�corresponds�to�your�descriptive�statistics��If� the� end� result� of� your� research� paper� is� to� present� results� from� inferential� statistics,� it�may�be�that�your�research�questions�correspond�only�to�those�inferential�questions� and�thus�no�question�is�presented�to�represent�the�descriptive�statistics��That�is�quite� common��On�the�other�hand,�if�the�ultimate�purpose�of�your�research�study�is�purely� descriptive� in� nature,� then� writing� one� or� more� research� questions� that� correspond� to� the� descriptive� statistics� is� not� only� entirely� appropriate� but� (in� most� cases)� abso- lutely� necessary�� At� this� time,� let� us� revisit� our� graduate� research� assistant,� Marie,� who�was�introduced�at�the�beginning�of�the�chapter��As�you�may�recall,�her�task�was�
70 An Introduction to Statistical Concepts
to�summarize�data�from�25�students�enrolled�in�a�statistics�course��The�questions�that� Marie’s�faculty�mentor�shared�with�her�were�as�follows:�How can quiz scores of students enrolled in an introductory statistics class be summarized using measures of central tendency? Measures of dispersion?�A�tem- plate�for�writing�descriptive�research�questions�for�summarizing�data�with�measures� of�central�tendency�and�dispersion�are�presented�as�follows:
How can [variable] be summarized using measures of central tendency? Measures of dispersion?
Next,�we�present�an�APA-like�paragraph�summarizing�the�results�of�the�statistics�quiz�data� example�answering�the�questions�posed�to�Marie:
As shown in Table 3.5, scores ranged from 9 to 20. The mean was 15.56, the approximate median was 17.00 (or 16.33 when calculated from grouped data), and the mode was 17.00. Thus, the scores tended to lump together at the high end of the scale. A negatively skewed dis- tribution is suggested given that the mean was less than the median and mode. The exclusive range was 11, H spread (interquartile range) was 5.0, variance was 10.007, and standard deviation was 3.1633. From this, we can tell that the scores tended to be quite variable. For example, the middle 50% of the scores had a range of 5 (H spread) indicating that there was a reasonable spread of scores around the median. Thus, despite a high “average” score, there were some low performing students as well. These results are consistent with those described in Section 2.4.
3.6 Summary
In�this�chapter,�we�continued�our�exploration�of�descriptive�statistics�by�considering�some� basic� univariate� population� parameters� and� sample� statistics�� First,� we� examined� sum- mation� notation� which� is� necessary� in� many� areas� of� statistics�� Then� we� looked� at� the� most�commonly�used�measures�of�central�tendency,�the�mode,�the�median,�and�the�mean�� The�next�section�of�the�chapter�dealt�with�the�most�commonly�used�measures�of�disper- sion��Here�we�discussed�the�range�(both�exclusive�and�inclusive�ranges),�H�spread,�and�the� population�variance�and�standard�deviation,�as�well�as�the�sample�variance�and�standard� deviation��We�concluded�the�chapter�with�a�look�at�SPSS,�a�template�for�writing�research� questions�for�summarizing�data�using�measures�of�central�tendency�and�dispersion,�and� then�developed�an�APA-style�paragraph�of�results��At�this�point,�you�should�have�met�the� following�objectives:�(a)�be�able�to�understand�and�utilize�summation�notation,�(b)�be�able� to�determine�and�interpret�the�three�commonly�used�measures�of�central�tendency,�and�(c)�be� able� to� determine� and� interpret� different� measures� of� dispersion�� A� summary� of� when� these�descriptive�statistics�are�most�appropriate�for�each�of�the�scales�of�measurement�is� shown�in�Box�3�3��In�the�next�chapter,�we�will�have�a�more�extended�discussion�of�the�nor- mal�distribution�(previously�introduced�in�Chapter�2),�as�well�as�the�use�of�standard�scores� as�an�alternative�to�raw�scores�
71Univariate Population Parameters and Sample Statistics
STOp aNd ThINk bOx 3.3
Appropriate�Descriptive�Statistics
Measurement Scale Measure of Central Tendency Measure of Dispersion
Nominal •�Mode
Ordinal •�Mode •�Range •�Median •�H�spread
Interval/ratio •�Mode •�Range •�Median •�H�spread •�Mean •�Variance�and�standard�deviation
Problems
Conceptual problems
3.1� �Adding�just�one�or�two�extreme�scores�to�the�low�end�of�a�large�distribution�of�scores� will�have�a�greater�effect�on�which�one�of�the�following?
� a�� Q�than�the�variance� � b�� The�variance�than�Q� � c�� The�mode�than�the�median� � d�� None�of�the�above�will�be�affected� 3.2� The�variance�of�a�distribution�of�scores�is�which�one�of�the�following? � a�� Always�1� � b�� May�be�any�number,�negative,�0,�or�positive� � c�� May�be�any�number�greater�than�0� � d�� May�be�any�number�equal�to�or�greater�than�0� 3.3� �A�20-item�statistics�test�was�graded�using�the�following�procedure:�a�correct�response�
is�scored�+1,�a�blank�response�is�scored�0,�and�an�incorrect�response�is�scored�−1��The� highest�possible�score�is�+20;�the�lowest�score�possible�is�−20��Because�the�variance�of� the�test�scores�for�the�class�was�−3,�we�conclude�which�one�of�the�following?
� a�� The�class�did�very�poorly�on�the�test� � b�� The�test�was�too�difficult�for�the�class� � c�� Some�students�received�negative�scores� � d�� A�computational�error�certainly�was�made� 3.4� �Adding� just� one� or� two� extreme� scores� to� the� high� end� of� a� large� distribution� of�
scores�will�have�a�greater�effect�on�which�one�of�the�following? � a�� The�mode�than�the�median� � b�� The�median�than�the�mode� � c�� The�mean�than�the�median� � d�� None�of�the�above�will�be�affected� 3.5� �In� a� negatively� skewed� distribution,� the� proportion� of� scores� between� Q1� and� the�
median�is�less�than��25��True�or�false?
72 An Introduction to Statistical Concepts
3.6� Median�is�to�ordinal�as�mode�is�to�nominal��True�or�false? 3.7� �I�assert�that�it�is�appropriate�to�utilize�the�mean�in�dealing�with�class�rank�data��Am�
I�correct? 3.8� �For� a� perfectly� symmetrical� distribution� of� data,� the� mean,� median,� and� mode� are�
calculated�� I� assert� that� the� values� of� all� three� measures� are� necessarily� equal�� Am� I correct?
3.9� �In� a� distribution� of� 100� scores,� the� top� 10� examinees� received� an� additional� bonus� of� 5� points�� Compared� to� the� original� median,� I� assert� that� the� median� of� the� new� (revised)�distribution�will�be�the�same�value��Am�I�correct?
3.10� �A�set�of�eight�scores�was�collected,�and�the�variance�was�found�to�be�0��I�assert�that�a� computational�error�must�have�been�made��Am�I�correct?
3.11� �Researcher�A�and�Researcher�B�are�using�the�same�dataset�(n�=�10),�where�Researcher� A� computes� the� sample� variance,� and� Researcher� B� computes� the� population� vari- ance�� The� values� are� found� to� differ� by� more� than� rounding� error�� I� assert� that� a� computational�error�must�have�been�made��Am�I�correct?
3.12� �For� a� set� of� 10� test� scores,� which� of� the� following� values� will� be� different� for� the� sample�statistic�and�population�parameter?
� a�� Mean � b�� H � c�� Range � d�� Variance 3.13� Median�is�to�H�as�mean�is�to�standard�deviation��True�or�false? 3.14� �The� inclusive� range� will� be� greater� than� the� exclusive� range� for� any� data�� True� or�
false? 3.15� �For�a�set�of�IQ�test�scores,�the�median�was�computed�to�be�95�and�Q1�to�be�100��I�assert�
that�the�statistician�is�to�be�commended�for�their�work��Am�I�correct? 3.16� �A� physical� education� teacher� is� conducting� research� related� to� elementary� chil-
dren’s� time� spent� in� physical� activity�� As� part� of� his� research,� he� collects� data� from�schools�related�to�the�number�of�minutes�that�they�require�children�to�par- ticipate� in� physical� education� classes�� She� finds� that� the� most� frequently� occur- ring�number�of�minutes�required�for�children�to�participate�in�physical�education� classes�is�22�00�minutes��Which�measure�of�central�tendency�does�this�statement� represent?
� a�� Mean � b�� Median � c�� Mode � d�� Range � e�� Standard�deviation 3.17� �A�physical�education�teacher�is�conducting�research�related�to�elementary�children’s�
time�spent�in�physical�activity��As�part�of�his�research,�he�collects�data�from�schools� related�to�the�number�of�minutes�that�they�require�children�to�participate�in�physical� education�classes��She�finds�that�the�fewest�number�of�minutes�required�per�week�is�
73Univariate Population Parameters and Sample Statistics
15�minutes�and�the�maximum�number�of�minutes�is�45��Which�measure�of�dispersion� do�these�values�reflect?
� a�� Mean � b�� Median � c�� Mode � d�� Range � e�� Standard�deviation 3.18� �A�physical�education�teacher�is�conducting�research�related�to�elementary�children’s�
time�spent�in�physical�activity��As�part�of�his�research,�he�collects�data�from�schools� related�to�the�number�of�minutes�that�they�require�children�to�participate�in�physical� education�classes��She�finds�that�50%�of�schools�required�20�or�more�minutes�of�par- ticipation�in�physical�education�classes��Which�measure�of�central�tendency�does�this� statement�represent?
� a�� Mean � b�� Median � c�� Mode � d�� Range � e�� Standard�deviation 3.19� �One�item�on�a�survey�of�recent�college�graduates�asks�students�to�indicate�if�they�plan�
to�live�within�a�50�mile�radius�of�the�university��Responses�to�the�question�include� “yes”�or�“no�”�The�researcher�who�gathers�these�data�computes�the�variance�of�this� variable��Is�this�appropriate�given�the�measurement�scale�of�this�variable?
3.20� �A�marriage�and�family�counselor�randomly�samples�250�clients�and�collects�data�on� the�number�of�hours�they�spent�in�counseling�during�the�past�year��What�is�the�most� stable� measure� of� central� tendency� to� compute� given� the� measurement� scale� of� this� variable?
� a�� Mean � b�� Median � c�� Mode � d�� Range � e�� Standard�deviation
Computational problems
3.1� �For�the�population�data�in�Computational�Problem�2�1,�and�again�assuming�an�inter- val�width�of�1,�compute�the�following:
� a�� Mode � b�� Median � c�� Mean � d�� Exclusive�and�inclusive�range � e�� H�spread � f�� Variance�and�standard�deviation
74 An Introduction to Statistical Concepts
3.2� �Given�a�negatively�skewed�distribution�with�a�mean�of�10,�a�variance�of�81,�and�N�=�500,� what�is�the�numerical�value�of�the�following?
( )Xi i
N
− =
∑ µ 1
3.3� �For�the�sample�data�in�Computational�Problem�2�2,�and�again�assuming�an�interval� width�of�1,�compute�the�following:
� a�� Mode � b�� Median � c�� Mean � d�� Exclusive�and�inclusive�range � e�� H�spread � f�� Variance�and�standard�deviation 3.4� �For�the�sample�data�in�Computational�Problem�4�(classroom�test�scores)�of�Chapter�2,�
and�again�assuming�an�interval�width�of�1,�compute�the�following: � a�� Mode � b�� Median � c�� Mean � d�� Exclusive�and�inclusive�range � e�� H�spread � f�� Variance�and�standard�deviation 3.5� A�sample�of�30�test�scores�is�as�follows:
X f
8 1 9 4
10 3 11 7 12 9 13 0 14 0 15 3 16 0 17 0 18 2 19 0 20 1
75Univariate Population Parameters and Sample Statistics
Compute�each�of�the�following�statistics: � a�� Mode � b�� Median � c�� Mean � d�� Exclusive�and�inclusive�range � e�� H�spread � f�� Variance�and�standard�deviation 3.6� �Without�doing�any�computations,�which�of�the�following�distributions�has�the�largest�
variance?
X f Y f Z f
15 6 15 4 15 2 16 7 16 7 16 7 17 9 17 11 17 13 18 9 18 11 18 13 19 7 19 7 19 7 20 6 20 4 20 2
3.7� �Without� doing� any� computations,� which� of� the� following� distributions� has� the� largest�variance?
X f Y f Z f
5 3 5 1 5 6 6 2 6 0 6 2 7 4 7 4 7 3 8 3 8 3 8 1 9 5 9 2 9 0
10 2 10 1 10 7
Interpretive problems
3.1� Select�one�interval�or�ratio�variable�from�the�survey1�sample�dataset�on�the�website� � a�� Calculate�all�of�the�measures�of�central�tendency�and�dispersion�discussed�in�this�
chapter�that�are�appropriate�for�this�measurement�scale� � b�� Write�an�APA-style�paragraph�which�summarizes�the�findings� 3.2� Select�one�ordinal�variable�from�the�survey1�sample�dataset�on�the�website� � a�� Calculate� the� measures� of� central� tendency� and� dispersion� discussed� in� this�
chapter�that�are�appropriate�for�this�measurement�scale� � b�� Write�an�APA-style�paragraph�which�summarizes�the�findings�
77
4 Normal Distribution and Standard Scores
Chapter Outline
4�1� Normal�Distribution 4�1�1� History 4�1�2� Characteristics
4�2� Standard�Scores 4�2�1� z�Scores 4�2�2� Other�Types�of�Standard�Scores
4�3� Skewness�and�Kurtosis�Statistics 4�3�1� Symmetry 4�3�2� Skewness 4�3�3� Kurtosis
4�4� SPSS 4�5� Templates�for�Research�Questions�and�APA-Style�Paragraph
Key Concepts
� 1�� Normal�distribution�(family�of�distributions,�unit�normal�distribution,�area�under� the�curve,�points�of�inflection,�asymptotic�curve)
� 2�� Standard�scores�[z,�College�Entrance�Examination�Board�(CEEB),�T,�IQ] � 3�� Symmetry � 4�� Skewness�(positively�skewed,�negatively�skewed) � 5�� Kurtosis�(leptokurtic,�platykurtic,�mesokurtic) � 6�� Moments�around�the�mean
In�Chapter�3,�we�continued�our�discussion�of�descriptive�statistics,�previously�defined� as�techniques�that�allow�us�to�tabulate,�summarize,�and�depict�a�collection�of�data�in� an� abbreviated� fashion�� There� we� considered� the� following� three� topics:� summation� notation� (method� for� summing� a� set� of� scores),� measures� of� central� tendency� (mea- sures� for� boiling� down� a� set� of� scores� into� a� single� value� used� to� represent� the� data),� and�measures�of�dispersion�(measures�dealing�with�the�extent�to�which�a�collection�of� scores�vary)�
78 An Introduction to Statistical Concepts
In�this�chapter,�we�delve�more�into�the�field�of�descriptive�statistics�in�terms�of�three�addi- tional�topics��First,�we�consider�the�most�commonly�used�distributional�shape,�the�normal� distribution��Although�in�this�chapter�we�discuss�the�major�characteristics�of�the�normal�dis- tribution�and�how�it�is�used�descriptively,�in�later�chapters�we�see�how�the�normal�distribu- tion�is�used�inferentially�as�an�assumption�for�certain�statistical�tests��Second,�several�types� of�standard�scores�are�considered��To�this�point,�we�have�looked�at�raw�scores�and�deviation� scores��Here�we�consider�scores�that�are�often�easier�to�interpret,�known�as�standard�scores�� Then� we� examine� two� other� measures� useful� for� describing� a� collection� of� data,� namely,� skewness�and�kurtosis��As�we�show�shortly,�skewness�refers�to�the�lack�of�symmetry�of�a�dis- tribution�of�scores,�and�kurtosis�refers�to�the�peakedness�of�a�distribution�of�scores��Finally,� we� provide� a� template� for� writing� research� questions,� develop� an� APA-style� paragraph� of� results�for�an�example�dataset,�and�also�illustrate�the�use�of�SPSS��Concepts�to�be�discussed� include�the�normal�distribution�(i�e�,�family�of�distributions,�unit�normal�distribution,�area� under�the�curve,�points�of�inflection,�asymptotic�curve),�standard�scores�(e�g�,�z,�CEEB,�T,�IQ),� symmetry,�skewness�(positively�skewed,�negatively�skewed),�kurtosis�(leptokurtic,�platykur- tic,�mesokurtic),�and�moments�around�the�mean��Our�objectives�are�that�by�the�end�of�this� chapter,�you�will�be�able�to�(a)�understand�the�normal�distribution�and�utilize�the�normal� table,� (b)� determine� and� interpret� different� types� of� standard� scores,� particularly� z� scores,� and�(c)�understand�and�interpret�skewness�and�kurtosis�statistics�
4.1 Normal Distribution
You�may�remember�the�following�research�scenario�that�was�first�introduced�in�Chapter�2�� We�will�revisit�Marie�in�this�chapter�
Marie,�a�graduate�student�pursuing�a�master’s�degree�in�educational�research,�has�been� assigned�to�her�first�task�as�a�research�assistant��Her�faculty�mentor�has�given�Marie� quiz�data�collected�from�25�students�enrolled�in�an�introductory�statistics�course�and� has� asked� Marie� to� summarize� the� data�� The� faculty� member,� who� continues� to� be� pleased� with� the� descriptive� analysis� and� presentation� of� results� previously� shared,� has� asked� Marie� to� revisit� the� following� research� question� related� to� distributional� shape:� What is the distributional shape of the statistics quiz score?� Additionally,� Marie’s� faculty� mentor� has� asked� Marie� to� standardize� the� quiz� score� and� compare� student� 1� to�student�3�relative�to�the�mean��The�corresponding�research�question�that�Marie�is� provided� for� this� analysis� is� as� follows:� In standard deviation units, what is the relative standing to the mean of student 1 compared to student 3?
Recall�from�Chapter�2�that�there�are�several�commonly�seen�distributions��The�most�com- monly�observed�and�used�distribution�is�the�normal�distribution��It�has�many�uses�both�in� descriptive�and�inferential�statistics,�as�we�show��In�this�section,�we�discuss�the�history�of� the�normal�distribution�and�the�major�characteristics�of�the�normal�distribution�
4.1.1 history
Let�us�first�consider�a�brief�history�of�the�normal�distribution��From�the�time�that�data�were� collected�and�distributions�examined,�a�particular�bell-shaped�distribution�occurred�quite� often�for�many�variables�in�many�disciplines�(e�g�,�many�physical,�cognitive,�physiological,�
79Normal Distribution and Standard Scores
and� motor� attributes)�� This� has� come� to� be� known� as� the� normal distribution�� Back� in� the� 1700s,� mathematicians� were� called� on� to� develop� an� equation� that� could� be� used� to� approximate�the�normal�distribution��If�such�an�equation�could�be�found,�then�the�prob- ability� associated� with� any� point� on� the� curve� could� be� determined,� and� the� amount� of� space�or�area�under�any�portion�of�the�curve�could�also�be�determined��For�example,�one� might� want� to� know� what� the� probability� of� being� taller� than� 6′2″� would� be� for� a� male,� given� that� height� is� normally� shaped� for� each� gender�� Until� the� 1920s,� the� development� of� this� equation� was� commonly� attributed� to� Karl� Friedrich� Gauss�� Until� that� time,� this� distribution�was�known�as�the�Gaussian�curve��However,�in�the�1920s,�Karl�Pearson�found� this�equation�in�an�earlier�article�written�by�Abraham�DeMoivre�in�1733�and�renamed�the� curve�as�the�normal�distribution��Today�the�normal�distribution�is�obviously�attributed�to� DeMoivre�
4.1.2 Characteristics
There�are�seven�important�characteristics�of�the�normal�distribution��Because�the�nor- mal�distribution�occurs�frequently,�features�of�the�distribution�are�standard�across�all� normal� distributions�� This� “standard� curve”� allows� us� to� make� comparisons� across� two�or�more�normal�distributions�as�well�as�look�at�areas�under�the�curve,�as�becomes� evident�
4.1.2.1 Standard Curve
First,�the�normal�distribution�is�a�standard�curve�because�it�is�always�(a)�symmetric�around� the�mean,�(b)�unimodal,�and�(c)�bell-shaped��As�shown�in�Figure�4�1,�if�we�split�the�distri- bution�in�one-half�at�the�mean�(μ),�the�left-hand�half�(below�the�mean)�is�the�mirror�image� of�the�right-hand�half�(above�the�mean)��Also,�the�normal�distribution�has�only�one�mode,� and�the�general�shape�of�the�distribution�is�bell-shaped�(some�even�call�it�the�bell-shaped� curve)��Given�these�conditions,�the�mean,�median,�and�mode�will�always�be�equal�to�one� another�for�any�normal�distribution�
–3 –2 –1 1 Mean
2 3
13.59%13.59%
34.13% 34.13%
2.14% 2.14%
FIGuRe 4.1 The�normal�distribution�
80 An Introduction to Statistical Concepts
4.1.2.2 Family of Curves
Second,�there�is�no�single�normal�distribution,�but�rather�the�normal�distribution�is�a�fam- ily�of�curves��For�instance,�one�particular�normal�curve�has�a�mean�of�100�and�a�vari- ance�of�225�(recall�that�the�standard�deviation�is�the�square�root�of�the�variance;�thus,� the�standard�deviation�in�this�instance�is�15)��This�normal�curve�is�exemplified�by�the� Wechsler� intelligence� scales�� Another� specific� normal� curve� has� a� mean� of� 50� and� a� variance�of�100�(standard�deviation�of�10)��This�normal�curve�is�used�with�most�behav- ior�rating�scales��In�fact,�there�are�an�infinite�number�of�normal�curves,�one�for�every� distinct�pair�of�values�for�the�mean�and�variance��Every�member�of�the�family�of�nor- mal� curves� has� the� same� characteristics;� however,� the� scale� of� X,� the� mean� of� X,� and� the�variance�(and�standard�deviation)�of�X�can�differ�across�different�variables�and/or� populations�
To� keep� the� members� of� the� family� distinct,� we� use� the� following� notation�� If� the� variable�X�is�normally�distributed,�we�write�X ∼ N(μ,�σ2)��This�is�read�as�“X�is�distrib- uted�normally�with�population�mean�μ�and�population�variance�σ2�”�This�is�the�general� notation;�for�notation�specific�to�a�particular�normal�distribution,�the�mean�and�vari- ance�values�are�given��For�our�examples,�the�Wechsler�intelligence�scales�are�denoted� by�X ∼ N(100,�225),�whereas�the�behavior�rating�scales�are�denoted�by�X ∼ N(50,�100)�� Narratively�speaking�therefore,�the�Wechsler�intelligence�scale�is�distributed�normally� with�a�population�mean�of�100�and�population�variance�of�225��A�similar�interpretation� can�be�made�on�the�behavior�rating�scale�
4.1.2.3 Unit Normal Distribution
Third,�there�is�one�particular�member�of�the�family�of�normal�curves�that�deserves�addi- tional�attention��This�member�has�a�mean�of�0�and�a�variance�(and�standard�deviation)�of�1� and�thus�is�denoted�by�X ∼ N(0,�1)��This�is�known�as�the�unit normal distribution�(unit� referring�to�the�variance�of�1)�or�as�the�standard unit normal distribution��On�a�related� matter,�let�us�define�a�z�score�as�follows:
z X
i i= −( )µ
σ
The� numerator� of� this� equation� is� actually� a� deviation� score,� previously� described� in� Chapter� 3,� and� indicates� how� far� above� or� below� the� mean� an� individual’s� score� falls�� When�we�divide�the�deviation�from�the�mean�(i�e�,�the�numerator)�by�the�standard�devia- tion�(i�e�,�denominator),�the�value�derived�indicates�how�many�deviations�above�or�below�the� mean�an�individual’s�score�falls��If�one�individual�has�a�z�score�of�+1�00,�then�the�person� falls�one�standard�deviation�above�the�mean��If�another�individual�has�a�z�score�of�−2�00,� then�that�person�falls�two�standard�deviations�below�the�mean��There�is�more�to�say�about�this� as�we�move�along�in�this�section�
4.1.2.4 Area
The� fourth� characteristic� of� the� normal� distribution� is� the� ability� to� determine� any� area� under�the�curve��Specifically,�we�can�determine�the�area�above�any�value,�the�area�below� any�value,�or�the�area�between�any�two�values�under�the�curve��Let�us�chat�about�what�we� mean�by�area��If�you�return�to�Figure�4�1,�areas�for�different�portions�of�the�curve�are�listed��
81Normal Distribution and Standard Scores
Here�area�is�defined�as�the�percentage�or�amount�of�space�of�a�distribution,�either�above� a� certain� score,� below� a� certain� score,� or� between� two� different� scores�� For� example,� we� see�that�the�area�between�the�mean�and�one�standard�deviation�above�the�mean�is�34�13%�� In�other�words,�roughly�a�third�of�the�entire�distribution�falls�into�that�region��The�entire� area� under� the� curve� then� represents� 100%,� and� smaller� portions� of� the� curve� represent� somewhat�less�than�that�
For�example,�say�you�wanted�to�know�what�percentage�of�adults�had�an�IQ�score�over�120,� what�percentage�of�adults�had�an�IQ�score�under�107,�or�what�percentage�of�adults�had�an�IQ� score�between�107�and�120��How�can�we�compute�these�areas�under�the�curve?�A�table�of�the� unit�normal�distribution�has�been�developed�for�this�purpose��Although�similar�tables�could� also�be�developed�for�every�member�of�the�normal�family�of�curves,�these�are�unnecessary,� as�any�normal�distribution�can�be�converted�to�a�unit�normal�distribution��The�unit�normal� table�is�given�in�Table�A�1�
Turn�to�Table�A�1�now�and�familiarize�yourself�with�its�contents��To�help�illustrate,�a� portion�of�the�table�is�presented�in�Figure�4�2��The�first�column�simply�lists�the�values� of�z��These�are�standardized�scores�on�the�X�axis��Note�that�the�values�of�z�only�range� from�0�to�4�0��There�are�two�reasons�for�this��First,�values�above�4�0�are�rather�unlikely,� as�the�area�under�that�portion�of�the�curve�is�negligible�(less�than��003%)��Second,�val- ues�below�0�(i�e�,�negative�z�scores)�are�not�really�necessary�to�present�in�the�table,�as�the� normal�distribution�is�symmetric�around�the�mean�of�0��Thus,�that�portion�of�the�table� would�be�redundant�and�is�not�shown�here�(we�show�how�to�deal�with�this�situation�for� some�example�problems�in�a�bit)�
The� second� column,� labeled� P(z),� gives� the� area� below� the� respective� value� of� z—in� other�words,�the�area�between�that�value�of�z�and�the�most�extreme�left-hand�portion�of� the�curve�[i�e�,�−∞�(negative�infinity)�on�the�far�negative�or�left-hand�side�of�0]��So�if�we� wanted�to�know�what�the�area�was�below�z�=�+1�00,�we�would�look�in�the�first�column� under�z�=�1�00�and�then�look�in�the�second�column�(P(z))�to�find�the�area�of��8413��This� value,��8413,�represents�the�percentage�of�the�distribution�that�is�smaller�than�z�of�+1�00��It� also�represents�the�probability�that�a�score�will�be�smaller�than�z�of�+1�00��In�other�words,� about�84%�of�the�distribution�is�less�than�z�of�+1�00,�and�the�probability�that�a�value�will� be�less�than�z�of�+1�00�is�about�84%��More�examples�are�considered�later�in�this�section�
z scores are standardized scores on the X axis.
.5000000
.5039694
.5079783
.5119665
.5159534
.5199388
.6914625
.6949743
.6984682
.7019440
.7054015
.7088403
.8414625
.8437524
.8461358
.8484950
.8508300
.8531409
.9331928
.9344783
.9357445
.9369916
.9382198
.9394292
.00
.01
.02
.03
.04
.04
.50
.51
.52
.53
.54
.55
1.00 1.01 1.02 1.03 1.04 1.05
1.50 1.51 1.52 1.53 1.54 1.55
P(z) P(z)P(z)P(z) zzzz
P(z) values indicate the percentage of the z distribution that is smaller than the respective z value and it also represents the probability that a value will be less than that respective z value.
FIGuRe 4.2 Portion�of�z�table�
82 An Introduction to Statistical Concepts
4.1.2.5 Transformation to Unit Normal Distribution
A� fifth� characteristic� is� any� normally� distributed� variable,� regardless� of� the� mean� and� variance,�can�be�converted�into�a�unit�normally�distributed�variable��Thus,�our�Wechsler� intelligence� scales� as� denoted� by� X ∼ N(100,� 225)� can� be� converted� into� z ∼ N(0,� 1)�� Conceptually,�this�transformation�is�done�by�moving�the�curve�along�the�X�axis�until�it� is�centered�at�a�mean�of�0�(by�subtracting�out�the�original�mean)�and�then�by�stretching� or� compressing� the� distribution� until� it� has� a� variance� of� 1� (remember,� however,� that� the�shape�of�the�distribution�does�not�change�during�the�standardization�process—only� those� values� on� the� X� axis)�� This� allows� us� to� make� the� same� interpretation� about� any� individual’s� score� on� any� normally� distributed� variable�� If� z� =� +1�00,� then� for� any� vari- able,�this�implies�that�the�individual�falls�one�standard�deviation�above�the�mean�
This� also� allows� us� to� make� comparisons� between� two� different� individuals� or� across� two� different� variables�� If� we� wanted� to� make� comparisons� between� two� different� indi- viduals�on�the�same�variable�X,�then�rather�than�comparing�their�individual�raw�scores,� X1�and�X2,�we�could�compare�their�individual�z�scores,�z1�and�z2,�where
z X
1 1=
−( )µ σ
and
z X
2 2=
−( )µ σ
This� is� the� reason� we� only� need� the� unit� normal� distribution� table� to� determine� areas� under� the� curve� rather� than� a� table� for� every� member� of� the� normal� distribution� fam- ily�� In� another� situation,� we� may� want� to� compare� scores� on� the� Wechsler� intelligence� scales�[X ∼ N(100,�225)]�to�scores�on�behavior�rating�scales�[X ∼ N(50,�100)]�for�the�same� individual��We�would�convert�to�z�scores�again�for�two�variables,�and�then�direct�com- parisons�could�be�made�
It�is�important�to�note�that�in�standardizing�a�variable,�it�is�only�the�values�on�the�X�axis� that�change��The�shape�of�the�distribution�(e�g�,�skewness�and�kurtosis)�remains�the�same�
4.1.2.6 Constant Relationship with Standard Deviation
The� sixth� characteristic� is� that� the� normal� distribution� has� a� constant� relationship� with� the�standard�deviation��Consider�Figure�4�1�again��Along�the�X�axis,�we�see�values�repre- sented�in�standard�deviation�increments��In�particular,�from�left�to�right,�the�values�shown� are�three,�two,�and�one�standard�deviation�units�below�the�mean�and�one,�two,�and�three� standard�deviation�units�above�the�mean��Under�the�curve,�we�see�the�percentage�of�scores� that�are�under�different�portions�of�the�curve��For�example,�the�area�between�the�mean�and� one�standard�deviation�above�or�below�the�mean�is�34�13%��The�area�between�one�standard� deviation�and�two�standard�deviations�on�the�same�side�of�the�mean�is�13�59%,�the�area� between�two�and�three�standard�deviations�on�the�same�side�is�2�14%,�and�the�area�beyond� three�standard�deviations�is��13%�
In�addition,�three�other�areas�are�often�of�interest��The�area�within�one�standard�devi- ation�of�the�mean,�from�one�standard�deviation�below�the�mean�to�one�standard�devia- tion�above�the�mean,�is�approximately�68%�(or�roughly�two-thirds�of�the�distribution)�� The� area� within� two� standard� deviations� of� the� mean,� from� two� standard� deviations�
83Normal Distribution and Standard Scores
below� the� mean� to� two� standard� deviations� above� the� mean,� is� approximately� 95%�� The�area�within�three�standard�deviations�of�the�mean,�from�three�standard�deviations� below�the�mean�to�three�standard�deviations�above�the�mean,�is�approximately�99%��In� other�words,�nearly�all�of�the�scores�will�be�within�two�or�three�standard�deviations�of� the�mean�for�any�normal�curve�
4.1.2.7 Points of Inflection and Asymptotic Curve
The� seventh� and� final� characteristic� of� the� normal� distribution� is� as� follows�� The� points of inflection� are� where� the� curve� changes� from� sloping� down� (concave)� to� sloping� up� (convex)��These�points�occur�precisely�at�one�standard�deviation�unit�above�and�below�the� mean��This�is�more�a�matter�of�mathematical�elegance� than�a�statistical�application��The� curve�also�never�touches�the�X�axis��This�is�because�with�the�theoretical�normal�curve,�all� values�from�negative�infinity�to�positive�infinity�have�a�nonzero�probability�of�occurring�� Thus,� while� the� curve� continues� to� slope� ever-downward� toward� more� extreme� scores,� it�approaches,�but�never�quite�touches,�the�X�axis��The�curve�is�referred�to�here�as�being� asymptotic��This�allows�for�the�possibility�of�extreme�scores�
Examples:�Now�for�the�long-awaited�examples�for�finding�area�using�the�unit�normal�dis- tribution��These�examples�require�the�use�of�Table�A�1��Our�personal�preference�is�to�draw� a�picture�of�the�normal�curve�so�that�the�proper�area�is�determined��Let�us�consider�four� examples�of�finding�the�area�below�a�certain�value�of�z:�(1)�below�z�=�−2�50,�(2)�below�z�=�0,� (3)�below�z�=�1�00,�and�(4)�between�z�=�−2�50�and�z�=�1�00�
To�determine�the�value�below�z�=�−2�50,�we�draw�a�picture�as�shown�in�Figure�4�3a��We� draw�a�vertical�line�at�the�value�of�z,�then�shade�in�the�area�we�want�to�find��Because�the� shaded�region�is�relatively�small,�we�know�the�area�must�be�considerably�smaller�than��50�� In�the�unit�normal�table,�we�already�know�negative�values�of�z�are�not�included��However,� because�the�normal�distribution�is�symmetric,�we�know�the�area�below�−2�50�is�the�same�as� the�area�above�+2�50��Thus,�we�look�up�the�area�below�+2�50�and�find�the�value�of��9938��We� subtract�this�from�1�0000�and�find�the�value�of��0062,�or��62%,�a�very�small�area�indeed�
How�do�we�determine�the�area�below�z�=�0�(i�e�,�the�mean)?�As�shown�in�Figure�4�3b,�we� already�know�from�reading�this�section�that�the�area�has�to�be��5000�or�one-half�of�the�total� area�under�the�curve��However,�let�us�look�in�the�table�again�for�area�below�z�=�0,�and�we� find�the�area�is��5000��How�do�we�determine�the�area�below�z�=�1�00?�As�shown�in�Figure� 4�3c,�this�region�exists�on�both�sides�of�0�and�actually�constitutes�two�smaller�areas,�the�first� area�below�0�and�the�second�area�between�0�and�1��For�this�example,�we�use�the�table�directly� and�find�the�value�of��8413��We�leave�you�with�two�other�problems�to�solve�on�your�own�� First,�what�is�the�area�below�z�=��50�(answer:��6915)?�Second,�what�is�the�area�below�z�=�1�96� (answer:��9750)?
Because�the�unit�normal�distribution�is�symmetric,�finding�the�area�above�a�certain�value� of�z�is�solved�in�a�similar�fashion�as�the�area�below�a�certain�value�of�z��We�need�not�devote� any�further�attention�to�that�particular�situation��However,�how�do�we�determine�the�area� between� two� values� of� z?� This� is� a� little� different� and� needs� some� additional� discussion�� Consider� as� an� example� finding� the� area� between� z� =� −2�50� and� z� =� 1�00,� as� depicted� in� Figure� 4�3d�� Here� we� see� that� the� shaded� region� consists� of� two� smaller� areas,� the� area� between�the�mean�and�−2�50�and�the�area�between�the�mean�(z�=�0)�and�1�00��Using�the�table� again,� we� find� the� area� below� 1�00� is� �8413� and� the� area� below� −2�50� is� �0062�� Thus,� the� shaded�region�is�the�difference�as�computed�by��8413�−��0062�=��8351��On�your�own,�determine� the�area�between�z�=�−1�27�and�z�=��50�(answer:��5895)�
84 An Introduction to Statistical Concepts
Finally,�what�if�we�wanted�to�determine�areas�under�the�curve�for�values�of�X�rather�than� z?�The�answer�here�is�simple,�as�you�might�have�guessed��First�we�convert�the�value�of�X� to�a�z�score;�then�we�use�the�unit�normal�table�to�determine�the�area��Because�the�normal� curve�is�standard�for�all�members�of�the�family�of�normal�curves,�the�scale�of�the�variable,� X�or�z,�is�irrelevant�in�terms�of�determining�such�areas��In�the�next�section,�we�deal�more� with�such�transformations�
4.2 Standard Scores
We�have�already�devoted�considerable�attention�to�z�scores,�which�are�one�type�of�standard� score��In�this�section,�we�describe�an�application�of�z�scores�leading�up�to�a�discussion�of� other� types� of� standard� scores�� As� we� show,� the� major� purpose� of� standard� scores� is� to� place�scores�on�the�same�standard�scale�so�that�comparisons�can�be�made�across�individu- als�and/or�variables��Without�some�standard�scale,�comparisons�across�individuals�and/or� variables�would�be�difficult�to�make��Examples�are�coming�right�up�
4.2.1 z Scores
A�child�comes�home�from�school�with�the�results�of�two�tests�taken�that�day��On�the�math� test,�she�receives�a�score�of�75,�and�on�the�social�studies�test,�she�receives�a�score�of�60�� As�a�parent,�the�natural�question�to�ask�is,�“Which�performance�was�the�stronger�one?”�
.0062 –2.5(a)
.5000
0(b)
(c) (d)
.8413
1.0
.8351
0–2.5 1.0
FIGuRe 4.3 Examples�of�area�under�the�unit�normal�distribution:�(a)�Area�below�z�=�−2�5��(b)�Area�below�z�=�0��(c)�Area�below� z�=�1�0��(d)�Area�between�z�=�−2�5�and�z�=�1�0�
85Normal Distribution and Standard Scores
No�information�about�any�of�the�following�is�available:�maximum�score�possible,�mean� of�the�class�(or�any�other�central�tendency�measure),�or�standard�deviation�of�the�class� (or�any�other�dispersion�measure)��It�is�possible�that�the�two�tests�had�a�different�number� of� possible� points,� different� means,� and/or� different� standard� deviations�� How� can� we� possibly�answer�our�question?
The�answer,�of�course,�is�to�use�z�scores�if�the�data�are�assumed�to�be�normally�distrib- uted,�once�the�relevant�information�is�obtained��Let�us�take�a�minor�digression�before�we� return�to�answer�our�question�in�more�detail��Recall�the�formula�for�standardizing�vari- able�X�into�a�z�score:
z X
i i X
X =
−( )µ σ
where�the�X�subscript�has�been�added�to�the�mean�and�standard�deviation�for�purposes� of�clarifying�which�variable�is�being�considered��If�variable�X�is�the�number�of�items�cor- rect�on�a�test,�then�the�numerator�is�the�deviation�of�a�student’s�raw�score�from�the�class� mean� (i�e�,� the� numerator� is� a� deviation� score� as� previously� defined� in� Chapter� 3),� mea- sured�in�terms�of�items�correct,�and�the�denominator�is�the�standard�deviation�of�the�class,� measured� in� terms� of� items� correct�� Because� both� the� numerator� and� denominator� are� measured�in�terms�of�items�correct,�the�resultant�z�score�is�measured�in�terms�of�no�units� (as�the�units�of�the�numerator�and�denominator�essentially�cancel�out)��As�z�scores�have� no�units�(i�e�,�the�z�score�is�interpreted�as�the�number�of�standard�deviation�units�above�or� below�the�mean),�this�allows�us�to�compare�two�different�raw�score�variables�with�different� scales,�means,�and/or�standard�deviations��By�converting�our�two�variables�to�z�scores,�the� transformed�variables�are�now�on�the�same�z�score�scale�with�a�mean�of�0,�and�a�variance� and�standard�deviation�of�1�
Let� us� return� to� our� previous� situation� where� the� math� test� score� is� 75� and� the� social� studies�test�score�is�60��In�addition,�we�are�provided�with�information�that�the�standard� deviation�for�the�math�test�is�15�and�the�standard�deviation�for�the�social�studies�test�is�10�� Consider�the�following�three�examples��In�the�first�example,�the�means�are�60�for�the�math� test�and�50�for�the�social�studies�test��The�z�scores�are�then�computed�as�follows:
z zmath ss= −
= = −
= ( )
. ( )
. 75 60
15 1 0
60 50 10
1 0
The�conclusion�for�the�first�example�is�that�the�performance�on�both�tests�is�the�same;�that� is,�the�child�scored�one�standard�deviation�above�the�mean�for�both�tests�
In�the�second�example,�the�means�are�60�for�the�math�test�and�40�for�the�social�studies� test��The�z�scores�are�then�computed�as�follows:
z zmath ss= −
= = −
= ( )
. ( )
. 75 60
15 1 0
60 40 10
2 0
The�conclusion�for�the�second�example�is�that�performance�is�better�on�the�social�studies� test;� that� is,� the� child� scored� two� standard� deviations� above� the� mean� for� the� social� studies�test�and�only�one�standard�deviation�above�the�mean�for�the�math�test�
86 An Introduction to Statistical Concepts
In�the�third�example,�the�means�are�60�for�the�math�test�and�70�for�the�social�studies�test�� The�z�scores�are�then�computed�as�follows:
z zmath ss= −
= = −
= − ( )
. ( )
. 75 60
15 1 0
60 70 10
1 0
The�conclusion�for�the�third�example�is�that�performance�is�better�on�the�math�test;�that�is,� the�child�scored�one�standard�deviation�above�the�mean�for�the�math�test�and�one�standard� deviation� below� the� mean� for� the� social� studies� test�� These� examples� serve� to� illustrate� a� few� of� the� many� possibilities,� depending� on� the� particular� combinations� of� raw� score,� mean,�and�standard�deviation�for�each�variable�
Let�us�conclude�this�section�by�mentioning�the�major�characteristics�of�z�scores��The�first� characteristic�is�that�z�scores�provide�us�with�comparable�distributions,�as�we�just�saw�in� the� previous� examples�� Second,� z� scores� take� into� account� the� entire� distribution� of� raw� scores��All�raw�scores�can�be�converted�to�z�scores�such�that�every�raw�score�will�have�a� corresponding�z�score��Third,�we�can�evaluate�an�individual’s�performance�relative�to�the� scores�in�the�distribution��For�example,�saying�that�an�individual’s�score�is�one�standard� deviation�above�the�mean�is�a�measure�of�relative�performance��This�implies�that�approxi- mately�84%�of�the�scores�will�fall�below�the�performance�of�that�individual��Finally,�nega- tive�values�(i�e�,�below�0)�and�decimal�values�(e�g�,�z�=�1�55)�are�obviously�possible�(and�will� most�certainly�occur)�with�z�scores��On�the�average,�about�one-half�of�the�z�scores�for�any� distribution�will�be�negative,�and�some�decimal�values�are�quite�likely��This�last�character- istic�is�bothersome�to�some�individuals�and�has�led�to�the�development�of�other�types�of� standard�scores,�as�described�in�the�next�section�
4.2.2 Other Types of Standard Scores
Over�the�years,�other�standard�scores�besides�z�scores�have�been�developed,�either�to�allevi- ate�the�concern�over�negative�and/or�decimal�values�associated�with�z�scores,�or�to�obtain�a� particular�mean�and�standard�deviation��Let�us�examine�three�common�examples��The�first� additional�standard�score�is�known�as�the�College�Entrance�Examination�Board�(CEEB)�score�� This�standard�score�is�used�in�exams�such�as�the�SAT�and�the�GRE��The�subtests�for�these� exams�all�have�a�mean�of�500�and�a�standard�deviation�of�100��A�second�additional�standard� score�is�known�as�the�T�score�and�is�used�in�tests�such�as�most�behavior�rating�scales,�as�pre- viously�mentioned��The�T�scores�have�a�mean�of�50�and�a�standard�deviation�of�10��A�third� additional�standard�score�is�known�as�the�IQ�score�and�is�used�in�the�Wechsler�intelligence� scales��The�IQ�score�has�a�mean�of�100�and�a�standard�deviation�of�15�(the�Stanford–Binet� intelligence�scales�have�a�mean�of�100�and�a�standard�deviation�of�16)�
Say�we�want�to�develop�our�own�type�of�standard�score,�where�we�determine�in�advance� the�mean�and�standard�deviation�that�we�would�like�to�have��How�would�that�be�done?�As� the�equation�for�z�scores�is�as�follows:
z X
i i X
X =
−( )µ σ
then�algebraically�the�following�can�be�shown:
X zi X X i= +µ σ
87Normal Distribution and Standard Scores
If,�for�example,�we�want�to�develop�our�own�“stat”�standardized�score,�then�the�following� equation�would�be�used:
stat zi stat stat i= +µ σ
where stati�is�the�“stat”�standardized�score�for�a�particular�individual μstat�is�the�desired�mean�of�the�“stat”�distribution σstat�is�the�desired�standard�deviation�of�the�“stat”�distribution
If� we� want� to� have� a� mean� of� 10� and� a� standard� deviation� of� 2,� then� our� equation� becomes
stat zi i= +10 2
We�would�then�have�the�computer�simply�plug�in�a�z�score�and�compute�an�individual’s� “stat”�score��Thus,�a�z�score�of�1�0�would�yield�a�“stat”�standardized�score�of�12�0�
Consider�a�realistic�example�where�we�have�a�raw�score�variable�we�want�to�transform� into�a�standard�score,�and�we�want�to�control�the�mean�and�standard�deviation��For�exam- ple,�we�have�statistics�midterm�raw�scores�with�225�points�possible��We�want�to�develop� a�standard�score�with�a�mean�of�50�and�a�standard�deviation�of�5��We�also�have�scores�on� other� variables� that� are� on� different� scales� with� different� means� and� different� standard� deviations�(e�g�,�statistics�final�exam�scores�worth�175�points,�a�set�of�20�lab�assignments� worth�a�total�of�200�points,�a�statistics�performance�assessment�worth�100�points)��We�can� standardize�each�of�those�variables�by�placing�them�on�the�same�scale�with�the�same�mean� and�same�standard�deviation,�thereby�allowing�comparisons�across�variables��This�is�pre- cisely� the�rationale�used�by�testing�companies�and�researchers� when�they�develop� stan- dard�scores��In�short,�from�z�scores,�we�can�develop�a�CEEB,�T,�IQ,�“stat,”�or�any�other�type� of�standard�score�
4.3 Skewness and Kurtosis Statistics
In� previous� chapters,� we� discussed� the� distributional� concepts� of� symmetry,� skewness,� central�tendency,�and�dispersion��In�this�section,�we�more�closely�define�symmetry�as�well� as�the�statistics�commonly�used�to�measure�skewness�and�kurtosis�
4.3.1 Symmetry
Conceptually,�we�define�a�distribution�as�being�symmetric�if�when�we�divide�the�dis- tribution� precisely� in� one-half,� the� left-hand� half� is� a� mirror� image� of� the� right-hand� half�� That� is,� the� distribution� above� the� mean� is� a� mirror� image� of� the� distribution� below�the�mean��To�put�it�another�way,�a�distribution�is�symmetric around the mean� if�for�every�score�q�units�below�the�mean,�there�is�a�corresponding�score�q�units�above� the�mean�
88 An Introduction to Statistical Concepts
Two� examples� of� symmetric� distributions� are� shown� in� Figure� 4�4�� In� Figure� 4�4a,� we� have�a�normal�distribution,�which�is�clearly�symmetric�around�the�mean��In�Figure�4�4b,� we� have� a� symmetric� distribution� that� is� bimodal,� unlike� the� previous� example�� From� these�and�other�numerous�examples,�we�can�make�the�following�two�conclusions��First,�if�a� distribution�is�symmetric,�then�the�mean�is�equal�to�the�median��Second,�if�a�distribution�is� symmetric�and�unimodal,�then�the�mean,�median,�and�mode�are�all�equal��This�indicates� we�can�determine�whether�a�distribution�is�symmetric�by�simply�comparing�the�measures� of�central�tendency�
4.3.2 Skewness
We� define� skewness� as� the� extent� to� which� a� distribution� of� scores� deviates� from� per- fect�symmetry��This�is�important�as�perfectly�symmetrical�distributions�rarely�occur�with� actual�sample�data�(i�e�,�“real”�data)��A�skewed�distribution�is�known�as�being�asymmetri- cal�� As� shown� in� Figure� 4�5,� there� are� two� general� types� of� skewness,� distributions� that� are�negatively�skewed,�as�in�Figure�4�5a,�and�those�that�are�positively�skewed,�as�in�Figure� 4�5b��Negatively�skewed�distributions,�which�are�skewed�to�the�left,�occur�when�most�of� the�scores�are�toward�the�high�end�of�the�distribution�and�only�a�few�scores�are�toward� the�low�end��If�you�make�a�fist�with�your�thumb�pointing�to�the�left�(skewed�to�the�left),� you� have� graphically� defined� a� negatively� skewed� distribution�� For� a� negatively� skewed�
(a) (b)
FIGuRe 4.4 Symmetric�distributions:�(a)�Normal�distribution��(b)�Bimodal�distribution�
(a) (b)
FIGuRe 4.5 Skewed�distributions:�(a)�Negatively�skewed�distribution��(b)�Positively�skewed�distribution�
89Normal Distribution and Standard Scores
distribution,�we�also�find�the�following:�mode > median > mean��This�indicates�that�we�can� determine�whether�a�distribution�is�negatively�skewed�by�simply�comparing�the�measures� of�central�tendency�
Positively�skewed�distributions,�which�are�skewed�to�the�right,�occur�when�most�of�the� scores� are� toward� the� low� end� of� the� distribution� and� only� a� few� scores� are� toward� the� high�end��If�you�make�a�fist�with�your�thumb�pointing�to�the�right�(skewed�to�the�right),� you� have� graphically� defined� a� positively� skewed� distribution�� For� a� positively� skewed� distribution,�we�also�find�the�following:�mode < median < mean��This�indicates�that�we�can� determine�whether�a�distribution�is�positively�skewed�by�simply�comparing�the�measures� of�central�tendency�
The� most� commonly� used� measure� of� skewness� is� known� as� γ1� (Greek� letter� gamma),� which�is�mathematically�defined�as�follows:
γ 1
3
1= = ∑ z N
i
i
N
where�we�take�the�z�score�for�each�individual,�cube�it,�sum�across�all�N�individuals,�and�then� divide� by� the� number� of� individuals� N�� This� measure� is� available� in� nearly� all� computer� packages,� so� hand� computations� are� not� necessary�� The� characteristics� of� this� measure� of� skewness�are�as�follows:�(a)�a�perfectly�symmetrical�distribution�has�a�skewness�value�of 0,� (b)�the�range�of�values�for�the�skewness�statistic�is�approximately�from�−3�to�+3,�(c) nega- tively�skewed�distributions�have�negative�skewness�values,�and�(d)�positively�skewed�dis- tributions�have�positive�skewness�values�
There�are�different�rules�of�thumb�for�determining�how�extreme�skewness�can�be�and� still�retain�a�relatively�normal�distribution��One�simple�rule�of�thumb�is�that�skewness� values� within� ±2�0� are� considered� relatively� normal,� with� more� conservative� research- ers� applying� a� ±3�0� guideline,� and� more� stringent� researchers� using� ±1�0�� Another� rule� of� thumb� for� determining� how� extreme� a� skewness� value� must� be� for� the� distribution� to� be� considered� nonnormal� is� as� follows:� Skewness� values� outside� the� range� of� ±� two� standard�errors�of�skewness�suggest�a�distribution�that�is�nonnormal��Applying�this�rule� of� thumb,� if� the� standard� error� of� skewness� is� �85,� then� anything� outside� of� −2(�85)� to� +2(�85),�or�−1�7�to +1�7,�would�be�considered�nonnormal��It�is�important�to�note�that�this� second�rule�of�thumb�is�sensitive�to�small�sample�sizes�and�should�only�be�considered�as� a�general�guide�
4.3.3 kurtosis
Kurtosis� is� the� fourth� and� final� property� of� a� distribution� (often� referred� to� as� the� moments around the mean)��These�four�properties�are�central�tendency�(first�moment),� dispersion� (second� moment),� skewness� (third� moment),� and� kurtosis� (fourth� moment)�� Kurtosis�is�conceptually�defined�as�the�“peakedness”�of�a�distribution�(kurtosis�is�Greek� for�peakedness)��Some�distributions�are�rather�flat,�and�others�have�a�rather�sharp�peak�� Specifically,�there�are�three�general�types�of�peakedness,�as�shown�in�Figure�4�6��A�distri- bution�that�is�very�peaked�is�known�as�leptokurtic�(“lepto”�meaning�slender�or�narrow)� (Figure�4�6a)��A�distribution�that�is�relatively�flat�is�known�as�platykurtic�(“platy”�mean- ing�flat�or�broad)�(Figure�4�6b)��A�distribution�that�is�somewhere�in�between�is�known�as� mesokurtic�(“meso”�meaning�intermediate)�(Figure�4�6c)�
90 An Introduction to Statistical Concepts
The�most�commonly�used�measure�of�kurtosis�is�known�as�γ2,�which�is�mathematically� defined�as
γ 2
4
1 3= −= ∑ z N
i
i
N
where�we�take�the�z�score�for�each�individual,�take�it�to�the�fourth�power�(being�the�fourth� moment),� sum� across� all� N� individuals,� divide� by� the� number� of� individuals� N,� and� then� subtract�3��This�measure�is�available�in�nearly�all�computer�packages,�so�hand�computations� are�not�necessary��The�characteristics�of�this�measure�of�kurtosis�are�as�follows:�(a)�a�perfectly� mesokurtic�distribution,�which�would�be�a�normal�distribution,�has�a�kurtosis�value�of�0,� (b)� platykurtic�distributions�have�negative�kurtosis�values�(being�flat�rather�than�peaked),� and�(c)�leptokurtic�distributions�have�positive�kurtosis�values�(being�peaked)��Kurtosis�values� can�range�from�negative�to�positive�infinity�
There�are�different�rules�of�thumb�for�determining�how�extreme�kurtosis�can�be�and�still� retain� a� relatively� normal� distribution�� One� simple� rule� of� thumb� is� that� kurtosis� values� within�±2�0�are�considered�relatively�normal,�with�more�conservative�researchers�applying� a�±3�0�guideline,�and�more�stringent�researchers�using�±1�0��A�rule�of�thumb�for�determin- ing�how�extreme�a�kurtosis�value�may�be�for�the�distribution�to�be�considered�nonnormal� is�as�follows:�Kurtosis�values�outside�the�range�of�±�two�standard�errors�of�kurtosis�suggest�
(c)
(a) (b)
FIGuRe 4.6 Distributions� of� different� kurtosis:� (a)� Leptokurtic� distribution�� (b)� Platykurtic� distribution�� (c)� Mesokurtic� distribution�
91Normal Distribution and Standard Scores
a� distribution� that� is� nonnormal�� Applying� this� rule� of� thumb,� if� the� standard� error� of� kurtosis� is� 1�20,� then� anything� outside� of� −2(1�20)� to� +2(1�20),� or� −2�40� to� +2�40,� would� be� considered�nonnormal��It�is�important�to�note�that�this�second�rule�of�thumb�is�sensitive�to� small�sample�sizes�and�should�only�be�considered�as�a�general�guide�
Skewness�and�kurtosis�statistics�are�useful�for�the�following�two�reasons:�(a)�as�descrip- tive�statistics�used�to�describe�the�shape�of�a�distribution�of�scores�and�(b)�in�inferential� statistics,�which�often�assume�a�normal�distribution,�so�the�researcher�has�some�indication� of�whether�the�assumption�has�been�met�(more�about�this�beginning�in�Chapter�6)�
4.4 SPSS
Here�we�review�what�SPSS�has�to�offer�for�examining�distributional�shape�and�computing� standard�scores��The�following�programs�have�proven�to�be�quite�useful�for�these�purposes:� “Explore,” “Descriptives,” “Frequencies,” “Graphs,”� and� “Transform.”� Instructions�for�using�each�are�provided�as�follows�
Explore
Explore: Step 1.� The� first� program,� “Explore,”� can� be� invoked� by� clicking� on� “Analyze”�in�the�top�pulldown�menu,�then�“Descriptive Statistics,”�and�then� “Explore.”�Following�the�screenshot�(step�1),�as�follows,�produces�the�“Explore”�dia- log�box��For�brevity,�we�have�not�reproduced�this�initial�screenshot�when�we�discuss�the� “Descriptives”� and�“Frequencies”� programs;� however,� you� see� here� where� they� can�be�found�from�the�pulldown�menus�
Explore: Step 1
B
A
C
Frequencies and Descriptives
can also be invoked from this menu.
92 An Introduction to Statistical Concepts
Explore: Step 2.� Next,� from� the� main�“Explore”� dialog� box,� click� the� variable� of� interest�from�the�list�on�the�left�(e�g�,�quiz),�and�move�it�into�the�“Dependent List”�box� by�clicking�on�the�arrow�button��Next,�click�on�the�“Statistics”�button�located�in�the� top�right�corner�of�the�main�dialog�box�
Select the variable of interest from
the list on the left and use the arrow
to move to the “Dependent List”
box on the right.
Clicking on “Statistics” will
allow you to select descriptive statistics.
Explore: Step 2
Explore: Step 3.�A�new�box�labeled�“Explore: Statistics”�will�appear��Simply� place�a�checkmark�in�the�“Descriptives”�box��Next�click�“Continue.”�You�will�then�be� returned�to�the�main�“Explore”�dialog�box��From�there,�click�“OK.”�This�will�automati- cally�generate�the�skewness�and�kurtosis�values,�as�well�as�measures�of�central�tendency� and� dispersion� which� were� covered� in� Chapter� 3�� The� output� from� this� was� previously� shown�in�the�top�panel�of�Table�3�5�
Explore: Step 3
Descriptives
Descriptives: Step 1. The� second� program� to� consider� is� “Descriptives.”� It� can� also� be� accessed� by� going� to�“Analyze”� in� the� top� pulldown� menu,� then� selecting� “Descriptive Statistics,”�and�then�“Descriptives”�(see�“Explore: Step 1”�for� screenshots�of�these�steps)�
Descriptives: Step 2.� This� will� bring� up� the� “Descriptives”� dialog� box� (see� screenshot,�step�2)��From�the�main�“Descriptives”�dialog�box,�click�the�variable�of�inter- est�(e�g�,�quiz)�and�move�into�the�“Variable(s)”�box�by�clicking�on�the�arrow��If�you�want�
93Normal Distribution and Standard Scores
to�obtain�z�scores�for�this�variable�for�each�case�(e�g�,�person�or�object�that�was�measured— your�unit�of�analysis),�check�the�“Save standardized values as variables”�box� located�in�the�bottom�left�corner�of�the�main�“Descriptives”�dialog�box��This�will�insert� a�new�variable�into�your�dataset�for�subsequent�analysis�(see�screenshot�for�how�this�will� appear�in�“Data View”)��Next,�click�on�the�“Options”�button�
Select the variable of interest from
the list on the left and use the arrow
to move to the “Variable” box on
the right.
Placing a checkmark here will generate a
new, standardized
variable in your datafile for each
variable selected.
Clicking on “Options” will allow you to select various statistics to
be generated.
Descriptives: Step 2
Descriptives: Step 3.�A�new�box�called�“Descriptives: Options”�will�appear� (see�screenshot,�step�3)�and�you�can�simply�place�a�checkmark�in�the�boxes�for�the�statistics� that� you� want� to� generate�� This� will� allow� you� to� obtain� the� skewness� and� kurtosis� val- ues,�as�well�as�measures�of�central�tendency�and�dispersion�discussed�in�Chapter�3��After� making� your� selections,� click� on� “Continue.”� You� will� then� be� returned� to� the� main� “Descriptives”�dialog�box��From�there,�click�“OK.”
Statistics available when clicking on “Options”
from the main dialog box for Descriptives.
Placing a checkmark will generate the respective statistic in the output.
Descriptives: Step 3
94 An Introduction to Statistical Concepts
X – μ σ
If “Save standardized values as variables”
was checked on the main “Descriptives” dialog
box, a new standardized variable will be created. By default, this variable name is the name of the original variable prefixed with a “Z” (denoting its
standardization). It is computed using the
unit normal formula:
Descriptives: Saving
standardized variable
–
–
–
–
–
–
–
–
–
Frequencies
Frequencies: Step 1.�The�third�program�to�consider�is�“Frequencies,”�which�is� also� accessible� by� clicking� on� “Analyze”� in� the� top� pulldown� menu,� then� clicking� on� “Descriptive Statistics,”� and� then� selecting� “Frequencies”� (see� “Explore: Step 1”�for�screenshots�of�these�steps)�
Frequencies: Step 2.�This�will�bring�up�the�“Frequencies”�dialog�box��Click�the� variable�of�interest�(e�g�,�quiz)�into�the�“Variable(s)”�box,�then�click�on�the�“Statistics”� button�
95Normal Distribution and Standard Scores
Select the variable of interest from
the list on the left and use the arrow
to move to the “Variable” box on
the right.
Clicking on “Charts” will
allow you to generate a
histogram with normal curve
(and other types of graphs).
Clicking on “Statistics” will allow you to
select various statistics to be generated.
Frequencies: Step 2
Frequencies: Step 3.�A�new�box�labeled�“Frequencies:�Statistics”�will�appear�� Again,�you�can�simply�place�a�checkmark�in�the�boxes�for�the�statistics�that�you�want�to� generate��Here�you�can�obtain�the�skewness�and�kurtosis�values,�as�well�as�measures�of� central�tendency�and�dispersion�from�Chapter�3��If�you�click�on�the�“Charts”�button,�you� can� also� obtain� a� histogram� with� a� normal� curve� overlay� by� clicking� the�“Histogram”� radio� button� and� checking� the�“With normal curve”� box�� This� histogram� output� is� shown�in�Figure�4�7��After�making�your�selections,�click�on�“Continue.”�You�will�then�be� returned�to�the�main�“Frequencies”�dialog�box��From�there,�click�“OK.”
9 10 11 12 13 14 15 Quiz
16 17 18 19 20
1
2
3
Fr eq
ue nc
y
4
5
FIGuRe 4.7 SPSS� histogram� of� statistics� quiz� data� with� nor- mal�distribution�overlay�
96 An Introduction to Statistical Concepts
Options available when clicking on “Statistics” from the main
dialog box for Frequencies. Placing a checkmark will generate the
respective statistic in the output.
Check this for better
accuracy with quartiles and percentiles
(i.e., the median).
Frequencies: Step 3
Graphs
Graphs:� Two� other� programs� also� yield� a� histogram� with� a� normal� curve� overlay�� Both� can� be� accessed� by� first� going� to�“Graphs”� in� the� top� pulldown� menu�� From� there,� select� “Legacy Dialogs,”�then�“Histogram.”�Another�option�for�creating�a�histogram,�starting� again�from�the�“Graphs”�option�in�the�top�pulldown�menu,�is�to�select�“Legacy Dialogs,”� then� “Interactive,”� and� finally� “Histogram.”� From� there,� both� work� similarly� to� the� “Frequencies”�program�described�earlier�
A
B
C
Graphs: Step 1
97Normal Distribution and Standard Scores
Transform
Transform: Step 1.�A�final�program�that�comes�in�handy�is�for�transforming�variables,� such� as� creating� a� standardized� version� of� a� variable� (most� notably� standardization� other� than�the�application�of�the�unit�normal�formula,�where�the�unit�normal�standardization�can� be� easily� performed� as� seen� previously� by� using�“Descriptives”)�� Go� to�“Transform”� from�the�top�pulldown�menu,�and�then�select�“Compute Variables.”�A�dialog�box�labeled� “Compute Variables”�will�appear�
A
B
Transform: Step 1
Transform: Step 2.�The�“Target Variable”�is�the�name�of�the�new�variable�you�are� creating,�and�the�“Numeric Expression”�box�is�where�you�insert�the�commands�of�which� original�variable�to�transform�and�how�to�transform�it�(e�g�,�stat�variable)��When�you�are�done� defining�the�formula,�simply�click�“OK”�to�generate�the�new�variable�in�the�data�file�
The name specified here becomes the
column header in “Data View.” This name must
begin with a letter, and no spaces can be
included.
“Numeric Expression” will be where you enter the formula for your new variable. For user’s convenience, a number of formulas are already defined within SPSS and accessible through the “Function group” formulas listed below.
Transform: Step 2
98 An Introduction to Statistical Concepts
4.5 Templates for Research Questions and APA-Style Paragraph
As�stated�in�the�previous�chapter,�depending�on�the�purpose�of�your�research�study,�you� may�or�may�not�write�a�research�question�that�corresponds�to�your�descriptive�statistics�� If� the� end� result� of� your� research� paper� is� to� present� results� from� inferential� statistics,� it� may� be� that� your� research� questions� correspond� only� to� those� inferential� questions,� and,� thus,� no� question� is� presented� to� represent� the� descriptive� statistics�� That� is� quite� common�� On� the� other� hand,� if� the� ultimate� purpose� of� your� research� study� is� purely� descriptive� in� nature,� then� writing� one� or� more� research� questions� that� correspond� to� the� descriptive� statistics� is� not� only� entirely� appropriate� but� (in� most� cases)� absolutely� necessary�
It�is�time�again�to�revisit�our�graduate�research�assistant,�Marie,�who�was�reintroduced� at�the�beginning�of�the�chapter��As�a�reminder,�her�task�was�to�continue�to�summarize�data� from� 25� students� enrolled� in� a� statistics� course,� this� time� paying� particular� attention� to� distributional�shape�and�standardization��The�questions�posed�this�time�by�Marie’s�faculty� mentor�were�as�follows:�What is the distributional shape of the statistics quiz score? In standard deviation units, what is the relative stand- ing to the mean of student 1 compared to student 3? A�template�for�writ- ing�a�descriptive�research�question�for�summarizing�distributional�shape�is�presented�as� follows�(this�may�sound�familiar�as�this�was�first�presented�in�Chapter�2�when�we�initially� discussed�distributional�shape)��This�is�followed�by�a�template�for�writing�a�research�ques- tion�related�to�standardization:
What is the distributional shape of the [variable]? In standard devi- ation units, what is the relative standing to the mean of [unit 1] compared to [unit 3]?
Next,� we� present� an� APA-style� paragraph� summarizing� the� results� of� the� statistics� quiz� data�example�answering�the�questions�posed�to�Marie:
As shown in the top panel of Table 3.5, the skewness value is −.598 (SE = .464) and the kurtosis value is −.741 (SE = .902). Skewness and kurtosis values within the range of +/−2(SE) are generally considered normal. Given our values, skewness is within the range of −.928 to +.928 and kurtosis is within the range of −1.804 and +1.804, and these would be considered normal. Another rule of thumb is that the skew- ness and kurtosis values should fall within an absolute value of 2.0 to be considered normal. Applying this rule, normality is still evi- dent. The histogram with a normal curve overlay is depicted in Figure 4.7. Taken with the skewness and kurtosis statistics, these results indicate that the quiz scores are reasonably normally distributed. There is a slight negative skew such that there are more scores at the high end of the distribution than a typical normal distribu- tion. There is also a slight negative kurtosis indicating that the distribution is slightly flatter than a normal distribution, with a few more extreme scores at the low end of the distribution. Again, however, the values are within the range of what is considered a reasonable approximation to the normal curve.
99Normal Distribution and Standard Scores
The quiz score data were standardized using the unit normal formula. After standardization, student 1’s score was −2.07 and student 3’s score was 1.40. This suggests that student 1 was slightly more than two stan- dard deviation units below the mean on the statistics quiz score, while student 3 was nearly 1.5 standard deviation units above the mean.
4.6 Summary
In� this� chapter,� we� continued� our� exploration� of� descriptive� statistics� by� considering� an� important� distribution,� the� normal� distribution,� standard� scores,� and� other� characteristics� of�a�distribution�of�scores��First�we�discussed�the�normal�distribution,�with�its�history�and� important� characteristics�� In� addition,� the� unit� normal� table� was� introduced� and� used� to� determine� various� areas� under� the� curve�� Next� we� examined� different� types� of� standard� scores,�in�particular�z�scores,�as�well�as�CEEB�scores,�T�scores,�and�IQ�scores��Examples�of� types�of�standard�scores�are�summarized�in�Box�4�1��The�next�section�of�the�chapter�included� a�detailed�description�of�symmetry,�skewness,�and�kurtosis��The�different�types�of�skewness� and� kurtosis� were� defined� and� depicted�� We� finished� the� chapter� by� examining� SPSS� for� these�statistics�as�well�as�how�to�write�up�an�example�set�of�results��At�this�point,�you�should� have� met� the� following� objectives:� (a)� understand� the� normal� distribution� and� utilize� the� normal�table;�(b)�determine�and�interpret�different�types�of�standard�scores,�particularly�z� scores;�and�(c)�understand�and�interpret�skewness�and�kurtosis�statistics��In�the�next�chapter,� we�move�toward�inferential�statistics�through�an�introductory�discussion�of�probability�as� well�as�a�more�detailed�discussion�of�sampling�and�estimation�
STOp aNd ThINk bOx 4.1
Examples�of�Types�of�Standard�Scores
Standard Score Distributiona
Z�(unit�normal) N(0,�1) CEEB�score N(500,�10,000) T�score N(50,�100) Wechsler�intelligence�scale N(100,�225) Stanford–Binet�intelligence�scale N(100,�256)
a� N(μ,�σ2)�
Problems
Conceptual problems
4.1� For�which�of�the�following�distributions�will�the�skewness�value�be�0? � a�� N(0,�1) � b�� N(0,�2) � c�� N(10,�50) � d�� All�of�the�above
100 An Introduction to Statistical Concepts
4.2� For�which�of�the�following�distributions�will�the�kurtosis�value�be�0? � a�� N(0,�1) � b�� N(0,�2) � c�� N(10,�50) � d�� All�of�the�above 4.3� A�set�of�400�scores�is�approximately�normally�distributed�with�a�mean�of�65�and�a�
standard�deviation�of�4�5��Approximately�95%�of�the�scores�would�fall�within�which� range�of�scores?
� a�� 60�5�and�69�5 � b�� 56�and�74 � c�� 51�5�and�78�5 � d�� 64�775�and�65�225 4.4� What�is�the�percentile�rank�of�60�in�the�distribution�of�N(60,100)? � a�� 10 � b�� 50 � c�� 60 � d�� 100 4.5� Which� of� the� following� parameters� can� be� found� on� the� X� axis� for� a� frequency�
polygon�of�a�population�distribution? � a�� Skewness � b�� Median � c�� Kurtosis � d�� Q 4.6� The�skewness�value�is�calculated�for�a�set�of�data�and�is�found�to�be�equal�to�+2�75��
This�indicates�that�the�distribution�of�scores�is�which�one�of�the�following? � a�� Highly�negatively�skewed � b�� Slightly�negatively�skewed � c�� Symmetrical � d�� Slightly�positively�skewed � e�� Highly�positively�skewed 4.7� The�kurtosis�value�is�calculated�for�a�set�of�data�and�is�found�to�be�equal�to�+2�75��This�
indicates�that�the�distribution�of�scores�is�which�one�of�the�following? � a�� Mesokurtic � b�� Platykurtic � c�� Leptokurtic � d�� Cannot�be�determined 4.8� For�a�normal�distribution,�all�percentiles�above�the�50th�must�yield�positive�z�scores��
True�or�false? 4.9� If� one� knows� the� raw� score,� the� mean,� and� the� z� score,� then� one� can� calculate� the�
value�of�the�standard�deviation��True�or�false?
101Normal Distribution and Standard Scores
4.10� In�a�normal�distribution,�a�z�score�of�1�0�has�a�percentile�rank�of�34��True�or�false? 4.11� The�mean�of�a�normal�distribution�of�scores�is�always�1��True�or�false? 4.12� If�in�a�distribution�of�200�IQ�scores,�the�mean�is�considerably�above�the�median,�then�
the�distribution�is�which�one�of�the�following? � a�� Negatively�skewed � b�� Symmetrical � c�� Positively�skewed � d�� Bimodal 4.13� Which� of� the� following� is� indicative� of� a� distribution� that� has� a� skewness� value� of�
−3�98�and�a�kurtosis�value�of�−6�72? � a�� A�left�tail�that�is�pulled�to�the�left�and�a�very�flat�distribution � b�� A�left�tail�that�is�pulled�to�the�left�and�a�distribution�that�is�neither�very�peaked�
nor�very�flat � c�� A�right�tail�that�is�pulled�to�the�right�and�a�very�peaked�distribution � d�� A�right�tail�that�is�pulled�to�the�right�and�a�very�flat�distribution 4.14� Which�of�the�following�is�indicative�of�a�distribution�that�has�a�kurtosis�value�of�+4�09? � a�� Leptokurtic�distribution � b�� Mesokurtic�distribution � c�� Platykurtic�distribution � d�� Positive�skewness � e�� Negative�skewness 4.15� For�which�of�the�following�distributions�will�the�kurtosis�value�be�greatest?
A f B f C f D f
11 3 11 4 11 1 11 1 12 4 12 4 12 3 12 5 13 6 13 4 13 12 13 8 14 4 14 4 14 3 14 5 15 3 15 4 15 1 15 1
� a�� Distribution�A � b�� Distribution�B � c�� Distribution�C � d�� Distribution�D
4.16� The�distribution�of�variable�X�has�a�mean�of�10�and�is�positively�skewed��The�distri- bution�of�variable�Y�has�the�same�mean�of�10�and�is�negatively�skewed��I�assert�that� the�medians�for�the�two�variables�must�also�be�the�same��Am�I�correct?
4.17� The�variance�of�z�scores�is�always�equal�to�the�variance�of�the�raw�scores�for�the�same� variable��True�or�false?
102 An Introduction to Statistical Concepts
4.18� The� mode� has� the� largest� value� of� the� central� tendency� measures� in� a� positively� skewed�distribution��True�or�false?
4.19� Which� of� the� following� represents� the� highest� performance� in� a� normal� distribution?
� a�� P90 � b�� z�=�+1�00 � c�� Q3 � d�� IQ�=�115 4.20� Suzie�Smith�came�home�with�two�test�scores,�z�=�+1�in�math�and�z�=�−1�in�biology��
For which�test�did�Suzie�perform�better? 4.21� A�psychologist�analyzing�data�from�creative�intelligence�scores�finds�a�relatively�nor-
mal�distribution�with�a�population�mean�of�100�and�population�standard�deviation� of� 10�� When� standardized� into� a� unit� normal� distribution,� what� is� the� mean� of� the� (standardized)�creative�intelligence�scores?
� a�� 0 � b�� 70 � c�� 100 � d�� Cannot�be�determined�from�the�information�provided
Computational problems
4.1� Give�the�numerical�value�for�each�of�the�following�descriptions�concerning�normal� distributions�by�referring�to�the�table�for�N(0,�1)�
� a�� The�proportion�of�the�area�below�z�=�−1�66 � b�� The�proportion�of�the�area�between�z�=�−1�03�and�z�=�+1�03 � c�� The�fifth�percentile�of�N(20,�36) � d�� The�99th�percentile�of�N(30,�49) � e�� The�percentile�rank�of�the�score�25�in�N(20,�36) � f�� The�percentile�rank�of�the�score�24�5�in�N(30,�49) � g�� The�proportion�of�the�area�in�N(36,�64)�between�the�scores�of�18�and�42 4.2� Give�the�numerical�value�for�each�of�the�following�descriptions�concerning�normal�
distributions�by�referring�to�the�table�for�N(0,�1)� � a�� The�proportion�of�the�area�below�z�=�−�80 � b�� The�proportion�of�the�area�between�z�=�−1�49�and�z�=�+1�49 � c�� The�2�5th�percentile�of�N(50,�81) � d�� The�50th�percentile�of�N(40,�64) � e�� The�percentile�rank�of�the�score�45�in�N(50,�81) � f�� The�percentile�rank�of�the�score�53�in�N(50,�81) � g�� The�proportion�of�the�area�in�N(36,�64)�between�the�scores�of�19�7�and�45�1
103Normal Distribution and Standard Scores
4.3� Give�the�numerical�value�for�each�of�the�following�descriptions�concerning�normal� distributions�by�referring�to�the�table�for�N(0,�1)�
� a�� The�proportion�of�the�area�below�z�=�+1�50 � b�� The�proportion�of�the�area�between�z�=�−�75�and�z�=�+2�25 � c�� The�15th�percentile�of�N(12,�9) � d�� The�80th�percentile�of�N(100,000,�5,000) � e�� The�percentile�rank�of�the�score�300�in�N(200,�2500) � f�� The�percentile�rank�of�the�score�61�in�N(60,�9) � g�� The�proportion�of�the�area�in�N(500,�1600)�between�the�scores�of�350�and�550
Interpretive problems
4.1� Select� one� interval� or� ratio� variable� from� the� survey� 1� dataset� on� the� website� (e�g�,� one�idea�is�to�select�the�same�variable�you�selected�for�the�interpretive�problem�from� Chapter�3)�
� a�� Determine�the�measures�of�central�tendency,�dispersion,�skewness,�and�kurtosis� � b�� Write�a�paragraph�which�summarizes�the�findings,�particularly�commenting�on�
the�distributional�shape� 4.2� Using�the�same�variable�selected�in�the�previous�problem,�standardize�it�using�SPSS� � a�� Determine�the�measures�of�central�tendency,�dispersion,�skewness,�and�kurtosis�
for�the�standardized�variable� � b�� Determine�the�measures�of�central�tendency,�dispersion,�skewness,�and�kurtosis�
for�the�variable�in�its�original�scale�(i�e�,�the�unstandardized�variable)� � c�� Compare� and� contrast� the� differences� between� the� standardized� and� unstan-
dardized�variables�
105
5 Introduction to Probability and Sample Statistics
Chapter Outline
5�1� Brief�Introduction�to�Probability 5�1�1� Importance�of�Probability 5�1�2� Definition�of�Probability 5�1�3� Intuition�Versus�Probability
5�2� Sampling�and�Estimation 5�2�1� Simple�Random�Sampling 5�2�2� Estimation�of�Population�Parameters�and�Sampling�Distributions
Key Concepts
� 1�� Probability � 2�� Inferential�statistics � 3�� Simple�random�sampling�(with�and�without�replacement) � 4�� Sampling�distribution�of�the�mean � 5�� Variance�and�standard�error�of�the�mean�(sampling�error) � 6�� Confidence�intervals�(CIs)�(point�vs��interval�estimation) � 7�� Central�limit�theorem
In�Chapter�4,�we�extended�our�discussion�of�descriptive�statistics��There�we�considered�the� following�three�general�topics:�the�normal�distribution,�standard�scores,�and�skewness�and� kurtosis��In�this�chapter,�we�begin�to�move�from�descriptive�statistics�into�inferential�statis- tics�(in�which�normally�distributed�data�play�a�major�role)��The�two�basic�topics�described� in�this�chapter�are�probability,�and�sampling�and�estimation��First,�as�a�brief�introduction� to�probability,�we�discuss�the�importance�of�probability�in�statistics,�define�probability�in�a� conceptual�and�computational�sense,�and�discuss�the�notion�of�intuition�versus�probabil- ity�� Second,� under� sampling� and� estimation,� we� formally� move� into� inferential� statistics� by�considering�the�following�topics:�simple�random�sampling�(and�briefly�other�types�of� sampling),�and�estimation�of�population�parameters�and�sampling�distributions��Concepts� to� be� discussed� include� probability,� inferential� statistics,� simple� random� sampling� (with� and�without�replacement),�sampling�distribution�of�the�mean,�variance�and�standard�error�
106 An Introduction to Statistical Concepts
of�the�mean�(sampling�error),�CIs�(point�vs��interval�estimation),�and�central�limit�theorem�� Our�objectives�are�that�by�the�end�of�this�chapter,�you�will�be�able�to�(a)�understand�the� most�basic�concepts�of�probability;�(b)�understand�and�conduct�simple�random�sampling;� and�(c)�understand,�determine,�and�interpret�the�results�from�the�estimation�of�population� parameters�via�a�sample�
5.1 Brief Introduction to Probability
The�area�of�probability�became�important�and�began�to�be�developed�during�the�Middle� Ages�(seventeenth�and�eighteenth�centuries),�when�royalty�and�other�well-to-do�gamblers� consulted� with� mathematicians� for� advice� on� games� of� chance�� For� example,� in� poker� if� you�hold�two�jacks,�what�are�your�chances�of�drawing�a�third�jack?�Or�in�craps,�what�is�the� chance�of�rolling�a�“7”�with�two�dice?�During�that�time,�probability�was�also�used�for�more� practical� purposes,� such� as� to� help� determine� life� expectancy� to� underwrite� life� insur- ance� policies�� Considerable� development� in� probability� has� obviously� taken� place� since� that� time�� In� this� section,� we� discuss� the� importance� of� probability,� provide� a� definition� of� probability,� and� consider� the� notion� of� intuition� versus� probability�� Although� there� is� much�more�to�the�topic�of�probability,�here�we�simply�discuss�those�aspects�of�probability� necessary�for�the�remainder�of�the�text��For�additional�information�on�probability,�take�a� look�at�texts�by�Rudas�(2004)�or�Tijms�(2004)�
5.1.1 Importance of probability
Let�us�first�consider�why�probability�is�important�in�statistics��A�researcher�is�out�collect- ing�some�sample�data�from�a�group�of�individuals�(e�g�,�students,�parents,�teachers,�voters,� corporations,�animals)��Some�descriptive�statistics�are�generated�from�the�sample�data��Say� the�sample�mean,�X
– ,�is�computed�for�several�variables�(e�g�,�number�of�hours�of�study�time�
per� week,� grade� point� average,� confidence� in� a� political� candidate,� widget� sales,� animal� food�consumption)��To�what�extent�can�we�generalize�from�these�sample�statistics�to�their� corresponding�population�parameters?�For�example,�if�the�mean�amount�of�study�time�per� week�for�a�given�sample�of�graduate�students�is�X
– �=�10�hours,�to�what�extent�are�we�able�to�
generalize�to�the�population�of�graduate�students�on�the�value�of�the�population�mean�μ? As�we�see,�beginning�in�this�chapter,�inferential�statistics�involve�making�an�inference�
about�population�parameters�from�sample�statistics��We�would�like�to�know�(a)�how�much� uncertainty�exists�in�our�sample�statistics�as�well�as�(b)�how�much�confidence�to�place�in� our�sample�statistics��These�questions�can�be�addressed�by�assigning�a�probability�value� to�an�inference��As�we�show�beginning�in�Chapter�6,�probability�can�also�be�used�to�make� statements�about�areas�under�a�distribution�of�scores�(e�g�,�the�normal�distribution)��First,� however,�we�need�to�provide�a�definition�of�probability�
5.1.2 definition of probability
In�order�to�more�easily�define�probability,�consider�a�simple�example�of�rolling�a�six-sided�die� (as�there�are�dice�with�different�numbers�of�sides)��Each�of�the�six�sides,�of�course,�has�any- where�from�one�to�six�dots��Each�side�has�a�different�number�of�dots��What�is�the�probability� of�rolling�a�“4”?�Technically,�there�are�six�possible�outcomes�or�events�that�can�occur��One�can�
107Introduction to Probability and Sample Statistics
also�determine�how�many�times�a�specific�outcome�or�event�actually�can�occur��These�two� concepts�are�used�to�define�and�compute�the�probability�of�a�particular�outcome�or�event�by
� p A
S T
( ) =
where p(A)�is�the�probability�that�outcome�or�event�A�will�occur S�is�the�number�of�times�that�the�specific�outcome�or�event�A�can�occur T�is�the�total�number�of�outcomes�or�events�possible
Let�us�revisit�our�example,�the�probability�of�rolling�a�“4�”�A�“4”�can�occur�only�once,�thus� S�=�1��There�are�six�possible�values�that�can�be�rolled,�thus�T�=�6��Therefore�the�probability� of�rolling�a�“4”�is�determined�by
� p
S T
( )4 1 6
= =
This�assumes,�however,�that�the�die�is�unbiased,�which�means�that�the�die�is�fair�and�that� the�probability�of�obtaining�any�of�the�six�outcomes�is�the�same��For�a�fair,�unbiased�die,� the�probability�of�obtaining�any�outcome�is�1/6��Gamblers�have�been�known�to�possess�an� unfair,�biased�die�such�that�the�probability�of�obtaining�a�particular�outcome�is�different� from�1/6�(e�g�,�to�cheat�their�opponent�by�shaving�one�side�of�the�die)�
Consider�one�other�classic�probability�example��Imagine�you�have�an�urn�(or�other�con- tainer)��Inside�of�the�urn�and�out�of�view�are�a�total�of�nine�balls�(thus�T�=�9),�six�of�the�balls� being�red�(event�A;�S�=�6)�and�the�other�three�balls�being�green�(event�B;�S�=�3)��Your�task� is�to�draw�one�ball�out�of�the�urn�(without�looking)�and�then�observe�its�color��The�prob- ability�of�each�of�these�two�events�occurring�on�the�first�draw�is�as�follows:
� p A
S T
( ) = = = 6 9
2 3
� p B
S T
( ) = = = 3 9
1 3
Thus�the�probability�of�drawing�a�red�ball�is�2/3,�and�the�probability�of�drawing�a�green� ball�is�1/3�
Two�notions�become�evident�in�thinking�about�these�examples��First,�the�sum�of�the� probabilities�for�all�distinct�or�independent�events�is�precisely�1��In�other�words,�if�we� take�each�distinct�event�and�compute�its�probability,�then�the�sum�of�those�probabilities� must�be�equal�to�one�so�as�to�account�for�all�possible�outcomes��Second,�the�probability� of�any�given�event�(a)�cannot�exceed�one�and�(b)�cannot�be�less�than�zero��Part�(a)�should� be� obvious� in� that� the� sum� of� the� probabilities� for� all� events� cannot� exceed� one,� and� therefore�the�probability�of�any�one�event�cannot�exceed�one�either�(it�makes�no�sense� to�talk�about�an�event�occurring�more�than�all�of�the�time)��An�event�would�have�a�prob- ability�of�one�if�no�other�event�can�possibly�occur,�such�as�the�probability�that�you�are� currently�breathing��For�part�(b)�no�event�can�have�a�negative�probability�(it�makes�no�
108 An Introduction to Statistical Concepts
sense�to�talk�about�an�event�occurring�less�than�never);�however,�an�event�could�have� a� zero� probability� if� the� event� can� never� occur�� For� instance,� in� our� urn� example,� one� could�never�draw�a�purple�ball�
5.1.3 Intuition Versus probability
At�this�point,�you�are�probably�thinking�that�probability�is�an�interesting�topic��However,� without�extensive�training�to�think�in�a�probabilistic�fashion,�people�tend�to�let�their�intu- ition�guide�them��This�is�all�well�and�good,�except�that�intuition�can�often�guide�you�to�a� different�conclusion�than�probability��Let�us�examine�two�classic�examples�to�illustrate�this� dilemma��The�first�classic�example�is�known�as�the�“birthday�problem�”�Imagine�you�are�in� a�room�of�23�people��You�ask�each�person�to�write�down�their�birthday�(month�and�day)�on� a�piece�of�paper��What�do�you�think�is�the�probability�that�in�a�room�of�23�people�at�least� two�will�have�the�same�birthday?
Assume� first� that� we� are� dealing� with� 365� different� possible� birthdays,� where� leap� year� (February�29)�is�not�considered��Also�assume�the�sample�of�23�people�is�randomly�drawn�from� some�population�of�people��Taken�together,�this�implies�that�each�of�the�365�different�possible� birthdays�has�the�same�probability�(i�e�,�1/365)��An�intuitive�thinker�might�have�the�following� thought�processing��“There�are�365�different�birthdays�in�a�year�and�there�are�23�people�in�the� sample��Therefore�the�probability�of�two�people�having�the�same�birthday�must�be�close�to�zero�”� We�try�this�on�our�introductory�students�each�year,�and�their�guesses�are�usually�around�zero�
Intuition� has� led� us� astray,� and� we� have� not� used� the� proper� thought� processing�� True,� there�are�365�days�and�23�people��However,�the�question�really�deals�with�pairs�of�people�� There� is� a� fairly� large� number� of� different� possible� pairs� of� people� [i�e�,� person� 1� with� 2,� 1� with�3,�etc�,�where�the�total�number�of�different�pairs�of�people�is�equal�to�n(n�−�1)/2�=�23(22)/� 2 = 253]��All�we�need�is�for�one�pair�to�have�the�same�birthday��While�the�probability�compu- tations�are�a�little�complex�(see�Appendix),�the�probability�that�at�least�two�individuals�will� have�the�same�birthday�in�a�group�of�23�is�equal�to��507��That�is�right,�about�one-half�of�the� time�a�group�of�23�people�will�have�two�or�more�with�the�same�birthday��Our�introductory� classes�typically�have�between�20�and�40�students��More�often�than�not,�we�are�able�to�find� two�students�with�the�same�birthday��One�year�one�of�us�wrote�each�birthday�on�the�board�so� that�students�could�see�the�data��The�first�two�students�selected�actually�had�the�same�birth- day,�so�our�point�was�very�quickly�shown��What�was�the�probability�of�that�event�occurring?
The� second� classic� example� is� the� “gambler’s� fallacy,”� sometimes� referred� to� as� the� “law�of�averages�”�This�works�for�any�game�of�chance,�so�imagine�you�are�flipping�a�coin�� Obviously�there�are�two�possible�outcomes�from�a�coin�flip,�heads�and�tails��Assume�the� coin�is�fair�and�unbiased�such�that�the�probability�of�flipping�a�head�is�the�same�as�flipping� a�tail,�that�is,��5��After�flipping�the�coin�nine�times,�you�have�observed�a�tail�every�time�� What�is�the�probability�of�obtaining�a�head�on�the�next�flip?
An�intuitive�thinker�might�have�the�following�thought�processing��“I�have�just�observed�a� tail�each�of�the�last�nine�flips��According�to�the�law�of�averages,�the�probability�of�observing�a� head�on�the�next�flip�must�be�near�certainty��The�probability�must�be�nearly�one�”�We�also�try� this�on�our�introductory�students�every�year,�and�their�guesses�are�almost�always�near�one�
Intuition� has� led� us� astray� once� again� as� we� have� not� used� the� proper� thought� pro- cessing��True,�we�have�just�observed�nine�consecutive�tails��However,�the�question�really� deals� with� the� probability� of� the� 10th� flip� being� a� head,� not� the� probability� of� obtaining� 10�consecutive�tails��The�probability�of�a�head�is�always��5�with�a�fair,�unbiased�coin��The� coin�has�no�memory;�thus�the�probability�of�tossing�a�head�after�nine�consecutive�tails�is� the�same�as�the�probability�of�tossing�a�head�after�nine�consecutive�heads,��5��In�technical�
109Introduction to Probability and Sample Statistics
terms,�the�probabilities�of�each�event�(each�toss)�are�independent�of�one�another��In�other� words,�the�probability�of�flipping�a�head�is�the�same�regardless�of�the�preceding�flips��This� is� not� the� same� as� the� probability� of� tossing� 10� consecutive� heads,� which� is� rather� small� (approximately��0010)��So�when�you�are�gambling�at�the�casino�and�have�lost�the�last�nine� games,�do�not�believe�that�you�are�guaranteed�to�win�the�next�game��You�can�just�as�easily� lose�game�10�as�you�did�game�1��The�same�goes�if�you�have�won�a�number�of�games��You� can�just�as�easily�win�the�next�game�as�you�did�game�1��To�some�extent,�the�casinos�count� on�their�customers�playing�the�gambler’s�fallacy�to�make�a�profit�
5.2 Sampling and Estimation
In�Chapter�3,�we�spent�some�time�discussing�sample�statistics,�including�the�measures�of� central�tendency�and�dispersion��In�this�section,�we�expand�upon�that�discussion�by�defin- ing�inferential�statistics,�describing�different�types�of�sampling,�and�then�moving�into�the� implications�of�such�sampling�in�terms�of�estimation�and�sampling�distributions�
Consider� the� situation� where� we� have� a� population� of� graduate� students�� Population� parameters�(characteristics�of�a�population)�could�be�determined,�such�as�the�population� size N,� the� population� mean� μ,� the� population� variance� σ2,� and� the� population� standard� deviation�σ��Through�some�method�of�sampling,�we�then�take�a�sample�of�students�from� this�population��Sample�statistics�(characteristics�of�a�sample)�could�be�determined,�such� as�the�sample�size�n,�the�sample�mean�X
– ,�the�sample�variance�s2,�and�the�sample�standard�
deviation�s� How� often� do� we� actually� ever� deal� with� population� data?� Except� when� dealing� with�
very� small,� well-defined� populations,� we� almost� never� deal� with� population� data�� The� main�reason�for�this�is�cost,�in�terms�of�time,�personnel,�and�economics��This�means�then� that� we� are� almost� always� dealing� with� sample� data�� With� descriptive� statistics,� dealing� with�sample�data�is�very�straightforward,�and�we�only�need�to�make�sure�we�are�using�the� appropriate�sample�statistic�equation��However,�what�if�we�want�to�take�a�sample�statistic� and�make�some�generalization�about�its�relevant�population�parameter?�For�example,�you� have�computed�a�sample�mean�on�grade�point�average�(GPA)�of�X
– �=�3�25�for�a�sample�of�25�
graduate�students�at�State�University��You�would�like�to�make�some�generalization�from� this� sample� mean� to� the� population� mean� μ� at� State� University�� How� do� we� do� this?� To� what�extent�can�we�make�such�a�generalization?�How�confident�are�we�that�this�sample� mean�actually�represents�the�population�mean?
This� brings� us� to� the� field� of� inferential� statistics�� We� define� inferential statistics� as� statistics�that�allow�us�to�make�an�inference�or�generalization�from�a�sample�to�the�popu- lation�� In� terms� of� reasoning,� inductive� reasoning� is� used� to� infer� from� the� specific� (the� sample)� to� the� general� (the� population)�� Thus� inferential� statistics� is� the� answer� to� all� of� our�preceding�questions�about�generalizing�from�sample�statistics�to�population�param- eters��How�the�sample�is�derived,�however,�is�important�in�determining�to�what�extent�the� statistical�results�we�derive�can�be�inferred�from�the�sample�back�to�the�population��Thus,� it�is�important�to�spend�a�little�time�talking�about�simple�random�sampling,�the�only�sam- pling�procedure�that�allows�generalizations�to�be�made�from�the�sample�to�the�population�� (Although�there�are�statistical�means�to�correct�for�non-simple�random�samples,�they�are� beyond� the� scope� of� this� textbook�)� In� the� remainder� of� this� section,� and� in� much� of� the� remainder� of� this� text,� we� take� up� the� details� of� inferential� statistics� for� many� different� procedures�
110 An Introduction to Statistical Concepts
5.2.1 Simple Random Sampling
There�are�several�different�ways�in�which�a�sample�can�be�drawn�from�a�population�� In�this�section�we�introduce�simple�random�sampling,�which�is�a�commonly�used�type� of� sampling� and� which� is� also� assumed� for� many� inferential� statistics� (beginning� in� Chapter� 6)�� Simple random sampling� is� defined� as� the� process� of� selecting� sample� observations�from�a�population�so�that�each�observation�has�an�equal�and�independent� probability� of� being� selected�� If� the� sampling� process� is� truly� random,� then� (a)� each� observation� in� the� population� has� an� equal� chance� of� being� included� in� the� sample,� and�(b)�each�observation�selected�into�the�sample�is�independent�of�(or�not�affected�by)� every�other�selection��Thus�a�volunteer�or�“street-corner”�sample�would�not�meet�the� first�condition�because�members�of�the�population�who�do�not�frequent�that�particular� street�corner�have�no�chance�of�being�included�in�the�sample�
In� addition,� if� the� selection� of� spouses� required� the� corresponding� selection� of� their� respective�mates,�then�the�second�condition�would�not�be�met��For�example,�if�the�selection� of�Mr��Joe�Smith�III�also�required�the�selection�of�his�wife,�then�these�two�selections�are�not� independent�of�one�another��Because�we�selected�Mr��Joe�Smith�III,�we�must�also�therefore� select�his�wife��Note�that�through�independent�sampling�it�is�possible�for�Mr��Smith�and� his�wife�to�both�be�sampled,�but�it�is�not�required��Thus,�independence�implies�that�each� observation�is�selected�without�regard�to�any�other�observation�sampled�
We� also� would� fail� to� have� equal� and� independent� probability� of� selection� if� the� sam- pling�procedure�employed�was�something�other�than�a�simple�random�sample—because�it� is�only�with�a�simple�random�sample�that�we�have�met�the�conditions�(a)�and�(b)�presented� earlier� in� the� paragraph�� (Although� there� are� statistical� means� to� correct� for� non-simple� random�samples,�they�are�beyond�the�scope�of�this�textbook�)�This�concept�of�independence� is� an� important� assumption� that� we� will� become� acquainted� with� more� in� the� remain- ing�chapters��If�we�have�independence,�then�generalizations�from�the�sample�back�to�the� population� can� be� made� (you� may� remember� this� as� external validity� which� was� likely� introduced� in� your� research� methods� course)� (see� Figure� 5�1)�� Because� of� the� connection� between�simple�random�sampling�and�independence,�let�us�expand�our�discussion�on�the� two�types�of�simple�random�sampling�
5.2.1.1 Simple Random Sampling With Replacement
There�are�two�specific�types�of�simple�random�sampling��Simple random sampling with replacement�is�conducted�as�follows��The�first�observation�is�selected�from�the�population� into�the�sample,�and�that�observation�is�then�replaced�back�into�the�population��The�second� observation�is�selected�and�then�replaced�in�the�population��This�continues�until�a�sample� of� the� desired� size� is� obtained�� The� key� here� is� that� each� observation� sampled� is� placed� back�into�the�population�and�could�be�selected�again�
This�scenario�makes�sense�in�certain�applications�and�not�in�others��For�example,�return� to�our�coin�flipping�example�where�we�now�want�to�flip�a�coin�100�times�(i�e�,�a�sample�size� of�100)��How�does�this�operate�in�the�context�of�sampling?�We�flip�the�coin�(e�g�,�heads)�and� record�the�result��This�“head”�becomes�the�first�observation�in�our�sample��This�observa- tion� is� then� placed� back� into� the� population�� Then� a� second� observation� is� made� and� is� placed�back�into�the�population��This�continues�until�our�sample�size�requirement�of�100�is� reached��In�this�particular�scenario�we�always�sample�with�replacement,�and�we�automati- cally�do�so�even�if�we�have�never�heard�of�sampling�with�replacement��If�no�replacement� took�place,�then�we�could�only�ever�have�a�sample�size�of�two,�one�“head”�and�one�“tail�”
111Introduction to Probability and Sample Statistics
5.2.1.2 Simple Random Sampling Without Replacement
In� other� scenarios,� sampling� with� replacement� does� not� make� sense�� For� example,� say� we�are�conducting�a�poll�for�the�next�major�election�by�randomly�selecting�100�students� (the� sample)� at� a� local� university� (the� population)�� As� each� student� is� selected� into� the� sample,�they�are�removed�and�cannot�be�sampled�again��It�simply�would�make�no�sense� if� our� sample� of� 100� students� only� contained� 78� different� students� due� to� replacement� (as� some� students� were� polled� more� than� once)�� Our� polling� example� represents� the� other�type�of�simple�random�sampling,�this�time�without�replacement��Simple random sampling without replacement� is� conducted� in� a� similar� fashion� except� that� once� an� observation� is� selected� for� inclusion� in� the� sample,� it� is� not� replaced� and� cannot� be� selected�a�second�time�
5.2.1.3 Other Types of Sampling
There� are� several� other� types� of� sampling�� These� other� types� of� sampling� include� con- venient� sampling� (i�e�,� volunteer� or� “street-corner”� sampling� previously� mentioned),� systematic� sampling� (e�g�,� select� every� 10th� observation� from� the� population� into� the� sample),�cluster�sampling�(i�e�,�sample�groups�or�clusters�of�observations�and�include�all� members�of�the�selected�clusters�in�the�sample),�stratified�sampling�(i�e�,�sampling�within� subgroups� or� strata� to� ensure� adequate� representation� of� each� strata),� and� multistage� sampling�(e�g�,�stratify�at�one�stage�and�randomly�sample�at�another�stage)��These�types� of� sampling� are� beyond� the� scope� of� this� text,� and� the� interested� reader� is� referred� to� sampling� texts� such� as� Sudman� (1976),� Kalton� (1983),� Jaeger� (1984),� Fink� (1995),� or� Levy� and�Lemeshow�(1999)�
Step 1: Population
Step 2: Draw simple
random sample
Step 3: Compute
sample statistics
Step 4: Make
inference back to the population
FIGuRe 5.1 Cycle�of�inference�
112 An Introduction to Statistical Concepts
5.2.2 estimation of population parameters and Sampling distributions
Take� as� an� example� the� situation� where� we� select� one� random� sample� of� n� females� (e�g�,� n = 20),�measure�their�weight,�and�then�compute�the�mean�weight�of�the�sample��We�find� the�mean�of�this�first�sample�to�be�102�pounds�and�denote�it�by�X
– 1�=�102,�where�the�subscript�
identifies�the�first�sample��This�one�sample�mean�is�known�as�a�point�estimate�of�the�popu- lation�mean�μ,�as�it�is�simply�one�value�or�point��We�can�then�proceed�to�collect�weight�data� from�a�second�sample�of�n�females�and�find�that�X
– 2�=�110��Next�we�collect�weight�data�from�
a�third�sample�of�n�females�and�find�that�X –
3�=�119��Imagine�that�we�go�on�to�collect�such�data� from�many�other�samples�of�size�n�and�compute�a�sample�mean�for�each�of�those�samples�
5.2.2.1 Sampling Distribution of the Mean
At� this� point,� we� have� a� collection� of� sample� means,� which� we� can� use� to� construct� a� frequency�distribution�of�sample�means��This�frequency�distribution�is�formally�known� as�the�sampling distribution of the mean��To�better�illustrate�this�new�distribution,�let� us�take�a�very�small�population�from�which�we�can�take�many�samples��Here�we�define� our�population�of�observations�as�follows:�1,�2,�3,�5,�9�(in�other�words,�we�have�five�values� in� our� population)�� As� the� entire� population� is� known� here,� we� can� better� illustrate� the� important�underlying�concepts��We�can�determine�that�the�population�mean�μX�=�4�and� the�population�variance�σX
2 �=�8,�where�X�indicates�the�variable�we�are�referring�to��Let�us�
first�take�all�possible�samples�from�this�population�of�size�2�(i�e�,�n�=�2)�with�replacement�� As� there� are� only� five� observations,� there� will� be� 25� possible� samples� as� shown� in� the� upper�portion�of�Table�5�1,�called�“Samples�”�Each�entry�represents�the�two�observations� for�a�particular�sample��For�instance,�in�row�1�and�column�4,�we�see�1,5��This�indicates�that� the�first�observation�is�a�1�and�the�second�observation�is�a�5��If�sampling�was�done�without� replacement,�then�the�diagonal�of�the�table�from�upper�left�to�lower�right�would�not�exist�� For�instance,�a�1,1�sample�could�not�be�selected�if�sampling�without�replacement�
Now� that� we� have� all� possible� samples� of� size� 2,� let� us� compute� the� sample� means� for� each� of� the� 25� samples�� The� sample� means� are� shown� in� the� middle� portion� of� Table� 5�1,� called�“Sample�means�”�Just�eyeballing�the�table,�we�see�the�means�range�from�1�to�9�with� numerous�different�values�in�between��We�then�compute�the�mean�of�the�25�sample�means� to�be�4,�as�shown�in�the�bottom�portion�of�Table�5�1,�called�“Mean�of�the�sample�means�”
This� is� a� matter� for� some� discussion,� so� consider� the� following� three� points�� First,� the� distribution�of�X
– �for�all�possible�samples�of�size�n�is�known�as�the�sampling�distribution�
of�the�mean��In�other�words,�if�we�were�to�take�all�of�the�“sample�mean”�values�in�Table�5�1� and�construct�a�histogram�of�those�values,�then�that�is�what�is�referred�to�as�a�“sampling� distribution�of�the�mean�”�It�is�simply�the�distribution�(i�e�,�histogram)�of�all�the�“sample� mean”�values��Second,�the�mean�of�the�sampling�distribution�of�the�mean�for�all�possible� samples�of�size�n�is�equal�to�μX–��As�the�mean�of�the�sampling�distribution�of�the�mean�is� denoted�by�μX–�(the�mean�of�the�X
– s),�then�we�see�for�the�example�that�μX–�=�μX�=�4��In�other�
words,�the�mean�of�the�sampling�distribution�of�the�mean�is�simply�the�average�of�all�of� the�“sample�means”�in�Table�5�1��The�mean�of�the�sampling�distribution�of�the�mean�will� always�be�equal�to�the�population�mean�
Third,�we�define�sampling error�in�this�context�as�the�difference�(or�deviation)�between� a�particular�sample�mean�and�the�population�mean,�denoted�as�X
– �−�μX��A�positive�sam-
pling�error�indicates�a�sample�mean�greater�than�the�population�mean,�where�the�sam- ple�mean�is�known�as�an�overestimate�of�the�population�mean��A�zero�sampling�error� indicates� a� sample� mean� exactly� equal� to� the� population� mean�� A� negative� sampling�
113Introduction to Probability and Sample Statistics
error�indicates�a�sample�mean�less�than�the�population�mean,�where�the�sample�mean� is� known� as� an� underestimate� of� the� population� mean�� As� a� researcher,� we� want� the� sampling�error�to�be�as�close�to�zero�as�possible�to�suggest�that�the�sample�reflects�the� population�well�
5.2.2.2 Variance Error of the Mean
Now�that�we�have�a�measure�of�the�mean�of�the�sampling�distribution�of�the�mean,�let�us� consider�the�variance�of�this�distribution��We�define�the�variance�of�the�sampling�distribu- tion�of�the�mean,�known�as�the�variance error of the mean,�as� σX
2 ��This�will�provide�us�
with� a� dispersion� measure� of� the� extent� to� which� the� sample� means� vary� and� will� also� provide�some�indication�of�the�confidence�we�can�place�in�a�particular�sample�mean��The� variance�error�of�the�mean�is�computed�as
� σ
σ X
X
n 2
2
=
where σX
2 �is�the�population�variance�of�X
n�is�the�sample�size
Table 5.1
All�Possible�Samples�and�Sample�Means�for�n�=�2�from�the�Population�of�1,�2,�3,�5,�9
First Observation Second Observation
Samples 1 2 3 5 9
1 1,1 1,2 1,3 1,5 1,9 2 2,1 2,2 2,3 2,5 2,9 3 3,1 3,2 3,3 3,5 3,9 5 5,1 5,2 5,3 5,5 5,9 9 9,1 9,2 9,3 9,5 9,9
Sample Means
1 1�0 1�5 2�0 3�0 5�0 2 1�5 2�0 2�5 3�5 5�5 3 2�0 2�5 3�0 4�0 6�0 5 3�0 3�5 4�0 5�0 7�0 9 5�0 5�5 6�0 7�0 9�0
X =∑ 12 5. X =∑ 15 0. X =∑ 17 5. X =∑ 22 5. X =∑ 32 5. Mean�of�the�sample�means:
µX X
number of samples = = =
∑ 100 25
4 0.
Variance�of�the�sample�means:
σX number of samples X X
number of samples 2
2 2
2
25 500 =
− ( ) =
−∑ ∑( ) ( )
( ) (1100 25
25 500 10 000 25
4 0 2
2 2
) ( )
( ) , ( )
.= −
=
114 An Introduction to Statistical Concepts
For�the�example,�we�have�already�determined�that�σX 2
�=�8�and�that�n�=�2;�therefore,
� σ
σ X
X
n 2
2 8 2
4= = =
This�is�verified�in�the�bottom�portion�of�Table�5�1,�called�“Variance�of�the�sample�means,”� where�the�variance�error�is�computed�from�the�collection�of�sample�means�
What�will�happen�if�we�increase�the�size�of�the�sample?�If�we�increase�the�sample�size�to� n�=�4,�then�the�variance�error�is�reduced�to�2��Thus�we�see�that�as�the�size�of�the�sample�n� increases,�the�magnitude�of�the�sampling�error�decreases��Why?�Conceptually,�as�sample� size�increases,�we�are�sampling�a�larger�portion�of�the�population��In�doing�so,�we�are�also� obtaining� a� sample� that� is� likely� more� representative� of� the� population�� In� addition,� the� larger�the�sample�size,�the�less�likely�it�is�to�obtain�a�sample�mean�that�is�far�from�the�popu- lation�mean��Thus,�as�sample�size�increases,�we�hone�in�closer�and�closer�to�the�population� mean�and�have�less�and�less�sampling�error�
For�example,�say�we�are�sampling�from�a�voting�district�with�a�population�of�5000�vot- ers��A�survey�is�developed�to�assess�how�satisfied�the�district�voters�are�with�their�local� state�representative��Assume�the�survey�generates�a�100-point�satisfaction�scale��First�we� determine�that�the�population�mean�of�satisfaction�is�75��Next�we�take�samples�of�different� sizes��For�a�sample�size�of�1,�we�find�sample�means�that�range�from�0�to�100�(i�e�,�each�mean� really�only�represents�a�single�observation)��For�a�sample�size�of�10,�we�find�sample�means� that�range�from�50�to�95��For�a�sample�size�of�100,�we�find�sample�means�that�range�from� 70�to�80��We�see�then�that�as�sample�size�increases,�our�sample�means�become�closer�and� closer�to�the�population�mean,�and�the�variability�of�those�sample�means�becomes�smaller� and�smaller�
5.2.2.3 Standard Error of the Mean
We� can� also� compute� the� standard� deviation� of� the� sampling� distribution� of� the� mean,� known�as�the�standard error of the mean,�by
� σ
σ X
X
n =
Thus�for�the�example�we�have
� σ
σ X
X
n = = =
2 8284 2
2 .
Because�the�applied�researcher�typically�does�not�know�the�population�variance,�the�pop- ulation�variance�error�of�the�mean�and�the�population�standard�error�of�the�mean�can�be� estimated�by�the�following,�respectively:
� s
s nX X2 2
=
115Introduction to Probability and Sample Statistics
and
� s
s n
X X=
5.2.2.4 Confidence Intervals
Thus� far� we� have� illustrated� how� a� sample� mean� is� a� point estimate� of� the� popula- tion� mean� and� how� a� variance� error� gives� us� some� sense� of� the� variability� among� the� sample� means�� Putting� these� concepts� together,� we� can� also� build� an� interval estimate� for� the� population� mean� to� give� us� a� sense� of� how� confident� we� are� in� our� particular�sample�mean��We�can�form�a�confidence interval (CI)�around�a�particular� sample�mean�as�follows��As�we�learned�in�Chapter�4,�for�a�normal�distribution,�68%�of� the�distribution�falls�within�one�standard�deviation�of�the�mean��A�68%�CI�of�a�sample� mean�can�be�formed�as follows:
� 68% CI = ±X Xσ
Conceptually,� this� means� that� if� we� form� 68%� CIs� for� 100� sample� means,� then� 68� of� those� 100� intervals� would� contain� or� include� the� population� mean� (it� does� not� mean� that� there� is� a� 68%� probability� of� the� interval� containing� the� population� mean—the� interval� either� contains� it� or� does� not)�� Because� the� applied� researcher� typically� only� has�one�sample�mean�and�does�not�know�the�population�mean,�he�or�she�has�no�way� of�knowing�if�this�one�CI�actually�contains�the�population�mean�or�not��If�one�wanted� to�be�more�confident�in�a�sample�mean,�then�a�90%�CI,�a�95%�CI,�or�a�99%�CI�could�be� formed�as�follows:
� 90 1 645% CI .= ±X Xσ
� 95 1 96% CI .= ±X Xσ
� 99 2 5758% CI .= ±X Xσ
Thus�for�the�90%�CI,�the�population�mean�will�be�contained�in�90�out�of�100�CIs;�for�the� 95%�CI,�the�population�mean�will�be�contained�in�95�out�of�100�CIs;�and�for�the�99%�CI,�the� population�mean�will�be�contained�in�99�out�of�100�CIs��The�critical�values�of�1�645,�1�96,� and�2�5758�come�from�the�standard�unit�normal�distribution�table�(Table�A�1)�and�indicate� the�width�of�the�CI��Wider�CIs,�such�as�the�99%�CI,�enable�greater�confidence��For�example,� with�a�sample�mean�of�70�and�a�standard�error�of�the�mean�of�3,�the�following�CIs�result:� 68%�CI�=�(67,�73)�[i�e�,�ranging�from�67�to�73];�90%�CI�=�(65�065,�74�935);�95%�CI�=�(64�12,�75�88);� and�99%�CI�=�(62�2726,�77�7274)��We�can�see�here�that�to�be�assured�that�99%�of�the�CIs�con- tain�the�population�mean,�then�our�interval�must�be�wider�(i�e�,�ranging�from�about�62�27�to� 77�73,�or�a�range�of�about�15)�than�the�CIs�that�are�lesser�(e�g�,�the�95%�CI�ranges�from�64�12� to�75�88,�or�a�range�of�about�11)�
116 An Introduction to Statistical Concepts
In�general,�a�CI�for�any�level�of�confidence�(i�e�,�XX%�CI)�can�be�computed�by�the�follow- ing�general�formula:
� XX X zcv X% CI = ± σ
where�zcv�is�the�critical� value�taken� from�the�standard�unit�normal�distribution�table�for� that�particular�level�of�confidence,�and�the�other�values�are�as�before�
5.2.2.5 Central Limit Theorem
In�our�discussion�of�CIs,�we�used�the�normal�distribution�to�help�determine�the�width� of�the�intervals��Many�inferential�statistics�assume�the�population�distribution�is�nor- mal� in� shape�� Because� we� are� looking� at� sampling� distributions� in� this� chapter,� does� the�shape�of�the�original�population�distribution�have�any�relationship�to�the�sampling� distribution�of�the�mean�we�obtain?�For�example,�if�the�population�distribution�is�non- normal,� what� form� does� the� sampling� distribution� of� the� mean� take� (i�e�,� is� the� sam- pling�distribution�of�the�mean�also�nonnormal)?�There�is�a�nice�concept,�known�as�the� central limit theorem,�to�assist�us�here��The�central�limit�theorem�states�that�as�sample� size�n�increases,�the�sampling�distribution�of�the�mean�from�a�random�sample�of�size� n� more� closely� approximates� a� normal� distribution�� If� the� population� distribution� is� normal�in�shape,�then�the�sampling�distribution�of�the�mean�is�also�normal�in�shape�� If� the� population� distribution� is� not� normal� in� shape,� then� the� sampling� distribution� of� the� mean� becomes� more� nearly� normal� as� sample� size� increases�� This� concept� is� graphically�depicted�in�Figure�5�2�
The�top�row�of�the�figure�depicts�two�population�distributions,�the�left�one�being�normal� and�the�right�one�being�positively�skewed��The�remaining�rows�are�for�the�various�sam- pling� distributions,� depending� on� the� sample� size�� The� second� row� shows� the� sampling� distributions�of�the�mean�for�n�=�1��Note�that�these�sampling�distributions�look�precisely� like�the�population�distributions,�as�each�observation�is�literally�a�sample�mean��The�next� row�gives�the�sampling�distributions�for�n�=�2;�here�we�see�for�the�skewed�population�that� the�sampling�distribution�is�slightly�less�skewed��This�is�because�the�more�extreme�obser- vations�are�now�being�averaged�in�with�less�extreme�observations,�yielding�less�extreme�
Normal Positively skewed
Population
------------------------------------------------------------------
n = 1
n = 2
n = 4
n = 25
FIGuRe 5.2 Central�limit�theorem�for�normal�and�positively�skewed�population�distributions�
117Introduction to Probability and Sample Statistics
means��For�n�=�4,�the�sampling�distribution�in�the�skewed�case�is�even�less�skewed�than�for� n = 2��Eventually�we�reach�the�n�=�25�sampling�distribution,�where�the�sampling�distribu- tion� for� the� skewed� case� is� nearly� normal� and� nearly� matches� the� sampling� distribution� for�the�normal�case��This�phenomenon�will�occur�for�other�nonnormal�population�distri- butions�as�well�(e�g�,�negatively�skewed)��The�morale�of�the�story�here�is�a�good�one��If�the� population�distribution�is�nonnormal,�then�this�will�have�minimal�effect�on�the�sampling� distribution� of� the� mean� except� for� rather� small� samples�� This� can� come� into� play� with� inferential�statistics�when�the�assumption�of�normality�is�not�satisfied,�as�we�see�in�later� chapters�
5.3 Summary
In� this� chapter,� we� began� to� move� from� descriptive� statistics� to� the� realm� of� inferential� statistics�
The� two� main� topics� we� considered� were� probability,� and� sampling� and� estimation�� First� we� briefly� introduced� probability� by� looking� at� the� importance� of� probability� in� statistics,� defining� probability,� and� comparing� conclusions� often� reached� by� intuition� versus�probability��The�second�topic�involved�sampling�and�estimation,�a�topic�we�return� to�in�most�of�the�remaining�chapters��In�the�sampling�section,�we�defined�and�described� simple�random�sampling,�both�with�and�without�replacement,�and�briefly�outlined�other� types�of�sampling��In�the�estimation�section,�we�examined�the�sampling�distribution�of� the� mean,� the� variance� and� standard� error� of� the� mean,� CIs� around� the� mean,� and� the� central�limit�theorem��At�this�point�you�should�have�met�the�following�objectives:�(a)�be� able�to�understand�the�most�basic�concepts�of�probability,�(b)�be�able�to�understand�and� conduct� simple� random� sampling,� and� (c)� be� able� to� understand,� determine,� and� inter- pret�the�results�from�the�estimation�of�population�parameters�via�a�sample��In�the�next� chapter� we� formally� discuss� our� first� inferential� statistics� situation,� testing� hypotheses� about�a�single�mean�
Appendix: Probability That at Least Two Individuals Have the Same Birthday
This� probability� can� be� shown� by� either� of� the� following� equations�� Note� that� there� are� n = 23�individuals�in�the�room��One�method�is�as�follows:
� 1
365 364 363 365 1 365
1 365 364 363 343
365 5023−
× × × × − + = −
× × × × =
� �( ) .
n n 77
An�equivalent�method�is�as�follows:
� 1
365 365
364 365
363 365
365 1 365
1 365 365
364 36
− × × × × − +
= − ×�
( )n 55
363 365
343 365
507× × ×
=� .
118 An Introduction to Statistical Concepts
Problems
Conceptual problems
5.1� The�standard�error�of�the�mean�is�which�one�of�the�following? � a�� Standard�deviation�of�a�sample�distribution � b�� Standard�deviation�of�the�population�distribution � c�� Standard�deviation�of�the�sampling�distribution�of�the�mean � d�� Mean�of�the�sampling�distribution�of�the�standard�deviation 5.2� An�unbiased�six-sided�die�is�tossed�on�two�consecutive�trials,�and�the�first�toss�results�
in�a�“2�”�What�is�the�probability�that�a�“2”�will�result�on�the�second�toss? � a�� Less�than�1/6 � b�� 1/6 � c�� More�than�1/6 � d�� Cannot�be�determined 5.3� An�urn�contains�9�balls:�3�green,�4�red,�and�2�blue��The�probability�that�a�ball�selected�
at�random�is�blue�is�equal�to�which�one�of�the�following? � a�� 2/9 � b�� 5/9 � c�� 6/9 � d�� 7/9 5.4� Sampling�error�is�which�one�of�the�following? � a�� The�amount�by�which�a�sample�mean�is�greater�than�the�population�mean � b�� The�amount�of�difference�between�a�sample�statistic�and�a�population�parameter � c�� The�standard�deviation�divided�by�the�square�root�of�n � d�� When�the�sample�is�not�drawn�randomly 5.5� What�does�the�central�limit�theorem�state? � a�� The� means� of� many� random� samples� from� a� population� will� be� normally�
distributed� � b�� The�raw�scores�of�many�natural�events�will�be�normally�distributed� � c�� z�scores�will�be�normally�distributed� � d�� None�of�the�above� 5.6� For� a� normal� population,� the� variance� of� the� sampling� distribution� of� the� mean�
increases�as�sample�size�increases��True�or�false? 5.7� All� other� things� being� equal,� as� the� sample� size� increases,� the� standard� error� of� a�
statistic�decreases��True�or�false? 5.8� I�assert�that�the�95%�CI�has�a�larger�(or�wider)�range�than�the�99%�CI�for�the�same�
parameter�using�the�same�data��Am�I�correct? 5.9� I�assert�that�the�90%�CI�has�a�smaller�(or�more�narrow)�range�than�the�68%�CI�for�the�
same�parameter�using�the�same�data��Am�I�correct?
119Introduction to Probability and Sample Statistics
5.10� I�assert�that�the�mean�and�median�of�any�random�sample�drawn�from�a�symmetric� population�distribution�will�be�equal��Am�I�correct?
5.11� A�random�sample�is�to�be�drawn�from�a�symmetric�population�with�mean�100�and� variance�225��I�assert�that�the�sample�mean�is�more�likely�to�have�a�value�larger�than� 105�if�the�sample�size�is�16�than�if�the�sample�size�is�25��Am�I�correct?
5.12� A� gambler� is� playing� a� card� game� where� the� known� probability� of� winning� is� �40� (win�40%�of�the�time)��The�gambler�has�just�lost�10�consecutive�hands��What�is�the� probability�of�the�gambler�winning�the�next�hand?
� a�� Less�than��40 � b�� Equal�to��40 � c�� Greater�than��40 � d�� Cannot�be�determined�without�observing�the�gambler
5.13� On� the� evening� news,� the� anchorwoman� announces� that� the� state’s� lottery� has� reached�$72�billion�and�reminds�the�viewing�audience�that�there�has�not�been�a�win- ner�in�over�5�years��In�researching�lottery�facts,�you�find�a�report�that�states�the�prob- ability� of� winning� the� lottery� is� 1� in� 2� million� (i�e�,� a� very,� very� small� probability)�� What�is�the�probability�that�you�will�win�the�lottery?
� a�� Less�than�1�in�2�million � b�� Equal�to�1�in�2�million � c�� Greater�than�1�in�2�million � d�� Cannot�be�determined�without�additional�statistics
5.14� The�probability�of�being�selected�into�a�sample�is�the�same�for�every�individual�in�the� population�for�the�convenient�method�of�sampling��True�or�false?
5.15� Malani�is�conducting�research�on�elementary�teacher�attitudes�toward�changes� in�mathematics�standards��Malani’s�population�consists�of�all�elementary�teach- ers� within� one� district� in� the� state�� Malani� wants� her� sampling� method� to� be� such� that� every� teacher� in� the� population� has� an� equal� and� independent� prob- ability� of� selection�� Which� of� the� following� is� the� most� appropriate� sampling� method?
� a�� Convenient�sampling � b�� Simple�random�sampling�with�replacement � c�� Simple�random�sampling�without�replacement � d�� Systematic�sampling
5.16� Sampling�error�increases�with�larger�samples��True�or�false?
5.17� If�a�population�distribution�is�highly�positively�skewed,�then�the�distribution�of�the� sample�means�for�samples�of�size�500�will�be
� a�� Highly�negatively�skewed � b�� Highly�positively�skewed � c�� Approximately�normally�distributed � d�� Cannot�be�determined�without�further�information
120 An Introduction to Statistical Concepts
Computational problems
5.1� The�population�distribution�of�variable�X,�the�number�of�pets�owned,�consists�of�the� five�values�of�1,�4,�5,�7,�and�8�
� a�� Calculate�the�values�of�the�population�mean�and�variance� � b�� List�all�possible�samples�of�size�2�where�samples�are�drawn�with�replacement� � c�� Calculate�the�values�of�the�mean�and�variance�of�the�sampling�distribution�of�the�
mean� 5.2� The�following�is�a�random�sampling�distribution�of�the�mean�number�of�children�for�
samples�of�size�3,�where�samples�are�drawn�with�replacement�
Sample Mean f
1 1 2 2 3 4 4 2 5 1
� a�� What�is�the�population�mean? � b�� What�is�the�population�variance? � c�� What�is�the�mean�of�the�sampling�distribution�of�the�mean? � d�� What�is�the�variance�error�of�the�mean? 5.3� In�a�study�of�the�entire�student�body�of�a�large�university,�if�the�standard�error�of�the�
mean�is�20�for�n�=�16,�what�must�the�sample�size�be�to�reduce�the�standard�error�to�5? 5.4� A�random�sample�of�13�statistics�texts�had�a�mean�number�of�pages�of�685�and�a�stan-
dard�deviation�of�42��First�calculate�the�standard�error�of�the�mean��Then�calculate� the�95%�CI�for�the�mean�length�of�statistics�texts�
5.5� A�random�sample�of�10�high�schools�employed�a�mean�number�of�guidance�counsel- ors�of�3�and�a�standard�deviation�of�2��First�calculate�the�standard�error�of�the�mean�� Then�calculate�the�90%�CI�for�the�mean�number�of�guidance�counselors�
Interpretive problems
5.1� Take�a�six-sided�die,�where�the�population�values�are�obviously�1,�2,�3,�4,�5,�and�6��Take� 20�samples,�each�of�size�2�(e�g�,�every�two�rolls�is�one�sample)��For�each�sample,�calcu- late�the�mean��Then�determine�the�mean�of�the�sampling�distribution�of�the�mean�and� the�variance�error�of�the�mean��Compare�your�results�to�those�of�your�colleagues�
5.2� You�will�need�20�plain�M&M�candy�pieces�and�one�cup��Put�the�candy�pieces�in�the� cup�and�toss�them�onto�a�flat�surface��Count�the�number�of�candy�pieces�that�land� with�the�“M”�facing�up��Write�down�that�number��Repeat�these�steps�five�times��These� steps� will� constitute� one sample�� Next,� generate� four� additional� samples� (i�e�,� repeat� the�process�of�tossing�the�candy�pieces,�counting�the�“Ms,”�and�writing�down�that� number)��Then�determine�the�mean�of�the�sampling�distribution�of�the�mean�and�the� variance�error�of�the�mean��Compare�your�results�to�those�of�your�colleagues�
121
6 Introduction to Hypothesis Testing: Inferences About a Single Mean
Chapter Outline
6�1� Types�of�Hypotheses 6�2� Types�of�Decision�Errors
6�2�1� Example�Decision-Making�Situation 6�2�2� Decision-Making�Table
6�3� Level�of�Significance�(α) 6�4� Overview�of�Steps�in�Decision-Making�Process 6�5� Inferences�About�μ�When�σ�Is�Known
6�5�1� z�Test 6�5�2� Example 6�5�3� Constructing�Confidence�Intervals�Around�the�Mean
6�6� Type�II�Error�(β)�and�Power�(1�−�β) 6�6�1� Full�Decision-Making�Context 6�6�2� Power�Determinants
6�7� Statistical�Versus�Practical�Significance 6�8� Inferences�About�μ�When�σ�Is�Unknown
6�8�1� New�Test�Statistic�t 6�8�2� t�Distribution 6�8�3� t�Test 6�8�4� Example
6�9� SPSS 6�10� G*Power 6�11� Template�and�APA-Style�Write-Up
Key Concepts
� 1�� Null�or�statistical�hypothesis�versus�scientific�or�research�hypothesis � 2�� Type�I�error�(α),�type�II�error�(β),�and�power�(1�−�β) � 3�� Two-tailed�versus�one-tailed�alternative�hypotheses � 4�� Critical�regions�and�critical�values
122 An Introduction to Statistical Concepts
� 5�� z�test�statistic � 6�� Confidence�interval�(CI)�around�the�mean � 7�� t�test�statistic � 8�� t�distribution,�degrees�of�freedom,�and�table�of�t�distributions
In�Chapter�5,�we�began�to�move�into�the�realm�of�inferential�statistics��There�we�considered� the� following� general� topics:� probability,� sampling,� and� estimation�� In� this� chapter,� we� move�totally�into�the�domain�of�inferential�statistics,�where�the�concepts�involved�in�prob- ability,�sampling,�and�estimation�can�be�implemented��The�overarching�theme�of�the�chap- ter�is�the�use�of�a�statistical�test�to�make�inferences�about�a�single�mean��In�order�to�properly� cover� this� inferential� test,� a� number� of� basic� foundational� concepts� are� described� in� this� chapter�� Many� of� these� concepts� are� utilized� throughout� the� remainder� of� this� text�� The� topics�described�include�the�following:�types�of�hypotheses,�types�of�decision�errors,�level� of� significance� (α),� overview� of� steps� in� the� decision-making� process,� inferences� about μ� when�σ�is�known,�Type�II�error�(β)�and�power�(1�−�β),�statistical�versus�practical�significance,� and� inferences� about� μ� when� σ� is� unknown�� Concepts� to� be� discussed� include� the� fol- lowing:�null�or�statistical�hypothesis�versus�scientific�or�research�hypothesis;�Type�I�error� (α),� Type II� error� (β),� and� power� (1� −� β);� two-tailed� versus� one-tailed� alternative� hypoth- eses;�critical�regions�and�critical�values;�z�test�statistic;�confidence�interval�(CI)�around�the� mean;� t� test� statistic;� and� t� distribution,� degrees� of� freedom,� and� table� of� t� distributions�� Our�objectives�are�that�by�the�end�of�this�chapter,�you�will�be�able�to�(a)�understand�the� basic�concepts�of�hypothesis�testing;�(b)�utilize�the�normal�and�t�tables;�and�(c)�understand,� determine,�and�interpret�the�results�from�the�z�test,�t�test,�and�CI�procedures�
6.1 Types of Hypotheses
You�may�remember�Marie�from�previous�chapters��We�now�revisit�Marie�in�this�chapter�
Marie,� a� graduate� student� pursuing� a� master’s� degree� in� educational� research,� has� completed� her� first� tasks� as� a� research� assistant—determining� a� number� of� descrip- tive�statistics�on�data�provided�to�her�by�her�faculty�mentor��The�faculty�member�was� so�pleased�with�the�descriptive�analyses�and�presentation�of�results�previously�shared� that�she�has�asked�Marie�to�consult�with�a�local�hockey�coach,�Oscar,�who�is�interested� in� examining� team� skating� performance�� Based� on� Oscar’s� research� question:� Is the mean skating speed of a hockey team different from the league mean speed of 12 seconds?�Marie� suggests�a�one-sample�test�of�means�as�the�test�of�inference��Her�task�is�to�assist�Oscar� in�generating�the�test�of�inference�to�answer�his�research�question�
Hypothesis�testing�is�a�decision-making�process�where�two�possible�decisions�are�weighed� in�a�statistical�fashion��In�a�way,�this�is�much�like�any�other�decision�involving�two�possi- bilities,�such�as�whether�to�carry�an�umbrella�with�you�today�or�not��In�statistical�decision- making,�the�two�possible�decisions�are�known�as�hypotheses��Sample�data�are�then�used� to�help�us�select�one�of�these�decisions��The�two�types�of�hypotheses�competing�against� one�another�are�known�as�the�null�or�statistical hypothesis,�denoted�by�H0,�and�the�scien- tific, alternative,�or�research hypothesis,�denoted�by�H1�
123Introduction to Hypothesis Testing: Inferences About a Single Mean
The�null�or�statistical�hypothesis�is�a�statement�about�the�value�of�an�unknown�popula- tion� parameter�� Considering� the� procedure� we� are� discussing� in� this� chapter,� the� one- sample� mean� test,� one� example� H0� might� be� that� the� population� mean� IQ� score� is� 100,� which�we�denote�as
� H H0 000 00 0: 1 or : 1µ µ= − =
Mathematically,� both� equations� say� the� same� thing�� The� version� on� the� left� is� the� more� traditional�form�of�the�null�hypothesis�involving�a�single�mean��However,�the�version�on� the� right� makes� clear� to� the� reader� why� the� term� “null”� is� appropriate�� That� is,� there� is� no�difference�or�a�“null”�difference�between�the�population�mean�and�the�hypothesized� mean�value�of�100��In�general,�the�hypothesized�mean�value�is�denoted�by�μ0�(here�μ0�=�100)�� Another� H0� might� be� the� statistics� exam� population� means� are� the� same� for� male� and� female�students,�which�we�denote�as
� H0 00: 11 2µ µ− =
where μ1�is�the�population�mean�for�males μ2�is�the�population�mean�for�females
Here�there�is�no�difference�or�a�“null”�difference�between�the�two�population�means��The� test�of�the�difference�between�two�means�is�presented�in�Chapter�7��As�we�move�through� subsequent�chapters,�we�become�familiar�with�null�hypotheses�that�involve�other�popula- tion�parameters�such�as�proportions,�variances,�and�correlations�
The�null�hypothesis�is�basically�set�up�by�the�researcher�in�an�attempt�to�reject�the�null� hypothesis�in�favor�of�our�own�personal�scientific,�alternative,�or�research�hypothesis��In� other�words,�the�scientific�hypothesis�is�what�we�believe�the�outcome�of�the�study�will�be,� based�on�previous�theory�and�research��Thus,�we�are�trying�to�reject�the�null�hypothesis� and�find�evidence�in�favor�of�our�scientific�hypothesis��The�scientific�hypotheses�H1�for�our� two�examples�are
� H H1 1: 1 or : 1µ µ≠ − ≠00 00 0
and
� H1 1 2 1: µ µ− ≠ 00
Based�on�the�sample�data,�hypothesis�testing�involves�making�a�decision�as�to�whether�the� null�or�the�research�hypothesis�is�supported��Because�we�are�dealing�with�sample�statistics� in�our�decision-making�process,�and�trying�to�make�an�inference�back�to�the�population� parameter(s),�there�is�always�some�risk�of�making�an�incorrect�decision��In�other�words,�the� sample�data�might�lead�us�to�make�a�decision�that�is�not�consistent�with�the�population�� We�might�decide�to�take�an�umbrella�and�it�does�not�rain,�or�we�might�decide�to�leave�the� umbrella�at�home�and�it�rains��Thus,�as�in�any�decision,�the�possibility�always�exists�that� an�incorrect�decision�may�be�made��This�uncertainty�is�due�to�sampling�error,�which,�we� will�see,�can�be�described�by�a�probability�statement��That�is,�because�the�decision�is�made� based�on�sample�data,�the�sample�may�not�be�very�representative�of�the�population�and� therefore�leads�us�to�an�incorrect�decision��If�we�had�population�data,�we�would�always�
124 An Introduction to Statistical Concepts
make�the�correct�decision�about�a�population�parameter��Because�we�usually�do�not,�we� use�inferential�statistics�to�help�make�decisions�from�sample�data�and�infer�those�results� back� to� the� population�� The� nature� of� such� decision� errors� and� the� probabilities� we� can� attribute�to�them�are�described�in�the�next�section�
6.2 Types of Decision Errors
In� this� section,� we� consider� more� specifically� the� types� of� decision� errors� that� might� be� made�in�the�decision-making�process��First�an�example�decision-making�situation�is�pre- sented��This�is�followed�by�a�decision-making�table�whereby�the�types�of�decision�errors� are�easily�depicted�
6.2.1 example decision-Making Situation
Let�us�propose�an�example�decision-making�situation�using�an�adult�intelligence�instru- ment��It�is�known�somehow�that�the�population�standard�deviation�of�the�instrument�is� 15�(i�e�,�σ2�=�225,�σ�=�15)��(In�the�real�world,�it�is�rare�that�the�population�standard�deviation� is�known,�and�we�return�to�reality�later�in�the�chapter�when�the�basic�concepts�have�been� covered��But�for�now,�assume�that�we�know�the�population�standard�deviation�)�Our�null� and�alternative�hypotheses,�respectively,�are�as�follows:
� H H0 000 00 0: 1 or : 1µ µ= − =
� H H1 1: 1 or : 1µ µ≠ − ≠00 00 0
Thus,�we�are�interested�in�testing�whether�the�population�mean�for�the�intelligence�instru- ment�is�equal�to�100,�our�hypothesized�mean�value,�or�not�equal�to�100�
Next�we�take�several�random�samples�of�individuals�from�the�adult�population��We�find� for�our�first�sample�Y
– 1�=�105�(i�e�,�denoting�the�mean�for�sample�1)��Eyeballing�the�informa-
tion�for�sample�1,�the�sample�mean�is�one-third�of�a�standard�deviation�above�the�hypoth- esized�value�[i�e�,�by�computing�a�z�score�of�(105�−�100)/15�=��3333],�so�our�conclusion�would� probably�be�to�fail�to�reject�H0��In�other�words,�if�the�population�mean�actually�is�100,�then� we�believe�that�one�is�quite�likely�to�observe�a�sample�mean�of�105��Thus,�our�decision�for� sample� 1� is� to� fail� to� reject� H0;� however,� there� is� some� likelihood� or� probability� that� our� decision�is�incorrect�
We� take� a� second� sample� and� find� Y –
2� =� 115� (i�e�,� denoting� the� mean� for� sample� 2)�� Eyeballing�the�information�for�sample�2,�the�sample�mean�is�one�standard�deviation�above� the�hypothesized�value�[i�e�,�z�=�(115�−�100)/15�=�1�0000],�so�our�conclusion�would�probably� be�to�fail�to�reject�H0��In�other�words,�if�the�population�mean�actually�is�100,�then�we�believe� that�it�is�somewhat�likely�to�observe�a�sample�mean�of�115��Thus,�our�decision�for�sample�2�is� to�fail�to�reject�H0��However,�there�is�an�even�greater�likelihood�or�probability�that�our�deci- sion�is�incorrect�than�was�the�case�for�sample�1;�this�is�because�the�sample�mean�is�further� away�from�the�hypothesized�value�
We�take�a�third�sample�and�find�Y –
3�=�190�(i�e�,�denoting�the�mean�for�sample�3)��Eyeballing� the� information� for� sample� 3,� the� sample� mean� is� six� standard� deviations� above� the� hypothesized�value�[i�e�,�z�=�(190�−�100)/15�=�6�0000],�so�our�conclusion�would�probably�be�
125Introduction to Hypothesis Testing: Inferences About a Single Mean
reject�H0��In�other�words,�if�the�population�mean�actually�is�100,�then�we�believe�that�it�is� quite�unlikely�to�observe�a�sample�mean�of�190��Thus,�our�decision�for�sample�3�is�to�reject� H0;�however,�there�is�some�small�likelihood�or�probability�that�our�decision�is�incorrect�
6.2.2 decision-Making Table
Let�us�consider�Table�6�1�as�a�mechanism�for�sorting�out�the�possible�outcomes�in�the�sta- tistical�decision-making�process��The�table�consists�of�the�general�case�and�a�specific�case�� First,� in� part� (a)� of� the� table,� we� have� the� possible� outcomes� for� the� general� case�� For� the� state�of�nature�or�reality�(i�e�,�how�things�really�are�in�the�population),�there�are�two�distinct� possibilities�as�depicted�by�the�rows�of�the�table��Either�H0�is�indeed�true�or�H0�is�indeed� false��In�other�words,�according�to�the�real-world�conditions�in�the�population,�either�H0�is� actually�true�or�H0�is�actually�false��Admittedly,�we�usually�do�not�know�what�the�state�of� nature�truly�is;�however,�it�does�exist�in�the�population�data��It�is�the�state�of�nature�that�we� are�trying�to�best�approximate�when�making�a�statistical�decision�based�on�sample�data�
For� our� statistical� decision,� there� are� two� distinct� possibilities� as� depicted� by� the� col- umns� of� the� table�� Either� we� fail� to� reject� H0� or� we� reject� H0�� In� other� words,� based� on� our�sample�data,�we�either�fail�to�reject�H0�or�reject�H0��As�our�goal�is�usually�to�reject�H0� in� favor� of� our� research� hypothesis,� we� prefer� the� term� fail to reject� rather� than� accept�� Accept�implies� you� are�willing�to�throw� out�your� research� hypothesis� and�admit�defeat� based� on� one� sample�� Fail to reject� implies� you� still� have� some� hope� for� your� research� hypothesis,�despite�evidence�from�a�single�sample�to�the�contrary�
If�we�look�inside�of�the�table,�we�see�four�different�outcomes�based�on�a�combination�of� our�statistical�decision�and�the�state�of�nature��Consider�the�first�row�of�the�table�where�H0� is�in�actuality�true��First,�if�H0�is�true�and�we�fail�to�reject�H0,�then�we�have�made�a�correct� decision;�that�is,�we�have�correctly�failed�to�reject�a�true�H0��The�probability�of�this�first�out- come�is�known�as�1�−�α�(where�α�represents�alpha)��Second,�if�H0�is�true�and�we�reject�H0,� then�we�have�made�a�decision�error�known�as�a�Type I error��That�is,�we�have�incorrectly� rejected�a�true�H0��Our�sample�data�have�led�us�to�a�different�conclusion�than�the�popula- tion�data�would�have��The�probability�of�this�second�outcome�is�known�as�α��Therefore,�if� H0�is�actually�true,�then�our�sample�data�lead�us�to�one�of�two�conclusions,�either�we�cor- rectly�fail�to�reject�H0,�or�we�incorrectly�reject�H0��The�sum�of�the�probabilities�for�these�two� outcomes�when�H0�is�true�is�equal�to�1�[i�e�,�(1�−�α)�+�α�=�1]�
Consider� now� the� second� row� of� the� table� where� H0� is� in� actuality� false�� First,� if� H0� is� really�false�and�we�fail�to�reject�H0,�then�we�have�made�a�decision�error�known�as�a�Type II
Table 6.1
Statistical�Decision�Table
State of Nature (Reality)
Decision
Fail to Reject H0 Reject H0
(a) General case H0�is�true Correct�decision�(1�−�α) Type�I�error�(α)
H0�is�false Type�II�error�(β) Correct�decision�(1�−�β)�=�power
(b) Example rain case H0�is�true�(no rain) Correct�decision�(do not take umbrella
and no umbrella needed)�(1�−�α) Type�I�error�(take umbrella and look silly)�(α)
H0�is�false�(rains) Type�II�error�(do not take umbrella and get wet)�(β)
Correct�decision�(take umbrella and stay dry)� (1�−�β)�=�power
126 An Introduction to Statistical Concepts
error��That�is,�we�have�incorrectly�failed�to�reject�a�false�H0��Our�sample�data�have�led�us� to�a�different�conclusion�than�the�population�data�would�have��The�probability�of�this�out- come�is�known�as�β�(beta)��Second,�if�H0�is�really�false�and�we�reject�H0,�then�we�have�made� a�correct�decision;�that�is,�we�have�correctly�rejected�a�false�H0��The�probability�of�this�sec- ond�outcome�is�known�as�1�−�β�or�power�(to�be�more�fully�discussed�later�in�this�chapter)�� Therefore,�if�H0�is�actually�false,�then�our�sample�data�lead�us�to�one�of�two�conclusions,� either�we�incorrectly�fail�to�reject�H0,�or�we�correctly�reject�H0��The�sum�of�the�probabilities� for�these�two�outcomes�when�H0�is�false�is�equal�to�1�[i�e�,�β�+�(1�−�β)�=�1]�
As�an�application�of�this�table,�consider�the�following�specific�case,�as�shown�in�part�(b)�of� Table�6�1��We�wish�to�test�the�following�hypotheses�about�whether�or�not�it�will�rain�tomorrow�
H0:�no�rain�tomorrow H1:�rains�tomorrow
We� collect� some� sample� data� from� prior� years� for� the� same� month� and� day,� and� go� to� make�our�statistical�decision��Our�two�possible�statistical�decisions�are�(a)�we�do�not�believe� it�will�rain�tomorrow�and�therefore�do�not�bring�an�umbrella�with�us,�or�(b)�we�do�believe�it� will�rain�tomorrow�and�therefore�do�bring�an�umbrella�
Again� there� are� four� potential� outcomes�� First,� if� H0� is� really� true� (no� rain)� and� we� do� not�carry�an�umbrella,�then�we�have�made�a�correct�decision�as�no�umbrella�is�necessary� (probability�=�1�−�α)��Second,�if�H0�is�really�true�(no�rain)�and�we�carry�an�umbrella,�then� we�have�made�a�Type�I�error�as�we�look�silly�carrying�that�umbrella�around�all�day�(prob- ability�=�α)��Third,�if�H0�is�really�false�(rains)�and�we�do�not�carry�an�umbrella,�then�we�have� made�a�Type�II�error�and�we�get�wet�(probability�=�β)��Fourth,�if�H0�is�really�false�(rains)� and�we�carry�an�umbrella,�then�we�have�made�the�correct�decision�as�the�umbrella�keeps� us�dry�(probability�=�1�−�β)�
Let� us� make� two� concluding� statements� about� the� decision� table�� First,� one� can� never� prove�the�truth�or�falsity�of�H0�in�a�single�study��One�only�gathers�evidence�in�favor�of�or� in�opposition�to�the�null�hypothesis��Something�is�proven�in�research�when�an�entire�col- lection�of�studies�or�evidence�reaches�the�same�conclusion�time�and�time�again��Scientific� proof�is�difficult�to�achieve�in�the�social�and�behavioral�sciences,�and�we�should�not�use� the�term�prove�or�proof�loosely��As�researchers,�we�gather�multiple�pieces�of�evidence�that� eventually�lead�to�the�development�of�one�or�more�theories��When�a�theory�is�shown�to�be� unequivocally�true�(i�e�,�in�all�cases),�then�proof�has�been�established�
Second,�let�us�consider�the�decision�errors�in�a�different�light��One�can�totally�eliminate� the� possibility� of� a� Type� I� error� by� deciding� to� never� reject� H0�� That� is,� if� we� always� fail� to� reject� H0� (do� not� ever� carry� umbrella),� then� we� can� never� make� a� Type� I� error� (look� silly�with�unnecessary�umbrella)��Although�this�strategy�sounds�fine,�it�totally�takes�the� decision-making�power�out�of�our�hands��With�this�strategy,�we�do�not�even�need�to�collect� any�sample�data,�as�we�have�already�decided�to�never�reject�H0�
One� can� totally� eliminate� the� possibility� of� a� Type� II� error� by� deciding� to� always� reject H0��That�is,�if�we�always�reject�H0�(always�carry�umbrella),�then�we�can�never�make� a� Type� II� error� (get� wet� without� umbrella)�� Although� this� strategy� also� sounds� fine,� it� totally�takes�the�decision-making�power�out�of�our�hands��With�this�strategy,�we�do�not� even� need� to� collect� any� sample� data� as� we� have� already� decided� to� always� reject H0�� Taken� together,� one� can� never� totally� eliminate� the� possibility� of� both� a� Type� I� and� a� Type�II�error��No�matter�what�decision�we�make,�there�is�always�some�possibility�of�mak- ing�a�Type�I�and/or�Type�II�error��Therefore,�as�researchers,�our�job�is�to�make�conscious� decisions�in�designing�and�conducting�our�study�and�in�analyzing�the�data�so�that�the� possibility�of�decision�error�is�minimized�
127Introduction to Hypothesis Testing: Inferences About a Single Mean
6.3 Level of Significance (α) We�have�already�stated�that�a�Type�I�error�occurs�when�the�decision�is�to�reject�H0�when� in�fact�H0�is�actually�true��We�defined�the�probability�of�a�Type�I�error�as�α,�which�is�also� known�as�the�level�of�significance�or�significance�level��We�now�examine�α�as�a�basis�for� helping� us� make� statistical� decisions�� Recall� from� a� previous� example� that� the� null� and� alternative�hypotheses,�respectively,�are�as�follows:
� H H0 000 00 0: 1 or : 1µ µ= − =
� H H1 1: 1 or : 1µ µ≠ − ≠00 00 0
We� need� a� mechanism� for� deciding� how� far� away� a� sample� mean� needs� to� be� from� the� hypothesized� mean� value� of� μ0� =� 100� in� order� to� reject� H0�� In� other� words,� at� a� certain� point�or�distance�away�from�100,�we�will�decide�to�reject�H0��We�use�α�to�determine�that� point� for� us,� where� in� this� context,� α� is� known� as� the� level of significance�� Figure� 6�1a� shows�a�sampling�distribution�of�the�mean�where�the�hypothesized�value�μ0�is�depicted� at�the�center�of�the�distribution��Toward�both�tails�of�the�distribution,�we�see�two�shaded� regions�known�as�the�critical regions�or�regions�of�rejection��The�combined�areas�of�the� two�shaded�regions�is�equal�to�α,�and,�thus,�the�area�of�either�the�upper�or�the�lower�tail� critical�region�is�equal�to�α/2�(i�e�,�we�split�α�in�half�by�dividing�by�two)��If�the�sample�mean�
(a)
α/2 Critical region
α/2 Critical region
Critical value
Critical value
µ0 Hypothesized
value (b)
α Critical region
Critical value
µ0 Hypothesized
value
(c)
α Critical region
Critical value
µ0 Hypothesized
value
FIGuRe 6.1 Alternative�hypotheses�and�critical�regions:�(a)�two-tailed�test;�(b)�one-tailed,�right�tailed�test;�(c)�one-tailed,�left� tailed�test�
128 An Introduction to Statistical Concepts
is�far�enough�away�from�the�hypothesized�mean�value,�μ0,�that�it�falls�into�either�critical� region,�then�our�statistical�decision�is�to�reject�H0��In�this�case,�our�decision�is�to�reject�H0� at�the�α�level�of�significance��If,�however,�the�sample�mean�is�close�enough�to�μ0�that�it�falls� into�the�unshaded�region�(i�e�,�not�into�either�critical�region),�then�our�statistical�decision� is� to� fail to reject� H0�� The� precise� points� on� the� X� axis� at� which� the� critical� regions� are� divided�from�the�unshaded�region�are�known�as�the�critical values��Determining�critical� values�is�discussed�later�in�this�chapter�
Note�that�under�the�alternative�hypothesis�H1,�we�are�willing�to�reject�H0�when�the�sample� mean�is�either�significantly�greater�than�or�significantly�less�than�the�hypothesized�mean� value�μ0��This�particular�alternative�hypothesis�is�known�as�a�nondirectional alternative hypothesis,�as�no�direction�is�implied�with�respect�to�the�hypothesized�value��That�is,�we� will� reject� the� null� hypothesis� in� favor� of� the� alternative� hypothesis� in� either� direction,� either� above� or� below� the� hypothesized� mean� value�� This� also� results� in� what� is� known� as�a�two-tailed test of significance�in�that�we�are�willing�to�reject�the�null�hypothesis�in� either�tail�or�critical�region�
Two� other� alternative� hypotheses� are� also� possible,� depending� on� the� researcher’s� sci- entific�hypothesis,�which�are�known�as�a�directional alternative hypothesis��One�direc- tional�alternative�is�that�the�population�mean�is�greater�than�the�hypothesized�mean�value,� also�known�as�a�right-tailed�test,�as�denoted�by
� H H1 1: 1 or : 1µ µ> − >00 00 0
Mathematically,�both�of�these�equations�say�the�same�thing��With�a�right-tailed�alternative� hypothesis,�the�entire�region�of�rejection�is�contained�in�the�upper�tail,�with�an�area�of�α,� known� as� a� one-tailed test of significance� (and� specifically� the� right� tail)�� If� the� sample� mean�is�significantly�greater�than�the�hypothesized�mean�value�of�100,�then�our�statistical� decision�is�to�reject�H0��If,�however,�the�sample�mean�falls�into�the�unshaded�region,�then� our�statistical�decision�is�to�fail�to�reject�H0��This�situation�is�depicted�in�Figure�6�1b�
A� second� directional� alternative� is� that� the� population� mean� is� less� than� the� hypoth- esized�mean�value,�also�known�as�a�left-tailed�test,�as�denoted�by
� H H1 1: 1 or : 1µ µ< − <00 00 0
Mathematically,�both�of�these�equations�say�the�same�thing��With�a�left-tailed�alternative� hypothesis,�the�entire�region�of�rejection�is�contained�in�the�lower�tail,�with�an�area�of�α,� also�known�as�a�one-tailed test of significance�(and�specifically�the�left�tail)��If�the�sam- ple�mean�is�significantly�less�than�the�hypothesized�mean�value�of�100,�then�our�statisti- cal�decision�is�to�reject�H0��If,�however,�the�sample�mean�falls�into�the�unshaded�region,� then�our�statistical�decision�is�to�fail�to�reject�H0��This�situation�is�depicted�in�Figure�6�1c�
There� is� some� potential� for� misuse� of� the� different� alternatives,� which� we� consider� to� be�an�ethical� matter��For�example,� a�researcher� conducts�a�one-tailed� test�with�an�upper� tail�critical�region�and�fails�to�reject�H0��However,�the�researcher�notices�that�the�sample� mean�is�considerably�below�the�hypothesized�mean�value�and�then�decides�to�change�the� alternative�hypothesis�to�either�a�nondirectional�test�or�a�one-tailed�test�in�the�other�tail�� This� is� unethical,� as� the� researcher� has� examined� the� data� and� changed� the� alternative� hypothesis��The�morale�of�the�story�is�this:�If there is previous and consistent empirical evidence to use a specific directional alternative hypothesis, then you should do so. If, however, there is mini- mal or inconsistent empirical evidence to use a specific directional alternative, then you should not. Instead, you should use a nondirectional alternative��Once�you�have�decided�which�alternative�
129Introduction to Hypothesis Testing: Inferences About a Single Mean
hypothesis�to�go�with,�then�you�need�to�stick�with�it�for�the�duration�of�the�statistical�deci- sion��If�you�find�contrary�evidence,�then�report�it�as�it�may�be�an�important�finding,�but�do� not�change�the�alternative�hypothesis�in�midstream�
6.4 Overview of Steps in Decision-Making Process
Before�we�get�into�the�specific�details�of�conducting�the�test�of�a�single�mean,�we�want�to� discuss�the�basic�steps�for�hypothesis�testing�of�any�inferential�test:
� 1�� State�the�null�and�alternative�hypotheses� � 2�� Select�the�level�of�significance�(i�e�,�alpha,�α)� � 3�� Calculate�the�test�statistic�value� � 4�� Make�a�statistical�decision�(reject�or�fail�to�reject�H0)�
Step 1:�The�first�step�in�the�decision-making�process�is�to�state�the�null�and�alternative� hypotheses��Recall�from�our�previous�example�that�the�null�and�nondirectional�alterna- tive�hypotheses,�respectively,�for�a�two-tailed�test�are�as�follows:
� H H0 000 00 0: 1 or : 1µ µ= − =
� H H1 1: 1 or : 1µ µ≠ − ≠00 00 0
One� could� also� choose� one� of� the� other� directional� alternative� hypotheses� described� previously�
If� we� choose� to� write� our� null� hypothesis� as� H0:� μ� =� 100,� we� would� want� to� write� our� research�hypothesis�in�a�consistent�manner,�H1:�μ�≠�100�(rather�than�H1:�μ�−�100�≠�0)��In�pub- lication,�many�researchers�opt�to�present�the�hypotheses�in�narrative�form�(e�g�,�“the�null� hypothesis�states�that�the�population�mean�will�equal�100,�and�the�alternative�hypothesis� states� that� the� population� mean� will� not� equal� 100”)�� How� you� present� your� hypotheses� (mathematically�or�using�statistical�notation)�is�up�to�you�
Step 2:�The�second�step�in�the�decision-making�process�is�to�select�a�level�of�significance�α�� There�are�two�considerations�to�make�in�terms�of�selecting�a�level�of�significance��One�con- sideration�is�the�cost�associated�with�making�a�Type�I�error,�which�is�what�α�really�is��Recall� that�alpha�is�the�probability�of�rejecting�the�null�hypothesis�if�in�reality�the�null�hypothesis� is�true��When�a�Type�I�error�is�made,�that�means�evidence�is�building�in�favor�of�the�research� hypothesis�(which�is�actually�false)��Let�us�take�an�example�of�a�new�drug��To�test�the�efficacy� of�the�drug,�an�experiment�is�conducted�where�some�individuals�take�the�new�drug�while� others� receive� a� placebo�� The� null� hypothesis,� stated� nondirectionally,� would� essentially� indicate�that�the�effects�of�the�drug�and�placebo�are�the�same��Rejecting�that�null�hypothesis� would�mean�that�the�effects�are�not�equal—suggesting�that�perhaps�this�new�drug,�which�in� reality�is�not�any�better�than�a�placebo,�is�being�touted�as�effective�medication��That�is�obvi- ously�problematic�and�potentially�very�hazardous�
Thus,�if�there�is�a�relatively�high�cost�associated�with�a�Type�I�error—for�example,�such� that�lives�are�lost,�as�in�the�medical�profession—then�one�would�want�to�select�a�relatively� small�level�of�significance�(e�g�,��01�or�smaller)��A�small�alpha�would�translate�to�a�very�small� probability�of�rejecting�the�null�if�it�were�really�true�(i�e�,�a�small�probability�of�making�an�
130 An Introduction to Statistical Concepts
incorrect�decision)��If�there�is�a�relatively�low�cost�associated�with�a�Type�I�error—for�exam- ple,�such�that�children�have�to�eat�the�second-rated�candy�rather�than�the�first—then�select- ing�a�larger�level�of�significance�may�be�appropriate�(e�g�,��05�or�larger)��Costs�are�not�always� known,�however��A�second�consideration�is�the�level�of�significance�commonly�used�in�your� field� of� study�� In� many� disciplines,� the� �05� level� of� significance� has� become� the� standard� (although�no�one�seems�to�have�a�really�good�rationale)��This�is�true�in�many�of�the�social� and�behavioral�sciences��Thus,�you�would�do�well�to�consult�the�published�literature�in�your� field�to�see�if�some�standard�is�commonly�used�and�to�consider�it�for�your�own�research�
Step 3:�The�third�step�in�the�decision-making�process�is�to�calculate�the�test�statistic��For� the� one-sample� mean� test,� we� will� compute� the� sample� mean� Y
– � and� compare� it� to� the�
hypothesized�value�μ0��This�allows�us�to�determine�the�size�of�the�difference�between� Y – � and� μ0,� and� subsequently,� the� probability� associated� with� the� difference�� The� larger�
the�difference,�the�more�likely�it�is�that�the�sample�mean�really�differs�from�the�hypoth- esized�mean�value�and�the�larger�the�probability�associated�with�the�difference�
Step 4:�The�fourth�and�final�step�in�the�decision-making�process�is�to�make�a�statistical�deci- sion�regarding�the�null�hypothesis�H0��That�is,�a�decision�is�made�whether�to�reject�H0�or�to� fail�to�reject�H0��If�the�difference�between�the�sample�mean�and�the�hypothesized�value�is� large�enough�relative�to�the�critical�value�(we�will�talk�about�critical�values�in�more�detail� later),�then�our�decision�is�to�reject�H0��If�the�difference�between�the�sample�mean�and�the� hypothesized�value�is�not�large�enough�relative�to�the�critical�value,�then�our�decision�is�to� fail�to�reject�H0��This�is�the�basic�four-step�process�for�hypothesis�testing�of�any�inferential� test��The�specific�details�for�the�test�of�a�single�mean�are�given�in�the�following�section�
6.5 Inferences About μ When σ Is Known In�this�section,�we�examine�how�hypotheses�about�a�single�mean�are�conducted�when�the� population�standard�deviation�is�known��Specifically,�we�consider�the�z�test,�an�example� illustrating�the�use�of�the�z�test,�and�how�to�construct�a�CI�around�the�mean�
6.5.1 z Test
Recall�from�Chapter�4�the�definition�of�a�z�score�as
�
z Yi
Y =
− µ σ
where Yi�is�the�score�on�variable�Y�for�individual�i μ�is�the�population�mean�for�variable�Y σY�is�the�population�standard�deviation�for�variable�Y
The�z�score�is�used�to�tell�us�how�many�standard�deviation�units�an�individual’s�score�is� from�the�mean�
In� the� context� of� this� chapter,� however,� we� are� concerned� with� the� extent� to� which� a� sample�mean�differs�from�some�hypothesized�mean�value��We�can�construct�a�variation�of�
131Introduction to Hypothesis Testing: Inferences About a Single Mean
the�z�score�for�testing�hypotheses�about�a�single�mean��In�this�situation,�we�are�concerned� with�the�sampling�distribution�of�the�mean�(introduced�in�Chapter�5),�so�the�equation�must� reflect�means�rather�than�raw�scores��Our�z�score�equation�for�testing�hypotheses�about�a� single�mean�becomes
� z
Y
Y
= − µ σ
0
where Y – �is�the�sample�mean�for�variable�Y
μ0�is�the�hypothesized�mean�value�for�variable�Y σY– is�the�population�standard�error�of�the�mean�for�variable�Y
From�Chapter�5,�recall�that�the�population�standard�error�of�the�mean�σY–�is�computed�by
� σ
σ Y
Y
n =
�
where σY�is�the�population�standard�deviation�for�variable�Y n�is�sample�size
Thus,� the� numerator� of� the� z� score� equation� is� the� difference� between� the� sample� mean� and�the�hypothesized�value�of�the�mean,�and�the�denominator�is�the�standard�error�of�the� mean��What�we�are�really�determining�here�is�how�many�standard�deviation�(or�standard� error)� units� the� sample� mean� is� from� the� hypothesized� mean�� Henceforth,� we� call� this� variation�of�the�z�score�the�test statistic for the test of a single mean,�also�known�as�the� z�test��This�is�the�first�of�several�test�statistics�we�describe�in�this�text;�every�inferential�test� requires�some�test�statistic�for�purposes�of�testing�hypotheses�
We�need�to�make�a�statistical�assumption�regarding�this�hypothesis�testing�situation��We� assume�that�z�is�normally�distributed�with�a�mean�of�0�and�a�standard�deviation�of�1��This� is�written�statistically�as�z ∼ N(0,�1)�following�the�notation�we�developed�in�Chapter�4��Thus,� the�assumption�is�that�z�follows�the�unit�normal�distribution�(in�other�words,�the�shape�of� the�distribution�is�approximately�normal)��An�examination�of�our�test�statistic�z�reveals�that� only� the� sample� mean� can� vary� from� sample� to� sample�� The� hypothesized� value� and� the� standard�error�of�the�mean�are�constant�for�every�sample�of�size�n�from�the�same�population�
In�order�to�make�a�statistical�decision,�the�critical�regions�need�to�be�defined��As�the�test� statistic�is�z�and�we�have�assumed�normality,�then�the�relevant�theoretical�distribution�we� compare�the�test�statistic�to�is�the�unit�normal�distribution��We�previously�discussed�this�dis- tribution�in�Chapter�4,�and�the�table�of�values�is�given�in�Table�A�1��If�the�alternative�hypoth- esis�is�nondirectional,�then�there�would�be�two�critical�regions,�one�in�the�upper�tail�and�one� in�the�lower�tail��Here�we�would�split�the�area�of�the�critical�region,�known�as�α,�in�two��If�the� alternative�hypothesis�is�directional,�then�there�would�only�be�one�critical�region,�either�in� the�upper�tail�or�in�the�lower�tail,�depending�on�which�direction�one�is�willing�to�reject�H0�
6.5.2 example
Let�us�illustrate�the�use�of�this�inferential�test�through�an�example��We�are�interested�in� testing�whether�the�population�of�undergraduate�students�from�Awesome�State�University� (ASU)�has�a�mean�intelligence�test�score�different�from�the�hypothesized�mean�value�of�
132 An Introduction to Statistical Concepts
μ0�=�100�(remember�that�the�hypothesized�mean�value�does�not�come�from�our�sample�but� from�another�source;�in�this�example,�let�us�say�that�this�value�of�100�is�the�national�norm� as�presented�in�the�technical�manual�of�this�particular�intelligence�test)�
Recall that our first step in hypothesis testing is to state the hypothesis��A�nondirectional�alter- native�hypothesis�is�of�interest�as�we�simply�want�to�know�if�this�population�has�a�mean� intelligence�different�from�the�hypothesized�value,�either�greater�than�or�less�than��Thus,� the�null�and�alternative�hypotheses�can�be�written�respectively�as�follows:
� H H0 000 00 0: 1 or : 1µ µ= − =
� H H1 1: 1 or : 1µ µ≠ − ≠00 00 0
A�sample�mean�of�Y – �=�103�is�observed�for�a�sample�of�n�=�100�ASU�undergraduate�students��
From� the� development� of� this� intelligence� test,� we� know� that� the� theoretical� population� standard�deviation�is�σY�=�15�(again,�for�purposes�of�illustration,�let�us�say�that�the�popula- tion�standard�deviation�of�15�was�noted�in�the�technical�manual�for�this�test)�
Our second step is to select a level of significance��The�standard�level�of�significance�in�this� field�is�the��05�level;�thus,�we�perform�our�significance�test�at�α�=��05�
The third step is to compute the test statistic value��To�compute�our�test�statistic�value,�first� we�compute�the�standard�error�of�the�mean�(the�denominator�of�our�test�statistic�formula)� as�follows:
� σ
σ Y
Y
n = = =
15 100
1 5000.
Then� we� compute� the� test� statistic� z,� where� the� numerator� is� the� difference� between� the� mean�of�our�sample�(Y
– �=�103)�and�the�hypothesized�mean�value�(μ0�=�100),�and�the�denomi-
nator�is�the�standard�error�of�the�mean:
� z
Y
Y
= −
= −
= µ
σ 0 103 100
1 5000 2 0000
. .
Finally, in the last step, we make our statistical decision by comparing the test statistic z to the critical values��To�determine�the�critical�values�for�the�z�test,�we�use�the�unit�normal�distribution�in� Table A�1�� Since� α� =� �05� and� we� are� conducting� a� nondirectional� test,� we� need� to� find� criti- cal�values�for�the�upper�and�lower�tails,�where�the�area�of�each�of�the�two�critical�regions�is� equal�to��025�(i�e�,�splitting�alpha�in�half:�α/2�or��05/2�=��025)��From�the�unit�normal�table,�we� find�these�critical�values�to�be�+1�96�(the�point�on�the�X�axis�where�the�area�above�that�point� is�equal�to��025)�and�−1�96�(the�point�on�the�X�axis�where�the�area�below�that�point�is�equal�to� �025)��As�shown�in�Figure�6�2,�the�test�statistic�z�=�2�00�falls�into�the�upper�tail�critical�region,� just�slightly�larger�than�the�upper�tail�critical�value�of�+1�96��Our�decision�is�to�reject�H0�and� conclude�that�the�ASU�population�from�which�the�sample�was�selected�has�a�mean�intelligence� score�that�is�statistically�significantly�different�from�the�hypothesized�mean�of�100�at�the��05� level�of�significance�
A�more�precise�way�of�thinking�about�this�process�is�to�determine�the�exact probability� of�observing�a�sample�mean�that�differs�from�the�hypothesized�mean�value��From�the�unit� normal�table,�the�area�above�z�=�2�00�is�equal�to��0228��Therefore,�the�area�below�z�=�−2�00�is� also�equal�to��0228��Thus,�the�probability�p�of�observing,�by�chance,�a�sample�mean�of�2�00� or�more�standard�errors�(i�e�,�z�=�2�00)�from�the�hypothesized�mean�value�of�100,�in�either� direction,�is�two�times�the�observed�probability�level�or�p�=�(2)(�0228)�= �0456��To�put�this�in�
133Introduction to Hypothesis Testing: Inferences About a Single Mean
the�context�of�the�values�in�this�example,�there�is�a�relatively�small�probability�(less�than�5%)� of�observing�a�sample�mean�of�103�just�by�chance�if�the�true�population�mean�is�really 100�� As� this� exact� probability� (p� =� �0456)� is� smaller� than� our� level� of� significance� α� = �05,� we� reject H0�� Thus,� there� are� two� approaches� to� dealing� with� probability�� One� approach� is� a� decision�based�solely�on�the�critical�values��We�reject�or�fail�to�reject�H0�at�a�given�α�level,� but�no�other�information�is�provided��The�other�approach�is�a�decision�based�on�compar- ing�the�exact�probability�to�the�given�α�level��We�reject�or�fail�to�reject�H0�at�a�given�α�level,� but�we�also�have�information�available�about�the�closeness�or�confidence�in�that�decision�
For�this�example,�the�findings�in�a�manuscript�would�be�reported�based�on�comparing� the�p�value�to�alpha�and�reported�either�as�z�=�2�(p�<��05)�or�as�z�=�2�(p�=��0456)��(You�may� want�to�refer�to�the�style�manual�relevant�to�your�discipline,�such�as�the�Publication Manual for the American Psychological Association� (2010),� for� information� on� which� is� the� recom- mended�reporting�style�)�Obviously�the�conclusion�is�the�same�with�either�approach;�it�is� just�a�matter�of�how�the�results�are�reported��Most�statistical�computer�programs,�includ- ing�SPSS,�report�the�exact�probability�so�that�the�reader�can�make�a�decision�based�on�their� own� selected� level� of� significance�� These� programs� do� not� provide� the� critical� value(s),� which�are�only�found�in�the�appendices�of�statistics�textbooks�
6.5.3 Constructing Confidence Intervals around the Mean
Recall�our�discussion�from�Chapter�5�on�CIs��CIs�are�often�quite�useful�in�inferential�sta- tistics� for� providing� the� researcher� with� an� interval� estimate� of� a� population� parameter�� Although�the�sample�mean�gives�us�a�point�estimate�(i�e�,�just�one�value)�of�a�population� mean,�a�CI�gives�us�an�interval�estimate�of�a�population�mean�and�allows�us�to�determine� the�accuracy�or�precision�of�the�sample�mean��For�the�inferential�test�of�a�single�mean,�a�CI� around�the�sample�mean�Y
– �is�formed�from
� Y zcv Y± σ
where zcv�is�the�critical�value�from�the�unit�normal�distribution σY–�is�the�population�standard�error�of�the�mean
α/2 Critical region
α/2 Critical region
–1.96 z critical
value
+1.96 z critical
value
+2.00 z test
statistic value
µ0 Hypothesized
value
FIGuRe 6.2 Critical�regions�for�example�
134 An Introduction to Statistical Concepts
CIs�are�typically�formed�for�nondirectional�or�two-tailed�tests�as�shown�in�the�equation�� A�CI�will�generate�a�lower�and�an�upper�limit��If�the�hypothesized�mean�value�falls�within� the�lower�and�upper�limits,�then�we�would�fail�to�reject�H0��In�other�words,�if�the�hypoth- esized�mean�is�contained�in�(or�falls�within)�the�CI�around�the�sample�mean,�then�we�con- clude�that�the�sample�mean�and�the�hypothesized�mean�are�not�significantly�different�and� that�the�sample�mean�could�have�come�from�a�population�with�the�hypothesized�mean�� If� the� hypothesized� mean� value� falls� outside� the� limits� of� the� interval,� then� we� would� reject�H0��Here�we�conclude�that�it�is�unlikely�that�the�sample�mean�could�have�come�from� a�population�with�the�hypothesized�mean�
One�way�to�think�about�CIs�is�as�follows��Imagine�we�take�100�random�samples�of�the� same�sample�size�n,�compute�each�sample�mean,�and�then�construct�each�95%�CI��Then�we� can�say�that�95%�of�these�CIs�will�contain�the�population�parameter�and�5%�will�not��In� short,� 95%� of� similarly� constructed� CIs� will� contain� the� population� parameter�� It� should� also� be� mentioned� that� at� a� particular� level� of� significance,� one� will� always� obtain� the� same�statistical�decision�with�both�the�hypothesis�test�and�the�CI��The�two�procedures�use� precisely� the� same� information�� The� hypothesis� test� is� based� on� a� point� estimate;� the� CI� is�based�on�an�interval�estimate�providing�the�researcher�with�a�little�more�information�
For�the�ASU�example�situation,�the�95%�CI�would�be�computed�by
� Y zcv Y± = ± = ± =σ 103 1 96 1 5 103 2 94 100 06 105 94. ( . ) . ( . , . )
Thus,�the�95%�CI�ranges�from�100�06�to�105�94��Because�the�interval�does�not�contain�the� hypothesized�mean�value�of�100,�we�reject�H0�(the�same�decision�we�arrived�at�by�walking� through�the�steps�for�hypothesis�testing)��Thus,�it�is�quite�unlikely�that�our�sample�mean� could�have�come�from�a�population�distribution�with�a�mean�of�100�
6.6 Type II Error (β) and Power (1 − β) In� this� section,� we� complete� our� discussion� of� Type� II� error� (β)� and� power� (1� −� β)�� First� we�return�to�our�rain�example�and�discuss�the�entire�decision-making�context��Then�we� describe�the�factors�which�determine�power�
6.6.1 Full decision-Making Context
Previously,� we� defined� Type� II� error� as� the� probability� of� failing� to� reject� H0� when� H0� is� really�false��In�other�words,�in�reality,�H0�is�false,�yet�we�made�a�decision�error�and�did�not� reject�H0��The�probability�associated�with�a�Type�II�error�is�denoted�by�β��Power�is�a�related� concept�and�is�defined�as�the�probability�of�rejecting�H0�when�H0�is�really�false��In�other� words,�in�reality,�H0�is�false,�and�we�made�the�correct�decision�to�reject�H0��The�probability� associated�with�power�is�denoted�by�1�−�β��Let�us�return�to�our�“rain”�example�to�describe� Type�I�and�Type�II�errors�and�power�more�completely�
The�full�decision-making�context�for�the�“rain”�example�is�given�in�Figure�6�3��The�dis- tribution�on�the�left-hand�side�of�the�figure�is�the�sampling�distribution�when�H0�is�true,� meaning�in�reality�it�does�not�rain��The�vertical�line�represents�the�critical�value�for�decid- ing� whether� to� carry� an� umbrella� or� not�� To� the� left� of� the� vertical� line,� we� do� not� carry� an� umbrella,� and� to� the� right� side� of� the� vertical� line,� we� do� carry� an� umbrella�� For� the�
135Introduction to Hypothesis Testing: Inferences About a Single Mean
no-rain�sampling�distribution�on�the�left,�there�are�two�possibilities��First,�we�do�not�carry� an�umbrella�and�it�does�not�rain��This�is�the�unshaded�portion�under�the�no-rain�sampling� distribution�to�the�left�of�the�vertical�line��This is a correct decision,�and�the�probability�asso- ciated�with�this�decision�is�1�−�α��Second,�we�do�carry�an�umbrella�and�it�does�not�rain�� This�is�the�shaded�portion�under�the�no-rain�sampling�distribution�to�the�right�of�the�verti- cal�line��This is an incorrect decision,�a�Type�I�error,�and�the�probability�associated�with�this� decision�is�α/2�in�either�the�upper�or�lower�tail,�and�α�collectively�
The�distribution�on�the�right-hand�side�of�the�figure�is�the�sampling�distribution�when� H0� is� false,� meaning� in� reality,� it� does� rain�� For� the� rain� sampling� distribution,� there� are� two�possibilities��First,�we�do�carry�an�umbrella�and�it�does�rain��This�is�the�unshaded�por- tion�under�the�rain�sampling�distribution�to�the�right�of�the�vertical�line��This�is�a�correct decision,�and�the�probability�associated�with�this�decision�is�1�−�β�or�power��Second,�we�do� not�carry�an�umbrella�and�it�does�rain��This�is�the�shaded�portion�under�the�rain�sampling� distribution�to�the�left�of�the�vertical�line��This�is�an�incorrect decision,�a�Type�II�error,�and� the�probability�associated�with�this�decision�is�β�
As�a�second�illustration,�consider�again�the�example�intelligence�test�situation��This�situ- ation�is� depicted� in� Figure� 6�4��The�distribution� on� the�left-hand� side�of�the�figure� is� the� sampling�distribution�of�Y
– �when�H0�is�true,�meaning�in�reality,�μ�=�100��The�distribution�on�
the�right-hand�side�of�the�figure�is�the�sampling�distribution�of�Y – �when�H1�is�true,�meaning�
in�reality,�μ�=�115�(and�in�this�example,�while�there�are�two�critical�values,�only�the�right� tail�matters�as�that�relates�to�the�H1�sampling�distribution)��The�vertical�line�represents�the� critical� value� for� deciding� whether� to� reject� the� null� hypothesis� or� not�� To� the� left� of� the� vertical�line,�we�do�not�reject�H0�and�to�the�right�of�the�vertical�line,�we�reject�H0��For�the�H0� is�true�sampling�distribution�on�the�left,�there�are�two�possibilities��First,�we�do�not�reject� H0�and�H0�is�really�true��This is the unshaded portion under the H0�is true sampling distribution to the left of the vertical line��This�is�a�correct�decision,�and�the�probability�associated�with� this�decision�is�1�−�α��Second,�we�reject�H0�and�H0�is�true��This is the shaded portion under the H0�is true sampling distribution to the right of the vertical line��This�is�an�incorrect�decision,�a�
Type II error (got wet)
Do not carry umbrella.
Correct decision
Correct decision
Do carry umbrella.
Sampling distribution when
H0 “No Rain” is true.
Sampling distribution when
H0 “No Rain” is false.
Type I error (did not need umbrella)
FIGuRe 6.3 Sampling�distributions�for�the�rain�case�
136 An Introduction to Statistical Concepts
Type�I�error,�and�the�probability�associated�with�this�decision�is�α/2�in�either�the�upper�or� lower�tail,�and�α�collectively�
The�distribution�on�the�right-hand�side�of�the�figure�is�the�sampling�distribution�when� H0�is�false,�and�in�particular,�when�H1:�μ�=�115�is�true��This�is�a�specific�sampling�distribu- tion�when�H0�is�false,�and�other�possible�sampling�distributions�can�also�be�examined�(e�g�,� μ�=�85,�110)��For�the�H1:�μ�=�115�is� true�sampling�distribution,� there�are� two�possibilities�� First,�we�do�reject�H0,�as�H0�is�really�false,�and�H1:�μ�=�115�is�really�true��This�is�the�unshaded� portion�under�the�H1:�μ�=�115�is�true�sampling�distribution�to�the�right�of�the�vertical�line�� This� is� a� correct� decision,� and� the� probability� associated� with� this� decision� is� 1� −� β� or� power��Second,�we�do�not�reject�H0,�H0�is�really�false,�and�H1:�μ�=�115�is�really�true��This� is�the�shaded�portion�under�the�H1:�μ�=�115�is�true�sampling�distribution�to�the�left�of�the� vertical�line��This�is�an�incorrect�decision,�a�Type�II�error,�and�the�probability�associated� with�this�decision�is�β�
6.6.2 power determinants
Power�is�determined�by�five�different�factors:�(1)�level�of�significance,�(2)�sample�size,�(3)�popu- lation�standard�deviation,�(4)�difference�between�the�true�population�mean�μ�and�the�hypoth- esized�mean�value�μ0,�and�(5)�directionality�of�the�test�(i�e�,�one-�or�two-tailed�test)��Let�us�talk� about�each�of�these�factors�in�more�detail�
First,�power�is�determined�by�the�level�of�significance�α��As�α�increases,�power�increases�� Thus,�if�α�increases�from��05�to��10,�then�power�will�increase��This�would�occur�in�Figure�6�4� if�the�vertical�line�were�shifted�to�the�left�(thus�creating�a�larger�critical�region�and�thereby� making� it� easier� to� reject� the� null� hypothesis)�� This� would� increase� the� α� level� and� also� increase�power��This�factor�is�under�the�control�of�the�researcher�
Second,�power�is�determined�by�sample�size��As�sample�size�n�increases,�power�increases�� Thus,�if�sample�size�increases,�meaning�we�have�a�sample�that�consists�of�a�larger�propor- tion�of�the�population,�this�will�cause�the�standard�error�of�the�mean�to�decrease,�as�there�
Type II error (β)
Do not reject H0. Correct decision (1 – α)
Correct decision (1 – β)
Reject H0.
Sampling distribution when
H0: µ = 100 is true.
Sampling distribution when
H1: µ = 115 is true
(i.e., H0: µ = 100 is false).
Type I error (α/2)
Type I error (α/2)
Critical value
Critical value
FIGuRe 6.4 Sampling�distributions�for�the�intelligence�test�case�
137Introduction to Hypothesis Testing: Inferences About a Single Mean
is�less�sampling�error�with�larger�samples��This�would�also�result�in�the�vertical�line�being� moved� to� the� left� (again� thereby� creating� a� larger� critical� region� and� thereby� making� it� easier�to�reject�the�null�hypothesis)��This�factor�is�also�under�the�control�of�the�researcher�� In� addition,� because� a� larger� sample� yields� a� smaller� standard� error,� it� will� be� easier� to� reject�H0�(all�else�being�equal),�and�the�CIs�generated�will�also�be�narrower�
Third,�power�is�determined�by�the�size�of�the�population�standard�deviation�σ��Although� not� under� the� researcher’s� control,� as� σ� increases,� power� decreases�� Thus,� if� σ� increases,� meaning�the�variability�in�the�population�is�larger,�this�will�cause�the�standard�error�of�the� mean�to�increase�as�there�is�more�sampling�error�with�larger�variability��This�would�result� in�the�vertical�line�being�moved�to�the�right��If�σ�decreases,�meaning�the�variability�in�the� population�is�smaller,�this�will�cause�the�standard�error�of�the�mean�to�decrease�as�there� is�less�sampling�error�with�smaller�variability��This�would�result�in�the�vertical�line�being� moved�to�the�left��Considering,�for�example,�the�one-sample�mean�test,�the�standard�error� of�the�mean�is�the�denominator�of�the�test�statistic�formula��When�the�standard�error�term� decreases,�the�denominator�is�smaller�and�thus�the�test�statistic�value�becomes�larger�(and� thereby�easier�to�reject�the�null�hypothesis)�
Fourth,�power�is�determined�by�the�difference�between�the�true�population�mean�μ�and� the�hypothesized�mean�value�μ0��Although�not�always�under�the�researcher’s�control�(only� in�true�experiments�as�described�in�Chapter�14),�as�the�difference�between�the�true�popula- tion�mean�and�the�hypothesized�mean�value�increases,�power�increases��Thus,�if�the�differ- ence�between�the�true�population�mean�and�the�hypothesized�mean�value�is�large,�it�will� be� easier� to� correctly� reject� H0�� This� would� result� in� greater� separation� between� the� two� sampling�distributions��In�other�words,�the�entire�H1�is�true�sampling�distribution�would� be� shifted� to� the� right�� Consider,� for� example,� the� one-sample� mean� test�� The� numerator� is�the�difference�between�the�means��The�larger�the�numerator�(holding�the�denominator� constant),�the�more�likely�it�will�be�to�reject�the�null�hypothesis�
Finally,� power� is� determined� by� directionality� and� type� of� statistical� procedure— whether� we� conduct� a� one-� or� a� two-tailed� test� as� well� as� the� type� of� test� of� inference�� There�is�greater�power�in�a�one-tailed�test,�such�as�when�μ�>�100,�than�in�a�two-tailed�test�� In�a�one-tailed�test,�the�vertical�line�will�be�shifted�to�the�left,�creating�a�larger�rejection� region��This�factor�is�under�the�researcher’s�control��There�is�also�often�greater�power�in� conducting�parametric�as�compared�to�nonparametric�tests�of�inference�(we�will�talk�more� about� parametric� versus� nonparametric� tests� in� later� chapters)�� This� factor� is� under� the� researcher’s�control�to�some�extent�depending�on�the�scale�of�measurement�of�the�variables� and�the�extent�to�which�the�assumptions�of�parametric�tests�are�met�
Power� has� become� of� much� greater� interest� and� concern� to� the� applied� researcher� in� recent�years��We�begin�by�distinguishing�between�a priori power,�when�power�is�deter- mined�as�a�study�is�being�planned�or�designed�(i�e�,�prior�to�the�study),�and�post hoc power,� when�power�is�determined�after�the�study�has�been�conducted�and�the�data�analyzed�
For�a�priori�power,�if�you�want�to�insure�a�certain�amount�of�power�in�a�study,�then�you�can� determine�what�sample�size�would�be�needed�to�achieve�such�a�level�of�power��This�requires� the�input�of�characteristics�such�as�α,�σ,�the�difference�between�μ�and�μ0,�and�one-�versus�two- tailed�test��Alternatively,�one�could�determine�power�given�each�of�those�characteristics��This� can� be� done� either� by� using� statistical� software� [such� as� Power� and� Precision,� Ex-Sample,� G*Power�(freeware),�or�a�CD�provided�with�the�Murphy,�Myors,�and�Wolach�(2008)�text]�or� by�using�tables�[the�most�definitive�collection�of�tables�being�in�Cohen�(1988)]�
For�post�hoc�power�(also�called�observed�power),�most�statistical�software�packages�(e�g�,� SPSS,�SAS,�STATGRAPHICS)�will�compute�this�as�part�of�the�analysis�for�many�types�of� inferential� statistics� (e�g�,� analysis� of� variance)�� However,� even� though� post� hoc� power� is�
138 An Introduction to Statistical Concepts
routinely�reported�in�some�journals,�it�has�been�found�to�have�some�flaws��For�example,� Hoenig�and�Heisey�(2001)�concluded�that�it�should�not�be�used�to�aid�in�interpreting�non- significant� results�� They� found� that� low� power� may� indicate� a� small� effect� (e�g�,� a� small� mean�difference)�rather�than�an�underpowered�study��Thus,�increasing�sample�size�may� not� make� much� of� a� difference�� Yuan� and� Maxwell� (2005)� found� that� observed� power� is� almost�always�biased�(too�high�or�too�low),�except�when�true�power�is��50��Thus,�we�do�not� recommend� the� sole� use� of� post� hoc� power� to� determine� sample� size� in� the� next� study;� rather�it�is�recommended�that�CIs�be�used�in�addition�to�post�hoc�power��(An�example�pre- sented�later�in�this�chapter�will�use�G*Power�to�illustrate�both�a�priori�sample�size�require- ments�given�desired�power�and�post�hoc�power�analysis�)
6.7 Statistical Versus Practical Significance
We�have�discussed�the�inferential�test�of�a�single�mean�in�terms�of�statistical�significance�� However,�are�statistically�significant�results�always�practically�significant?�In�other�words,� if�a�result�is�statistically�significant,�should�we�make�a�big�deal�out�of�this�result�in�a�practi- cal�sense?�Consider�again�the�simple�example�where�the�null�and�alternative�hypotheses� are�as�follows:
� H H0 000 00 0: 1 or : 1µ µ= − = �
� H H1 1: 1 or : 1µ µ≠ − ≠00 00 0
A�sample�mean�intelligence�test�score�of�Y – �=�101�is�observed�for�a�sample�size�of�n�=�2000�and�
a�known�population�standard�deviation�of�σY�=�15��If�we�perform�the�test�at�the��01�level�of� significance,�we�find�we�are�able�to�reject�H0�even�though�the�observed�mean�is�only�1�unit� away�from�the�hypothesized�mean�value��The�reason�is,�because�the�sample�size�is�rather� large,�a�rather�small�standard�error�of�the�mean�is�computed�(σY–�=�0�3354),�and�we�thus�reject� H0�as�the�test�statistic�(z�=�2�9815)�exceeds�the�critical�value�(z�=�2�5758)��Holding�the�mean� and�standard�deviation�constant,�if�we�had�a�sample�size�of�200�instead�of�2000,�the�standard� error�becomes�much�larger�(σY–�=�1�0607),�and�we�thus�fail�to�reject�H0�as�the�test�statistic� (z�=�0�9428)�does�not�exceed�the�critical�value�(z�=�2�5758)��From�this�example,�we�can�see�how� the�sample�size�can�drive�the�results�of�the�hypothesis�test,�and�how�it�is�possible�that�statisti- cal�significance�can�be�influenced�simply�as�an�artifact�of�sample�size�
Should�we�make�a�big�deal�out�of�an�intelligence�test�sample�mean�that�is�1�unit�away� from�the�hypothesized�mean�intelligence?�The�answer�is�“maybe�not�”�If�we�gather�enough� sample� data,� any� small� difference,� no� matter� how� small,� can� wind� up� being� statistically� significant�� Thus,� larger� samples� are� more� likely� to� yield� statistically� significant� results�� Practical�significance�is�not�entirely�a�statistical�matter��It�is�also�a�matter�for�the�substan- tive� field� under� investigation�� Thus,� the� meaningfulness� of� a� small� difference� is� for� the� substantive�area�to�determine��All�that�inferential�statistics�can�really�determine�is�statis- tical� significance�� However,� we� should� always� keep� practical� significance� in� mind� when� interpreting�our�findings�
In�recent�years,�a�major�debate�has�been�ongoing�in�the�statistical�community�about�the� role�of�significance�testing��The�debate�centers�around�whether�null�hypothesis�significance� testing�(NHST)�best�suits�the�needs�of�researchers��At�one�extreme,�some�argue�that�NHST�is�
139Introduction to Hypothesis Testing: Inferences About a Single Mean
fine�as�is��At�the�other�extreme,�others�argue�that�NHST�should�be�totally�abandoned��In�the� middle,�yet�others�argue�that�NHST�should�be�supplemented�with�measures�of�effect�size��In� this�text,�we�have�taken�the�middle�road�believing�that�more�information�is�a�better�choice�
Let�us�formally�introduce�the�notion�of�effect size��While�there�are�a�number�of�different� measures�of�effect�size,�the�most�commonly�used�measure�is�Cohen’s�δ�(delta)�or�d�(1988)�� For�the�population�case�of�the�one-sample�mean�test,�Cohen’s�delta�is�computed�as�follows:
� δ µ µ
σ =
− 0
For�the�corresponding�sample�case,�Cohen’s�d�is�computed�as�follows:
� d Y
s =
− µ0
For�the�one-sample�mean�test,�d�indicates�how�many�standard�deviations�the�sample�mean� is�from�the�hypothesized�mean��Thus,�if�d�=�1�0,�the�sample�mean�is�one�standard�deviation� away� from� the� hypothesized� mean�� Cohen� has� proposed� the� following� subjective� stan- dards�for�the�social�and�behavioral�sciences�as�a�convention�for�interpreting�d:�small�effect� size,�d�=��2;�medium�effect�size,�d�=��5;�large�effect�size,�d�=��8��Interpretation�of�effect�size� should�always�be�made�first�based�on�a�comparison�to�similar�studies;�what�is�considered� a�“small”�effect�using�Cohen’s�rule�of�thumb�may�actually�be�quite�large�in�comparison�to� other�related�studies�that�have�been�conducted��In�lieu�of�a�comparison�to�other�studies,� such�as�in�those�cases�where�there�are�no�or�minimal�related�studies,�then�Cohen’s�subjec- tive�standards�may�be�appropriate�
Computing�CIs�for�effect�sizes�is�also�valuable��The�benefit�in�creating�CIs�for�effect�size� values�is�similar�to�that�of�creating�CIs�for�parameter�estimates—CIs�for�the�effect�size�pro- vide�an�added�measure�of�precision�that�is�not�obtained�from�knowledge�of�the�effect�size� alone�� Computing� CIs� for� effect� size� indices,� however,� is� not� as� straightforward� as� simply� plugging�in�known�values�into�a�formula��This�is�because�d�is�a�function�of�both�the�popula- tion�mean�and�population�standard�deviation�(Finch�&�Cumming,�2009)��Thus,�specialized� software�must�be�used�to�compute�the�CIs�for�effect�sizes,�and�interested�readers�are�referred� to�appropriate�sources�(e�g�,�Algina�&�Keselman,�2003;�Algina,�Keselman,�&�Penfield,�2005;� Cumming�&�Finch,�2001)�
While�a�complete�discussion�of�these�issues�is�beyond�this�text,�further�information�on� effect� sizes� can� be� seen� in� special� sections� of� Educational and Psychological Measurement� (2001a;� 2001b)� and� Grissom� and� Kim� (2005),� while� additional� material� on� NHST� can� be� viewed� in� Harlow,� Mulaik,� and� Steiger� (1997)� and� a� special� section� of� Educational and Psychological Measurement� (2000,� October)�� Additionally,� style� manuals� (e�g�,� American� Psychological�Association,�2010)�often�provide�useful�guidelines�on�reporting�effect�size�
6.8 Inferences About μ When σ Is Unknown We�have�already�considered�the�inferential�test�involving�a�single�mean�when�the�popula- tion�standard�deviation�σ�is�known��However,�rarely�is�σ�known�to�the�applied�researcher�� When�σ�is�unknown,�then�the�z�test�previously�discussed�is�no�longer�appropriate��In�this�
140 An Introduction to Statistical Concepts
section,�we�consider�the�following:�the�test�statistic�for�inferences�about�the�mean�when�the� population�standard�deviation�is�unknown,�the�t�distribution,�the�t�test,�and�an�example� using�the�t�test�
6.8.1 New Test Statistic t
What�is�the�applied�researcher�to�do�then�when�σ�is�unknown?�The�answer�is�to�estimate� σ�by�the�sample�standard�deviation�s��This�changes�the�standard�error�of�the�mean�to�be
� s s nY Y=
Now�we�are�estimating�two�population�parameters:�(1)�the�population�mean,�μY,�is�being� estimated�by�the�sample�mean,�Y
– ;�and�(2)�the�population�standard�deviation,�σY,�is�being�
estimated�by�the�sample�standard�deviation,�sY��Both�Y – �and�sY�can�vary�from�sample�to�
sample��Thus,�although�the�sampling�error�of�the�mean�is�taken�into�account�explicitly�in� the�z�test,�we�also�need�to�take�into�account�the�sampling�error�of�the�standard�deviation,� which�the�z�test�does�not�at�all�consider��We�now�develop�a�new�inferential�test�for�the� situation�where�σ�is�unknown��The�test�statistic�is�known�as�the�t�test�and�is�computed� as�follows:
� t
Y sY
= − µ0
The� t� test� was� developed� by� William� Sealy� Gossett,� also� known� by� the� pseudonym� Student,�previously�mentioned�in�Chapter�1��The�unit�normal�distribution�cannot�be�used� here� for� the� unknown� σ� situation�� A� different� theoretical� distribution� must� be� used� for� determining�critical�values�for�the�t�test,�known�as�the�t�distribution�
6.8.2 t distribution
The�t�distribution�is�the�theoretical�distribution�used�for�determining�the�critical�values�of� the�t�test��Like�the�normal�distribution,�the�t�distribution�is�actually�a�family�of�distribu- tions�� There� is� a� different� t� distribution� for� each� value� of� degrees� of� freedom�� However,� before�we�look�more�closely�at�the�t�distribution,�some�discussion�of�the�degrees of free- dom�concept�is�necessary�
As�an�example,�say�we�know�a�sample�mean�Y – �=�6�for�a�sample�size�of�n�=�5��How�many�
of� those� five� observed� scores� are� free� to� vary?� The� answer� is� that� four� scores� are� free� to� vary��If�the�four�known�scores�are�2,�4,�6,�and�8�and�the�mean�is�6,�then�the�remaining�score� must�be�10��The�remaining�score�is�not�free�to�vary,�but�is�already�totally�determined��We� see�this�in�the�following�equation�where,�to�arrive�at�a�solution�of�6,�the�sum�in�the�numera- tor�must�equal�30,�and�Y5�must�be�10:
� Y
Y
n
Y Y
i
i
n
i
i= = = + + + +
== = ∑ ∑
1 1
5
5
5 2 4 6 8
5 6
Therefore,�the�number�of�degrees�of�freedom�is�equal�to�4�in�this�particular�case�and�n�−�1� in�general��For�the�t�test�being�considered�here,�we�specify�the�degrees�of�freedom�as�
141Introduction to Hypothesis Testing: Inferences About a Single Mean
ν� =� n� −� 1� (ν� is� the� Greek� letter� “nu”)�� We� use� ν� often� in� statistics� to� denote� some� type� of� degrees�of�freedom�
Another�way�to�think�about�degrees�of�freedom�is�that�we�know�the�sum�of�the�devia- tions� from� the� mean� must� equal� 0� (recall� the� unsquared� numerator� of� the� variance� con- ceptual�formula)��For�example,�if�n�=�10,�there�are�10�deviations�from�the�mean��Once�the� mean�is�known,�only�nine�of�the�deviations�are�free�to�vary��A�final�way�to�think�about�this� is�that,�in�general,�df�=�(n�−�number�of�restrictions)��For�the�one-sample�t�test,�because�the� population�variance�is�unknown,�we�have�to�estimate�it�resulting�in�one�restriction��Thus,� df�=�(n�−�1)�for�this�particular�inferential�test�
Several�members�of�the�family�of�t�distributions�are�shown�in�Figure�6�5��The�distribu- tion�for�ν�=�1�has�thicker�tails�than�the�unit�normal�distribution�and�a�shorter�peak��This� indicates�that�there�is�considerable�sampling�error�of�the�sample�standard�deviation�with� only�two�observations�(as�ν�=�2�−�1�=�1)��For�ν�=�5,�the�tails�are�thinner�and�the�peak�is� taller�than�for�ν�=�1��As�the�degrees�of�freedom�increase,�the�t�distribution�becomes�more� nearly� normal�� For� ν� =� ∞� (i�e�,� infinity),� the� t� distribution� is� precisely� the� unit� normal� distribution�
A� few� important� characteristics� of� the� t� distribution� are� worth� mentioning�� First,� like� the�unit�normal�distribution,�the�mean�of�any�t�distribution�is�0,�and�the�t�distribution�is� symmetric�around�the�mean�and�unimodal��Second,�unlike�the�unit�normal�distribution,� which�has�a�variance�of�1,�the�variance�of�a�t�distribution�is�as�follows:
� σ ν
ν ν2
2 2=
− >for
Thus,�the�variance�of�a�t�distribution�is�somewhat�greater�than�1�but�approaches�1�as� ν�increases�
The�table�for�the�t�distribution�is�given�in�Table�A�2,�and�a�snapshot�of�the�table�is�pre- sented�in�Figure�6�6�for�illustration�purposes��In�looking�at�the�table,�each�column�header� has�two�values��The�top�value�is�the�significance�level�for�a�one-tailed�test,�denoted�by�α1�� Thus,�if�you�were�doing�a�one-tailed�test�at�the��05�level�of�significance,�you�want�to�look�in� the�second�column�of�numbers��The�bottom�value�is�the�significance�level�for�a�two-tailed� test,�denoted�by�α2��Thus,�if�you�were�doing�a�two-tailed�test�at�the��05�level�of�significance,� you�want�to�look�in�the�third�column�of�numbers��The�rows�of�the�table�denote�the�various� degrees�of�freedom�ν�
0.4
0.3
0.2 Re
la tiv
e fr
eq ue
nc y
0.1
0 –4 0
t 4
1 5 Normal
FIGuRe 6.5 Several�members�of�the�family�of�t�distributions�
142 An Introduction to Statistical Concepts
Thus,�if�ν�=�3,�meaning�n�=�4,�you�want�to�look�in�the�third�row�of�numbers��If�ν�=�3�for� α1�=��05,�the�tabled�value�is�2�353��This�value�represents�the�95th�percentile�point�in�a�t�dis- tribution�with�three�degrees�of�freedom��This�is�because�the�table�only�presents�the�upper� tail�percentiles��As�the�t�distribution�is�symmetric�around�0,�the�lower�tail�percentiles�are� the�same�values�except�for�a�change�in�sign��The�fifth�percentile�for�three�degrees�of�free- dom�then�is�−2�353��Thus,�for�a�right-tailed�directional�hypothesis,�the�critical�value�will�be� +2�353,�and�for�a�left-tailed�directional�hypothesis,�the�critical�value�will�be�−2�353�
If�ν�=�120�for�α1�=��05,�then�the�tabled�value�is�1�658��Thus,�as�sample�size�and�degrees�of� freedom�increase,�the�value�of�t�decreases��This�makes�it�easier�to�reject�the�null�hypothesis� when�sample�size�is�large�
6.8.3 t Test
Now�that�we�have�covered�the�theoretical�distribution�underlying�the�test�of�a�single�mean� for�an�unknown�σ,�we�can�go�ahead�and�look�at�the�inferential�test��First,�the�null�and�alter- native�hypotheses�for�the�t�test�are�written�in�the�same�fashion�as�for�the�z�test�presented� earlier��Thus,�for�a�two-tailed�test,�we�have�the�same�notation�as�previously�presented:
� H H0 000 00 0: 1 or : 1µ µ= − =
� H H1 1: 1 or : 1µ µ≠ − ≠00 00 0 �
The�test�statistic�t�is�written�as�follows:
� t
Y sY
= − µ0
In�order�to�use�the�theoretical�t�distribution�to�determine�critical�values,�we�must�assume� that�Yi�∼�N(μ,�σ2)�and�that�the�observations�are�independent�of�each�other�(also�referred�to� as�“independent�and�identically�distributed”�or�IID)��In�terms�of�the�distribution�of�scores� on� Y,� in� other� words,� we� assume� that� the� population� of� scores� on� Y� is� normally� distrib- uted�with�some�population�mean�μ�and�some�population�variance�σ2��The�most�important� assumption�for�the�t�test�is�normality�of�the�population��Conventional�research�has�shown� that� the� t� test� is� very� robust� to� nonnormality� for� a� two-tailed� test� except� for� very� small� samples�(e�g�,�n�<�5)��The�t�test�is�not�as�robust�to�nonnormality�for�a�one-tailed�test,�even� for�samples�as�large�as�40�or�more�(e�g�,�Noreen,�1989;�Wilcox,�1993)��Recall�from�Chapter�5� on�the�central�limit�theorem�that�when�sample�size�increases,�the�sampling�distribution�of� the�mean�becomes�more�nearly�normal��As�the�shape�of�a�population�distribution�may�be� unknown,�conservatively�one�would�do�better�to�conduct�a�two-tailed�test�when�sample� size�is�small,�unless�some�normality�evidence�is�available�
1ν = .10 1 = .20
.05
.10 .025 .050
.01
.02 .005 .010
.0025
.0050 .001 .002
.0005
.0010 1 3.078 6.314 12.706 31.821 63.657 127.32 318.31 636.62 2 1.886 2.920 4.303 6. 965 9.925 14.089 22.327 31.598 3 1.638 2.353 3.182 4.541 5.841 7.453 10.214 12.924 … … … … … … … … …
FIGuRe 6.6 Snapshot�of�t�distribution�table�
143Introduction to Hypothesis Testing: Inferences About a Single Mean
However,�recent�research�(e�g�,�Basu�&�DasGupta,�1995;�Wilcox,�1997,�2003)�suggests�that� small�departures�from�normality�can�inflate�the�standard�error�of�the�mean�(as�the�stan- dard�deviation�is�larger)��This�can�reduce�power�and�also�affect�control�over�Type�I�error�� Thus,�a�cavalier�attitude�about�ignoring�nonnormality�may�not�be�the�best�approach,�and� if� nonnormality� is� an� issue,� other� procedures,� such� as� the� nonparametric� Kolmogorov– Smirnov�one-sample�test,�may�be�considered��In�terms�of�the�assumption�of�independence,� this�assumption�is�met�when�the�cases�or�units�in�your�sample�have�been�randomly�selected� from� the� population�� Thus,� the� extent� to� which� this� assumption� is� met� is� dependent� on� your�sampling�design��In�reality,�random�selection�is�often�difficult�in�education�and�the� social�sciences�and�may�or�may�not�be�feasible�given�your�study�
The� critical� values� for� the� t� distribution� are� obtained� from� the� t� table� in� Table� A�2,� where�you�take�into�account�the�α�level,�whether�the�test�is�one-�or�two-tailed,�and�the� degrees�of�freedom�ν�=�n�−�1��If�the�test�statistic�falls�into�a�critical�region,�as�defined�by� the�critical�values,�then�our�conclusion�is�to�reject�H0��If�the�test�statistic�does�not�fall�into� a�critical�region,�then�our�conclusion�is�to�fail�to�reject�H0��For�the�t�test,�the�critical�values� depend�on�sample�size,�whereas�for�the�z�test,�the�critical�values�do�not�
As�was�the�case�for�the�z�test,�for�the�t�test,�a�CI�for�μ0�can�be�developed��The�(1�−�α)%�CI� is�formed�from
� Y t scv Y±
where�tcv�is�the�critical�value�from�the�t�table��If�the�hypothesized�mean�value�μ0�is�not�con- tained�in�the�interval,�then�our�conclusion�is�to�reject�H0��If�the�hypothesized�mean�value� μ0�is�contained�in�the�interval,�then�our�conclusion�is�to�fail�to�reject�H0��The�CI�procedure� for�the�t�test�then�is�comparable�to�that�for�the�z�test�
6.8.4 example
Let�us�consider�an�example�of�the�entire�t�test�process��A�hockey�coach�wanted�to�determine� whether� the� mean� skating� speed� of� his� team� differed� from� the� hypothesized� league� mean� speed�of�12�seconds��The�hypotheses�are�developed�as�a�two-tailed�test�and�written�as�follows:
� H H0 0 0: 12 or : 12µ µ= − =
� H H1 1: 12 or : 12µ µ≠ − ≠ 0
Skating�speed�around�the�rink�was�timed�for�each�of�16�players�(data�are�given�in�Table�6�2�and� on�the�website�as�chap6data)��The�mean�speed�of�the�team�was�Y
– �=�10�seconds�with�a�standard�
deviation�of�sY�=�1�7889�seconds��The�standard�error�of�the�mean�is�then�computed�as�follows:
� s
s nY Y= = =
1 7889 16
0 4472 .
.
We�wish�to�conduct�a�t�test�at�α�=��05,�where�we�compute�the�test�statistic�t�as
� t
Y sY
= −
= −
= − µ0 10 12
0 4472 4 4722
. .
144 An Introduction to Statistical Concepts
Table 6.2
SPSS�Output�for�Skating�Example
Raw data: 8, 12, 9, 7, 8, 10, 9, 11, 13.5, 8.5, 10.5, 9.5, 11.5, 12.5, 9.5, 10.5
One-Sample Statistics
N Mean Std. Deviation Std. Error Mean
Time 16 10.000 1.7889 .4472
One-Sample Test
Test Value = 12
95% Confidence Interval of the Difference
t df Sig. (2-Tailed) Mean Difference Lower Upper
Time –4.472 15
“t” is the t test statistic value.
.000 –2.0000 –2.953 –1.047
“Sig.” is the observed p value.
It is interpreted as: there is less than a 1% probability of
a sample mean of 10.00 occurring by chance if the null
hypothesis is really true (i.e., if the population mean
is really 12).
The mean difference is simply the difference
between the sample mean value (in this case, 10)
and the hypothesized mean value (in this example, 12).
In other words, 10 – 12 = –2.00
df are the degrees of freedom. For the one sample t test, they
are calculated as n – 1
The table labeled “One-Sample Statistics” provides basic
descriptive statistics for the sample.
The standard error of the mean is:
sY sY
n =–
sY 1.7889= =–
16 0.4472
.4472 t
10 – 12= = –4.472
SPSS reports the 95% confidence interval of the difference which means that in 95% of sample CIs, the true population mean difference will fall between –2.953 and –1.047. It is computed as:
The 95% confidence interval of the mean (although not provided by SPSS) could also be calculated as:
–2.00 ± (2.131)(.4472) ± sYtcv
– –Ydifference
10 ± 2.131(0.4472) = 10 ± ( .9530) [9.047, 10.953]
± sYtcv ––Y
sY– t =
– Y – μ0
=
145Introduction to Hypothesis Testing: Inferences About a Single Mean
We� turn� to� the�t� table� in�Table� A�2�and� determine� the�critical� values� based� on� α2�=��05� and�ν�=�15�degrees�of�freedom��The�critical�values�are�+�2�131,�which�defines�the�upper�tail� critical�region,�and�−2�131,�which�defines�the�lower�tail�critical�region��As�the�test�statistic t� (i�e�, −4�4722)� falls� into� the� lower� tail� critical� region� (i�e�,� the� test� statistic� is� less� than� the� lower� tail� critical� value),� our� decision� is� to� reject� H0� and� conclude� that� the� mean� skating� speed�of�this�team�is�significantly�different�from�the�hypothesized�league�mean�speed�at�the� �05�level�of�significance��A�95%�CI�can�be�computed�as�follows:
� Y t scv Y± = ± = ± =10 2 131 0 4472 10 9530 9 0470 10 9530. ( . ) (. ) ( . , . )
As�the�CI�does�not�contain�the�hypothesized�mean�value�of�12,�our�conclusion�is�to�again� reject�H0��Thus,�there�is�evidence�to�suggest�that�the�mean�skating�speed�of�the�team�differs� from�the�hypothesized�league�mean�speed�of�12�seconds�
6.9 SPSS
Here�we�consider�what�SPSS�has�to�offer�in�the�way�of�testing�hypotheses�about�a�single� mean��As�with�most�statistical�software,�the�t�test�is�included�as�an�option�in�SPSS,�but�the� z�test�is�not��Instructions�for�determining�the�one-sample�t�test�using�SPSS�are�presented� first��This�is�followed�by�additional�steps�for�examining�the�normality�assumption�
One-Sample t Test
Step 1:�To�conduct�the�one-sample�t�test,�go�to�“Analyze”�in�the�top�pulldown�menu,� then�select�“Compare Means,”�and�then�select�“One-Sample T Test.”�Following�the� screenshot�(step�1)�as�follows�produces�the�“One-Sample T Test”�dialog�box�
A
B C
Step 1
146 An Introduction to Statistical Concepts
Step 2:�Next,�from�the�main�“One-Sample T Test”�dialog�box,�click�the�variable�of� interest�from�the�list�on�the�left�(e�g�,�time),�and�move�it�into�the�“Test Variable”�box�by� clicking�on�the�arrow�button��At�the�bottom�right�of�the�screen�is�a�box�for�“Test Value,”� where�you�indicate�the�hypothesized�value�(e�g�,�12)�
Select the variable of interest from
the list on the left and use the arrow
to move to the “Test Variable”
box on the right.
Clicking on “Options” will
allow you to define a confidence interval
percentage. �e default is 95% (corresponding to an alpha of .05).
Step 2
Step 3 (Optional):�The�default�alpha�level�in�SPSS�is��05,�and,�thus,�the�default�cor- responding�CI�is�95%��If�you�wish�to�test�your�hypothesis�at�an�alpha�level�other�than��05� (and�thus�obtain�CIs�other�than�95%),�then�click�on�the�“Options”�button�located�in�the� top�right�corner�of�the�main�dialog�box��From�here,�the�CI�percentage�can�be�adjusted�to� correspond�to�the�alpha�level�at�which�your�hypothesis�is�being�tested��(For�purposes�of� this�example,�the�test�has�been�generated�using�an�alpha�level�of��05�)
Step 3
The�one-sample�t�test�output�for�the�skating�example�is�provided�in�Table�6�2�
Using Explore to Examine Normality of Sample Distribution
Generating normality evidence:�As�alluded�to�earlier�in�the�chapter,�understanding� the�distributional�shape�of�your�variable,�specifically�the�extent�to�which�normality�is�a�reason- able�assumption,�is�important��In�earlier�chapters,�we�saw�how�we�could�use�the�“Explore”� tool�in�SPSS�to�generate�a�number�of�useful�descriptive�statistics��In�conducting�our�one-sample� t�test,�we�can�again�use�“Explore”�to�examine�the�extent�to�which�the�assumption�of�normal- ity� is� met� for� our� sample� distribution�� As� the� general� steps� for� accessing�“Explore”� have�
147Introduction to Hypothesis Testing: Inferences About a Single Mean
been�presented�in�previous�chapters�(e�g�,�Chapter�4),�they�will�not�be�reiterated�here��After�the� variable�of�interest�has�been�selected�and�moved�to�the�“Dependent List”�box�on�the�main� “Explore”�dialog�box,�click�on�“Plots”�in�the�upper�right�corner��Place�a�checkmark�in�the� boxes�for�“Normality plots with tests”�and�also�for “Histogram.”
Select the variable of interest from
the list on the left and use the arrow
to move to the “Dependent
List” box on the right. Then click on “Plots.”
Generating normality
evidence
Interpreting normality evidence:� We� have� already� developed� a� good� under- standing�of�how�to�interpret�some�forms�of�evidence�of�normality,�including�skewness�and� kurtosis,�histograms,�and�boxplots��Using�our�hockey�data,�the�skewness�statistic�is��299� and�kurtosis�is�−�483—both�within�the�range�of�an�absolute�value�of�2�0,�suggesting�some� evidence�of�normality��The�histogram�also�suggests�relative�normality�
3
Histogram
2
1
Fr eq
ue nc
y
0 8.0 10.0
Time 12.0 14.0
Mean = 10.0 Std. dev. = 1.789 N = 16
148 An Introduction to Statistical Concepts
There�are�a�few�other�statistics�that�can�be�used�to�gauge�normality�as�well��Using�SPSS,�we� can�obtain�two�statistical�tests�of�normality��The�Kolmogorov–Smirnov�(K–S)�(Chakravart,� Laha,�&�Roy,�1967)�with�Lilliefors�significance�(Lilliefors,�1967)�and�the�Shapiro-Wilk�(S–W)� (Shapiro�&�Wilk,�1965)�are�tests�that�provide�evidence�of�the�extent�to�which�our�sample� distribution�is�statistically�different�from�a�normal�distribution��The�K–S�test�tends�to�be� conservative,�whereas�the�S–W�test�is�usually�considered�the�more�powerful�of�the�two�for� testing�normality�and�is�recommended�for�use�with�small�sample�sizes�(n�<�50)��Both�of� these�statistics�are�generated�from�the�selection�of�“Normality plots with tests.”� The�output�for�the�K–S�and�S–W�tests�is�presented�as�follows��As�we�have�learned�in�this� chapter,�when�the�observed�probability�(i�e�,�p�value�which�is�reported�in�SPSS�as�“Sig�”)�is� less�than�our�stated�alpha�level,�then�we�reject�the�null�hypothesis��We�follow�those�same� rules�of�interpretation�here��Regardless�of�which�test�(K–S�or�S–W)�we�examine,�both�pro- vide�the�same�evidence—our�sample�distribution�is�not�statistically�significantly�different� than�what�would�be�expected�from�a�normal�distribution�
Time a Lilliefors significance correction. * This is a lower bound of the true significance.
.110 16 .200 .982 16 .978
Statistic Statisticdf dfSig.
Tests of Normality
Kolmogorov–Smirnova Shapiro–Wilk
Sig.
Quantile–quantile� (Q–Q)� plots� are� also� often� examined� to� determine� evidence� of� nor- mality��Q–Q�plots�are�graphs�that�depict�quantiles�of�the�sample�distribution�to�quantiles� of� the� theoretical� normal� distribution�� Points� that� fall� on� or� closely� to� the� diagonal� line� suggest�evidence�of�normality��The�Q–Q�plot�of�our�hockey�skating�time�provides�another� form�of�evidence�of�normality�
3
2
1
Ex pe
ct ed
n or
m al
0
–1
–2
6 8 10 Observed value
12 14
Normal Q–Q plot of time
149Introduction to Hypothesis Testing: Inferences About a Single Mean
The� detrended� normal� Q–Q� plot� shows� deviations� of� the� observed� values� from� the� theoretical� normal� distribution�� Evidence� of� normality� is� suggested� when� the� points� exhibit�little�or�no�pattern�around�0�(the�horizontal�line);�however�due�to�subjectivity�in� determining�the�extent�of�a�pattern,�this�graph�can�often�be�difficult�to�interpret��Thus,� in� many� cases,� you� may� wish� to� rely� more� heavily� on� the� other� forms� of� evidence� of� normality�
0.4
0.3
0.2
0.1
D ev
. fr
om n
or m
al
0.0
–0.1
–0.2
8 10 Observed value
12 14
Detrended normal Q–Q plot of time
6.10 G*Power
In�our�discussion�of�power�presented�earlier�in�this�chapter,�we�indicated�that�the�sample� size� to� achieve� a� desired� level� of� power� can� be� determined� a� priori� (before� the� study� is� conducted),�and�observed�power�can�also�be�determined�post�hoc�(after�the�study�is�con- ducted)�using�statistical�software�or�power�tables��One�freeware�program�for�calculating� power� is� G*Power� (http://www�psycho�uni-duesseldorf�de/abteilungen/aap/gpower3/),� which� can� be� used� to� compute� both� a� priori� sample� size� and� post� hoc� power� analyses� (among�other�things)��Using�the�results�of�the�one-sample�t�test�just�conducted,�let�us�uti- lize�G*Power�to�first�determine�the�required�sample�size�given�various�estimated�param- eters�and�then�compute�the�post�hoc�power�of�our�test�
A Priori Sample Size Using G*Power
Step 1 (A priori sample size):�As�seen�in�step�1,�there�are�several�decisions�that� need�to�be�made�from�the�initial�G*Power�screen��First,�the�correct�test�family�needs�to�be� selected��In�our�case,�we�conducted�a�one-sample�t�test;�therefore,�the�default�selection�of�
150 An Introduction to Statistical Concepts
“t tests”�is�the�correct�test�family��Next,�we�need�to�select�the�appropriate�statistical�test�� The�default�is�“Correlation: Point biserial model.”�This�is�not�the�correct�option� for� us,� and� so� we� use� the� arrow� to� toggle� to�“Means: Difference from constant (one sample case).”
�e default selection for “Test Family” is “t tests.”
�e default selection for “Statistical Test” is “Correlation: Point biserial model.” Use the arrow to toggle to the desired statistical test. For the one sample t test, we need “Means: Difference from constant (one sample case).”
Step 1
�is is the test needed for a one sample t test.
Step 2 (A priori sample size):�The�“Type of Power Analysis”�desired�then� needs� to� be� selected�� The� default� is“� A� priori:� Compute� required� sample� size—given� α,� power,�and�effect�size�”�For�this�illustration,�we�will�first�conduct�an�example�of�comput- ing� the� a� priori� sample� size� (i�e�,� the� default� option),� and� then�we� will�compute� post�hoc� power��Although�we�do�not�illustrate�the�use�of�these�here,�we�see�that�there�are�also�three� additional�forms�of�power�analysis�that�can�be�conducted�using�G*Power:�(1)�compromise,� (2)�criterion,�and�(3)�sensitivity�
151Introduction to Hypothesis Testing: Inferences About a Single Mean
The default selection for “Type of Power Analysis” is “A priori: Compute required sample size–given ,
power, and effect size.”
Step 2
Step 3 (A priori sample size):� The� “Input Parameters”� must� then� be� specified�� The� first� parameter� is� the� selection� of� whether� your� test� is� one-tailed� (i�e�,� directional)� or� two-tailed� (i�e�,� nondirectional)�� In� this� example,� we� have� a� two-tailed� test,� so� we� use� the� arrow� to� toggle�“Tails”� to�“Two.”� For� a� priori� power,� we� have� to� indicate� the� anticipated� effect� size�� Your� best� estimate� of� the� effect� size� you� can� anticipate� achieving� is� usually� to� rely� on� previous� studies� that� have� been� conducted� that� are� similar� to� yours�� In� G*Power,� the� default� effect� size� is� d� =� �50�� For� purposes� of� this� illustration,� let� us� use� the� default�� The� alpha� level� must� also� be� defined�� The� default� significance� level� in� G*Power� is� �05,� which� is� the� alpha� level� we� will� be� using� for�our�example��The�desired�level�of�power�must�also�be�defined��The�G*Power�default� for� power� is� �95�� Many� researchers� in� education� and� the� behavioral� sciences� indicate� that�a�desired�power�of��80�or�above�is�usually�desired��Thus,��95�may�be�higher�than�
152 An Introduction to Statistical Concepts
what� many� would� consider� sufficient� power�� For� purposes� of� this� example,� however,� we�will�use�the�default�power�of��95��Once�the�parameters�are�specified,�simply�click�on� “Calculate”�to�generate�the�a�priori�power�statistics�
Once the parameters are
specified, click on “Calculate.”
Step 3
The “Input Parameters” to determine a prior sample size must be specified including:
1. One versus two tailed test; 2. Anticipated effect size; 3. Alpha level; and 4. Desired power.
Step 4 (A priori sample size):�The�“Output Parameters”�provide�the�relevant� statistics�given�the�input�specified��In�this�example,�we�were�interested�in�determining�the� a�priori�sample�size�given�a�two-tailed�test,�with�an�anticipated�effect�size�of��50,�an�alpha� level�of��05,�and�desired�power�of��95��Based�on�those�criteria,�the�required�sample�size�for� our�one-sample�t�test�is�54��In�other�words,�if�we�have�a�sample�size�of�54�individuals�or� cases�in�our�study,�testing�at�an�alpha�level�of��05,�with�a�two-tailed�test,�and�achieving�a� moderate�effect�size�of��50,�then�the�power�of�our�test�will�be��95—the�probability�of�reject- ing�the�null�hypothesis�when�it�is�really�false�will�be�95%�
153Introduction to Hypothesis Testing: Inferences About a Single Mean
Step 4
The “Output Parameters” provide the relevant statistics given the input specified.
In this case, we were interested in determining the required sample size given
various parameters. Based on the parameters specified, we need a sample size
of 54 for our one sample t test.
If�we�had�anticipated�a�smaller�effect�size,�say��20,�but�left�all�of�the�other�input�parameters� the�same,�the�required�sample�size�needed�to�achieve�a�power�of��95�increases�greatly—to�327�
If a small e�ect is anticipated, the needed sample size increases greatly
to achieve the desired power.
Post Hoc Power Using G*Power
Now,�let�us�use�G*Power�to�compute�post�hoc�power��Step�1,�as�presented�earlier�for�a�priori� power,�remains�the�same;�thus,�we�will�start�from�step�2�
154 An Introduction to Statistical Concepts
Step 2 (Post hoc power):�The�“Type of Power Analysis”�desired�then�needs�to� be�selected��The�default�is�“A�priori:�Compute�required�sample�size—given�α,�power,�and� effect�size�”�To�compute�post�hoc�power,�we�need�to�select�“Post�hoc:�Compute�achieved� power—given�α,�sample�size,�and�effect�size�” Step 3 (Post hoc power):�The�“Input Parameters”�must�then�be�specified��The� first�parameter�is�the�selection�of�whether�your�test�is�one-tailed�(i�e�,�directional)�or�two- tailed�(i�e�,�nondirectional)��In�this�example,�we�have�a�two-tailed�test�so�we�use�the�arrow� to�toggle�to�“Tails”�to�“Two.”�The�achieved�or�observed�effect�size�was�−1�117��The�alpha� level� we� tested� at� was� �05,� and� the� actual� sample� size� was� 16�� Once� the� parameters� are� specified,�simply�click�on�“Calculate”�to�generate�the�achieved�power�statistics�
Step 4 (Post hoc power):�The�“Output Parameters”�provide�the�relevant�statis- tics�given�the�input�specified��In�this�example,�we�were�interested�in�determining�post�hoc� power�given�a�two-tailed�test,�with�an�observed�effect�size�of�−1�117,�an�alpha�level�of��05,�and� sample�size�of�16��Based�on�those�criteria,�the�post�hoc�power�was��96��In�other�words,�with�a� sample�size�of�16�skaters�in�our�study,�testing�at�an�alpha�level�of��05,�with�a�two-tailed�test,� and�observing�a�large�effect�size�of�−1�117,�then�the�power�of�our�test�was��96—the�probability� of�rejecting�the�null�hypothesis�when�it�is�really�false�will�be�96%,�an�excellent�level�of�power�� Keep�in�mind�that�conducting�power�analysis�a�priori�is�highly�recommended�so�that�you� avoid�a�situation�where,�post�hoc,�you�find�that�the�sample�size�was�not�sufficient�to�reach�the� desired�power�(given�the�observed�effect�size�and�alpha�level)�
The “Input Parameters” must be specified including:
Once the parameters are
specified, click on “Calculate.”
Steps 2–4
1. One versus two tailed test; 2. Actual effect size (for post hoc power); 3. Alpha level; and 4. Total sample size.
155Introduction to Hypothesis Testing: Inferences About a Single Mean
6.11 Template and APA-Style Write-Up
Let� us� revisit� our� graduate� research� assistant,� Marie,� who� was� working� with� Oscar,� a� local� hockey� coach,� to� assist� in� analyzing� his� team’s� data�� As� a� reminder,� her� task� was� to� assist� Oscar� in� generating� the� test� of� inference� to� answer� his� research� question,� “Is the mean skating speed of our hockey team different from the league mean speed of 12 seconds”?� Marie�suggested�a�one-sample�test�of�means�as�the�test�of�inference��A�template�for�writ- ing�a�research�question�for�a�one-sample�test�of�inference�(i�e�,�one-sample�t�test)�is�pre- sented�as�follows:
Is the mean of [sample variable] different from [hypothesized mean value]?
It�may�be�helpful�to�preface�the�results�of�the�one-sample�t�test�with�the�information�we� gathered�to�examine�the�extent�to�which�the�assumption�of�normality�was�met��This�assists� the�reader�in�understanding�that�you�were�thorough�in�data�screening�prior�to�conducting� the�test�of�inference�
The distributional shape of skating speed was examined to determine the extent to which the assumption of normality was met. Skewness (.299, SE = .564), kurtosis (−.483, SE = 1.091), and the Shapiro-Wilk test of normality (S-W = .982, df = 16, p = .978) suggest that normality is a reasonable assumption. Visually, a relatively bell-shaped distribution displayed in the histogram (reflected similarly in the boxplot) as well as a Q–Q plot with points adhering closely to the diagonal line also suggest evidence of normality. Additionally, the boxplot did not suggest the presence of any potential outliers. These indices suggest evidence that the assumption of normality was met.
An� additional� assumption� of� the� one-sample� t� test� is� the� assumption� of� independence�� This�assumption�is�met�when�the�cases�in�our�sample�have�been�randomly�selected�from� the� population�� This� is� an� often� overlooked,� but� important,� assumption� for� researchers� when� presenting� the� results� of� their� test�� One� or� two� sentences� are� usually� sufficient� to� indicate�if�this�assumption�was�met�
Because the skaters in this sample represented a random sample, the assumption of independence was met.
It�is� also� desirable� to�include� a� measure� of� effect� size�� Recall� our� formula�for� computing� the�effect�size,�d,�presented�earlier�in�the�chapter��Plugging�in�the�values�for�our�skating� example,�we�find�an�effect�size�of�−1�117,�interpreted�according�to�Cohen’s�(1988)�guidelines� as�a�large�effect:
� d
Y s
= −
= −
= − µ0 10 12
1 7889 1 117
. .
Remember�that�for�the�one-sample�mean�test,�d�indicates�how�many�standard�deviations� the�sample�mean�is�from�the�hypothesized�mean��Thus,�with�an�effect�size�of�−1�117,�there�
156 An Introduction to Statistical Concepts
are�nearly�one�and�one-quarter�standard�deviation�units�between�our�sample�mean�and� the�hypothesized�mean��The�negative�sign�simply�indicates�that�our�sample�mean�was�the� smaller� mean� (as� it� is� the� first� value� in� the� numerator� of� the� formula)�� In� this� particular� example,� the� negative� effect� is� desired� as� it� suggests� the� team’s� average� skating� time� is� quicker�than�the�league�mean�
Here�is�an�example�APA-style�paragraph�of�results�for�the�skating�data�(remember�that� this�will�be�prefaced�by�the�paragraph�reporting�the�extent�to�which�the�assumptions�of� the�test�were�met)�
A one-sample t test was conducted at an alpha level of .05 to answer the research question: Is the mean skating speed of a hockey team dif- ferent from the league mean speed of 12 seconds? The null hypothesis stated that the team mean speed would not differ from the league mean speed of 12. The alternative hypothesis stated that the team average speed would differ from the league mean. As depicted in Table 6.2, based on a random sample of 16 skaters, there was a mean time of 10 seconds, and a standard deviation of 1.7889 seconds. When compared against the hypothesized mean of 12 seconds, the one-sample t test was shown to be statistically significant (t = −4.472, df = 15, p < .001). Therefore, the null hypothesis that the team average time would be 12 seconds was rejected. This provides evidence to suggest that the sample mean skating time for this particular team was statistically different from the hypothesized mean skating time of the league. Additionally, the effect size d was −1.117, generally interpreted as a large effect (Cohen, 1988), and indicating that there is more than a one standard deviation difference between the team and league mean skating times. The post hoc power of the test, given the sample size, two-tailed test, alpha level, and observed effect size, was .96.
6.12 Summary
In� this� chapter,� we� considered� our� first� inferential� testing� situation,� testing� hypotheses� about�a�single�mean��A�number�of�topics�and�new�concepts�were�discussed��First,�we�intro- duced�the�types�of�hypotheses�utilized�in�inferential�statistics,�that�is,�the�null�or�statistical� hypothesis�versus�the�scientific�or�alternative�or�research�hypothesis��Second,�we�moved� on�to�the�types�of�decision�errors�(i�e�,�Type�I�and�Type�II�errors)�as�depicted�by�the�deci- sion�table�and�illustrated�by�the�rain�example��Third,�the�level�of�significance�was�intro- duced� as� well� as� the� types� of� alternative� hypotheses� (i�e�,� nondirectional� vs�� directional� alternative�hypotheses)��Fourth,�an�overview�of�the�steps�in�the�decision-making�process� of� inferential� statistics� was� given�� Fifth,� we� examined� the� z� test,� which� is� the� inferential� test�about�a�single�mean�when�the�population�standard�deviation�is�known��This�was�fol- lowed� by� a� more� formal� description� of� Type� II� error� and� power�� We� then� discussed� the� notion�of�statistical�significance�versus�practical�significance��Finally,�we�considered�the�t� test,�which�is�the�inferential�test�about�a�single�mean�when�the�population�standard�devia- tion�is�unknown,�and�then�completed�the�chapter�with�an�example,�SPSS�information,�a� G*Power�illustration,�and�an�APA-style�write-up�of�results��At�this�point,�you�should�have�
157Introduction to Hypothesis Testing: Inferences About a Single Mean
met�the�following�objectives:�(a)�be�able�to�understand�the�basic�concepts�of�hypothesis�testing,� (b)�be�able�to�utilize�the�normal�and�t�tables,�and�(c)�be�able�to�understand,�determine,�and� interpret�the�results�from�the�z�test,�t test,�and�CI�procedures��Many�of�the�concepts�in�this� chapter�carry�over�into�other�inferential�tests��In�the�next�chapter,�we�discuss�inferential� tests�involving�the�difference�between�two�means��Other�inferential�tests�will�be�consid- ered�in�subsequent�chapters�
Problems
Conceptual problems
6.1� In� hypothesis� testing,� the� probability� of� failing� to� reject� H0� when� H0� is� false� is� denoted by
� a�� α � b�� 1�−�α � c�� β � d�� 1�−�β 6.2� The� probability� of� observing� the� sample� mean� (or� some� value� greater� than� the�
sample� mean)� by� chance� if� the� null� hypothesis� is� really� true� is� which� one� of� the� following?
� a�� α � b�� Level�of�significance � c�� p�value � d�� Test�statistic�value 6.3� When�testing�the�hypotheses�presented�in�the�following,�at�a��05�level�of�significance�
with�the�t�test,�where�is�the�rejection�region?
�
H
H
0
1
100
100
:
:
µ
µ
=
<
� a�� The�upper�tail � b�� The�lower�tail � c�� Both�the�upper�and�lower�tails � d�� Cannot�be�determined 6.4� A�research�question�asks,�“Is�the�mean�age�of�children�who�enter�preschool�different�
from�48�months”?�Which�one�of�the�following�is�implied? � a�� Left-tailed�test � b�� Right-tailed�test � c�� Two-tailed�test � d�� Cannot�be�determined�based�on�this�information
158 An Introduction to Statistical Concepts
6.5� The�probability�of�making�a�Type�II�error�when�rejecting�H0�at�the��05�level�of�signifi- cance�is�which�one�of�the�following?
� a�� 0 � b�� �05 � c�� Between��05�and��95 � d�� �95 6.6� If�the�90%�CI�does�not�include�the�value�for�the�parameter�being�estimated�in�H0,�then�
which�one�of�the�following�is�a�correct�statement? � a�� H0�cannot�be�rejected�at�the��10�level� � b�� H0�can�be�rejected�at�the��10�level� � c�� A�Type�I�error�has�been�made� � d�� A�Type�II�error�has�been�made� 6.7� Other�things�being�equal,�which�of�the�values�of�t�given�next�is�least�likely�to�result�
when�H0�is�true,�for�a�two-tailed�test? � a�� 2�67 � b�� 1�00 � c�� 0�00 � d�� −1�96 � e�� −2�70 6.8� The�fundamental�difference�between�the�z�test�and�the�t�test�for�testing�hypotheses�
about�a�population�mean�is�which�one�of�the�following? � a�� Only�z�assumes�the�population�distribution�be�normal� � b�� z�is�a�two-tailed�test,�whereas�t�is�one-tailed� � c�� Only�t�becomes�more�powerful�as�sample�size�increases� � d�� Only�z�requires�the�population�variance�be�known� 6.9� If�one�fails�to�reject�a�true�H0,�one�is�making�a�Type�I�error��True�or�false? 6.10� Which�one�of�the�following�is�a�correct�interpretation�of�d? � a�� Alpha�level � b�� CI � c�� Effect�size � d�� Observed�probability � e�� Power 6.11� A�one-sample�t�test�is�conducted�at�an�alpha�level�of��10��The�researcher�finds�a�p�value�
of��08�and�concludes�that�the�test�is�statistically�significant��Is�the�researcher�correct? 6.12� When�testing�the�following�hypotheses�at�the��01�level�of�significance�with�the�t�test,�a�
sample�mean�of�301�is�observed��I�assert�that�if�I�calculate�the�test�statistic�and�compare�it� to�the�t�distribution�with�n�−�1�degrees�of�freedom,�it�is�possible�to�reject�H0��Am�I�correct?
�
H
H
0
1
295
295
:
:
µ
µ
=
<
159Introduction to Hypothesis Testing: Inferences About a Single Mean
6.13� If�the�sample�mean�exceeds�the�hypothesized�mean�by�200�points,�I�assert�that�H0�can� be�rejected��Am�I�correct?
6.14� I� assert� that� H0� can� be� rejected� with� 100%� confidence� if� the� sample� consists� of� the� entire�population��Am�I�correct?
6.15� I� assert� that� the� 95%� CI� has� a� larger� width� than� the� 99%� CI� for� a� population� mean� using�the�same�data��Am�I�correct?
6.16� I�assert�that�the�critical�value�of�z,�for�a�test�of�a�single�mean,�will�increase�as�sample� size�increases��Am�I�correct?
6.17� The� mean� of� the� t� distribution� increases� as� degrees� of� freedom� increase?� True� or� false?
6.18� It�is�possible�that�the�results�of�a�one-sample�t�test�and�for�the�corresponding�CI�will� differ�for�the�same�dataset�and�level�of�significance��True�or�false?
6.19� The�width�of�the�95%�CI�does�not�depend�on�the�sample�mean��True�or�false? 6.20� The�null�hypothesis�is�a�numerical�statement�about�which�one�of�the�following? � a�� An�unknown�parameter � b�� A�known�parameter � c�� An�unknown�statistic � d�� A�known�statistic
Computational problems
6.1� Using�the�same�data�and�the�same�method�of�analysis,�the�following�hypotheses�are� tested� about� whether� mean� height� is� 72� inches�� Researcher� A� uses� the� �05� level� of� significance,�and�Researcher�B�uses�the��01�level�of�significance:
�
H
H
0
1
72
72
:
:
µ
µ
=
≠
� a�� If�Researcher�A�rejects�H0,�what�is�the�conclusion�of�Researcher�B? � b�� If�Researcher�B�rejects�H0,�what�is�the�conclusion�of�Researcher�A? � c�� If�Researcher�A�fails�to�reject�H0,�what�is�the�conclusion�of�Researcher�B? � d�� If�Researcher�B�fails�to�reject�H0,�what�is�the�conclusion�of�Researcher�A? 6.2� Give�a�numerical�value�for�each�of�the�following�descriptions�by�referring�to�the�
t�table� � a�� The�percentile�rank�of�t5�=�1�476 � b�� The�percentile�rank�of�t10�=�3�169 � c�� The�percentile�rank�of�t21�=�2�518 � d�� The�mean�of�the�distribution�of�t23 � e�� The�median�of�the�distribution�of�t23 � f�� The�variance�of�the�distribution�of�t23 � g�� The�90th�percentile�of�the�distribution�of�t27
160 An Introduction to Statistical Concepts
6.3� Give�a�numerical�value�for�each�of�the�following�descriptions�by�referring�to�the� t�table�
� a�� The�percentile�rank�of�t5�=�2�015 � b�� The�percentile�rank�of�t20�=�1�325 � c�� The�percentile�rank�of�t30�=�2�042 � d�� The�mean�of�the�distribution�of�t10 � e�� The�median�of�the�distribution�of�t10 � f�� The�variance�of�the�distribution�of�t10 � g�� The�95th�percentile�of�the�distribution�of�t14 6.4� The� following� random� sample� of� weekly� student� expenses� is� obtained� from� a�
normally� distributed� population� of� undergraduate� students� with� unknown� parameters:
68 56 76 75 62 81 72 69 91 84 49 75 69 59 70 53 65 78 71 87 71 74 69 65 64
� a�� Test�the�following�hypotheses�at�the��05�level�of�significance:
�
H
H
0
1
74
74
:
:
µ
µ
=
≠
� b�� Construct�a�95%�CI� 6.5� The�following�random�sample�of�hours�spent�per�day�answering�e-mail�is�obtained�
from�a�normally�distributed�population�of�community�college�faculty�with�unknown� parameters:
2 3�5 4 1�25 2�5 3�25 4�5 4�25 2�75 3�25 1�75 1�5 2�75 3�5 3�25 3�75 2�25 1�5 1�25 3�25
� a�� Test�the�following�hypotheses�at�the��05�level�of�significance:
�
H
H
0
1
3 0
3 0
:
:
µ
µ
=
≠
.
.
� b�� Construct�a�95%�CI�
161Introduction to Hypothesis Testing: Inferences About a Single Mean
6.6� In�the�population,�it�is�hypothesized�that�flags�have�a�mean�usable�life�of�100�days�� Twenty-five�flags�are�flown�in�the�city�of�Tuscaloosa�and�are�found�to�have�a�sample� mean�usable�life�of�200�days�with�a�standard�deviation�of�216�days��Does�the�sample� mean�in�Tuscaloosa�differ�from�that�of�the�population�mean?
� a�� Conduct�a�two-tailed�t�test�at�the��01�level�of�significance� � b�� Construct�a�99%�CI�
Interpretive problems
6.1� Using�item�7�from�the�survey�1�dataset�accessible�from�the�website,�use�SPSS�to�con- duct� a� one-sample� t� test� to� determine� whether� the� mean� number� of� compact� disks� owned�is�significantly�different�from�25,�at�the��05�level�of�significance��Test�for�the� extent�to�which�the�assumption�of�normality�has�been�met��Calculate�an�effect�size�as� well�as�post�hoc�power��Then�write�an�APA-style�paragraph�reporting�your�results�
6.2� Using� item� 14� from� the� survey� 1� dataset� accessible� from� the� website,� use� SPSS� to� conduct�a�one-sample�t�test�to�determine�whether�the�mean�number�of�hours�slept� is�significantly�different�from�8,�at�the��05�level�of�significance��Test�for�the�extent�to� which�the�assumption�of�normality�has�been�met��Calculate�an�effect�size�as�well�as� post�hoc�power��Then�write�an�APA-style�paragraph�reporting�your�results�
163
7 Inferences About the Difference Between Two Means
Chapter Outline
7�1� New�Concepts 7�1�1� Independent�Versus�Dependent�Samples 7�1�2� Hypotheses
7�2� Inferences�About�Two�Independent�Means 7�2�1� Independent�t�Test 7�2�2� Welch�t′�Test 7�2�3� Recommendations
7�3� Inferences�About�Two�Dependent�Means 7�3�1� Dependent�t�Test 7�3�2� Recommendations
7�4� SPSS 7�5� G*Power 7�6� Template�and�APA-Style�Write-Up
Key Concepts
� 1�� Independent�versus�dependent�samples � 2�� Sampling�distribution�of�the�difference�between�two�means � 3�� Standard�error�of�the�difference�between�two�means � 4�� Parametric�versus�nonparametric�tests
In�Chapter�6,�we�introduced�hypothesis�testing�and�ultimately�considered�our�first�inferen- tial�statistic,�the�one-sample�t�test��There�we�examined�the�following�general�topics:�types� of�hypotheses,�types�of�decision�errors,�level�of�significance,�steps�in�the�decision-making� process,�inferences�about�a�single�mean�when�the�population�standard�deviation�is�known� (the� z� test),� power,� statistical� versus� practical� significance,� and� inferences� about� a� single� mean�when�the�population�standard�deviation�is�unknown�(the�t�test)�
In�this�chapter,�we�consider�inferential�tests�involving�the�difference�between�two�means�� In�other�words,�our�research�question�is�the�extent�to�which�two�sample�means�are�statis- tically�different�and,�by�inference,�the�extent�to�which�their�respective�population�means� are�different��Several�inferential�tests�are�covered�in�this�chapter,�depending�on�whether�
164 An Introduction to Statistical Concepts
the� two� samples� are� selected� in� an� independent� or� dependent� manner,� and� on� whether� the�statistical�assumptions�are�met��More�specifically,�the�topics�described�include�the�fol- lowing�inferential�tests:�for�two�independent�samples—the�independent�t�test,�the�Welch� t′�test,�and�briefly�the�Mann–Whitney–Wilcoxon�test;�and�for�two�dependent�samples—the� dependent�t�test�and�briefly�the�Wilcoxon�signed�ranks�test��We�use�many�of�the�founda- tional�concepts�previously�covered�in�Chapter�6��New�concepts�to�be�discussed�include�the� following:� independent� versus� dependent� samples,� the� sampling� distribution� of� the� dif- ference�between�two�means,�and�the�standard�error�of�the�difference�between�two�means�� Our�objectives�are�that�by�the�end�of�this�chapter,�you�will�be�able�to�(a)�understand�the� basic�concepts�underlying�the�inferential�tests�of�two�means,�(b)�select�the�appropriate�test,� and�(c)�determine�and�interpret�the�results�from�the�appropriate�test�
7.1 New Concepts
Remember� Marie,� our� very� capable� educational� researcher� graduate� student?� Let� us� see� what�Marie�has�in�store�for�her�now…�
Marie’s�first�attempts�at�consulting�went�so�well�that�her�faculty�advisor�has�assigned� Marie�two�additional�consulting�responsibilities�with�individuals�from�their�commu- nity�� Marie� has� been� asked� to� consult� with� a� local� nurse� practitioner,� JoAnn,� who� is� studying�cholesterol�levels�of�adults�and�how�they�differ�based�on�gender��Marie�sug- gests�the�following�research�question:�Is there a mean difference in cholesterol level between males and females?� Marie� suggests� an� independent� samples� t� test� as� the� test� of� infer- ence��Her�task�is�then�to�assist�JoAnn�in�generating�the�test�of�inference�to�answer�her� research�question�
Marie�has�also�been�asked�to�consult�with�the�swimming�coach,�Mark,�who�works� with� swimming� programs� that� are� offered� through� their� local� Parks� and� Recreation� Department�� Mark� has� just� conducted� an� intensive� 2� month� training� program� for� a� group�of�10�swimmers��He�wants�to�determine�if,�on�average,�their�time�in�the�50�meter� freestyle�event�is�different�after�the�training��The�following�research�question�is�sug- gested�by�Marie:�Is there a mean difference in swim time for the 50-meter freestyle event before participation in an intensive training program as compared to swim time for the 50-meter free- style event after participation in an intensive training program?�Marie�suggests�a�dependent� samples�t�test�as�the�test�of�inference��Her�task�is�then�to�assist�Mark�in�generating�the� test�of�inference�to�answer�his�research�question�
Before� we� proceed� to� inferential� tests� of� the� difference� between� two� means,� a� few� new� concepts� need� to� be� introduced�� The� new� concepts� are� the� difference� between� the� selec- tion�of�independent�samples�and�dependent�samples,�the�hypotheses�to�be�tested,�and�the� sampling�distribution�of�the�difference�between�two�means�
7.1.1 Independent Versus dependent Samples
The�first�new�concept�to�address�is�to�make�a�distinction�between�the�selection�of�indepen- dent samples�and�dependent samples��Two�samples�are�independent�when�the�method� of� sample� selection� is� such� that� those� individuals� selected� for� sample� 1� do� not� have� any�
165Inferences About the Difference Between Two Means
relationship� to� those� individuals� selected� for� sample� 2�� In� other� words,� the� selection� of� individuals�to�be�included�in�the�two�samples�are�unrelated�or�uncorrelated�such�that�they� have�absolutely�nothing�to�do�with�one�another��You�might�think�of�the�samples�as�being� selected� totally� separate� from� one� another�� Because� the� individuals� in� the� two� samples� are�independent�of�one�another,�their�scores�on�the�dependent�variable,�Y,�should�also�be� independent�of�one�another��The�independence�condition�leads�us�to�consider,�for�example,� the�independent samples�t�test��(This�should�not,�however,�be�confused�with�the�assump- tion�of�independence,�which�was�introduced�in�the�previous�chapter��The�assumption�of� independence�still�holds�for�the�independent�samples�t�test,�and�we�will�talk�later�about� how�this�assumption�can�be�met�with�this�particular�procedure�)
Two� samples� are� dependent� when� the� method� of� sample� selection� is� such� that� those� individuals�selected�for�sample�1�do�have�a�relationship�to�those�individuals�selected�for� sample�2��In�other�words,�the�selections�of�individuals�to�be�included�in�the�two�samples� are�related�or�correlated��You�might�think�of�the�samples�as�being�selected�simultaneously� such�that�there�are�actually�pairs�of�individuals��Consider�the�following�two�typical�exam- ples��First,�if�the�same�individuals�are�measured�at�two�points�in�time,�such�as�during�a� pretest�and�a�posttest,�then�we�have�two�dependent�samples��The�scores�on�Y�at�time�1�will� be�correlated�with�the�scores�on�Y�at�time�2�because�the�same�individuals�are�assessed�at� both�time�points��Second,�if�husband-and-wife�pairs�are�selected,�then�we�have�two�depen- dent�samples��That�is,�if�a�particular�wife�is�selected�for�the�study,�then�her�corresponding� husband�is�also�automatically�selected—this�is�an�example�where�individuals�are�paired� or�matched�in�some�way�such�that�they�share�characteristics�that�makes�the�score�of�one� person�related�to�(i�e�,�dependent�on)�the�score�of�the�other�person��In�both�examples,�we� have�natural�pairs�of�individuals�or�scores��The�dependence�condition�leads�us�to�consider� the�dependent samples�t�test,�alternatively�known�as�the�correlated samples�t�test�or�the� paired samples�t�test��As�we�show�in�this�chapter,�whether�the�samples�are�independent� or�dependent�determines�the�appropriate�inferential�test�
7.1.2 hypotheses
The�hypotheses�to�be�evaluated�for�detecting�a�difference�between�two�means�are�as�fol- lows�� The� null� hypothesis� H0� is� that� there� is� no� difference� between� the� two� population� means,�which�we�denote�as�the�following:
� H H0 1 2 0 1 20: :µ µ µ µ− = =or
where μ1�is�the�population�mean�for�sample�1 μ2�is�the�population�mean�for�sample�2
Mathematically,�both�equations�say�the�same�thing��The�version�on�the�left�makes�it�clear� to�the�reader�why�the�term�“null”�is�appropriate��That�is,�there�is�no�difference�or�a�“null”� difference�between�the�two�population�means��The�version�on�the�right�indicates�that�the� population� mean� of� sample� 1� is� the� same� as� the� population� mean� of� sample� 2—another� way�of�saying�that�there�is�no�difference�between�the�means�(i�e�,�they�are�the�same)��The� nondirectional�scientific�or�alternative�hypothesis�H1�is�that�there�is�a�difference�between� the�two�population�means,�which�we�denote�as�follows:
� H H1 1 2 1 1 20: :µ µ µ µ− ≠ or ≠
166 An Introduction to Statistical Concepts
The�null�hypothesis�H0�will�be�rejected�here�in�favor�of�the�alternative�hypothesis�H1�if�the� population�means�are�different��As�we�have�not�specified�a�direction�on�H1,�we�are�will- ing�to�reject�either�if�μ1�is�greater�than�μ2�or�if�μ1�is�less�than�μ2��This�alternative�hypothesis� results�in�a�two-tailed�test�
Directional�alternative�hypotheses�can�also�be�tested�if�we�believe�μ1�is�greater�than�μ2,� denoted�as�follows:
� H H1 1 2 1 1 20: :µ µ µ µ− > >or
In�this�case,�the�equation�on�the�left�tells�us�that�when�μ2�is�subtracted�from�μ1,�a�positive� value�will�result�(i�e�,�some�value�greater�than�0)��The�equation�on�the�right�makes�it�some- what�clearer�what�we�hypothesize�
Or�if�we�believe�μ1�is�less�than�μ2,�the�directional�alternative�hypotheses�will�be�denoted� as�we�see�here:
� H H1 1 2 1 1 20: :µ − < <µ µ µor
In�this�case,�the�equation�on�the�left�tells�us�that�when�μ2�is�subtracted�from�μ1,�a�negative� value�will�result�(i�e�,�some�value�less�than�0)��The�equation�on�the�right�makes�it�somewhat� clearer�what�we�hypothesize��Regardless�of�how�they�are�denoted,�directional�alternative� hypotheses�result�in�a�one-tailed�test�
The� underlying� sampling� distribution� for� these� tests� is� known� as� the� sampling dis- tribution of the difference between two means��This�makes�sense,�as�the�hypotheses� examine�the�extent�to�which�two�sample�means�differ��The�mean�of�this�sampling�dis- tribution�is�0,�as�that�is�the�hypothesized�difference�between�the�two�population�means� μ1�−�μ2��The�more�the�two�sample�means�differ,�the�more�likely�we�are�to�reject�the�null� hypothesis��As�we�show�later,�the�test�statistics�in�this�chapter�all�deal�in�some�way�with� the�difference�between�the�two�means�and�with�the�standard�error�(or�standard�devia- tion)�of�the�difference�between�two�means�
7.2 Inferences About Two Independent Means
In� this� section,� three� inferential� tests� of� the� difference� between� two� independent� means� are� described:� the� independent� t� test,� the� Welch� t′� test,� and� briefly� the� Mann–Whitney– Wilcoxon�test��The�section�concludes�with�a�list�of�recommendations�
7.2.1 Independent t Test
First,�we�need�to�determine�the�conditions�under�which�the�independent�t�test�is�appropri- ate��In�part,�this�has�to�do�with�the�statistical�assumptions�associated�with�the�test�itself�� The�assumptions�of�the�independent�t�test�are�that�the�scores�on�the�dependent�variable�Y� (a)�are�normally�distributed�within�each�of�the�two�populations,�(b)�have�equal�population� variances�(known�as�homogeneity�of�variance�or�homoscedasticity),�and�(c)�are�indepen- dent�� (The� assumptions� of� normality� and� independence� should� sound� familiar� as� they� were�introduced�as�we�learned�about�the�one-sample�t�test�)�Later�in�the�chapter,�we�more�
167Inferences About the Difference Between Two Means
fully�discuss�the�assumptions�for�this�particular�procedure��When�these�assumptions�are� not�met,�other�procedures�may�be�more�appropriate,�as�we�also�show�later�
The�measurement�scales�of�the�variables�must�also�be�appropriate��Because�this�is�a�test� of�means,�the�dependent�variable�must�be�measured�on�an�interval�or�ratio�scale��The�inde- pendent�variable,�however,�must�be�nominal�or�ordinal,�and�only�two�categories�or�groups� of�the�independent�variable�can�be�used�with�the�independent�t�test��(In�later�chapters,�we� will�learn�about�analysis�of�variance�(ANOVA)�which�can�accommodate�an�independent� variable� with� more� than� two� categories�)� It� is� not� a� condition� of� the� independent� t� test� that�the�sample�sizes�of�the�two�groups�be�the�same��An�unbalanced�design�(i�e�,�unequal� sample�sizes)�is�perfectly�acceptable�
The�test�statistic�for�the�independent�t�test�is�known�as�t�and�is�denoted�by�the�following� formula:
� t
Y Y sY Y
= −
−
1 2
1 2
where Y –
1�and�Y –
2�are�the�means�for�sample�1�and�sample�2,�respectively sY Y1 2− �is�the�standard�error�of�the�difference�between�two�means
This�standard�error�is�the�standard�deviation�of�the�sampling�distribution�of�the�difference� between�two�means�and�is�computed�as�follows:
� s s
n nY Y p1 2
1 1
1 2 − = +
where�sp�is�the�pooled�standard�deviation�computed�as
� s
n s n s n n
p = − + −
+ − ( ) ( )1 1
2 2 2
2
1 2
1 1 2
and�where s1
2�and� s2 2 �are�the�sample�variances�for�groups�1�and�2,�respectively
n1�and�n2�are�the�sample�sizes�for�groups�1�and�2,�respectively
Conceptually,�the�standard�error� sY Y1 2− �is�a�pooled�standard�deviation�weighted�by�the� two� sample� sizes;� more� specifically,� the� two� sample� variances� are� weighted� by� their� respective� sample� sizes� and� then� pooled�� This� is� conceptually� similar� to� the� standard� error�for�the�one-sample�t�test,�which�you�will�recall�from�Chapter�6�as
� s
s n
Y Y=
where�we�also�have�a�standard�deviation�weighted�by�sample�size��If�the�sample�variances� are�not�equal,�as�the�test�assumes,�then�you�can�see�why�we�might�not�want�to�take�a�pooled� or�weighted�average�(i�e�,�as�it�would�not�represent�well�the�individual�sample�variances)�
168 An Introduction to Statistical Concepts
The� test� statistic� t� is� then� compared� to� a� critical� value(s)� from� the� t� distribution�� For� a� two-tailed� test,� from� Table� A�2,� we� would� use� the� appropriate� α2� column� depending� on� the� desired� level� of� significance� and� the� appropriate� row� depending� on� the� degrees� of� freedom�� The� degrees� of� freedom� for� this� test� are� n1� +� n2� −� 2�� Conceptually,� we� lose� one� degree�of�freedom�from�each�sample�for�estimating�the�population�variances�(i�e�,�there�are� two�restrictions�along�the�lines�of�what�was�discussed�in�Chapter�6)��The�critical�values�are� denoted�as� ± + −α2 1 2 2tn n ��The�subscript�α2�of�the�critical�values�reflects�the�fact�that�this�is�a� two-tailed�test,�and�the�subscript�n1�+�n2�−�2�indicates�these�particular�degrees�of�freedom�� (Remember�that�the�critical�value�can�be�found�based�on�the�knowledge�of�the�degrees�of� freedom�and�whether�it�is�a�one-�or�two-tailed�test�)�If�the�test�statistic�falls�into�either�criti- cal�region,�then�we�reject�H0;�otherwise,�we�fail�to�reject�H0�
For�a�one-tailed�test,�from�Table�A�2,�we�would�use�the�appropriate�α1�column�depend- ing�on�the�desired�level�of�significance�and�the�appropriate�row�depending�on�the�degrees� of�freedom��The�degrees�of�freedom�are�again�n1�+�n2�−�2��The�critical�value�is�denoted�as� +α1 1 2 2tn n+ − �for�the�alternative�hypothesis�H1:�μ1�−�μ2�>�0�(i�e�,�right-tailed�test�so�the�critical� value�will�be�positive),�and�as�− + −α1 1 2 2tn n �for�the�alternative�hypothesis�H1:�μ1�−�μ2�<�0�(i�e�,� left-tailed�test�and�thus�a�negative�critical�value)��If�the�test�statistic�t�falls�into�the�appro- priate�critical�region,�then�we�reject�H0;�otherwise,�we�fail�to�reject�H0�
7.2.1.1 Confidence Interval
For�the�two-tailed�test,�a�(1�−�α)%�confidence�interval�(CI)�can�also�be�examined��The�CI�is� formed�as�follows:
� ( ) ( )Y Y t sn n Y Y1 2 22 1 21 2− ± + − −α
If�the�CI�contains�the�hypothesized�mean�difference�of�0,�then�the�conclusion�is�to�fail�to� reject�H0;�otherwise,�we�reject�H0��The�interpretation�and�use�of�CIs�is�similar�to�that�of�the� one-sample�test�described�in�Chapter�6��Imagine�we�take�100�random�samples�from�each�of� two�populations�and�construct�95%�CIs��Then�95%�of�the�CIs�will�contain�the�true�popula- tion�mean�difference�μ1�−�μ2�and�5%�will�not��In�short,�95%�of�similarly�constructed�CIs�will� contain�the�true�population�mean�difference�
7.2.1.2 Effect Size
Next�we�extend�Cohen’s�(1988)�sample�measure�of�effect�size�d�from�Chapter�6�to�the�two� independent�samples�situation��Here�we�compute�d�as�follows:
� d
Y Y sp
= −1 2
The�numerator�of�the�formula�is�the�difference�between�the�two�sample�means��The�denomi- nator� is� the� pooled� standard� deviation,� for� which� the� formula� was� presented� previously�� Thus,�the�effect�size�d�is�measured�in�standard�deviation�units,�and�again�we�use�Cohen’s� proposed�subjective�standards�for�interpreting�d:�small�effect�size,�d�=��2;�medium�effect�size,� d�=��5;�large�effect�size,�d�=��8��Conceptually,�this�is�similar�to�d�in�the�one-sample�case�from� Chapter�6��The�effect�size�d�is�considered�a�standardized�group�difference�type�of�effect�size� (Huberty,�2002)��There�are�other�types�of�effect�sizes,�however��Another�is�eta�squared�(η2),�
169Inferences About the Difference Between Two Means
also�a�standardized�effect�size,�and�it�is�considered�a�relationship�type�of�effect�size�(Huberty,� 2002)��For�the�independent�t�test,�eta�squared�can�be�calculated�as�follows:
� η2
2
2
2
2 1 2 2
= +
= + + −
t t df
t t n n( )
Here� the� numerator� is� the� squared� t� test� statistic� value,� and� the� denominator� is� sum� of� the�squared�t�test�statistic�value�and�the�degrees�of�freedom��Values�for�eta�squared�range� from�0�to�+1�00,�where�values�closer�to�one�indicate�a�stronger�association��In�terms�of�what� this�effect�size�tells�us,�eta�squared�is�interpreted�as�the�proportion�of�variance�accounted� for�in�the�dependent�variable�by�the�independent�variable�and�indicates�the�degree�of�the� relationship�between�the�independent�and�dependent�variables��If�we�use�Cohen’s�(1988)� metric�for�interpreting�eta�squared:�small�effect�size,�η2�=��01;�moderate�effect�size,�η2�=��06;� large�effect�size,�η2�=��14�
7.2.1.3 Example of the Independent t Test
Let�us�now�consider�an�example�where�the�independent�t�test�is�implemented��Recall�from� Chapter�6�the�basic�steps�for�hypothesis�testing�for�any�inferential�test:�(1)�State�the�null� and� alternative� hypotheses,� (2)� select� the� level� of� significance� (i�e�,� alpha,� α),� (3)� calculate� the�test�statistic�value,�and�(4)�make�a�statistical�decision�(reject�or�fail�to�reject�H0)��We�will� follow�these�steps�again�in�conducting�our�independent�t�test��In�our�example,�samples�of�8� female�and�12�male�middle-age�adults�are�randomly�and�independently�sampled�from�the� populations�of�female�and�male�middle-age�adults,�respectively��Each�individual�is�given� a�cholesterol�test�through�a�standard�blood�sample��The�null�hypothesis�to�be�tested�is�that� males�and�females�have�equal�cholesterol�levels��The�alternative�hypothesis�is�that�males� and�females�will�not�have�equal�cholesterol�levels,�thus�necessitating�a�nondirectional�or� two-tailed�test��We�will�conduct�our�test�using�an�alpha�level�of��05��The�raw�data�and�sum- mary�statistics�are�presented�in�Table�7�1��For�the�female�sample�(sample�1),�the�mean�and� variance�are�185�0000�and�364�2857,�respectively,�and�for�the�male�sample�(sample�2),�the� mean�and�variance�are�215�0000�and�913�6363,�respectively�
In�order�to�compute�the�test�statistic�t,�we�first�need�to�determine�the�standard�error�of� the�difference�between�the�two�means��The�pooled�standard�deviation�is�computed�as
� s
n s n s n n
p = − + −
+ − =
− + −( ) ( ) ( ) . ( ) .1 1 2
2 2 2
1 2
1 1 2
8 1 364 2857 12 1 913 6363 88 12 2
26 4575 + −
= .
and�the�standard�error�of�the�difference�between�two�means�is�computed�as
� s s
n nY Y p1 2
1 1 26 4575
1 8
1 12
12 0752 1 2
− = + = + =. .
The�test�statistic�t�can�then�be�computed�as
� t
Y Y sY Y
= −
= −
= − −
1 2
1 2
185 215 12 0752
2 4844 .
.
170 An Introduction to Statistical Concepts
The�next�step�is�to�use�Table�A�2�to�determine�the�critical�values��As�there�are�18�degrees� of�freedom�(n1�+�n2�−�2�=�8�+�12�−�2�=�18),�using�α�=��05�and�a�two-tailed�or�nondirectional� test,�we�find�the�critical�values�using�the�appropriate�α2�column�to�be�+2�101�and�−2�101�� Since�the�test�statistic�falls�beyond�the�critical�values�as�shown�in�Figure�7�1,�we�therefore� reject�the�null�hypothesis�that�the�means�are�equal�in�favor�of�the�nondirectional�alterna- tive�that�the�means�are�not�equal��Thus,�we�conclude�that�the�mean�cholesterol�levels�for� males�and�females�are�not�equal�at�the��05�level�of�significance�(denoted�by�p�<��05)�
The� 95%� CI� can� also� be� examined�� For� the� cholesterol� example,� the� CI� is� formed� as� follows:
( ) ( ) ( ) . ( . )Y Y t sn n Y Y1 2 22 1 2 1 2 185 215 2 101 12 0752 30 25− ± = − ± = − ±+ − −α .. ( . , . )3700 55 3700 4 6300= − −
Table 7.1
Cholesterol�Data�for�Independent� Samples
Female (Sample 1) Male (Sample 2)
205 245 160 170 170 180 180 190 190 200 200 210 210 220 165 230
240 250 260 185
X –
1�=�185�0000 X –
2�=�215�0000
s1 2 364 2857= . s22 913 6363= .
FIGuRe 7.1 Critical� regions� and� test� statistics� for� the� cholesterol�example�
α = .025 α = .025
+2.101 Critical value
–2.101 Critical value
–2.4884 t test
statistic value
–2.7197 Welch t΄ test
statistic value
171Inferences About the Difference Between Two Means
As�the�CI�does�not�contain�the�hypothesized�mean�difference�value�of�0,�then�we�would� again�reject�the�null�hypothesis�and�conclude�that�the�mean�gender�difference�in�choles- terol�levels�was�not�equal�to�0�at�the��05�level�of�significance�(p�<��05)��In�other�words,�there� is�evidence�to�suggest�that�the�males�and�females�differ,�on�average,�on�cholesterol�level�� More�specifically,�the�mean�cholesterol�level�for�males�is�greater�than�the�mean�cholesterol� level�for�females�
The�effect�size�for�this�example�is�computed�as�follows:
� d
Y Y sp
= −
= −
= −1 2 185 215 26 4575
1 1339 .
.
According�to�Cohen’s�recommended�subjective�standards,�this�would�certainly�be�a�rather� large�effect�size,�as�the�difference�between�the�two�sample�means�is�larger�than�one�stan- dard�deviation��Rather�than�d,�had�we�wanted�to�compute�eta�squared,�we�would�have�also� found�a�large�effect:
� η2
2
2
2
2
2 4844 2 4844 18
2553= +
= −
− + =
t t df
( . ) ( . ) ( )
.
An� eta� squared� value� of� �26� indicates� a� large� relationship� between� the� independent� and� dependent�variables,�with�26%�of�the�variance�in�the�dependent�variable�(i�e�,�cholesterol� level)�accounted�for�by�the�independent�variable�(i�e�,�gender)�
7.2.1.4 Assumptions
Let�us�return�to�the�assumptions�of�normality,�independence,�and�homogeneity�of�vari- ance�� For� the� independent� t� test,� the� assumption� of� normality� is� met� when� the� depen- dent�variable�is�normally�distributed�for�each�sample�(i�e�,�each�category�or�group)�of�the� independent�variable��The�normality�assumption�is�made�because�we�are�dealing�with�a� parametric�inferential�test��Parametric tests�assume�a�particular�underlying�theoretical� population� distribution,� in� this� case,� the� normal� distribution�� Nonparametric tests� do� not�assume�a�particular�underlying�theoretical�population�distribution�
Conventional� wisdom� tells� us� the� following� about� nonnormality�� When� the� normality� assumption�is�violated�with�the�independent�t�test,�the�effects�on�Type�I�and�Type�II�errors� are�minimal�when�using�a�two-tailed�test�(e�g�,�Glass,�Peckham,�&�Sanders,�1972;�Sawilowsky�&� Blair,�1992)��When�using�a�one-tailed�test,�violation�of�the�normality�assumption�is�minimal� for� samples� larger� than� 10� and� disappears� for� samples� of� at� least� 20� (Sawilowsky� &� Blair,� 1992;� Tiku� &� Singh,� 1981)�� The� simplest� methods� for� detecting� violation� of� the� normality� assumption� are� graphical� methods,� such� as� stem-and-leaf� plots,� box� plots,� histograms,� or� Q–Q�plots,�statistical�procedures�such�as�the�Shapiro–Wilk�(S–W)�test�(1965),�and/or�skew- ness�and�kurtosis�statistics��However,�more�recent�research�by�Wilcox�(2003)�indicates�that� power� for� both� the� independent� t� and� Welch� t′� can� be� reduced� even� for� slight� departures� from�normality,�with�outliers�also�contributing�to�the�problem��Wilcox�recommends�several� procedures�not�readily�available�and�beyond�the�scope�of�this�text�(such�as�bootstrap�meth- ods,�trimmed�means,�medians)��Keep�in�mind,�though,�that�the�independent�t�test�is�fairly� robust�to�nonnormality�in�most�situations�
The�independence�assumption�is�also�necessary�for�this�particular�test��For�the�indepen- dent� t� test,� the� assumption� of� independence� is� met� when� there� is� random� assignment� of�
172 An Introduction to Statistical Concepts
individuals�to�the�two�groups�or�categories�of�the�independent�variable��Random�assignment� to�the�two�samples�being�studied�provides�for�greater�internal�validity—the�ability�to�state� with�some�degree�of�confidence�that�the�independent�variable�caused�the�outcome�(i�e�,�the� dependent�variable)��If�the�independence�assumption�is�not�met,�then�probability�statements� about� the� Type� I� and� Type� II� errors� will� not� be� accurate;� in� other� words,� the� probability� of�a�Type�I�or�Type�II�error�may�be�increased�as�a�result�of�the�assumption�not�being�met�� Zimmerman�(1997)�found�that�Type�I�error�was�affected�even�for�relatively�small�relations� or�correlations�between�the�samples�(i�e�,�even�as�small�as��10�or��20)��In�general,�the�assump- tion�can�be�met�by�(a)�keeping�the�assignment�of�individuals�to�groups�separate�through�the� design�of�the�experiment�(specifically�random�assignment—not�to�be�confused�with�random� selection),�and�(b)�keeping�the�individuals�separate�from�one�another�through�experimen- tal�control�so�that�the�scores�on�the�dependent�variable�Y�for�sample�1�do�not�influence�the� scores�for�sample�2��Zimmerman�also�stated�that�independence�can�be�violated�for�suppos- edly� independent� samples� due� to� some� type� of� matching� in� the� design� of� the� experiment� (e�g�,�matched�pairs�based�on�gender,�age,�and�weight)��If�the�observations�are�not�indepen- dent,�then�the�dependent�t�test,�discussed�further�in�the�chapter,�may�be�appropriate�
Of�potentially�more�serious�concern�is�violation�of�the�homogeneity�of�variance�assump- tion��Homogeneity�of�variance�is�met�when�the�variances�of�the�dependent�variable�for�the� two�samples�(i�e�,�the�two�groups�or�categories�of�the�independent�variables)�are�the�same�� Research� has� shown� that� the� effect� of� heterogeneity� (i�e�,� unequal� variances)� is� minimal� when�the�sizes�of�the�two�samples,�n1�and�n2,�are�equal;�this�is�not�the�case�when�the�sample� sizes�are�not�equal��When�the�larger�variance�is�associated�with�the�smaller�sample�size� (e�g�,� group� 1� has� the� larger� variance� and� the� smaller� n),� then� the� actual� α� level� is� larger� than�the�nominal�α�level��In�other�words,�if�you�set�α�at��05,�then�you�are�not�really�conduct- ing�the�test�at�the��05�level,�but�at�some�larger�value��When�the�larger�variance�is�associated� with�the�larger�sample�size�(e�g�,�group�1�has�the�larger�variance�and�the�larger�n),�then�the� actual�α�level�is�smaller�than�the�nominal�α�level��In�other�words,�if�you�set�α�at��05,�then� you�are�not�really�conducting�the�test�at�the��05�level,�but�at�some�smaller�value�
One�can�use�statistical�tests�to�detect�violation�of�the�homogeneity�of�variance�assump- tion,� although� the� most� commonly� used� tests� are� somewhat� problematic�� These� tests� include�Hartley’s�Fmax�test�(for�equal�n’s,�but�sensitive�to�nonnormality;�it�is�the�unequal� n’s�situation�that�we�are�concerned�with�anyway),�Cochran’s�test�(for�equal�n’s,�but�even� more� sensitive� to� nonnormality� than� Hartley’s� test;� concerned� with� unequal� n’s� situa- tion�anyway),�Levene’s�test�(for�equal�n’s,�but�sensitive�to�nonnormality;�concerned�with� unequal� n’s� situation� anyway)� (available� in� SPSS),� the� Bartlett� test� (for� unequal� n’s,� but� very� sensitive� to� nonnormality),� the� Box–Scheffé–Anderson� test� (for� unequal� n’s,� fairly� robust�to�nonnormality),�and�the�Browne–Forsythe�test�(for�unequal�n’s,�more�robust�to� nonnormality�than�the�Box–Scheffé–Anderson�test�and�therefore�recommended)��When� the�variances�are�unequal�and�the�sample�sizes�are�unequal,�the�usual�method�to�use�as� an�alternative�to�the�independent�t�test�is�the�Welch�t′�test�described�in�the�next�section�� Inferential� tests� for� evaluating� homogeneity� of� variance� are� more� fully� considered� in� Chapter�9�
7.2.2 Welch t′ Test
The�Welch�t′�test�is�usually�appropriate�when�the�population�variances�are�unequal�and� the� sample� sizes� are� unequal�� The� Welch� t′� test� assumes� that� the� scores� on� the� depen- dent� variable� Y� (a)� are� normally� distributed� in� each� of� the� two� populations� and� (b)� are� independent�
173Inferences About the Difference Between Two Means
The�test�statistic�is�known�as�t′�and�is�denoted�by
�
′ = −
= − +
= −
+− t
Y Y s
Y Y
s s
Y Y
s n
s n
Y Y Y Y
1 2 1 2
2 2
1 2
1 2
1
2 2
2
1 2 1 2
where Y –
1�and�Y –
2�are�the�means�for�samples�1�and�2,�respectively sY1
2 �and� sY2 2 �are�the�variance�errors�of�the�means�for�samples�1�and�2,�respectively
Here�we�see�that�the�denominator�of�this�test�statistic�is�conceptually�similar�to�the�one- sample� t� and� the� independent� t� test� statistics�� The� variance� errors� of� the� mean� are� com- puted�for�each�group�by
� s
s nY1
2 1 2
1 =
� s
s nY2
2 2 2
2 =
where� s1 2 �and� s2
2 �are�the�sample�variances�for�groups�1�and�2,�respectively��The�square�root� of�the�variance�error�of�the�mean�is�the�standard�error�of�the�mean�(i�e�,� sY1 �and� sY2 )��Thus,� we�see�that�rather�than�take�a�pooled�or�weighted�average�of�the�two�sample�variances�as� we�did�with�the�independent�t�test,�the�two�sample�variances�are�treated�separately�with� the�Welch�t′�test�
The�test�statistic�t′�is�then�compared�to�a�critical�value(s)�from�the�t�distribution�in�Table� A�2��We�again�use�the�appropriate�α�column�depending�on�the�desired�level�of�significance� and�whether�the�test�is�one-�or�two-tailed�(i�e�,�α1�and�α2),�and�the�appropriate�row�for�the� degrees�of�freedom��The�degrees�of�freedom�for�this�test�are�a�bit�more�complicated�than� for� the� independent� t� test�� The� degrees� of� freedom� are� adjusted� from� n1� +� n2� −� 2� for� the� independent�t�test�to�the�following�value�for�the�Welch�t′�test:
�
ν = +( )
( ) −
+ ( )
−
s s
s
n
s
n
Y Y
Y Y
1 2
1 2
2 2 2
2 2
1
2 2
21 1
The� degrees� of� freedom� ν� are� approximated� by� rounding� to� the� nearest� whole� number� prior� to� using� the� table�� If� the� test� statistic� falls� into� a� critical� region,� then� we� reject� H0;� otherwise,�we�fail�to�reject�H0�
For�the�two-tailed�test,�a�(1�−�α)%�CI�can�also�be�examined��The�CI�is�formed�as�follows:
� ( ) ( )Y Y t sY Y1 2 2 1 2− ± −α ν
If�the�CI�contains�the�hypothesized�mean�difference�of�0,�then�the�conclusion�is�to�fail�to� reject�H0;�otherwise,�we�reject�H0��Thus,�interpretation�of�this�CI�is�the�same�as�with�the� independent�t�test�
174 An Introduction to Statistical Concepts
Consider� again� the� example� cholesterol� data� where� the� sample� variances� were� some- what� different� and� the� sample� sizes� were� different�� The� variance� errors� of� the� mean� are� computed�for�each�sample�as�follows:
� s
s nY1
2 1 2
1
364 2857 8
45 5357= = = .
.
� s
s nY2
2 2 2
2
913 6363 12
76 1364= = = .
.
The�t′�test�statistic�is�computed�as
�
′ = − +
= − +
= −
= −t Y Y
s sY Y
1 2
2 2 1 2
185 215 45 5357 76 1364
30 11 0305
2 719 . . .
. 77
Finally,�the�degrees�of�freedom�ν�are�determined�to�be
�
ν = +( )
( ) −
+ ( )
−
= +s s
s
n
s
n
Y Y
Y Y
1 2
1 2
2 2 2
2 2
1
2 2
2
2
1 1
45 5357 76 1364 4 ( . . )
( 55 5357 8 1
76 1364 12 1
17 98382 2. ) ( . ) .
− +
−
=
which� is� rounded� to� 18,� the� nearest� whole� number�� The� degrees� of� freedom� remain� 18� as� they�were�for�the�independent�t�test,�and�thus,�the�critical�values�are�still�+2�101�and�−2�101�� As�the�test�statistic�falls�beyond�the�critical�values�as�shown�in�Figure�7�1,�we�therefore�reject� the�null�hypothesis�that�the�means�are�equal�in�favor�of�the�alternative�that�the�means�are� not�equal��Thus,�as�with�the�independent�t�test,�with�the�Welch�t′�test,�we�conclude�that�the� mean�cholesterol�levels�for�males�and�females�are�not�equal�at�the��05�level�of�significance��In� this�particular�example,�then,�we�see�that�the�unequal�sample�variances�and�unequal�sample� sizes�did�not�alter�the�outcome�when�comparing�the�independent�t�test�result�with�the�Welch� t′�test�result��However,�note�that�the�results�for�these�two�tests�may�differ�with�other�data�
Finally,�the�95%�CI�can�be�examined��For�the�example,�the�CI�is�formed�as�follows:
� ( ) ( ) ( ) . ( . ) .Y Y t sY Y1 2 2 1 2 185 215 2 101 11 0305 30 23 1751− ± = − ± = − ± =−α ν (( . , . )− −53 1751 6 8249
As�the�CI�does�not�contain�the�hypothesized�mean�difference�value�of�0,�then�we�would� again� reject� the� null� hypothesis� and� conclude� that� the� mean� gender� difference� was� not� equal�to�0�at�the��05�level�of�significance�(p�<��05)�
7.2.3 Recommendations
The�following�four�recommendations�are�made�regarding�the�two�independent�samples� case�� Although� there� is� no� total� consensus� in� the� field,� our� recommendations� take� into� account,� as� much� as� possible,� the� available� research� and� statistical� software�� First,� if� the� normality� assumption� is� satisfied,� the� following� recommendations� are� made:� (a)� the�
175Inferences About the Difference Between Two Means
independent�t�test�is�recommended�when�the�homogeneity�of�variance�assumption�is�met;� (b)�the�independent�t�test�is�recommended�when�the�homogeneity�of�variance�assumption� is�not�met�and�when�there�are�an�equal�number�of�observations�in�the�samples;�and�(c)�the� Welch�t′�test�is�recommended�when�the�homogeneity�of�variance�assumption�is�not�met� and�when�there�are�an�unequal�number�of�observations�in�the�samples�
Second,�if�the�normality�assumption�is�not�satisfied,�the�following�recommendations�are� made:� (a)� if� the� homogeneity� of� variance� assumption� is� met,� then� the� independent� t� test� using�ranked�scores�(Conover�&�Iman,�1981),�rather�than�raw�scores,�is�recommended;�and� (b)�if�homogeneity�of�variance�assumption�is�not�met,�then�the�Welch�t′�test�using�ranked� scores�is�recommended,�regardless�of�whether�there�are�an�equal�number�of�observations� in�the�samples��Using�ranked�scores�means�you�rank�order�the�observations�from�highest� to�lowest�regardless�of�group�membership,�then�conduct�the�appropriate�t�test�with�ranked� scores�rather�than�raw�scores�
Third,�the�dependent�t�test�is�recommended�when�there�is�some�dependence�between� the� groups� (e�g�,� matched� pairs� or� the� same� individuals� measured� on� two� occasions),� as� described� later� in� this� chapter�� Fourth,� the� nonparametric� Mann-Whitney-Wilcoxon� test� is�not�recommended��Among�the�disadvantages�of�this�test�are�that�(a)�the�critical�values� are�not�extensively�tabled,�(b)�tied�ranks�can�affect�the�results�and�no�optimal�procedure� has�yet�been�developed�(Wilcox,�1996),�and�(c)�Type�I�error�appears�to�be�inflated�regard- less� of� the� status� of� the� assumptions� (Zimmerman,� 2003)�� For� these� reasons,� the� Mann– Whitney–Wilcoxon� test� is� not� further� described� here�� Note� that� most� major� statistical� packages,�including�SPSS,�have�options�for�conducting�the�independent�t�test,�the�Welch�t′� test,�and�the�Mann-Whitney-Wilcoxon�test��Alternatively,�one�could�conduct�the�Kruskal– Wallis�nonparametric�one-factor�ANOVA,�which�is�also�based�on�ranked�data,�and�which� is�appropriate�for�comparing�the�means�of�two�or�more�independent�groups��This�test�is� considered�more�fully�in�Chapter�11��These�recommendations�are�summarized�in�Box�7�1�
STOp aNd ThINk bOx 7.1
Recommendations�for�the�Independent�and�Dependent�Samples�Tests�Based�on�Meeting� or Violating�the�Assumption�of�Normality
Assumption Independent Samples Tests Dependent Samples Tests
Normality�is�met •��Use�the�independent�t�test�when� homogeneity�of�variances�is�met
•�Use�the�dependent�t�test
•��Use�the�independent�t�test�when� homogeneity�of�variances�is�not�met,�but� there�are�equal�sample�sizes�in�the�groups
•��Use�the�Welch�t′�test�when�homogeneity�of� variances�is�not�met�and�there�are�unequal� sample�sizes�in�the�groups
Normality�is�not�met •��Use�the�independent�t�test�with�ranked� scores�when�homogeneity�of�variances�is� met
•��Use�the�Welch�t′�test�with�ranked�scores� when�homogeneity�of�variances�is�not�met,� regardless�of�equal�or�unequal�sample� sizes�in�the�groups
•��Use�the�Kruskal–Wallis�nonparametric� procedure
•��Use�the�dependent�t�test�with�ranked� scores,�or�alternative�procedures� including�bootstrap�methods,� trimmed�means,�medians,�or�Stein’s� method
•��Use�the�Wilcoxon�signed�ranks�test� when�data�are�both�nonnormal�and� have�extreme�outliers
•��Use�the�Friedman�nonparametric� procedure
176 An Introduction to Statistical Concepts
7.3 Inferences About Two Dependent Means
In�this�section,�two�inferential�tests�of�the�difference�between�two�dependent�means�are� described,� the� dependent� t� test� and� briefly� the� Wilcoxon� signed� ranks� test�� The� section� concludes�with�a�list�of�recommendations�
7.3.1 dependent t Test
As�you�may�recall,�the�dependent�t�test�is�appropriate�to�use�when�there�are�two�samples� that�are�dependent—the�individuals�in�sample�1�have�some�relationship�to�the�individuals� in�sample�2��First,�we�need�to�determine�the�conditions�under�which�the�dependent�t�test�is� appropriate��In�part,�this�has�to�do�with�the�statistical�assumptions�associated�with�the�test� itself—that�is,�(a)�normality�of�the�distribution�of�the�differences�of�the�dependent�variable� Y,�(b)�homogeneity�of�variance�of�the�two�populations,�and�(c)�independence�of�the�obser- vations�within�each�sample��Like�the�independent�t�test,�the�dependent�t�test�is�reasonably� robust�to�violation�of�the�normality�assumption,�as�we�show�later��Because�this�is�a�test�of� means,�the�dependent�variable�must�be�measured�on�an�interval�or�ratio�scale��For�example,� the� same� individuals� may� be� measured� at� two� points� in� time� on� the� same� interval-scaled� pretest�and�posttest,�or�some�matched�pairs�(e�g�,�twins�or�husbands–wives)�may�be�assessed� with�the�same�ratio-scaled�measure�(e�g�,�weight�measured�in�pounds)�
Although�there�are�several�methods�for�computing�the�test�statistic�t,�the�most�direct�method� and�the�one�most�closely�aligned�conceptually�with�the�one-sample�t�test�is�as�follows:
� t
d sd
=
where d – �is�the�mean�difference
sd–�is�the�standard�error�of�the�mean�difference
Conceptually,� this� test� statistic� looks� just� like� the� one-sample� t� test� statistic,� except� now� that�the�notation�has�been�changed�to�denote�that�we�are�dealing�with�difference�scores� rather�than�raw�scores�
The�standard�error�of�the�mean�difference�is�computed�by
� s
s n
d d=
where sd�is�the�standard�deviation�of�the�difference�scores�(i�e�,�like�any�other�standard�devia-
tion,�only�this�one�is�computed�from�the�difference�scores�rather�than�raw�scores) n�is�the�total�number�of�pairs
Conceptually,�this�standard�error�looks�just�like�the�standard�error�for�the�one-sample�t�test�� If�we�were�doing�hand�computations,�we�would�compute�a�difference�score�for�each�pair�of� scores�(i�e�,�Y1�−�Y2)��For�example,�if�sample�1�were�wives�and�sample�2�were�their�husbands,� then�we�calculate�a�difference�score�for�each�couple��From�this�set�of�difference�scores,�we� then�compute�the�mean�of�the�difference�scores�d
– �and�standard�deviation�of�the�difference�
177Inferences About the Difference Between Two Means
scores�sd��This�leads�us�directly�into�the�computation�of�the�t�test�statistic��Note�that�although� there�are�n�scores�in�sample�1,�n�scores�in�sample�2,�and�thus�2n�total�scores,�there�are�only�n� difference�scores,�which�is�what�the�analysis�is�actually�based�upon�
The�test�statistic�t�is�then�compared�with�a�critical�value(s)�from�the�t�distribution��For�a� two-tailed�test,�from�Table�A�2,�we�would�use�the�appropriate�α2�column�depending�on�the� desired� level� of� significance� and� the� appropriate� row� depending� on� the� degrees� of� free- dom��The�degrees�of�freedom�for�this�test�are�n�−�1��Conceptually,�we�lose�one�degree�of� freedom�from�the�number�of�differences�(or�pairs)�because�we�are�estimating�the�popula- tion�variance�(or�standard�deviation)�of�the�difference��Thus,�there�is�one�restriction�along� the� lines� of� our� discussion� of� degrees� of� freedom� in� Chapter� 6�� The� critical� values� are� denoted�as�± −α2 1tn ��The�subscript,�α2,�of�the�critical�values�reflects�the�fact�that�this�is�a�two- tailed�test,�and�the�subscript�n�−�1�indicates�the�degrees�of�freedom��If�the�test�statistic�falls� into�either�critical�region,�then�we�reject�H0;�otherwise,�we�fail�to�reject�H0�
For�a�one-tailed�test,�from�Table�A�2,�we�would�use�the�appropriate�α1�column�depending� on�the�desired�level�of�significance�and�the�appropriate�row�depending�on�the�degrees�of� freedom��The�degrees�of�freedom�are�again�n�−�1��The�critical�value�is�denoted�as�+ −α1 1tn �for� the�alternative�hypothesis�H1:�μ1�−�μ2�>�0�and�as� − −α1 1tn �for�the�alternative�hypothesis�H1:� μ1�−�μ2�<�0��If�the�test�statistic�t�falls�into�the�appropriate�critical�region,�then�we�reject�H0;� otherwise,�we�fail�to�reject�H0�
7.3.1.1 Confidence Interval
For�the�two-tailed�test,�a�(1�−�α)%�CI�can�also�be�examined��The�CI�is�formed�as�follows:
� d t sn d± −α 2 1( )
If�the�CI�contains�the�hypothesized�mean�difference�of�0,�then�the�conclusion�is�to�fail�to� reject�H0;�otherwise,�we�reject�H0��The�interpretation�of�these�CIs�is�the�same�as�those�previ- ously�discussed�for�the�one-sample�t�and�the�independent�t�
7.3.1.2 Effect Size
The�effect�size�can�be�measured�using�Cohen’s�(1988)�d�computed�as�follows:
� Cohen d
d sd
=
where�Cohen’s�d�is�simply�used�to�distinguish�among�the�various�uses�and�slight�differ- ences� in� the� computation� of� d�� Interpretation� of� the� value� of� d� would� be� the� same� as� for� the�one-sample�t�and�the�independent�t�previously�discussed—specifically,�the�number�of� standard�deviation�units�for�which�the�mean(s)�differ(s)�
7.3.1.3 Example of the Dependent t Test
Let�us�consider�an�example�for�purposes�of�illustrating�the�dependent�t�test��Ten�young� swimmers�participated�in�an�intensive�2�month�training�program��Prior�to�the�program,� each�swimmer�was�timed�during�a�50�meter�freestyle�event��Following�the�program,�the�
178 An Introduction to Statistical Concepts
same�swimmers�were�timed�in�the�50�meter�freestyle�event�again��This�is�a�classic�pretest- posttest�design��For�illustrative�purposes,�we�will�conduct�a�two-tailed�test��However,�a� case� might� also� be� made� for� a� one-tailed� test� as� well,� in� that� the� coach� might� want� to� see�improvement�only��However,�conducting�a�two-tailed�test�allows�us�to�examine�the� CI�for�purposes�of�illustration��The�raw�scores,�the�difference�scores,�and�the�mean�and� standard�deviation�of�the�difference�scores�are�shown�in�Table�7�2��The�pretest�mean�time� was�64�seconds�and�the�posttest�mean�time�was�59�seconds�
To�determine�our�test�statistic�value,�t,�first�we�compute�the�standard�error�of�the�mean� difference�as�follows:
� s
s n
d d= = =
2 1602 10
0 6831 .
.
Next,�using�this�value�for�the�denominator,�the�test�statistic�t�is�then�computed�as�follows:
� t
d sd
= = = 5
0 6831 7 3196
. .
We�then�use�Table�A�2�to�determine�the�critical�values��As�there�are�nine�degrees�of�free- dom�(n�−�1�=�10�−�1�=�9),�using�α�=��05�and�a�two-tailed�or�nondirectional�test,�we�find�the� critical�values�using�the�appropriate�α2�column�to�be�+2�262�and�−2�262��Since�the�test�sta- tistic�falls�beyond�the�critical�values,�as�shown�in�Figure�7�2,�we�reject�the�null�hypothesis� that�the�means�are�equal�in�favor�of�the�nondirectional�alternative�that�the�means�are�not� equal��Thus,�we�conclude�that�the�mean�swimming�performance�changed�from�pretest�to� posttest�at�the��05�level�of�significance�(p�<��05)�
The�95%�CI�is�computed�to�be�the�following:
� d t sn d± = ± = ± =−α 2 1 5 2 262 0 6831 5 1 5452 3 4548 6 5452( ) . ( . ) . ( . , . )
Table 7.2
Swimming�Data�for�Dependent�Samples
Swimmer Pretest Time (in Seconds)
Posttest Time (in Seconds) Difference (d)
1 58 54 (i�e�,�58�−�54)�=�4 2 62 57 5 3 60 54 6 4 61 56 5 5 63 61 2 6 65 59 6 7 66 64 2 8 69 62 7 9 64 60 4
10 72 63 9 d – �=�5�0000
sd�=�2�1602
179Inferences About the Difference Between Two Means
As�the�CI�does�not�contain�the�hypothesized�mean�difference�value�of�0,�we�would�again� reject�the�null�hypothesis�and�conclude�that�the�mean�pretest-posttest�difference�was�not� equal�to�0�at�the��05�level�of�significance�(p�<��05)�
The�effect�size�is�computed�to�be�the�following:
� Cohen d
d sd
= = = 5
2 1602 2 3146
. .
which� is� interpreted� as� there� is� approximately� a� two� and� one-third� standard� deviation� difference�between�the�pretest�and�posttest�mean�swimming�times,�a�very�large�effect�size� according�to�Cohen’s�subjective�standard�
7.3.1.4 Assumptions
Let� us� return� to� the� assumptions� of� normality,� independence,� and� homogeneity� of� vari- ance�� For� the� dependent� t� test,� the� assumption� of� normality� is� met� when� the� difference� scores� are� normally� distributed�� Normality� of� the� difference� scores� can� be� examined� as� discussed� previously—graphical� methods� (such� as� stem-and-leaf� plots,� box� plots,� histo- grams,�and/or�Q–Q�plots),�statistical�procedures�such�as�the�S–W�test�(1965),�and/or�skew- ness�and�kurtosis�statistics��The�assumption�of�independence�is�met�when�the�cases�in�our� sample�have�been�randomly�selected�from�the�population��If�the�independence�assump- tion�is�not�met,�then�probability�statements�about�the�Type�I�and�Type�II�errors�will�not�be� accurate;�in�other�words,�the�probability�of�a�Type�I�or�Type�II�error�may�be�increased�as�a� result�of�the�assumption�not�being�met��Homogeneity�of�variance�refers�to�equal�variances� of�the�two�populations��In�later�chapters,�we�will�examine�procedures�for�formally�testing� for�equal�variances��For�the�moment,�if�the�ratio�of�the�smallest�to�largest�sample�variance� is� within� 1:4,� then� we� have� evidence� to� suggest� the� assumption� of� homogeneity� of� vari- ances�is�met��Research�has�shown�that�the�effect�of�heterogeneity�(i�e�,�unequal�variances)� is�minimal�when�the�sizes�of�the�two�samples,�n1�and�n2,�are�equal,�as�is�the�case�with�the� dependent�t�test�by�definition�(unless�there�are�missing�data)�
α = .025
–2.262 Critical value
+2.262 Critical value
+7.3196 t test
statistic value
α = .025
FIGuRe 7.2 Critical� regions� and� test� statistic� for� the� swimming�example�
180 An Introduction to Statistical Concepts
7.3.2 Recommendations
The� following� three� recommendations� are� made� regarding� the� two� dependent� samples� case��First,�the�dependent�t�test�is�recommended�when�the�normality�assumption�is�met�� Second,�the�dependent�t�test�using�ranks�(Conover�&�Iman,�1981)�is�recommended�when� the�normality�assumption�is�not�met��Here�you�rank�order�the�difference�scores�from�high- est�to�lowest,�then�conduct�the�test�on�the�ranked�difference�scores�rather�than�on�the�raw� difference�scores��However,�more�recent�research�by�Wilcox�(2003)�indicates�that�power�for� the�dependent�t�can�be�reduced�even�for�slight�departures�from�normality��Wilcox�recom- mends� several� procedures� not� readily� available� and� beyond� the� scope� of� this� text� (boot- strap�methods,�trimmed�means,�medians,�Stein’s�method)��Keep�in�mind,�though,�that�the� dependent�t�test�is�fairly�robust�to�nonnormality�in�most�situations�
Third,�the�nonparametric�Wilcoxon�signed�ranks�test�is�recommended�when�the�data� are�nonnormal�with�extreme�outliers�(one�or�a�few�observations�that�behave�quite�differ- ently�from�the�rest)��However,�among�the�disadvantages�of�this�test�are�that�(a)�the�criti- cal�values�are�not�extensively�tabled�and�two�different�tables�exist�depending�on�sample� size,� and� (b)� tied� ranks� can� affect� the� results� and� no� optimal� procedure� has� yet� been� developed� (Wilcox,� 1996)�� For� these� reasons,� the� details� of� the� Wilcoxon� signed� ranks� test� are� not� described� here�� Note� that� most� major� statistical� packages,� including� SPSS,� include�options�for�conducting�the�dependent�t�test�and�the�Wilcoxon�signed�ranks�test�� Alternatively,�one�could�conduct�the�Friedman�nonparametric�one-factor�ANOVA,�also� based�on�ranked�data,�and�which�is�appropriate�for�comparing�two�or�more�dependent� sample�means��This�test�is�considered�more�fully�in�Chapter�15��These�recommendations� are�summarized�in�Box�7�1�
7.4 SPSS
Instructions�for�determining�the�independent�samples�t�test�using�SPSS�are�presented�first�� This� is� followed� by� additional� steps� for� examining� the� assumption� of� normality� for� the� independent�t�test��Next,�instructions�for�determining�the�dependent�samples�t�test�using� SPSS�are�presented�and�are�then�followed�by�additional�steps�for�examining�the�assump- tions�of�normality�and�homogeneity�
Independent t Test
Step 1:� In� order� to� conduct� an� independent� t� test,� your� dataset� needs� to� include� a� dependent�variable�Y�that�is�measured�on�an�interval�or�ratio�scale�(e�g�,�cholesterol),�as� well�as�a�grouping�variable�X�that�is�measured�on�a�nominal�or�ordinal�scale�(e�g�,�gen- der)��For�the�grouping�variable,�if�there�are�more�than�two�categories�available,�only�two� categories�can�be�selected�when�running�the�independent�t�test�(the�ANOVA�is�required� for� examining� more� than� two� categories)�� To� conduct� the� independent� t� test,� go� to� the� “Analyze”�in�the�top�pulldown�menu,�then�select�“Compare Means,”�and�then�select� “Independent-Samples T Test.”�Following�the�screenshot�(step�1)�as�follows�pro- duces�the�“Independent-Samples T Test”�dialog�box�
181Inferences About the Difference Between Two Means
A
B C
Independent t test: Step 1
Step 2:�Next,�from�the�main�“Independent-Samples T Test”�dialog�box,�click�the� dependent�variable�(e�g�,�cholesterol)�and�move�it�into�the�“Test Variable”�box�by�click- ing�on�the�arrow�button��Next,�click�the�grouping�variable�(e�g�,�gender)�and�move�it�into� the�“Grouping Variable”� box� by� clicking� on� the� arrow� button�� You� will� notice� that� there�are�two�question�marks�next�to�the�name�of�your�grouping�variable��This�is�SPSS�let- ting�you�know�that�you�need�to�define�(numerically)�which�two�categories�of�the�grouping� variable�you�want�to�include�in�your�analysis��To�do�that,�click�on�“Define Groups.”
Clicking on “Options” will
allow you to define a confidence interval
percentage. e default is 95% (corresponding to an alpha of .05).
Select the variable of interest from
the list on the left and use the arrow
to move to the “Test Variable”
box on the right.
Clicking on “Define Groups”
will allow you to define the two
numeric values of the categories for the independent
variable.
Independent t test: Step 2
182 An Introduction to Statistical Concepts
Step 3:�From�the�“Define Groups”�dialog�box,�enter�the�numeric�value�designated�for� each�of�the�two�categories�or�groups�of�your�independent�variable��Where�it�says�“Group 1,”�type�in�the�value�designated�for�your�first�group�(e�g�,�1,�which�in�our�case�indicated� that�the�individual�was�a�female),�and�where�it�says�“Group 2,”�type�in�the�value�desig- nated�for�your�second�group�(e�g�,�2,�in�our�example,�a�male)�(see�step�3�screenshot)�
Independent t test: Step 3
Click�on�“Continue”�to�return�to�the�original�dialog�box�(see�step�2�screenshot)�and�then� click�on�“OK”�to�run�the�analysis��The�output�is�shown�in�Table�7�3�
Changing the alpha level (optional):�The�default�alpha�level�in�SPSS�is��05,�and� thus,�the�default�corresponding�CI�is�95%��If�you�wish�to�test�your�hypothesis�at�an�alpha� level�other�than��05�(and�thus�obtain�CIs�other�than�95%),�click�on�the�“Options”�button� located�in�the�top�right�corner�of�the�main�dialog�box�(see�step�2�screenshot)��From�here,� the�CI�percentage�can�be�adjusted�to�correspond�to�the�alpha�level�at�which�you�wish�your� hypothesis�to�be�tested�(see�Chapter�6�screenshot�step�3)��(For�purposes�of�this�example,�the� test�has�been�generated�using�an�alpha�level�of��05�)
Interpreting the output:�The�top�table�provides�various�descriptive�statistics�for� each� group,� while� the� bottom� box� gives� the� results� of� the� requested� procedure�� There� you� see� the� following� three� different� inferential� tests� that� are� automatically� provided:� (1)� Levene’s� test� of� the� homogeneity� of� variance� assumption� (the� first� two� columns� of� results),�(2)�the�independent�t�test�(which�SPSS�calls�“Equal variances assumed”)� (the�top�row�of�the�remaining�columns�of�results),�and�(3)�the�Welch�t′�test�(which�SPSS� calls�“Equal variances not assumed”)�(the�bottom�row�of�the�remaining�columns� of�results)�
The� first� interpretation� that� must� be� made� is� for� Levene’s� test� of� equal� variances�� The� assumption� of� equal� variances� is� met� when� Levene’s� test� is� not� statistically� significant�� We� can� determine� statistical� significance� by� reviewing� the� p� value� for� the� F� test�� In� this� example,�the�p�value�is��090,�greater�than�our�alpha�level�of��05�and�thus�not�statistically� significant��Levene’s�test�tells�us�that�the�variance�for�cholesterol�level�for�males�is�not�sta- tistically�significantly�different�than�the�variance�for�cholesterol�level�for�females��Having� met� the� assumption� of� equal� variances,� the� values� in� the� rest� of� the� table� will� be� drawn� from�the�row�labeled�“Equal Variances Assumed.”�Had�we�not�met�the�assumption� of�equal�variances�(p�<�α),�we�would�report�Welch�t′�for�which�the�statistics�are�presented� on�the�row�labeled�“Equal Variances Not Assumed.”
After� determining� that� the� variances� are� equal,� the� next� step� is� to� examine� the� results�of�the�independent�t�test��The�t�test�statistic�value�is�−2�4842,�and�the�associated� p�value�is��023��Since�p�is�less�than�α,�we�reject�the�null�hypothesis��There�is�evidence�to� suggest�that�the�mean�cholesterol�level�for�males�is�different�than�the�mean�cholesterol� level�for�females�
183Inferences About the Difference Between Two Means
Table 7.3
SPSS�Results�for�Independent�t�Test
Group Statistics
Gender N Mean Std. Deviation Std. Error Mean
Female 8 185.0000 19.08627 6.74802Cholesterol level
Male 12 215.0000 30.22642 8.72562
Independent Samples Test
Levene's Test for Equality of Variances t-Test for Equality of Means
95% Confidence Interval of the Difference
F Sig. t df Sig.
(2-Tailed) Mean
Difference Std. Error Difference Lower Upper
3.201 .090 –2.484 .023 –30.00000 12.07615 –55.37104
–2.720
18
17.984 .014 –30.00000 11.03051 –53.17573
–4.62896
– 6.82427
“t” is the t test statistic value. �e t value in the top row is used when the assumption of equal variances has been met and is calculated as:
The t value in the bottom row is the Welch t΄and is used when the assumption of equal variances has not been met.
�e table labeled “Group Statistics” provides basic descriptive statistics for the
dependent variable by group.
SPSS reports the95% confidence interval of the difference. This is
interpreted to mean that 95% of the CIs generated
across samples will contain the true population mean
difference of 0.
�e F test (and p value) of Levene’s Test for Equality of Variances is reviewed to determine
if the equal variances assumption has been met. �e result of this test determines which
row of statistics to utilize. In this case, we meet the assumption and use the statistics
reported in the top row.
“Sig.” is the observed p value for the independent t test.
It is interpreted as: there is less than a 3% probability of a sample mean difference of –30 or greater
occurring by chance if the null hypothesis is really true (i.e., if
the population mean difference is 0).
�e mean difference is simply the difference between the sample mean cholesterol values. In other words, 185 – 215 = – 30
The standard error of the mean difference is calculated as:
=–sY1 sp n1 n2
11 +
df are the degrees of freedom. For the independent samples t test, they are calculated as
Equal variances assumed
Cholesterol level
Equal variances
not assumed
Y2
n1 + n2 – 2.
–2.484 12.075
Y1 – Y2t sY1 – Y2
185 – 215 ===
184 An Introduction to Statistical Concepts
Using “Explore” to Examine Normality of Distribution of Dependent Variable by Categories of Independent Variable
Generating normality evidence: As�alluded�to�earlier�in�the�chapter,�understanding� the�distributional�shape,�specifically�the�extent�to�which�normality�is�a�reasonable�assump- tion,�is�important��For�the�independent�t�test,�the�distributional�shape�for�the�dependent�vari- able�should�be�normally�distributed�for�each�category/group�of�the�independent�variable��As� with�our�one-sample�t�test,�we�can�again�use�“Explore”�to�examine�the�extent�to�which�the� assumption�of�normality�is�met�
The� general� steps� for� accessing�“Explore”� have� been� presented� in� previous� chapters� (e�g�,�Chapter�4),�and�they�will�not�be�reiterated�here��Normality�of�the�dependent�variable� must�be�examined�for�each�category�of�the�independent�variable,�so�we�must�tell�SPSS�to� split�the�examination�of�normality�by�group��Click�the�dependent�variable�(e�g�,�cholesterol)� and�move�it�into�the�“Dependent List”�box�by�clicking�on�the�arrow�button��Next,�click� the�grouping�variable�(e�g�,�gender)�and�move�it�into�the�“Factor List”�box�by�clicking� on�the�arrow�button��The�procedures�for�selecting�normality�statistics�were�presented�in� Chapter� 6,� and� they� remain� the� same� here:� click� on�“Plots”� in� the� upper� right� corner�� Place� a� checkmark� in� the� boxes� for� “Normality plots with tests”� and� also� for� “Histogram.”�Then�click�“Continue”�to�return�to�the�main�“Explore”�dialog�screen�� From�there,�click�“OK”�to�generate�the�output�
Select the dependent variable from the list
on the left and use the arrow to move to the “Dependent List” box on the right and the
independent variable from the list
on the left and use the arrow to move to the “Factor List”
box on the right. �en click on “Plots.”
Generating normality
evidence by group
Interpreting normality evidence:�We�have�already�developed�a�good�under- standing�of�how�to�interpret�some�forms�of�evidence�of�normality�including�skewness�
185Inferences About the Difference Between Two Means
and� kurtosis,� histograms,� and� boxplots�� As� we� examine� the� “Descriptives”� table,� we� see� the� output� for� the� cholesterol� statistics� is� separated� for� male� (top� portion)� and� female�(bottom�portion)�
Mean 95% Con�dence interval for mean
5% Trimmed mean Median Variance Std. deviation Minimum Maximum Range Interquartile range Skewness Kurtosis Mean 95% Con�dence interval for mean
5% Trimmed mean Median Variance Std. deviation Minimum
Maximum Range Interquartile range Skewness Kurtosis
Female Lower bound Upper bound
Lower bound Upper bound
Cholesterol level Male
Gender Descriptives
Statistic Std. Error 215.0000 195.7951 234.2049
215.0000 215.0000
913.636
170.00 260.00
90.00 57.50
.000 –1.446
185.0000 169.0435 200.9565 185.0000 185.0000
364.286 19.08627
160.00 210.00
50.00 37.50
.000 –1.790
1.232 6.74802
.637
30.22642
8.72562
.752 1.481
The� skewness� statistic� of� cholesterol� level� for� the� males� is� �000� and� kurtosis� is� −1�446—both� within� the� range� of� an� absolute� value� of� 2�0,� suggesting� some� evidence� of� normality� of� the� dependent� variable� for� males�� Evidence� of� normality� for� the� dis- tributional�shape�of�cholesterol�level�for�females�is�also�present:�skewness�=��000�and� kurtosis�=�−1�790�
The�histogram�of�cholesterol�level�for�males�is�not�exactly�what�most�researchers�would� consider� a� classic� normally� shaped� distribution�� Although� the� histogram� of� cholesterol� level�for�females�is�not�presented�here,�it�follows�a�similar�distributional�shape�
186 An Introduction to Statistical Concepts
2.0
1.5
1.0
Fr eq
ue nc
y
0.5
0.0 160.00 180.00 200.00 220.00
Cholesterol level 240.00 260.00
Histogram for group = Male
Mean = 215.00 Std. dev. = 30.226 N = 12
There�are�a�few�other�statistics�that�can�be�used�to�gauge�normality�as�well��Our�formal� test�of�normality,�the�Shapiro–Wilk�test�(SW)�(Shapiro�&�Wilk,�1965),�provides�evidence�of� the�extent�to�which�our�sample�distribution�is�statistically�different�from�a�normal�distri- bution�� The� output� for� the� S–W� test� is� presented� in� the� following� and� suggests� that� our� sample�distribution�for�cholesterol�level�is�not�statistically�significantly�different�than�what� would�be�expected�from�a�normal�distribution—and�this�is�true�for�both�males�(SW�=��949,� df�=�12,�p�=��617)�and�females�(SW�=��931,�df�=�8,�p�=��525)�
Gender
Male Female
Cholesterol level Statistic Statisticdf
Tests of Normality
Kolmogorov–Smirnova
df Shapiro–Wilk
Sig. Sig. .129 .159 8
12
8
12 .617 .525
.200
.200 .931 .949
a Lilliefors significance correction * This is a lower bound of the true significance.
Quantile–quantile� (Q–Q)� plots� are� also� often� examined� to� determine� evidence� of� nor- mality�� Q–Q� plots� are� graphs� that� plot� quantiles� of� the� theoretical� normal� distribution� against�quantiles�of�the�sample�distribution��Points�that�fall�on�or�close�to�the�diagonal�line� suggest�evidence�of�normality��Similar�to�what�we�saw�with�the�histogram,�the�Q–Q�plot� of�cholesterol�level�for�both�males�and�females�(although�not�shown�here)�suggests�some� nonnormality��Keep�in�mind�that�we�have�a�relatively�small�sample�size��Thus,�interpreting� the�visual�graphs�(e�g�,�histograms�and�Q–Q�plots)�can�be�challenging,�although�we�have� plenty�of�other�evidence�for�normality�
187Inferences About the Difference Between Two Means
2
1
0
–1
–2
Ex pe
ct ed
n or
m al
175 200 225 Observed value
250 275
Normal Q–Q plot of cholesterol level
For group = male
Examination�of�the�boxplots�suggests�a�relatively�normal�distributional�shape�of�choles- terol�level�for�both�males�and�females�and�no�outliers�
260.00
240.00
220.00
200.00
C ho
le st
er ol
le ve
l
180.00
160.00
Male Female Gender
Considering�the�forms�of�evidence�we�have�examined,�skewness�and�kurtosis�statistics,� the�S–W�test,�and�the�boxplots,�all�suggest�normality�is�a�reasonable�assumption��Although� the� histograms� and� Q–Q� plots� suggest� some� nonnormality,� this� is� somewhat� expected� given� the� small� sample� size�� Generally,� we� can� be� reasonably� assured� we� have� met� the� assumption� of� normality� of� the� dependent� variable� for� each� group� of� the� independent� variable��Additionally,�recall�that�when�the�assumption�of�normality�is�violated�with�the� independent�t�test,�the�effects�on�Type�I�and�Type�II�errors�are�minimal�when�using�a�two- tailed�test,�as�we�are�conducting�here�(e�g�,�Glass,�Peckham,�&�Sanders,�1972;�Sawilowsky� &�Blair,�1992)�
188 An Introduction to Statistical Concepts
Dependent t Test
Step 1:�To�conduct�a�dependent�t�test,�your�dataset�needs�to�include�the�two�variables� (i�e�,�for�the�paired�samples)�whose�means�you�wish�to�compare�(e�g�,�pretest�and�posttest)�� To� conduct� the� dependent� t� test,� go� to� the�“Analyze”� in� the� top� pulldown� menu,� then� select�“Compare Means,”�and�then�select�“Paired-Samples T Test.”�Following�the� screenshot�(step�1)�as�follows�produces�the�“Paired-Samples T Test”�dialog�box�
A
B
C
Dependent t test: Step 1
Step 2:�Click�both�variables�(e�g�,�pretest�and�posttest�as�variable�1�and�variable�2,�respec- tively)�and�move�them�into�the�“Paired Variables”�box�by�clicking�the�arrow�button�� Both�variables�should�now�appear�in�the�box�as�shown�in�screenshot�step�2��Then�click�on� “Ok” to�run�the�analysis�and�generate�the�output�
Select the paired samples from the list on the left and use the arrow to move to the “Paired Variables” box on the right. Then click on “Ok.”
Dependent t test: Step 2
The�output�appears�in�Table�7�4,�where�again�the�top�box�provides�descriptive�statistics,� the�middle�box�provides�a�bivariate�correlation�coefficient,�and�the�bottom�box�gives�the� results�of�the�dependent�t�test�procedure�
189Inferences About the Difference Between Two Means
Table 7.4
SPSS�Results�for�Dependent�t�Test
Paired Samples Statistics
Mean N Std. Deviation Std. Error Mean
Pretest 64.0000 10 4.21637 1.33333Pair 1 Posttest 59.0000 10 3.62093 1.14504
Paired Samples Correlations
N Correlation Sig.
Pair 1 Pretest and posttest 10 .859 .001
Paired Samples Test
Paired Differences
95% Confidence Interval of the
Difference
Mean Std.
Deviation Std. Error
Mean Lower Upper t df Sig. (2-Tailed) Pair 1 Pretest -
posttest 5.00000 2.16025 .68313 3.45465 6.54535 7.319 9 .000
�e table labeled “Paired Samples Statistics” provides basic descriptive
statistics for the paired samples.
The table labeled “Paired Samples Correlations” provides the Pearson
Product Moment Correlation Coefficient value, a bivariate
correlation coefficient, between the pretest and posttest values.
In this example, there is a strong correlation (r = .859) and it is
statistically significant ( p = .001).�e values in this section of the table are calculated based on paired differences (i.e., the
difference values between pretest and posttest scores).
“Sig.” is the observed p value for the dependent t test.
It is interpreted as: there is less than a 1% probability of a sample mean difference of 5 or
greater occurring by chance if the null
hypothesis is really true (i.e., if the population mean difference is 0).
df are the degrees of freedom. For the
dependent samples
t test, they are calculated as
n – 1.
“t” is the t test statistic value. The t value is calculated as:
5 0.6831
== d sd
t 7.3196=
190 An Introduction to Statistical Concepts
Using “Explore” to Examine Normality of Distribution of Difference Scores
Generating normality evidence:�As�with�the�other�t�tests�we�have�studied,�under- standing� the� distributional� shape� and� the� extent� to� which� normality� is� a� reasonable� assumption� is� important�� For� the� dependent� t� test,� the� distributional� shape� for� the� dif- ference scores� should� be� normally� distributed�� Thus,� we� first� need� to� create� a� new� vari- able�in�our�dataset�to�reflect�the�difference�scores�(in�this�case,�the�difference�between�the� pre-�and�posttest�values)��To�do�this,�go�to�“Transform”�in�the�top�pulldown�menu,�then� select�“Compute Variable.”�Following�the�screenshot�(step�1)�as�follows�produces�the� “Compute Variable”�dialog�box�
A
B
Computing the
difference score:
Step 1
From�the�“Compute Variable”�dialog�screen,�we�can�define�the�column�header�for�our� variable�by�typing�in�a�name�in�the�“Target Variable”�box�(no�spaces,�no�special�char- acters,�and�cannot�begin�with�a�numeric�value)��The�formula�for�computing�our�difference� score�is�inserted�in�the�“Numeric Expression”�box��To�create�this�formula,�(1)�click�on� “pretest”�in�the�left�list�of�variables�and�use�the�arrow�key�to�move�it�into�the�“Numeric Expression” box;�(2)�use�your�keyboard�or�the�keyboard�within�the�dialog�box�to�insert� a�minus�sign�(i�e�,�dash)�after�“pretest”�in�the�“Numeric Expression”�box;�(3)�click�on� “posttest”�in�the�left�list�of�variables�and�use�the�arrow�key�to�move�it�into�the�“Numeric Expression”�box;�and�(4)�click�on�“OK”�to�create�the�new�difference�score�variable�in� your�dataset�
191Inferences About the Difference Between Two Means
Computing the
difference
score: Step 2
We�can�again�use�“Explore”�to�examine�the�extent�to�which�the�assumption�of�normal- ity�is�met�for�the�distributional�shape�of�our�newly�created�difference score��The�general�steps� for� accessing� “Explore”� (see,� e�g�,� Chapter� 4)� and� for� generating� normality� evidence� for� one�variable�(see�Chapter�6)�have�been�presented�in�previous�chapters,�and�they�will�not� be�reiterated�here�
Interpreting normality evidence:� We� have� already� developed� a� good� under- standing�of�how�to�interpret�some�forms�of�evidence�of�normality�including�skewness�and� kurtosis,�histograms,�and�boxplots��The�skewness�statistic�for�the�difference�score�is��248� and� kurtosis� is� �050—both� within� the� range� of� an� absolute� value� of� 2�0,� suggesting� one� form�of�evidence�of�normality�of�the�differences�
The�histogram�for�the�difference�scores�(not�presented�here)�is�not�necessarily�what�most� researchers� would� consider� a� normally� shaped� distribution�� Our� formal� test� of� normal- ity,� the� S–W� (SW)� test� (Shapiro� &� Wilk,� 1965),� suggests� that� our� sample� distribution� for� differences�is�not�statistically�significantly�different�than�what�would�be�expected�from�a� normal�distribution�(S–W�=��956,�df�=�10,�p�=��734)��Similar�to�what�we�saw�with�the�histo- gram,�the�Q–Q�plot�of�differences�suggests�some�nonnormality�in�the�tails�(as�the�farthest� points� are� not� falling� on� the� diagonal� line)�� Keep� in� mind� that� we� have� a� small� sample� size��Thus,�interpreting�the�visual�graphs�(e�g�,�histograms�and�Q–Q�plots)�can�be�difficult�� Examination�of�the�boxplot�suggests�a�relatively�normal�distributional�shape��Considering� the�forms�of�evidence�we�have�examined,�skewness�and�kurtosis,�the�S–W�test�of�normal- ity,� and� boxplots,� all� suggest� normality� is� a� reasonable� assumption�� Although� the� histo- grams�and�Q–Q�plots�suggested�some�nonnormality,�this�is�somewhat�expected�given�the� small�sample�size��Generally,�we�can�be�reasonably�assured�we�have�met�the�assumption� of�normality�of�the�difference�scores�
Generating evidence of homogeneity of variance of difference scores:� Without�conducting�a�formal�test�of�equality�of�variances�(as�we�do�in�Chapter�9),�a�rough� benchmark�for�having�met�the�assumption�of�homogeneity�of�variances�when�conducting�
192 An Introduction to Statistical Concepts
the�dependent�t�test�is�that�the�ratio�of�the�smallest�to�largest�variance�of�the�paired�samples� is�no�greater�than�1:4��The�variance�can�be�computed�easily�by�any�number�of�procedures� in�SPSS�(e�g�,�refer�back�to�Chapter�3),�and�these�steps�will�not�be�repeated�here��For�our� paired�samples,�the�variance�of�the�pretest�score�is�17�778�and�the�variance�of�the�posttest� score�is�13�111—well�within�the�range�of�1:4,�suggesting�that�homogeneity�of�variances�is� reasonable�
7.5 G*Power
Using�the�results�of�the�independent�samples�t�test�just�conducted,�let�us�use�G*Power�to� compute�the�post�hoc�power�of�our�test�
Post Hoc Power for the Independent t Test Using G*Power
The� first� thing� that� must� be� done� when� using� G*Power� for� computing� post� hoc� power� is� to� select� the� correct� test� family�� In� our� case,� we� conducted� an� independent� samples� t� test;� therefore,� the� default� selection� of�“t tests”� is� the� correct� test� family�� Next,� we� need� to� select� the� appropriate� statistical� test�� We� use� the� arrow� to� toggle� to� “Means: Difference between two independent means (two groups).”�The�“Type of Power Analysis”� desired� then� needs� to� be� selected�� To� compute� post� hoc� power,� we� need�to�select�“Post hoc: Compute achieved power–given�α, sample size, and effect size.”
The�“Input Parameters”�must�then�be�specified��The�first�parameter�is�the�selection�of� whether�your�test�is�one-tailed�(i�e�,�directional)�or�two-tailed�(i�e�,�nondirectional)��In�this� example,�we�have�a�two-tailed�test�so�we�use�the�arrow�to�toggle�to�“Two.”�The�achieved� or�observed�effect�size�was�−1�1339��The�alpha�level�we�tested�at�was��05,�and�the�sample� size�for�females�was�8�and�for�males,�12��Once�the�parameters�are�specified,�simply�click�on� “Calculate”�to�generate�the�achieved�power�statistics�
The�“Output Parameters”�provide�the�relevant�statistics�given�the�input�just�speci- fied�� In� this� example,� we� were� interested� in� determining� post� hoc� power� given� a� two- tailed�test,�with�an�observed�effect�size�of�−1�1339,�an�alpha�level�of��05,�and�sample�sizes� of� 8� (females)� and� 12� (males)�� Based� on� those� criteria,� the� post� hoc� power� was� �65�� In� other� words,� with� a� sample� size� of� 8� females� and� 12� males� in� our� study,� testing� at� an� alpha�level�of��05�and�observing�a�large�effect�of�−1�1339,�then�the�power�of�our�test�was� �65—the�probability�of�rejecting�the�null�hypothesis�when�it�is�really�false�will�be�65%,� which�is�only�moderate�power��Keep�in�mind�that�conducting�power�analysis�a�priori�is� recommended�so�that�you�avoid�a�situation�where,�post�hoc,�you�find�that�the�sample�size� was�not�sufficient�to�reach�the�desired�power�(given�the�observed�effect�size�and�alpha� level)��We�were�fortunate�in�this�example�in�that�we�were�still�able�to�detect�a�statistically� significant�difference�in�cholesterol�levels�between�males�and�females;�however�we�will� likely�not�always�be�that�lucky�
193Inferences About the Difference Between Two Means
The “Input Parameters” for computing post hoc power must be specified including:
Once the parameters are
specified, click on “Calculate.”
Independent
t test
1. One versus two tailed test; 2. Observed effect size d; 3. Alpha level; and 4. Sample size for each group of the independent variable.
Post Hoc Power for the Dependent t Test Using G*Power
Now,�let�us�use�G*Power�to�compute�post�hoc�power�for�the�dependent�t�test��First,�the�cor- rect�test�family�needs�to�be�selected��In�our�case,�we�conducted�a�dependent�samples�t�test;� therefore,�the�default�selection�of�“t tests”�is�the�correct�test�family��Next,�we�need�to� select�the�appropriate�statistical�test��We�use�the�arrow�to�toggle�to�“Means: Difference between two dependent means (matched pairs).”� The� “Type of Power Analysis”� desired� then� needs� to� be� selected�� To� compute� post� hoc� power,� we� need� to� select�“Post hoc: Compute achieved power–given α, sample size, and effect size.”
The�“Input Parameters”� must� then� be� specified�� The� first� parameter� is� the� selec- tion�of�whether�your�test�is�one-tailed�(i�e�,�directional)�or�two-tailed�(i�e�,�nondirectional)��
194 An Introduction to Statistical Concepts
In�this�example,�we�have�a�two-tailed�test,�so�we�use�the�arrow�to�toggle�to�“Two.”�The� achieved�or�observed�effect�size�was�2�3146��The�alpha�level�we�tested�at�was��05,�and�the� total�sample�size�was�10��Once�the�parameters�are�specified,�simply�click�on�“Calculate”� to�generate�the�achieved�power�statistics�
The�“Output Parameters”�provide�the�relevant�statistics�given�the�input�specified��In� this�example,�we�were�interested�in�determining�post�hoc�power�given�a�two-tailed�test,� with� an� observed� effect� size� of� 2�3146,� an� alpha� level� of� �05,� and� total� sample� size� of� 10�� Based�on�those�criteria,�the�post�hoc�power�was��99��In�other�words,�with�a�total�sample�size� of�10,�testing�at�an�alpha�level�of��05�and�observing�a�large�effect�of�2�3146,�then�the�power� of�our�test�was�over��99—the�probability�of�rejecting�the�null�hypothesis�when�it�is�really� false�will�be�greater�than�99%,�about�the�strongest�power�that�can�be�achieved��Again,�con- ducting�power�analysis�a�priori�is�recommended�so�that�you�avoid�a�situation�where,�post� hoc,�you�find�that�the�sample�size�was�not�sufficient�to�reach�the�desired�power�(given�the� observed�effect�size�and�alpha�level)�
Once the parameters are
specified, click on “Calculate.”
Dependent
t test
�e “Input Parameters” for computing post hoc power must be specified including:
1. One versus two tailed test; 2. Observed effect size d; 3. Alpha level; and 4. Sample size for each group of the independent variable.
195Inferences About the Difference Between Two Means
7.6 Template and APA-Style Write-Up
Next�we�develop�APA-style�paragraphs�describing�the�results�for�both�examples��First�is�a� paragraph�describing�the�results�of�the�independent�t�test�for�the�cholesterol�example,�and� this�is�followed�by�dependent�t�test�for�the�swimming�example�
Independent t Test
Recall�that�our�graduate�research�assistant,�Marie,�was�working�with�JoAnn,�a�local�nurse� practitioner,� to� assist� in� analyzing� cholesterol� levels�� Her� task� was� to� assist� JoAnn� with� writing�her�research�question�(Is there a mean difference in cholesterol level between males and females?)�and�generating�the�test�of�inference�to�answer�her�question��Marie�suggested�an� independent�samples�t�test�as�the�test�of�inference��A�template�for�writing�a�research�ques- tion�for�an�independent�t�test�is�presented�as�follows:
Is there a mean difference in [dependent variable] between [group 1 of the independent variable] and [group 2 of the independent variable]?
It�may�be�helpful�to�preface�the�results�of�the�independent�samples�t�test�with�informa- tion� on� an� examination� of� the� extent� to� which� the� assumptions� were� met� (recall� there� are� three� assumptions:� normality,� homogeneity� of� variances,� and� independence)�� This� assists� the� reader� in� understanding� that� you� were� thorough� in� data� screening� prior� to� conducting�the�test�of�inference�
An independent samples t test was conducted to determine if the mean cholesterol level of males differed from females. The assumption of normality was tested and met for the distributional shape of the dependent variable (cholesterol level) for females. Review of the S-W test for normality (SW = .931, df = 8, p = .525) and skewness (.000) and kurtosis (−1.790) statistics suggested that normality of cholesterol levels for females was a reasonable assumption. Similar results were found for male cholesterol levels. Review of the S-W test for normality (S-W = .949, df = 12, p = .617) and skewness (.000) and kurtosis (−1.446) statistics suggested that normality of males cholesterol levels was a reasonable assumption. The boxplots suggested a relatively normal distributional shape (with no outliers) of cholesterol levels for both males and females. The Q–Q plots and histograms suggested some minor nonnormality for both male and female cholesterol levels. Due to the small sample, this was anticipated. Although normality indices gener- ally suggest the assumption is met, even if there are slight depar- tures from normality, the effects on Type I and Type II errors will be minimal given the use of a two-tailed test (e.g., Glass, Peckham, & Sanders, 1972; Sawilowsky & Blair, 1992). According to Levene’s test, the homogeneity of variance assumption was satisfied (F = 3.2007, p = .090). Because there was no random assignment of the individuals to gender, the assumption of independence was not met, creating a poten- tial for an increased probability of a Type I or Type II error.
196 An Introduction to Statistical Concepts
It�is�also�desirable�to�include�a�measure�of�effect�size��Recall�our�formula�for�computing� the�effect�size,�d,�presented�earlier�in�the�chapter��Plugging�in�the�values�for�our�cholesterol� example,� we� find� an� effect� size� d� of� −1�1339,� which� is� interpreted� according� to� Cohen’s� (1988)�guidelines�as�a�large�effect:
� d
Y Y sp
= −
= −
= −1 2 185 215 26 4575
1 1339 .
.
Remember�that�for�the�two-sample�mean�test,�d�indicates�how�many�standard�deviations� the�mean�of�sample�1�is�from�the�mean�of�sample�2��Thus,�with�an�effect�size�of�−1�1339,� there�are�nearly�one�and�one-quarter�standard�deviation�units�between�the�mean�choles- terol�levels�of�males�as�compared�to�females��The�negative�sign�simply�indicates�that�group� 1�has�the�smaller�mean�(as�it�is�the�first�value�in�the�numerator�of�the�formula;�in�our�case,� the�mean�cholesterol�level�of�females)�
Here�is�an�APA-style�example�paragraph�of�results�for�the�cholesterol�level�data�(remem- ber�that�this�will�be�prefaced�by�the�paragraph�reporting�the�extent�to�which�the�assump- tions�of�the�test�were�met)�
As shown in Table 7.3, cholesterol data were gathered from samples of 12 males and 8 females, with a female sample mean of 185 (SD = 19.09) and a male sample mean of 215 (SD = 30.22). The independent t test indi- cated that the cholesterol means were statistically significantly dif- ferent for males and females (t = −2.4842, df = 18, p = .023). Thus, the null hypothesis that the cholesterol means were the same by gender was rejected at the .05 level of significance. The effect size d (calculated using the pooled standard deviation) was −1.1339. Using Cohen’s (1988) guidelines, this is interpreted as a large effect. The results provide evidence to support the conclusion that males and females differ in cholesterol levels, on average. More specifically, males were observed to have larger cholesterol levels, on average, than females.
Parenthetically,�notice�that�the�results�of�the�Welch�t′�test�were�the�same�as�for�the�inde- pendent� t� test� (Welch� t′� =� −2�7197,� rounded� df� =� 18,� p� =� �014)�� Thus,� any� deviation� from� homogeneity�of�variance�did�not�affect�the�results�
Dependent t Test
Marie,�as�you�recall,�was�also�working�with�Mark,�a�local�swimming�coach,�to�assist�in�analyz- ing�freestyle�swimming�time�before�and�after�swimmers�participated�in�an�intensive�training� program�� Marie� suggested� a� research� question� (Is there a mean difference in swim time for the 50-meter freestyle event before participation in an intensive training program as compared to swim time for the 50-meter freestyle event after participation in an intensive training program?)�and�assisted�in� generating�the�test�of�inference�(specifically�the�dependent�t�test)�to�answer�her�question��A� template�for�writing�a�research�question�for�a�dependent�t�test�is�presented�as�follows�
Is there a mean difference in [paired sample 1] as compared to [paired sample 2]?
197Inferences About the Difference Between Two Means
It�may�be�helpful�to�preface�the�results�of�the�dependent�samples�t�test�with�information�on� the�extent�to�which�the�assumptions�were�met�(recall�there�are�three�assumptions:�normal- ity,�homogeneity�of�variance,�and�independence)��This�assists�the�reader�in�understanding� that�you�were�thorough�in�data�screening�prior�to�conducting�the�test�of�inference�
A dependent samples t test was conducted to determine if there was a difference in the mean swim time for the 50 meter freestyle before participation in an intensive training program as compared to the mean swim time for the 50 meter freestyle after participation in an intensive training program. The assumption of normality was tested and met for the distributional shape of the paired differences. Review of the S-W test for normality (SW = .956, df = 10, p = .734) and skew- ness (.248) and kurtosis (.050) statistics suggested that normality of the paired differences was reasonable. The boxplot suggested a rela- tively normal distributional shape, and there were no outliers pres- ent. The Q–Q plot and histogram suggested minor nonnormality. Due to the small sample, this was anticipated. Homogeneity of variance was tested by reviewing the ratio of the raw score variances. The ratio of the smallest (posttest = 13.111) to largest (pretest = 17.778) variance was less than 1:4; therefore, there is evidence of the equal variance assumption. The individuals were not randomly selected; therefore, the assumption of independence was not met, creating a potential for an increased probability of a Type I or Type II error.
It�is�also�important�to�include�a�measure�of�effect�size��Recall�our�formula�for�computing� the�effect�size,�d,�presented�earlier�in�the�chapter��Plugging�in�the�values�for�our�swimming� example,�we�find�an�effect�size�d�of�2�3146,�which�is�interpreted�according�to�Cohen’s�(1988)� guidelines�as�a�large�effect:
� Cohen d
d sd
= = = 5
2 1602 2 3146
. .
With� an� effect� size� of� 2�3146,� there� are� about� two� and� a� third� standard� deviation� units� between�the�pretraining�mean�swim�time�and�the�posttraining�mean�swim�time�
Here�is�an�APA-style�example�paragraph�of�results�for�the�swimming�data�(remember� that�this�will�be�prefaced�by�the�paragraph�reporting�the�extent�to�which�the�assumptions� of�the�test�were�met)�
From Table 7.4, we see that pretest and posttest data were collected from a sample of 10 swimmers, with a pretest mean of 64 seconds (SD = 4.22) and a posttest mean of 59 seconds (SD = 3.62). Thus, swimming times decreased from pretest to posttest. The dependent t test was conducted to determine if this difference was statistically significantly dif- ferent from 0, and the results indicate that the pretest and posttest means were statistically different (t = 7.319, df = 9, p < .001). Thus, the null hypothesis that the freestyle swimming means were the same at both points in time was rejected at the .05 level of significance. The effect size d (calculated as the mean difference divided by the standard
198 An Introduction to Statistical Concepts
deviation of the difference) was 2.3146. Using Cohen’s (1988) guidelines, this is interpreted as a large effect. The results provide evidence to support the conclusion that the mean 50 meter freestyle swimming time prior to intensive training is different than the mean 50 meter free- style swimming time after intensive training.
7.7 Summary
In�this�chapter,�we�considered�a�second�inferential�testing�situation,�testing�hypotheses� about� the� difference� between� two� means�� Several� inferential� tests� and� new� concepts� were�discussed��New�concepts�introduced�were�independent�versus�dependent�samples,� the� sampling� distribution� of� the� difference� between� two� means,� the� standard� error� of� the�difference�between�two�means,�and�parametric�versus�nonparametric�tests��We�then� moved�on�to�describe�the�following�three�inferential�tests�for�determining�the�difference� between� two� independent� means:� the� independent� t� test,� the� Welch� t′� test,� and� briefly� the� Mann–Whitney–Wilcoxon� test�� The� following� two� tests� for� determining� the� differ- ence� between� two� dependent� means� were� considered:� the� dependent� t� test� and� briefly� the�Wilcoxon�signed�ranks�test��In�addition,�examples�were�presented�for�each�of�the�t� tests,� and� recommendations� were� made� as� to� when� each� test� is� most� appropriate�� The� chapter�concluded�with�a�look�at�SPSS�and�G*Power�(for�post�hoc�power)�as�well�as�devel- oping�an�APA-style�write-up�of�results��At�this�point,�you�should�have�met�the�following� objectives:�(a)�be�able�to�understand�the�basic�concepts�underlying�the�inferential�tests� of�two�means,�(b)�be�able�to�select�the�appropriate�test,�and�(c)�be�able�to�determine�and� interpret�the�results�from�the�appropriate�test��In�the�next�chapter,�we�discuss�inferential� tests�involving�proportions��Other�inferential�tests�are�covered�in�subsequent�chapters�
Problems
Conceptual problems
7.1� We�test�the�following�hypothesis:
� H0 1 2 0: µ µ− =
� H1 1 2 0: µ µ− ≠
� The�level�of�significance�is��05�and�H0�is�rejected��Assuming�all�assumptions�are�met�and� H0�is�true,�the�probability�of�committing�a�Type�I�error�is�which�one�of�the�following?
� a�� 0 � b�� 0�05 � c�� Between��05�and��95 � d�� 0�95 � e�� 1�00
199Inferences About the Difference Between Two Means
7.2� When�H0�is�true,�the�difference�between�two�independent�sample�means�is�a�func- tion�of�which�one�of�the�following?
� a�� Degrees�of�freedom � b�� The�standard�error � c�� The�sampling�distribution � d�� Sampling�error 7.3� The�denominator�of�the�independent�t�test�is�known�as�the�standard�error�of�the�
difference�between�two�means,�and�may�be�defined�as�which�one�of�the�following? � a�� The�difference�between�the�two�group�means � b�� The�amount�by�which�the�difference�between�the�two�group�means�differs�from�
the�population�mean � c�� The� standard� deviation� of� the� sampling� distribution� of� the� difference� between�
two�means � d�� All�of�the�above � e�� None�of�the�above 7.4� In�the�independent�t�test,�the�homoscedasticity�assumption�states�what? � a�� The�two�population�means�are�equal� � b�� The�two�population�variances�are�equal� � c�� The�two�sample�means�are�equal� � d�� The�two�sample�variances�are�equal� 7.5� Sampling�error�increases�with�larger�samples��True�or�false? 7.6� At� a� given� level� of� significance,� it� is� possible� that� the� significance� test� and� the� CI�
results�will�differ�for�the�same�dataset��True�or�false? 7.7� I� assert� that� the� critical� value� of� t� required� for� statistical� significance� is� smaller� (in�
absolute�value�or�ignoring�the�sign)�when�using�a�directional�rather�than�a�nondirec- tional�test��Am�I�correct?
7.8� If�a�95%�CI�from�an�independent�t�test�ranges�from�−�13�to�+1�67,�I�assert�that�the�null� hypothesis�would�not�be�rejected�at�the��05�level�of�significance��Am�I�correct?
7.9� A� group� of� 15� females� was� compared� to� a� group� of� 25� males� with� respect� to� intel- ligence��To�test�if�the�sample�sizes�are�significantly�different,�which�of�the�following� tests�would�you�use?
� a�� Independent�t�test � b�� Dependent�t�test � c�� z�test � d�� None�of�the�above 7.10� The� mathematic� ability� of� 10� preschool� children� was� measured� when� they� entered�
their�first�year�of�preschool�and�then�again�in�the�spring�of�their�kindergarten�year�� To�test�for�pre-�to�post-mean�differences,�which�of�the�following�tests�would�be�used?
� a�� Independent�t�test � b�� Dependent�t�test � c�� z�test � d�� None�of�the�above
200 An Introduction to Statistical Concepts
7.11� A� researcher� collected� data� to� answer� the� following� research� question:� Are� there� mean�differences�in�science�test�scores�for�middle�school�students�who�participate�in� school-sponsored�athletics�as�compared�to�students�who�do�not�participate?�Which� of�the�following�tests�would�be�used�to�answer�this�question?
� a�� Independent�t�test � b�� Dependent�t�test � c�� z�test � d�� None�of�the�above 7.12� The�number�of�degrees�of�freedom�for�an�independent�t�test�with�15�females�and�
25�males�is�40��True�or�false? 7.13� I�assert�that�the�critical�value�of�t,�for�a�test�of�two�dependent�means,�will�increase�as�
the�samples�become�larger��Am�I�correct? 7.14� Which�of�the�following�is�NOT�an�assumption�of�the�independent�t�test? � a�� Normality � b�� Independence � c�� Equal�sample�sizes � d�� Homogeneity�of�variance 7.15� For� which� of� the� following� assumptions� of� the� independent� t� test� is� evidence� pro-
vided�in�the�SPSS�output�by�default? � a�� Normality � b�� Independence � c�� Equal�sample�sizes � d�� Homogeneity�of�variance
Computational problems
7.1� The�following�two�independent�samples�of�older�and�younger�adults�were�measured� on�an�attitude�toward�violence�test:
Sample 1 (Older Adult) Data Sample 1 (Younger Adult) Data
42 36 47 45 50 57 35 46 37 58 43 52 52 44 47 43 60 41 51 56 54 49 44 51 55 50 40 49� 55� 56 40 46 41
� a�� Test�the�following�hypotheses�at�the��05�level�of�significance:
� H0 1 2 0: µ µ− =
� H1 1 2 0: µ µ− ≠
� b�� Construct�a�95%�CI�
201Inferences About the Difference Between Two Means
7.2� The�following�two�independent�samples�of�male�and�female�undergraduate�students� were�measured�on�an�English�literature�quiz:
Sample 1 (Male) Data Sample 1 (Female) Data
5 7 8 9 9 11 10 11 11 13 15 18 13 15 19 20
� a�� Test�the�following�hypotheses�at�the��05�level�of�significance:
� H0 1 2 0: µ µ− =
� H1 1 2 0: µ µ− ≠
� b�� Construct�a�95%�CI� 7.3� The� following� two� independent� samples� of� preschool� children� (who� were� demo-
graphically� similar� but� differed� in� Head� Start� participation)� were� measured� on� teacher-reported�social�skills�during�the�spring�of�kindergarten:
Sample 1 (Head Start) Data Sample 1 (Non-Head Start) Data
18 14 12 15 12 9 16 10 17 10 18 12 20 16 19 11 8 11 15 13 22 13 10 14
� a�� Test�the�following�hypothesis�at�the��05�level�of�significance:
� H0 1 2 0: µ µ− =
� H1 1 2 0: µ µ− ≠
� b�� Construct�a�95%�CI�
202 An Introduction to Statistical Concepts
7.4� The�following�is�a�random�sample�of�paired�values�of�weight�measured�before�(time�1)� and�after�(time�2)�a�weight-reduction�program:
Pair 1 2
1 127 130 2 126 124 3 129 135 4 123 127 5 124 127 6 129 128 7 132 136 8 125 130 9 135 131
10 126 128
� a�� Test�the�following�hypothesis�at�the��05�level�of�significance:
� H0 1 2 0: µ µ− =
� H1 1 2 0: µ µ− ≠
� b�� Construct�a�95%�CI� 7.5� Individuals�were�measured�on�the�number�of�words�spoken�during�the�1�minute�prior�
to�exposure�to�a�confrontational�situation��During�the�1�minute�after�exposure,�the�indi- viduals�were�again�measured�on�the�number�of�words�spoken��The�data�are�as�follows:
Person Pre Post
1 60 50
2 80 70 3 120 80 4 100 90 5 90 100 6 85 70 7 70 40 8 90 70 9 100 60
10 110 100 11 80 100 12 100 70 13 130 90 14 120 80 15 90 50
� a�� Test�the�following�hypotheses�at�the��05�level�of�significance:
� H0 1 2 0: µ µ− =
� H1 1 2 0: µ µ− ≠
� b�� Construct�a�95%�CI�
203Inferences About the Difference Between Two Means
7.6� The�following�is�a�random�sample�of�scores�on�an�attitude�toward�abortion�scale�for� husband�(sample�1)�and�wife�(sample�2)�pairs:
Pair 1 2
1 1 3 2 2 3 3 4 6 4 4 5 5 5 7 6 7 8 7 7 9 8 8 10
� a�� Test�the�following�hypotheses�at�the��05�level�of�significance:
� H0 1 2 0: µ µ− =
� H1 1 2 0: µ µ− ≠
� b�� Construct�a�95%�CI� 7.7� For� two� dependent� samples,� test� the� following� hypothesis� at� the� �05� level� of�
significance: � Sample�statistics:�n�=�121;�d
– �=�10;�sd�=�45�
� H0 1 2 0: µ µ− =
� H1 1 2 0: µ µ− >
7.8� For� two� dependent� samples,� test� the� following� hypothesis� at� the� �05� level� of� significance�
� Sample�statistics:�n�=�25;�d – �=�25;�sd�=�14�
� H0 1 2 0: µ µ− =
� H1 1 2 0: µ µ− >
Interpretive problems
7.1� Using� the� survey� 1� dataset� from� the� website,� use� SPSS� to� conduct� an� independent� t� test,� where� gender� is� the� grouping� variable� and� the� dependent� variable� is� a� vari- able�of�interest�to�you��Test�for�the�extent�to�which�the�assumptions�have�been�met�� Calculate� an� effect� size� as� well� as� post� hoc� power�� Then� write� an� APA-style� para- graph�describing�the�results�
7.2� Using�the�survey�1�dataset�from�the�website,�use�SPSS�to�conduct�an�independent�t test,� where� the� grouping� variable� is� whether� or� not� the� person� could� tell� the� difference� between�Pepsi�and�Coke�and�the�dependent�variable�is�a�variable�of�interest�to�you��Test� for�the�extent�to�which�the�assumptions�have�been�met��Calculate�an�effect�size�as�well�as� post�hoc�power��Then�write�an�APA-style�paragraph�describing�the�results�
205
8 Inferences About Proportions
Chapter Outline
8�1� Inferences�About�Proportions�Involving�the�Normal�Distribution 8�1�1� Introduction 8�1�2� Inferences�About�a�Single�Proportion 8�1�3� Inferences�About�Two�Independent�Proportions 8�1�4� Inferences�About�Two�Dependent�Proportions
8�2� Inferences�About�Proportions�Involving�the�Chi-Square�Distribution 8�2�1� Introduction 8�2�2� Chi-Square�Goodness-of-Fit�Test 8�2�3� Chi-Square�Test�of�Association
8�3� SPSS 8�4� G*Power 8�5� Template�and�APA-Style�Write-Up
Key Concepts
� 1�� Proportion � 2�� Sampling�distribution�and�standard�error�of�a�proportion � 3�� Contingency�table � 4�� Chi-square�distribution � 5�� Observed�versus�expected�proportions
In� Chapters� 6� and� 7,� we� considered� testing� inferences� about� means,� first� for� a� single� mean� (Chapter�6)�and�then�for�two�means�(Chapter�7)��The�major�concepts�discussed�in�those�two� chapters� included� the� following:� types� of� hypotheses,� types� of� decision� errors,� level� of� sig- nificance,�power,�confidence�intervals�(CIs),�effect�sizes,�sampling�distributions�involving�the� mean,�standard�errors�involving�the�mean,�inferences�about�a�single�mean,�inferences�about� the�difference�between�two�independent�means,�and�inferences�about�the�difference�between� two� dependent� means�� In� this� chapter,� we� consider� inferential� tests� involving� proportions�� We�define�a�proportion�as�the�percentage�of�scores�falling�into�particular�categories��Thus,� the�tests�described�in�this�chapter�deal�with�variables�that�are�categorical�in�nature�and�thus� are� nominal� or� ordinal� variables� (see� Chapter� 1),� or� have� been� collapsed� from� higher-level� variables�into�nominal�or�ordinal�variables�(e�g�,�high�and�low�scorers�on�an�achievement�test)�
206 An Introduction to Statistical Concepts
The�tests�that�we�cover�in�this�chapter�are�considered�nonparametric�procedures,�also� sometimes�referred�to�as�distribution-free�procedures,�as�there�is�no�requirement�that�the� data�adhere�to�a�particular�distribution�(e�g�,�normal�distribution)��Nonparametric�pro- cedures�are�often�less�preferable�than�parametric�procedures�(e�g�,�t�tests�which�assume� normality� of� the� distribution)� for� the� following� reasons:� (1)� parametric� procedures� are� often� robust� to� assumption� violations;� in� other� words,� the� results� are� often� still� inter- pretable�even�if�there�may�be�assumption�violations;�(2)�nonparametric�procedures�have� lower�power�relative�to�sample�size;�in�other�words,�rejecting�the�null�hypothesis�if�it�is� false�requires�a�larger�sample�size�with�nonparametric�procedures;�and�(3)�the�types�of� research�questions�that�can�be�addressed�by�nonparametric�procedures�are�often�quite� simple� (e�g�,� while� complex� interactions� of� many� different� variables� can� be� tested� with� parametric� procedures� such� as� factorial� analysis� of� variance,� this� cannot� be� done� with� nonparametric�procedures)��Nonparametric�procedures�can�still�be�valuable�to�use�given� the� measurement� scale(s)� of� the� variable(s)� and� the� research� question;� however,� at� the� same�time,�it�is�important�that�researchers�recognize�the�limitations�in�using�these�types� of�procedures�
Research�questions�to�be�asked�of�proportions�include�the�following�examples:
� 1�� Is� the� quarter� in� my� hand� a� fair� or� biased� coin;� in� other� words,� over� repeated� samples,�is�the�proportion�of�heads�equal�to��50�or�not?
� 2�� Is�there�a�difference�between�the�proportions�of�Republicans�and�Democrats�who� support�the�local�school�bond�issue?
� 3�� Is�there�a�relationship�between�ethnicity�(e�g�,�African-American,�Caucasian)�and� type�of�criminal�offense�(e�g�,�petty�theft,�rape,�murder);�in�other�words,�is�the�pro- portion�of�one�ethnic�group�different�from�another�in�terms�of�the�types�of�crimes� committed?
Several�inferential�tests�are�covered�in�this�chapter,�depending�on�(a)�whether�there�are�one� or�two�samples,�(b)�whether�the�two�samples�are�selected�in�an�independent�or�dependent� manner,�and�(c)�whether�there�are�one�or�more�categorical�variables��More�specifically,�the� topics� described� include� the� following� inferential� tests:� testing� whether� a� single� propor- tion�is�different�from�a�hypothesized�value,�testing�whether�two�independent�proportions� are�different,�testing�whether�two�dependent�proportions�are�different,�and�the�chi-square� goodness-of-fit� test� and� chi-square� test� of� association�� We� use� many� of� the� foundational� concepts� previously� covered� in� Chapters� 6� and� 7�� New� concepts� to� be� discussed� include� the�following:�proportion,�sampling�distribution�and�standard�error�of�a�proportion,�con- tingency� table,� chi-square� distribution,� and� observed� versus� expected� frequencies�� Our� objectives�are�that�by�the�end�of�this�chapter,�you�will�be�able�to�(a)�understand�the�basic� concepts�underlying�tests�of�proportions,�(b)�select�the�appropriate�test,�and�(c)�determine� and�interpret�the�results�from�the�appropriate�test�
8.1 Inferences About Proportions Involving Normal Distribution
We�have�been�following�Marie,�an�educational�research�graduate�student,�as�she�completes� tasks�assigned�to�her�by�her�faculty�advisor�
207Inferences About Proportions
Marie’s�advisor�has�received�two�additional�calls�from�individuals�in�other�states�who� are� interested� in� assistance� with� statistical� analysis�� Knowing� the� success� Marie� has� had� with� the� previous� consultations,� Marie’s� advisor� requests� that� Marie� work� with� Tami,�a�staff�member�in�the�Undergraduate�Services�Office�at�Ivy-Covered�University� (ICU),�and�Matthew,�a�lobbyist�from�a�state�that�is�considering�legalizing�gambling�
In�conversation�with�Marie,�Tami�shares�that�she�recently�read�a�report�that�provided� national�statistics�on�the�proportion�of�students�that�major�in�various�disciplines��Tami� wants�to�know�if�there�are�similar�proportions�at�their�institution��Marie�suggests�the� following� research� question:� Are the sample proportions of undergraduate student college majors at Ivy Covered University in the same proportions of those nationally?�Marie�suggests� a�chi-square�goodness-of-fit�test�as�the�test�of�inference��Her�task�is�then�to�assist�Tami� in�generating�the�test�of�inference�to�answer�her�research�question�
Marie�then�speaks�with�Matthew,�a�lobbyist�who�is�lobbying�against�legalizing�gam- bling�in�his�state��Matthew�wants�to�determine�if�there�is�a�relationship�between�level� of�education�and�stance�on�a�proposed�gambling�amendment��Matthew�suspects�that� the�proportions�supporting�gambling�vary�as�a�function�of�their�education�level��The� following�research�question�is�suggested�by�Marie:�Is there an association between level of education and stance on gambling?�Marie�suggests�a�chi-square�test�of�association�as�the� test�of�inference��Her�task�is�then�to�assist�Matthew�in�generating�the�test�of�inference� to�answer�his�research�question�
This�section�deals�with�concepts�and�procedures�for�testing�inferences�about�proportions� that�involve�the�normal�distribution��Following�a�discussion�of�the�concepts�related�to�tests� of�proportions,�inferential�tests�are�presented�for�situations�when�there�is�a�single�propor- tion,�two�independent�proportions,�and�two�dependent�proportions�
8.1.1 Introduction
Let�us�examine�in�greater�detail�the�concepts�related�to�tests�of�proportions��First,�a�propor- tion�represents�the�percentage�of�individuals�or�objects�that�fall�into�a�particular�category�� For� instance,� the� proportion� of� individuals� who� support� a� particular� political� candidate� might�be�of�interest��Thus,�the�variable�here�is�a�dichotomous,�categorical,�nominal�variable,� as�there�are�only�two�categories�represented,�support�or�do�not�support�the�candidate�
For�notational�purposes,�we�define�the�population proportion�π�(pi)�as
π = f N
where f� is� the� number� of� frequencies� in� the� population� who� fall� into� the� category� of� interest�
(e�g�,�the�number�of�individuals�in�the�population�who�support�the�candidate) N�is�the�total�number�of�individuals�in�the�population
For� example,� if� the� population� consists� of� 100� individuals� and� 58� support� the� candidate,� then�π�=��58�(i�e�,�58/100)��If�the�proportion�is�multiplied�by�100%,�this�yields�the�percent- age� of� individuals� in� the� population� who� support� the� candidate,� which� in� the� example� would�be�58%��At�the�same�time,�1�−�π�represents�the�population�proportion�of�individuals� who�do�not�support�the�candidate,�which�for�this�example�would�be�1�−��58�=��42��If�this�is� multiplied�by�100%,�this�yields�the�percentage�of�individuals�in�the�population�who�do�not� support�the�candidate,�which�in�the�example�would�be�42%�
208 An Introduction to Statistical Concepts
In�a�fashion,�the�population�proportion�is�conceptually�similar�to�the�population�mean�if� the�category�of�interest�(support�of�candidate)�is�coded�as�1�and�the�other�category�(no�sup- port)�is�coded�as�0��In�the�case�of�the�example�with�100�individuals,�there�are�58�individuals� coded�1,�42�individuals�coded�0,�and�therefore,�the�mean�would�be��58��To�this�point�then,�we� have�π�representing�the�population�proportion�of�individuals�supporting�the�candidate�and� 1�−�π�representing�the�population�proportion�of�individuals�not�supporting�the�candidate�
The�population variance of a proportion�can�also�be�determined�by�σ2�=�π(1�−�π),�and� thus,� the� population� standard� deviation� of� a� proportion� is� σ π π= −( )1 �� These� provide� us�with�measures�of�variability�that�represent�the�extent�to�which�the�individuals�in�the� population� vary� in� their� support� of� the� candidate�� For� the� example� population� then,� the� variance�is�computed�to�be�σ2�=�π(1�−�π)�=��58(1�−��58)�=��58(�42)�=��2436,�and�the�standard� deviation�is�σ π π= − = − =( ) . ( . ) . (. )1 58 1 58 58 42 �=��4936�
For�the�population�parameters,�we�now�have�the�population�proportion�(or�mean),�the�pop- ulation�variance,�and�the�population�standard�deviation��The�next�step�is�to�discuss�the�cor- responding�sample�statistics�for�the�proportion��The�sample proportion�p�is�defined�as
p f n
=
where f�is�the�number�of�frequencies�in�the�sample�that�fall�into�the�category�of�interest�(e�g�,�the�
number�of�individuals�who�support�the�candidate) n�is�the�total�number�of�individuals�in�the�sample
The� sample� proportion� p� is� thus� a� sample� estimate� of� the� population� proportion� π�� One� way�we�can�estimate�the�population�variance�is�by�the�sample�variance�s2�=�p(1�−�p),�and�the� population�standard�deviation�of�a�proportion�can�be�estimated�by�the�sample�standard� deviation�s p p= −( )1 �
The�next�concept�to�discuss�is�the�sampling�distribution�of�the�proportion��This�is�com- parable� to� the� sampling� distribution� of� the� mean� discussed� in� Chapter� 5�� If� one� were� to� take�many�samples,�and�for�each�sample,�compute�the�sample�proportion�p,�then�we�could� generate�a�distribution�of�p��This�is�known�as�the�sampling distribution of the proportion�� For�example,�imagine�that�we�take�50�samples�of�size�100�and�determine�the�proportion� for�each�sample��That�is,�we�would�have�50�different�sample�proportions�each�based�on�100� observations��If�we�construct�a�frequency�distribution�of�these�50�proportions,�then�this�is� actually�the�sampling�distribution�of�the�proportion�
In�theory,�the�sample�proportions�for�this�example�could�range�from��00�(p�=�0/100)�to�1�00� (p�=�100/100)�given�that�there�are�100�observations�in�each�sample��One�could�also�examine� the�variability�of�these�50�sample�proportions��That�is,�we�might�be�interested�in�the�extent� to�which�the�sample�proportions�vary��We�might�have,�for�one�example,�most�of�the�sample� proportions�falling�near�the�mean�proportion�of��60��This�would�indicate�for�the�candidate� data�that�(a)�the�samples�generally�support�the�candidate,�as�the�average�proportion�is��60,� and�(b)�the�support�for�the�candidate�is�fairly�consistent�across�samples,�as�the�sample�pro- portions�tend�to�fall�close�to��60��Alternatively,�in�a�second�example,�we�might�find�the�sample� proportions� varying� quite� a� bit� around� the� mean� of� �60,� say� ranging� from� �20� to� �80�� This� would� indicate� that� (a)� the� samples� generally� support� the� candidate� again,� as� the� average� proportion�is��60,�and�(b)�the�support�for�the�candidate�is�not�very�consistent�across�samples,� leading�one�to�believe�that�some�groups�support�the�candidate�and�others�do�not�
209Inferences About Proportions
The�variability�of�the�sampling�distribution�of�the�proportion�can�be�determined�as�fol- lows��The�population�variance�of�the�sampling�distribution�of�the�proportion�is�known�as� the�variance error of the proportion,�denoted�by�σ p
2��The�variance�error�is�computed�as
σ π π
p n
2 1= −( )
where π�is�again�the�population�proportion n�is�sample�size�(i�e�,�the�number�of�observations�in�a�single�sample)
The�population�standard�deviation�of�the�sampling�distribution�of�the�proportion�is�known� as� the� standard error of the proportion,� denoted� by� σp�� The� standard� error� is� an� index� of� how� variable� a� sample� statistic� (in� this� case,� the� sample� proportion)� is� when� multiple� samples�of�the�same�size�are�drawn,�and�is�computed�as�follows:
σ π π
p n
= −( )1
This�situation�is�quite�comparable�to�the�sampling�distribution�of�the�mean�discussed�in� Chapter�5��There�we�had�the�variance�error�and�standard�error�of�the�mean�as�measures�of� the�variability�of�the�sample�means�
Technically� speaking,� the� binomial� distribution� is� the� exact� sampling� distribution� for� the� proportion;�binomial�here�refers�to�a�categorical�variable�with�two�possible�categories,�which�is� certainly�the�situation�here��However,�except�for�rather�small�samples,�the�normal�distribution� is�a�reasonable�approximation�to�the�binomial�distribution�and�is�therefore�typically�used��The� reason�we�can�rely�on�the�normal�distribution�is�due�to�the�central�limit�theorem,�previously� discussed�in�Chapter�5��For�proportions,�the�central�limit�theorem�states�that�as�sample�size�n� increases,�the�sampling�distribution�of�the�proportion�from�a�random�sample�of�size�n�more� closely�approximates�a�normal�distribution��If�the�population�distribution�is�normal�in�shape,� then� the� sampling� distribution� of� the� proportion� is� also� normal� in� shape�� If� the� population� distribution�is�not�normal�in�shape,�then�the�sampling�distribution�of�the�proportion�becomes� more�nearly�normal�as�sample�size�increases��As�previously�shown�in�Figure�5�1�in�the�context� of�the�mean,�the�bottom�line�is�that�if�the�population�is�nonnormal,�this�will�have�a�minimal� effect�on�the�sampling�distribution�of�the�proportion�except�for�rather�small�samples�
Because� nearly� always� the� applied� researcher� only� has� access� to� a� single� sample,� the� population� variance� error� and� standard� error� of� the� proportion� must� be� estimated�� The� sample�variance�error�of�the�proportion�is�denoted�by�sp
2�and�computed�as
s p p
n p 2 1=
−( )
where p�is�again�the�sample�proportion n�is�sample�size
The�sample�standard�error�of�the�proportion�is�denoted�by�sp�and�computed�as
s p p
n p =
−( )1
210 An Introduction to Statistical Concepts
8.1.2 Inferences about a Single proportion
In�the�first�inferential�testing�situation�for�proportions,�the�researcher�would�like�to� know� whether� the� population� proportion� is� equal� to� some� hypothesized� proportion� or� not�� This� is� comparable� to� the� one-sample� t� test� described� in� Chapter� 6� where� a� population�mean�was�compared�against�some�hypothesized�mean��First,�the�hypoth- eses� to� be� evaluated� for� detecting� whether� a� population� proportion� differs� from� a� hypothesized� proportion� are� as� follows�� The� null� hypothesis� H0� is� that� there� is� no� difference�between�the�population�proportion�π�and�the�hypothesized�proportion�π0,� which�we�denote�as
H0 0: π π=
Here�there�is�no�difference,�or�a�“null”�difference,�between�the�population�proportion�and� the� hypothesized� proportion�� For� example,� if� we� are� seeking� to� determine� whether� the� quarter� you� are� flipping� is� a� biased� coin� or� not,� then� a� reasonable� hypothesized� value� would�be��50,�as�an�unbiased�coin�should�yield�“heads”�about�50%�of�the�time�
The� nondirectional,� scientific,� or� alternative� hypothesis� H1� is� that� there� is� a� difference� between� the� population� proportion� π� and� the� hypothesized� proportion� π0,� which� we� denote�as
H1 0: π π≠
The�null�hypothesis�H0�will�be�rejected�here�in�favor�of�the�alternative�hypothesis�H1� if� the� population� proportion� is� different� from� the� hypothesized� proportion�� As� we� have�not�specified�a�direction�on�H1,�we�are�willing�to�reject�H0�either�if�π�is�greater� than�π0�or�if�π�is�less�than�π0��This�alternative�hypothesis�results�in�a�two-tailed�test�� Directional� alternative� hypotheses� can� also� be� tested� if� we� believe� either� that� π� is� greater�than�π0�or�that�π�is�less�than�π0��In�either�case,�the�more�the�resulting�sample� proportion�differs�from�the�hypothesized�proportion,�the�more�likely�we�are�to�reject� the�null�hypothesis�
It�is�assumed�that�the�sample�is�randomly�drawn�from�the�population�(i�e�,�the�assump- tion�of�independence)�and�that�the�normal�distribution�is�the�appropriate�sampling�distri- bution��The�next�step�is�to�compute�the�test�statistic�z�as
z p s
p
n p
= −
= −
− π π
π π 0 0
0 01ˆ ( )
where�sp̂�is�estimated�based�on�the�hypothesized�proportion�π0� The�test�statistic�z�is�then�compared�to�a�critical�value(s)�from�the�unit�normal�distribu-
tion��For�a�two-tailed�test,�the�critical�values�are�denoted�as�±α/2z�and�are�found�in�Table� A�1��If�the�test�statistic�z�falls�into�either�critical�region,�then�we�reject�H0;�otherwise,�we�fail� to� reject� H0�� For� a� one-tailed� test,� the� critical� value� is� denoted� as� +αz� for� the� alternative� hypothesis�H1:�π�>�π0�(i�e�,�a�right-tailed�test)�and�as�−αz�for�the�alternative�hypothesis�
211Inferences About Proportions
H1:� π� <� π0� (i�e�,� a� left-tailed� test)�� If� the� test� statistic� z� falls� into� the� appropriate� critical� region,�then�we�reject�H0;�otherwise,�we�fail�to�reject�H0�
For� the� two-tailed� test,� a� (1� −� α)%� CI� can� also� be� examined�� The� CI� is� formed� as� follows:
p z sp± α/ ( )2 ˆ
where p�is�the�observed�sample�proportion ±α/2z�is�the�tabled�critical�value sp̂�is�the�sample�standard�error�of�the�proportion
If�the�CI�contains�the�hypothesized�proportion�π0,�then�the�conclusion�is�to�fail�to�reject� H0;� otherwise,� we� reject� H0�� Simulation� research� has� shown� that� this� CI� procedure� works�fine�for�small�samples�when�the�sample�proportion�is�near��50;�that�is,�the�normal� distribution� is� a� reasonable� approximation� in� this� situation�� However,� as� the� sample� proportion�moves�closer�to�0�or�1,�larger�samples�are�required�for�the�normal�distribu- tion� to� be� reasonably� approximate�� Alternative� approaches� have� been� developed� that� appear�to�be�more�widely�applicable��The�interested�reader�is�referred�to�Ghosh�(1979)� and�Wilcox�(1996)�
Several�points�should�be�noted�about�each�of�the�z�tests�for�proportions�developed�in� this�chapter��First,�the�interpretation�of�CIs�described�in�this�chapter�is�the�same�as�those� in�Chapter�7��Second,�Cohen’s�(1988)�measure�of�effect�size�for�proportion�tests�using�z�is� known�as�h��Unfortunately,�h�involves�the�use�of�arcsine�transformations�of�the�propor- tions,� which� is� beyond� the� scope� of� this� test�� In� addition,� standard� statistical� software,� such�as�SPSS,�does�not�provide�measures�of�effect�size�for�any�of�these�tests�
Let�us�consider�an�example�to�illustrate�the�use�of�the�test�of�a�single�proportion��We�fol- low�the�basic�steps�for�hypothesis�testing�that�we�applied�in�previous�chapters��These�steps� include�the�following:
� 1�� State�the�null�and�alternative�hypotheses� � 2�� Select�the�level�of�significance�(i�e�,�alpha,�α)� � 3�� Calculate�the�test�statistic�value� � 4�� Make�a�statistical�decision�(reject�or�fail�to�reject�H0)�
Suppose�a�researcher�conducts�a�survey�in�a�city�that�is�voting�on�whether�or�not�to�have� an�elected�school�board��Based�on�informal�conversations�with�a�small�number�of�influ- ential�citizens,�the�researcher�is�led�to�hypothesize�that�50%�of�the�voters�are�in�favor�of� an�elected�school�board��Through�the�use�of�a�scientific�poll,�the�researcher�would�like�to� know�whether�the�population�proportion�is�different�from�this�hypothesized�value;�thus,� a� nondirectional,� two-tailed� alternative� hypothesis� is� utilized�� The� null� and� alternative� hypotheses�are�denoted�as�follows:
H
H
0 0
1 0
:
:
π π
π π
=
≠
212 An Introduction to Statistical Concepts
If�the�null�hypothesis�is�rejected,�this�would�indicate�that�scientific�polls�of�larger�samples� yield� different� results� and� are� important� in� this� situation�� If� the� null� hypothesis� is� not� rejected,�this�would�indicate�that�informal�conversations�with�a�small�sample�are�just�as� accurate�as�a�scientific�larger-sized�sample�
A� random� sample� of� 100� voters� is� taken,� and� 60� indicate� their� support� of� an� elected� school�board�(i�e�,�p�=��60)��In�an�effort�to�minimize�the�Type�I�error�rate,�the�significance� level�is�set�at�α�=��01��The�test�statistic�z�is�computed�as
z p
n
= −
− =
− −
= = π
π π 0
0 01 60 50 50 1 50
100
10 50 50
100
10 05( )
. . . ( . )
. . (. )
. . 000
2 0000= .
Note� that� the� final� value� for� the� denominator� is� the� standard� error� of� the� proportion� (i�e�,�sp̂�=��0500),�which�we�will�need�for�computing�the�CI��From�Table�A�1,�we�determine� the�critical�values�to�be�±α/2�z�=�±�005�z�=�±2�58;�in�other�words,�the�z�value�that�corresponds� to�the�P(z)�value�closest�to��995�is�when�z�is�equal�to�2�58��As�the�test�statistic�(i�e�,�z�=�2�000)� does�not�exceed�the�critical�values�and�thus�fails�to�fall�into�a�critical�region,�our�decision� is�to�fail�to�reject�H0��Our�conclusion�then�is�that�the�accuracy�of�the�scientific�poll�is�not� any�different�from�the�hypothesized�value�of��50�as�determined�informally�
The�99%�CI�for�the�example�would�be�computed�as�follows:
p z sp± = ± = ± =α/ ( ) . . (. ) . . (. , . )2 60 2 58 0500 60 129 471 729ˆ
Because� the� CI� contains� the� hypothesized� value� of� �50,� our� conclusion� is� to� fail� to� reject� H0�(the�same�result�found�when�we�conducted�the�statistical�test)��The�conclusion�derived� from�the�test�statistic�is�always�consistent�with�the�conclusion�derived�from�the�CI��We�can� interpret�the�CI�as�follows:�99%�of�similarly�constructed�CIs�will�contain�the�hypothesized� value�of��50�
8.1.3 Inferences about Two Independent proportions
In� our� second� inferential� testing� situation� for� proportions,� the� researcher� would� like� to� know�whether�the�population�proportion�for�one�group�is�different�from�the�population� proportion�for�a�second�independent�group��This�is�comparable�to�the�independent�t�test� described�in�Chapter�7,�where�one�population�mean�was�compared�to�a�second�indepen- dent� population� mean�� Once� again,� we� have� two� independently� drawn� samples,� as� dis- cussed�in�Chapter�7�
First,� the� hypotheses� to� be� evaluated� for� detecting� whether� two� independent� popula- tion�proportions�differ�are�as�follows��The�null�hypothesis�H0�is�that�there�is�no�difference� between�the�two�population�proportions�π1�and�π2,�which�we�denote�as
H0 1 2 0: π π− =
Here� there� is� no� difference,� or� a� “null”� difference,� between� the� two� population� propor- tions��For�example,�we�may�be�seeking�to�determine�whether�the�proportion�of�Democratic� senators�who�support�gun�control�is�equal�to�the�proportion�of�Republican�senators�who� support�gun�control�
213Inferences About Proportions
The� nondirectional,� scientific,� or� alternative� hypothesis� H1� is� that� there� is� a� difference� between�the�population�proportions�π1�and�π2,�which�we�denote�as
H1 1 2 0: π π− ≠
The�null�hypothesis�H0�will�be�rejected�here�in�favor�of�the�alternative�hypothesis�H1�if�the�pop- ulation�proportions�are�different��As�we�have�not�specified�a�direction�on�H1,�we�are�willing� to�reject�either�if�π1�is�greater�than�π2�or�if�π1�is�less�than�π2��This�alternative�hypothesis�results� in�a�two-tailed�test��Directional�alternative�hypotheses�can�also�be�tested�if�we�believe�either� that�π1�is�greater�than�π2�or�that�π1�is�less�than�π2��In�either�case,�the�more�the�resulting�sample� proportions�differ�from�one�another,�the�more�likely�we�are�to�reject�the�null�hypothesis�
It� is� assumed� that� the� two� samples� are� independently� and� randomly� drawn� from� their� respective�populations�(i�e�,�the�assumption�of�independence)�and�that�the�normal�distribu- tion�is�the�appropriate�sampling�distribution��The�next�step�is�to�compute�the�test�statistic�z�as
z p p s
p p
p p n n
p p
= −
= −
− +
−
1 2 1 2
1 2
1 2 1
1 1 ( )
where�n1�and�n2�are�the�sample�sizes�for�samples�1�and�2,�respectively,�and
p f f n n
= + +
1 2
1 2
where�f1�and�f2�are�the�number�of�observed�frequencies�for�samples�1�and�2,�respectively�� The�denominator�of�the�z�test�statistic�sp p1 2− �is�known�as�the�standard error of the differ- ence between two proportions�and�provides�an�index�of�how�variable�the�sample�statistic� (in�this�case,�the�sample�proportion)�is�when�multiple�samples�of�the�same�size�are�drawn�� This�test�statistic�is�conceptually�similar�to�the�test�statistic�for�the�independent�t�test�
The�test�statistic�z�is�then�compared�to�a�critical�value(s)�from�the�unit�normal�distribu- tion��For�a�two-tailed�test,�the�critical�values�are�denoted�as�±α/2z�and�are�found�in�Table� A�1�� If� the� test� statistic� z� falls� into� either� critical� region,� then� we� reject� H0;� otherwise,� we� fail�to�reject�H0��For�a�one-tailed�test,�the�critical�value�is�denoted�as�+αz�for�the�alternative� hypothesis�H1:�π1�−�π2�>�0�(i�e�,�a�right-tailed�test)�and�as�−αz�for�the�alternative�hypothesis� H1:�π1�−�π2�<�0�(i�e�,�a�left-tailed�test)��If�the�test�statistic�z�falls�into�the�appropriate�critical� region,�then�we�reject�H0;�otherwise,�we�fail�to�reject�H0��It�should�be�noted�that�other�alter- natives�to�this�test�have�been�proposed�(e�g�,�Storer�&�Kim,�1990)�
For�the�two-tailed�test,�a�(1�−�α)%�CI�can�also�be�examined��The�CI�is�formed�as�follows:
( ) ( )/p p z sp p1 2 2 1 2− ± −α
If� the� CI� contains� 0,� then� the� conclusion� is� to� fail� to� reject� H0;� otherwise,� we� reject� H0�� Alternative�methods�are�described�by�Beal�(1987)�and�Coe�and�Tamhane�(1993)�
Let�us�consider�an�example�to�illustrate�the�use�of�the�test�of�two�independent�propor- tions��Suppose�a�researcher�is�taste-testing�a�new�chocolate�candy�(“chocolate�yummies”)� and� wants� to� know� the� extent� to� which� individuals� would� likely� purchase� the� product��
214 An Introduction to Statistical Concepts
As�taste�in�candy�may�be�different�for�adults�versus�children,�a�study�is�conducted�where� independent� samples� of� adults� and� children� are� given� “chocolate� yummies”� to� eat� and� asked�whether�they�would�buy�them�or�not��The�researcher�would�like�to�know�whether� the� population� proportion� of� individuals� who� would� purchase� “chocolate� yummies”� is� different�for�adults�and�children��Thus,�a�nondirectional,�two-tailed�alternative�hypothesis� is�utilized��The�null�and�alternative�hypotheses�are�denoted�as�follows:
H
H
0 1 2
1 1 2
0
0
:
:
π π
π π
− =
− ≠
If�the�null�hypothesis�is�rejected,�this�would�indicate�that�interest�in�purchasing�the�prod- uct�is�different�in�the�two�groups,�and�this�might�result�in�different�marketing�and�packag- ing�strategies�for�each�group��If�the�null�hypothesis�is�not�rejected,�then�this�would�indicate� the�product�is�equally�of�interest�to�both�adults�and�children,�and�different�marketing�and� packaging�strategies�are�not�necessary�
A�random�sample�of�100�children�(sample�1)�and�a�random�sample�of�100�adults�(sam- ple� 2)� are� independently� selected�� Each� individual� consumes� the� product� and� indicates� whether�or�not�he�or�she�would�purchase�it��Sixty-eight�of�the�children�and�54�of�the�adults� state�they�would�purchase�“chocolate�yummies”�if�they�were�available��The�level�of�signifi- cance�is�set�at�α�=��05��The�test�statistic�z�is�computed�as�follows��We�know�that�n1�=�100,� n2�=�100,�f1�=�68,�f2�=�54,�p1�=��68,�and�p2�=��54��We�compute�p�to�be
p f f n n
= + +
= + +
= =1 2 1 2
68 54 100 100
122 200
6100.
This�allows�us�to�compute�the�test�statistic�z�as
z p p
p p n n
= −
− +
= . − .
− +
1 2
61 1 61 1
100 1
100 (1 )
1 1
68 54
. ( . ) 1 2
= = = .
(. )(. )(. ) .
. 2.0290
14 61 39 02
14 0690
The�denominator�of�the�z�test�statistic,�sp p1 2− �=��0690,�is�the�standard�error�of�the�difference� between�two�proportions,�which�we�will�need�for�computing�the�CI�
The�test�statistic�z�is�then�compared�to�the�critical�values�from�the�unit�normal�distribu- tion��As�this�is�a�two-tailed�test,�the�critical�values�are�denoted�as�±α/2z�and�are�found�in� Table�A�1�to�be�±α/2z�=�±�025z�=�±1�9600��In�other�words,�this�is�the�z�value�that�is�closest�to� a�P(z)�of��975��As�the�test�statistic�z�falls�into�the�upper�tail�critical�region,�we�reject�H0�and� conclude�that�the�adults�and�children�are�not�equally�interested�in�the�product�
Finally,�we�can�compute�the�95%�CI�as�follows:
( ) ( ) (. . ) . (. ) (. ) (. ) (/p p z sp p1 2 2 1 2 68 54 1 96 0690 14 1352− ± = − ± = ± =−α .. , . )0048 2752
Because�the�CI�does�not�include�0,�we�would�again�reject�H0�and�conclude�that�the�adults� and�children�are�not�equally�interested�in�the�product��As�previously�stated,�the�conclusion�
215Inferences About Proportions
derived�from�the�test�statistic�is�always�consistent�with�the�conclusion�derived�from�the�CI� at�the�same�level�of�significance��We�can�interpret�the�CI�as�follows:�for�95%�of�similarly� constructed�CIs,�the�true�population�proportion�difference�will�not�include�0�
8.1.4 Inferences about Two dependent proportions
In�our�third�inferential�testing�situation�for�proportions,�the�researcher�would�like�to�know� whether�the�population�proportion�for�one�group�is�different�from�the�population�propor- tion�for�a�second�dependent�group��This�is�comparable�to�the�dependent�t�test�described�in� Chapter�7,�where�one�population�mean�was�compared�to�a�second�dependent�population� mean��Once�again,�we�have�two�dependently�drawn�samples,�as�discussed�in�Chapter�7��For� example,�we�may�have�a�pretest-posttest�situation�where�a�comparison�of�proportions�over� time�for�the�same�individuals�is�conducted��Alternatively,�we�may�have�pairs�of�matched� individuals�(e�g�,�spouses,�twins,�brother-sister)�for�which�a�comparison�of�proportions�is� of�interest�
First,�the�hypotheses�to�be�evaluated�for�detecting�whether�two�dependent�population� proportions� differ� are� as� follows�� The� null� hypothesis� H0� is� that� there� is� no� difference� between�the�two�population�proportions�π1�and�π2,�which�we�denote�as
H0 1 2 0: π π− =
Here�there�is�no�difference�or�a�“null”�difference�between�the�two�population�proportions�� For� example,� a� political� analyst� may� be� interested� in� determining� whether� the� approval� rating�of�the�president�is�the�same�just�prior�to�and�immediately�following�his�annual�State� of� the� Union� address� (i�e�,� a� pretest–posttest�situation)�� As� a� second� example,� a� marriage� counselor�wants�to�know�whether�husbands�and�wives�equally�favor�a�particular�training� program�designed�to�enhance�their�relationship�(i�e�,�a�couple�situation)�
The� nondirectional,� scientific,� or� alternative� hypothesis� H1� is� that� there� is� a� difference� between�the�population�proportions�π1�and�π2,�which�we�denote�as�follows:
H1 1 2 0: π π− ≠
The�null�hypothesis�H0�will�be�rejected�here�in�favor�of�the�alternative�hypothesis�H1�if�the� population�proportions�are�different��As�we�have�not�specified�a�direction�on�H1,�we�are�will- ing�to�reject�either�if�π1�is�greater�than�π2�or�if�π1�is�less�than�π2��This�alternative�hypothesis� results�in�a�two-tailed�test��Directional�alternative�hypotheses�can�also�be�tested�if�we�believe� either�that�π1�is�greater�than�π2�or�that�π1�is�less�than�π2��The�more�the�resulting�sample�pro- portions�differ�from�one�another,�the�more�likely�we�are�to�reject�the�null�hypothesis�
Before�we�examine�the�test�statistic,�let�us�consider�a�table�in�which�the�proportions�are� often�presented��As�shown�in�Table�8�1,�the�contingency table�lists�proportions�for�each�of�
Table 8.1
Contingency�Table�for�Two�Samples
Sample 1
Sample 2 “Unfavorable” “Favorable” Marginal Proportions
“Favorable” a b p2 “Unfavorable” c d 1�−�p2
Marginal�proportions 1�−�p1 p1
216 An Introduction to Statistical Concepts
the�different�possible�outcomes��The�columns�indicate�the�proportions�for�sample�1��The�left� column�contains�those�proportions�related�to�the�“unfavorable”�condition�(or�disagree�or� no,�depending�on�the�situation),�and�the�right�column,�those�proportions�related�to�the�“favor- able”�condition�(or�agree�or�yes,�depending�on�the�situation)��At�the�bottom�of�the�columns� are� the� marginal� proportions� shown� for� the� “unfavorable”� condition,� denoted� by� 1� −� p1,� and� for� the� “favorable”� condition,� denoted� by� p1�� The� rows� indicate� the� proportions� for� sample�2��The�top�row�contains�those�proportions�for�the�“favorable”� condition,�and�the� bottom�row�contains�those�proportions�for�the�“unfavorable”�condition��To�the�right�of�the� rows�are�the�marginal�proportions�shown�for�the�“favorable”�condition,�denoted�by�p2,�and� for�the�“unfavorable”�condition,�denoted�by�1�−�p2�
Within�the�box�of�the�table�are�the�proportions�for�the�different�combinations�of�condi- tions� across� the� two� samples�� The� upper� left-hand� cell� is� the� proportion� of� observations� that�are�“unfavorable”�in�sample�1�and�“favorable”�in�sample�2�(i�e�,�dissimilar�across�sam- ples),� denoted�by�a��The�upper�right-hand�cell�is�the�proportion� of�observations� who�are� “favorable”�in�sample�1�and�“favorable”�in�sample�2�(i�e�,�similar�across�samples),�denoted� by�b��The�lower�left-hand�cell�is�the�proportion�of�observations�who�are�“unfavorable”�in� sample� 1� and� “unfavorable”� in� sample� 2� (i�e�,� similar� across� samples),� denoted� by� c�� The� lower�right-hand�cell�is�the�proportion�of�observations�who�are�“favorable”�in�sample�1�and� “unfavorable”�in�sample�2�(i�e�,�dissimilar�across�samples),�denoted�by�d�
It�is�assumed�that�the�two�samples�are�randomly�drawn�from�their�respective�popula- tions�and�that�the�normal�distribution�is�the�appropriate�sampling�distribution��The�next� step�is�to�compute�the�test�statistic�z�as
z p p s
p p d a n
p p
= −
= − +−
1 2 1 2
1 2
where�n�is�the�total�number�of�pairs��The�denominator�of�the�z�test�statistic�sp p1 2− �is�again� known�as�the�standard�error�of�the�difference�between�two�proportions�and�provides�an� index�of�how�variable�the�sample�statistic�(i�e�,�the�difference�between�two�sample�propor- tions)�is�when�multiple�samples�of�the�same�size�are�drawn��This�test�statistic�is�conceptu- ally�similar�to�the�test�statistic�for�the�dependent�t�test�
The�test�statistic�z�is�then�compared�to�a�critical�value(s)�from�the�unit�normal�distribu- tion��For�a�two-tailed�test,�the�critical�values�are�denoted�as�±α/2z�and�are�found�in�Table� A�1�� If� the� test� statistic� z� falls� into� either� critical� region,� then� we� reject� H0;� otherwise,� we� fail�to�reject�H0��For�a�one-tailed�test,�the�critical�value�is�denoted�as�+αz�for�the�alternative� hypothesis�H1:�π1�−�π2�>�0�(i�e�,�right-tailed�test)�and�as�−αz�for�the�alternative�hypothesis� H1:� π1� −� π2� <� 0� (i�e�,� left-tailed� test)�� If� the� test� statistic� z� falls� into� the� appropriate� critical� region,�then�we�reject�H0;�otherwise,�we�fail�to�reject�H0��It�should�be�noted�that�other�alter- natives�to�this�test�have�been�proposed�(e�g�,�the�chi-square�test�as�described�in�the�follow- ing�section)��Unfortunately,�the�z�test�does�not�yield�an�acceptable�CI�procedure�
Let�us�consider�an�example�to�illustrate�the�use�of�the�test�of�two�dependent�propor- tions��Suppose�a�medical�researcher�is�interested�in�whether�husbands�and�wives�agree� on� the� effectiveness� of� a� new� headache� medication� “No-Head�”� A� random� sample� of� 100�husband-wife�couples�were�selected�and�asked�to�try�“No-Head”�for�2�months��At� the�end�of�2�months,�each�individual�was�asked�whether�the�medication�was�effective� or�not�at�reducing�headache�pain��The�researcher�wants�to�know�whether�the�medica- tion�is�differentially�effective�for�husbands�and�wives��Thus,�a�nondirectional,�two-tailed� alternative�hypothesis�is�utilized�
217Inferences About Proportions
The�resulting�proportions�are�presented�as�a�contingency�table�in�Table�8�2��The�level�of� significance�is�set�at�α�=��05��The�test�statistic�z�is�computed�as�follows:
z p p s
p p d a n
p p
= −
= − +
= − +
= −
= − −
1 2 1 2
1 2
40 65 15 40
100
25 0742
3 3 (. . )
. . .
. . 6693
The�test�statistic�z�is�then�compared�to�the�critical�values�from�the�unit�normal�distribu- tion�� As� this� is� a� two-tailed� test,� the� critical� values� are� denoted� as� ±α/2z� and� are� found� in�Table�A�1�to�be�±α/2z�=�±�025z�=�±1�96��In�other�words,�this�is�the�z�value�that�is�closest� to�a�P(z)�of��975��As�the�test�statistic�z�falls�into�the�lower�tail�critical�region,�we�reject�H0� and�conclude�that�the�husbands�and�wives�do�not�believe�equally�in�the�effectiveness�of� “No-Head�”
8.2 Inferences About Proportions Involving Chi-Square Distribution
This�section�deals�with�concepts�and�procedures�for�testing�inferences�about�proportions� that� involve� the� chi-square� distribution�� Following� a� discussion� of� the� chi-square� distri- bution� relevant� to� tests� of� proportions,� inferential� tests� are� presented� for� the� chi-square� goodness-of-fit�test�and�the�chi-square�test�of�association�
8.2.1 Introduction
The�previous�tests�of�proportions�in�this�chapter�were�based�on�the�unit�normal�distri- bution,� whereas� the� tests� of� proportions� in� the� remainder� of� the� chapter� are� based� on� the�chi-square distribution��Thus,�we�need�to�become�familiar�with�this�new�distribu- tion��Like�the�normal�and�t�distributions,�the�chi-square�distribution�is�really�a�family�of� distributions�� Also,� like� the� t� distribution,� the� chi-square� distribution� family� members� depend�on�the�number�of�degrees�of�freedom�represented��As�we�shall�see,�the�degrees� of� freedom� for� the� chi-square� goodness-of-fit� test� are� calculated� as� the� number� of� cat- egories�(denoted�as�J)�minus�1��For�example,�the�chi-square�distribution�for�one�degree� of� freedom� (i�e�,� for� a� variable� which� has� two� categories)� is� denoted� by� χ1
2 � as� shown� in�
Figure� 8�1�� This� particular� chi-square� distribution� is� especially� positively� skewed� and� leptokurtic�(sharp�peak)�
Table 8.2
Contingency�Table�for�Headache�Example
Husband Sample
Wife Sample “Ineffective” “Effective” Marginal Proportions
“Effective” a�=��40 b�=��25 p2�=��65 “Ineffective” c�=��20 d�=��15 1�−�p2�=��35
Marginal�proportions 1�−�p1�=��60 p1�=��40
218 An Introduction to Statistical Concepts
The�figure�also�describes�graphically�the�distributions�for�χ5 2 �and� χ10
2 ��As�you�can�see�
in�the�figure,�as�the�degrees�of�freedom�increase,�the�distribution�becomes�less�skewed� and� less� leptokurtic;� in� fact,� the� distribution� becomes� more� nearly� normal� in� shape� as� the� number� of� degrees� of� freedom� increase�� For� extremely� large� degrees� of� freedom,� the�chi-square�distribution�is�approximately�normal��In�general,�we�denote�a�particular�
chi-square� distribution� with� ν� degrees� of� freedom� as� χ ν2�� The� mean� of� any� chi-square� distribution�is�ν,�the�mode�is�ν�−�2�when�ν�is�at�least�2,�and�the�variance�is�2ν��The�value�of� chi-square�can�range�from�0�to�positive�infinity��A�table�of�different�percentile�values�for� many�chi-square�distributions�is�given�in�Table�A�3��This�table�is�utilized�in�the�following� two�chi-square�tests�
One�additional�point�that�should�be�noted�about�each�of�the�chi-square�tests�of�propor- tions�developed�in�this�chapter�is�that�there�are�no�CI�procedures�for�either�the�chi-square� goodness-of-fit�test�or�the�chi-square�test�of�association�
8.2.2 Chi-Square Goodness-of-Fit Test
The�first�test�to�consider�is�the�chi-square goodness-of-fit test��This�test�is�used�to�determine� whether� the� observed� proportions� in� two� or� more� categories� of� a� categorical� variable� dif- fer�from�what�we�would�expect�a�priori��For�example,�a�researcher�is�interested�in�whether� the�current�undergraduate�student�body�at�ICU�is�majoring�in�disciplines�according�to�an�a� priori�or�expected�set�of�proportions��Based�on�research�at�the�national�level,�the�expected� proportions�of�undergraduate�college�majors�are�as�follows:��20�education,��40�arts�and�sci- ences,��10�communications,�and��30�business��In�a�random�sample�of�100�undergraduates�at� ICU,�the�observed�proportions�are�as�follows:��25�education,��50�arts�and�sciences,��10�com- munications,�and��15�business��Thus,�the�researcher�would�like�to�know�whether�the�sample� proportions�observed�at�ICU�fit�the�expected�national�proportions��In�essence,�the�chi-square� goodness-of-fit�test�is�used�to�test�proportions�for�a�single�categorical�variable�(i�e�,�nominal� or�ordinal�measurement�scale)�
The� observed proportions� are� denoted� by� pj,� where� p� represents� a� sample� proportion� and�j�represents�a�particular�category�(e�g�,�education�majors),�where�j�=�1,�…,�J�categories�� The�expected proportions�are�denoted�by�πj,�where�π�represents�an�expected�proportion�
FIGuRe 8.1 Several� members� of� the� family� of� the� chi- square�distribution�
0.3
0.2
0.1Re la
tiv e
fr eq
ue nc
y 0
0 5 10 Chi-square
15 20 25
1 5 10
219Inferences About Proportions
and� j� represents� a� particular� category�� The� null� and� alternative� hypotheses� are� denoted� as�follows,�where�the�null�hypothesis�states�that�the�difference�between�the�observed�and� expected�proportions�is�0�for�all�categories:
H p for all jj j0 0: ( )− =π
H p for all jj j1 0: ( )− ≠π
The�test�statistic�is�a�chi-square�and�is�computed�by
χ π
π 2
2
1
= −
= ∑n pj j
jj
J ( )
where�n�is�the�size�of�the�sample��The�test�statistic�is�compared�to�a�critical�value�from�the� chi-square�table�(Table�A�3)�α νχ
2 ,�where�ν�=�J�−�1��The�degrees�of�freedom�are�1�less�than�the�
total�number�of�categories�J,�because�the�proportions�must�total�to�1�00;�thus,�only�J�−�1�are� free�to�vary�
If� the� test� statistic� is� larger� than� the� critical� value,� then� the� null� hypothesis� is� rejected� in� favor� of� the� alternative�� This� would� indicate� that� the� observed� and� expected� propor- tions�were�not�equal�for�all�categories��The�larger�the�differences�are�between�one�or�more� observed�and�expected�proportions,�the�larger�the�value�of�the�test�statistic,�and�the�more� likely�it�is�to�reject�the�null�hypothesis��Otherwise,�we�would�fail�to�reject�the�null�hypoth- esis,�indicating�that�the�observed�and�expected�proportions�were�approximately�equal�for� all�categories�
If�the�null�hypothesis�is�rejected,�one�may�wish�to�determine�which�sample�proportions� are� different� from� their� respective� expected� proportions�� Here� we� recommend� you� con- duct�tests�of�a�single�proportion�as�described�in�the�preceding�section��If�you�would�like�to� control�the�experimentwise�Type�I�error�rate�across�a�set�of�such�tests,�then�the�Bonferroni� method�is�recommended�where�the�α�level�is�divided�up�among�the�number�of�tests�con- ducted��For�example,�with�an�overall�α�=��05�and�five�categories,�one�would�conduct�five� tests�of�a�single�proportion,�each�at�the��01�level�of�α�
Another� way� to� determine� which� cells� are� statistically� different� in� observed� to� expected�proportions�is�to�examine�the�standardized�residuals�which�can�be�computed� as�follows:
R O E
E =
−
Standardized�residuals�that�are�greater�(in�absolute�value�terms)�than�1�96�(when�α�=��05)�or� 2�58�(when�α�=��01)�have�different�observed�to�expected�frequencies�and�are�contributing�to� the�statistically�significant�chi-square�statistic��The�sign�of�the�residual�provides�informa- tion�on�whether�the�observed�frequency�is�greater�than�the�expected�frequency�(i�e�,�posi- tive�value)�or�less�than�the�expected�frequency�(i�e�,�negative�value)�
220 An Introduction to Statistical Concepts
Let� us� return� to� the� example� and� conduct� the� chi-square� goodness-of-fit� test�� The� test� statistic�is�computed�as�follows:
�
χ π
π 2
2
1
= −
= ∑n pj j
jj
J ( )
�
= −
+ −
+ −
+ −
100 25 20
20 50 40
40 10 10
10 15 30
3
2 2 2 2(. . ) .
(. . ) .
(. . ) .
(. . ) . 00
1
4
= ∑
j
�
= + + + = = =
∑100 0125 0250 0000 0750 100 1125 11 25 1
4
(. . . . ) (. ) . j
The� test� statistic� is� compared� to� the� critical� value,� from� Table� A�3,� of� .05 3 2χ � =� 7�8147��
Because� the� test� statistic� is� larger� than� the� critical� value,� we� reject� the� null� hypothesis� and�conclude�that�the�sample�proportions�from�ICU�are�different�from�the�expected�pro- portions�at�the�national�level��Follow-up�tests�to�determine�which�cells�are�statistically� different�in�their�observed�to�expected�proportions�involve�examining�the�standardized� residuals��In�this�example,�the�standardized�residuals�are�computed�as�follows:
R O E
E
R
Education
Arts and sciences
= −
= −
=
= −
=
25 20 20
1 118
50 40 40
1 58
.
. 11
10 10 10
0
15 30 30
2 739
R
R
Communication
Bu ess
= −
=
= −
= −sin .
The�standardized�residual�for�business�is�greater�(in�absolute�value�terms)�than�1�96�(as�α�=� �05)�and�thus�suggests�that�there�are�different�observed�to�expected�frequencies�for�students� majoring�in�business�at�ICU�compared�to�national�estimates,�and�that�this�category�is�the� one�which�is�contributing�most�to�the�statistically�significant�chi-square�statistic�
8.2.2.1 Effect Size
An�effect�size�for�the�chi-square�goodness-of-fit�test�can�be�computed�by�hand�as�follows,� where�N�is�the�total�sample�size�and�J�is�the�number�of�categories�in�the�variable:
Effect size N J
= −
χ2
1( )
This�effect�size�statistic�can�range�from�0�to�+1,�where�0�indicates�no�difference�between� the�sample�and�hypothesized�proportions�(and�thus�no�effect)��Positive�one�indicates�the�
221Inferences About Proportions
maximum� difference� between� the� sample� and� hypothesized� proportions� (and� thus� a� large�effect)��Given�the�range�of�this�value�(0�to�+1�0)�and�similarity�to�a�correlation�coeffi- cient,�it�is�reasonable�to�apply�Cohen’s�interpretations�for�correlations�as�a�rule�of�thumb�� These�include�the�following:�small�effect�size�=��10;�medium�effect�size�=��30;�and�large� effect�size�=��50��For�the�previous�example,�the�effect�size�would�be�calculated�as�follows� and�would�be�interpreted�as�a�small�effect:
Effectsize N J
= −
= −
= = χ2
1 11 25
100 4 1 11 25 300
0375 ( )
. ( )
. .
8.2.2.2 Assumptions
Two� assumptions� are� made� for� the� chi-square� goodness-of-fit� test:� (1)� observations� are� independent�(which�is�met�when�a�random�sample�of�the�population�is�selected),�and� (2)�expected�frequency�is�at�least�5�per�cell�(and�in�the�case�of�the�chi-square�goodness-of- fit�test,�this�translates�to�an�expected�frequency�of�at�least�5�per�category�as�there�is�only� one� variable� included� in� the� analysis)�� When� the� expected� frequency� is� less� than� 5,� that� particular� cell� (i�e�,� category)� has� undue� influence� on� the� chi-square� statistic�� In� other� words,�the�chi-square�goodness-of-fit�test�becomes�too�sensitive�when�the�expected�values� are�less�than�5�
8.2.3 Chi-Square Test of association
The�second�test�to�consider�is�the�chi-square test of association��This�test�is�equivalent� to�the�chi-square�test�of�independence�and�the�chi-square�test�of�homogeneity,�which� are�not�further�discussed��The�chi-square�test�of�association�incorporates�both�of�these� tests�(e�g�,�Glass�&�Hopkins,�1996)��The�chi-square�test�of�association�is�used�to�deter- mine�whether�there�is�an�association�or�relationship�between�two�or�more�categorical� (i�e�,� nominal� or� ordinal)� variables�� Our� discussion� is,� for� the� most� part,� restricted� to� the� two-variable� situation� where� each� variable� has� two� or� more� categories�� The� chi- square�test�of�association�is�the�logical�extension�to�the�chi-square�goodness-of-fit�test,� which�is�concerned�with�one�categorical�variable��Unlike�the�chi-square�goodness-of- fit� test� where� the� expected� proportions� are� known� a� priori,� for� the� chi-square� test� of� association,� the� expected� proportions� are� not� known� a� priori� but� must� be� estimated� from�the�sample�data�
For� example,� suppose� a� researcher� is� interested� in� whether� there� is� an� association� between� level� of� education� and� stance� on� a� proposed� amendment� to� legalize� gambling�� Thus,� one� categorical� variable� is� level� of� education� with� the� categories� being� as� follows:� (1)�less�than�a�high�school�education,�(2)�high�school�graduate,�(3)�undergraduate�degree,� and�(4)�graduate�school�degree��The�other�categorical� variable� is�stance�on�the�gambling� amendment�with�the�following�categories:�(1)�in�favor�of�the�gambling�bill�and�(2)�opposed� to�the�gambling� bill��The�null�hypothesis� is� that�there�is� no�association�between�level� of� education�and�stance�on�gambling,�whereas�the�alternative�hypothesis�is�that�there�is�some� association�between�level�of�education�and�stance�on�gambling��The�alternative�would�be� supported�if�individuals�at�one�level�of�education�felt�differently�about�the�bill�than�indi- viduals�at�another�level�of�education�
The�data�are�shown�in�Table�8�3,�known�as�a�contingency table�(or�crosstab�table)��As� there�are�two�categorical�variables,�we�have�a�two-way�or�two-dimensional�contingency�
222 An Introduction to Statistical Concepts
table��Each�combination�of�the�two�variables�is�known�as�a�cell��For�example,�the�cell� for�row�1,�favor�bill,�and�column�2,�high�school�graduate,�is�denoted�as�cell 12,�the�first� value� (i�e�,� 1)� referring� to� the� row� and� the� second� value� (i�e�,� 2)� to� the� column�� Thus,� the� first� subscript� indicates� the� particular� row� r,� and� the� second� subscript� indicates� the� particular� column� c�� The� row� subscript� ranges� from� r� =� 1,…,� R,� and� the� column� subscript�ranges�from�c�=�1,…,�C,�where�R�is�the�last�row�and�C�is�the�last�column��This� example�contains�a�total�of�eight�cells,�two�rows�times�four�columns,�denoted�by�R�×� C�=�2�×�4�=�8�
Each� cell� in� the� table� contains� two� pieces� of� information,� the� number� (or� count� or� frequencies)�of�observations�in�that�cell�and�the�observed�proportion�in�that�cell��For� cell 12,� there� are� 13� observations� denoted� by� n12� =� 13� and� an� observed� proportion� of� �65�denoted�by�p12�=��65��The�observed�proportion�is�computed�by�taking�the�number� of�observations�in�the�cell�and�dividing�by�the�number�of�observations�in�the�column�� Thus,�for�cell 12,�13�of�the�20�high�school�graduates�favor�the�bill,�or�13/20�=��65��The�col- umn�information�is�given�at�the�bottom�of�each�column,�known�as�the�column margin- als��Here�we�are�given�the�number�of�observations�in�a�column,�denoted�by�n�c,�where� the�“�”�indicates�we�have�summed�across�rows�and�c�indicates�the�particular�column�� For�column�2�(reflecting�high�school�graduates),�there�are�20�observations�denoted�by� n�2�=�20�
There�is�also�row�information�contained�at�the�end�of�each�row,�known�as�the�row mar- ginals�� Two� values� are� listed� in� the� row� marginals�� First,� the� number� of� observations� in� a�row�is�denoted�by�nr�,�where�r�indicates�the�particular�row�and�the�“�”�indicates�we�have� summed�across�the�columns��Second,�the�expected�proportion�for�a�specific�row�is�denoted� by�πr�,�where�again� r�indicates� the�particular�row�and�the�“�”�indicates� we�have�summed� across�the�columns��The�expected�proportion�for�a�particular�row�is�computed�by�taking� the�number�of�observations�in�that�row�nr��and�dividing�by�the�number�of�total�observa- tions�n�����Note�that�the�total�number�of�observations�is�given�in�the�lower�right-hand�por- tion�of�the�figure�and�denoted�as�n���=�80��Thus,�for�the�first�row,�the�expected�proportion�is� computed�as�π1��=�n1�/n���=�44/80�=��55�
The�null�and�alternative�hypotheses�can�be�written�as�follows:
H p for all cellsrc r0 0: ( ).− =π
H p for all cellsrc r1 0: ( ).− ≠π
Table 8.3
Contingency�Table�for�Gambling�Example
Level of Education
Stance on Gambling
Less than High School High School Undergraduate Graduate Row Marginals
“Favor” n11�=�16 n12�=�13 n13�=�10 n14�=�5 n1��=�44 p11�=��80 p12�=��65 p13�=��50 p14�=��25 π1��=��55
“Opposed” n21�=�4 n22�=�7 n23�=�10 n24�=�15 n2��=�36 p21�=��20 p22�=��35 p23�=��50 p24�=��75 π2��=��45
Column�marginals n�1�=�20 n�2�=�20 n�3�=�20 n�4�=�20 n���=�80
223Inferences About Proportions
The�test�statistic�is�a�chi-square�and�is�computed�by
χ π
π 2
1
2
1
= −
== ∑∑ n pc c
C rc r
rr
R
. .
.
( )
The�test�statistic�is�compared�to�a�critical�value�from�the�chi-square�table�(Table�A�3)� α χν 2 ,�
where�ν�=�(R�−�1)(C�−�1)��That�is,�the�degrees�of�freedom�are�1�less�than�the�number�of�rows� times�1�less�than�the�number�of�columns�
If�the�test�statistic�is�larger�than�the�critical�value,�then�the�null�hypothesis�is�rejected�in� favor�of�the�alternative��This�would�indicate�that�the�observed�and�expected�proportions� were�not�equal�across�cells�such�that�the�two�categorical�variables�have�some�association�� The�larger�the�differences�between�the�observed�and�expected�proportions,�the�larger�the� value�of�the�test�statistic�and�the�more�likely�it�is�to�reject�the�null�hypothesis��Otherwise,� we� would� fail� to� reject� the� null� hypothesis,� indicating� that� the� observed� and� expected� proportions� were� approximately� equal,� such� that� the� two� categorical� variables� have� no� association�
If�the�null�hypothesis�is�rejected,�then�one�may�wish�to�determine�for�which�combina- tion�of�categories�the�sample�proportions�are�different�from�their�respective�expected� proportions��Here�we�recommend�you�construct�2�×�2�contingency�tables�as�subsets�of� the�larger�table�and�conduct�chi-square�tests�of�association��If�you�would�like�to�con- trol� the� experimentwise� Type� I� error� rate� across� the� set� of� tests,� then� the� Bonferroni� method� is� recommended� where� the� α� level� is� divided� up� among� the� number� of� tests� conducted�� For� example,� with� α� =� �05� and� five� 2� ×� 2� tables,� one� would� conduct� five� tests� each� at� the� �01� level� of� α�� As� with� the� chi-square� goodness-of-fit� test,� it� is� also� possible� to� examine� the� standardized� residuals� (which� can� be� requested� in� SPSS)� to� determine�the�cells�that�have�significantly�different�observed�to�expected�proportions�� Cells�where�the�standardized�residuals�are�greater�(in�absolute�value�terms)�than�1�96� (when�α�=��05)�or�2�58�(when�α�=��01)�are�significantly�different�in�observed�to�expected� frequencies�
Finally,�it�should�be�noted�that�we�have�only�considered�two-way�contingency�tables�here�� Multiway�contingency�tables�can�also�be�constructed�and�the�chi-square�test�of�association� utilized�to�determine�whether�there�is�an�association�among�several�categorical�variables�
Let�us�complete�the�analysis�of�the�example�data��The�test�statistic�is�computed�as
� χ
π π
2
1
2
1
= −
== ∑∑ n pc c
C rc r
rr
R
. .
.
( )
� =
− +
− +
− +
− 20
80 55 55
20 20 45
45 20
65 55 55
20 35 42 2 2(. . )
. (. . )
. (. . )
. (. . 55
45
2) .
� +
− +
− +
− +
− 20
50 55 55
20 50 45
45 20
25 55 55
20 75 42 2 2(. . )
. (. . )
. (. . )
. (. . 55
45
2) .
� = + + + + + + + =2 2727 2 7778 3636 4444 0909 1111 3 2727 4 0000 13 33. . . . . . . . . 332
224 An Introduction to Statistical Concepts
The�test�statistic�is�compared�to�the�critical�value,�from�Table�A�3,�of�.05 3 2χ �=�7�8147��Because�the�
test�statistic�is�larger�than�the�critical�value,�we�reject�the�null�hypothesis�and�conclude� that� there� is� an� association� between� level� of� education� and� stance� on� the� gambling� bill��In�other�words,�stance�on�gambling�is�not�the�same�for�all�levels�of�education��The� cells�with�the�largest�contribution�to�the�test�statistic�give�some�indication�as�to�where� the� observed� and� expected� proportions� are� most� different�� Here� the� first� and� fourth� columns�have�the�largest�contributions�to�the�test�statistic�and�have�the�greatest�differ- ences�between�the�observed�and�expected�proportions;�these�would�be�of�interest�in�a� 2��2�follow-up�test�
8.2.3.1 Effect Size
Several�measures�of�effect�size,�such�as�correlation�coefficients�and�measures�of�association,� can� be� requested� in� SPSS� and� are� commonly� reported� effect� size� indices� for� results� from� chi-square� test� of� association�� Which� effect� size� value� is� selected� depends� in� part� on� the� measurement�scale�of�the�variable��For�example,�researchers�working�with�nominal�data�can� select�a�contingency�coefficient,�phi�(for�2�×�2�tables),�Cramer’s�V�(for�tables�larger�than�2�×�2),� lambda,�or�an�uncertainty�coefficient��Correlation�options�available�for�ordinal�data�include� gamma,�Somer’s�d,�Kendall’s�tau-b,�and�Kendall’s�tau-c��From�the�contingency�coefficient,�C,� we�can�compute�Cohen’s�w�as�follows:
w C C
= −
2
21
Cohen’s�recommended�subjective�standard�for�interpreting�w�(as�well�as�the�other�correla- tion�coefficients�presented)�is�as�follows:�small�effect�size,�w�=��10;�medium�effect�size,�w�=��30;� and�large�effect�size,�w�=��50��See�Cohen�(1988)�for�further�details�
8.2.3.2 Assumptions
The� same� two� assumptions� that� apply� to� the� chi-square� goodness-of-fit� test� also� apply� to� the� chi-square� test� of� association,� as� follows:� (1)� observations� are� independent� (which� is� met� when� a� random� sample� of� the� population� is� selected),� and� (2)� expected� frequency� is�at�least�5�per�cell��When�the�expected�frequency�is�less�than�5,�that�particular�cell�has� undue�influence�on�the�chi-square�statistic��In�other�words,�the�chi-square�test�of�association� becomes�too�sensitive�when�the�expected�values�are�less�than�5�
8.3 SPSS
Once� again,� we� consider� the� use� of� SPSS� for� the� example� datasets�� While� SPSS� does� not� have�any�of�the�z�procedures�described�in�the�first�part�of�this�chapter,�it�is�capable�of�con- ducting�both�of�the�chi-square�procedures�covered�in�the�second�part�of�this�chapter,�as� described�in�the�following�
225Inferences About Proportions
Chi-Square Goodness-of-Fit Test
Step 1: To� conduct� the� chi-square� goodness-of-fit� test,� you� need� one� variable� that� is� either�nominal�or�ordinal�in�scale�(e�g�,�major)��To�conduct�the�chi-square�goodness-of-fit� test,�go�to�“Analyze”�in�the�top�pulldown�menu,�then�select�“Nonparametric Tests,” followed� by�“Legacy Dialogs,”� and� then�“Chi-Square.”� Following� the� screenshot� (step�1)�as�follows�produces�the�“Chi-Square Goodness-of-Fit”�dialog�box�
A
B
C D
Chi-square goodness-of-fit:
Step 1
Step 2:� Next,� from� the� main�“Chi-Square Goodness-of-Fit”� dialog� box,� click� the� variable�(e�g�,�major)�and�move�it�into�the�“Test Variable List”�box�by�clicking�on�the� arrow�button��In�the�lower�right-hand�portion�of�the�screen�is�a�section�labeled�“Expected Values.”� The� default� is� to� conduct� the� analysis� with� the� expected� values� equal� for� each� category� (you� will� see� that� the� radio� button� for� “All categories equal”� is� prese- lected)��Much�of�the�time,�you�will�want�to�use�different�expected�values��To�define�different� expected�values,�click�on�the�“Values”�radio�button��Enter�each�expected�value�in�the�box� below�“Values,”�in�the�same�order�as�the�categories�(e�g�,�first�enter�the�expected�value�for� category�1�and�then�the�expected�value�for�category�2),�and�then�click�“Add”�to�define�the� value�in�the�box��This�sets�up�an�expected�value�for�each�category��Repeat�this�process�for� every�category�of�your�variable�
226 An Introduction to Statistical Concepts
Chi-square goodness-of-fit:
Step 2a
Enter the expected value for the category that
corresponds to the first numeric value
(e.g., 1).
Click on “Add” to define the
value expected in each group.
Repeat this for each category.
Chi-square goodness-of-fit:
Step 2b
The expected values will appear in rank order from the first category
to the last category.
227Inferences About Proportions
Then�click�on�“OK”�to�run�the�analysis��The�output�is�shown�in�Table�8�4�
Table 8.4
SPSS�Results�for�Undergradute�Majors�Example
Observed N reflects the observed frequencies from your sample.
Expected N reflects the expected values that were input by the researcher.
The residual is simply the difference between the observed and expected frequencies.
“Asymp. sig.” is the observed p value for the chi-square
goodness-of-fit test. It is interpreted as: there is about
a 1% probability of the sample proportions occurring by chance if the null hypothesis is really true (i.e., if the population proportions
are 20, 40, 10, and 30).
df are the degrees of freedom.
For the chi-square goodness-of-fit test, they are calculated
as J – 1 (i.e., one less than the
number of categories).
College Major
Chi-Square Test
Frequencies
Education 5.0 10.0
.0 –15.0
20.0 40.0 10.0 30.0
25 50 10 15
100
Arts and sciences Communications Business Total
College Major
Observed N Expected N Residual
Chi-square df Asymp. sig. .010 a 0 cells (.0%) have expected frequencies less than 5. The minimum expected cell frequency is 10.0.
3 11.250a
Test Statistics
“Chi-square” is the test statistic value and is calculated as:
= 100 j=1
j=1
J
4 (.25 – .20)2
.20 .40 .10 .30 +++ = 11.25
(.50 – .40)2
(pj – πj)2
(.10 – .10)2 (.15 – .30)2 Σ
Σ2 = n πj
Interpreting the output:� The� top� table� provides� the� frequencies� observed� in� the� sample� (“Observed N”)� and� the� expected� frequencies� given� the� values� defined� by� the� researcher�(“Expected N”)��The�“Residual”�is�simply�the�difference�between�the�two� Ns��The�chi-square�test�statistic�value�is�11�25,�and�the�associated�p�value�is��01��Since�p�is� less�than�α,�we�reject�the�null�hypothesis��Let�us�translate�this�back�to�the�purpose�of�our� null� hypothesis� statistical� test�� There� is� evidence� to� suggest� that� the� sample� proportions� observed�differ�from�the�proportions�of�college�majors�nationally��Follow-up�tests�to�deter- mine�which�cells�are�statistically�different�in�the�observed�to�expected�proportions�can�be�
228 An Introduction to Statistical Concepts
conducted� by� examining� the� standardized� residuals�� In� this� example,� the� standardized� residuals�were�computed�previously�as�follows:
R O E
E
R
Education
Arts and sciences
= −
= −
=
= −
=
25 20 20
1 118
50 40 40
1 58
.
. 11
10 10 10
0
15 30 30
2 739
R
R
Communication
Bu ess
= −
=
= −
= −sin .
The�standardized�residual�for�business�is�greater�(in�absolute�value�terms)�than�1�96�(given� α� =� �05)� and� thus� suggests� that� there� are� different� observed� to� expected� frequencies� for� students�majoring�in�business�at�ICU�compared�to�national�estimates��This�category�is�the� one�contributing�most�to�the�statistically�significant�chi-square�statistic�
The�effect�size�can�be�calculated�as�follows:
Effect size N J
= −
= −
= = χ2
1 11 25
100 4 1 11 25 300
0375 ( )
. ( )
. .
Chi-Square Test of Association
Step 1:�To�conduct�a�chi-square�test�of�association,�you�need�two�categorical�variables�(nomi- nal�and/or�ordinal)�whose�frequencies�you�wish�to�associate�(e�g�,�education�level�and�gambling� stance)��To�compute�the�chi-square�test�of�association,�go�to�“Analyze”�in�the�top�pulldown,� then�select�“Descriptive Statistics,”�and�then�select�the�“Crosstabs”�procedure�
A
B
C
Chi-square test of association:
Step 1
229Inferences About Proportions
Step 2:�Select�the�dependent�variable�and�move�it�into�the�“Row(s)”�box�by�clicking�on� the�arrow�key�[e�g�,�here�we�used�stance�on�gambling�as�the�dependent�variable�(1�=�support;� 0�=�not�support)]��Then�select�the�independent�variable�and�move�it�into�the�“Column(s)”� box� [in� this� example,� level� of� education� is� the� independent� variable� (1� =� less� than� high� school;�2�=�high�school;�3�=�undergraduate;�4�=�graduate)]�
Select the variable of interest from
the list on the left and use the arrow
to move to the boxes on the right.
Clicking on “Cells” provides options for what statistics
to display in the cells. See step 4.
Clicking on “Format” allows the option of displaying
the categories in ascending or descending
order. See step 5.
Chi-square test of association:
Step 2
Clicking on “Statistics” will allow you to select
various statistics to generate (including the chi-square test statistics
value and various correlation coefficients).
See step 3.
The dependent variable should be
displayed in the row(s) and the independent
variable in the column(s).
Step 3:�In�the�top�right�corner�of�the�“Crosstabs”�dialog�box�(see�screenshot�step�2),� click� on� the� button� labeled�“Statistics.”� From� here,� placing� a� checkmark� in� the� box� for� “Chi-square”� will� produce� the� chi-square� test� statistic� value� and� resulting� null� hypothesis� statistical� test� results� (including� degrees� of� freedom� and� p� value)�� Also� from� “Statistics,”�you�can�select�various�measures�of�association�that�can�serve�as�an�effect� size� (i�e�,� correlation� coefficient� values)�� Which� correlation� is� selected� should� depend� on� the� measurement� scales� of� your� variables�� We� are� working� with� two� nominal� variables;� thus,�for�purposes�of�this�illustration,�we�will�select�both�“Phi and Cramer’s V”�and� “Contingency coefficient”�just�to�illustrate�two�different�effect�size�indices�(although� it�is�standard�practice�to�use�and�report�only�one�effect�size)��We�will�use�the�contingency� coefficient�to�compute�Cohen’s�w��Click�on�“Continue”�to�return�to�the�main�“Crosstabs”� dialog�box�
230 An Introduction to Statistical Concepts
Chi-square test of association:
Step 3
Step 4:�In�the�top�right�corner�of�the�“Crosstabs”�dialog�box�(see�screenshot�step�2),� click�on�the�button�labeled�“Cells.”�From�the “Cells”�dialog�box,�options�are�available� for�selecting�counts�and�percentages��We�have�requested�“Observed”�and�“Expected”� counts,� “Column”� percentages,� and� “Standardized”� residuals�� We� will� review� the� expected� counts� to� determine� if� the� assumption� of� five� expected� frequencies� per� cell� is� met��We�will�use�the�standardized�residuals�post�hoc�if�the�results�of�the�test�are�statisti- cally�significant�to�determine�which�cell(s)�is�most�influencing�the�chi-square�value��Click� “Continue”�to�return�to�the�main�“Crosstabs”�dialog�box�
Ch-square test of association:
Step 4
231Inferences About Proportions
Step 5:� In� the� top� right� corner� of� the� “Crosstabs”� dialog� box� (see� screenshot� step�2),�click�on�the�button�labeled�“Format.”�From�the�“Format”�dialog�box,�options� are� available� for� determining� which� order,� “Ascending”� or� “Descending,”� you� want� the� row� values� presented� in� the� contingency� table� (we� asked� for� descending� in� this� example,� such� that� row� 1� was� gambling� =� 1� and� row� 2� was� gambling� =� 0)�� Click� “Continue”� to� return� to� the� main�“Crosstabs”� dialog� box�� Then� click� on�“OK”� to� run�the�analysis�
Chi-square test of association:
Step 5
Interpreting the Output:�The�output�appears�in�Table�8�5,�where�the�top�box�(“Case Processing Summary”)�provides�information�on�the�sample�size�and�frequency�of�miss- ing� data� (if� any)�� The�“Crosstabulation”� table� is� next� and� provides� the� contingency� table� (i�e�,� counts,� percentages,� and� standardized� residuals)�� The�“Chi-Square Tests”� box� gives� the� results� of� the� procedure� (including� chi-square� test� statistic� value� labeled� “Pearson Chi-Square,”�degrees�of�freedom,�and�p�value�labeled�as�“Asymp. Sig.”)�� The� likelihood� ratio� chi-square� uses� a� different� mathematical� formula� than� the� Pearson� chi-square;�however�for�large�sample�sizes,�the�values�for�the�likelihood�ratio�chi-square� and�the�Pearson�chi-square�should�be�similar�(and�rarely�should�the�two�statistics�suggest� different�conclusions�in�terms�of�rejecting�or�failing�to�reject�the�null�hypothesis)��The�lin- ear�by�linear�association�statistic,�also�known�as�the�Mantel-Haenszel�chi-square,�is�based� on�the�Pearson�correlation�and�tests�whether�there�is�a�linear�association�between�the�two� variables�(and�thus�should�not�be�used�for�nominal�variables)�
For�the�contingency�coefficient,�C,�of��378,�we�compute�Cohen’s�w�effect�size�as�follows:�
w C C
= −
= −
= −
= = 2
2
2
21 378
1 378 143
1 143 167 408
. .
. .
. .
Cohen’s�w�of��408�would�be�interpreted�as�a�moderate�to�large�effect��Cramer’s�V,�as�seen�in� the�output,�is��408�and�would�be�interpreted�similarly—a�moderate�to�large�effect�
8.4 G*Power
A�priori�power�can�be�determined�using�specialized�software�(e�g�,�Power�and�Precision,� Ex-Sample,�G*Power)�or�power�tables�(e�g�,�Cohen,�1988),�as�previously�described��However,� since� SPSS� does� not� provide� power� information� for� the� results� of� the� chi-square� test� of� association�just�conducted,�let�us�use�G*Power�to�compute�the�post�hoc�power�of�our�test�
232 An Introduction to Statistical Concepts
Table 8.5
SPSS�Results�for�Gambling�Example
Review the standardized residuals to determine which cell(s) are
contributing to the statistically significant chi-square value.
Standardized residuals greater than an absolute value of 1.96 (critical
value when alpha=.05) indicate that cell is contributing to the association
between the variables. In this case, only one cell,
graduate/do not support, has a standardized residual of 2.0 and thus
is contributing to the relationship.
When analyzing the percentages in the crosstab
table, compare the categories of the dependent
variable (rows) across the columns of the independent
variable (columns). For example, of respondents with a high school diploma,
65% support gambling
Zero cells have expected counts less than five, thus we have met this assumption of the
chi-square test of association.
Degrees of freedom are computed as: (Rows–1)(Columns – 1) = (2 – 1)(4 – 1) = 3
The probability is less than 1% (see “Asymp. sig.”) that we would see these proportions by random chance if the proportions were all equal
(i.e., if the null hypothesis were really true).
We have a 2 × 4 table thus Cramer’s V is appropriate. It is statistically significant,
and using Cohen’s interpretations, reflects a moderate to large effect size.
“Pearson Chi-square” is the test statistic value and is calculated as (see Section 8.2.3 for the full computation):
2 = n.c πr.
R
r=1 Σ
C (prc–πr.) 2
c=1 Σ
Observed and expected counts
The contingency coefficient can be used to compute Cohen’s w, a measure of effect size as follows:
w = = = .408 C 2 .3782
1–.37821–C 2
Stance on Gambling * Level of Education Crosstabulation
Chi-Square Tests
Symmetric Measures
Cases Valid Missing Total
N N NPercent Percent Percent 100.080.00100.080Stance on gambling*
Level of education
Level of Education Less Than
High School High School Undergraduate Graduate Total Stance on gambling Support
Do not support
Count
% Within level of education Std. residual Count Expected count % Within level of education Std. residual Count Expected count % Within level of education
Pearson chi-square Likelihood ratio Linear-by-linear association N of valid cases
16 11.0
80.0%
1.5 4
9.0 20.0%
–1.7 20
20.0 100.0%
20 –.7
35.0% 9.0
7 .6
65.0% 11.0
13
20.0 100.0%
20 2.0
75.0% 9.0 15
–1.8
25.0% 11.0
5
20.0 100.0%
45.0% 36.0
36
55.0% 44.0
44
80 80.0
100.0%
20 20.0
100.0%
10 11.0
50.0%
–.3 10 9.0
50.0%
.3
Value 13.333a 13.969 12.927
80 a0 cells (.0%) have expected co unt less than 5. The minimum expected count is 9.00.
3 3 1 .000
Value .408 .408 .378
80
Nominal by nominal
N of valid cases .004 .004 .004
Approx. Sig.
.003
.004 df
Asymp. Sig. (2-Sided)
Phi Cramer’s V Contingency coefficient
Case Processing Summary
Expected count
233Inferences About Proportions
Post Hoc Power for the Chi-Square Test of Association Using G*Power
The�first�thing�that�must�be�done�when�using�G*Power�for�computing�post�hoc�power�is� to� select� the� correct� test� family�� In� our� case,� we� conducted� a� chi-square� test� of� associa- tion;� therefore,� the� toggle� button� must� be� used� to� change� the� test� family� to� χ2�� Next,� we� need�to�select� the�appropriate� statistical� test��We� toggle�to�“Goodness-of-fit tests: Contingency tables.”�The�“Type of power analysis”�desired�then�needs�to�be� selected��To�compute�post�hoc�power,�we�need�to�select�“Post hoc: Compute achieved power—given α, sample size, and effect size.”
The� “Input Parameters”� must� then� be� specified�� The� first� parameter� is� speci- fication� of� the� effect� size� w� (this� was� computed� by� hand� from� the� contingency� coef- ficient� and� w� =� �408)�� The� alpha� level� we� tested� at� was� �05,� the� sample� size� was� 80,� and�the�degrees�of�freedom�were�3��Once�the�parameters�are�specified,�simply�click�on� “Calculate”�to�generate�the�achieved�power�statistics�
The�“Output Parameters”�provide�the�relevant�statistics�given�the�input�just�speci- fied��In�this�example,�we�were�interested�in�determining�post�hoc�power�given�a�two-tailed� test,�with�an�observed�effect�size�of��408,�an�alpha�level�of��05,�and�sample�size�of�80��Based� on�those�criteria,�the�post�hoc�power�was��88��In�other�words,�with�a�sample�size�of�80,�test- ing�at�an�alpha�level�of��05�and�observing�a�moderate�to�large�effect�of��408,�then�the�power� of�our�test�was��88—the�probability�of�rejecting�the�null�hypothesis�when�it�is�really�false� was�88%,�which�is�very�high�power��Keep�in�mind�that�conducting�power�analysis�a�priori� is�recommended�so�that�you�avoid�a�situation�where,�post�hoc,�you�find�that�the�sample� size�was�not�sufficient�to�reach�the�desired�level�of�power�(given�the�observed�effect�size� and�alpha�level)�
Once the parameters are
specified, click on “Calculate.”�e “Input Parameters” for computing
post hoc power must be specified including:
Chi-square test of association
1. Observed effect size w 2. Alpha level 3. Total sample size 4. Degrees of freedom
234 An Introduction to Statistical Concepts
8.5 Template and APA-Style Write-Up
We�finish�the�chapter�by�presenting�templates�and�APA-style�write-ups�for�our�examples�� First�we�present�an�example�paragraph�detailing�the�results�of�the�chi-square�goodness-of- fit�test�and�then�follow�this�by�the�chi-square�test�of�association�
Chi-Square Goodness-of-Fit Test
Recall�that�our�graduate�research�assistant,�Marie,�was�working�with�Tami,�a�staff�member�in� the�Undergraduate�Services�Office�at�ICU,�to�assist�in�analyzing�the�proportions�of�students� enrolled� in� undergraduate� majors�� Her� task� was� to� assist� Tami� with� writing� her� research� question�(Are the sample proportions of undergraduate student college majors at ICU in the same proportions as those nationally?)�and�generating�the�statistical�test�of�inference�to�answer�her� question��Marie�suggested�a�chi-square�goodness-of-fit�test�as�the�test�of�inference��A�tem- plate� for� writing� a� research� question� for� a� chi-square� goodness-of-fit� test� is� presented� as� follows:
Are the sample proportions of [units in categories] in the same pro- portions of those [identify the source to which the comparison is being made]?
It�may�be�helpful�to�include�in�the�results�of�the�chi-square�goodness-of-fit�test�information� on�an�examination�of�the�extent�to�which�the�assumptions�were�met�(recall�there�are�two� assumptions:�independence�and�expected�frequency�of�at�least�5�per�cell)��This�assists�the� reader�in�understanding�that�you�were�thorough�in�data�screening�prior�to�conducting�the� test�of�inference�
A chi-square goodness-of-fit test was conducted to determine if the sample proportions of undergraduate student college majors at ICU were in the same proportions of those reported nationally. The test was conducted using an alpha of .05. The null hypothesis was that the proportions would be as follows: .20 education, .40 arts and sciences, .10 communications, and .30 business. The assumption of an expected frequency of at least 5 per cell was met. The assumption of indepen- dence was met via random selection.
As shown in Table 8.4, there was a statistically significant differ- ence between the proportion of undergraduate majors at ICU and those reported nationally (χ2 = 11.250, df = 3, p = .010). Thus, the null hypothesis that the proportions of undergraduate majors at ICU par- allel those expected at the national level was rejected at the .05 level of significance. The effect size (χ2/[N(J − 1)]) was .0375, and interpreted using Cohen’s guide (1988) as a very small effect.
Follow-up tests were conducted by examining the standardized residu- als. The standardized residual for business was −2.739 and thus sug- gests that there are different observed to expected frequencies for students majoring in business at ICU compared to national estimates.
235Inferences About Proportions
Therefore, business is the college major that is contributing most to the statistically significant chi-square statistic.
Chi-Square Test of Association
Marie,�our�graduate�research�assistant,�was�also�working�with�Matthew,�a�lobbyist�inter- ested�in�examining�the�association�between�education�level�and�stance�on�gambling��Marie� was�tasked�with�assisting�Matthew�in�writing�his�research�question�(Is there an association between level of education and stance on gambling?)� and� generating� the� test� of� inference� to� answer�his�question��Marie�suggested�a�chi-square�test�of�association�as�the�test�of�infer- ence��A�template�for�writing�a�research�question�for�a�chi-square�test�of�association�is�pre- sented�as�follows:
Is there an association between [independent variable] and [dependent variable]?
It� may� be� helpful� to� include� in� the� results� of� the� chi-square� test� of� association� informa- tion�on�the�extent�to�which�the�assumptions�were�met�(recall�there�are�two�assumptions:� independence� and� expected� frequency� of� at� least� 5� per� cell)�� This� assists� the� reader� in� understanding� that� you� were� thorough� in� data� screening� prior� to� conducting� the� test� of� inference�� It� is� also� desirable� to� include� a� measure� of� effect� size�� Given� the� contingency� coefficient,�C,�of��378,�we�computed�Cohen’s�w�effect�size�to�be��408,�which�would�be�inter- preted�as�a�moderate�to�large�effect�
A chi-square test of association was conducted to determine if there was a relationship between level of education and stance on gambling. The test was conducted using an alpha of .01. It was hypothesized that there was an association between the two variables. The assump- tion of an expected frequency of at least 5 per cell was met. The assumption of independence was not met since the respondents were not randomly selected; thus, there is an increased probability of a Type I error.
From Table 8.5, we can see from the row marginals that 55% of the individuals overall support gambling. However, lower levels of edu- cation have a much higher percentage of support, while the highest level of education has a much lower percentage of support. Thus, there appears to be an association or relationship between gambling stance and level of education. This is subsequently supported sta- tistically from the chi-square test (χ2 = 13.333, df = 3, p = .004). Thus, the null hypothesis that there is no association between stance on gambling and level of education was rejected at the .01 level of significance. Examination of the standardized residuals suggests that respondents who hold a graduate degree are signifi- cantly more likely not to support gambling (standardized residual = 2.0) as compared to all other respondents. The effect size, Cohen’s w, was computed to be .408, which is interpreted to be a moderate to large effect (Cohen, 1988).
236 An Introduction to Statistical Concepts
8.6 Summary
In�this�chapter,�we�described�a�third�inferential�testing�situation:�testing�hypotheses�about� proportions��Several�inferential�tests�and�new�concepts�were�discussed��The�new�concepts� introduced� were� proportions,� sampling� distribution� and� standard� error� of� a� proportion,� contingency�table,�chi-square�distribution,�and�observed�versus�expected�frequencies��The� inferential� tests� described� involving� the� unit� normal� distribution� were� tests� of� a� single� proportion,� of� two� independent� proportions,� and� of� two� dependent� proportions�� These� tests�are�parallel�to�the�tests�of�one�or�two�means�previously�discussed�in�Chapters�6�and�7�� The�inferential�tests�described�involving�the�chi-square�distribution�consisted�of�the�chi- square� goodness-of-fit� test� and� the� chi-square� test� of� association�� In� addition,� examples� were�presented�for�each�of�these�tests��Box�8�1�summarizes�the�tests�reviewed�in�this�chap- ter�and�the�key�points�related�to�each�(including�the�distribution�involved�and�recommen- dations�for�when�to�use�the�test)�
STOp aNd ThINk bOx 8.1
Characteristics�and�Recommendations�for�Inferences�About�Proportions
Test Distribution When to Use
Inferences�about�a� single�proportion� (akin�to�one-sample� mean�test)
Unit�normal,�z •��To�determine�if�the�sample�proportion� differs�from�a�hypothesized�proportion
•�One�variable,�nominal�or�ordinal�in�scale
Inferences�about�two� independent� proportions�(akin�to� the�independent�t�test)
Unit�normal,�z •��To�determine�if�the�population�proportion� for�one�group�differs�from�the�population� proportion�for�a�second�independent�group
•��Two�variables,�both�nominal�and�ordinal� in scale
Inferences�about�two� dependent� proportions�(akin�to� the�dependent�t�test)
Unit�normal,�z •��To�determine�if�the�population�proportion� for�one�group�is�different�than�the� population�proportion�for�a�second� dependent�group
•��Two�variables�of�the�same�measure,�both� nominal�and�ordinal�in�scale
Chi-square�goodness- of-fit�test
Chi-square •��To�determine�if�observed�proportions�differ� from�what�would�be�expected�a�priori
•�One�variable,�nominal�or�ordinal�in�scale
Chi-square�test�of� association
Chi-square •��To�determine�association/relationship� between�two�variables�based�on�observed� proportions
•��Two�variables,�both�nominal�and�ordinal� in scale
At�this�point,�you�should�have�met�the�following�objectives:�(a)�be�able�to�understand�the� basic�concepts�underlying�tests�of�proportions,�(b)�be�able�to�select�the�appropriate�test,�and� (c)�be�able�to�determine�and�interpret�the�results�from�the�appropriate�test��In�Chapter�9,�we� discuss�inferential�tests�involving�variances�
237Inferences About Proportions
Problems
Conceptual problems
8.1� How�many�degrees�of�freedom�are�there�in�a�5��7�contingency�table�when�the�chi- square�test�of�association�is�used?
� a�� 12 � b�� 24 � c�� 30 � d�� 35 8.2� The�more�that�two�independent�sample�proportions�differ,�all�else�being�equal,�the�
smaller�the�z�test�statistic��True�or�false? 8.3� The�null�hypothesis�is�a�numerical�statement�about�an�unknown�parameter��True�or�
false? 8.4� In�testing�the�null�hypothesis�that�the�proportion�is��50,�the�critical�value�of�z�increases�
as�degrees�of�freedom�increase��True�or�false? 8.5� A�consultant�found�a�sample�proportion�of�individuals�favoring�the�legalization�of�
drugs� to� be� −�50�� I� assert� that� a� test� of� whether� that� sample� proportion� is� different� from�0�would�be�rejected��Am�I�correct?
8.6� Suppose�we�wish�to�test�the�following�hypotheses�at�the��10�level�of�significance:
H
H
0
1
60
60
: .
: .
π
π
=
>
A�sample�proportion�of��15�is�observed��I�assert�if�I�conduct�the�z�test�that�it�is�possible� to�reject�the�null�hypothesis��Am�I�correct?
8.7� When� the� chi-square� test� statistic� for� a� test� of� association� is� less� than� the� cor- responding� critical� value,� I� assert� that� I� should� reject� the� null� hypothesis�� Am� I� correct?
8.8� Other�things�being�equal,�the�larger�the�sample�size,�the�smaller�the�value�of�sp��True� or�false?
8.9� In� the� chi-square� test� of� association,� as� the� difference� between� the� observed� and� expected�proportions�increases,
� a�� The�critical�value�for�chi-square�increases� � b�� The�critical�value�for�chi-square�decreases� � c�� The�likelihood�of�rejecting�the�null�hypothesis�decreases� � d�� The�likelihood�of�rejecting�the�null�hypothesis�increases� 8.10� When� the� hypothesized� value� of� the� population� proportion� lies� outside� of� the� CI�
around�a�single�sample�proportion,�I�assert�that�the�researcher�should�reject�the�null� hypothesis��Am�I�correct?
238 An Introduction to Statistical Concepts
8.11� Statisticians�at�a�theme�park�want�to�know�if�the�same�proportions�of�visitors�select� the� Jungle� Safari� as� their� favorite� ride� as� compared� to� the� Mountain� Rollercoaster�� They�sample�150�visitors�and�collect�data�on�one�variable:�favorite�ride�(two�catego- ries:�Jungle�Safari�and�Mountain�Rollercoaster)��Which�statistical�procedure�is�most� appropriate�to�use�to�test�the�hypothesis?
� a�� Chi-square�goodness-of-fit�test � b�� Chi-square�test�of�association 8.12� Sophie� is� a� reading� teacher�� She� is� researching� the� following� question:� is� there� a�
relationship�between�a�child’s�favorite�genre�of�book�and�their�socioeconomic�sta- tus�category?�She�collects�data�from�35�children�on�two�variables:�(a)�favorite�genre� of�book�(two�categories:�fiction,�nonfiction)�and�(b)�socioeconomic�status�category� (three�categories:�low,�middle,�high)��Which�statistical�procedure�is�most�appropri- ate�to�use�to�test�the�hypothesis?
� a�� Chi-square�goodness-of-fit�test � b�� Chi-square�test�of�association
Computational problems
8.1� For�a�random�sample�of�40�widgets�produced�by�the�Acme�Widget�Company,�30�suc- cesses�and�10�failures�are�observed��Test�the�following�hypotheses�at�the��05�level�of� significance:
H
H
0
1
60
60
: .
: .
π
π
=
≠
8.2� The� following� data� are� calculated� for� two� independent� random� samples� of� male� and� female� teenagers,� respectively,� on� whether� they� expect� to� attend� graduate� school:�n1�=�48,�p1�=�18/48,�n2�=�52,�p2�=�33/52��Test�the�following�hypotheses�at�the� �05�level�of�significance:
H
H
0 1 2
1 1 2
0
0
:
:
π π
π π
− =
− ≠
8.3� The�following�frequencies�of�successes�and�failures�are�obtained�for�two�dependent� random�samples�measured�at�the�pretest�and�posttest�of�a�weight�training�program:
Pretest
Posttest Success Failure
Failure 18 30 Success 33 19
Test�the�following�hypotheses�at�the��05�level�of�significance:
H
H
0 1 2
1 1 2
0
0
:
:
π π
π π
− =
− ≠
239Inferences About Proportions
8.4� A� chi-square� goodness-of-fit� test� is� to� be� conducted� with� six� categories� of� profes- sions�to�determine�whether�the�sample�proportions�of�those�supporting�the�current� government�differ�from�a�priori�national�proportions��The�chi-square�test�statistic�is� equal�to�16�00��Determine�the�result�of�this�test�by�looking�up�the�critical�value�and� making�a�statistical�decision,�using�α�=��01�
8.5� A�chi-square�goodness-of-fit�test�is�to�be�conducted�to�determine�whether�the�sample� proportions�of�families�in�Florida�who�select�various�schooling�options�(five�catego- ries�including�home�school,�public�school,�public�charter�school,�private�school,�and� other)� differ� from� the� proportions� reported� nationally�� The� chi-square� test� statistic� is�equal�to�9�00��Determine�the�result�of�this�test�by�looking�up�the�critical�value�and� making�a�statistical�decision,�using�α�=��05�
8.6� A� random� sample� of� 30� voters� was� classified� according� to� their� general� political� beliefs� (liberal� vs�� conservative)� and� also� according� to� whether� they� voted� for� or� against�the�incumbent�representative�in�their�town��The�results�were�placed�into�the� following�contingency�table:
Liberal Conservative
Yes 10 5 No 5 10
Use� the� chi-square� test� of� association� to� determine� whether� political� belief� is� inde- pendent�of�voting�behavior�at�the��05�level�of�significance�
8.7� A�random�sample�of�40�kindergarten�children�was�classified�according�to�whether� they� attended� at� least� 1� year� of� preschool� prior� to� entering� kindergarten� and� also� according�to�gender��The�results�were�placed�into�the�following�contingency�table:
Boy Girl
Preschool 12 10 No�preschool 8 10
Use�the�chi-square�test�of�association�to�determine�whether�enrollment�in�preschool� is�independent�of�gender�at�the��10�level�of�significance�
Interpretive problem
There�are�numerous�ways�to�use�the�survey�1�dataset�from�the�website�as�there�are�several� categorical�variables��Here�are�some�examples�for�the�tests�described�in�this�chapter:
� a�� �Conduct�a�test�of�a�single�proportion:�Is�the�sample�proportion�of�females�equal� to��50?
� b�� Conduct�a�test�of�two�independent�proportions:�Is�there�a�difference�between�the� sample�proportion�of�females�who�are�right-handed�and�the�sample�proportion�of� males�who�are�right-handed?
� c�� Conduct� a� test� of� two� dependent� proportions:� Is� there� a� difference� between� the� sample�proportion�of�students’�mothers�who�are�right-handed�and�the�sample�pro- portion�of�students’�fathers�who�are�right-handed?
240 An Introduction to Statistical Concepts
� d�� Conduct� a� chi-square� goodness-of-fit� test:� Do� the� sample� proportions� for� the� political�view�categories�differ�from�their�expected�proportions�(very�liberal�=��10,� liberal�=��15,�middle�of�the�road�=��50,�conservative�=��15,�very�conservative�=��10)?� Determine�if�the�assumptions�of�the�test�are�met��Determine�and�interpret�the�cor- responding�effect�size�
� e�� Conduct�a�chi-square�goodness-of-fit�test�to�determine�if�there�are�similar�propor- tions� of� respondents� who� can� (vs�� cannot)� tell� the� difference� between� Pepsi� and� Coke��Determine�if�the�assumptions�of�the�test�are�met��Determine�and�interpret� the�corresponding�effect�size�
� f�� Conduct�a�chi-square�test�of�association:�Is�there�an�association�between�political� view� and� gender?� Determine� if� the� assumptions� of� the� test� are� met�� Determine� and�interpret�the�corresponding�effect�size�
� g�� Compute�a�chi-square�test�of�association�to�examine�the�relationship�between�if�a� person�smokes�and�their�political�view��Determine�if�the�assumptions�of�the�test� are�met��Determine�and�interpret�the�corresponding�effect�size�
241
9 Inferences About Variances
Chapter Outline
9�1� New�Concepts 9�2� Inferences�About�Single�Variance 9�3� Inferences�About�Two�Dependent�Variances 9�4� �Inferences�About�Two�or�More�Independent�Variances�(Homogeneity�of�
Variance�Tests) 9�4�1� Traditional�Tests 9�4�2� Brown–Forsythe�Procedure 9�4�3� O’Brien�Procedure
9�5� SPSS 9�6� Template�and�APA-Style�Write-Up
Key Concepts
� 1�� Sampling�distributions�of�the�variance � 2�� The�F�distribution � 3�� Homogeneity�of�variance�tests
In�Chapters�6�through�8,�we�looked�at�testing�inferences�about�means�(Chapters�6�and�7)� and�about�proportions�(Chapter�8)��In�this�chapter,�we�examine�inferential�tests�involving� variances�� Tests� of� variances� are� useful� in� two� applications,� (a)� as� an� inferential� test� by� itself�and�(b)�as�a�test�of�the�homogeneity�of�variance�assumption�for�another�procedure� (e�g�,�t�test,�analysis�of�variance�[ANOVA])��First,�a�researcher�may�want�to�perform�infer- ential�tests�on�variances�for�their�own�sake,�in�the�same�fashion�that�we�described�for�the� one-�and�two-sample�t�tests�on�means��For�example,�we�may�want�to�assess�whether�the� variance�of�undergraduates�at�Ivy-Covered�University�(ICU)�on�an�intelligence�measure�is� the�same�as�the�theoretically�derived�variance�of�225�(from�when�the�test�was�developed� and�normed)��In�other�words,�is�the�variance�at�a�particular�university�greater�than�or�less� than� 225?� As� another� example,� we� may� want� to� determine� whether� the� variances� on� an� intelligence�measure�are�consistent�across�two�or�more�groups;�for�example,�is�the�variance� of�the�intelligence�measure�at�ICU�different�from�that�at�Podunk�University?
242 An Introduction to Statistical Concepts
Second,�for�some�procedures�such�as�the�independent�t�test�(Chapter�7)�and�the�ANOVA� (Chapter� 11),� it� is� assumed� that� the� variances� for� two� or� more� independent� samples� are� equal� (known� as� the� homogeneity� of� variance� assumption)�� Thus,� we� may� want� to� use� an� inferential� test� of� variances� to� assess� whether� this� assumption� has� been� violated� or� not�� The� following� inferential� tests� of� variances� are� covered� in� this� chapter:� (a)� testing� whether�a�single�variance�is�different�from�a�hypothesized�value,�(b)�testing�whether�two� dependent�variances�are�different,�and�(c)�testing�whether�two�or�more�independent�vari- ances� are� different�� We� utilize� many� of� the� foundational� concepts� previously� covered� in� Chapters�6�through�8��New�concepts�to�be�discussed�include�the�following:�the�sampling� distributions� of� the� variance,� the� F� distribution,� and� homogeneity� of� variance� tests�� Our� objectives�are�that�by�the�end�of�this�chapter,�you�will�be�able�to�(a)�understand�the�basic� concepts� underlying� tests� of� variances,� (b)� select� the� appropriate� test,� and� (c)� determine� and�interpret�the�results�from�the�appropriate�test�
9.1 New Concepts
As� you� remember,� Marie� is� a� graduate� student� working� on� a� degree� in� educational� research��She�has�been�building�her�statistical�skills�and�is�becoming�quite�adept�at�apply- ing� her� skills� as� she� completes� tasks� assigned� to� her� by� her� faculty� advisor�� We� revisit� Marie�again�
Another�call�has�been�fielded�by�Marie’s�advisor�for�assistance�with�statistical�analysis�� This�time,�it�is�Jessica,�an�elementary�teacher�within�the�community��Having�built�quite� a�reputation�for�success�in�statistical�consultations,�Marie’s�advisor�requests�that�Marie� work�with�Jessica�
Jessica�shares�with�Marie�that�she�is�conducting�a�teacher�research�project�related�to� achievement�of�first-grade�students�at�her�school��Jessica�wants�to�determine�if�the�vari- ances�of�the�achievement�scores�differ�when�children�begin�school�in�the�fall�as�com- pared� to� when� they� end� school� in� the� spring�� Marie� suggests� the� following� research� question:�Are�the�variances�of�achievement�scores�for�first-grade�children�the�same�in� the�fall�as�compared�to�the�spring?�Marie�suggests�a�test�of�variance�as�the�test�of�infer- ence��Her�task�is�then�to�assist�Jessica�in�generating�the�test�of�inference�to�answer�her� research�question�
This�section�deals�with�concepts�for�testing�inferences�about�variances,�in�particular,�the� sampling�distributions�underlying�such�tests��Subsequent�sections�deal�with�several�infer- ential�tests�of�variances��Although�the�sampling�distribution�of�the�mean�is�a�normal�distri- bution�(Chapters�6�and�7),�and�the�sampling�distribution�of�a�proportion�is�either�a�normal� or�chi-square�distribution�(Chapter�8),�the�sampling distribution of a variance�is�either�a� chi-square�distribution�for�a�single�variance,�a�t�distribution�for�two�dependent�variances,� or� an� F� distribution� for� two� or� more� independent� variances�� Although� we� have� already� discussed�the�t�distribution�in�Chapter�6�and�the�chi-square�distribution�in�Chapter�8,�we� need�to�discuss�the�F�distribution�(named�in�honor�of�the�famous�statistician�Sir�Ronald�A�� Fisher)�in�some�detail�here�
243Inferences About Variances
Like�the�normal,�t,�and�chi-square�distributions,�the�F�distribution�is�really�a�family�of� distributions��Also,�like�the�t�and�chi-square�distributions,�the�F�distribution�family�mem- bers� depend� on� the� number� of� degrees� of� freedom� represented�� Unlike� any� previously� discussed�distribution,�the�F�distribution�family�members�actually�depend�on�a�combina- tion�of�two�different�degrees�of�freedom,�one�for�the�numerator�and�one�for�the�denomina- tor��The�reason�is�that�the�F�distribution�is�a�ratio�of�two�chi-square�variables��To�be�more� precise,�F�with�ν1�degrees�of�freedom�for�the�numerator�and�ν2�degrees�of�freedom�for�the� denominator�is�actually�a�ratio�of�the�following�chi-square�variables:
� Fν ν
ν
ν
χ ν χ ν1 2
1
2
2 1
2 2
, / /
=
For�example,�the�F�distribution�for�1�degree�of�freedom�numerator�and�10�degrees�of�free- dom�denominator�is�denoted�by�F1,10��The�F�distribution�is�generally�positively�skewed� and�leptokurtic�in�shape�(like�the�chi-square�distribution)�and�has�a�mean�of�ν2/(ν2�−�2)� when�ν2�>�2�(where�ν2�represents�the�denominator�degrees�of�freedom)��A�few�examples� of�the�F�distribution�are�shown�in�Figure�9�1�for�the�following�pairs�of�degrees�of�freedom� (i�e�,�numerator,�denominator):�F10,10;�F20,20;�and�F40,40�
Critical� values� for� several� levels� of� α� of� the� F� distribution� at� various� combinations� of� degrees�of�freedom�are�given�in�Table�A�4��The�numerator�degrees�of�freedom�are�given� in�the�columns�of�the�table�(ν1),�and�the�denominator�degrees�of�freedom�are�shown�in�the� rows�of�the�table�(ν2)��Only�the�upper-tail�critical�values�are�given�in�the�table�(e�g�,�percen- tiles�of��90,��95,��99�for�α�=��10,��05,��01,�respectively)��The�reason�is�that�most�inferential�tests� involving�the�F�distribution�are�one-tailed�tests�using�the�upper-tail�critical�region��Thus,� to�find�the�upper-tail�critical�value�for��05F1,10,�we�look�on�the�second�page�of�the�table�(α�=��05),� in�the�first�column�of�values�on�that�page�for�ν1�=�1,�and�where�it�intersects�with�the�10th� row�of�values�for�ν2�=�10��There�you�should�find��05F1,10�=�4�96�
1.5
1.2
0.9
0.6
Re la
tiv e
fr eq
ue nc
y
0.3
0 0 1 2 3
F 4 5
10,10 20,20 40,40
FIGuRe 9.1 Several� members� of� the� family� of� F distributions�
244 An Introduction to Statistical Concepts
9.2 Inferences About Single Variance
In�our�initial�inferential�testing�situation�for�variances,�the�researcher�would�like�to�know� whether�the�population�variance�is�equal�to�some�hypothesized�variance�or�not��First,�the� hypotheses� to� be� evaluated� for� detecting� whether� a� population� variance� differs� from� a� hypothesized�variance�are�as�follows��The�null�hypothesis�H0�is�that�there�is�no�difference� between�the�population�variance�σ2�and�the�hypothesized�variance�σ02,�which�we�denote�as
� H0 2
0 2: σ σ=
Here�there�is�no�difference�or�a�“null”�difference�between�the�population�variance�and�the� hypothesized�variance��For�example,�if�we�are�seeking�to�determine�whether�the�variance� on� an� intelligence� measure� at� ICU� is� different� from� the� overall� adult� population,� then� a� reasonable�hypothesized�value�would�be�225,�as�this�is�the�theoretically�derived�variance� for�the�adult�population�
The� nondirectional,� scientific,� or� alternative� hypothesis� H1� is� that� there� is� a� differ- ence�between�the�population�variance�σ2�and�the�hypothesized�variance� σ02 ,�which�we� denote�as
� H1 2
0 2: σ σ≠
The�null�hypothesis�H0�will�be�rejected�here�in�favor�of�the�alternative�hypothesis�H1�if� the� population� variance� is� different� from� the� hypothesized� variance�� As� we� have� not� specified�a�direction�on�H1,�we�are�willing�to�reject�either�if�σ2�is�greater�than� σ0
2 �or�if�σ2� is�less�than�σ0
2��This�alternative�hypothesis�results�in�a�two-tailed�test��Directional�alter- native�hypotheses�can�also�be�tested�if�we�believe�either�that�σ2�is�greater�than� σ02 �or�that� σ2�is�less�than�σ02��In�either�case,�the�more�the�resulting�sample�variance�differs�from�the� hypothesized�variance,�the�more�likely�we�are�to�reject�the�null�hypothesis�
It�is�assumed�that�the�sample�is�randomly�drawn�from�the�population�(i�e�,�the�assump- tion�of�independence)�and�that�the�population�of�scores�is�normally�distributed��Because� we�are�testing�a�variance,�a�condition�of�the�test�is�that�the�variable�must�be�interval�or�ratio� in�scale�
The�next�step�is�to�compute�the�test�statistic�χ2�as
� χ
ν σ
2 2
0 2=
s
where s2�is�the�sample�variance ν�=�n�−�1
The�test�statistic�χ2�is�then�compared�to�a�critical�value(s)�from�the�chi-square�distribu- tion�� For� a� two-tailed� test,� the� critical� values� are� denoted� as� α νχ/2
2 � and� 1 2 2
− α νχ/ � and� are� found�in�Table�A�3�(recall�that�unlike�z�and�t�critical�values,�two�unique�χ2�critical�values� must�be�found�from�the�table�as�the�χ2�distribution�is�not�symmetric�like�z�or�t)��If�the�test� statistic�χ2�falls�into�either�critical�region,�then�we�reject�H0;�otherwise,�we�fail�to�reject�H0�� For�a�one-tailed�test,�the�critical�value�is�denoted�as� α νχ
2 �for�the�alternative�hypothesis�
245Inferences About Variances
H1:� σ2� <� σ0 2 � and� as� 1
2 −α νχ � for� the� alternative� hypothesis� H1:�σ2�>� σ0
2��If�the�test�statistic� χ2�falls�into�the�appropriate�critical�region,�then�we�reject�H0;�otherwise,�we�fail�to�reject� H0��It�has�been�noted�by�statisticians�such�as�Wilcox�(1996)�that�the�chi-square�distribu- tion�does�not�perform�adequately�when�sampling�from�a�nonnormal�distribution,�as�the� actual�Type�I�error�rate�can�differ�greatly�from�the�nominal�α�level�(the�level�set�by�the� researcher)�� However,� Wilcox� stated� “it� appears� that� a� completely� satisfactory� solution� does�not�yet�exist,�although�many�attempts�have�been�made�to�find�one”�(p��85)�
For�the�two-tailed�test,�a�(1�−�α)%�confidence�interval�(CI)�can�also�be�examined�and�is� formed�as�follows��The�lower�limit�of�the�CI�is
�
ν χα ν
s2
1 2 2
− /
whereas�the�upper�limit�of�the�CI�is
�
ν χα ν
s2
2 2
/
If� the� CI� contains� the� hypothesized� value� σ0 2 ,� then� the� conclusion� is� to� fail� to� reject� H0;�
otherwise,�we�reject�H0� Now�consider�an�example�to�illustrate�use�of�the�test�of�a�single�variance��We�follow�the�
basic�steps�for�hypothesis�testing�that�we�applied�in�previous�chapters��These�steps�include� the�following:
� 1�� State�the�null�and�alternative�hypotheses� � 2�� Select�the�level�of�significance�(i�e�,�alpha,�α)� � 3�� Calculate�the�test�statistic�value� � 4�� Make�a�statistical�decision�(reject�or�fail�to�reject�H0)�
A�researcher�at�the�esteemed�ICU�is�interested�in�determining�whether�the�population� variance�in�intelligence�at�the�university�is�different�from�the�norm-developed�hypoth- esized�variance�of�225��Thus,�a�nondirectional,�two-tailed�alternative�hypothesis�is�uti- lized�� If� the� null� hypothesis� is� rejected,� this� would� indicate� that� the� intelligence� level� at� ICU� is� more� or� less� diverse� or� variable� than� the� norm�� If� the� null� hypothesis� is� not� rejected,� this� would� indicate� that� the� intelligence� level� at� ICU� is� as� equally� diverse� or� variable�as�the�norm�
The�researcher�takes�a�random�sample�of�101�undergraduates�from�throughout�the�uni- versity�and�computes�a�sample�variance�of�149��The�test�statistic�χ2�is�computed�as�follows:
� χ
ν σ
2 2
0 2
100 149 225
66 2222= = = s ( )
.
From� the� Table� A�3,� and� using� an� α� level� of� �05,� we� determine� the� critical� values� to� be� . .025 100
2 74 2219χ = �and� . .975 100 2 129 561χ = ��As�the�test�statistic�does�exceed�one�of�the�critical�
values�by�falling�into�the�lower-tail�critical�region�(i�e�,�66�2222�<�74�2219),�our�decision�is�to�
246 An Introduction to Statistical Concepts
reject�H0��Our�conclusion�then�is�that�the�variance�of�the�undergraduates�at�ICU�is�different� from�the�hypothesized�value�of�225�
The�95%�CI�for�the�example�is�computed�as�follows��The�lower�limit�of�the�CI�is
�
ν χα ν
s2
1 2 2
100 149 129 561
115 0037 −
= = /
( ) .
.
and�the�upper�limit�of�the�CI�is
�
ν χα ν
s2
2 2
100 149 74 2219
200 7494 /
( ) .
.= =
As�the�limits�of�the�CI�(i�e�,�115�0037,�200�7494)�do�not�contain�the�hypothesized�variance�of� 225,�the�conclusion�is�to�reject�H0��As�always,�the�CI�procedure�leads�us�to�the�same�conclu- sion�as�the�hypothesis�testing�procedure�for�the�same�α�level�
9.3 Inferences About Two Dependent Variances
In�our�second�inferential�testing�situation�for�variances,�the�researcher�would�like�to�know� whether�the�population�variance�for�one�group�is�different�from�the�population�variance� for� a� second� dependent� group�� This� is� comparable� to� the� dependent� t� test� described� in� Chapter�7�where�one�population�mean�was�compared�to�a�second�dependent�population� mean��Once�again,�we�have�two�dependently�drawn�samples�
First,�the�hypotheses�to�be�evaluated�for�detecting�whether�two�dependent�population� variances�differ�are�as�follows��The�null�hypothesis�H0�is�that�there�is�no�difference�between� the�two�population�variances�σ1
2�and�σ2 2,�which�we�denote�as
� H0 1 2
2 2 0: σ σ− =
Here�there�is�no�difference�or�a�“null”�difference�between�the�two�population�variances�� For�example,�we�may�be�seeking�to�determine�whether�the�variance�of�income�of�husbands� is�equal�to�the�variance�of�their�wives’�incomes��Thus,�the�husband�and�wife�samples�are� drawn�as�couples�in�pairs�or�dependently,�rather�than�individually�or�independently�
The� nondirectional,� scientific,� or� alternative� hypothesis� H1� is� that� there� is� a� difference� between�the�population�variances�σ1
2�and�σ2 2,�which�we�denote�as
� H1 1 2
2 2 0: σ σ− ≠
The�null�hypothesis�H0�is�rejected�here�in�favor�of�the�alternative�hypothesis�H1�if�the�popu- lation�variances�are�different��As�we�have�not�specified�a�direction�on�H1,�we�are�willing� to�reject�either�if� σ1
2�is�greater�than�σ2 2 �or�if�σ1
2�is�less�than�σ2 2 ��This�alternative�hypothesis�
results� in� a� two-tailed� test�� Directional� alternative� hypotheses� can� also� be� tested� if� we� believe�either�that�σ1
2�is�greater�than�σ2 2�or�that�σ1
2�is�less�than� σ2 2 ��In�either�case,�the�more�
the�resulting�sample�variances�differ�from�one�another,�the�more�likely�we�are�to�reject�the� null�hypothesis�
247Inferences About Variances
It� is� assumed� that� the� two� samples� are� dependently� and� randomly� drawn� from� their� respective�populations,�that�both�populations�are�normal�in�shape,�and�that�the�t�distribu- tion�is�the�appropriate�sampling�distribution��Since�we�are�testing�variances,�a�condition�of� the�test�is�that�the�variable�must�be�interval�or�ratio�in�scale�
The�next�step�is�to�compute�the�test�statistic�t�as�follows:
�
t s s
s s r
= −
− 1 2
2 2
1 2 12 2
2 1
ν
where s1
2�and�s2 2�are�the�sample�variances�for�samples�1�and�2�respectively
s1�and�s2�are�the�sample�standard�deviations�for�samples�1�and�2�respectively r12�is�the�correlation�between�the�scores�from�sample�1�and�sample�2�(which�is�then�squared) ν� is� the� number� of� degrees� of� freedom,� ν� =� n� −� 2,� with� n� being� the� number� of� paired�
observations�(not�the�number�of�total�observations)
Although�correlations�are�not�formally�discussed�until�Chapter�10,�conceptually�the�cor- relation�is�a�measure�of�the�relationship�between�two�variables��This�test�statistic�is�concep- tually�somewhat�similar�to�the�test�statistic�for�the�dependent�t�test�
The� test� statistic� t� is� then� compared� to� a� critical� value(s)� from� the� t� distribution�� For� a� two-tailed�test,�the�critical�values�are�denoted�as� ± α ν2 t �and�are�found�in�Table�A�2��If�the� test�statistic�t�falls�into�either�critical�region,�then�we�reject�H0;�otherwise,�we�fail�to�reject�H0�� For�a�one-tailed�test,�the�critical�value�is�denoted�as� + α ν1 t �for�the�alternative�hypothesis�H1:� σ1
2 �−� σ2 2 �>�0�and�as� − α ν1 t �for�the�alternative�hypothesis�H1:� σ1
2 �−� σ2 2 �<�0��If�the�test�statistic�t�
falls�into�the�appropriate�critical�region,�then�we�reject�H0;�otherwise,�we�fail�to�reject�H0�� It�is�thought�that�this�test�is�not�particularly�robust�to�nonnormality�(Wilcox,�1987)��As�a� result,�other�procedures�have�been�developed�that�are�thought�to�be�more�robust��However,� little�in�the�way�of�empirical�results�is�known�at�this�time��Some�of�the�new�procedures�can� also�be�used�for�testing�inferences�involving�the�equality�of�two�or�more�dependent�variances�� In�addition,�note�that�acceptable�CI�procedures�are�not�currently�available�
Let�us�consider�an�example�to�illustrate�use�of�the�test�of�two�dependent�variances��The� same�basic�steps�for�hypothesis�testing�that�we�applied�in�previous�chapters�will�be�applied� here�as�well��These�steps�include�the�following:
� 1�� State�the�null�and�alternative�hypotheses� � 2�� Select�the�level�of�significance�(i�e�,�alpha,�α)� � 3�� Calculate�the�test�statistic�value� � 4�� Make�a�statistical�decision�(reject�or�fail�to�reject�H0)�
A�researcher�is�interested�in�whether�there�is�greater�variation�in�achievement�test�scores� at�the�end�of�the�first�grade�as�compared�to�the�beginning�of�the�first�grade��Thus,�a�direc- tional,�one-tailed�alternative�hypothesis�is�utilized��If�the�null�hypothesis�is�rejected,�this� would�indicate�that�first�graders’�achievement�test�scores�are�more�variable�at�the�end�of� the�year�than�at�the�beginning�of�the�year��If�the�null�hypothesis�is�not�rejected,�this�would� indicate�that�first�graders’�achievement�test�scores�have�approximately�the�same�variance� at�both�the�end�of�the�year�and�at�the�beginning�of�the�year�
248 An Introduction to Statistical Concepts
A�random�sample�of�62�first-grade�children�is�selected�and�given�the�same�achievement� test� at� the� beginning� of� the� school� year� (September)� and� at� the� end� of� the� school� year� (April)��Thus,�the�same�students�are�tested�twice�with�the�same�instrument,�thereby�result- ing�in�dependent�samples�at�time�1�and�time�2��The�level�of�significance�is�set�at�α�=��01��The� test�statistic�t�is�computed�as�follows��We�determine�that�n�=�62,�ν�=�60,� s12 �=�100,�s1�=�10,� s2
2 �=�169,� s2�=�13,�and�r12�=��80��We�compute�the�test�statistic�t�to�be�as�follows:
�
t s s
s s r
= −
− =
− −
= −1 2
2 2
1 2 12 2
2 1
100 169
2 10 13 1 64
60
3 4261
ν ( )
. .
The�test�statistic�t�is�then�compared�to�the�critical�value�from�the�t�distribution��As�this�is� a�one-tailed�test,�the�critical�value�is�denoted�as�− α ν1t �and�is�determined�from�Table�A�2�to� be�−�01t60�=�−2�390��The�test�statistic�t�falls�into�the�lower-tail�critical�region,�as�it�is�less�than� the�critical�value�(i�e�,�−3�4261�<�−2�390),�so�we�reject�H0�and�conclude�that�the�variance�in� achievement�test�scores�increases�from�September�to�April�
9.4 Inferences About Two or More Independent Variances (Homogeneity of Variance Tests)
In�our�third�and�final�inferential�testing�situation�for�variances,�the�researcher�would� like�to�know�whether�the�population�variance�for�one�group�is�different�from�the�pop- ulation� variance� for� one� or� more� other� independent� groups�� In� this� section,� we� first� describe� the� somewhat� cloudy� situation� that� exists� for� the� traditional� tests�� Then� we� provide� details� on� two� recommended� tests,� the� Brown–Forsythe� procedure� and� the� O’Brien�procedure�
9.4.1 Traditional Tests
One�of�the�more�heavily�studied�inferential�testing�situations�in�recent�years�has�been�for� testing�whether�differences�exist�among�two�or�more�independent�group�variances��These� tests�are�often�referred�to�as�homogeneity of variance tests��Here�we�briefly�discuss�the� more�traditional�tests�and�their�associated�problems��In�the�sections�that�follow,�we�then� recommend�two�of�the�“better”�tests��As�was�noted�in�the�previous�procedures,�the�vari- able�for�which�the�variance(s)�is�computed�must�be�interval�or�ratio�in�scale�
Several� tests� have� traditionally� been� used� to� test� for� the� equality� of� independent� vari- ances�� An� early� simple� test� for� two� independent� variances� is� to� form� a� ratio� of� the� two� sample�variances,�which�yields�the�following�F�test�statistic:
� F
s s
= 1 2
2 2
This�F�ratio�test�assumes�that�the�two�populations�are�normally�distributed��However,�it� is�known�that�the�F�ratio�test�is�not�very�robust�to�violation�of�the�normality�assumption,�
249Inferences About Variances
except�for�when�the�sample�sizes�are�equal�(i�e�,�n1�=�n2)��In�addition,�the�F�ratio�test�can�only� be�used�for�the�two-group�situation�
Subsequently,� more� general� tests� were� developed� to� cover� the� multiple-group� situation�� One�such�popular�test�is�Hartley’s�Fmax�test�(developed�in�1950),�which�is�simply�a�more�general� version�of�the�F�ratio�test�just�described��The�test�statistic�for�Hartley’s�Fmax�test�is�the�following:
� F
s s
max largest
smallest =
2
2
where slargest
2 �is�the�largest�variance�in�the�set�of�variances ssmallest
2 �is�the�smallest�variance�in�the�set
Hartley’s�Fmax�test�assumes�normal�population�distributions�and�requires�equal�sample�sizes�� We�also�know�that�Hartley’s�Fmax�test�is�not�very�robust�to�violation�of�the�normality�assump- tion��Cochran’s�C�test�(developed�in�1941)�is�also�an�F�test�statistic�and�is�computed�by�taking�the� ratio�of�the�largest�variance�to�the�sum�of�all�of�the�variances��Cochran’s�C�test�also�assumes�nor- mality,�requires�equal�sample�sizes,�and�has�been�found�to�be�even�less�robust�to�nonnormality� than�Hartley’s�Fmax�test��As�we�see�in�Chapter�11�for�the�ANOVA,�it�is�when�we�have�unequal� sample�sizes�that�unequal�variances�are�a�problem;�for�these�reasons,�none�of�these�tests�can�be� recommended,�which�is�the�same�situation�we�encountered�with�the�independent�t�test�
Bartlett’s�χ2�test�(developed�in�1937)�does�not�have�the�stringent�requirement�of�equal�sam- ple�sizes;�however,�it�does�still�assume�normality��Bartlett’s�test�is�very�sensitive�to�nonnor- mality�and�is�therefore�not�recommended�either��Since�1950,�the�development�of�homogeneity� tests�has�proliferated,�with�the�goal�being�to�find�a�test�that�is�fairly�robust�to�nonnormality�� Seemingly�as�each�new�test�was�developed,�later�research�would�show�that�the�test�was�not� very�robust��Today�there�are�well�over�60�such�tests�available�for�examining�homogeneity�of� variance�(e�g�,�a�bootstrap�method�developed�by�Wilcox�[2002])��Rather�than�engage�in�a�pro- tracted�discussion�of�these�tests�and�their�associated�limitations,�we�simply�present�two�tests� that�have�been�shown�to�be�most�robust�to�nonnormality�in�several�recent�studies��These�are� the�Brown–Forsythe�procedure�and�the�O’Brien�procedure��Unfortunately,�neither�of�these� tests�is�available�in�the�major�statistical�packages�(e�g�,�SPSS),�which�only�include�some�of�the� problematic�tests�previously�described�
9.4.2 brown–Forsythe procedure
The�Brown–Forsythe�procedure�is�a�variation�of�Levene’s�test�developed�in�1960��Levene’s� test�is�essentially�an�ANOVA�on�the�transformed�variable:
� Z Y Yij ij j= − �
where ij�designates�the�ith�observation�in�group�j Zij�is�computed�for�each�individual�by�taking�their�score�Yij,�subtracting�from�it�the�group�
mean�Y –
�j�(the�“�”�indicating�we�have�averaged�across�all�i�observations�in�group j),�and� then�taking�the�absolute�value�(i�e�,�by�removing�the�sign)
Unfortunately,� Levene’s� test� is� not� very� robust� to� nonnormality,� except� when� sample� sizes�are�equal�
250 An Introduction to Statistical Concepts
Developed�in�1974,�the�Brown–Forsythe�procedure�has�been�shown�to�be�quite�robust�to� nonnormality�in�numerous�studies�(e�g�,�Olejnik�&�Algina,�1987;�Ramsey,�1994)��Based�on� this� and� other� research,� the� Brown–Forsythe� procedure� is� recommended� for� leptokurtic� distributions� (i�e�,� those� with� sharp� peaks),� as� it� is� robust� to� nonnormality� and� provides� adequate� Type� I� error� protection� and� excellent� power�� In� the� next� section,� we� describe� the� O’Brien� procedure,� which� is� recommended� for� other� distributions� (i�e�,� mesokurtic� and�platykurtic�distributions)��In�cases�where�you�are�unsure�of�which�procedure�to�use,� Algina,� Blair,� and� Combs� (1995)� recommend� using� a� maximum� procedure,� where� both� tests�are�conducted�and�the�procedure�with�the�maximum�test�statistic�is�selected�
Let�us�now�examine�in�detail�the�Brown–Forsythe�procedure��The�null�hypothesis�is�that� H0:� σ1
2 �=� σ2 2 �=�…�=� σJ
2,�and�the�alternative�hypothesis�is�that�not�all�of�the�population�group� variances�are�the�same��The�Brown–Forsythe�procedure�is�essentially�an�ANOVA�on�the� transformed�variable:
� Z Y Mdnij ij j= − �
which� is� computed� for� each� individual� by� taking� their� score� Yij,� subtracting� from� it� the� group�median�Mdn�j,�and�then�taking�the�absolute�value�(i�e�,�by�removing�the�sign)��The� test�statistic�is�an�F�and�is�computed�by
�
F
n Z Z J
Z Z N J
j j
j
J
ij j
j
J
i
nj =
− −
− −
=
==
∑
∑∑
( ) /( )
( ) /( )
. ..
.
2
1
2
11
1
where nj�designates�the�number�of�observations�in�group�j J�is�the�number�of�groups�(where�j�ranges�from�1�to�J) Z –
�j�is�the�mean�for�group�j�(computed�by�taking�the�sum�of�the�observations�in�group�j� and�dividing�by�the�number�of�observations�in�group�j,�which�is�nj)
Z –
���is�the�overall�mean�regardless�of�group�membership�(computed�by�taking�the�sum�of�all� of�the�observations�across�all�groups�and�dividing�by�the�total�number�of�observations�N)
The� test� statistic� F� is� compared� against� a� critical� value� from� the� F� table� (Table� A�4)� with� J�−�1�degrees�of�freedom�in�the�numerator�and�N�−�J�degrees�of�freedom�in�the�denomina- tor,�denoted�by� αFJ−1,�N−J��If�the�test�statistic�is�greater�than�the�critical�value,�we�reject�H0;� otherwise,�we�fail�to�reject�H0�
An� example� using� the� Brown–Forsythe� procedure� is� certainly� in� order� now�� Three� dif- ferent� groups� of� children,� below-average,� average,� and� above-average� readers,� play� a� com- puter�game��The�scores�on�the�dependent�variable�Y�are�their�total�points�from�the�game��We� are� interested� in� whether� the� variances� for� the� three� student� groups� are� equal� or� not�� The� example�data�and�computations�are�given�in�Table�9�1��First�we�compute�the�median�for�each� group,� and� then� compute� the� deviation� from� the� median� for� each� individual� to� obtain� the� transformed�Z�values��Then�the�transformed�Z�values�are�used�to�compute�the�F�test�statistic�
The�test�statistic�F�=�1�6388�is�compared�against�the�critical�value�for�α�=��05�of��05F2,9�=� 4�26�� As� the� test� statistic� is� smaller� than� the� critical� value� (i�e�,� 1�6388� <� 4�26),� we� fail� to� reject�the�null�hypothesis�and�conclude�that�the�three�student�groups�do�not�have�different� variances�
251Inferences About Variances
9.4.3 O’brien procedure
The� final� test� to� consider� in� this� chapter� is� the� O’Brien� procedure�� While� the� Brown– Forsythe�procedure�is�recommended�for�leptokurtic�distributions,�the�O’Brien�procedure� is�recommended�for�other�distributions�(i�e�,�mesokurtic�and�platykurtic�distributions)�� Let�us�now�examine�in�detail�the�O’Brien�procedure��The�null�hypothesis�is�again�that� H0:� σ1
2 � =� σ2 2� =� …� =� σJ
2,� and� the� alternative� hypothesis� is� that� not� all� of� the� population� group�variances�are�the�same�
Table 9.1
Example�Using�the�Brown–Forsythe�and�O’Brien�Procedures
Group 1 Group 2 Group 3
Y Z r Y Z r Y Z r
6 4 124�2499 9 4 143 10 8 704 8 2 14�2499 12 1 −7 16 2 −16
12 2 34�2499 14 1 −7 20 2 −96 13 3 89�2499 17 4 143 30 12 1104
Mdn Z –
r– Mdn Z –
r– Mdn Z –
r–
10 2�75 65�4999 13 2�50 68 18 6 424
Overall�Z –
Overall�r–
3�75 185�8333
Computations�for�the�Brown–Forsythe�procedure:
F
n Z Z J
Z Z N J
j j
j
J
ij j
j
J
i
nj =
− −
− −
=
=
==
∑
∑∑
( ) /( )
( ) /( )
[ ( .
. ..
.
2
1
2
11
1
4 2 775 3 75 4 2 50 3 75 4 6 3 75 2 4 2 75 2 2 75
2 2 2
2 2
− + − + − − + −
. ) ( . . ) ( . ) ]/ [( . ) ( . ) ++ + −
= � ( ) ]/
. 12 6 9
1 63882
Computations�for�the�O’Brien�procedure: Sample�means:�Y
– 1�=�9�75,�Y
– 2�=�13�0,�Y
– 3�=�19�0
Sample�variances:�s1 2 0.= 1 9167,�s22 = 11 3333. ,�s32 0= 7 6667.
Example�computation�for�rij:
r n n Y Y s n
n n ij
j j ij j j j
j j =
− − − − − −
= −( . ) ( ) . ( )
( )( ) ( . ) (.1 5 5 1
1 2 4 1 5 42 2 66 9 75 5 10 9167 4 1
4 1 4 2 124 2499
2− − − − −
= . ) . ( . )( )
( )( ) .
Test�statistic:
F
n r r J
r r N J
j j
j
J
ij j
j
J
i
nj =
− −
− −
=
=
==
∑
∑∑
( ) /( )
( ) / ( )
[ (
. ..
.
2
1
2
11
1
4 65.. . ) ( . ) ( . ) ]/ [(
4999 185 8333 4 68 185 8333 4 424 185 8333 2 124
2 2 2− + − + − .. . ) ( . . ) ( ) ]/
.
2499 65 4999 14 2499 65 4999 1104 424 9
1 479
2 2 2− + − + + −
=
�
99
252 An Introduction to Statistical Concepts
The�O’Brien�procedure�is�an�ANOVA�on�a�different�transformed�variable:
� r
n n Y Y s n n n
ij j j ij j j j
j j
= − − − −
− − ( . ) ( ) . ( )
( )( ) .1 5 5 1
1 2
2 2
which�is�computed�for�each�individual,�where nj�is�the�size�of�group�j Y –
�j�is�the�mean�for�group�j sj
2�is�the�sample�variance�for�group�j
The�test�statistic�is�an�F�and�is�computed�by
�
F
n r r J
r r N J
j j
j
J
ij j
j
J
i
nj =
− −
− −
=
==
∑
∑∑
( ) /( )
( ) /( )
. ..
.
2
1
2
11
1
where nj�designates�the�number�of�observations�in�group�j J�is�the�number�of�groups�(where�j�ranges�from�1�to�J) r–�j�is�the�mean�for�group�j�(computed�by�taking�the�sum�of�the�observations�in�group�j�
and�dividing�by�the�number�of�observations�in�group�j,�which�is�nj) r���is�the�overall�mean�regardless�of�group�membership�(computed�by�taking�the�sum�of�
all�of�the�observations�across�all�groups�and�dividing�by�the�total�number�of�observa- tions�N)
The�test�statistic�F�is�compared�against�a�critical�value�from�the�F�table�(Table�A�4)�with�J�−�1� degrees�of�freedom�in�the�numerator�and�N�−�J�degrees�of�freedom�in�the�denominator,� denoted�by�αFJ−1,N−J��If�the�test�statistic�is�greater�than�the�critical�value,�then�we�reject�H0;� otherwise,�we�fail�to�reject�H0�
Let�us�return�to�the�example�in�Table�9�1�and�consider�the�results�of�the�O’Brien�proce- dure��From�the�computations�shown�in�the�table,�the�test�statistic�F�=�1�4799�is�compared� against�the�critical�value�for�α�=��05�of��05F2,9�=�4�26��As�the�test�statistic�is�smaller�than�the� critical�value�(i�e�,�1�4799�<�4�26),�we�fail�to�reject�the�null�hypothesis�and�conclude�that� the�three�student�groups�do�not�have�different�variances�
9.5 SPSS
Unfortunately,�there�is�not�much�to�report�on�tests�of�variances�for�SPSS��There�are�no�tests� available�for�inferences�about�a�single�variance�or�for�inferences�about�two�dependent�vari- ances��For�inferences�about�independent�variances,�SPSS�does�provide�Levene’s�test�as�part� of�the�“Independent�t�Test”�procedure�(previously�discussed�in�Chapter�7),�and�as�part�of�the�
253Inferences About Variances
“One-Way�ANOVA”�and�“Univariate�ANOVA”�procedures�(to�be�discussed�in�Chapter�11)�� Given�our�previous�concerns�with�Levene’s�test,�use�it�with�caution��There�is�also�little�infor- mation�published�in�the�literature�on�power�and�effect�sizes�for�tests�of�variances�
9.6 Template and APA-Style Write-Up
Consider�an�example�paragraph�for�one�of�the�tests�described�in�this�chapter,�more�spe- cifically,� testing� inferences� about� two� dependent� variances�� As� you� may� remember,� our� graduate�research�assistant,�Marie,�was�working�with�Jessica,�a�classroom�teacher,�to�assist� in�analyzing�the�variances�of�first-grade�students��Her�task�was�to�assist�Jessica�with�writ- ing� her� research� question� (Are the variances of achievement scores for first-grade children the same in the fall as compared to the spring?)�and�generating�the�test�of�inference�to�answer�her� question��Marie�suggested�a�dependent�variances�test�as�the�test�of�inference��A�template� for�writing�a�research�question�for�the�dependent�variances�is�presented�as�follows:
Are the Variances of [Variable] the Same in [Time 1] as Compared to [Time 2]?
An�example�write-up�is�presented�as�follows:
A dependent variances test was conducted to determine if variances of achievement scores for first-grade children were the same in the fall as compared to the spring. The test was conducted using an alpha of .05. The null hypothesis was that the variances would be the same.
There was a statistically significant difference in variances of achievement scores of first-grade children in the fall as compared to the spring (t = −3.4261, df = 60, p < .05). Thus, the null hypothesis that the variances would be equal at the beginning and end of the first grade was rejected. The variances of achievement test scores significantly increased from September to April.
9.7 Summary
In� this� chapter,� we� described� testing� hypotheses� about� variances�� Several� inferential� tests� and� new� concepts� were� discussed�� The� new� concepts� introduced� were� the� sam- pling� distribution� of� the� variance,� the� F� distribution,� and� homogeneity� of� variance� tests�� The� first� inferential� test� discussed� was� the� test� of� a� single� variance,� followed� by�a�test�of�two�dependent�variances��Next�we�examined�several�tests�of�two�or�more� independent�variances��Here�we�considered�the�following�traditional�procedures:�the� F� ratio� test,� Hartley’s� Fmax� test,� Cochran’s� C� test,� Bartlett’s� χ2� test,� and� Levene’s� test�� Unfortunately,�these�tests�are�not�very�robust�to�violation�of�the�normality�assumption�� We� then� discussed� two� newer� procedures� that� are� relatively� robust� to� nonnormality,�
254 An Introduction to Statistical Concepts
the�Brown–Forsythe�procedure�and�the�O’Brien�procedure��Examples�were�presented� for� each� of� the� recommended� tests�� At� this� point,� you� should� have� met� the� following� objectives:� (a)� be� able� to� understand� the� basic� concepts� underlying� tests� of� variances,� (b)�be�able�to�select�the�appropriate�test,�and�(c)�be�able�to�determine�and�interpret�the� results�from�the�appropriate�test��In�Chapter�10,�we�discuss�correlation�coefficients,�as� well�as�inferential�tests�involving�correlations�
Problems
Conceptual problems
9.1� Which�of�the�following�tests�of�homogeneity�of�variance�is�most�robust�to�assump- tion�violations?
� a�� F�ratio�test � b�� Bartlett’s�chi-square�test � c�� The�O’Brien�procedure � d�� Hartley’s�Fmax�test 9.2� Cochran’s�C�test�requires�equal�sample�sizes��True�or�false? 9.3� I�assert�that�if�two�dependent�sample�variances�are�identical,�I�would�not�be�able�to�
reject�the�null�hypothesis��Am�I�correct? 9.4� Suppose�that�I�wish�to�test�the�following�hypotheses�at�the��01�level�of�significance:
�
H
H
0 2
1 2
250
250
:
:
σ
σ
=
>
� A� sample� variance� of� 233� is� observed�� I� assert� that� if� I� compute� the� χ2� test� statistic� and�compare�it�to�the�χ2�table,�it�is�possible�that�I�could�reject�the�null�hypothesis��Am� I�correct?
9.5� Suppose�that�I�wish�to�test�the�following�hypotheses�at�the��05�level�of�significance:
�
H
H
0 2
1 2
16
16
:
:
σ
σ
=
>
� A�sample�variance�of�18�is�observed��I�assert�that�if�I�compute�the�χ2�test�statistic�and� compare� it� to� the� χ2� table,� it� is� possible� that� I� could� reject� the� null� hypothesis�� Am� I� correct?
9.6� If� the� 90%� CI� for� a� single� variance� extends� from� 25�7� to� 33�6,� I� assert� that� the� null� hypothesis�would�definitely�be�rejected�at�the��10�level��Am�I�correct?
9.7� If� the� 95%� CI� for� a� single� variance� ranges� from� 82�0� to� 93�5,� I� assert� that� the� null� hypothesis�would�definitely�be�rejected�at�the��05�level��Am�I�correct?
9.8� If�the�mean�of�the�sampling�distribution�of�the�difference�between�two�variances�equals� 0,�I�assert�that�both�samples�probably�represent�a�single�population��Am�I�correct?
255Inferences About Variances
9.9� Which�of�the�following�is�an�example�of�two�dependent�samples? � a�� Pretest�scores�of�males�in�one�course�and�posttest�scores�of�females�in�another�course � b�� Husbands�and�their�wives�in�your�neighborhood � c�� Softball�players�at�your�school�and�football�players�at�your�school � d�� Professors�in�education�and�professors�in�psychology 9.10� The� mean� of� the� F� distribution� increases� as� degrees� of� freedom� denominator� (ν2)�
increase��True�or�false?
Computational problems
9.1� The�following�random�sample�of�scores�on�a�preschool�ability�test�is�obtained�from�a� normally�distributed�population�of�4�year�olds:
20 22 24 30 18 22 29 27 25 21 19 22 38 26 17 25
� a�� Test�the�following�hypotheses�at�the��10�level�of�significance:
�
H
H
0 2
1 2
75
75
:
:
σ
σ
=
≠
� b�� Construct�a�90%�CI� 9.2� The� following� two� independent� random� samples� of� number� of� CDs� owned� are�
obtained�from�two�populations�of�undergraduate�(sample�1)�and�graduate�students� (sample�2),�respectively:
Sample 1 Data Sample 2 Data
42 36 47 35 46 45 50 57 58 43 37 52 44 47 51 52 43 60 41 49 56 54 55 50 40 44 51 49 55 56 40 46 41
� Test� the� following� hypotheses� at� the� �05� level� of� significance� using� the� Brown– Forsythe�and�O’Brien�procedures:
�
H
H
0 1 2
2 2
1 1 2
2 2
0
0
:
:
σ σ
σ σ
− =
− ≠
9.3� The�following�summary�statistics�are�available�for�two�dependent�random�samples� of�brothers�and�sisters,�respectively,�on�their�allowance�for�the�past�month:�s1
2�=�49,� s2
2�=�25,�n =�32,�r12�=��60� �Test�the�following�hypotheses�at�the��05�level�of�significance:
�
H
H
0 1 2
2 2
1 1 2
2 2
0
0
:
:
σ σ
σ σ
− =
− ≠
256 An Introduction to Statistical Concepts
9.4� The�following�summary�statistics�are�available�for�two�dependent�random�samples� of�first�semester�college�students�who�were�measured�on�their�high�school�and�first� semester�college�GPAs,�respectively:� s1
2 �=�1�56,� s2 2 �=�4�42,�n�=�62,�r12�=��72�
�Test�the�following�hypotheses�at�the��05�level�of�significance:
�
H
H
0 1 2
2 2
1 1 2
2 2
0
0
:
:
σ σ
σ σ
− =
− ≠
9.5� A�random�sample�of�21�statistics�exam�scores�is�collected�with�a�sample�mean�of� 50� and� a� sample� variance� of� 10�� Test� the� following� hypotheses� at� the� �05� level� of� significance:
�
H
H
0 2
1 2
25
25
:
:
σ
σ
=
≠
9.6� A� random� sample� of� 30� graduate� entrance� exam� scores� is� collected� with� a� sample� mean�of�525�and�a�sample�variance�of�16,900��Test�the�following�hypotheses�at�the��05� level�of�significance:
�
H
H
0 2
1 2
10 000
10 000
: ,
: ,
σ
σ
=
≠
9.7� A� pretest� was� given� at� the� beginning� of� a� history� course� and� a� posttest� at� the� end� of� the� course�� The� pretest� variance� is� 36,� the� posttest� variance� is� 64,� sample� size� is� 31,� and� the� pretest-posttest� correlation� is� �80�� Test� the� null� hypothesis� that� the� two� dependent�variances�are�equal�against�a�nondirectional�alternative�at�the��01�level�of� significance�
Interpretive problems
9.1� Use� the� survey� 1� dataset� from� the� website� to� determine� if� there� are� gender� differ- ences�among�the�variances�for�any�items�of�interest�that�are�at�least�interval�or�ratio� in�scale��Some�example�items�might�include�the�following:
� a�� Item�#1:�height�in�inches � b�� Item�#6:�amount�spent�at�last�hair�appointment � c�� Item�#7:�number�of�compact�disks�owned � d�� Item�#9:�current�GPA � e�� Item�#10:�amount�of�exercise�per�week � f�� Item�#15:�number�of�alcoholic�drinks�per�week � g�� Item�#21:�number�of�hours�studied�per�week
257Inferences About Variances
9.2� Use� the� survey� 1� dataset� from� the� website� to� determine� if� there� are� differences� between�the�variances�for�left-�versus�right-handed�individuals�on�any�items�of�inter- est�that�are�at�least�interval�or�ratio�in�scale��Some�example�items�might�include�the� following:
� a�� Item�#1:�height�in�inches � b�� Item�#6:�amount�spent�at�last�hair�appointment � c�� Item�#7:�number�of�compact�disks�owned � d�� Item�#9:�current�GPA � e�� Item�#10:�amount�of�exercise�per�week � f�� Item�#15:�number�of�alcoholic�drinks�per�week � g�� Item�#21:�number�of�hours�studied�per�week
259
10 Bivariate Measures of Association
Chapter Outline
10�1� Scatterplot 10�2� Covariance 10�3� Pearson�Product–Moment�Correlation�Coefficient 10�4� Inferences�About�the�Pearson�Product–Moment�Correlation�Coefficient
10�4�1� Inferences�for�a�Single�Sample 10�4�2� Inferences�for�Two�Independent�Samples
10�5� Assumptions�and�Issues�Regarding�Correlations 10�5�1� Assumptions 10�5�2� Correlation�and�Causality 10�5�3� Restriction�of�Range
10�6� Other�Measures�of�Association 10�6�1� Spearman’s�Rho 10�6�2� Kendall’s�Tau 10�6�3� Phi 10�6�4� Cramer’s�Phi 10�6�5� Other�Correlations
10�7� SPSS 10�8� G*Power 10�9� Template�and�APA-Style�Write-Up
Key Concepts
� 1�� Scatterplot � 2�� Strength�and�direction � 3�� Covariance � 4�� Correlation�coefficient � 5�� Fisher’s�Z�transformation � 6�� Linearity�assumption,�causation,�and�restriction�of�range�issues
260 An Introduction to Statistical Concepts
We� have� considered� various� inferential� tests� in� the� last� four� chapters,� specifically� those� that�deal�with�tests�of�means,�proportions,�and�variances��In�this�chapter,�we�examine�mea- sures�of�association�as�well�as�inferences�involving�measures�of�association��Methods�for� directly�determining�the�relationship�among�two�variables�are�known�as�bivariate analy- sis,� rather� than� univariate analysis� which� is� only� concerned� with� a� single� variable�� The� indices�used�to�directly�describe�the�relationship�among�two�variables�are�known�as�cor- relation coefficients�(in�the�old�days,�known�as�co-relation)�or�as�measures of association�
These�measures�of�association�allow�us�to�determine�how�two�variables�are�related�to� one� another� and� can� be� useful� in� two� applications,� (a)� as� a� descriptive� statistic� by� itself� and�(b)�as�an�inferential�test��First,�a�researcher�may�want�to�compute�a�correlation�coeffi- cient�for�its�own�sake,�simply�to�tell�the�researcher�precisely�how�two�variables�are�related� or� associated�� For� example,� we� may� want� to� determine� whether� there� is� a� relationship� between�the�GRE-Quantitative�(GRE-Q)�subtest�and�performance�on�a�statistics�exam��Do� students�who�score�relatively�high�on�the�GRE-Q�perform�higher�on�a�statistics�exam�than� do�students�who�score�relatively�low�on�the�GRE-Q?�In�other�words,�as�scores�increase�on� the�GRE-Q,�do�they�also�correspondingly�increase�their�performance�on�a�statistics�exam?
Second,� we� may� want� to� use� an� inferential� test� to� assess� whether� (a)� a� correlation� is� significantly�different�from�0�or�(b)�two�correlations�are�significantly�different�from�one� another��For�example,�is�the�correlation�between�GRE-Q�and�statistics�exam�performance� significantly�different�from�0?�As�a�second�example,�is�the�correlation�between�GRE-Q� and�statistics�exam�performance�the�same�for�younger�students�as�it�is�for�older�students?
The�following�topics�are�covered�in�this�chapter:�scatterplot,�covariance,�Pearson�product- moment�correlation�coefficient,�inferences�about�the�Pearson�product–moment�correlation� coefficient,� some� issues� regarding� correlations,� other� measures� of� association,� SPSS,� and� power��We�utilize�some�of�the�basic�concepts�previously�covered�in�Chapters�6�through�9�� New�concepts�to�be�discussed�include�the�following:�scatterplot;�strength�and�direction;� covariance;� correlation� coefficient;� Fisher’s� Z� transformation;� and� linearity� assumption,� causation,�and�restriction�of�range�issues��Our�objectives�are�that�by�the�end�of�this�chapter,� you�will�be�able�to�(a)�understand�the�concepts�underlying�the�correlation�coefficient�and� correlation�inferential�tests,�(b)�select�the�appropriate�type�of�correlation,�and�(c)�determine� and�interpret�the�appropriate�correlation�and�inferential�test�
10.1 Scatterplot
Marie,�the�graduate�student�pursuing�a�degree�in�educational�research,�continues�to�work� diligently�on�her�coursework��Additionally,�as�we�will�once�again�see�in�this�chapter,�Marie� continues�to�assist�her�faculty�advisor�with�various�research�tasks�
Marie’s� faculty� advisor� received� a� telephone� call� from� Matthew,� the� director� of� mar- keting�for�the�local�animal�shelter��Based�on�the�donor�list,�it�appears�that�the�donors� who� contribute� the� largest� donations� also� have� children� and� pets�� In� an� effort� to� attract�more�donors�to�the�animal�shelter,�Matthew�is�targeting�select�groups—one�of� which�he�believes�may�be�families�that�have�children�at�home�and�who�also�have�pets�� Matthew�believes�if�there�is�a�relationship�between�these�variables,�he�can�more�easily� reach� the� intended� audience� with� his� marketing� materials� which� will� then� translate� into� increased� donations� to� the� animal� shelter�� However,� Matthew� wants� to� base� his�
261Bivariate Measures of Association
decision�on�solid�evidence�and�not�just�a�hunch��Having�built�a�good�knowledge�base� with� previous� consulting� work,� Marie’s� faculty� advisor� puts� Matthew� in� touch� with� Marie�� After� consulting� with� Matthew,� Marie� suggests� a� Pearson� correlation� as� the� test�of�inference�to�test�his�research�question:�Is there a correlation between the number of children in a family and the number of pets?�Marie’s�task�is�then�to�assist�in�generating�the� test�of�inference�to�answer�Matthew’s�research�question�
This�section�deals�with�an�important�concept�underlying�the�relationship�among�two�vari- ables,�the�scatterplot��Later�sections�move�us�into�ways�of�measuring�the�relationship�among� two� variables�� First,� however,� we� need� to� set� up� the� situation� where� we� have� data� on� two� different�variables�for�each�of�N�individuals�in�the�population��Table�10�1�displays�such�a�situ- ation��The�first�column�is�simply�an�index�of�the�individuals�in�the�population,�from�i�=�1,…,�N,� where� N� is� the� total� number� of� individuals� in� the� population�� The� second� column� denotes� the�values�obtained�for�the�first�variable�X��Thus,�X1�=�10�means�that�the�first�individual�had� a�score�of�10�on�variable�X��The�third�column�provides�the�values�for�the�second�variable�Y�� Thus,�Y1�=�20�indicates�that�the�first�individual�had�a�score�of�20�on�variable�Y��In�an�actual� data�table,�only�the�scores�would�be�shown,�not�the�Xi�and�Yi�notation��Thus,�we�have�a�tabular� method�for�depicting�the�data�of�a�two-variable�situation�in�Table�10�1�
A�graphical�method�for�depicting�the�relationship�among�two�variables�is�to�plot�the� pair�of�scores�on�X�and�Y�for�each�individual�on�a�two-dimensional�figure�known�as�a� scatterplot�(or�scattergram)��Each�individual�has�two�scores�in�a�two-dimensional�coor- dinate�system,�denoted�by�(X,�Y)��For�example,�our�individual�1�has�the�paired�scores� of� (10,� 20)�� An� example� scatterplot� is� shown� in� Figure� 10�1�� The� X� axis� (the� horizontal�
Table 10.1
Layout�for�Correlational�Data
Individual X Y
1 X1�=�10 Y1�=�20 2 X2�=�12 Y2�=�28 3 X3�=�20 Y3�=�33 � � � � � � � � �
N XN�=�44 YN�=�65
20
Y
10 X FIGuRe 10.1 Scatterplot�
262 An Introduction to Statistical Concepts
axis�or�abscissa)�represents�the�values�for�variable�X,�and�the�Y�axis�(the�vertical�axis�or� ordinate)�represents�the�values�for�variable�Y��Each�point�on�the�scatterplot�represents�a� pair�of�scores�(X,�Y)�for�a�particular�individual��Thus,�individual�1�has�a�point�at�X�=�10� and�Y�=�20�(the�circled�point)��Points�for�other�individuals�are�also�shown��In�essence,� the�scatterplot�is�actually�a�bivariate�frequency�distribution��When�there�is�a�moderate� degree�of�relationship,�the�points�may�take�the�shape�of�an�ellipse�(i�e�,�a�football�shape� where� the� direction� of� the� relationship,� positive� or� negative,� may� make� the� football� appear�to�point�up�to�the�right—as�with�a�positive�relation�depicted�in�this�figure),�as� in�Figure�10�1�
The�scatterplot�allows�the�researcher�to�evaluate�both�the�direction�and�the�strength�of� the�relationship�among�X�and�Y��The�direction�of�the�relationship�has�to�do�with�whether� the�relationship�is�positive�or�negative��A�positive�relationship�occurs�when�as�scores�on� variable�X�increase�(from�left�to�right),�scores�on�variable�Y�also�increase�(from�bottom�to� top)��Thus,�Figure�10�1�indicates�a�positive�relationship�among�X�and�Y��Examples�of�dif- ferent�scatterplots�are�shown�in�Figure�10�2��Figure�10�2a�and�d�displays�positive�relation- ships��A�negative�relationship,�sometimes�called�an�inverse�relationship,�occurs�when�as� scores�on�variable�X�increase�(from�left�to�right),�scores�on�variable�Y�decrease�(from�top�to� bottom)��Figure�10�2b�and�e�shows�examples�of�negative�relationships��There�is�no�relation- ship�between�X�and�Y�when�for�a�large�value�of�X,�a�large�or�a�small�value�of�Y�can�occur,� and�for�a�small�value�of�X,�a�large�or�a�small�value�of�Y�can�also�occur��In�other�words,�X� and�Y�are�not�related,�as�shown�in�Figure�10�2c�
The� strength� of� the� relationship� among� X� and� Y� is� determined� by� the� scatter� of� the� points� (hence� the� name� scatterplot)�� First,� we� draw� a� straight� line� through� the� points� which�cuts�the�bivariate�distribution�in�half,�as�shown�in�Figures�10�1�and�10�2��In�Chapter�17,� we�note�that�this�line�is�known�as�the�regression�line��If�the�scatter�is�such�that�the�points� tend�to�fall�close�to�the�line,�then�this�is�indicative�of�a�strong�relationship�among�X�and�Y�� Figure�10�2a�and�b�denotes�strong�relationships��If�the�scatter�is�such�that�the�points�are� widely�scattered�around�the�line,�then�this�is�indicative�of�a�weak�relationship�among�
(e)
(c)(a)
(d)
(b)
FIGuRe 10.2 Examples�of�possible�scatterplots�
263Bivariate Measures of Association
X� and� Y�� Figure� 10�2d� and� e� denotes� weak� relationships�� To� summarize� Figure� 10�2,� part�(a)�represents�a�strong�positive�relationship,�part�(b)�a�strong�negative�relationship,�part� (c)� no� relationship,� part� (d)� a� weak� positive� relationship,� and� part� (e)� a� weak� negative� relationship��Thus,�the�scatterplot�is�useful�for�providing�a�quick�visual�indication�of�the� nature�of�the�relationship�among�variables�X�and�Y�
10.2 Covariance
The�remainder�of�this�chapter�deals�with�statistical�methods�for�measuring�the�relationship� among�variables�X�and�Y��The�first�such�method�is�known�as�the�covariance��The�covariance� conceptually� is� the� shared� variance� (or� co-variance)� among� X� and� Y�� The� covariance� and� correlation� share� commonalities� as� the� correlation� is� simply� the� standardized� covariance�� The�population�covariance�is�denoted�by�σXY,�and�the�conceptual�formula�is�given�as�follows:
σ µ µ
XY
i X i Y
i
N
X Y
N =
− − =
∑( )( ) 1
where Xi�and�Yi�are�the�scores�for�individual�i�on�variables�X�and�Y,�respectively μX�and�μY�are�the�population�means�for�variables�X�and�Y,�respectively N�is�the�population�size
This� equation� looks� similar� to� the� computational� formula� for� the� variance� presented� in� Chapter�3,�where�deviation�scores�from�the�mean�are�computed�for�each�individual��The� conceptual� formula� for� the� covariance� is� essentially� an� average� of� the� paired� deviation� score�products��If�variables�X�and�Y�are�positively�related,�then�the�deviation�scores�will� tend�to�be�of�the�same�sign,�their�products�will�tend�to�be�positive,�and�the�covariance�will� be�a�positive�value�(i�e�,�σXY�>�0)��If�variables�X�and�Y�are�negatively�related,�then�the�devia- tion�scores�will�tend�to�be�of�opposite�signs,�their�products�will�tend�to�be�negative,�and� the�covariance�will�be�a�negative�value�(i�e�,�σXY�<�0)��Finally,�if�variables�X�and�Y�are�not� related,�then�the�deviation�scores�will�consist�of�both�the�same�and�opposite�signs,�their� products�will�be�both�positive�and�negative�and�sum�to�0,�and�the�covariance�will�be�a�zero� value�(i�e�,�σXY�=�0)�
The� sample� covariance� is� denoted� by� sXY,� and� the� conceptual� formula� becomes� as� follows:
s
X X Y Y
n XY
i i
i
n
= − −
− =
∑( )( ) 1
1
where X –
�and�Y – �are�the�sample�means�for�variables�X�and�Y,�respectively
n�is�sample�size
264 An Introduction to Statistical Concepts
Note� that� the� denominator� becomes� n� −� 1� so� as� to� yield� an� unbiased� sample� esti- mate�of�the�population�covariance�(i�e�,�similar�to�what�we�did�in�the�sample�variance� situation)�
The� conceptual� formula� is� unwieldy� and� error� prone� for� other� than� small� samples�� Thus,� a� computational� formula� for� the� population� covariance� has� been� developed� as� seen�here:
σXY
i i
i
N
i
i
N
i
i
N
N X Y X Y
N =
−
= = = ∑ ∑ ∑
1 1 1 2
where�the�first�summation�involves�the�cross�product�of�X�multiplied�by�Y�for�each�indi- vidual� summed� across� all� N� individuals,� and� the� other� terms� should� be� familiar�� The� computational�formula�for�the�sample�covariance�is�the�following:
s
n X Y X Y
n n XY
i i
i
n
i
i
n
i
i
n
=
−
− = = =
∑ ∑ ∑ 1 1 1
1( )
where�the�denominator�is�n(n�−�1)�so�as�to�yield�an�unbiased�sample�estimate�of�the�popula- tion�covariance�
Table�10�2�gives�an�example�of�a�population�situation�where�a�strong�positive�relation- ship�is�expected�because�as�X�(number�of�children�in�a�family)�increases,�Y�(number�of�pets� in�a�family)�also�increases��Here�σXY�is�computed�as�follows:
σXY
i i
i
N
i
i
N
i
i
N
N X Y X Y
N =
−
= −= = =
∑ ∑ ∑ 1 1 1 5 108
2
( ) (115 30 25
3 6000 )( )
.=
The�sign�indicates� that�the�relationship�between�X�and�Y�is�indeed�positive��That�is,�the� more�children�a�family�has,�the�more�pets�they�tend�to�have��However,�like�the�variance,�
Table 10.2
Example�Correlational�Data�(X�=�#�Children,�Y�=�#�Pets)
Individual X Y XY X 2 Y 2 Rank X Rank Y (Rank X − Rank Y )2
1 1 2 2 1 4 1 1 0 2 2 6 12 4 36 2 3 1 3 3 4 12 9 16 3 2 1 4 4 8 32 16 64 4 4 0 5 5 10 50 25 100 5 5 0
Sums 15 30 108 55 220 2
265Bivariate Measures of Association
the� value� of� the� covariance� depends� on� the� scales� of� the� variables� involved�� Thus,� inter- pretation�of�the�magnitude�of�a�single�covariance�is�difficult,�as�it�can�take�on�literally�any� value�� We� see� shortly� that� the� correlation� coefficient� takes� care� of� this� problem�� For� this� reason,� you� are� only� likely� to� see� the� covariance� utilized� in� the� analysis� of� covariance� (Chapter� 14)� and� advanced� techniques� such� as� structural� equation� modeling� and� multi- level�modeling�(beyond�the�scope�of�this�text)�
10.3 Pearson Product–Moment Correlation Coefficient
Other�methods�for�measuring�the�relationship�among�X�and�Y�have�been�developed�that� are�easier�to�interpret�than�the�covariance��We�refer�to�these�measures�as�correlation coeffi- cients��The�first�correlation�coefficient�we�consider�is�the�Pearson product–moment corre- lation coefficient,�developed�by�the�famous�statistician�Karl�Pearson,�and�simply�referred� to�as�the�Pearson�here��The�Pearson�can�be�considered�in�several�different�forms,�where�the� population�value�is�denoted�by�ρXY�(rho)�and�the�sample�value�by�rXY��One�conceptual�form� of�the�Pearson�is�a�product�of�standardized�z�scores�(previously�described�in�Chapter�4)�� This�formula�for�the�Pearson�is�given�as�follows:
ρXY X Y
i
N
z z
N = = ∑( )
1
where�zX�and�zY�are�the�z�scores�for�variables�X�and�Y,�respectively,�whose�product�is�taken� for�each�individual,�and�then�summed�across�all�N�individuals�
Because�z�scores�are�standardized�versions�of�raw�scores,�then�the�Pearson�correla- tion�is�simply�a�standardized�version�of�the�covariance��The�sign�of�the�Pearson�denotes� the�direction�of�the�relationship�(e�g�,�positive�or�negative),�and�the�value�of�the�Pearson� denotes� the� strength� of� the� relationship�� The� Pearson� falls� on� a� scale� from� −1�00� to� +1�00,� where� −1�00� indicates� a� perfect� negative� relationship,� 0� indicates� no� relation- ship,� and� +1�00� indicates� a� perfect� positive� relationship�� Values� near� �50� or� −�50� are� considered�as�moderate�relationships,�values�near�0�as�weak�relationships,�and�values� near�+1�00�or�−1�00�as�strong�relationships�(although�these�are�subjective�terms)��Cohen� (1988)� also� offers� rules� of� thumb,� which� are� presented� later� in� this� chapter,� for� inter- preting� the� value� of� the� correlation�� As� you� may� see� as� you� read� more� statistics� and� research� methods� textbooks,� there� are� other� guidelines� offered� for� interpreting� the� value�of�the�correlation�
There� are� other� forms� of� the� Pearson�� A� second� conceptual� form� of� the� Pearson� is� in� terms�of�the�covariance�and�the�standard�deviations�and�is�given�as�follows:
ρ σ
σ σ XY
XY
X Y =
266 An Introduction to Statistical Concepts
This�form�is�useful�when�the�covariance�and�standard�deviations�are�already�known��A�final� form�of�the�Pearson�is�the�computational�formula,�written�as�follows:
ρXY
i i
i
N
i
i
N
i
i
N
i
i
N
N X Y X Y
N X
=
−
= = =
=
∑ ∑ ∑
∑ 1 1 1
2
1
−
−
= = = ∑ ∑ ∑X N Y Yi i
N
i
i
N
i
i
N
1
2
2
1 1
2
where�all�terms�should�be�familiar�from�the�computational�formulas�of�the�variance�and� covariance��This�is�the�formula�to�use�for�hand�computations,�as�it�is�more�error-free�than� the�other�previously�given�formulas�
For�the�example�children-pet�data�given�in�Table�10�2,�we�see�that�the�Pearson�correlation� is�computed�as�follows:
ρXY
i i
i
N
i
i
N
i
i
N
i
i
N
N X Y X Y
N X
=
−
= = =
=
∑ ∑ ∑
∑ 1 1 1
2
1
−
−
= = = ∑ ∑ ∑X N Y Yi i
N
i
i
N
i
i
N
1
2
2
1 1
2
= −
− − =
5 108 15 30
5 55 15 5 220 302 2 ( ) ( )( )
( ) ( ) ( ) ( ) .99000
Thus,�there�is�a�very�strong�positive�relationship�among�variables�X�(the�number�of�chil- dren)�and�Y�(the�number�of�pets)�
The�sample�correlation�is�denoted�by�rXY��The�formulas�are�essentially�the�same�for�the� sample�correlation�rXY�and�the�population�correlation�ρXY,�except�that�n�is�substituted�for�N�� For�example,�the�computational�formula�for�the�sample�correlation�is�noted�here:
r
n X Y X Y
n X
XY
i i
i
n
i
i
n
i
i
n
i
i
n =
−
= = =
=
∑ ∑ ∑
∑ 1 1 1
2
1
−
−
= = = ∑ ∑ ∑X n Y Yi i
n
i
i
n
i
i
n
1
2
2
1 1
2
Unlike�the�sample�variance�and�covariance,�the�sample�correlation�has�no�correction�for�bias�
10.4 Inferences About Pearson Product–Moment Correlation Coefficient
Once�a�researcher�has�determined�one�or�more�Pearson�correlation�coefficients,�it�is�often� useful�to�know�whether�the�sample�correlations�are�significantly�different�from�0��Thus,� we�need�to�visit�the�world�of�inferential�statistics�again��In�this�section,�we�consider�two�
267Bivariate Measures of Association
different� inferential� tests:� first� for� testing� whether� a� single� sample� correlation� is� signifi- cantly�different�from�0�and�second�for�testing�whether�two�independent�sample�correla- tions�are�significantly�different�
10.4.1 Inferences for a Single Sample
Our�first�inferential�test�is�appropriate�when�you�are�interested�in�determining�whether� the� correlation� among� variables� X� and� Y� for� a� single� sample� is� significantly� different� from� 0�� For� example,� is� the� correlation� between� the� number� of� years� of� education� and� current�income�significantly�different�from�0?�The�test�of�inference�for�the�Pearson�cor- relation�will�be�conducted�following�the�same�steps�as�those�in�previous�chapters��The� null�hypothesis�is�written�as
H0 0: ρ =
A�nondirectional�alternative�hypothesis,�where�we�are�willing�to�reject�the�null�if�the�sam- ple�correlation�is�either�significantly�greater�than�or�less�than�0,�is�nearly�always�utilized�� Unfortunately,�the�sampling�distribution�of�the�sample�Pearson�r�is�too�complex�to�be�of� much� value� to� the� applied� researcher�� For� testing� whether� the� correlation� is� different� from�0�(i�e�,�where�the�alternative�hypothesis�is�specified�as�H1:�ρ�≠�0),�a�transformation�of�r� can�be�used�to�generate�a�t-distributed�test�statistic��The�test�statistic�is
t r n
r =
− −
2 1 2
which�is�distributed�as�t�with�ν�=�n�−�2�degrees�of�freedom,�assuming�that�both�X�and�Y� are�normally�distributed�(although�even�if�one�variable�is�normal�and�the�other�is�not,�the� t�distribution�may�still�apply;�see�Hogg�&�Craig,�1970)�
There�are�two�assumptions�with�the�Pearson�correlation��First,�the�Pearson�correlation� is� appropriate� only� when� there� is� a� linear� relationship� assumed� between� the� variables� (given�that�both�variables�are�at�least�interval�in�scale)��In�other�words,�when�a�curvilinear� or�some�type�of�polynomial�relationship�is�present,�the�Pearson�correlation�should�not�be� computed�� Testing� for� linearity� can� be� done� by� simply� graphing� a� bivariate� scatterplot� and�reviewing�it�for�a�general�linear�display�of�points��Also,�and�as�we�have�seen�with�the� other�inferential�procedures�discussed�in�previous�chapters,�we�need�to�again�assume�that� the�scores�of�the�individuals�are�independent�of�one�another��For�the�Pearson�correlation,�the� assumption� of� independence� is� met� when� a� random� sample� of� units� have� been� selected� from�the�population�
It� should� be� noted� for� inferential� tests� of� correlations� that� sample� size� plays� a� role� in� determining� statistical� significance�� For� instance,� this� particular� test� is� based� on� n� −� 2� degrees� of� freedom�� If� sample� size� is� small� (e�g�,� 10),� then� it� is� difficult� to� reject� the� null� hypothesis�except�for�very�strong�correlations��If�sample�size�is�large�(e�g�,�200),�then�it�is� easy� to� reject� the� null� hypothesis� for� all� but� very� weak� correlations�� Thus,� the� statistical� significance�of�a�correlation�is�definitely�a�function�of�sample�size,�both�for�tests�of�a�single� correlation�and�for�tests�of�two�correlations�
Effect�size�and�power�are�always�important,�particularly�here�where�sample�size�plays� such� a� large� role�� Cohen� (1988)� proposed� using� r� as� a� measure� of� effect� size,� using� the� subjective�standard�(ignoring�the�sign�of�the�correlation)�of�r�=��1�as�a�weak�effect,�r�=��3�
268 An Introduction to Statistical Concepts
as�a�moderate�effect,�and�r�=��5�as�a�strong�effect��These�standards�were�developed�for�the� behavioral�sciences,�but�other�standards�may�be�used�in�other�areas�of�inquiry��Cohen�also� has�a�nice�series�of�power�tables�in�his�Chapter�3�for�determining�power�and�sample�size� when�planning�a�correlational�study��As�for�confidence�intervals�(CIs),�Wilcox�(1996)�notes� that�“many�methods�have�been�proposed�for�computing�CIs�for�ρ,�but�it�seems�that�a�satis- factory�method�for�applied�work�has�yet�to�be�derived”�(p��303)��Thus,�a�CI�procedure�is�not� recommended,�even�for�large�samples�
From�the�example�children-pet�data,�we�want�to�determine�whether�the�sample�Pearson� correlation�is�significantly�different�from�0,�with�a�nondirectional�alternative�hypothesis� and�at�the��05�level�of�significance��The�test�statistic�is�computed�as�follows:
t r n
r =
− −
= −
− =
2 1
9000 5 2
1 8100 3 57622 . . .
The� critical� values� from� Table� A�2� are� ± = ±α2 3 3 182t . �� Thus,� we� would� reject� the� null� hypothesis,� as� the� test� statistic� exceeds� the� critical� value,� and� conclude� the� correlation� among�variables�X�and�Y�is�significantly�different�from�0��In�summary,�there�is�a�strong,� positive,�statistically�significant�correlation�between�the�number�of�children�and�the�num- ber�of�pets�
10.4.2 Inferences for Two Independent Samples
In�a�second�situation,�the�researcher�may�have�collected�data�from�two�different�indepen- dent�samples��It�can�be�determined�whether�the�correlations�among�variables�X�and�Y�are� equal�for�these�two�independent�samples�of�observations��For�example,�is�the�correlation� among�height�and�weight�the�same�for�children�and�adults?�Here�the�null�and�alternative� hypotheses�are�written�as
H
H
0 1 2
1 1 2
0
0
:
:
ρ ρ
ρ ρ
− =
− ≠
where�ρ1�is�the�correlation�among�X�and�Y�for�sample�1�and�ρ2�is�the�correlation�among�X� and�Y�for�sample�2��However,�because�correlations�are�not�normally�distributed�for�every� value�of�ρ,�a�transformation�is�necessary��This�transformation�is�known�as�Fisher’s�Z�trans- formation,� named� after� the� famous� statistician� Sir� Ronald� A�� Fisher,� which� is� approxi- mately� normally� distributed� regardless� of� the� value� of� ρ�� Table� A�5� is� used� to� convert� a� sample�correlation�r�to�a�Fisher’s�Z�transformed�value��Note�that�Fisher’s�Z�is�a�totally�dif- ferent�statistic�from�any�z�score�or�z�statistic�previously�covered�
The�test�statistic�for�this�situation�is
z Z Z
n n
= −
− +
−
1 2
1 2
1 3
1 3
where n1�and�n2�are�the�sizes�of�the�two�samples Z1�and�Z2�are�the�Fisher’s�Z�transformed�values�for�the�two�samples
269Bivariate Measures of Association
The� test� statistic� is� then�compared� to� critical� values� from� the� z� distribution� in� Table� A�1�� For� a� nondirectional� alternative� hypothesis� where� the� two� correlations� may� be� different� in� either� direction,� the� critical� values� are� ± α2z�� Directional� alternative� hypotheses� where� the�correlations�are�different�in�a�particular�direction�can�also�be�tested�by�looking�in�the� appropriate�tail�of�the�z�distribution�(i�e�,�either�+ α1 z�or�− α1 z)�
Cohen�(1988)�proposed�a�measure�of�effect�size�for�the�difference�between�two�indepen- dent�correlations�as�q�=�Z1�−�Z2��The�subjective�standards�proposed�(ignoring�the�sign)�are� q�=��1�as�a�weak�effect,�q�=��3�as�a�moderate�effect,�and�q�=��5�as�a�strong�effect�(these�are�the� standards�for�the�behavioral�sciences,�although�standards�vary�across�disciplines)��A�nice� set�of�power�tables�for�planning�purposes�is�contained�in�Chapter�4�of�Cohen��Once�again,� while�CI�procedures�have�been�developed,�none�of�these�have�been�viewed�as�acceptable� (Marascuilo�&�Serlin,�1988;�Wilcox,�2003)�
Consider� the� following� example�� Two� samples� have� been� independently� drawn� of� 28� children� (sample� 1)� and� 28� adults� (sample� 2)�� For� each� sample,� the� correlations� among� height�and�weight�were�computed�to�be�rchildren�=��8�and�radults�=��4��A�nondirectional�alter- native�hypothesis�is�utilized�where�the�level�of�significance�is�set�at��05��From�Table�A�5,�we� first�determine�the�Fisher’s�Z�transformed�values�to�be�Zchildren�=�1�099�and�Zadults�=��4236�� Then�the�test�statistic�z�is�computed�as�follows:
z Z Z
n n
= −
− +
−
= −
+ =1 2
1 2
1 3
1 3
1 099 4236 1 25
1 25
2 3878 . .
.
From�Table�A�1,�the�critical�values�are�± = ±α2 1 96z . ��Our�decision�then�is�to�reject�the�null� hypothesis�and�conclude�that�height�and�weight�do�not�have�the�same�correlation�for�chil- dren�and�adults��In�other�words,�there�is�a�statistically�significant�difference�of�the�height- weight�correlation�between�children�and�adults�with�a�strong�effect�size�(q�=��6754)��This� inferential�test�assumes�both�variables�are�normally�distributed�for�each�population�and� that�scores�are�independent�across�individuals;�however,�the�procedure�is�not�very�robust� to� nonnormality� as� the� Z� transformation� assumes� normality� (Duncan� &� Layard,� 1973;� Wilcox,�2003;�Yu�&�Dunn,�1982)��Thus,�caution�should�be�exercised�in�using�the�z�test�when� data�are�nonnormal�(e�g�,�Yu�&�Dunn�recommend�the�use�of�Kendall’s�τ�as�discussed�later� in�this�chapter)�
10.5 Assumptions and Issues Regarding Correlations
There�are�several�issues�about�the�Pearson�and�other�types�of�correlations�that�you�should� be�aware�of��These�issues�are�concerned�with�the�assumption�of�linearity,�correlation�and� causation,�and�restriction�of�range�
10.5.1 assumptions
First,� as� mentioned� previously,� the� Pearson� correlation� assumes� that� the� relationship� among�X�and�Y�is�a�linear relationship.�In�fact,�the�Pearson�correlation,�as�a�measure�of� relationship,�is�really�a�linear�measure�of�relationship��Recall�from�earlier�in�the�chapter�
270 An Introduction to Statistical Concepts
the� scatterplots� to� which� we� fit� a� straight� line�� The� linearity� assumption� means� that� a� straight�line�provides�a�reasonable�fit�to�the�data��If�the�relationship�is�not�a�linear�one,� then�the�linearity�assumption�is�violated��However,�these�correlational�methods�can�still� be�computed,�fitting�a�straight�line�to�the�data,�albeit�inappropriately��The�result�of�such� a� violation� is� that� the� strength� of� the� relationship� will� be� reduced�� In� other� words,� the� linear�correlation�will�be�much�closer�to�0�than�the�true�nonlinear�relationship�
For�example,�there�is�a�perfect�curvilinear�relationship�shown�by�the�data�in�Figure�10�3,� where�all�of�the�points�fall�precisely�on�the�curved�line��Something�like�this�might�occur�if� you�correlate�age�with�time�in�the�mile�run,�as�younger�and�older�folks�would�take�longer� to�run�this�distance�than�others��If�these�data�are�fit�by�a�straight�line,�then�the�correlation� will�be�severely�reduced,�in�this�case,�to�a�value�of�0�(i�e�,�the�horizontal�straight�line�that� runs�through�the�curved�line)��This�is�another�good�reason�to�always�examine�your�data�� The� computer� may� determine� that� the� Pearson� correlation� among� variables� X� and� Y� is� small�or�around�0��However,�on�examination�of�the�data,�you�might�find�that�the�relation- ship�is�indeed�nonlinear;�thus,�you�should�get�to�know�your�data��We�return�to�the�assess- ment�of�nonlinear�relationships�in�Chapter�17�
Second,�the�assumption�of�independence�applies�to�correlations��This�assumption�is�met� when�units�or�cases�are�randomly�sampled�from�the�population�
10.5.2 Correlation and Causality
A�second�matter�to�consider�is�an�often-made�misinterpretation�of�a�correlation��Many�indi- viduals�(e�g�,�researchers,�the�public,�and�the�media)�often�infer�a�causal�relationship�from�a� strong�correlation��However,�a�correlation�by�itself�should�never�be�used�to�infer�causation�� In�particular,�a�high�correlation�among�variables�X�and�Y�does�not�imply�that�one�variable�is� causing�the�other;�it�simply�means�that�these�two�variables�are�related�in�some�fashion��There� are�many�reasons�why�variables�X�and�Y�are�highly�correlated��A�high�correlation�could�be� the�result�of�(a)�X�causing�Y,�(b)�Y�causing�X,�(c)�a�third�variable�Z�causing�both�X�and�Y,�or� (d)�even�many�more�variables�being�involved��The�only�methods�that�can�strictly�be�used�to� infer�cause�are�experimental�methods�that�employ�random�assignment�where�one�variable� is� manipulated� by� the� researcher� (the� cause),� a� second� variable� is� subsequently� observed� (the�effect),�and�all�other�variables�are�controlled��[There�are,�however,�some�excellent�quasi- experimental�methods,�propensity�score�analysis�and�regression�discontinuity,�that�can�be� used�in�some�situations�and�that�mimic�random�assignment�and�increase�the�likelihood�of� speaking�to�causal�inference�(Shadish,�Cook,�&�Campbell,�2002)�]
FIGuRe 10.3 Nonlinear�relationship�
Y
X
271Bivariate Measures of Association
10.5.3 Restriction of Range
A�final�issue�to�consider�is�the�effect�of�restriction of the range�of�scores�on�one�or�both� variables��For�example,�suppose�that�we�are�interested�in�the�relationship�among�GRE� scores�and�graduate�grade�point�average�(GGPA)��In�the�entire�population�of�students,� the� relationship� might� be� depicted� by� the� scatterplot� shown� in� Figure� 10�4�� Say� the� Pearson�correlation�is�found�to�be��60�as�depicted�by�the�entire�sample�in�the�full�scat- terplot��Now�we�take�a�more�restricted�population�of�students,�those�students�at�highly� selective� Ivy-Covered� University� (ICU)�� ICU� only� admits� students� whose� GRE� scores� are�above�the�cutoff�score�shown�in�Figure�10�4��Because�of�restriction�of�range�in�the� scores�of�the�GRE�variable,�the�strength�of�the�relationship�among�GRE�and�GGPA�at� ICU�is�reduced�to�a�Pearson�correlation�of��20,�where�only�the�subsample�portion�of�the� plot�to�the�right�of�the�cutoff�score�is�involved��Thus,�when�scores�on�one�or�both�vari- ables�are�restricted�due�to�the�nature�of�the�sample�or�population,�then�the�magnitude� of�the�correlation�will�usually�be�reduced�(although�see�an�exception�in�Figure�6�3�from� Wilcox,�2003)�
It�is�difficult�for�two�variables�to�be�highly�related�when�one�or�both�variables�have�little� variability��This�is�due�to�the�nature�of�the�formula��Recall�that�one�version�of�the�Pearson� formula�consisted�of�standard�deviations�in�the�denominator��Remember�that�the�standard� deviation�measures�the�distance�of�the�sample�scores�from�the�mean��When�there�is�restric- tion�of�range,�the�distance�of�the�individual�scores�from�the�mean�is�minimized��In�other� words,� there� is� less� variation� or� variability� around� the� mean�� This� translates� to� smaller� correlations�(and�smaller�covariances)��If�the�size�of�the�standard�deviation�for�one�variable� is�reduced,�everything�else�being�equal,�then�the�size�of�correlations�with�other�variables� will�also�be�reduced��In�other�words,�we�need�sufficient�variation�for�a�relationship�to�be� evidenced�through�the�correlation�coefficient�value��Otherwise�the�correlation�is�likely�to� be�reduced�in�magnitude,�and�you�may�miss�an�important�correlation��If�you�must�use�a� restrictive�subsample,�we�suggest�you�choose�measures�of�greater�variability�for�correla- tional�purposes�
Outliers,�observations�that�are�different�from�the�bulk�of�the�observations,�also�reduce� the� magnitude� of� correlations�� If� one� observation� is� quite� different� from� the� rest� such� that�it�fell�outside�of�the�ellipse,�then�the�correlation�would�be�smaller�in�magnitude�(e�g�,� closer�to�0)�than�the�correlation�without�the�outlier��We�discuss�outliers�in�this�context� in�Chapter�17�
GGPA
Cuto� GRE FIGuRe 10.4 Restriction�of�range�example�
272 An Introduction to Statistical Concepts
10.6 Other Measures of Association
Thus�far,�we�have�considered�one�type�of�correlation,�the�Pearson�product–moment�cor- relation� coefficient�� The� Pearson� is� most� appropriate� when� both� variables� are� at� least� interval�level��That�is,�both�variables�X�and�Y�are�interval-�and/or�ratio-level�variables�� The�Pearson�is�considered�a�parametric�procedure�given�the�distributional�assumptions� associated� with� it�� If� both� variables� are� not� at� least� interval� level,� then� other� measures� of� association,� considered� nonparametric� procedures,� should� be� considered� as� they� do� not�have�distributional�assumptions�associated�with�them��In�this�section,�we�examine� in� detail� the� Spearman’s� rho� and� phi� types� of� correlation� coefficients� and� briefly� men- tion�several�other�types��While�a�distributional�assumption�for�these�correlations�is�not� necessary,�the�assumption�of�independence�still�applies�(and�thus�a�random�sample�from� the�population�is�assumed)�
10.6.1 Spearman’s Rho
Spearman’s�rho�rank�correlation�coefficient�is�appropriate�when�both�variables�are�ordinal� level�� This� type� of� correlation� was� developed� by� Charles� Spearman,� the� famous� quanti- tative� psychologist�� Recall� from� Chapter� 1� that� ordinal� data� are� where� individuals� have� been�rank-ordered,�such�as�class�rank��Thus,�for�both�variables,�either�the�data�are�already� available�in�ranks,�or�the�researcher�(or�computer)�converts�the�raw�data�to�ranks�prior�to� the�analysis�
The�equation�for�computing�Spearman’s�rho�correlation�is
ρS i i
i
N
X Y
N N = −
−
− =
∑ 1
6
1
2
1 2
( )
( )
where ρS�denotes�the�population�Spearman’s�rho�correlation (Xi�−�Yi)�represents�the�difference�between�the�ranks�on�variables�X�and�Y�for�individual�i�
The�sample�Spearman’s�rho�correlation�is�denoted�by�rS�where�n�replaces�N,�but�other- wise�the�equation�remains�the�same��In�case�you�were�wondering�where�the�“6”�in�the� equation�comes�from,�you�will�find�an�interesting�article�by�Lamb�(1984)��Unfortunately,� this�particular�computational�formula�is�only�appropriate�when�there�are�no�ties�among� the�ranks�for�either�variable��An�example�of�a�tie�in�rank�would�be�if�two�cases�scored�the� same�value�on�either�X�or�Y��With�ties,�the�formula�given�is�only�approximate,�depending� on�the�number�of�ties��In�the�case�of�ties,�particularly�when�there�are�more�than�a�few,� many�researchers�recommend�using�Kendall’s�τ�(tau)�as�an�alternative�correlation�(e�g�,� Wilcox,�1996)�
As� with� the� Pearson� correlation,� Spearman’s� rho� ranges� from� −1�0� to� +1�0�� The� rules� of� thumb�that�we�used�for�interpreting�the�Pearson�correlation�(e�g�,�Cohen,�1988)�can�be�applied� to�Spearman’s�rho�correlation�values�as�well��The�sign�of�the�coefficient�can�be�interpreted�as� with�the�Pearson��A�negative�sign�indicates�that�as�the�values�for�one�variable�increase,�the� values�for�the�other�variable�decrease��A�positive�sign�indicates�that�as�one�variable�increases� in�value,�the�value�of�the�second�variable�also�increases�
273Bivariate Measures of Association
As�an�example,�consider�the�children-pets�data�again�in�Table�10�2��To�the�right�of�the�table,� you� see� the� last� three� columns� labeled� as� rank� X,� rank� Y,� and� (rank� X� −� rank� Y)2�� The� raw� scores�were�converted�to�ranks,�where�the�lowest�raw�score�received�a�rank�of�1��The�last�col- umn�lists�the�squared�rank�differences��As�there�were�no�ties,�the�computations�are�as�follows:
ρS i i
i
N
X Y
N N = −
−
− = − ==
∑ 1
6
1 1
6 2 5 24
9000
2
1 2
( )
( ) ( )
( ) .
Thus,�again�there�is�a�strong�positive�relationship�among�variables�X�and�Y��It�is�a�coincidence� that�ρ�=�ρS�for�this�dataset,�but�not�so�for�computational�problem�1�at�the�end�of�this�chapter�
To�test�whether�a�sample�Spearman’s�rho�correlation�is�significantly�different�from�0,� we� examine� the� following� null� hypothesis� (the� alternative� hypothesis� would� be� stated� as�H1:�ρS�≠�0):
H S0 0: ρ =
The�test�statistic�is�given�as
t r n
r S
S
= −
− 2
1 2
which�is�approximately�distributed�as�a�t�distribution�with�ν�=�n�−�2�degrees�of�freedom� (Ramsey,�1989)��The�approximation�works�best�when�n�is�at�least�10��A�nondirectional�alter- native�hypothesis,�where�we�are�willing�to�reject�the�null�if�the�sample�correlation�is�either� significantly�greater�than�or�less�than�0,�is�nearly�always�utilized��From�the�example,�we� want�to�determine�whether�the�sample�Spearman’s�rho�correlation�is�significantly�different� from�0�at�the��05�level�of�significance��For�a�nondirectional�alternative�hypothesis,�the�test� statistic�is�computed�as
t r n
r S
S
= −
− =
− −
= 2
1
9000 5 2 1 81
3 5762 2
. .
.
where�the�critical�values�from�Table�A�2�are�± = ±α2 3 3 182t . ��Thus,�we�would�reject�the�null� hypothesis� and� conclude� that� the� correlation� is� significantly� different� from� 0,� strong� in� magnitude�(suggested�by�the�value�of�the�correlation�coefficient;�using�Cohen’s�guidelines� for� interpretation� as� an� effect� size,� this� would� be� considered� a� large� effect),� and� positive� in� direction� (evidenced� from� the� sign� of� the� correlation� coefficient)�� The� exact� sampling� distribution�for�when�3�≤�n�≤�18�is�given�by�Ramsey�
10.6.2 kendall’s Tau
Another�correlation�that�can�be�computed�with�ordinal�data�is�Kendall’s�tau,�which�also� uses�ranks�of�data�to�calculate�the�correlation�coefficient�(and�has�an�adjustment�for�tied� ranks)�� The� ranking� for� Kendall’s� tau� differs� from� Spearman’s� rho� in� the� following� way��
274 An Introduction to Statistical Concepts
With�Kendall’s�tau,�the�values�for�one�variable�are�rank-ordered,�and�then�the�order�of�the� second�variable�is�examined�to�see�how�many�pairs�of�values�are�out�of�order��A�perfect� positive�correlation�(+1�0)�is�achieved�with�Kendall’s�tau�when�no�scores�are�out�of�order,� and�a�perfect�negative�correlation�(−1�0)�is�obtained�when�all�scores�are�out�of�order��Values� for�Kendall’s�tau�range�from�−1�0�to�+1�0��The�rules�of�thumb�that�we�used�for�interpreting� the�Pearson�correlation�(e�g�,�Cohen,�1988)�can�be�applied�to�Kendall’s�tau�correlation�val- ues�as�well��The�sign�of�the�coefficient�can�be�interpreted�as�with�the�Pearson:�A�negative� sign�indicates�that�as�the�values�for�one�variable�increase,�the�values�for�the�second�vari- able�decrease��A�positive�sign�indicates�that�as�one�variable�increases�in�value,�the�value� of�the�second�variable�also�increases��While�similar�in�some�respects,�Spearman’s�rho�and� Kendall’s�tau�are�based�on�different�calculations,�and,�thus,�finding�different�results�is�not� uncommon�� While� both� are� appropriate� when� ordinal� data� are� being� correlated,� it� has� been�suggested�that�Kendall’s�tau�provides�a�better�estimation�of�the�population�correla- tion�coefficient�value�given�the�sample�data�(Howell,�1997),�especially�with�smaller�sample� sizes�(e�g�,�n�≤�10)�
10.6.3 phi
The�phi�coefficient�ϕ�is�appropriate�when�both�variables�are�dichotomous�in�nature�(and�is� statistically� equivalent� to� the� Pearson)�� Recall� from� Chapter� 1� that� a� dichotomous� variable� is�one�consisting�of�only�two�categories�(i�e�,�binary),�such�as�gender,�pass/fail,�or�enrolled/ dropped�out��Thus,�the�variables�being�correlated�would�be�either�nominal�or�ordinal�in�scale�� When�correlating�two�dichotomous�variables,�one�can�think�of�a�2�×�2�contingency�table�as� previously�discussed�in�Chapter�8��For�instance,�to�determine�if�there�is�a�relationship�among� gender�and�whether�students�are�still�enrolled�since�their�freshman�year,�a�contingency�table� like�Table�10�3�can�be�constructed��Here�the�columns�correspond�to�the�two�levels�of�the�enroll- ment�status�variable,�enrolled�(coded�1)�or�dropped�out�(0),�and�the�rows�correspond�to�the� two�levels�of�the�gender�variable,�female�(1)�or�male�(0)��The�cells�indicate�the�frequencies�for� the�particular�combinations�of�the�levels�of�the�two�variables��If�the�frequencies�in�the�cells�are� denoted�by�letters,�then�a�represents�females�who�dropped�out,�b�represents�females�who� are�enrolled,�c�indicates�males�who�dropped�out,�and�d�indicates�males�who�are�enrolled�
The�equation�for�computing�the�phi�coefficient�is
ρφ = −
+ + + + ( )
( )( )( )( ) bc ad
a c b d a b c d
where�ρϕ�denotes�the�population�phi�coefficient�(for�consistency’s�sake,�although�typically� written�as�ϕ),�and�rϕ�denotes�the�sample�phi�coefficient�using�the�same�equation��Note�that�
Table 10.3
Contingency�Table�for�Phi�Correlation
Enrollment Status
Student Gender Dropped Out (0) Enrolled (1)
Female�(1) a�=�5 b�=�20 a�+�b�=�25 Male�(0) c�=�15 d�=�10 c�+�d�=�25
a�+�c�=�20 b�+�d�=�30 a�+�b�+�c�+�d�=�50
275Bivariate Measures of Association
the�bc�product�involves�the�consistent�cells,�where�both�values�are�the�same,�either�both�0�or� both�1,�and�the�ad�product�involves�the�inconsistent�cells,�where�both�values�are�different�
Using�the�example�data�from�Table�10�3,�we�compute�the�phi�coefficient�to�be�the�following:
ρφ = −
+ + + + =
− =
( ) ( )( )( )( )
( ) ( )( )( )( )
. bc ad
a c b d a b c d 300 50
20 30 25 25 40082
Thus,�there�is�a�moderate,�positive�relationship�between�gender�and�enrollment�status��We� see�from�the�table�that�a�larger�proportion�of�females�than�males�are�still�enrolled�
To� test� whether� a� sample� phi� correlation� is� significantly� different� from� 0,� we� test� the� following�null�hypothesis�(the�alternative�hypothesis�would�be�stated�as�H1:�ρϕ�≠�0):
H0 0: ρφ =
The�test�statistic�is�given�as
χ φ 2 2= nr
which�is�distributed�as�a�χ2�distribution�with�one�degree�of�freedom��From�the�example,� we�want�to�determine�whether�the�sample�phi�correlation�is�significantly�different�from�0� at�the��05�level�of�significance��The�test�statistic�is�computed�as
χ φ 2 2 250 4082 8 3314= = =nr (. ) .
and�the�critical�value�from�Table�A�3�is�. .05 1 2 3 84χ = ��Thus,�we�would�reject�the�null�hypoth-
esis�and�conclude�that�the�correlation�among�gender�and�enrollment�status�is�significantly� different�from�0�
10.6.4 Cramer’s phi
When�the�variables�being�correlated�have�more�than�two�categories,�Cramer’s�phi�(Cramer’s�V� in�SPSS)�can�be�computed��Thus,�Cramer’s�phi�is�appropriate�when�both�variables�are�nominal� (and�at�least�one�variable�has�more�than�two�categories)�or�when�one�variable�is�nominal�and� the�other�variable�is�ordinal�(and�at�least�one�variable�has�more�than�two�categories)�� As� with�the�other�correlation�coefficients�that�we�have�discussed,�values�range�from�−1�0�to� +1�0�� Cohen’s� guidelines� (1988)� for� interpreting� the� correlation� in� terms� of� effect� size� can� be� applied�to�Cramer’s�phi�correlations,�as�they�can�with�any�other�correlation�examined�
10.6.5 Other Correlations
Other� types� of� correlations� have� been� developed� for� different� combinations� of� types� of� variables,�but�these�are�rarely�used�in�practice�and�are�unavailable�in�most�statistical�pack- ages�(e�g�,�rank�biserial�and�point�biserial)��Table�10�4�provides�suggestions�for�when�dif- ferent�types�of�correlations�are�most�appropriate��We�mention�briefly�the�two�other�types� of�correlations�in�the�table:�the�rank�biserial�correlation�is�appropriate�when�one�variable� is�dichotomous�and�the�other�variable�is�ordinal,�whereas�the�point�biserial�correlation�is� appropriate�when�one�variable�is�dichotomous�and�the�other�variable�is�interval�or�ratio� (statistically�equivalent�to�the�Pearson;�thus,�the�Pearson�correlation�can�be�computed�in� this�situation)�
276 An Introduction to Statistical Concepts
10.7 SPSS
Next� let� us� see� what� SPSS� has� to� offer� in� terms� of� measures� of� association� using� the� children-pets�example�dataset��There�are�two�programs�for�obtaining�measures�of�asso- ciation� in� SPSS,� dependent� on� the� measurement� scale� of� your� variables—the� Bivariate� Correlation�program�(for�computing�the�Pearson,�Spearman’s�rho,�and�Kendall’s�tau)�and� the�Crosstabs�program�(for�computing�the�Pearson,�Spearman’s�rho,�Kendall’s�tau,�phi,� Cramer’s�phi,�and�several�other�types�of�measures�of�association)�
Bivariate Correlations
Step 1:�To�locate�the�Bivariate�Correlations�program,�we�go�to�“Analyze”�in�the�top�pull- down�menu,�then�select�“Correlate,”�and�then�“Bivariate.”�Following�the�screenshot� (step�1),�as�follows,�produces�the�“Bivariate”�dialog�box�
A
B C
Bivariate correlations:
Step 1
Table 10.4
Different�Types�of�Correlation�Coefficients
Variable X
Variable Y Nominal Ordinal Interval/Ratio
Nominal Phi�(when�both�variables�are� dichotomous)�or�Cramer’s�V� (when�one�or�both�variables�have� more�than�two�categories)
Rank�biserial�or� Cramer’s�V
Point�biserial�(Pearson� in lieu�of�point�biserial)
Ordinal Rank�biserial�or�Cramer’s�V Spearman’s�rho�or� Kendall’s�tau
Spearman’s�rho�or� Kendall’s�tau�or�Pearson
Interval/ratio Point�biserial�(Pearson�in�lieu� of point�biserial)
Spearman’s�rho�or� Kendall’s�tau�or� Pearson
Pearson
277Bivariate Measures of Association
Step 2:�Next,�from�the�main�“Bivariate Correlations”�dialog�box,�click�the�variables� to�correlate�(e�g�,�number�of�children�and�pets)�and�move�them�into�the�“Variables”�box� by�clicking�on�the�arrow�button��In�the�bottom�half�of�this�dialog�box,�options�are�available� for�selecting�the�type�of�correlation,�one-�or�two-tailed�test�(i�e�,�directional�or�nondirectional� test),�and�whether�to�flag�statistically�significant�correlations��For�illustrative�purposes,�we� will� place� a� checkmark� to� generate� the�“Pearson”� and�“Spearman’s rho”� correlation� coefficients��We�will�also�select�the�radio�button�for�a�“Two-tailed”�test�of�significance,�and� at�the�very�bottom�check,�we�will�“Flag significant correlations”�(which�simply� means�an�asterisk�will�be�placed�next�to�significant�correlations�in�the�output)�
Select the variables of
interest from the list on the left and use the arrow to
move to the “Variables” box
on the right.
Clicking on “Options” will
allow you to obtain the means, standard deviations, and/or
covariances. Place a checkmark in the
box that corresponds to the type of correlation to
generate. This decision will be based on the
measurement scale of your variables.
“Test of significance” selected is based on a non- directional (two-tailed) or
directional (one-tailed) test.
“Flag significant correlations” will generate asterisks in the output for statistically
significant correlations.
Bivariate correlations:
Step 2
Step 3 (optional):� To� obtain� means,� standard� deviations,� and/or� covariances,� as� well�as�options�for�dealing�with�missing�data�(listwise�or�pairwise�deletion),�click�on�the� “Options”�button�located�in�the�top�right�corner�of�the�main�dialog�box�
Step 3
278 An Introduction to Statistical Concepts
From� the� main� dialog� box,� click� on� “Ok”� to� run� the� analysis� and� to� generate� the� output�
Interpreting the output:�The�output�for�generation�of�the�Pearson�and�Spearman’s� rho� bivariate� correlations� between� number� of� children� and� number� of� pets� appears� in� Table� 10�5�� For� illustrative� purposes,� we� asked� for� both� the� Pearson� and� Spearman’s� rho� correlations�(although�the�Pearson�is�the�appropriate�correlation�given�the�measurement� scales�of�our�variables,�we�have�also�generated�the�Spearman’s�rho�so�that�the�output�can� be�reviewed)��Thus,�the�top�Correlations�box�gives�the�Pearson�results�and�the�bottom� Correlations�box�the�Spearman’s�rho�results��In�both�cases,�the�output�presents�the�cor- relation,�sample�size�(N�in�SPSS�language,�although�usually�denoted�as�n�by�everyone�else),� observed�level�of�significance,�and�asterisks�denoting�statistically�significant�correlations�� In� reviewing� Table� 10�5,� we� see� that� SPSS� does� not� provide� any� output� in� terms� of� CIs,� power,� or� effect� size�� Later� in� the� chapter,� we� illustrate� the� use� of� G*Power� for� comput- ing�power��Effect�size�is�easily�interpreted�from�the�correlation�coefficient�value�utilizing� Cohen’s�(1988)�subjective�standards�previously�described,�and�we�have�not�recommended� any�CI�procedures�for�correlations�
Table 10.5
SPSS�Results�for�Child—Pet�Data
The bivariate Pearson correlations are presented in the top row. The value of “1” indicates the Pearson correlation of the variable with itself. The correlation of interest (relationship of number of children to number of pets) is .900.
The asterisk indicates the correlation is statistically significant at an alpha of .05.
�e probability is less than 4% (see “Sig. (two-tailed)”)
that we would see this relationship by random
chance if the relationship between variables was zero (i.e., if the null hypothesis
was really true).
N represents the total sample size.
�e bottom half of the table presents the same
information as that presented in the top half.
�e results for the same data
computed with Spearman’s rho are presented here and
interpreted similarly.
Children
Pets
Pearson correlation Sig. (two-tailed) N Pearson correlation Sig. (two-tailed) N
Correlation coe�cient
Correlations Children Pets
Sig. (two-tailed) N Correlation coe�cient Sig. (two-tailed) N
* Correlation is significant at the 0.05 level (two-tailed).
Children Correlations
Pets .900*
5 1
5
1
5
5 .037
.900*
.037
* Correlation is significant at the 0.05 level (two-tailed).
Pets
ChildrenSpearman’s rho 1.000 .
5 .900*
.037 5
5
.900* .037
5 .
1.000
279Bivariate Measures of Association
Using Scatterplots to Examine Linearity for Bivariate Correlations
Step 1:�As�alluded�to�earlier�in�the�chapter,�understanding�the�extent�to�which�linear- ity� is� a� reasonable� assumption� is� an� important� first� step� prior� to� computing� a� Pearson� correlation� coefficient�� To� generate� a� scatterplot,� go� to� “Graphs”� in� the� top� pulldown� menu��From�there,�select�“Legacy Dialogs,”�then�“Scatter/Dot”�(see�screenshot�for� “Scatterplots: Step 1”)�
A
B
C
Scatterplots: Step 1
Step 2:� This� will� bring� up� the� “Scatter/Dot”� dialog� box� (see� screenshot� for� “Scatterplots: Step 2”).�The�default�selection�is�“Simple Scatter,”�and�this�is� the�option�we�will�use��Then�click�“Define.”
Scatterplots: Step 2
Step 3:� This� will� bring� up� the�“Simple Scatterplot”� dialog� box� (see� screenshot� for�“Scatterplots: Step 3”)��Click�the�dependent�variable�(e�g�,�number�of�pets)�and� move�it�into�the�“Y�Axis”�box�by�clicking�on�the�arrow��Click�the�independent�variable� (e�g�,� number� of� children)� and� move� it� into� the�“X� Axis”� box� by� clicking� on� the� arrow�� Then�click�“Ok.”
280 An Introduction to Statistical Concepts
Scatterplots: Step 3
Interpreting linearity evidence:� Scatterplots� are� also� often� examined� to� deter- mine�visual�evidence�of�linearity�prior�to�computing�Pearson�correlations��Scatterplots�are� graphs�that�depict�coordinate�values�of�X�and�Y��Linearity�is�suggested�by�points�that�fall�in� a�straight�line��This�line�may�suggest�a�positive�relation�(as�scores�on�X�increase,�scores�on�Y� increase,�and�vice�versa),�a�negative�relation�(as�scores�on�X�increase,�scores�on�Y�decrease,� and� vice� versa),� little� or� no� relation� (relatively� random� display� of� points),� or� a� polynomial� relation�(e�g�,�curvilinear)��In�this�example,�our�scatterplot�suggests�evidence�of�linearity�and,� more�specifically,�a�positive�relationship�between�number�of�children�and�number�of�pets�� Thus,�proceeding�to�compute�a�bivariate�Pearson�correlation�coefficient�is�reasonable�
10.00
8.00
6.00
4.00
2.00
1.00 2.00 3.00 Number of children
4.00 5.00
N um
be r o
f p et
s
281Bivariate Measures of Association
Using Crosstabs to Compute Correlations
The�Crosstabs�program�has�already�been�discussed�in�Chapter�8,�but�it�can�also�be�used� for� obtaining� many� measures� of� association� (specifically� Spearman’s� rho,� Kendall’s� tau,� Pearson,�phi�and�Cramer’s�phi)��We�will�illustrate�the�use�of�Crosstabs�for�two�nominal� variables,�thus�generating�phi�and�Cramer’s�phi�
Step 1:�To�compute�phi�or�Cramer’s�phi�correlations,�go�to�“Analyze”�in�the�top�pull- down,� then� select� “Descriptive Statistics,”� and� then� select� the� “Crosstabs”� procedure�
A
B
C
Phi and Cramers’s phi:
Step 1
Step 2:�Select�the�dependent�variable�(if�applicable;�many�times,�there�are�not�depen- dent�and�independent�variables,�per�se,�with�bivariate�correlations,�and�in�those�cases,� determining� which� variable� is� X� and� which� variable� is� Y� is� largely� irrelevant)� and� move�it�into�the�“Row(s)”�box�by�clicking�on�the�arrow�key�[e�g�,�here�we�used�enroll- ment�status�as�the�dependent�variable�(1�=�enrolled;�0�=�not�enrolled)]��Then�select�the� independent�variable�and�move�it�into�the�“Column(s)”�box�[in�this�example,�gender� is�the�independent�variable�(0�=�male;�1�=�female)]�
282 An Introduction to Statistical Concepts
Clicking on “Statistics” will
allow you to select various statistics to generate
(including various measures of association).
Select the variable of interest from
the list on the left and use the arrow
to move to the boxes on the right. If applicable, the
dependent variable should be displayed in the row(s) and the independent variable in the
column(s).
Phi and Cramers’s phi:
Step 2
Step 3:�In�the�top�right�corner�of�the�“Crosstabs”�dialog�box�(see�screenshot�step�2),� click� on� the� button� labeled�“Statistics.”� From� here,� you� can� select� various� measures� of�association�(i�e�,�types�of�correlation�coefficients)��Which�correlation�is�selected�should� depend� on� the� measurement� scales� of� your� variables�� With� two� nominal� variables,� the� appropriate�correlation�to�select� is�“Phi and Cramer’s V.”� Click�on�“Continue”�to� return�to�the�main�“Crosstabs”�dialog�box�
Clicking on “Correlations” will
generate Pearson, Spearman’s rho, and
Kendall’s tau correlations.
Phi and Cramer’s phi:
Step 3
From�the�main�dialog�box,�click�on�“Ok”�to�run�the�analysis�and�generate�the�output�
283Bivariate Measures of Association
10.8 G*Power
A� priori� and� post� hoc� power� could� again� be� determined� using� the� specialized� software� described�previously�in�this�text�(e�g�,�G*Power),�or�you�can�consult�a�priori�power�tables� (e�g�,�Cohen,�1988)��As�an�illustration,�we�use�G*Power�to�compute�the�post�hoc�power�of� our�test�
Post Hoc Power for the Pearson Bivariate Correlation Using G*Power
The�first�thing�that�must�be�done�when�using�G*Power�for�computing�post�hoc�power�is�to� select�the�correct�test�family��In�our�case,�we�conducted�a�Pearson�correlation��To�find�the� Pearson,�we�will�select�“Tests”�in�the�top�pulldown�menu,�then�“Correlations and regression,”� and� then�“Correlations: Bivariate normal model.”� Once� that� selection�is�made,�the�“Test family”�automatically�changes�to�“Exact.”
A
B C
Step 1
284 An Introduction to Statistical Concepts
The�“Type of power analysis”�desired�then�needs�to�be�selected��To�compute� post hoc�power,�select�“Post hoc: Compute achieved power—given�α,�sample size, and effect size.”
�e default selection for “Test Family” is“t tests.” Following the procedures presented in Step 1 will automatically change the test family to “exact.”
�e default selection for “Statistical Test” is “Correlation: Point biserial model.” Following the procedures presented in Step 1 will automatically change the statistical test to “correlation: bivariate normal model.”
Step 2
The� “Input Parameters”� must� then� be� specified�� The� first� parameter� is� specifica- tion� of� the� number� of� tail(s)�� For� a� directional� hypothesis,� “One”� is� selected,� and� for� a� nondirectional�hypothesis,�“Two”�is�selected��In�our�example,�we�chose�a�nondirectional� hypothesis�and�thus�will�select�“Two”�tails��We�then�input�the�observed�correlation�coef- ficient�value�in�the�box�for�“Correlation�ρ�H1�”�In�this�example,�our�Pearson�correlation� coefficient�value�was��90��The�alpha�level�we�tested�at�was��05,�the�total�sample�size�was�5,� and� the�“Correlation� ρ�H0”� will� remain� as� the� default� 0� (this� is� the� correlation� value� expected�if�the�null�hypothesis�is�true;�in�other�words,�there�is�zero�correlation�between� variables� given� the� null� hypothesis)�� Once� the� parameters� are� specified,� simply� click� on� “Calculate”�to�generate�the�power�results�
285Bivariate Measures of Association
The “Input Parameters” for computing post hoc power must be specified for: 1. One or two tailed test 2. Observed correlation coefficient value 3. Alpha level 4. Total sample size 5. Hypothesized correlation coefficient value
Once the parameters are
specified, click on “Calculate.”
The�“Output Parameters”�provide�the�relevant�statistics�given�the�input�just�speci- fied��In�this�example,�we�were�interested�in�determining�post�hoc�power�for�a�Pearson�cor- relation�given�a�two-tailed�test,�with�a�computed�correlation�value�of��90,�an�alpha�level�of� �05,�total�sample�size�of�5,�and�a�null�hypothesis�correlation�value�of�0�
Based� on� those� criteria,� the� post� hoc� power� was� �67�� In� other� words,� with� a� two-tailed� test,�an�observed�Pearson�correlation�of��90,�an�alpha�level�of��05,�sample�size�of�5,�and�a� null� hypothesis� correlation� value� of� 0,� the� power� of� our� test� was� �67—the� probability� of� rejecting�the�null�hypothesis�when�it�is�really�false�(in�this�case,�the�probability�that�there� is�not�a�zero�correlation�between�our�variables)�was�67%,�which�is�slightly�less�than�what� would�be�usually�considered�sufficient�power�(sufficient�power�is�often��80�or�above)��Keep� in�mind�that�conducting�power�analysis�a�priori�is�recommended�so�that�you�avoid�a�situ- ation�where,�post�hoc,�you�find�that�the�sample�size�was�not�sufficient�to�reach�the�desired� level�of�power�(given�the�observed�parameters)�
286 An Introduction to Statistical Concepts
10.9 Template and APA-Style Write-Up
Finally�we�conclude�the�chapter�with�a�template�and�an�APA-style�paragraph�detailing�the� results�from�an�example�dataset�
Pearson Correlation Test
As� you� may� recall,� our� graduate� research� assistant,� Marie,� was� working� with� the� mar- keting� director� of� the� local� animal� shelter,� Matthew�� Marie’s� task� was� to� assist� Matthew� in�generating�the�test�of�inference�to�answer�his�research�question,�“Is there a relationship between the number of children in a family and the number of pets”?�A�Pearson�correlation�was� the�test�of�inference�suggested�by�Marie��A�template�for�writing�a�research�question�for�a� correlation�(regardless�of�which�type�of�correlation�coefficient�is�computed)�is�presented� in�the�following:
Is There a Correlation Between [Variable 1] and [Variable 2]?
It� may� be� helpful� to� include� in� the� results� information� on� the� extent� to� which� the� assumptions�were�met�(recall�there�are�two�assumptions:�independence�and�linearity)�� This�assists�the�reader�in�understanding�that�you�were�thorough�in�data�screening�prior� to�conducting�the�test�of�inference��Recall�that�the�assumption�of�independence�is�met� when�the�cases�in�our�sample�have�been�randomly�selected�from�the�population��One� or�two�sentences�are�usually�sufficient�to�indicate�if�the�assumptions�are�met��It�is�also� important� to� address� effect� size� in� the� write-up�� Correlations� are� unique� in� that� they� are� already� effect� size� measures,� so� computing� an� effect� size� in� addition� to� the� cor- relation�value�is�not�needed��However,�it�is�desirable�to�interpret�the�correlation�value� as� an�effect�size�� Effect�size�is�easily�interpreted�from�the�correlation�coefficient�value� utilizing�Cohen’s�(1988)�subjective�standards�previously�described��Here�is�an�APA-style� example�paragraph�of�results�for�the�correlation�between�number�of�children�and�num- ber�of�pets�
A Pearson correlation coefficient was computed to determine if there is a relationship between the number of children in a family and the number of pets in the family. The test was conducted using an alpha of .05. The null hypothesis was that the relationship would be 0. The assumption of independence was met via random selection. The assump- tion of linearity was reasonable given a review of a scatterplot of the variables.
The Pearson correlation between children and pets is .90, which is positive, is interpreted as a large effect size (Cohen, 1988), and is statistically different from 0 (r =�.90,�n�= 5, p = .037).�Thus, the null hypothesis that the correlation is 0 was rejected at the .05 level of significance. There is a strong, positive correlation between the number of children in a family and the number of pets in the family.
287Bivariate Measures of Association
10.10 Summary
In�this�chapter,�we�described�various�measures�of�the�association�or�correlation�among�two� variables�� Several� new� concepts� and� descriptive� and� inferential� statistics� were� discussed�� The� new� concepts� covered� were� as� follows:� scatterplot;� strength� and� direction;� covariance;� correlation� coefficient;� Fisher’s� Z� transformation;� and� linearity� assumption,� causation,� and� restriction�of�range�issues��We�began�by�introducing�the�scatterplot�as�a�graphical�method�for� visually�depicting�the�association�among�two�variables��Next�we�examined�the�covariance�as� an�unstandardized�measure�of�association��Then�we�considered�the�Pearson�product–moment� correlation�coefficient,�first�as�a�descriptive�statistic�and�then�as�a�method�for�making�infer- ences�when�there�are�either�one�or�two�samples�of�observations��Some�important�issues�about� the�correlational�measures�were�also�discussed��Finally,�a�few�other�measures�of�association� were�introduced,�in�particular,�the�Spearman’s�rho�and�Kendall’s�tau�rank-order�correlation� coefficients�and�the�phi�and�Cramer’s�phi�coefficients��At�this�point,�you�should�have�met�the� following�objectives:�(a)�be�able�to�understand�the�concepts�underlying�the�correlation�coef- ficient�and�correlation�inferential�tests,�(b)�be�able�to�select�the�appropriate�type�of�correlation,� and�(c)�be�able�to�determine�and�interpret�the�appropriate�correlation�and�correlation�inferen- tial�test��In�Chapter�11,�we�discuss�the�one-factor�analysis�of�variance,�the�logical�extension�of� the�independent�t�test,�for�assessing�mean�differences�among�two�or�more�groups�
Problems
Conceptual problems
10.1� The�variance�of�X�is�9,�the�variance�of�Y�is�4,�and�the�covariance�between�X�and�Y�is�2�� What�is�rXY?
� a�� �039 � b�� �056 � c�� �233 � d�� �333 10.2� The�standard�deviation�of�X�is�20,�the�standard�deviation�of�Y�is�50,�and�the�covari-
ance�between�X�and�Y�is�30��What�is�rXY? � a�� �030 � b�� �080 � c�� �150 � d�� �200 10.3� Which�of�the�following�correlation�coefficients,�each�obtained�from�a�sample�of�1000�
children,�indicates�the�weakest�relationship? � a�� −�90 � b�� −�30 � c�� +�20 � d�� +�80
288 An Introduction to Statistical Concepts
10.4� �Which�of�the�following�correlation�coefficients,�each�obtained�from�a�sample�of�1000� children,�indicates�the�strongest�relationship?
� a�� −�90 � b�� −�30 � c�� +�20 � d�� +�80 10.5� �If�the�relationship�between�two�variables�is�linear,�which�of�the�following�is�neces-
sarily�true? � a�� The�relation�can�be�most�accurately�represented�by�a�straight�line� � b�� All�the�points�will�fall�on�a�curved�line� � c�� The�relationship�is�best�represented�by�a�curved�line� � d�� All�the�points�must�fall�exactly�on�a�straight�line� 10.6� �In� testing� the� null� hypothesis� that� a� correlation� is� equal� to� 0,� the� critical� value�
decreases�as�α�decreases��True�or�false? 10.7� �If�the�variances�of�X�and�Y�are�increased,�but�their�covariance�remains�constant,�the�
value�of�rXY�will�be�unchanged��True�or�false? 10.8� �We�compute�rXY�=��50�for�a�sample�of�students�on�variables�X�and�Y��I�assert�that�if�
the�low-scoring�students�on�variable�X�are�removed,�then�the�new�value�of�rXY�would� most�likely�be�less�than��50��Am�I�correct?
10.9� �Two�variables�are�linearly�related�such�that�there�is�a�perfect�relationship�between�X� and�Y��I�assert�that�rXY�must�be�equal�to�either�+1�00�or�−1�00��Am�I�correct?
10.10� �If� the� number� of� credit� cards� owned� and� the� number� of� cars� owned� are� strongly positively�correlated,�then�those�with�more�credit�cards�tend�to�own�more�cars��True� or�false?
10.11� �If� the� number� of� credit� cards� owned� and� the� number� of� cars� owned� are� strongly negatively� correlated,� then� those� with� more� credit� cards� tend� to� own� more� cars�� True�or�false?
10.12� �A�statistical�consultant�at�a�rival�university�found�the�correlation�between�GRE-Q� scores� and� statistics� grades� to� be� +2�0�� I� assert� that� the� administration� should� be� advised�to�congratulate�the�students�and�faculty�on�their�great�work�in�the�class- room��Am�I�correct?
10.13� �If� X� correlates� significantly� with� Y,� then� X� is� necessarily� a� cause� of� Y�� True� or� false?
10.14� �A�researcher�wishes�to�correlate�the�grade�students�earned�from�a�pass/fail�course� (i�e�,�pass�or�fail)�with�their�cumulative�GPA��Which�is�the�most�appropriate�correla- tion�coefficient�to�examine�this�relationship?
� a�� Pearson � b�� Spearman’s�rho�or�Kendall’s�tau � c�� Phi � d�� None�of�the�above 10.15� �If�both�X�and�Y�are�ordinal�variables,�then�the�most�appropriate�measure�of�associa-
tion�is�the�Pearson��True�or�false?
289Bivariate Measures of Association
Computational problems
10.1� You�are�given�the�following�pairs�of�sample�scores�on�X�(number�of�credit�cards�in� your�possession)�and�Y�(number�of�those�credit�cards�with�balances):
X Y
5 4 6 1 4 3 8 7 2 2
� a�� Graph�a�scatterplot�of�the�data� � b�� Compute�the�covariance� � c�� Determine�the�Pearson�product–moment�correlation�coefficient� � d�� Determine�the�Spearman’s�rho�correlation�coefficient� 10.2� If� rXY� =� �17� for� a� random� sample� of� size� 84,� test� the� hypothesis� that� the� population�
Pearson�is�significantly�different�from�0�(conduct�a�two-tailed�test�at�the��05�level�of� significance)�
10.3� If� rXY� =� �60� for� a� random� sample� of� size� 30,� test� the� hypothesis� that� the� population� Pearson�is�significantly�different�from�0�(conduct�a�two-tailed�test�at�the��05�level�of� significance)�
10.4� The�correlation�between�vocabulary�size�and�mother’s�age�is��50�for�12�rural�children� and��85�for�17�inner-city�children��Does�the�correlation�for�rural�children�differ�from� that�of�the�inner-city�children�at�the��05�level�of�significance?
10.5� You�are�given�the�following�pairs�of�sample�scores�on�X�(number�of�coins�in�posses- sion)�and�Y�(number�of�bills�in�possession):
X Y
2 1 3 3
4 5 5 5 6 3 7 1
� a�� Graph�a�scatterplot�of�the�data� � b�� Describe�the�relationship�between�X�and�Y� � c�� What�do�you�think�the�Pearson�correlation�will�be?
290 An Introduction to Statistical Concepts
10.6� Six� adults� were� assessed� on� the� number� of� minutes� it� took� to� read� a� government� report�(X)�and�the�number�of�items�correct�on�a�test�of�the�content�of�that�report�(Y)�� Use�the�following�data�to�determine�the�Pearson�correlation�and�the�effect�size�
X Y
10 17 8 17
15 13 12 16 14 15 16 12
10.7� Ten�kindergarten�children�were�observed�on�the�number�of�letters�written�in�proper� form�(given�26�letters)�(X)�and�the�number�of�words�that�the�child�could�read�(given� 50�words)�(Y)��Use�the�following�data�to�determine�the�Pearson�correlation�and�the� effect�size�
X Y
10 5 16 8 22 40 8 15
12 28 20 37 17 29 21 30 15 18 9 4
Interpretive problems
10.1� Select�two�interval/ratio�variables�from�the�survey�1�dataset�on�the�website��Use�SPSS� to�generate�the�appropriate�correlation,�determine�statistical�significance,�interpret�the� correlation�value�(including�interpretation�as�an�effect�size),�and�examine�and�inter- pret�the�scatterplot�
10.2� Select� two� ordinal� variables� from� the� survey� 1� dataset� on� the� website�� Use� SPSS� to�generate�the�appropriate�correlation,�determine�statistical�significance,�interpret� the�correlation�value�(including�interpretation�as�an�effect�size),�and�examine�and� interpret�the�scatterplot�
10.3� Select�one�ordinal�variable�and�one�interval/ratio�variable�from�the�survey�1�dataset� on�the�website��Use�SPSS�to�generate�the�appropriate�correlation,�determine�statisti- cal�significance,�interpret�the�correlation�value�(including�interpretation�as�an�effect� size),�and�examine�and�interpret�the�scatterplot�
10.4� Select� one� dichotomous� variable� and� one� interval/ratio� variable� from� the� survey� 1� dataset�on�the�website��Use�SPSS�to�generate�the�appropriate�correlation,�determine� statistical�significance,�interpret�the�correlation�value�(including�interpretation�as�an� effect�size),�and�examine�and�interpret�the�scatterplot�
291
11 One-Factor Analysis of Variance: Fixed-Effects Model
Chapter Outline
11�1� Characteristics�of�the�One-Factor�ANOVA�Model 11�2� Layout�of�Data 11�3� ANOVA�Theory � 11�3�1� General�Theory�and�Logic � 11�3�2� Partitioning�the�Sums�of�Squares � 11�3�3� ANOVA�Summary�Table 11�4� ANOVA�Model � 11�4�1� Model � 11�4�2� Estimation�of�the�Parameters�of�the�Model � 11�4�3� Effect�Size�Measures,�Confidence�Intervals,�and�Power � 11�4�4� Example � 11�4�5� Expected�Mean�Squares 11�5� Assumptions�and�Violation�of�Assumptions � 11�5�1� Independence � 11�5�2� Homogeneity�of�Variance � 11�5�3� Normality 11�6� Unequal�n’s�or�Unbalanced�Design 11�7� Alternative�ANOVA�Procedures � 11�7�1� Kruskal–Wallis�Test � 11�7�2� Welch,�Brown–Forsythe,�and�James�Procedures 11�8� SPSS�and�G*Power 11�9� Template�and�APA-Style�Write-Up
Key Concepts
� 1�� Between-�and�within-groups�variability � 2�� Sources�of�variation � 3�� Partitioning�the�sums�of�squares � 4�� The�ANOVA�model � 5�� Expected�mean�squares
292 An Introduction to Statistical Concepts
In�the�last�five�chapters,�our�discussion�has�dealt�with�various�inferential�statistics,�includ- ing�inferences� about� means��The�next�six� chapters�are�concerned�with� different� analysis� of�variance�(ANOVA)�models��In�this�chapter,�we�consider�the�most�basic�ANOVA�model,� known� as� the� one-factor� ANOVA� model�� Recall� the� independent� t� test� from� Chapter� 7� where� the� means� from� two� independent� samples� were� compared�� What� if� you� wish� to� compare� more� than� two� means?� The� answer� is� to� use� the� analysis of variance�� At� this� point,�you�may�be�wondering�why�the�procedure�is�called�the�analysis�of�variance�rather� than�the�analysis�of�means,�because�the�intent�is�to�study�possible�mean�differences��One� way�of�comparing�a�set�of�means�is�to�think�in�terms�of�the�variability�among�those�means�� If�the�sample�means�are�all�the�same,�then�the�variability�of�those�means�would�be�0��If�the� sample�means�are�not�all�the�same,�then�the�variability�of�those�means�would�be�somewhat� greater�than�0��In�general,�the�greater�the�mean�differences�are,�the�greater�is�the�variabil- ity�of�the�means��Thus,�mean�differences�are�studied�by�looking�at�the�variability�of�the� means;�hence,�the�term�analysis�of�variance�is�appropriate�rather�than�analysis�of�means� (further�discussed�in�this�chapter)�
We� use� X� to� denote� our� single� independent variable,� which� we� typically� refer� to� as� a� factor,� and� Y� to� denote� our� dependent� (or� criterion)� variable�� Thus,� the� one-factor� ANOVA� is� a� bivariate,� or� two-variable,� procedure�� Our� interest� here� is� in� determin- ing�whether�mean�differences�exist�on�the�dependent�variable��Stated�another�way,�the� researcher� is� interested� in� the� influence� of� the� independent� variable� on� the� dependent� variable�� For� example,� a� researcher� may� want� to� determine� the� influence� that� method� of�instruction�has�on�statistics�achievement��The�independent�variable,�or�factor,�would� be� method� of� instruction� and� the� dependent� variable� would� be� statistics� achievement�� Three� different� methods� of� instruction� that� might� be� compared� are� large� lecture� hall� instruction,�small-group�instruction,�and�computer-assisted�instruction��Students�would� be�randomly�assigned�to�one�of�the�three�methods�of�instruction�and�at�the�end�of�the� semester�evaluated�as�to�their�level�of�achievement�in�statistics��These�results�would�be�of� interest�to�a�statistics�instructor�in�determining�the�most�effective�method�of�instruction� (where�“effective”�is�measured�by�student�performance�in�statistics)��Thus,�the�instructor� may�opt�for�the�method�of�instruction�that�yields�the�highest�mean�achievement�
There� are� a� number� of� new� concepts� introduced� in� this� chapter� as� well� as� a� refresher� of� concepts� that� have� been� covered� in� previous� chapters�� The� concepts� addressed� in� this� chapter� include� the� following:� independent� and� dependent� variables;� between-� and� within-groups�variability;�fixed�and�random�effects;�the�linear�model;�partitioning�of�the� sums�of�squares;�degrees�of�freedom,�mean�square�terms,�and�F�ratios;�the�ANOVA�sum- mary� table;� expected� mean� squares;� balanced� and� unbalanced� models;� and� alternative� ANOVA�procedures��Our�objectives�are�that�by�the�end�of�this�chapter,�you�will�be�able�to� (a)�understand�the�characteristics�and�concepts�underlying�a�one-factor�ANOVA,�(b)�gener- ate�and�interpret�the�results�of�a�one-factor�ANOVA,�and�(c)�understand�and�evaluate�the� assumptions�of�the�one-factor�ANOVA�
11.1 Characteristics of One-Factor ANOVA Model
We� have� been� following� Marie,� our� very� capable� educational� research� graduate� student,� as�she�develops�her�statistical�skills��As�we�will�see,�Marie�is�embarking�on�a�very�exciting� research�adventure�of�her�own�
293One-Factor Analysis of Variance: Fixed-Effects Model
Marie�is�enrolled�in�an�independent�study�class��As�part�of�the�course�requirement,�she� has�to�complete�a�research�study��In�collaboration�with�the�statistics�faculty�in�her�pro- gram,�Marie�designs�an�experimental�study�to�determine�if�there�is�a�mean�difference� in�student�attendance�in�the�statistics�lab�based�on�the�attractiveness�of�the�statistics�lab� instructor��Marie’s�research�question�is:�Is there a mean difference in the number of statistics labs attended by students based on the attractiveness of the lab instructor?�Marie�determined� that�a�one-way�ANOVA�was�the�best�statistical�procedure�to�use�to�answer�her�ques- tion��Her�next�task�is�to�collect�and�analyze�the�data�to�address�her�research�question�
This� section� describes� the� distinguishing� characteristics� of� the� one-factor� ANOVA� model�� Suppose�you�are�interested�in�comparing�the�means�of�two�independent�samples��Here�the� independent�t�test�would�be�the�method�of�choice�(or�perhaps�the�Welch�t′�test)��What�if�your� interest�is�in�comparing�the�means�of�more�than�two�independent�samples?�One�possibility� is�to�conduct�multiple�independent�t�tests�on�each�pair�of�means��For�example,�if�you�wished� to�determine�whether�the�means�from�five�independent�samples�are�the�same,�you�could�do� all�possible�pairwise�t�tests��In�this�case,�the�following�null�hypotheses�could�be�evaluated:� μ1�=�μ2,�μ1�=�μ3,�μ1�=�μ4,�μ1�=�μ5,�μ2�=�μ3,�μ2�=�μ4,�μ2�=�μ5,�μ3�=�μ4,�μ3�=�μ5,�and�μ4�=�μ5��Thus,�we� would�have�to�carry�out�10�different�independent�t�tests��The�number�of�possible�pairwise� t�tests�that�could�be�done�for�J�means�is�equal�to�½[J(J�−�1)]�
Is� there� a� problem� in� conducting�so�many�t� tests?� Yes;� the� problem� has� to�do� with� the� probability�of�making�a�Type�I�error�(i�e�,�α),�where�the�researcher�incorrectly�rejects�a�true� null�hypothesis��Although�the�α�level�for�each�t�test�can�be�controlled�at�a�specified�nominal� α�level�that�is�set�by�the�researcher,�say��05,�what�happens�to�the�overall�α�level�for�the�entire� set�of�tests?�The�overall�α�level�for�the�entire�set�of�tests�(i�e�,�αtotal),�often�called�the�experi- mentwise Type I error rate,�is�larger�than�the�α�level�for�each�of�the�individual�t�tests�
In�our�example,�we�are�interested�in�comparing�the�means�for�10�pairs�of�groups�(again,� these�would�be�μ1�=�μ2,�μ1�=�μ3,�μ1�=�μ4,�μ1�=�μ5,�μ2�=�μ3,�μ2�=�μ4,�μ2�=�μ5,�μ3�=�μ4,�μ3�=�μ5,�and� μ4�=�μ5)��A�t�test�is�conducted�for�each�of�the�10�pairs�of�groups�at�α�=��05��Although�each� test�controls�the�α�level�at��05,�the�overall�α�level�will�be�larger�because�the�risk�of�a�Type� I�error�accumulates�across�the�tests��For�each�test,�we�are�taking�a�risk;�the�more�tests�we� do,�the�more�risks�we�are�taking��This�can�be�explained�by�considering�the�risk�you�take� each�day�you�drive�your�car�to�school�or�work��The�risk�of�an�accident�is�small�for�any�1�day;� however,�over�the�period�of�a�year,�the�risk�of�an�accident�is�much�larger�
For�C�independent�(or�orthogonal)�tests,�the�experimentwise�error�is�as�follows:
α αtotal = − −1 1( ) C
Assume�for�the�moment�that�our�10�tests�are�independent�(although�they�are�not�because� within� those� 10� tests,� each� group� is� actually� being� compared� to� another� group� in� four� different�instances)��If�we�go�ahead�with�our�10�t�tests�at�α�=��05,�then�the�experimentwise� error�rate�is
α total = − − = − =1 1 05 1 60 40 10( . ) . .
Although�we�are�seemingly�controlling�our�α�level�at�the��05�level,�the�probability�of�making�a� Type�I�error�across�all�10�tests�is��40��In�other�words,�in�the�long�run,�if�we�conduct�10�indepen- dent�t�tests,�4�times�out�of�10,�we�will�make�a�Type�I�error��For�this�reason,�we�do�not�want�to�do�
294 An Introduction to Statistical Concepts
all�possible�t�tests��Before�we�move�on,�the�experimentwise�error�rate�for�C�dependent�tests�αtotal� (which�would�be�the�case�when�doing�all�possible�pairwise�t�tests,�as�in�our�example)�is�more� difficult�to�determine,�so�let�us�just�say�that
α α α≤ ≤total C
Are� there� other� options� available� to� us� where� we� can� maintain� better� control� over� our� experimentwise�error�rate?�The�optimal�solution,�in�terms�of�maintaining�control�over� our� overall� α� level� as� well� as� maximizing� power,� is� to� conduct� one� overall� test,� often� called� an� omnibus test�� Recall� that� power� has� to� do� with� the� probability� of� correctly� rejecting� a� false� null� hypothesis�� The� omnibus� test� could� assess� the� equality� of� all� of� the�means�simultaneously�and�is�the�one�used�in�ANOVA��The�one-factor�ANOVA�then� represents� an� extension� of� the� independent� t� test� for� two� or� more� independent� sample� means,�where�the�experimentwise�error�rate�is�controlled�
In�addition,�the�one-factor�ANOVA�has�only�one�independent�variable�or�factor�with�two� or� more� levels�� The� independent� variable� is� a� discrete� or� grouping� variable,� where� each� subject� responds� to� only� one� level�� The� levels� represent� the� different� samples� or� groups� or�treatments�whose�means�are�to�be�compared��In�our�example,�method�of�instruction�is� the�independent�variable�with�three�levels:�large�lecture�hall,�small-group,�and�computer- assisted�� There� are� two� ways� of� conceptually� thinking� about� the� selection� of� levels�� In� the� fixed-effects� model,� all� levels� that� the� researcher� is� interested� in� are� included� in� the� design� and� analysis� for� the� study�� As� a� result,� generalizations� can� only� be� made� about� those�particular�levels�of�the�independent�variable�that�are�actually�selected��For�instance,� if�a�researcher�is�only�interested�in�these�three�methods�of�instruction—large�lecture�hall,� small-group,� and� computer-assisted—then� only� those� levels� are� incorporated� into� the� study�� Generalizations� about� other� methods� of� instruction� cannot� be� made� because� no� other�methods�were�considered�for�selection��Other�examples�of�fixed-effects�independent� variables� might� be� SES,� gender,� specific� types� of� drug� treatment,� age� group,� weight,� or� marital�status�
In�the�random-effects�model,�the�researcher�randomly�samples�some�levels�of�the�inde- pendent�variable�from�the�population�of�levels��As�a�result,�generalizations�can�be�made� about�all�of�the�levels�in�the�population,�even�those�not�actually�sampled��For�instance,�a� researcher�interested�in�teacher�effectiveness�may�have�randomly�sampled�history�teach- ers� (i�e�,� the� independent� variable)� from� the� population� of� history� teachers� in� a� particu- lar�school�district��Generalizations�can�then�be�made�about�other�history�teachers�in�that� school�district�not�actually�sampled��The�random�selection�of�levels�is�much�the�same�as� the� random� selection� of� individuals� or� objects� in� the� random� sampling� process�� This� is� the�nature�of�inferential�statistics,�where�inferences�are�made�about�a�population�(of�indi- viduals,�objects,�or�levels)�from�a�sample��Other�examples�of�random-effects�independent� variables� might� be� randomly� selected� classrooms,� types� of� medication,� animals,� or� time� (e�g�,�hours,�days)��The�remainder�of�this�chapter�is�concerned�with�the�fixed-effects�model�� Chapter�15�discusses�the�random-effects�model�in�more�detail�
In�the�fixed-effects�model,�once�the�levels�of�the�independent�variable�are�selected,�sub- jects�(i�e�,�persons�or�objects)�are�randomly�assigned�to�the�levels�of�the�independent�vari- able��In�certain�situations,�the�researcher�does�not�have�control�over�which�level�a�subject�is� assigned�to��The�groups�may�already�be�in�place�when�the�researcher�arrives�on�the�scene�� For�instance,�students�may�be�assigned�to�their�classes�at�the�beginning�of�the�year�by�the� school�administration��Researchers�typically�have�little�input�regarding�class�assignments��
295One-Factor Analysis of Variance: Fixed-Effects Model
In� another� situation,� it� may� be� theoretically� impossible� to� assign� subjects� to� groups�� For� example,�as�much�as�we�might�like,�researchers�cannot�randomly�assign�individuals�to�an� age� level�� Thus,� a� distinction� needs� to� be� made� about� whether� or� not� the� researcher� can� control� the� assignment� of� subjects� to� groups�� Although� the� analysis� will� not� be� altered,� the�interpretation�of�the�results�will�be��When�researchers�have�control�over�group�assign- ments,� the� extent� to� which� they� can� generalize� their� findings� is� greater� than� for� those� researchers� who� do� not� have� such� control�� For� further� information� on� the� differences� between�true experimental designs�(i�e�,�with�random�assignment)�and�quasi-experimental designs� (i�e�,� without� random� assignment),� take� a� look� at� Campbell� and� Stanley� (1966),� Cook�and�Campbell�(1979),�and�Shadish,�Cook,�and�Campbell�(2002)�
Moreover,�in�the�model�being�considered�here,�each�subject�is�exposed�to�only�one�level� of� the� independent� variable�� Chapter� 15� deals� with� models� where� a� subject� is� exposed� to� multiple� levels� of� an� independent� variable;� these� are� known� as� repeated-measures models��For�example,�a�researcher�may�be�interested�in�observing�a�group�of�young�chil- dren�repeatedly�over�a�period�of�several�years��Thus,�each�child�might�be�observed�every� 6� months� from� birth� to� 5� years� of� age�� This� would� require� a� repeated-measures� design� because� the� observations� of� a� particular� child� over� time� are� obviously� not� independent� observations�
One� final� characteristic� is� the� measurement� scale� of� the� independent� and� dependent� variables��In�ANOVA,�because�this�is�a�test�of�means,�a�condition�of�the�test�is�that�the�scale� of�measurement�on�the�dependent�variable�is�at�the�interval�or�ratio�level��If�the�dependent� variable�is�measured�at�the�ordinal�level,�then�the�nonparametric�equivalent,�the�Kruskal– Wallis�test,�should�be�considered�(discussed�later�in�this�chapter)��If�the�dependent�vari- able� shares� properties� of� both� the� ordinal� and� interval� levels� (e�g�,� grade� point� average� [GPA]),� then� both� the� ANOVA� and� Kruskal–Wallis� procedures� could� be� considered� to� cross-reference� any� potential� effects� of� the� measurement� scale� on� the� results�� As� previ- ously�mentioned,�the�independent�variable�is�a�grouping�or�discrete�variable,�so�it�can�be� measured�on�any�scale�
However,� there� is� one� caveat� to� the� measurement� scale� of� the� independent� variable�� Technically�the�condition�is�that�the�independent�variable�be�a�grouping�or�discrete�variable�� Most�often,�ANOVAs�are�conducted�with�independent�variables�which�are�categorical— nominal� or� ordinal� in� scale�� ANOVAs� can� also� be� used� in� the� case� of� interval� or� ratio� values�that�are�discrete��Recall�that�discrete�variables�are�variables�that�can�only�take�on� certain�values�and�that�arise�from�the�counting�process��An�example�of�a�discrete�variable� that�could�be�a�good�candidate�for�being�an�independent�variable�in�an�ANOVA�model�is� number�of�children��What�would�make�this�a�good�candidate?�The�responses�to�this�vari- able� would� likely� be� relatively� limited� (in� the� general� population,� it� may� be� anticipated� that�the�range�would�be�from�zero�children�to�five�or�six—although�outliers�may�be�a�pos- sibility),�and�each�discrete�value�would�likely�have�multiple�cases�(with�fewer�cases�having� larger� numbers� of� children)�� Applying� this� is� obviously� at� the� researcher’s� discretion;� at� some�point,�the�number�of�discrete�values�can�become�so�numerous�as�to�be�unwieldy�in� an�ANOVA�model��Thus,�while�at�first�glance�we�may�not�consider�it�appropriate�to�use� interval�or�ratio�variables�as�independent�variables�in�ANOVA�models,�there�are�situations� where�it�is�feasible�and�appropriate�
In�summary,�the�characteristics�of�the�one-factor�ANOVA�fixed-effects�model�are�as� follows:� (a)� control� of� the� experimentwise� error� rate� through� an� omnibus� test;� (b)� one� independent�variable�with�two�or�more�levels;�(c)�the�levels�of�the�independent�variable� are�fixed�by�the�researcher;�(d)�subjects�are�randomly�assigned�to�these�levels;�(e)�sub- jects�are�exposed�to�only�one�level�of�the�independent�variable;�and�(f)�the�dependent�
296 An Introduction to Statistical Concepts
variable� is� measured� at� least� at� the� interval� level,� although� the� Kruskal–Wallis� one- factor�ANOVA�can�be�considered�for�an�ordinal�level�dependent�variable��In�the�context� of� experimental� design,� the� one-factor� ANOVA� is� often� referred� to� as� the� completely randomized design�
11.2 Layout of Data
Before� we� get� into� the� theory� and� analysis� of� the� data,� let� us� examine� one� tabular� form� of� the� data,� known� as� the� layout� of� the� data�� We� designate� each� observation� as� Yij,� where� the� j� subscript� tells� us� what� group� or� level� the� observation� belongs� to� and� the�i�subscript�tells�us�the�observation�or�identification�number�within�that�group��For� instance,� Y34� would� mean� this� is� the� third� observation� in� the� fourth� group,� or� level,� of�the�independent�variable��The�first�subscript�ranges�over�i�=�1,…,�n,�and�the�second� subscript�ranges�over�j�=�1,…,�J��Thus,�there�are�J�levels�(or�categories�or�groups)�of�the� independent�variable�and�n�subjects�in�each�group,�for�a�total�of�Jn�=�N�total�observa- tions��For�now,�presume�there�are�n�subjects�(or�cases�or�units)�in�each�group�in�order� to�simplify�matters;�this�is�referred�to�as�the�equal�n’s�or�balanced case��Later�on�in�this� chapter,�we�consider�the�unequal�n’s�or�unbalanced case�
The�layout�of�the�data�is�shown�in�Table�11�1��Here�we�see�that�each�column�represents� the�observations�for�a�particular�group�or�level�of�the�independent�variable��At�the�bottom� of�each�column�are�the�sample�group�means�(Y
– �j),�with�the�overall�sample�mean�(Y
– ���)�to�the�
far�right��In�conclusion,�the�layout�of�the�data�is�one�form�in�which�the�researcher�can�think� about�the�data�
11.3 ANOVA Theory
This�section�examines�the�underlying�theory�and�logic�of�ANOVA,�the�sums�of�squares,� and�the�ANOVA�summary�table��As�noted�previously,�in�ANOVA,� mean�differences� are� tested�by�looking�at�the�variability�of�the�means��Here�we�show�precisely�how�this�is�done�
Table 11.1
Layout�for�the�One-Factor�ANOVA�Model
Level of the Independent Variable
1 2 3 … J
Y11 Y12 Y13 … Y1J Y21 Y22 Y23 … Y2J Y31 Y32 Y33 … Y3J � � � � � � � � � � Yn1 Yn2 Yn3 … YnJ
Means Y –
�1 Y –
�2 Y –
�3 … Y –
�J Y –
��
297One-Factor Analysis of Variance: Fixed-Effects Model
11.3.1 General Theory and logic
We�begin�with�the�hypotheses�to�be�tested�in�ANOVA��In�the�two-group�situation�of�the� independent�t�test,�the�null�and�alternative�hypotheses�for�a�two-tailed�(i�e�,�nondirectional)� test�are�as�follows:
H
H
0 1 2
1 1 2
:
:
µ µ
µ µ
=
≠
In� the� multiple-group� situation� (i�e�,� more� than� two� groups),� we� have� already� seen� the� problem�that�occurs�when�multiple�independent�t�tests�are�conducted�for�all�pairs�of�popu- lation�means�(i�e�,�increased�likelihood�of�a�Type�I�error)��We�concluded�that�the�solution� was�to�use�an�omnibus test�where�the�equality�of�all�of�the�means�could�be�assessed�simul- taneously��The�hypotheses�for�the�omnibus�ANOVA�test�are�as�follows:
H J0 1 2 3: µ µ µ µ= = = =…
H j1 : not all the are equalµ
Here� H1� is� purposely� written� in� a� general� form� to� cover� the� multitude� of� possible� mean� differences� that� could� arise�� These� range� from� only� two� of� the� means� being� different� to� all�of�the�means�being�different�from�one�another��Thus,�because�of�the�way�H1�has�been� written,� only� a� nondirectional� alternative� is� appropriate�� If� H0� were� to� be� rejected,� then� the� researcher� might� want� to� consider� a� multiple� comparison� procedure� (MCP)� so� as� to� determine�which�means�or�combination�of�means�are�significantly�different�(we�cover�this� in�greater�detail�in�Chapter�12)�
As�was�mentioned�in�the�introduction�to�this�chapter,�the�analysis�of�mean�differences� is�actually�carried�out�by�looking�at�variability�of�the�means��At�first,�this�seems�strange�� If� one� wants� to� test� for� mean� differences,� then� do� a� test� of� means�� If� one� wants� to� test� for�variance�differences,�then�do�a�test�of�variances��These�statements�should�make�sense� because�logic�pervades�the�field�of�statistics��And�they�do�for�the�two-group�situation��For� the�multiple-group�situation,�we�already�know�things�get�a�bit�more�complicated�
Say�a�researcher�is�interested�in�the�influence�of�amount�of�daily�study�time�on�statistics� achievement��Three�groups�were�formed�based�on�the�amount�of�daily�study�time�in�sta- tistics,�half�an�hour,�1�hour,�and�2�hours��Is�there�a�differential�influence�of�amount�of�time� studied�on�subsequent�mean�statistics�achievement�(e�g�,�statistics�final�exam)?�We�would� expect� that� the� more� one� studied� statistics,� the� higher� the� statistics� mean� achievement� would�be��One�possible�situation�in�the�population�is�where�the�amount�of�study�time�does� not�influence�statistics�achievement;�here�the�population�means�will�be�equal��That�is,�the� null�hypothesis�of�equal�group�means�is�actually�true��Thus,�the�three�groups�are�really� three�samples�from�the�same�population�of�students,�with�mean�μ��The�means�are�equal;� thus,�there�is�no�variability�among�the�three�group�means��A�second�possible�situation�in� the�population�is�where�the�amount�of�study�time�does�influence�statistics�achievement;� here�the�population�means�will�not�be�equal��That�is,�the�null�hypothesis�is�actually�false�� Thus,�the�three�groups�are�not�really�three�samples�from�the�same�population�of�students,� but�rather,�each�group�represents�a�sample�from�a�distinct�population�of�students�receiv- ing�that�particular�amount�of�study�time,�with�mean�μj��The�means�are�not�equal,�so�there�
298 An Introduction to Statistical Concepts
is�variability�among�the�three�group�means��In�summary,�the�statistical�question�becomes� whether�the�difference�between�the�sample�means�is�due�to�the�usual�sampling�variability� expected�from�a�single�population,�or�the�result�of�a�true�difference�between�the�sample� means�from�different�populations�
We�conceptually�define�within-groups variability�as�the�variability�of�the�observations� within�a�group�combined�across�groups�(e�g�,�variability�on�test�scores�within�children�in� the�same�proficiency�level,�such�as�low,�moderate,�and�high,�and�then�combined�across�all� proficiency�levels),�and�between-groups variability�as�the�variability�between�the�groups� (e�g�,� variability� among� the� test� scores� from� one� proficiency� level� to� another� proficiency� level)�� In� Figure� 11�1,� the� columns� represent� low� and� high� variability� within� the� groups�� The� rows� represent� low� and� high� variability� between� the� groups�� In� the� upper� left-hand� plot,�there�is�low�variability�both�within�and�between�the�groups��That�is,�performance�is� very�consistent,�both�within�each�group�as�well�as�across�groups��We�see�that�there�is�little� variability� within� the� groups� since� the� individual� distributions� are� not� very� spread� out� and�little�variability�between�the�groups�because�the�distributions�are�not�very�distinct,�as� they�are�nearly�lying�on�top�of�one�another��Here�within-�and�between-group�variability� are� both� low,� and� it� is� quite� unlikely� that� one� would� reject� H0�� In� the� upper� right-hand� plot,�there�is�high�variability�within�the�groups�and�low�variability�between�the�groups�� That�is,�performance�is�very�consistent�across�groups�(i�e�,�the�distributions�largely�overlap)� but� quite� variable� within� each� group�� We� see� high� variability� within� the� groups� because� the� spread� of� each� individual� distribution� is� quite� large� and� low� variability� between� the� groups� because� the� distributions� are� lying� so� closely� together�� Here� within-groups� vari- ability� exceeds� between-group� variability,� and� again� it� is� quite� unlikely� that� one� would� reject�H0��In�the�lower�left-hand�plot,�there�is�low�variability�within�the�groups�and�high� variability�between�the�groups��That�is,�performance�is�very�consistent�within�each�group� but� quite� variable� across� groups�� We� see� low� variability� within� the� groups� because� each� distribution� is� very� compact� with� little� spread� to� the� data� and� high� variability� between� the�groups�because�each�distribution�is�nearly�isolated�from�one�another�with�very�little� overlap��Here�between-group�variability�exceeds�within-groups�variability,�and�it�is�quite�
Variability within-groups HighLow
Low
V ar
ia bi
lit y
be tw
ee n-
gr ou
ps
High
FIGuRe 11.1 Conceptual�look�at�between-�and�within-groups�variability�
299One-Factor Analysis of Variance: Fixed-Effects Model
likely�that�one�would�reject�H0��In�the�lower�right-hand�plot,�there�is�high�variability�both� within�and�between�the�groups��That�is,�performance�is�quite�variable�within�each�group,� as�well�as�across�the�groups��We�see�high�variability�within�groups�because�the�spread�of� each�individual�distribution�is�quite�large�and�high�variability�between�groups�because�of� the�minimal�overlap�from�one�distribution�to�another��Here�within-�and�between-group� variability�are�both�high,�and�depending�on�the�relative�amounts�of�between-�and�within- groups�variability,�one�may�or�may�not�reject�H0��In�summary,�the�optimal�situation�when� seeking�to�reject�H0�is�the�one�represented�by�high�variability�between�the�groups�and�low� variability�within�the�groups�
11.3.2 partitioning the Sums of Squares
The�partitioning�of�the�sums�of�squares�in�ANOVA�is�a�new�concept�in�this�chapter,�which� is�also�an�important�concept�in�regression�analysis�(from�Chapters�17�and�18)��In�part,�this� is�because�ANOVA�and�regression�are�both�forms�of�the�same�general�linear�model�(GLM)� (to�be�further�discussed)��Let�us�begin�with�the�total�sum�of�squares�in�Y,�denoted�as�SStotal�� The�term�SStotal�represents�the�amount�of�total�variation�in�Y��The�next�step�is�to�partition� the� total� variation� into� variation� between� the� groups� (i�e�,� the� categories� or� levels� of� the� independent� variable),� denoted� by� SSbetw,� and� variation� within� the� groups� (i�e�,� units� or� cases�within�each�category�or�level�of�the�independent�variable),�denoted�by�SSwith��In�the� one-factor�ANOVA,�we�therefore�partition�SStotal�as�follows:
SS SS SStotal betw with= +
or
( ) ( ) ( ).. . .. .Y Y Y Y Y Yij j
J
i
n
j
j
J
i
n
ij j
j
J
i
− = − + − == == =
∑∑ ∑∑ ∑2 11
2
11
2
1== ∑
1
n
where SStotal�is�the�total�sum�of�squares�due�to�variation�among�all�of�the�observations�without�
regard�to�group�membership SSbetw�is�the�between-groups�sum�of�squares�due�to�the�variation�between�the�groups SSwith�is�the�within-groups�sum�of�squares�due�to�the�variation�within�the�groups�com-
bined�across�groups
We� refer� to� this� particular� formulation� of� the� partitioned� sums� of� squares� as� the� definitional� (or� conceptual)� formula� because� each� term� literally� defines� a� form� of� variation�
Due� to� computational� complexity� and� the� likelihood� of� a� computational� error,� the� definitional� formula� is� rarely� used� with� real� data�� Instead,� a� computational formula� for� the� partitioned� sums� of� squares� is� used� for� hand� computations�� However,� since� nearly� all� data� analysis� at� this� level� utilizes� computer� software,� we� defer� to� the� soft- ware�to�actually�perform�an�ANOVA�(SPSS�details�are�provided�toward�the�end�of�this� chapter)��A�complete�example�of�the�one-factor�ANOVA�is�also�considered�later�in�this� chapter�
300 An Introduction to Statistical Concepts
11.3.3 aNOVa Summary Table
An� important� result� of� the� analysis� is� the� ANOVA summary table�� The� purpose� of� the� summary� table� is� to� literally� summarize� the� ANOVA�� A� general� form� of� the� summary� table�is�shown�in�Table�11�2��The�first�column�lists�the�sources�of�variation�in�the�model��As� we�already�know,�in�the�one-factor�model,�the�total�variation�is�partitioned�into�between- groups� variation� and� within-groups� variation�� The� second� column� notes� the� sums� of� squares�terms�computed�for�each�source�(i�e�,�SSbetw,�SSwith,�and�SStotal)�
The�third�column�gives�the�degrees�of�freedom�for�each�source��Recall�that,�in�general,� the�degrees�of�freedom�have�to�do�with�the�number�of�observations�that�are�free�to�vary�� For�example,�if�a�sample�mean�and�all�of�the�sample�observations�except�for�one�are�known,� then�the�final�observation�is�not�free�to�vary��That�is,�the�final�observation�is�predetermined� to�be�a�particular�value��For�instance,�say�the�mean�is�10�and�there�are�three�observations,� 7,�11,�and�an�unknown�observation��Based�on�that�information,�first,�the�sum�of�the�three� observations�must�be�30�for�the�mean�to�be�10��Second,�the�sum�of�the�known�observations� is�18��Therefore,�the�unknown�observation�must�be�12��Otherwise�the�sample�mean�would� not�be�exactly�equal�to�10�
For�the�between-groups�source,�the�definitional�formula�is�concerned�with�the�deviation� of� each� group� mean� from� the� overall� mean�� There� are� J� group� means� (where� J� represents� the�number�of�groups�or�categories�or�levels�of�the�independent�variable),�so�the�dfbetw�(also� known�as�the�degrees�of�freedom�numerator)�must�be�J�−�1��Why?�If�we�have�J�group�means� and�we�know�the�overall�mean,�then�only�J�−�1�of�the�group�means�are�free�to�vary��In�other� words,� if� we� know� the� overall� mean� and� all� but� one� of� the� group� means,� then� the� final� unknown�group�mean�is�predetermined��For�the�within-groups�source,�the�definitional�for- mula�is�concerned�with�the�deviation�of�each�observation�from�its�respective�group�mean�� There� are� n� observations� (i�e�,� cases� or� units)� in� each� group;� consequently,� there� are� n� −� 1� degrees�of�freedom�in�each�group�and�J�groups��Why�are�there�n�−�1�degrees�of�freedom�in� each�group?�If�there�are�n�observations�in�each�group,�then�only�n�−�1�of�the�observations�are� free�to�vary��In�other�words,�if�we�know�one�group�mean�and�all�but�one�of�the�observations� for�that�group,�then�the�final�unknown�observation�for�that�group�is�predetermined��There� are� J� groups,� so� the� dfwith� (also� known� as� the� degrees� of� freedom� denominator)� is� J(n� −� 1),� or�more�simply�as�N�−�J��Thus,�we�lose�one�degree�of�freedom�for�each�group��For�the�total� source,�the�definitional�formula�is�concerned�with�the�deviation�of�each�observation�from�the� overall�mean��There�are�N�total�observations;�therefore,�the�dftotal�must�be�N�−�1��Why?�If�there� are�N�total�observations�and�we�know�the�overall�mean,�then�only�N�−�1�of�the�observations� are�free�to�vary��In�other�words,�if�we�know�the�overall�mean�and�all�but�one�of�the�N�obser- vations,�then�the�final�unknown�observation�is�predetermined�
Why� is� the� number� of� degrees� of� freedom� important� in� the� ANOVA?� Suppose� two� researchers�have�conducted�similar�studies,�except�Researcher�A�uses�20�observations�per� group�and�Researcher�B�uses�10�observations�per�group��Each�researcher�obtains�a�SSwith� value�of�15��Would�it�be�fair�to�say�that�this�particular�result�for�the�two�studies�is�the�same?�
Table 11.2
ANOVA�Summary�Table
Source SS df MS F
Between�groups SSbetw J�−�1 MSbetw MSbetw/MSwith Within�groups SSwith N�−�J MSwith Total SStotal N�−�1
301One-Factor Analysis of Variance: Fixed-Effects Model
Such�a�comparison�would�be�unfair�because�SSwith�is�influenced�by�the�number�of�observa- tions� per� group�� A� fair� comparison� would� be� to� weight� the� SSwith� terms� by� their� respec- tive� number� of� degrees� of� freedom�� Similarly,� it� would� not� be� fair� to� compare� the� SSbetw� terms�from�two�similar�studies�based�on�different�numbers�of�groups��A�fair�comparison� would� be� to� weight� the� SSbetw� terms� by� their� respective� number� of� degrees� of� freedom�� The�method�of�weighting�a�sum�of�squares�term�by�the�respective�number�of�degrees�of� freedom�on�which�it�is�based�yields�what�is�called�a�mean squares�term��Thus,�MSbetw�=� SSbetw/dfbetw�and�MSwith�=�SSwith/dfwith,�as�shown�in�the�fourth�column�of�Table�11�2��They�are� referred�to�as�mean�squares�because�they�represent�a�summed�quantity�that�is�weighted�by� the�number�of�observations�used�in�the�sum�itself,�like�the�mean��The�mean�squares�terms� are�also�variance�estimates�because�they�represent�the�sum�of�the�squared�deviations�from� a�mean�divided�by�their�degrees�of�freedom,�like�the�sample�variance�s2�
The�last�column�in�the�ANOVA�summary�table,�the�F�value,�is�the�summary�test�statistic� of�the�summary�table��The�F�value�is�computed�by�taking�the�ratio�of�the�two�mean�squares� or�variance�terms��Thus,�for�the�one-factor�ANOVA�fixed-effects�model,�the�F�value�is�com- puted� as� F� =� MSbetw/MSwith�� When� developed� by� Sir� Ronald� A�� Fisher� in� the� 1920s,� this� test�statistic�was�originally�known�as�the�variance�ratio�because�it�represents�the�ratio�of� two� variance� estimates�� Later,� the� variance� ratio� was� renamed� the� F� ratio� by� George� W�� Snedecor�(who�worked�out�the�table�of�F�values,�discussed�momentarily)�in�honor�of�Fisher� (F�for�Fisher)�
The�F�ratio�tells�us�whether�there�is�more�variation�between�groups�than�there�is�within� groups,�which�is�required�if�we�are�to�reject�H0��Thus,�if�there�is�more�variation�between� groups�than�there�is�within�groups,�then�MSbetw�will�be�larger�than�MSwith��As�a�result�of� this,�the�F�ratio�of�MSbetw/MSwith�will�be�greater�than�1��If,�on�the�other�hand,�the�amount� of�variation�between�groups�is�about�the�same�as�there�is�within�groups,�then�MSbetw�and� MSwith�will�be�about�the�same,�and�the�F�ratio�will�be�approximately�1��Thus,�we�want� to� find� large� F� values� in� order� to� reject� the� null� hypothesis�� The� F� test� statistic� is� then� compared�with�the�F�critical�value�so�as�to�make�a�decision�about�the�null�hypothesis�� The� critical� value� is� found� in� the� F� table� of� Table� A�4� as� αF(J−1,N−J)�� Thus,� the� degrees� of� freedom� are� df betw� =� J� −� 1� for� the� numerator� of� the� F� ratio� and� dfwith� =� N� −� J� for� the� denominator�of�the�F�ratio��The�significance�test�is�a�one-tailed�test�in�order�to�be�consis- tent�with�the�alternative�hypothesis��The�null�hypothesis�is�rejected�if�the�F�test�statistic� exceeds� the� F� critical� value�� This� is� the� omnibus� F� test� which,� again,� simply� provides� evidence�of the extent�to�which�there�is�at�least�one�statistically�significant�mean�differ- ence�between�the�groups�
If� the� F� test� statistic� exceeds� the� F� critical� value,� and� there� are� more� than� two� groups,� then� it� is� not� clear� where� the� differences� among� the� means� lie�� In� this� case,� some� MCP� should� be� used� to� determine� where� the� mean� differences� are� in� the� groups;� this� is� the� topic�of�Chapter�12��When�there�are�only�two�groups,�it�is�obvious�where�the�mean�differ- ence�falls,�that�is,�between�groups�1�and�2��A�researcher�can�simply�look�at�the�descriptive� statistics�to�determine�which�group�had�the�higher�mean�relative�to�the�other�group��For� the�two-group�situation,�it�is�also�interesting�to�note�that�the�F�and�t�test�statistics�follow� the�rule�of�F�=�t2,�for�a�nondirectional�alternative�hypothesis�in�the�independent�t�test��In� other�words,�the�one-way�ANOVA�with�two�groups�and�the�independent�t�test�will�gener- ate�the�same�conclusion�such�that�F�=�t2��This�result�occurs�when�the�numerator�degrees�of� freedom�for�the�F�ratio�is�1��In�an�actual�ANOVA�summary�table�(shown�in�the�next�sec- tion),�except�for�the�source�of�variation�column,�it�is�the�values�for�each�of�the�other�entries� generated�from�the�data�that�are�listed�in�the�table��For�example,�instead�of�seeing�SSbetw,� we�would�see�the�computed�value�of�SSbetw�
302 An Introduction to Statistical Concepts
11.4 ANOVA Model
In�this�section,�we�introduce�the�ANOVA�linear�model,�the�estimation�of�parameters�of�the� model,�effect�size�measures,�confidence�intervals�(CIs),�power,�and�an�example,�and�finish� up�with�expected�mean�squares�
11.4.1 Model
The�one-factor�ANOVA�fixed-effects�model�can�be�written�in�terms�of�population�param- eters�as
Yij j ij= + +µ α ε
where Y�is�the�observed�score�on�the�dependent�(or�criterion)�variable�for�individual�i�in�group j μ�is�the�overall�or�grand�population�mean�(i�e�,�regardless�of�group�designation) αj�is�the�group�effect�for�group�j ɛij�is�the�random�residual�error�for�individual�i�in�group�j
The�residual�error�can�be�due�to�individual�differences,�measurement�error,�and/or�other� factors�not�under�investigation�(i�e�,�other�than�the�independent�variable�X)��The�popula- tion�group�effect�and�residual�error�are�computed�as
α µ µj j= −�
and
ε µij ij jY= − �
respectively,�and�μ�j�is�the�population�mean�for�group�j,�where�the�initial�dot�subscript� indicates�we�have�averaged�across�all�i�individuals�in�group�j��That�is,�the�group�effect� is� equal� to� the� difference� between� the� population� mean� of� group� j� and� the� overall� population�mean��The�residual�error�is�equal�to�the�difference�between�an�individual’s� observed� score� and� the� population� mean� of� the� group� that� the� individual� is� a� mem- ber� of� (i�e�,� group� j)�� The� group� effect� can� also� be� thought� of� as� the� average� effect� of� being� a� member� of� a� particular� group�� A� positive� group� effect� implies� a� group� mean� greater� than� the� overall� mean,� whereas� a� negative� group� effect� implies� a� group� mean� less�than�the�overall�mean��Note�that�in�a�one-factor�fixed-effects�model,�the�population� group�effects�sum�to�0��The�residual�error�in�ANOVA�represents�that�portion�of�Y�not� accounted�for�by�X�
11.4.2 estimation of the parameters of the Model
Next�we�need�to�estimate�the�parameters�of�the�model�μ,�αj,�and�ɛij��The�sample�estimates� are�represented�by�Y
– ��,�aj,�and�eij,�respectively,�where�the�latter�two�are�computed�as
303One-Factor Analysis of Variance: Fixed-Effects Model
a Y Yj j= −� ��
and
e Y Yij ij j= − �
respectively��Note�that�Y –
���represents�the�overall�sample�mean,�where�the�double�dot�sub- script�indicates�we�have�averaged�across�both�the�i�and�j�subscripts,�and�Y
– �j�represents�the�
sample�mean�for�group�j,�where�the�initial�dot�subscript�indicates�we�have�averaged�across� all�i�individuals�in�group�j�
11.4.3 effect Size Measures, Confidence Intervals, and power
11.4.3.1 Effect Size Measures
There�are�various�effect�size�measures�to�indicate�the�strength�of�association�between�X�and�Y,� that�is,�the�relative�strength�of�the�group�effect��Let�us�briefly�examine�η2,�ω2,�and�Cohen’s� (1988)� f�� First,� η2� (eta� squared),� ranging� from� 0� to� +1�00,� is� known� as� the� correlation� ratio� (generalization� of� R2)� and� represents� the� proportion� of� variation� in� Y� explained� by� the� group�mean�differences�in�X��An�eta�squared�of�0�suggests�that�none�of�the�total�variance� in�the�dependent�variable�is�due�to�differences�between�the�groups��An�eta�squared�of�1�00� indicates�that�all�the�variance�in�the�dependent�variable�is�due�to�the�group�mean�differ- ences��We�find�η2�to�be�as�follows:
η2 = SS SS
betw
total
It�is�well�known�that�η2�is�a�positively�biased�statistic�(i�e�,�overestimates�the�association)�� The�bias�is�most�evident�for�n’s�(i�e�,�group�sample�sizes)�less�than�30�
Another�effect�size�measure�is�ω2�(omega�squared),�interpreted�similarly�to�eta�squared� (specifically�proportion�of�variation�in�Y�explained�by�the�group�mean�differences�in�X)� but�which�is�less�biased�than�η2��We�determine�ω2�through�the�following�formula:
ω2 1
= − −
+ SS J MS
SS MS betw with
total with
( )
A�final�effect�size�measure�is�f�developed�by�Cohen�(1988)��The�effect�f�can�take�on�values� from�0�(when�the�means�are�equal)�to�an�infinitely�large�positive�value��This�effect�is�inter- preted� as� an� approximate� correlation� index� but� can� also� be� interpreted� as� the� standard� deviation�of�the�standardized�means�(Cohen,�1988)��We�compute�f�through�the�following:
f = − η
η
2
21
We�can�also�use�f�to�compute�the�effect�size�d,�which�you�recall�from�the�t�test�is�interpreted� as�the�standardized�mean�difference��The�formulas�for�translating�f�to�d�are�dependent�on� whether�there�is�minimum,�moderate,�or�maximum�variability�between�the�means�of�the� groups��Interested�readers�are�referred�to�Cohen�(1988)�
304 An Introduction to Statistical Concepts
These� are� the� most� common� measures� of� effect� size� used� for� ANOVA� models,� both� in�statistics�software�and�in�print��Cohen’s�(1988)�subjective�standards�can�be�used�as� follows�to�interpret�these�effect�sizes:�small�effect,�f�=��1,�η2�or�ω2�=��01;�medium�effect,� f�=��25,�η2�or�ω2�=��06;�and�large�effect,�f�=��40,�η2�or�ω2�=��14��Note�that�these�are�subjective� standards� developed� for� the� behavioral� sciences;� your� discipline� may� use� other� stan- dards��For�further�discussion,�see�Keppel�(1982),�O’Grady�(1982),�Wilcox�(1987),�Cohen� (1988),� Keppel� and� Wickens� (2004),� and� Murphy,� Myors,� and� Wolach� (2008;� which� includes�software)�
11.4.3.2 Confidence Intervals
CI�procedures�are�often�useful�in�providing�an�interval�estimate�of�a�population�parameter� (i�e�,� mean� or� mean� difference);� these� allow� us� to� determine� the� accuracy� of� the� sample� estimate��One�can�form�CIs�around�any�sample�group�mean�from�an�ANOVA�(provided�in� software�such�as�SPSS),�although�CIs�for�means�have�more�utility�for�MCPs,�as�discussed� in� Chapter� 12�� CI� procedures� have� also� been� developed� for� several� effect� size� measures� (Fidler�&�Thompson,�2001;�Smithson,�2001)�
11.4.3.3 Power
As� for� power� (the� probability� of� correctly� rejecting� a� false� null� hypothesis),� one� can� consider�either�planned�power�(a�priori)�or�observed�power�(post�hoc),�as�discussed�in� previous�chapters��In�the�ANOVA�context,�we�know�that�power�is�primarily�a�function� of� α,� sample� size,� and� effect� size�� For� planned� power,� one� inputs� each� of� these� compo- nents�either�into�a�statistical�table�or�power�chart�(nicely�arrayed�in�texts�such�as�Cohen,� 1988,� or� Murphy� et� al�,� 2008),� or� into� statistical� software� (such� as� Power� and� Precision,� Ex-Sample,�G*Power,�or�the�software�contained�in�Murphy�et�al�,�2008)��Planned�power� is�most�often�used�by�researchers�to�determine�adequate�sample�sizes�in�ANOVA�mod- els,� which� is� highly� recommended�� Many� disciplines� recommend� a� minimum� power� value,� such� as� �80�� Thus,� these� methods� are� a� useful� way� to� determine� the� sample� size� that� would� generate� a� desired� level� of� power�� Observed� power� is� determined� by� some� statistics�software,�such�as�SPSS,�and�indicates�the�power�that�was�actually�observed�in� a�completed�study�
11.4.4 example
Consider�now�an�example�problem�used�throughout�this�chapter��Our�dependent�variable� is� the� number� of� times� a� student� attends� statistics� lab� during� one� semester� (or� quarter),� whereas�the�independent�variable�is�the�attractiveness�of�the�lab�instructor�(assuming�each� instructor�is�of�the�same�gender�and�is�equally�competent)��The�researcher�is�interested�in� whether�the�attractiveness�of�the�instructor�influences�student�attendance�at�the�statistics� lab��The�attractiveness�groups�are�defined�as�follows:
•� Group�1,�unattractive •� Group�2,�slightly�attractive •� Group�3,�moderately�attractive •� Group�4,�very�attractive
305One-Factor Analysis of Variance: Fixed-Effects Model
Students� were� randomly� assigned� to� one� group� at� the� beginning� of� the� semester,� and� attendance�was�taken�by�the�instructor��There�were�8�students�in�each�group�for�a�total�of� 32��Students�could�attend�a�maximum�of�30�lab�sessions��In�Table�11�3,�we�see�the�raw�data� and�sample�statistics�(means�and�variances)�for�each�group�and�overall�(far�right)�
The�results�are�summarized�in�the�ANOVA�summary�table�as�shown�in�Table�11�4�� The� test� statistic,� F� =� 6�1877,� is� compared� to� the� critical� value,� �05F3,28� =� 2�95� obtained� from�Table�A�4,�using�the��05�level�of�significance��To�use�the�F�table,�find�the�numera- tor� degrees� of� freedom,� df betw,� which� are� represented� by� the� columns,� and� then� the� denominator�degrees�of�freedom,�dfwith,�which�are�represented�by�the�rows��The�inter- section� of� the� two� provides� the� F� critical� value�� The� test� statistic� exceeds� the� critical� value,�so�we�reject�H0�and�conclude�that�level�of�attractiveness�is�related�to�mean�dif- ferences� in� statistics� lab� attendance�� The� exact� probability� value� (p� value)� given� by� SPSS�is��001�
Next�we�examine�the�group�effects�and�residual�errors��The�group�effects�are�estimated� as�follows�where�the�grand�mean�(irrespective�of�the�group�membership;�here�18�4063)�is� subtracted�from�the�group�mean�(e�g�,�11�125�for�group�1)��The�subscript�of�a�indicates�the� level�or�group�of�the�independent�variable�(e�g�,�1�=�unattractive;�2�=�slightly�attractive;� 3�=�moderately�attractive;�4�=�very�attractive)��A�negative�group�effect�indicates�that�group� had�a�smaller�mean�than�the�overall�average�and�thus�exerted�a�negative�effect�on�the�depen- dent�variable�(in�our�case,�lower�attendance�in�the�statistics�lab)��A�positive�group�effect�indi-
Table 11.4
ANOVA�Summary�Table—Statistics�Lab�Example
Source SS df MS F
Between�groups 738�5938 3 246�1979 6�8177a
Within�groups 1011�1250 28 36�1116 Total 1749�7188 31
a� �05F3,28�=�2�95�
Table 11.3
Data�and�Summary�Statistics�for�the�Statistics�Lab�Example
Number of Statistics Labs Attended by Group
Group 1: Unattractive
Group 2: Slightly
Unattractive
Group 3: Moderately Attractive
Group 4: Very
Attractive Overall
15 20 10 30 10 13 24 22 12 9 29 26 8 22 12 20
21 24 27 29 7 25 21 28
13 18 25 25 3 12 14 15
Means 11�1250 17�8750 20�2500 24�3750 18�4063 Variances 30�1250 35�2679 53�0714 25�9821 56�4425
306 An Introduction to Statistical Concepts
cates�that�group�had�a�larger�mean�than�the�overall�average�and�thus�exerted�a�positive�effect� on�the�dependent�variable�(in�our�case,�higher�attendance�in�the�statistics�lab):
a Y Y1 11.125 18.4063 7.2813= − = − = −. ..1
a Y Y2 17.875 18.4063 .5313= − = − = −. ..2
a Y Y3 20.250 18.4063 1.8437= − = − = +. ..3
a Y Y4 24.375 18.4063 5.9687= − = − = +. ..4
Thus,�group�4�(very attractive)�has�the�largest�positive�group�effect�(i�e�,�higher�attendance� than�average),�while�group�1�(unattractive)�has�the�largest�negative�group�effect�(i�e�,�lower� attendance� than� average)�� In� Chapter� 12,� we� use� the� same� data� to� determine� which� of� these�group�means,�or�combination�of�group�means,�are�statistically�different��The�resid- ual�errors�(computed�as�the�difference�between�the�observed�value�and�the�group�mean)� for�each�individual�by�group�are�shown�in�Table�11�5�and�discussed�later�in�this�chapter�
Finally�we�determine�the�effect�size�measures��For�illustrative�purposes,�all�effect�size�mea- sures�that�were�previously�discussed�have�been�computed��In�practice,�only�one�effect�size� is�usually�computed�and�interpreted��First,�the�correlation�ratio�η2�is�computed�as�follows:
η2 738 5938
1749 7188 4221= = =
SS SS
betw
total
. .
.
Next�ω2�is�found�to�be�the�following:
ω2 1 738 5938 3 36 1116
174 =
− − +
= −SS J MS
SS MS betw with
total with
( ) . ( ) . 99 7188 36 1116
3529 . .
. +
=
Lastly�f�is�computed�as�follows:
f = −
= −
= η
η
2
21 4221
1 4221 8546
. .
.
Table 11.5
Residuals�for�the�Statistics�Lab�Example� by Group
Group 1 Group 2 Group 3 Group 4
3�875 2�125 −10�250 5�625 −1�125 −4�875 3�750 −2�375
�875 −8�875 8�750 1�625 −3�125 4�125 −8�250 −4�375
9�875 6�125 6�750 4�625 −4�125 7�125 �750 3�625
1�875 �125 4�750 �625 −8�125 −5�875 −6�250 −9�375
307One-Factor Analysis of Variance: Fixed-Effects Model
Recall�Cohen’s�(1988)�subjective�standards�that�can�be�used�to�interpret�these�effect�sizes:� small�effect,�f�=��1,�η2�or�ω2�=��01;�medium�effect,�f�=��25,�η2�or�ω2�=��06;�and�large�effect,�f�=��40,� η2�or�ω2�=��14��Based�on�these�effect�size�measures,�all�measures�lead�to�the�same�conclusion:� there�is�a�large�effect�size�for�the�influence�of�instructor�attractiveness�on�lab�attendance�� Examining�η2�or�ω2,�we�can�also�state�that�42%�or�35%,�respectively,�of�the�variation�in� Y�(attendance�at�the�statistics�lab)�can�be�explained�by�X�(attractiveness�of�the�instructor)�� The�effect�f�suggests�a�strong�correlation�
In� addition,� if� we� rank� the� instructor� group� means� from� unattractive� (with� the� lowest� mean)�to�very�attractive�(with�the�highest�mean),�we�see�that�the�more�attractive�the�instruc- tor,� the� more� inclined� the� student� is� to� attend� lab�� While� visual� inspection� of� the� means� suggests� descriptively� that� there� are� differences� in� statistics� lab� attendance� by� group,� we� examine�MCPs�with�these�same�data�in�Chapter�12�to�determine�which�groups�are�statisti- cally�significantly�different�from�each�other�
11.4.5 expected Mean Squares
There�is�one�more�theoretical�concept�called�expected mean squares�to�introduce�in�this� chapter��The�notion�of�expected�mean�squares�provides�the�basis�for�determining�what�the� appropriate�error�term�is�when�forming�an�F�ratio�(recall�this�ratio�is�F�=�MSbetw/MSwith)�� That� is,� when� forming� an� F� ratio� to� test� a� certain� hypothesis,� how� do� we� know� which� source�of�variation�to�use�as�the�error�term�in�the�denominator?�For�instance,�in�the�one- factor� fixed-effects� ANOVA� model,� how� did� we� know� to� use� MSwith� as� the� error� term� in� testing�for�differences�between�the�groups?�There�is�a�good�rationale,�as�becomes�evident�
Before�we�get�into�expected�mean�squares,�consider�the�definition�of�an�expected�value�� An�expected�value�is�defined�as�the�average�value�of�a�statistic�that�would�be�obtained�with� repeated�sampling��Using�the�sample�mean�as�an�example�statistic,�the�expected�value�of� the�mean�would�be�the�average�value�of�the�sample�means�obtained�from�an�infinite�num- ber�of�samples��The�expected�value�of�a�statistic�is�also�known�as�the�mean�of�the�sampling� distribution�of�that�statistic��In�this�case,�the�expected�value�of�the�mean�is�the�mean�of�the� sampling�distribution�of�the�mean�
An�expected�mean�square�for�a�particular�source�of�variation�represents�the�average�mean� square�value�for�that�source�obtained�if�the�same�study�were�to�be�repeated�an�infinite�num- ber�of�times��For�instance,�the�expected�value�of�MSbetw,�denoted�by�E(MSbetw),�is�the�average� value�of�MSbetw�over�repeated�samplings��At�this�point,�you�might�be�asking,�“why�not�only� be�concerned�about�the�values�of�the�mean�square�terms�for�my�own�little�study”?�Well,�the� mean�square�terms�from�your�little�study�do�represent�a�sample�from�a�population�of�mean� square�terms��Thus,�sampling�distributions�and�sampling�variability�are�as�much�a�concern� in�ANOVA�as�they�are�in�other�situations�previously�described�in�this�text�
Now� we� are� ready� to� see� what� the� expected� mean� square� terms� actually� look� like�� Consider� the� two� situations� of� H0� actually� being� true� and� H0� actually� being� false�� If� H0� is� actually� true,� such� that� there� really� are� no� differences� between� the� population� group� means,�then�the�expected mean squares�[represented�in�statistical�notation�as�either�E(MSbetw)� or�E(MSwith)]�are�as�follows:
E betw( )MS = σε 2
E with( )MS = σε 2
308 An Introduction to Statistical Concepts
and�thus�the�ratio�of�expected�mean�squares�is�as�follows:
E Ebetw with( )/ ( )MS MS = 1
where�the�expected�value�of�F�is�then�E(F)�=�dfwith/(dfwith�−�2),�and�σε 2�is�the�population�vari-
ance�of�the�residual�errors��What�this�tells�us�is�the�following:�if�H0�is�actually�true,�then� each�of�the�J�samples�really�comes�from�the�same�population�with�mean�μ�
If�H0�is�actually�false,�such�that�there�really�are�differences�between�the�population�group� means,�then�the�expected�mean�squares�are�as�follows:
E betwMS n Jj j
J
( ) = +
− =
∑σ αε2 2 1
1/( )
E withMS( ) = σε2
and�thus�the�ratio�of�the�expected�mean�squares�is�as�follows:
E Ebetw with( )/ ( )MS MS > 1
where�E(F)�>�dfwith/(dfwith�−�2)��If�H0�is�actually�false,�then�the�J�samples�do�really�come�from� different�populations�with�different�means�μj �
There�is� a�difference� in�the�expected� mean�square�between� [i�e�,�E(MSbetw)]�when�H0�is� actually�true�as�compared�to�when�H0�is�actually�false,�as�in�the�latter�situation,�there�is�a�
second�term��The�important�part�of�this�second�term�is� α j j
J 2
1= ∑ ,�which�represents�the�sum�of�
the�squared�group�effects��The�larger�this�part�becomes,�the�larger�MSbetw�is,�and�thus�the� larger�the�F�ratio�becomes��In�comparing�the�two�situations,�we�also�see�that�E(MSwith)�is� the�same�whether�H0�is�actually�true�or�false�and�thus�represents�a�reliable�estimate�of�σε
2�� This�term�is�mean-free�because�it�does�not�depend�on�group�mean�differences��Just�to�cover� all�of�the�possibilities,�F�could�be�less�than�1�[or�technically�less�than�dfwith/(dfwith�−�2)]�due� to�sampling�error,�nonrandom�samples,�and/or�assumption�violations��For�a�mathematical� proof�of�the�E(MS)�terms,�see�Kirk�(1982,�pp��66–71)�
Finally�let�us�try�to�put�all�of�this�information�together��In�general,�the�F�ratio�represents� the�following:
F = +(systematic variability error variability)/(error variabiility)
where,�for�the�one-factor�fixed-effects�model,�systematic variability�is�variability�between�the� groups�and�error variability�is�variability�within�the�groups��The�F�ratio�is�formed�in�a�par- ticular� way� because� we� want� to� isolate� the� systematic� variability� in� the� numerator�� For� this� model,� the� only� appropriate� F� ratio� is� MSbetw/MSwith� because� it� does� serve� to� isolate� the�systematic�variability�(i�e�,�the�variability�between�the�groups)��That�is,�the�appropri- ate�error�term�for�testing�a�particular�effect�(e�g�,�mean�differences�between�groups)�is�the� mean�square�that�is�identical�to�the�mean�square�of�that�effect,�except�that�it�lacks�a�term� due�to�the�effect�of�interest��For�this�model,�the�appropriate�error�term�to�use�for�testing�
309One-Factor Analysis of Variance: Fixed-Effects Model
differences�between�groups�is�the�mean�square�identical�to�the�numerator�MSbetw,�except�
it�lacks�a�term�due�to�the�between�groups�effect�[i�e�,� n Jj j
J
α 2
1
1 =
∑
−/( )];�this,�of�course,�is�
MSwith��It�should�also�be�noted�that�the�F�ratio�is�a�ratio�of�two�independent�variance�esti- mates,�here�being�MSbetw�and�MSwith�
11.5 Assumptions and Violation of Assumptions
There�are�three�standard�assumptions�made�in�ANOVA�models,�which�we�are�already�famil- iar�with�from�the�independent�t�test��We�see�these�assumptions�often�in�the�remainder�of�this� text��The�assumptions�are�concerned�with�independence,�homogeneity�of�variance,�and�nor- mality��We�also�mention�some�techniques�appropriate�to�use�in�evaluating�each�assumption�
11.5.1 Independence
The� first� assumption� is� that� observations� are� independent� of� one� another� (both� within� samples� and� across� samples)�� In� general,� the� assumption� of� independence� for� ANOVA� designs� can� be� met� by� (a)� keeping� the� assignment� of� individuals� to� groups� separate� through� the� design� of� the� experiment� (specifically� random� assignment—not� to� be� con- fused�with�random�selection),�and�(b)�keeping�the�individuals�separate�from�one�another� through�experimental�control�so�that�the�scores�on�the�dependent�variable�Y�for�group�1� do�not�influence�the�scores�for�group�2�and�so�forth�for�other�groups�of�the�independent� variable��Zimmerman�(1997)�also�stated�that�independence�can�be�violated�for�supposedly� independent� samples� due� to� some� type� of� matching� in� the� design� of� the� experiment� (e�g�,�matched�pairs�based�on�gender,�age,�and�weight)�
The�use�of�independent�random�samples�is�crucial�in�ANOVA��The�F�ratio�is�very�sensi- tive�to�violation�of�the�independence�assumption�in�terms�of�increased�likelihood�of�a�Type�I� and/or�Type�II�error�(e�g�,�Glass,�Peckham,�&�Sanders,�1972)��This�effect�can�sometimes�even� be� worse� with� larger� samples� (Keppel� &� Wickens,� 2004)�� A� violation� of� the� independence� assumption� may� affect� the� standard� errors� of� the� sample� means� and� thus� influence� any� inferences�made�about�those�means��One�purpose�of�random�assignment�of�individuals�to� groups�is�to�achieve�independence��If�each�individual�is�only�observed�once�and�individuals� are�randomly�assigned�to�groups,�then�the�independence�assumption�is�usually�met��If�indi- viduals�work�together�during�the�study�(e�g�,�through�discussion�groups�or�group�work),�then� independence� may� be� compromised�� Thus,� a� carefully� planned,� controlled,� and� conducted� research�design�is�the�key�to�satisfying�this�assumption�
The�simplest�procedure�for�assessing�independence�is�to�examine�residual�plots�by�group�� If�the�independence�assumption�is�satisfied,�then�the�residuals�should�fall�into�a�random� display�of�points�for�each�group��If�the�assumption�is�violated,�then�the�residuals�will�fall� into�some�type�of�pattern��The�Durbin–Watson�statistic�(1950,�1951,�1971)�can�be�used�to�test� for� autocorrelation�� Violations� of� the� independence� assumption� generally� occur� in� three� situations:�(1)�when�observations�are�collected�over�time,�(2)�when�observations�are�made� within� blocks,� or� (3)� when� observation� involves� replication�� For� severe� violations� of� the� independence�assumption,�there�is�no�simple�“fix”�(e�g�,�Scariano�&�Davenport,�1987)��For� the�example�data,�a�plot�of�the�residuals�by�group�is�shown�in�Figure�11�2,�and�there�does� appear�to�be�a�random�display�of�points�for�each�group�
310 An Introduction to Statistical Concepts
11.5.2 homogeneity of Variance
The� second� assumption� is� that� the� variances� of� each� population� are� equal�� This� is� known�as�the�assumption�of�homogeneity of variance�or�homoscedasticity��A�viola- tion� of� the� homogeneity� assumption� can� lead� to� bias� in� the� SSwith� term,� as� well� as� an� increase�in�the�Type�I�error�rate�and�possibly�an�increase�in�the�Type�II�error�rate��Two� sets� of� research� studies� have� investigated� violations� of� this� assumption,� classic� work� and�more�modern�work�
The� classic� work� largely� resulted� from� Box� (1954a)� and� Glass� et� al�� (1972)�� Their� results� indicated�that�the�effect�of�the�violation�was�small�with�equal�or�nearly�equal�n’s�across�the� groups��There�is�a�more�serious�problem�if�the�larger�n’s�are�associated�with�the�smaller�vari- ances�(actual�observed�α�>�nominal�α,�which�is�a�liberal�result;�for�example,�if�a�researcher� desires�a�nominal�alpha�of��05,�the�alpha�actually�observed�will�be�greater�than��05),�or�if�the� larger� n’s� are� associated� with� the� larger� variances� (actual� observed� α� <� nominal� α,� which� is�a�conservative�result)��[Note�that�Bradley’s�(1978)�criterion�is�used�in�this�text,�where�the� actual�α�should�not�exceed�1�1–1�5�times�the�nominal�α�]�Thus,�the�suggestion�from�the�classic� work�was�that�heterogeneity�was�only�a�concern�when�there�were�unequal�n’s��However,�the� classic�work�only�examined�minor�violations�of�the�assumption�(the�ratio�of�largest�variance� to�smallest�variance�being�relatively�small),�and�unfortunately,�has�been�largely�adapted�in� textbooks�and�by�users�
There� has� been� some� research� conducted� since� that� time� by� researchers� such� as� Brown�and�Forsythe�(1974)�and�Wilcox�(1986,�1987,�1988,�1989)�and�nicely�summarized�by� Coombs,�Algina,�and�Ottman�(1996)��In�short,�this�more�modern�work�indicates�that�the� effect�of�heterogeneity�is�more�severe�than�previously�thought�(e�g�,�poor�power;�α�can�be� greatly�affected),�even�with�equal�n’s�(although�having�equal�n’s�does�reduce�the�magni- tude�of�the�problem)��Thus,�F�is�not�even�robust�to�heterogeneity�with�equal�n’s�(equal�n’s� are�sometimes�referred�to�as�a�balanced�design)��Suggestions�for�dealing�with�such�a�vio- lation�include�(a)�using�alternative�procedures�such�as�the�Welch,�Brown–Forsythe,�and�
10.000
5.000
.000
–5.000
–10.000
Re si
du al
fo r l
ab s
–15.000
1.00 1.50 2.00 2.50 Level of attractiveness
3.00 3.50 4.00
FIGuRe 11.2 Residual�plot�by�group�for�statistics�lab�example�
311One-Factor Analysis of Variance: Fixed-Effects Model
James�procedures�(e�g�,�Coombs�et�al�,�1996;�Glass�&�Hopkins,�1996;�Keppel�&�Wickens,� 2004;�Myers�&�Well,�1995;�Wilcox,�1996,�2003);�(b)�reducing�α�and�testing�at�a�more�strin- gent� alpha� level� (e�g�,� �01� rather� than� the� common� �05)� (e�g�,� Keppel� &� Wickens,� 2004;� Weinberg� &� Abramowitz,� 2002);� or� (c)� transforming� Y� (such� as� Y ,� 1/Y,� or� log� Y)� (e�g�,� Keppel� &� Wickens,� 2004;� Weinberg� &� Abramowitz,� 2002)�� The� alternative� procedures� will�be�more�fully�described�later�in�this�chapter�
In�a�plot�of�residuals�versus�each�value�of�X,�the�consistency�of�the�variance�of�the�con- ditional� residual� distributions� may� be� examined� simply� by� eyeballing� the� plot�� Another� method� for� detecting� violation� of� the� homogeneity� assumption� is� the� use� of� formal� sta- tistical� tests,� as� discussed� in� Chapter� 9�� The� traditional� homogeneity� tests� (e�g�,� Levene’s� test)� are� commonly� available� in� statistical� software,� but� are� not� robust� to� nonnormality�� Unfortunately�the�more�robust�homogeneity�tests�are�not�readily�available��For�the�exam- ple� data,� the� residual� plot� of� Figure� 11�2� shows� similar� variances� across� the� groups,� and� Levene’s�test�suggests�the�variances�are�not�different�[F(3,�28)�=��905,�p�=��451]�
11.5.3 Normality
The�third�assumption�is�that�each�of�the�populations�follows�the�normal�distribution�(i�e�,�there� is�normality�of�the�dependent�variable�for�each�category�or�group�or�level�of�the�indepen- dent�variable)��The�F�test�is�relatively�robust�to�moderate�violations�of�this�assumption� (i�e�,�in�terms�of�Type�I�and�II�error�rates)��Specifically,�effects�of�the�violation�will�be�mini- mal�except�for�small�n’s,�for�unequal�n’s,�and/or�for�extreme�nonnormality��Violation�of�the� normality�assumption�may�be�a�result�of�outliers��The�simplest�outlier�detection�procedure� is�to�look�for�observations�that�are�more�than�two�or�three�standard�deviations�from�their� respective�group�mean��We�recommend�(and�will�illustrate�later)�inspection�of�residuals�for� examination�of�evidence�of�normality��Formal�procedures�for�the�detection�of�outliers�are� now�available�in�many�statistical�packages�
The� following� graphical� techniques� can� be� used� to� detect� violations� of� the� normality� assumption:�(a)�the�frequency�distributions�of�the�scores�or�the�residuals�for�each�group� (through�stem-and-leaf�plots,�boxplots,�histograms,�or�residual�plots),�(b)�the�normal�prob- ability� or� quantile–quantile� (Q–Q)� plot,� or� (c)� a� plot� of� group� means� versus� group� vari- ances� (which� should� be� independent� of� one� another)�� There� are� also� several� statistical� procedures�available�for�the�detection�of�nonnormality�[e�g�,�the�Shapiro–Wilk�(S–W)�test,� 1965]��Transformations�can�also�be�used�to�normalize�the�data��For�instance,�a�nonlinear� relationship�between�X�and�Y�may�result�in�violations�of�the�normality�and/or�homosce- dasticity�assumptions��Readers�interested�in�learning�more�about�potential�data�transfor- mations� are� referred� to� sources� such� as� Bradley� (1982),� Box� and� Cox� (1964),� or� Mosteller� and�Tukey�(1977)�
In�the�example�data,�the�residuals�shown�in�Figure�11�2�appear�to�be�somewhat�normal� in�shape,�especially�considering�the�groups�have�fairly�small�n’s��This�is�suggested�by�the� random�display�of�points��In�addition,�for�the�residuals�overall,�skewness�=�−�2389�and�kur- tosis�=�−�0191,�indicating�a�small�departure�from�normality��Thus,�it�appears�that�all�of�our� assumptions�have�been�satisfied�for�the�example�data��We�will�delve�further�into�examina- tion�of�assumptions�later�as�we�illustrate�how�to�use�SPSS�to�conduct�a�one-way�ANOVA�
A� summary� of� the� assumptions� and� the� effects� of� their� violation� for� the� one-factor� ANOVA�design�are�presented�in�Table�11�6��Note�that�in�some�texts,�the�assumptions�are� written�in�terms�of�the�residuals�rather�than�the�raw�scores,�but�this�makes�no�difference� for�our�purposes�
312 An Introduction to Statistical Concepts
11.6 Unequal n’s or Unbalanced Procedure
Up� to� this� point� in� the� chapter,� we� have� only� considered� the� equal� n’s� or� balanced� case� where�the�number�of�observations� is� equal�for�each�group��This�was�done�only�to�make� things�simple�for�presentation�purposes��However,�we�do�not�need�to�assume�that�the�n’s� must�be�equal�(as�some�textbooks�incorrectly�do)��This�section�briefly�describes�the�unequal� n’s� or� unbalanced case�� For� our� purposes,� the� major� statistical� software� can� handle� the� analysis�of�this�case�for�the�one-factor�ANOVA�model�without�any�special�attention��Thus,� interpretation�of�the�analysis,�the�assumptions,�and�so�forth�are�the�same�as�with�the�equal� n’s�case��However,�once�we�get�to�factorial�designs�in�Chapter�13,�things�become�a�bit�more� complicated�for�the�unequal�n’s�or�unbalanced�case�
11.7 Alternative ANOVA Procedures
There�are�several�alternatives�to�the�parametric�one-factor�fixed-effects�ANOVA��These� include�the�Kruskal�and�Wallis�(1952)�one-factor�ANOVA,�the�Welch�(1951)�test,�the�Brown� and�Forsythe�(1974)�procedure,�and�the�James�(1951)�procedures��You�may�recognize�the� Welch�and�Brown–Forsythe�procedures�as�similar�alternatives�to�the�independent�t�test�
11.7.1 kruskal–Wallis Test
The�Kruskal–Wallis� test�makes� no�normality�assumption�about�the�population�distribu- tions,�although�it�assumes�similar�distributional�shapes,�but�still�assumes�equal�popula- tion� variances� across� the� groups� (although� heterogeneity� does� have� some� effect� on� this� test,�it�is�less�than�with�the�parametric�ANOVA)��When�the�normality�assumption�is�met,� or�nearly�so�(i�e�,�with�mild�nonnormality),�the�parametric�ANOVA�is�slightly�more�pow- erful� than� the� Kruskal–Wallis� test� (i�e�,� less� likelihood� of� a� Type� II� error)�� Otherwise� the� Kruskal–Wallis�test�is�more�powerful�
Table 11.6
Assumptions,�Evidence�to�Examine,�and�Effects�of�Violations:�One-Factor�ANOVA�Design
Assumption Evidence to Examine Effect of Assumption Violation
Independence •�Scatterplot�of�residuals�by�group Increased�likelihood�of�a�Type�I�and/or� Type�II�error�in�the�F�statistic;�influences� standard�errors�of�means�and�thus� inferences�about�those�means
Homogeneity� of�variance
•�Scatterplot�of�residuals�by�X •��Formal�test�of�equal�variances�(e�g�, Levene’s�
test)
Bias�in�SSwith;�increased�likelihood�of�a� Type I�and/or�Type�II�error;�less�effect� with�equal�or�nearly�equal�n’s;�effect� decreases�as�n�increases
Normality •��Graphs�of�residuals�(or�scores)�by�group�(e�g�,� boxplots,�histograms,�stem-and-leaf�plots)
•�Skewness�and�kurtosis�of�residuals •�Q–Q�plots�of�residuals •�Formal�tests�of�normality�of�residuals •�Plot�of�group�means�by�group�variances
Minimal�effect�with�moderate�violation;� effect�less�severe�with�large�n’s,�with�equal� or�nearly�equal�n’s,�and/or�with� homogeneously�shaped�distributions
313One-Factor Analysis of Variance: Fixed-Effects Model
The� Kruskal–Wallis� procedure� works� as� follows�� First,� the� observations� on� the� depen- dent�measure�are�rank�ordered,�regardless�of�group�assignment�(the�ranking�is�done�by� the�computer)��That�is,�the�observations�are�ranked�from�highest�to�lowest,�disregarding� group� membership�� The� procedure� essentially� tests� whether� the� mean� ranks� are� differ- ent�across�the�groups�such�that�they�are�unlikely�to�represent�random�samples�from�the� same� population�� Thus,� according� to� the� null� hypothesis,� the� mean� rank� is� the� same� for� each�group,�whereas�for�the�alternative�hypothesis,�the�mean�rank�is�not�the�same�across� groups��The�test�statistic�is�denoted�by�H�and�is�compared�to�the�critical�value�α χ J −1
2 ��The�
null�hypothesis�is�rejected�if�the�test�statistic�H�exceeds�the�χ2�critical�value� There� are� two� situations� to� consider� with� this� test�� First,� the� χ2� critical� value� is� really� only�
appropriate�when�there�are�at�least�three�groups�and�at�least�five�observations�per�group�(i�e�,�the� χ2�is�not�an�exact�sampling�distribution�of�H)��The�second�situation�is�that�when�there�are�tied� ranks,�the�sampling�distribution�of�H�can�be�affected��Typically�a�midranks�procedure�is�used,� which�results�in�an�overly�conservative�Kruskal–Wallis�test��A�correction�for�ties�is�commonly� used��Unless�the�number�of�ties�is�relatively�large,�the�effect�of�the�correction�is�minimal�
Using�the�statistics�lab�data�as�an�example,�we�perform�the�Kruskal–Wallis�ANOVA��The� test�statistic�H�=�13�0610�is�compared�with�the�critical�value�. .05 3
2 7 81χ = ,�from�Table�A�3,�and� the�result�is�that�H0�is�rejected�(p�=��005)��Thus,�the�Kruskal–Wallis�result�agrees�with�the�result� of�the�parametric�ANOVA��This�should�not�be�surprising�because�the�normality�assumption� apparently�was�met��Thus,�we�would�probably�not�have�done�the�Kruskal–Wallis�test�for�the� example�data��We�merely�provide�it�for�purposes�of�explanation�and�comparison�
In� summary,� the� Kruskal–Wallis� test� can� be� used� as� an� alternative� to� the� parametric� one-factor�ANOVA�under�nonnormality�and/or�when�data�on�the�dependent�variable�are� ordinal��Under�normality�and�with�interval/ratio�dependent�variable�data,�the�parametric� ANOVA�is�more�powerful�than�the�Kruskal–Wallis�test�and�thus�is�the�preferred�method�
11.7.2 Welch, brown–Forsythe, and James procedures
Next� we� briefly� consider� the� following� procedures� for� the� heteroscedasticity� condition:� the�Welch�(1951)�test,�the�Brown�and�Forsythe�(1974)�procedure,�and�the�James�(1951)�first-� and�second-order�procedures�(more�fully�described�by�Coombs�et�al�,�1996;�Myers�&�Well,� 1995;�Wilcox,�1996,�2003)��These�procedures�do�not�require�homogeneity��Current�research� suggests�that�(a)�under�homogeneity,�the�F�test�is�slightly�more�powerful�than�any�of�these� procedures,�and�(b)�under�heterogeneity,�each�of�these�alternative�procedures�is�more�pow- erful�than�the�F,�although�the�choice�among�them�depends�on�several�conditions,�making�a� recommendation�among�these�alternative�procedures�somewhat�complicated�(e�g�,�Clinch� &�Keselman,�1982;�Coombs�et�al�,�1996;�Tomarken�&�Serlin,�1986)��The�Kruskal–Wallis�test� is�widely�available� in�the�major�statistical�software,�and�the�Welch�and�Brown–Forsythe� procedures� are� available� in� the� SPSS� one-way� ANOVA� module�� Wilcox� (1996,� 2003)� also� provides�assistance�for�these�alternative�procedures�
11.8 SPSS and G*Power
Next�we�consider�the�use�of�SPSS�for�the�statistics�lab�example��Instructions�for�determining� the�one-way�ANOVA�using�SPSS�are�presented�first,�followed�by�additional�steps�for�examin- ing�the�assumptions�for�the�one-way�ANOVA��Next,�instructions�for�computing�the�Kruskal– Wallis�and�Brown�and�Forsythe�are�presented��Finally�we�return�to�G*Power�for�this�model�
314 An Introduction to Statistical Concepts
One-Way ANOVA
Note� that� SPSS� needs� the� data� to� be� in� a� specific� form� for� any� of� the� following� analyses� to� proceed,� which� is� different� from� the� layout� of� the� data� in� Table� 11�1�� For� a� one-factor� ANOVA,� the� dataset� must� consist� of� at� least� two� variables� or� columns�� One� column� or� variable� indicates� the� levels� or� categories� of� the� independent� variable,� and� the� second� is� for�the�dependent�variable��Each�row�then�represents�one�individual,�indicating�the�level� or�group�that�individual�is�a�member�of�(1,�2,�3,�or�4�in�our�example),�and�their�score�on�the� dependent�variable��Thus,�we�wind�up�with�two�long�columns�of�group�values�and�scores� as�shown�in�the�following�screenshot�
The “independent variable” is labeled “Group”
where each value represents the attractiveness of the statistics
lab instructor to which the student was assigned. One,
you recall, represented “unattractive”. Thus there
were eight students randomly assigned to an
“unattractive” instructor. Since each of
these eight students was in the same group, each is
coded with the same value (1, which represents that their
group was assigned to an “unattractive” instructor).
The “dependent variable” is “Labs” and represents the number of statistics labs the
student attended.
The other groups (2, 3, and 4) follow this pattern as well.
315One-Factor Analysis of Variance: Fixed-Effects Model
Step 1.� To� conduct� a� one-way� ANOVA,� go� to� “Analyze”� in� the� top� pulldown� menu,� then�select�“General Linear Model,”�and�then�select�“Univariate.”�Following�the� screenshot�(step�1)�as�follows�produces�the�“Univariate”�dialog�box�
A
B C
One-way ANOVA:
Step 1
Step 2.�Click�the�dependent�variable�(e�g�,�number�of�statistics�labs�attended)�and�move� it�into�the�“Dependent Variable”�box�by�clicking�the�arrow�button��Click�the�indepen- dent�variable�(e�g�,�level�of�attractiveness)�and�move�it�into�the�“Fixed Factors”�box�by� clicking�the�arrow�button��Next,�click�on�“Options.”
Select the dependent variable from the list on the
left and use the arrow to move to the “Dependent Variable” box
on the right. Select the
independent variable from the list on the left and use the arrow to
move to the“Fixed Factor(s)” box
on the right.
Clicking on “Plots” will allow
you to generate profile plots.
Clicking on “Save” will allow you to
save various forms of residuals, among
other variables.
Clicking on “Options” will allow you to obtain a number of other statistics (e.g., descriptive statistics,
effect size, power, homogeneity tests).
One-way ANOVA:
Step 2
316 An Introduction to Statistical Concepts
Step 3.� Clicking� on� “Options”� will� provide� the� option� to� select� such� information� as�“Descriptive Statistics,” “Estimates of effect size,” “Observed power,”�and�“Homogeneity tests”�(i�e�,�Levene’s�test�for�equal�variances)�(those�are� the� options� that� we� typically� utilize)�� Click� on� “Continue”� to� return� to� the� original� dialog�box�
Select from the list on the left those variables
that you wish to display means for and use the arrow to move to the
“Display Means for” box on the right.
One-way ANOVA:
Step 3
Step 4.�From�the�“Univariate”�dialog�box,�click�on�“Plots”�to�obtain�a�profile�plot�of� means��Click�the�independent�variable�(e�g�,�level�of�attractiveness�labeled�as�“Group”)�and� move�it�into�the�“Horizontal Axis”�box�by�clicking�the�arrow�button�(see�screenshot� step� 4a)�� Then� click� on�“Add”� to� move� the� variable� into� the�“Plots”� box� at� the� bottom� of�the�dialog�box�(see�screenshot�step�4b)��Click�on�“Continue”�to�return�to�the�original� dialog�box�
317One-Factor Analysis of Variance: Fixed-Effects Model
Select the independent variable from the list on the
left and use the arrow to move to the “Horizontal Axis” box on the right.
Then click “Add” to move the variable into the “Plots” box at the bottom.
One-way ANOVA: Step 4b
One-way ANOVA: Step 4a
Step 5.� From� the� “Univariate”� dialog� box,� click� on�“Save”� to� select� those� elements� that�you�want�to�save�(in�our�case,�we�want�to�save�the�unstandardized�residuals�which� will�be�used�later�to�examine�the�extent�to�which�normality�and�independence�are�met)�� From�the�“Univariate”�dialog�box,�click�on�“OK”�to�return�to�generate�the�output�
One-way ANOVA: Step 5
Interpreting the output:� Annotated� results� are� presented� in� Table� 11�7,� and� the� profile�plot�is�shown�in�Figure�11�3�
318 An Introduction to Statistical Concepts
Table 11.7
Selected�SPSS�Results�for�the�Statistics�Lab�Example
Between-Subjects Factors
Value Label N
1.00 Unattractive 8
2.00 Slightly
attractive 8
3.00 Moderately
attractive 8
Level of attractiveness
4.00 Very attractive 8
Descriptive Statistics Dependent Variable: Number of Statistics Labs Attended
Level of Attractiveness Mean Std. Deviation N
Unattractive 11.1250 Slightly attractive 17.8750 Moderately attractive 20.2500 Very attractive 24.3750 Total 18.4062
5.48862 5.93867 7.28501 5.09727 7.51283
8 8 8 8
32
Levene's Test of Equality of Error Variancesa
Dependent Variable: Number of Statistics Labs Attended
F df 1 df 2 Sig.
.905 3 28 .451
Tests the null hypothesis that the error variance of the dependent variable is equal across groups.
a Design: intercept + group
The table labeled “Between- Subjects Factors” provides
sample sizes for each of the categories of the independent variable (recall that the independent variable is the
“between subjects factor”).
The table labeled “Descriptive Statistics” provides basic
descriptive statistics (means, standard deviations, and sample sizes) for each group of the independent
variable.
The F test (and associated p value) for Levene’s Test for Equality of Error Variances is reviewed to determine if equal variances can be assumed.
In this case, we meet the assumption (as p is greater than α). Note that df 1 is degrees of
freedom for the numerator (calculated as J – 1) and df 2 are the degrees of freedom for the
denominator (calculated as N – J ).
319One-Factor Analysis of Variance: Fixed-Effects Model
Table 11.7 (continued)
Selected�SPSS�Results�for�the�Statistics�Lab�Example
The row labeled “GROUP” is the independent variable or between-groups variable. The between-groups mean square (246.198) tells how much the group means vary. The degrees of freedom for between groups is J – 1 (3 in this example).
The p value for the omnibus F test is .001. This indicates there is a statistically significant difference in the mean number of statistics labs attended based on attractiveness of the instructor. The probability of observing these mean differences or more extreme mean differences by chance if the null hypothesis is really true (i.e., if the means really are equal) is substantially less than 1%. We reject the null hypothesis that all the population means are equal. For this example, this provides evidence to suggest that number of stats labs attended differs based on attractiveness of the instructor.
The omnibus F test is computed as: 246.198 36.112
6.818 MSbetw MSwith
F = = =
Partial eta squared is one measure of effect size: SSbetw 738.594 SStotal 1749.719
η2p = = = .422
We can interpret this to mean that approximately 42% of the variation in the dependent variable (in this case, number of statistics labs attended) is accounted for by
the attractiveness of the statistics lab instructor.
Tests of Between-Subjects Effects Dependent Variable: Number of Statistics Labs Attended
Source Type III Sum of
Squares df Mean
Square F Sig. Partial Eta Squared
Noncent. Parameter
Observed Powerb
Corrected model Intercept
738.594a 3 246.198 6.818 .001 .422 20.453 .956 10841.281 1 10841.281 300.216 .000 .915 300.216 1.000
Group 738.594 3 246.198 6.818 .001 .422 20.453 .956 Error 1011.125 28 36.112 Total 12591.000 32 Corrected total 1749.719 31
a R squared = .422 (adjusted R squared = .360). b Computed using alpha = .05. The row labeled “Error” is within
groups. The within groups mean square tells us how much the
observations within the groups vary (i.e., 36.112). The degrees of freedom
for within groups is (N – J ) or the total sample size minus the number of levels
of the independent variable. The row labeled “corrected total” is the sum of squares total. The degrees of freedom for the total is (N – 1) or the
total sample size minus 1.
Observed power tells whether our test is powerful enough to
detect mean differences if they
really exist. Power of .956 indicates that the probability of rejecting the null hypothesis if it is really false is about 96%; this represents
strong power.
R squared is listed as a footnote underneath the table. R squared is the ratio of sum of squares between divided by sum of squares total:
and, in the case of one-way ANOVA, is also the simple bivariate Pearson correlation between the independent variable and dependent variable squared.
R2 = = = .422 738.594
1749.719 SSbetw SStotal
(continued)
320 An Introduction to Statistical Concepts
Table 11.7 (continued)
Selected�SPSS�Results�for�the�Statistics�Lab�Example
Estimated Marginal Means 1. Grand Mean
Dependent Variable: Number of Statistics Labs Attended
95% Confidence IntervalMean Std. Error Lower Bound Upper Bound
18.406 1.062 16.230 20.582
2. Level of Attractiveness
Dependent Variable: Number of Statistics Labs Attended
95% Confidence Interval Level of Attractiveness Mean Std. Error Lower Bound Upper Bound
Unattractive 11.125 2.125 6.773 15.477 Slightly attractive 17.875 2.125 13.523 22.227 Moderately attractive 20.250 2.125 15.898 24.602 Very attractive 24.375 2.125 20.023 28.727
The “Grand Mean” (in this case, 18.406) represents the overall mean, regardless of group membership, on the dependent variable. The 95% CI represents the CI of the grand mean.
The table labeled “Level of attractiveness” provides descriptive statistics for each of the categories of the independent variable (notice that these are the same means reported previously). In addition to means, the SE and 95% CI of the means are reported.
The Kruskal–Wallis procedure is shown here. The p value (denoted here as Asymp. sig. for
asymptotic significance) is less than α, therefore the null hypothesis is also rejected
for this nonparametric test.
The Welch and Brown–Forsythe robust ANOVA procedures are shown here. For
both tests, the p value is less than α, therefore the null hypothesis is also rejected
for these robust tests.
Chi-square df
Test Statisticsa,b
13.061 dv
dv
3 .005
Robust Tests of Equality of Means
Asymp. sig. a Kruskal–Wallis test. b Grouping variable: group.
Welch Brown–Forsythe
7.862 6.818 3 25.882 .002
.002 Sig.df 2df 1Statistica
15.4543
a Asymptotically F distributed.
FIGuRe 11.3 Profile�plot�for�statistics�lab�example�
1.00 2.00 3.00 Level of attractiveness
4.00
24.00
22.00
20.00
18.00
Es tim
at ed
m ar
gi na
l m ea
ns
16.00
14.00
10.00
12.00
Estimated marginal means of number of statistics labs attended
321One-Factor Analysis of Variance: Fixed-Effects Model
Examining Data for Assumptions
Normality
The residuals are computed by subtracting the group
mean from the dependent variable value for each
observation. For example, the mean
number of labs attended for group 1 was 11.125.
The residual for person 1 is then (15 – 11.125 = 3.88).
As we look at our raw data, we see a new variable has been added to our dataset
labeled RES_1. This is our residual.
The residual will be used to review the assumptions of
normality and independence.
Generating normality evidence:�As�alluded�to�earlier�in�the�chapter,�understand- ing� the� distributional� shape,� specifically� the� extent� to� which� normality� is� a� reasonable� assumption,�is�important��For�the�one-way�ANOVA,�the�distributional�shape�for�the�resid- uals�should�be�a�normal�distribution��We�can�again�use�“Explore”�to�examine�the�extent� to�which�the�assumption�of�normality�is�met�
The� general� steps� for� accessing� “Explore”� have� been� presented� in� previous� chapters� and�will�not�be�repeated�here��Click�the�residual�and�move�it�into�the�“Dependent List”� box�by�clicking�on�the�arrow�button��The�procedures�for�selecting�normality�statistics�were� presented� in� Chapter� 6� and� remain� the� same� here:� Click� on� “Plots”� in� the� upper� right� corner��Place�a�checkmark�in�the�boxes�for�“Normality plots with tests”�and�also� for�“Histogram.”�Then�click�“Continue”�to�return�to�the�main�“Explore”�dialog�box�� Then�click�“OK”�to�generate�the�output�
322 An Introduction to Statistical Concepts
Select residuals from the list on the left and
use the arrow to move to the
“Dependent List” box on the right.
Then click on “Plots.”
Generating normality evidence
Interpreting normality evidence:� We� have� already� developed� a� good� under- standing�of�how�to�interpret�some�forms�of�evidence�of�normality�including�skewness�and� kurtosis,�histograms,�and�boxplots�
Mean 95% Confidence interval Lower bound
Upper boundfor mean 5% Trimmed mean Median Variance Std. deviation Minimum Maximum Range Interquartile range Skewness Kurtosis
Residual for labs
Descriptives Statistic Std. Error
.0000 1.00959 –2.0591
2.0591 .0260 .8125
32.617 5.71112 –10.25
9.88 20.13
9.25 –.239
–1.019 .809 .414
The�skewness�statistic�of�the�residuals�is�−�239�and�kurtosis�is�−1�019—both�within�the� range�of�an�absolute�value�of�2�0,�suggesting�some�evidence�of�normality�
The�histogram�of�residuals�is�not�exactly�what�most�researchers�would�consider�a�classic� normally�shaped�distribution,�but�it�approaches�a�normal�distribution�and�there�is�nothing� to�suggest�normality�may�be�an�unreasonable�assumption�
323One-Factor Analysis of Variance: Fixed-Effects Model
6
4
Fr eq
ue nc
y
2
0 –10.00 –5.00 .00
Residual for labs 5.00 10.00
Histogram
Mean = –6.66E – 16 Std. dev. = 5.711 N = 32
There�are�a�few�other�statistics�that�can�be�used�to�gauge�normality��The�formal�test�of� normality,� the� S–W� test� (SW)� (Shapiro� &� Wilk,� 1965),� provides� evidence� of� the� extent� to� which� our� sample� distribution� is� statistically� different� from� a� normal� distribution�� The� output�for�the�S–W�test�is�presented�in�the�following�and�suggests�that�our�sample�distri- bution�for�residuals�is�not�statistically�significantly�different�than�what�would�be�expected� from�a�normal�distribution�(SW�=��958,�df�=�32,�p�=��240)�
Tests of Normality
Residual for labs
a Lilliefors significance correction. * This is a lower bound of the true significance.
Statistic Statisticdf dfSig. Sig. .112 32 .200 .958 32 .240
Shapiro–WilkKolmogorov–Smirnova
Q–Q� plots� are� also� often� examined� to� determine� evidence� of� normality�� Q–Q� plots� are�graphs�that�plot�quantiles�of�the�theoretical�normal�distribution�against�quantiles� of�the�sample�distribution��Points�that�fall�on�or�close�to�the�diagonal�line�suggest�evi- dence�of�normality��The�Q–Q�plot�of�residuals�shown�in�the�following�suggests�relative� normality�
324 An Introduction to Statistical Concepts
2
1
0
–1
–2
–3
–15 –10 –5 0 Observed value
5 10 15
Ex pe
ct ed
n or
m al
Normal Q–Q plot of residual for labs
Examination�of�the�following�boxplot�suggests�a�relatively�normal�distributional�shape� of�residuals�and�no�outliers�
10.00
5.00
.00
–5.00
–10.00
–15.00
Residual for labs
Considering�the�forms�of�evidence�we�have�examined,�skewness�and�kurtosis�statistics,� the�S–W�test,�the�Q–Q�plot,�and�the�boxplot,�all�suggest�normality�is�a�reasonable�assump- tion��We�can�be�reasonably�assured�we�have�met�the�assumption�of�normality�of�the�depen- dent�variable�for�each�group�of�the�independent�variable�
Independence
The� only� assumption� we� have� not� tested� for� yet� is� independence�� If� subjects� have� been� randomly�assigned�to�conditions�(in�other�words,�the�different�levels�of�the�independent�
325One-Factor Analysis of Variance: Fixed-Effects Model
variable),�the�assumption�of�independence�has�been�met��In�this�illustration,�students�were� randomly� assigned� to� instructor,� and� thus,� the� assumption� of� independence� was� met�� However,�we�often�use�independent�variables�that�do�not�allow�random�assignment,�such� as� preexisting� characteristics� such� as� education� level� (high� school� diploma,� bachelor’s,� master’s,� or� terminal� degrees)�� We� can� plot� residuals� against� levels� of� our� independent� variable�using�a�scatterplot�to�get�an�idea�of�whether�or�not�there�are�patterns�in�the�data� and� thereby� provide� an� indication� of� whether� we� have� met� this� assumption�� Remember� that� these� variables� were� added� to� the� dataset� by� saving� the� unstandardized� residuals� when�we�generated�the�ANOVA�model�
Please�note�that�some�researchers�do�not�believe�that�the�assumption�of�independence� can� be� tested�� If� there� is� not� a� random� assignment� to� groups,� then� these� researchers� believe�this�assumption�has�been�violated—period��The�plot�that�we�generate�will�give� us�a�general�idea�of�patterns,�however,�in�situations�where�random�assignment�was�not� performed�
The�general�steps�for�generating�a�simple�scatterplot�through�“Scatter/dot”�have� been�presented�in�a�previous�chapter�(e�g�,�Chapter�10),�and�they�will�not�be�reiterated� here��From�the�“Simple Scatterplot”�dialog�screen,�click�the�residual�variable�and� move�it�into�the�“Y Axis”�box�by�clicking�on�the�arrow��Click�the�independent�vari- able�(e�g�,�level�of�attractiveness)�and�move�it�into�the�“X Axis”�box�by�clicking�on�the� arrow��Then�click�“OK.”
Double�click�on�the�graph�in�the�output�to�activate�the�chart�editor�
326 An Introduction to Statistical Concepts
10.00
5.00
.00
–5.00
Re si
du al
fo r l
ab s
–10.00
–15.00
1.00 1.50 2.00 2.50 Level of attractiveness
3.00 3.50 4.00
327One-Factor Analysis of Variance: Fixed-Effects Model
10.00
5.00
.00
–5.00
Re si
du al
fo r l
ab s
–10.00
–15.00
1.00 1.50 2.00 2.50 Level of attractiveness
3.00 3.50 4.00
Interpreting independence evidence:� In� examining� the� scatterplot� for� evi- dence� of� independence,� the� points� should� be� falling� relatively� randomly� above� and� below�the�reference�line��In�this�example,�our�scatterplot�suggests�evidence�of�indepen- dence�with�a�relatively�random�display�of�points�above�and�below�the�horizontal�line� at� 0�� Thus,� had� we� not� met� the� assumption� of� independence� through� random� assign- ment�of�cases�to�groups,�this�would�have�provided�evidence�that�independence�was�a� reasonable�assumption�
Nonparametric Procedures
Results�from�some�of�the�recommended�alternative�procedures�can�be�obtained�from�two� other� SPSS� modules�� Here� we� discuss� the� Kruskal–Wallis,� Welch,� and� Brown–Forsythe� procedures�
Kruskal–Wallis
Step 1:� To� conduct� a� Kruskal–Wallis� test,� go� to� the�“Analyze”� in� the� top� pulldown� menu,� then� select� “Nonparametric Tests,”� then� select� “Legacy Dialogs,”� and� finally�select�“K Independent Samples.”�Following�the�screenshot�(step�1)�as�follows� produces�the�“Tests for Several Independent Samples”�dialog�box�
328 An Introduction to Statistical Concepts
A
B
C
D
Kruskal–Wallis: Step 1
Step 2:�Next,�from�the�main�“Tests for Several Independent Samples”�dia- log�box,�click�the�dependent�variable�(e�g�,�number�of�statistics�labs�attended)�and�move� it�into�the�“Test Variable List”�box�by�clicking�on�the�arrow�button��Next,�click�the� grouping� variable� (e�g�,� attractiveness� of� instructor)� and� move� it� into� the� “Grouping Variable”� box� by� clicking� on� the� arrow� button�� You� will� notice� that� there� are� two� question� marks� next� to� the� name� of� your� grouping� variable�� This� is� SPSS� letting� you� know�that�you�need�to�define�(numerically)�which�categories�of�the�grouping�variable� you�want�to�include�in�the�analysis�(this�must�be�done�by�identifying�a�range�of�values� for�all�groups�of�interest)��To�do�that,�click�on�“Define Range�”�We�have�four�groups� or�levels�of�our�independent�variable�(labeled�1,�2,�3,�and�4�in�our�raw�data);�thus,�enter� 1�as�the�minimum�and�4�as�the�maximum��In�the�lower�left�portion�of�the�screen�under� “Test Type,”� check� “Kruskal-Wallis H”� to� generate� this� nonparametric� test�� Then�click�on�“OK”�to�generate�the�results�presented�as�follows�
Select the dependent variable from the list on the left and use the arrow to move to the “Test Variable
List” box on the right. Select the independent variable
from the list on the left and use the arrow to move to the “Grouping Variable” box on the right.
Select “Kruskal– Wallis H” as the “Test Type”.
Clicking on “Define Range” will allow you to define the numeric values of the categories for the independent variable.
Kruskal–Wallis:
Step 2b
Kruskal–Wallis:
Step 2a
329One-Factor Analysis of Variance: Fixed-Effects Model
Interpreting the output:�The�Kruskal–Wallis�is�literally�an�ANOVA�of�ranks��Thus,� the�null�hypothesis�is�that�the�mean�ranks�of�the�groups�of�the�independent�variable�will� not� be� significantly� different�� In� this� example,� the� results� (p� =� �005)� suggest� statistically� significant�differences�in�the�mean�ranks�of�the�dependent�variable�by�group�of�the�inde- pendent�variable�
The mean rank is the rank order, from smallest to largest, of the means of
the dependent variable (statistic labs attended) by group (attractiveness
of the lab instructor).
The p value (labeled “Asymp.Sig.”) for the Kruskal– Wallis test is .005. This indicates there is a statistically
significant difference in the mean ranks [i.e., rank order of the mean number of statistic labs attended
by group (i.e., attractiveness of the instructor)].
The probability of observing these mean ranks or more extreme mean ranks by chance if the null hypothesis is
really true (i.e., if the mean ranks are really equal) is substantially less than 1%. We reject the null
hypothesis that all the population mean ranks are equal. For the example, this provides evidence to suggest that the number of statistic labs attended
differs based on the attractiveness of the instructor.
Ranks
Slightly attractive Moderately attractive Very attractive Total
Test Statisticsa,b
Chi-square 13.061 3
.005
Number of Statistics Labs
Attended
df Asymp. sig.
a Kruskal–Wallis test. b Grouping variable: Level of attractiveness.
Level of Attractiveness N Mean Rank
Number of statistics labs Unattractive attended
8 8 15.25
18.75 24.25
8 8
32
7.75
Welch and Brown–Forsythe
Step 1:�To�conduct�the�Welch�and�Brown–Forsythe�procedures,�go�to�the�“Analyze”�in�the� top�pulldown�menu,�then�select�“Compare Means,”�and�then�select�“One-way ANOVA.”� Following�the�screenshot�(step�1)�as�follows�produces�the�“One-way ANOVA”�dialog�box�
A
B
C
Welch and Brown– Forsythe: Step 1
330 An Introduction to Statistical Concepts
Step 2:�Click�the�dependent�variable�(e�g�,�number�of�stats�labs�attended)�and�move�it�into� the�“Dependent List”�box�by�clicking�the�arrow�button��Click�the�independent�variable� (e�g�,�level�of�attractiveness)�and�move�it�into�the�“Factor”�box�by�clicking�the�arrow�but- ton��Next,�click�on�“Options.”
Select the dependent variable from the list on the
left and use the arrow to move to the “Dependent
Variable” box on the right. Select the independent
variable from the list on the left and use the arrow to
move to the “Factor:” box on the right.
Clicking on “Options” will allow
you to obtain a number of other
statistics (including the Welch and
Brown–Forsythe).
Welch and Brown–Forsythe:
Step 2
Step 3:� Clicking� on�“Options”� will� provide� the� option� to� select� such� information� as� “Descriptive,” “Homogeneity of variance test”� (i�e�,� Levene’s� test� for� equal� variances),�“Brown-Forsythe,” “Welch,”�and�“Means plot.”�Click�on�“Continue”� to�return�to�the�original�dialog�box��From�the�“One-way ANOVA”�dialog�box,�click�on�“OK”� to�return�and�to�generate�the�output�
Welch and Brown–Forsythe:
Step 3
Interpreting the output:�For�illustrative�purposes�and�because�the�remainder� of� the� one-way� ANOVA� results� have� been� interpreted� previously,� only� the� results� for� the� Welch�and�Brown–Forsythe�procedures�are�displayed��Both�tests�suggest�there�are�statisti- cal�differences�between�the�groups�in�terms�of�the�number�of�stats�labs�attended�
331One-Factor Analysis of Variance: Fixed-Effects Model
�e p values for the Welch and Brown– Forsythe tests are .002. �ese indicate there is a statistically significant di�erence in the mean number of statistics labs attended per group (i.e., attractiveness of the instructor). �e probability of observing the F statistics (7.862 and 6.818) or larger by chance if the
means of the groups are really equal is substantially less than 1%. We reject the null hypothesis that all the population means are
equal. For this example, this provides evidence to suggest that the number of statistic labs attended di�ers based on
attractiveness of the instructor.
Robust Tests of Equality of Means
Number of Statistics Labs Attended
Welch 7.862 6.818 3
3 15.454 25.882 .002
.002 Brown–Forsythe
a Asymptotically F distributed.
Statistica df 1 df 2 Sig.
For�further�details�on�the�use�of�SPSS�for�these�procedures,�be�sure�to�examine�books�such� as�Page,�Braver,�and�MacKinnon�(2003),�or�Morgan,�Leech,�Gloeckner,�and�Barrett�(2011)�
A� priori� and� post� hoc� power� can� again� be� determined� using� the� specialized� software� described�previously�in�this�text�(e�g�,�G*Power),�or�you�can�consult�a�priori�power�tables�(e�g�,� Cohen,�1988)��As�an�illustration,�we�use�G*Power�to�compute�the�post�hoc�power�of�our�test�
Post Hoc Power for One-Way ANOVA Using G*Power
The�first�thing�that�must�be�done�when�using�G*Power�for�computing�post�hoc�power�is� to�select�the�correct�test�family��In�our�case,�we�conducted�a�one-way�ANOVA��To�find�the� one-way�ANOVA,�we�will�select�“Tests”�in�the�top�pulldown�menu,�then�“Means,”�and� then�“Many groups: ANOVA: One-way (one independent variable).”�Once�that� selection�is�made,�the�“Test family”�automatically�changes�to�“F tests.”
A
B
C
Step 1
332 An Introduction to Statistical Concepts
The�“Type of Power Analysis”�desired�then�needs�to�be�selected��To�compute�post�hoc� power,�we�need�to�select�“Post hoc: Compute achieved power—given�α, sample size, and effect size.”
The default selection for “Test Family” is “t tests”. Following the procedures presented in step 1 will automatically change the test family to “F tests”.
The default selection for “Statistical Test” is “Correlation: Point biserial model.” Following the procedures presented in Step 1 will automatically change the statistical test to “ANOVA: Fixed effects, omnibus, one-way”.
Step 2
Once the parameters are
specified, click on “Calculate”.
The “Input Parameters” for computing post hoc power must be specified (the default values are shown here) including:
1. Effect size f 2. Alpha level 3. Total sample size 4. Number of groups in the independent variable
The�“Input Parameters”�must�then�be�specified��The�first�parameter�is�the�effect� size,� f�� In� our� example,� the� computed� f� effect� size� was� �8546�� The� alpha� level� we� used� was��05,�the�total�sample�size�was�32,�and�the�number�of�groups�(i�e�,�levels�of�the�inde- pendent�variable)�was�4��Once�the�parameters�are�specified,�click�on�“Calculate”�to� find�the�power�statistics�
333One-Factor Analysis of Variance: Fixed-Effects Model
Post hoc power
Here are the post- hoc power results.
The� “Output Parameters”� provide� the� relevant� statistics� given� the� input� just� speci- fied�� In� this� example,� we� were� interested� in� determining� post� hoc� power� for� a� one-way� ANOVA�with�a�computed�effect�size�f�of��8546,�an�alpha�level�of��05,�total�sample�size�of� 32,�and�4�groups�(or�categories)�in�our�independent�variable�
Based� on� those� criteria,� the� post� hoc� power� was� �98�� In� other� words,� with� a� one-way� ANOVA,�computed�effect�size�f�of��8546,�alpha�level�of��05,�total�sample�size�of�32,�and� 4�groups�(or�categories)�in�our�independent�variable,�the�post�hoc�power�of�our�test�was� �98—the�probability�of�rejecting�the�null�hypothesis�when�it�is�really�false�(in�this�case,� the�probability�that�the�means�of�the�dependent�variable�would�be�equal�for�each�level� of�the�independent�variable)�was�98%,�which�would�be�considered�more�than�sufficient� power� (sufficient� power� is� often� �80� or� above)�� Note� that� this� value� is� slightly� different� than�the�observed�value�reported�in�SPSS��Keep�in�mind�that�conducting�power�analysis� a�priori�is�recommended�so�that�you�avoid�a�situation�where,�post�hoc,�you�find�that�the� sample� size� was� not� sufficient� to� reach� the� desired� level� of� power� (given� the� observed� parameters)�
334 An Introduction to Statistical Concepts
A Priori Power for One-Way ANOVA Using G*Power
For� a� priori� power,� we� can� determine� the� total� sample� size� needed� given� an� estimated� effect�size�f,�alpha�level,�desired�power,�and�number�of�groups�of�our�independent�variable�� In�this�example,�had�we�estimated�a�moderate�effect�f�of��25,�alpha�of��05,�desired�power�of� �80,�and�4�groups�in�the�independent�variable,�we�would�need�a�total�sample�size�of�180�(or� 45�per�group)�
Here are the a priori power
results.
A priori power
11.9 Template and APA-Style Write-Up
Finally� we� come� to� an� example� paragraph� of� the� results� for� the� statistics� lab� example�� Recall�that�our�graduate�research�assistant,�Marie,�was�working�on�a�research�project�for� an� independent� study� class� to� determine� if� there� was� a� mean� difference� in� the� number�
335One-Factor Analysis of Variance: Fixed-Effects Model
of� statistics� labs� attended� based� on� the� attractiveness� of� the� lab� instructor�� Her� research� question�was�as�follows:�Is there a mean difference in the number of statistics labs students attend based on the attractiveness of the lab instructor?�Marie�then�generated�a�one-way�ANOVA�as� the� test� of� inference�� A� template� for� writing� a� research� question� for� a� one-way� ANOVA� is�presented�as�follows��Please�note�that�it�is�important�to�ensure�the�reader�understands� the�levels�or�groups�of�the�independent�variable��This�may�be�done�parenthetically�in�the� actual� research� question,� as� an� operational� definition,� or� specified� within� the� methods� section��In�this�example,�parenthetically�we�could�have�stated�the�following:�Is there a mean difference in the number of statistics labs students attend based on the attractiveness of the lab instructor (unattractive, slightly attractive, moderately attractive, very attractive)?
Is there a mean difference in [dependent variable] between [indepen- dent variable]?
It�may�be�helpful�to�preface�the�results�of�the�one-way�ANOVA�with�information�on�an�exam- ination�of�the�extent�to�which�the�assumptions�were�met�(recall�there�are�three�assumptions:� normality,� homogeneity� of� variance,� and� independence)�� This� assists� the� reader� in� under- standing�that�you�were�thorough�in�data�screening�prior�to�conducting�the�test�of�inference�
A one-way ANOVA was conducted to determine if the mean number of statistics labs attended by students differed on the level of attractiveness of the statistics lab instructor. The assumption of normality was tested and met via examination of the residuals. Review of the S-W test for normality (SW = .958, df = 32, p = .240) and skewness (−.239) and kurtosis (−1.019) statistics suggested that normality was a reasonable assumption. The boxplot suggested a rela- tively normal distributional shape (with no outliers) of the residu- als. The Q–Q plot and histogram suggested normality was reasonable. According to Levene’s test, the homogeneity of variance assumption was satisfied [F(3, 28) = .905, p = .451]. Random assignment of indi- viduals to groups helped ensure that the assumption of independence was met. Additionally, a scatterplot of residuals against the levels of the independent variable was reviewed. A random display of points around 0 provided further evidence that the assumption of indepen- dence was met.
Here� is� an� APA-style� example� paragraph� of� results� for� the� one-way� ANOVA� (remember� that� this� will� be� prefaced� by� the� previous� paragraph� reporting� the� extent� to� which� the� assumptions�of�the�test�were�met)�
From Table 11.7, we see that the one-way ANOVA is statistically sig- nificant (F = 6.818, df = 3, 28, p = .001), the effect size is rather large (η2 = .422; suggesting about 42% of the variance of number of statistics labs attended is due to differences in the attractive- ness of the instructor), and observed power is quite strong (.956). The means and standard deviations of the number of statistics labs attended for each group of the independent variable were as follows: 11.125 (SD = 5.489) for the unattractive level, 17.875 (SD = 5.939) for the slightly attractive level, 20.250 (SD = 7.285) for the moderately
336 An Introduction to Statistical Concepts
attractive level, and 24.375 (SD = 5.097) for the very attractive level. The means and profile plot (Figure 11.3) suggest that with increas- ing instructor attractiveness, there was a corresponding increase in mean lab attendance. For completeness, we also conducted several alternative procedures. The Kruskal-Wallis test (χ2 = 13.061, df = 3, p = .005), the Welch procedure (Fasymp = 7.862, df1 = 3, df2 = 15.454, p = .002), and the Brown-Forsythe procedure (Fasymp = 6.818, df1 = 3, df2 = 25.882, p = .002) also indicated a statistically significant effect of instructor attractiveness on statistics lab attendance, providing further support for the assumptions being satisfied.
11.10 Summary
In�this�chapter,�methods�involving�the�comparison�of�multiple�group�means�for�a�single� independent�variable�were�considered��The�chapter�began�with�a�look�at�the�characteristics� of� the� one-factor� fixed-effects� ANOVA� including� (a)� control� of� the� experimentwise� error� rate�through�an�omnibus�test,�(b)�one�independent�variable�with�two�or�more�fixed�levels,� (c)� individuals� are� randomly� assigned� to� levels� or� groups� and� then� exposed� to� only� one� level�of�the�independent�variable,�and�(d)�the�dependent�variable�is�measured�at�least�at�the� interval�level��Next,�a�discussion�of�the�theory�underlying�ANOVA�was�conducted��Here� we�examined�the�concepts�of�between-�and�within-groups�variability,�sources�of�variation,� and� partitioning� the� sums� of� squares�� The� ANOVA� model� was� examined�� Some� discus- sion� was� also� devoted� to� the� ANOVA� assumptions,� their� assessment,� and� how� to� deal� with� assumption� violations�� Finally,� alternative� ANOVA� procedures� were� described�� At� this� point,� you� should� have� met� the� following� objectives:� (a)� be� able� to� understand� the� characteristics�and�concepts�underlying�the�one-factor�ANOVA,�(b)�be�able�to�determine� and�interpret�the�results�of�a�one-factor�ANOVA,�and�(c)�be�able�to�understand�and�evalu- ate�the�assumptions�of�the�one-factor�ANOVA��Chapter�12�considers�a�number�of�MCPs�for� further�examination�of�sets�of�means��Chapter�13�returns�to�ANOVA�and�discusses�models� which�have�more�than�one�independent�variable�
Problems
Conceptual problems
11.1� Data�for�three�independent�random�samples,�each�of�size�4,�are�analyzed�by�a�one- factor�ANOVA�fixed-effects�model��If�the�values�of�the�sample�means�are�all�equal,� what�is�the�value�of�MSbetw?
� a�� 0 � b�� 1 � c�� 2 � d�� 3
337One-Factor Analysis of Variance: Fixed-Effects Model
11.2� For�a�one-factor�ANOVA�fixed-effects�model,�which�of�the�following�is�always�true? � a�� dfbetw�+�dfwith�=�dftotal � b�� SSbetw�+�SSwith�=�SStotal � c�� MSbetw�+�MSwith�=�MStotal � d�� All�of�the�above � e�� Both�a�and�b 11.3� Suppose�n1�=�19,�n2�=�21,�and�n3�=�23��For�a�one-factor�ANOVA,�the�dfwith�would�be � a�� 2 � b�� 3 � c�� 60 � d�� 62
11.4� Suppose�n1�=�19,�n2�=�21,�and�n3�=�23��For�a�one-factor�ANOVA,�the�dfbetw�would�be � a�� 2 � b�� 3 � c�� 60 � d�� 62
11.5� Suppose�n1�=�19,�n2�=�21,�and�n3�=�23��For�a�one-factor�ANOVA,�the�dftotal�would�be � a�� 2 � b�� 3 � c�� 60 � d�� 62
11.6� Suppose�n1�=�19,�n2�=�21,�and�n3�=�23��For�a�one-factor�ANOVA,�the�df�for�the�numerator� of�the�F�ratio�would�be�which�one�of�the�following?
� a�� 2 � b�� 3 � c�� 60 � d�� 62
11.7� In�a�one-factor�ANOVA,�H0�asserts�that � a�� All�of�the�population�means�are�equal� � b�� The�between-groups�variance�estimate�and�the�within-groups�variance�estimate�
are�both�estimates�of�the�same�population�residual�variance� � c�� The�within-groups�sum�of�squares�is�equal�to�the�between-groups�sum�of�squares� � d�� Both�a�and�b�
11.8� For�a�one-factor�ANOVA�comparing�three�groups�with�n�=�10�in�each�group,�the�F�ratio� has�degrees�of�freedom�equal�to
� a�� 2,�27 � b�� 2,�29 � c�� 3,�27 � d�� 3,�29
338 An Introduction to Statistical Concepts
11.9� For�a�one-factor�ANOVA�comparing�five�groups�with�n�=�50�in�each�group,�the�F�ratio� has�degrees�of�freedom�equal�to
� a�� 4,�245 � b�� 4,�249 � c�� 5,�245 � d�� 5,�249 11.10� Which�of�the�following�is�not�necessary�in�ANOVA? � a�� Observations�are�from�random�and�independent�samples� � b�� The�dependent�variable�is�measured�on�at�least�the�interval�scale� � c�� Populations�have�equal�variances� � d�� Equal�sample�sizes�are�necessary�
11.11� If�you�find�an�F�ratio�of�1�0�in�a�one-factor�ANOVA,�it�means�that � a�� Between-groups�variation�exceeds�within-groups�variation� � b�� Within-groups�variation�exceeds�between-groups�variation� � c�� Between-groups�variation�is�equal�to�within-groups�variation� � d�� Between-groups�variation�exceeds�total�variation�
11.12� �Suppose�students�in�grades�7,�8,�9,�10,�11,�and�12�were�compared�on�absenteeism�� If�ANOVA�were�used�rather�than�multiple�t�tests,�then�the�probability�of�a�Type�I� error�will�be�less��True�or�false?
11.13� Mean�square�is�another�name�for�variance�or�variance�estimate��True�or�false?
11.14� In�ANOVA,�each�independent�variable�is�known�as�a�level��True�or�false?
11.15� A�negative�F�ratio�is�impossible��True�or�false?
11.16� �Suppose�that�for�a�one-factor�ANOVA�with�J�=�4�and�n�=�10,�the�four�sample�means� are�all�equal�to�15��I�assert�that�the�value�of�MSwith�is�necessarily�equal�to�0��Am�I� correct?
11.17� �With� J� =� 3� groups,� I� assert� that� if� you� reject� H0� in� the� one-factor� ANOVA,� you� will� necessarily� conclude� that� all� three� group� means� are� different�� Am� I� correct?
11.18� �The� homoscedasticity� assumption� is� that� the� populations� from� which� each� of� the� samples�are�drawn�are�normally�distributed��True�or�false?
11.19� �When�analyzing�mean�differences�among�more�than�two�samples,�doing�indepen- dent�t�tests�on�all�possible�pairs�of�means
� a�� Decreases�the�probability�of�a�Type�I�error � b�� Does�not�change�the�probability�of�a�Type�I�error � c�� Increases�the�probability�of�a�Type�I�error � d�� Cannot�be�determined�from�the�information�provided
11.20� �Suppose�for�a�one-factor�fixed-effects�ANOVA�with�J�=�5�and�n�=�15,�the�five�sample� means�are�all�equal�to�50��I�assert�that�the�F�test�statistic�cannot�be�significant��Am�I�correct?
339One-Factor Analysis of Variance: Fixed-Effects Model
11.21� �The�independence�assumption�in�ANOVA�is�that�the�observations�in�the�samples�do� not�depend�on�one�another��True�or�false?
11.22� �For�J�=�2�and�α�=��05,� if�the�result�of�the�independent� t�test�is� significant,�then�the� result�of�the�one-factor�fixed-effects�ANOVA�is�uncertain��True�or�false?
11.23� �A�statistician�conducted�a�one-factor�fixed-effects�ANOVA�and�found�the�F�ratio�to� be� less� than� 0�� I� assert� this� means� the� between-groups� variability� is� less� than� the� within-groups�variability��Am�I�correct?
Computational problems
11.1� Complete� the� following� ANOVA� summary� table� for� a� one-factor� ANOVA,� where� there�are�4�groups�receiving�different�headache�medications,�each�with�16�observa- tions,�and�α�=��05�
Source SS df MS F Critical Value and Decision
Between 9�75 — — — Within — — — Total 18�75 —
11.2� A�social�psychologist�wants�to�determine�if�type�of�music�has�any�effect�on�the�num- ber�of�beers�consumed�by�people�in�a�tavern��Four�taverns�are�selected�that�have�dif- ferent�musical�formats��Five�people�are�randomly�sampled�in�each�tavern�and�their� beer�consumption�monitored�for�3�hours��Complete�the�following�one-factor�ANOVA� summary�table�using�α�=��05�
Source SS df MS F Critical Value and Decision
Between — — 7�52 5�01 Within — — — Total — —
11.3� A� psychologist� would� like� to� know� whether� the� season� (fall,� winter,� spring,� and� summer)�has�any�consistent�effect�on�people’s�sexual�activity��In�the�middle�of�each� season,�a�psychologist�selects�a�random�sample�of�n�=�25�students��Each�individual� is�given�a�sexual�activity�questionnaire��A�one-factor�ANOVA�was�used�to�analyze� these�data��Complete�the�following�ANOVA�summary�table�(α�=��05)�
Source SS df MS F Critical Value and Decision
Between — — — 5�00 Within 960 — — Total — —
340 An Introduction to Statistical Concepts
11.4� The� following� five� independent� random� samples� are� obtained� from� five� normally� distributed�populations�with�equal�variances��The�dependent�variable�is�the�number� of�bank�transactions�in�1�month,�and�the�groups�are�five�different�banks�
Group 1 Group 2 Group 3 Group 4 Group 5
16 16 2 5 7 5 10 9 8 12
11 7 11 1 14 23 12 13 5 16 18 7 10 8 11 12 4 13 11 9 12 23 9 9 19 19 13 9 9 24
Use�SPSS�to�conduct�a�one-factor�ANOVA�to�determine�if�the�group�means�are�equal� using� α� =� �05�� Test� the� assumptions,� plot� the� group� means,� consider� an� effect� size,� interpret�the�results,�and�write�an�APA-style�summary�
11.5� The�following�three�independent�random�samples�are�obtained�from�three�normally� distributed� populations� with� equal� variances�� The� dependent� variable� is� starting� hourly�wage,�and�the�groups�are�the�types�of�position�(internship,�co-op,�work�study)�
Group 1: Internship Group 2: Co-op Group 3: Work Study
10 9 8 12 8 9 11 10 8 11 12 10 12 9 8 10 11 9 10 12 9 13 10 8
Use�SPSS�to�conduct�a�one-factor�ANOVA�to�determine�if�the�group�means�are�equal� using� α� =� �05�� Test� the� assumptions,� plot� the� group� means,� consider� an� effect� size,� interpret�the�results,�and�write�an�APA-style�summary�
Interpretive problems
11.1� Using�the�survey�1�dataset�from�the�website,�use�SPSS�to�conduct�a�one-factor�fixed- effects�ANOVA,�including�effect�size,�where�political�view�is�the�grouping�variable� (i�e�,�independent�variable)�(J�=�5)�and�the�dependent�variable�is�a�variable�of�interest� to� you� [the� following� variables� look� interesting:� books,� TV,� exercise,� drinks,� GPA,� GRE-Quantitative�(GRE-Q),�CDs,�hair�appointment]��Then�write�an�APA-style�para- graph�describing�the�results�
11.2� Using�the�survey�1�dataset�from�the�website,�use�SPSS�to�conduct�a�one-factor�fixed- effects�ANOVA,�including�effect�size,�where�hair�color�is�the�grouping�variable�(i�e�,� independent�variable)�(J�=�5)�and�the�dependent�variable�is�a�variable�of�interest�to� you�(the�following�variables�look�interesting:�books,�TV,�exercise,�drinks,�GPA,�GRE- Q,�CDs,�hair�appointment)��Then�write�an�APA-style�paragraph�describing�the results�
341
12 Multiple Comparison Procedures
Chapter Outline
12�1� Concepts�of�Multiple�Comparison�Procedures 12�1�1� Contrasts 12�1�2� Planned�Versus�Post�Hoc�Comparisons 12�1�3� Type�I�Error�Rate 12�1�4� Orthogonal�Contrasts
12�2� Selected�Multiple�Comparison�Procedures 12�2�1� Planned�Analysis�of�Trend 12�2�2� Planned�Orthogonal�Contrasts 12�2�3� Planned�Contrasts�with�Reference�Group:�Dunnett�Method 12�2�4� Other�Planned�Contrasts:�Dunn�(or Bonferroni)�and�Dunn–Sidak�Methods 12�2�5� Complex�Post�Hoc�Contrasts:�Scheffe’�and�Kaiser–Bowden�Methods 12�2�6� Simple�Post�Hoc�Contrasts:�Tukey�HSD,�Tukey– Kramer,�Fisher�LSD,�and�
Hayter�Tests 12�2�7� Simple�Post�Hoc�Contrasts�for�Unequal�Variances:�Games–Howell,�
Dunnett�T3�and�C�Tests 12�2�8� Follow-Up�Tests�to�Kruskal–Wallis
12�3� SPSS 12�4� Template�and�APA-Style�Write-Up
Key Concepts
� 1�� Contrast � 2�� Simple�and�complex�contrasts � 3�� Planned�and�post�hoc�comparisons � 4�� Contrast-�and�family-based�Type�I�error�rates � 5�� Orthogonal�contrasts
In�this�chapter,�our�concern�is�with�multiple comparison procedures�(MCPs)�that�involve� comparisons�among�the�group�means��Recall�from�Chapter�11�the�one-factor�analysis�of� variance�(ANOVA)�where�the�means�from�two�or�more�samples�were�compared��What�do�
342 An Introduction to Statistical Concepts
we�do�if�the�omnibus�F�test�leads�us�to�reject�H0?�First,�consider�the�situation�where�there� are�only�two�samples�(e�g�,�assessing�the�effectiveness�of�two�types�of�medication),�and�H0� has�already�been�rejected�in�the�omnibus�test��Why�was�H0�rejected?�The�answer�should�be� obvious��Those�two�sample�means�must�be�significantly�different�as�there�is�no�other�way� that�the�omnibus�H0�could�have�been�rejected�(e�g�,�one�type�of�medication�is�significantly� more�effective�than�the�other�based�on�an�inspection�of�the�means)�
Second,�consider�the�situation�where�there�are�more�than�two�samples�(e�g�,�three�types� of�medication),�and�H0�has�already�been�rejected�in�the�omnibus�test��Why�was�H0�rejected?� The� answer� is� not� so� obvious�� This� situation� is� one� where� a� multiple� comparison� proce- dure�(MCP)��would�be�quite�informative��Thus,�for�situations�where�there�are�at�least�three� groups�and�the�ANOVA�H0�has�been�rejected,�some�sort�of�MCP�is�necessary�to�determine� which�means�or�combination�of�means�are�different��Third,�consider�the�situation�where�the� researcher�is�not�even�interested�in�the�ANOVA�omnibus�test�but�is�only�interested�in�com- parisons�involving�particular�means�(e�g�,�certain�medications�are�more�effective�than�a�pla- cebo)��This�is�a�situation�where�an�MCP�is�useful�for�evaluating�those�specific�comparisons�
If�the�ANOVA�omnibus�H0�has�been�rejected,�why�not�do�all�possible�independent�t�tests?� First�let�us�return�to�a�similar�question�from�Chapter�11��There�we�asked�about�doing�all� possible�pairwise�independent�t�tests�rather�than�an�ANOVA��The�answer�there�was�to� do�an�omnibus�F�test��The�reasoning�was�related�to�the�probability�of�making�a�Type�I� error� (i�e�,� α),� where� the� researcher� incorrectly� rejects� a� true� null� hypothesis�� Although� the� α� level� for� each� t� test� can� be� controlled� at� a� specified� nominal� level,� say� �05,� what� would�happen�to�the�overall�α�level�for�the�set�of�t�tests?�The�overall�α�level�for�the�set�of� tests,�often�called�the�family-wise�Type�I�error�rate,�would�be�larger�than�the�α�level�for� each�of�the�individual�t�tests��The�optimal�solution,�in�terms�of�maintaining�control�over� our�overall�α�level�as�well�as�maximizing�power,�is�to�conduct�one�overall�omnibus�test�� The�omnibus�test�assesses�the�equality�of�all�of�the�means�simultaneously�
Let�us�apply�the�same�concept�to�the�situation�involving�multiple�comparisons��Rather� than� doing� all� possible� pairwise� independent� t� tests,� where� the� family-wise� error� rate� could�be�quite�large,�one�should�use�a�procedure�that�controls�the�family-wise�error�rate�in� some�way��This�can�be�done�with�MCPs��As�pointed�out�later�in�the�chapter,�there�are�two� main�methods�for�taking�the�Type�I�error�rate�into�account�
This� chapter� is� concerned� with� several� important� new� concepts,� such� as� a� contrast,� planned�versus�post�hoc�comparisons,�the�Type�I�error�rate,�and�orthogonal�contrasts��The� remainder�of�the�chapter�consists�of�selected�MCPs,�including�when�and�how�to�apply�them�� The� terms� comparison� and� contrast� are� used� here� synonymously�� Also,� MCPs� are� only� applicable�for�comparing�levels�of�an�independent�variable�that�are�fixed,�in�other�words,� for�fixed-effects�independent�variables�and�not�for�random-effects�independent�variables�� Our�objectives�are�that�by�the�end�of�this�chapter,�you�will�be�able�to�(a)�understand�the� concepts�underlying�the�MCPs,�(b)�select�the�appropriate�MCP�for�a�given�research�situa- tion,�and�(c)�determine�and�interpret�the�results�of�MCPs�
12.1 Concepts of Multiple Comparison Procedures
In�the�previous�chapter,�Marie,�our�very�capable�educational�researcher�graduate�student,� was�embarking�on�a�very�exciting�research�adventure�of�her�own��She�continues�to�work� toward�completion�of�this�project�
343Multiple Comparison Procedures
Marie�is�enrolled�in�an�independent�study�class��As�part�of�the�course�requirement,�she� has�to�complete�a�research�study��In�collaboration�with�the�statistics�faculty�in�her�pro- gram,�Marie�designs�an�experimental�study�to�determine�if�there�is�a�mean�difference� in�student�attendance�in�the�statistics�lab�based�on�the�attractiveness�of�the�statistics�lab� instructor��Marie’s�research�question�is�as�follows:�Is there a mean difference in the number of statistics labs attended by students based on the attractiveness of the lab instructor?�Marie� determined�that�a�one-way�ANOVA�was�the�best�statistical�procedure�to�use�to�answer� her� question�� Marie� has� collected� the� data� to� analyze� her� research� question� and� has� conducted�a�one-way�ANOVA,�where�she�rejected�the�null�hypothesis��Now,�her�task�is� to�determine�which�groups�(recall�there�were�four�statistics�labs,�each�with�an�instruc- tor� with� a� different� attractiveness� rating)� are� statistically� different� on� the� outcome� (i�e�,�number�of�statistics�labs�attended)�
This�section�describes�the�most�important�characteristics�of�the�MCPs��We�begin�by�defin- ing�a�contrast�and�then�move�into�planned�versus�post�hoc�contrasts,�the�Type�I�error�rates,� and�orthogonal�contrasts�
12.1.1 Contrasts
A�contrast�is�a�weighted�combination�of�the�means��For�example,�one�might�wish�to�form� contrasts�involving�the�following�means:�(a)�group�1�with�group�2�and�(b)�the�combination� (or�average)�of�groups�1�and�2�with�group�3��Statistically�a�contrast�is�defined�as
ψ µ µ µi J Jc c c= + + +1 1 2 2. . .…
where�the�cj�represents�contrast�coefficients�(or�weights),�which�are�positive,�zero,�and�neg- ative�values�used�to�define�a�particular�contrast�ψi,�and�the�μ�j�represents�population�group� means��In�other�words,�a�contrast�is�simply�a�particular�combination�of�the�group�means,� depending� on� which� means� the� researcher� is� interested� in� comparing�� It� should� also� be� noted�that�to�form�a�fair�or�legitimate�contrast,�Σcj�=�0�for�the�equal�n’s�or�balanced�case,� and�Σ(njcj)�=�0�for�the�unequal�n’s�or�unbalanced�case�
For�example,�suppose�we�wish�to�compare�the�means�of�groups�1�and�3�for�J�=�4�groups� or�levels,�and�we�call�this�contrast�1��The�contrast�would�be�written�as
ψ µ µ µ µ
µ µ µ µ
µ
1 1 1 2 2 3 3 4 4
1 2 3 41 0 1 0
= + + +
= + + + − +
=
c c c c. . . .
. . . .( ) ( ) ( ) ( )
.. .1 3− µ
What� hypotheses� are� we� testing� when� we� evaluate� a� contrast?� The� null� and� alternate� hypotheses�of�any�specific�contrast�can�be�written,�respectively,�simply�as
H i0 0: ψ =
and
H i1 0: ψ ≠
344 An Introduction to Statistical Concepts
Thus� we� are� testing� whether� a� particular� combination� of� means,� as� defined� by� the� contrast coefficients,� are� different�� How� does� this� relate� back� to� the� omnibus� F� test?� The null� and� alternate� hypotheses� for� the� omnibus� F� test� can� be� written� in� terms� of� contrasts�as
H i0 0: all ψ =
H i1 0: at least one ψ ≠
Here�the�omnibus�test�is�used�to�determine�whether�any�contrast�that�could�be�formulated� for�the�set�of�J�means�is�significant�or�not�
Contrasts�can�be�divided�into�simple�or�pairwise�contrasts,�and�complex�or�nonpairwise� contrasts��A�simple�or�pairwise�contrast�is�a�comparison�involving�only�two�means��Take�as� an�example�the�situation�where�there�are�J�=�3�groups��There�are�three�possible�distinct�pair- wise�contrasts�that�could�be�formed:�(a)�μ�1�−�μ�2�=�0�(comparing�the�mean�of�group�1�to�the� mean�of�group�2),�(b)�μ�1�−�μ�3�=�0�(comparing�the�mean�of�group�1�to�the�mean�of�group�3),�and� (c)�μ�2�−�μ�3�=�0�(comparing�the�mean�of�group�2�to�the�mean�of�group�3)��It�should�be�obvious� that�a�pairwise�contrast�involving�groups�1�and�2�is�the�same�contrast�whether�it�is�written� as�μ�1�−�μ�2�=�0�or�as�μ�2�−�μ�1�=�0�
In�terms�of�contrast�coefficients,�these�three�contrasts�could�be�written�in�the�form�of�a� table�as�follows:
c1 c2 c3
ψ1:�μ�1�−�μ�2�=�0 +1 −1 0 ψ2:�μ�1�−�μ�3�=�0 +1 0 −1 ψ3:�μ�2�−�μ�3�=�0 0 +1 −1
where� each� contrast� (i�e�,� ψ1,� ψ2,� ψ3)� is� read� across� the� table� (left� to� right)� to� determine� its�contrast�coefficients�(i�e�,�c1, c2, c3)��For�example,�the�first�contrast,�ψ1,�does�not�involve� group�3�because�that�contrast�coefficient�is�0�(see�c3�for�ψ1),�but�does�involve�groups�1�and�2� because�those�contrast�coefficients�are�not�0�(see�c1�and�c2�for�ψ1)��The�contrast�coefficients� are� +1� for� group� 1� (see� c1)� and� −1� for� group� 2� (see� c2);� consequently� we� are� interested� in� examining�the�difference�between�the�means�of�groups�1�and�2�
Written�in�long�form�so�that�we�can�see�where�the�contrast�coefficients�come�from,�the� three�contrasts�are�as�follows:
ψ µ µ µ µ µ1 1 2 3 1 21 1 0= + + − + = −( ) ( ) ( ). . . . .
ψ µ µ µ µ µ2 1 2 3 1 31 0 1= + + + − = −( ) ( ) ( ). . . . .
ψ µ µ µ µ µ3 1 2 3 2 30 1 1= + + + − = −( ) ( ) ( ). . . . .
An�easy�way�to�remember�the�number�of�possible�unique�pairwise�contrasts�that�could�be� written�is�½[J(J�−�1)]��Thus�for�J�=�3,�the�number�of�possible�unique�pairwise�contrasts�is�3,� whereas�for�J�=�4,�the�number�of�such�contrasts�is�6�(or�1/2[4�(4�−�1)]�=�1/2(4)(3)�=�1/2(12)�=�6)�
345Multiple Comparison Procedures
A�complex�contrast�is�a�comparison�involving�more�than�two�means��Continuing�with� the� example� of� J� =� 3� groups,� we� might� be� interested� in� testing� the� contrast� of� μ�1� −� (½)
(μ�2� +� μ�3)� which could also be written as ( )
.1 . .µ
µ µ −
+
2 3
2 �� This� contrast� is� a� compari-
son� of� the� mean� for� group� 1� (i�e�,� μ�1)� with� the� average� of� the� means� for� groups� 2� and� 3�
i . e ., ( )
2 . .µ µ2 3+
��In�terms�of�contrast�coefficients,�this�contrast�would�be�written�as�seen�here:
c1 c2 c3
ψ µ µ µ
4 1 2 3
2 2 : 0.
. .− − = +1 −1/2 −1/2
Written� in� long� form� so� that� we� can� see� where� the� contrast� coefficients� come� from,� this� complex�contrast�is�as�follows:
� ψ µ µ µ µ µ µ µ
µ 4 1 2 3 1 2 3 11 1 2 1 2 1 2 1 2= + + − + − = − − = −( ) ( ) ( ) ( ) ( ). . . . . . ./ / / /
.. .2 3
2 2 0− =
µ
The�number�of�unique�complex�contrasts�is�greater�than�½[J(J�−�1)]�when�J�is�at�least�4��In� other�words,�the�number�of�such�contrasts�that�could�be�formed�can�be�quite�large�when� there�are�more�than�three�groups��It�should�be�noted�that�the�total�number�of�unique�pair- wise�and�complex�contrasts�is�[1�+�½(3J�−�1)�−�2J ]�(Keppel,�1982)��Thus�for�J�=�4,�one�could� form�25�total�contrasts�
Many�of�the�MCPs�are�based�on�the�same�test�statistic,�which�we�introduce�here�as�the� “standard�t�”�The�standard�t�ratio�for�a�contrast�is�given�as�follows:
t s
= ψ
ψ
′ ′
where�sψ′ =�represents�the�standard�error�of�the�contrast�as�follows:
s MS c n
error j
jj
J
ψ ′ =
= ∑
2
1
where�the�prime�(i�e�,�′)�indicates�that�this�is�a�sample�estimate�of�the�population�value�of�the� contrast�(i�e�,�based�on�sample�data),�and�nj�refers�to�the�number�of�observations�in�group�j�
12.1.2 planned Versus post hoc Comparisons
This�section�examines�specific�types�of�contrasts�or�comparisons��One�way�of�classifying� contrasts�is�whether�the�contrasts�are�formulated�prior�to�the�research�or�following�a�sig- nificant� omnibus� F� test�� Planned contrasts� (also� known� as� specific� or� a� priori� contrasts)� involve� particular� comparisons� that� the� researcher� is� interested� in� examining� prior� to�
346 An Introduction to Statistical Concepts
data�collection��These�planned�contrasts�are�generally�based�on�theory,�previous�research,� and/or�specific�hypotheses��Here�the�researcher�is�interested�in�certain�specific�contrasts� a�priori,�where�the�number�of�such�contrasts�is�usually�small��Planned�contrasts�are�done� without�regard�to�the�result�of�the�omnibus�F�test�(i�e�,�whether�or�not�the�overall�F�test�is� statistically�significant)��In�other�words,�the�researcher�is�interested�in�certain�specific�con- trasts,�but�not�in�the�omnibus�F�test�that�examines�all�possible�contrasts��In�this�situation,� the�researcher�could�care�less�about�the�multitude�of�possible�contrasts�and�need�not�even� examine�the�overall�F�test,�but�rather�the�concern�is�only�with�a�few�contrasts�of�substan- tive� interest�� In� addition,� the� researcher� may� not� be� as� concerned� with� the� family-wise� error�rate�for�planned�comparisons�because�only�a�few�of�them�will�actually�be�carried�out�� Fewer�planned�comparisons�are�usually�conducted�(due�to�their�specificity)�than�post�hoc� comparisons�(due�to�their�generality),�so�planned�contrasts�generally�yield�narrower�con- fidence�intervals�(CIs),�are�more�powerful,�and�have�a�higher�likelihood�of�a�Type�I�error� than�post�hoc�comparisons�
Post hoc contrasts�are�formulated�such�that�the�researcher�provides�no�advance�speci- fication�of�the�actual�contrasts�to�be�tested��This�type�of�contrast�is�done�only�following�a� statistically�significant�omnibus�F�test��Post�hoc�is�Latin�for�“after�the�fact,”�referring�to�con- trasts�tested�after�a�statistically�significant�omnibus�F�in�the�ANOVA��Here�the�researcher� may�want�to�take�the�family-wise�error�rate�into�account�somehow�to�achieve�better�overall� Type�I�error�protection��Post�hoc�contrasts�are�also�known�as�unplanned,�a�posteriori,�or� postmortem�contrasts��It�should�be�noted�that�most�MCPs�are�not�derived�or�based�on�find- ing�a�statistically�significant�F�in�the�ANOVA�
12.1.3 Type I error Rate
How�does�the�researcher�deal�with�the�family-wise�Type�I�error�rate?�Depending�on�the� MCP�selected,�one�may�either�set�α�for�each�contrast�or�set�α�for�a�family�of�contrasts��In� the� former� category,� α� is� set� for� each� individual� contrast�� The� MCPs� in� this� category� are� known�as�contrast-based��We�designate�the�α�level�for�contrast-based�procedures�as�αpc ,�as� it�represents�the�per contrast�Type�I�error�rate��Thus�αpc�represents�the�probability�of�mak- ing�a�Type�I�error�for�that�particular�contrast��In�the�latter�category,�α�is�set�for�a�family�or� set�of�contrasts��The�MCPs�in�this�category�are�known�as�family-wise��We�designate�the�α� level�for�family-wise�procedures�as�αfw ,�as�it�represents�the�family-wise�Type�I�error�rate�� Thus�αfw�represents�the�probability�of�making�at�least�one�type�I�error�in�the�family�or�set� of�contrasts�
For�orthogonal�(or�independent�or�unrelated)�contrasts,�the�following�property�holds:
α αfw pc c= − −1 1( )
where� c� =� J� −� 1� orthogonal� contrasts� (as� defined� in� the� next� section)�� For� nonorthogonal� (or�related�or�oblique)�contrasts,�this�property�is�more�complicated,�so�we�simply�say�the� following:
α αfw pcc≤
These� properties� should� be� familiar� from� the� discussion� in� Chapter� 11,� where� we� were� looking�at�the�probability�of�a�Type�I�error�in�the�use�of�multiple�independent�t�tests�
347Multiple Comparison Procedures
12.1.4 Orthogonal Contrasts
Let�us�begin�this�section�by�defining�orthogonal�contrasts��A�set�of�contrasts�is�orthogonal� if�they�represent�nonredundant� and�independent� (if� the�usual� ANOVA� assumptions�are� met)�sources�of�variation��For�J�groups,�you�will�only�be�able�to�construct�J�−�1�orthogonal� contrasts�in�a�set��However,�more�than�one�set�of�orthogonal�contrasts�may�exist��Note�that� although�the�contrasts�within�each�set�are�orthogonal,�contrasts�across�such�sets�may�not� be�orthogonal�
For� purposes� of� simplicity,� we� first� consider� the� equal� n’s� or� balanced� case� (in� other� words,�the�sample�sizes�are�the�same�for�each�group)��With�equal�observations�per�group,� two�contrasts�are�defined�to�be�orthogonal�if�the�products�of�their�contrast�coefficients�sum� to�0��That�is,�two�contrasts�are�orthogonal�if�the�following�holds:
( ) ...’ ’ ’ ’c c c c c c c cj j j
J
J J
= ∑ = + + + =
1
1 1 2 2 0
where�j�and�j′�represent�two�distinct�contrasts��Thus�we�see�that�orthogonality�depends�on� the�contrast�coefficients,�the�cj,�and�not�the�group�means,�the�μ��j�
For�example,�if�J�=�3,�then�we�can�form�a�set�of�two�orthogonal�contrasts��One�such�set� is�as�follows��In�this�set�of�contrasts,�the�first�contrast�(ψ1)�compares�the�mean�of�group�1� (c1�=�+1)�to�the�mean�of�group�2�(c2�=�−1)��The�second�contrast�(ψ2)�compares�the�average�of� the�means�of�group�1�(c1�=�+1/2)�and�group�2�(c2�=�+1/2)�to�the�mean�of�group�3�(c3�=�−1):
c1 c2 c3
ψ1:�μ�1�−�μ�2�=�0 +1 −1 0 ψ2:�(1/2)μ�1�+�(1/2)μ�2�−�μ�3�=�0 +1/2 +1/2 −1
( )’c cj j j
J
= =
∑ 1
+1/2 −1/2 0 =�0
Thus,�plugging�these�values�into�our�equation�produces�the�following:
( ) ( )( ) ( )( ) ( )(’ ’ ’ ’c c c c c c c cj j j
J
= ∑ = + + = + + + − + + −
1
1 1 2 2 3 3 1 1 2 1 1 2 0/ / 11 1 2 1 2 0 0) ( ) ( )= + + − + =/ /
If�the�sum�of�the�contrast�coefficient�products�for�a�set�of�contrasts�is�equal�to�0,�then�we� define�this�as�an�orthogonal�set�of�contrasts�
A�set�of�two�contrasts�that�are�not�orthogonal�is�the�following,�where�we�see�that�the�set� of�contrasts�does�not�sum�to�0:
c1 c2 c3
ψ3:�μ�1�−�μ�2�=�0 +1 −1 0 ψ4:�μ�1�−�μ�3�=�0 +1 0 −1
( )’c cj j j
J
= =
∑ 1
+1 0 0 =�+1
348 An Introduction to Statistical Concepts
Thus,�plugging�these�values�into�our�equation�produces�the�following,�where�we�see�that� the�product�of�the�contrasts�also�does�not�sum�to�0:
( ) ( )( ) ( )( ) ( )( ) (’ ’ ’ ’c c c c c c c cj j j
J
= + + = + + + − + − = + =
∑ 1
1 1 2 2 3 3 1 1 1 0 0 1 11 0 0 1) + + = +
Consider�a�situation�where�there�are�three�groups�and�we�decide�to�form�three�pairwise� contrasts,�knowing�full�well�that�they�cannot�all�be�orthogonal�to�one�another��For�this�set� of�contrasts,�the�first�contrast�(ψ1)�compares�the�mean�of�group�1�(c1�=�+1)�to�the�mean�of� group�2�(c2�=�−1)��The�second�contrast�(ψ2)�compares�the�mean�of�group�2�(c2�=�+1)�to�the� mean�of�group�3�(c3�=�−1),�and�the�third�contrast�compares�the�mean�of�group�1�(c1�=�+1)�to� the�mean�of�group�3�(c3�=�−1)�
c1 c2 c3
ψ1:�μ�1�−�μ�2�=�0 +1 −1 0 ψ2:�μ�2�−�μ�3�=�0 0 +1 −1 ψ3:�μ�1�−�μ�3�=�0 +1 0 −1
Say�that�the�group�population�means�are�μ�1�=�30,�μ�2�=�24,�and�μ�3�=�20��We�find�ψ1�=�6�for�the� first�contrast�(i�e�,�ψ1:�μ�1�−�μ�2�=�30�−�24�=�6)�and�ψ2�=�4�for�the�second�contrast�(i�e�,�ψ2:�μ�2�−�μ�3�=� 24�−�20�=�4)��Because�these�three�contrasts�are�not�orthogonal�and�contain�totally�redundant� information�about�these�means,�ψ3�=�10�for�the�third�contrast�by�definition�(i�e�,�ψ3:�μ�1�−�μ�3�=� 30�−�20�=�10)��Thus�the�third�contrast�contains�no�additional�information�beyond�that�contained� in�the�first�two�contrasts�
Finally,� for� the� unequal� n’s� or� unbalanced� case,� two� contrasts� are� orthogonal� if� the� following�holds:
c c n j j
jj
J ’ 0
=
= ∑
1
The�denominator�nj�makes�it�more�difficult�to�find�an�orthogonal�set�of�contrasts�that�is�of� any�interest�to�the�applied�researcher�(see�Pedhazur,�1997,�for�an�example)�
12.2 Selected Multiple Comparison Procedures
This�section�considers�a�selection�of�MCPs��These�represent�the�“best”�procedures�in�some� sense,�in�terms�of�ease�of�utility,�popularity,�and�control�of�Type�I�and�Type�II�error�rates�� Other�procedures�are�briefly�mentioned��In�the�interest�of�consistency,�each�procedure�is� discussed� in� the� hypothesis� testing� situation� based� on� a� test� statistic�� Most,� but� not� all,� of� these� procedures� can� also� be� formulated� as� CIs� (sometimes� called� a� critical differ- ence),�although�these�will�not�be�discussed�here��The�first�few�procedures�discussed�are� for� planned� comparisons,� whereas� the� remainder� of� the� section� is� devoted� to� post� hoc� comparisons��For�each�MCP,�we�describe�its�major�characteristics�and�then�present�the�test� statistic�with�an�example�using�the�data�from�Chapter�11�
349Multiple Comparison Procedures
Unless� otherwise� specified,� each� MCP� makes� the� standard� assumptions� of� normality,� homogeneity�of�variance,�and�independence�of�observations��Some�of�the�procedures�do� have�additional�restrictions,�such�as�equal�n’s�per�group��Throughout�this�section,�we�also� presume�that�a�two-tailed�alternative�hypothesis�is�of�interest,�although�some�of�the�MCPs� can�also�be�used�with�a�one-tailed�alternative�hypothesis��In�general,�the�MCPs�are�fairly� robust� to� nonnormality� (but� not� for� extreme� cases),� but� are� not� as� robust� to� departures� from�homogeneity�of�variance�or�from�independence�(see�Pavur,�1988)�
12.2.1 planned analysis of Trend
Trend�analysis�is�a�planned�MCP�useful�when�the�groups�represent�different�quantitative� levels�of�a�factor�(i�e�,�an�interval�or�ratio�level�independent�variable)��Examples�of�such�a� factor�might�be�age,�drug�dosage,�and�different�amounts�of�instruction,�practice,�or�trials�� Here�the�researcher�is�interested�in�whether�the�sample�means�vary�with�a�change�in�the� amount�of�the�independent�variable��We�define�trend analysis�in�the�form�of�orthogonal� polynomials� and� assume� that� the� levels� of� the� independent� variable� are� equally� spaced� (i�e�,�same�distances�between�the�levels�of�the�independent�variable,�such�as�100,�200,�300,� and�400cc)�and�that�the�number�of�observations�per�group�is�the�same��This�is�the�standard� case;�other�cases�are�briefly�discussed�at�the�end�of�this�section�
Orthogonal�polynomial�contrasts�use�the�standard�t�test�statistic,�which�is�compared�to� the�critical�values�of�±α/2�tdf(error)�obtained�from�the�t�table�in�Table�A�2��The�form�of�the�con- trasts�is�a�bit�different�and�requires�a�bit�of�discussion��Orthogonal�polynomial�contrasts� incorporate�two�concepts,�orthogonal�contrasts�(recall�these�are�unrelated�or�independent� contrasts)� and� polynomial� regression�� For� J� groups,� there� can� be� only� J� −� 1� orthogonal� contrasts�in�a�set��In�polynomial�regression,�we�have�terms�in�the�model�for�a�linear�trend,� a�quadratic�trend,�a�cubic�trend,�and�so�on��For�example,�linear�trend�is�represented�by�a� straight�line�(no�bends),�quadratic�trend�by�a�curve�with�one�bend�(e�g�,�U�or�upside-down� U�shapes),�and�cubic�trend�by�a�curve�with�two�bends�(e�g�,�S�shape)�
Now�put�those�two�ideas�together��A�set�of�orthogonal�contrasts�can�be�formed�where�the� first�contrast�evaluates�a�linear�trend,�the�second�a�quadratic�trend,�the�third�a�cubic�trend,� and�so�forth��Thus�for�J�groups,�the�highest�order�polynomial�that�can�be�formed�is�J�−�1�� With�four�groups,�for�example,�one�could�form�a�set�of�three�orthogonal�contrasts�to�assess� linear,�quadratic,�and�cubic�trends�
You�may�be�wondering�just�how�these�contrasts�are�formed?�For�J�=�4�groups,�the�contrast� coefficients�for�the�linear,�quadratic,�and�cubic�trends�are�as�follows:
c1 c2 c3 c4
ψlinear −3 −1 +1 +3 ψquadratic +1 −1 −1 +1 ψcubic −1 +3 −3 +1
where�the�contrasts�can�be�written�out�as�follows:
ψ µ µ µ µlinear = − + − + + + +( ) ( ) ( ) ( ). . . .3 1 1 31 2 3 4
� ψ µ µ µ µquadratic = + + − + − + +( ) ( ) ( ) ( ). . . .1 1 1 11 2 3 4
� ψ µ µ µ µcubic = − + + + − + +( ) ( ) ( ) ( ). . . .1 3 3 11 2 3 4
350 An Introduction to Statistical Concepts
These�contrast�coefficients,�for�a�number�of�different�values�of�J,�can�be�found�in�Table� A�6��If�you�look�in�the�table�of�contrast�coefficients�for�values�of�J�greater�than�6,�you�see� that�the�coefficients�for�the�higher-order�polynomials�are�not�included��As�an�example,� for�J�=�7,�coefficients�only�up�through�a�quintic�trend�are�included��Although�they�could� easily�be�derived�and�tested,�these�higher-order�polynomials�are�usually�not�of�inter- est�to�the�researcher��In�fact,�it�is�rare�to�find�anyone�interested�in�polynomials�beyond� the�cubic�because�they�are�difficult�to�understand�and�interpret�(although�statistically� sophisticated,� they� say� little� to� the� applied� researcher� as� the� results� must� be� inter- preted� in� values� that� are� highly� complex)�� The� contrasts� are� typically� tested� sequen- tially� beginning� with� the� linear� trend� and� proceeding� to� higher-order� trends� (cubic� then�quadratic)�
Using�the�example�data�on�the�attractiveness�of�the�lab�instructors�from�Chapter�11,�let� us�test�for�linear,�quadratic,�and�cubic�trends��Trend�analysis�may�be�relevant�for�these�data� because� the� groups� do� represent� different� quantitative� levels� of� an� attractiveness� factor�� Because�J�=�4,�we�can�use�the�contrast�coefficients�given�previously�
The�following�are�the�computations,�based�on�these�mean�values,�to�test�the�trend�analy- sis��The�critical�values�(where�dferror�is�calculated�as�N�−�J,�or�32�−�4�=�28)�are�determined�to� be�as�follows:
± = ± = ±α/ ( ) . .2 025 28 2 048t tdf error
The� standard� error� for� linear trend� is� computed� as� follows� (where� nj� =� 8� for� each� of� the�J�=�4�groups;�MSerror�was�computed�in�the�previous�chapter�and�found�to�be�36�1116)�� Recall� that� the� contrast� equation� for� the� linear� trend� is� ψlinear� =� (−3)μ�1� +� (−1)μ�2� +� (+1)μ�3� +� (+3)μ�4,� and� thus� these� are� the� cj� values� in� the� following� equation� (−3,� −1,� +1,� and� +3,� respectively):
s MS c n
error j
jj
J
ψ ’ . ( ) ( )
=
=
− +
− + +
= ∑
2
1
2 2 2 2
36 1116 3 8
1 8
1 8
3 8
= + + +
=36 1116 9 8
1 8
1 8
9 8
9 5015. .
The� standard� error� for� quadratic trend� is� determined� similarly�� Recall� that� the� contrast� equation�for�the�quadratic�trend�is�ψquadratic�=�(+1)μ�1�+�(−1)μ�2�+�(−1)μ�3�+�(+1)μ�4,�and�thus�these� are�the�cj�values�in�the�following�equation�(+1,�−1,�−1,�and�+1,�respectively):
s MS c n
error j
jj
J
ψ ’ . ( ) ( )
=
= +
− +
− +
= ∑
2
1
2 2 2 2
36 1116 1 8
1 8
1 8
1 8
= + + +
=36 1116 1 8
1 8
1 8
1 8
4 2492. .
351Multiple Comparison Procedures
The�standard�error�for�cubic trend�is�computed�similarly��Recall�that�the�contrast�equation� for�the�cubic�trend�is�ψcubic�=�(−1)μ�1�+�(+3)μ�2�+�(−3)μ�3�+�(+1)μ�4,�and�thus�these�are�the�cj�values� in�the�following�equation�(−1,�+3,�−3,�and�+1,�respectively):
s MS c n
error j
jj
J
ψ ’ . ( ) ( )
=
=
− + +
− +
= ∑
2
1
2 2 2 2
36 1116 1 8
3 8
3 8
1 8
= + + +
=36 1116 1 8
9 8
9 8
1 8
9 5015. .
Recall�the�following�means�for�each�group�(as�presented�in�the�previous�chapter):
Number of Statistics Labs Attended by Group
Group 1: Unattractive
Group 2: Slightly
Unattractive
Group 3: Moderately Attractive
Group 4: Very
Attractive Overall
15 20 10 30 10 13 24 22 12 9 29 26 8 22 12 20
21 24 27 29 7 25 21 28
13 18 25 25 3 12 14 15
Means 11�1250 17�8750 20�2500 24�3750 18�4063 Variances 30�1250 35�2679 53�0714 25�9821 56�4425
Thus,�using�the�contrast�coefficients�(represented�by�the�constant�c�values�in�the�numerator� of�each�term)�and�the�values�of�the�means�for�each�of�the�four�groups�(represented�by� Y –
�1,�Y –
�2,�Y –
�3,�Y –
�4),�the�test�statistics�are�computed�as�follows:
t Y Y Y Y
s linear =
− − + + =
− − +3 1 1 3 3 11 1250 1 17 8750 1 21 2 3 4. . . . ’
( . ) ( . ) (
ψ
00 2500 3 24 3750 9 5015
4 4335 . ) ( . )
. .
+ =
t Y Y Y Y
s quadratic =
− − + =
− −1 1 1 1 1 11 1250 1 17 8750 11 2 3 4. . . . ’
( . ) ( . ) (
ψ
220 2500 1 24 3750 4 2492
0 6178 . ) ( . )
. .
+ = −
t Y Y Y Y
s cubic =
− + − + =
− + −1 3 3 1 1 11 1250 3 17 8750 3 201 2 3 4. . . . ’
( . ) ( . ) (
ψ
.. ) ( . ) .
. 2500 1 24 3750
9 5015 0 6446
+ =
The�t�test�statistic�for�the�linear�trend�exceeds�the�t�critical�value��Thus�we�see�that�there� is� a� statistically� significant� linear trend� in� the� means� but� no� significant� higher-order� trend� (in�other�words,�no�significant�quadratic�or�cubic�trend)��This�should�not�be�surprising�as� shown�in�the�profile�plot�of�the�means�of�Figure�12�1,�where�there�is�a�very�strong�linear�
352 An Introduction to Statistical Concepts
trend,�and�that�is�about�it��In�other�words,�there�is�a�steady�increase�in�mean�attendance�as� the�level�of�attractiveness�of�the�instructor�increases��Always�plot�the�means�so�that�you� can�interpret�the�results�of�the�contrasts�
Let�us�make�some�final�points�about�orthogonal�polynomial�contrasts��First,�be�particu- larly� careful� about� extrapolating� beyond� the� range� of� the� levels� investigated�� The� trend� may�or�may�not�be�the�same�outside�of�this�range;�that�is,�given�only�those�sample�means,� we�have�no�way�of�knowing�what�the�trend�is�outside�of�the�range�of�levels�investigated�� Second,� in� the� unequal� n’s� or� unbalanced� case,� it� becomes� difficult� to� formulate� a� set� of� orthogonal�contrasts�that�make�any�sense�to�the�researcher��See�the�discussion�in�the�next� section�on�planned�orthogonal�contrasts,�as�well�as�Kirk�(1982)��Third,�when�the�levels�are� not� equally� spaced,� this� needs� to� be� taken� into� account� in� the� contrast� coefficients� (see� Kirk,�1982)�
12.2.2 planned Orthogonal Contrasts
Planned�orthogonal�contrasts�(POC)�are�an�MCP�where�the�contrasts�are�defined�ahead�of� time�by�the�researcher�(i�e�,�planned)�and�the�set�of�contrasts�are�orthogonal�(or�unrelated)�� The� POC� method� is� a� contrast-based� procedure� where� the� researcher� is� not� concerned� with�control�of�the�family-wise�Type�I�error�rate�across�the�set�of�contrasts��The�set�of�con- trasts�are�orthogonal,� so�the�number� of�contrasts� should�be� small,� and�concern� with� the� family-wise�error�rate�is�lessened�
Computationally,� planned� orthogonal� contrasts� use� the� standard� t� test� statistic� that� is� compared�to�the�critical�values�of�±α/2tdf(error)�obtained�from�the�t�table�in�Table�A�2��Using� the�example�dataset�from�Chapter�11,�let�us�find�a�set�of�orthogonal�contrasts�and�complete� the�computations��Since�J�=�4,�we�can�find�at�most�a�set�of�three�(or�J�−�1)�orthogonal�contrasts�� One�orthogonal�set�that�seems�reasonable�for�these�data�is�as�follows:
c1 c2 c3 c4
ψ µ µ µ µ
1 1 2 3 4
2 2 0: . . . .
+
− +
= +1/2 +1/2 −1/2 −1/2
ψ2:�μ�1�−�μ�2�=�0 +1 −1 0 0 ψ3:�μ�3�−�μ�4�=�0 0 0 +1 −1
FIGuRe 12.1 Profile�plot�for�statistics�lab�example�
25
22
20
18
N um
be r o
f la
bs a
tt en
de d
15
12
10
1 2 3 Group
4
353Multiple Comparison Procedures
Here�we�see�that�the�first�contrast�compares�the�average�of�the�two�least�attractive�groups� (i�e�,�unattractive�and�slightly�attractive)�with�the�average�of�the�two�most�attractive�groups� (i�e�,� moderately� attractive� and� very� attractive),� the� second� contrast� compares� the� means� of� the� two� least� attractive� groups� (i�e�,� unattractive�and� slightly� attractive),� and� the� third� contrast�compares�the�means�of�the�two�most�attractive�groups�(moderately�attractive�and� very�attractive)��Note�that�the�design�is�balanced�(i�e�,�the�equal�n’s�case�as�all�groups�had� a�sample�size�of�8)��What�follows�are�the�computations��The�critical�values�are�as�follows:
± = ± = ±α/ ( ) . 2.0482 025 28t tdf error
The�standard�error�for�contrast�1�is�computed�as�follows�(where�nj�=�8�for�each�of�the�J�=�4� groups;�MSerror�was�computed�in�the�previous�chapter�and�found�to�be�36�1116)��The�equa-
tion�for�contrast�1�is� ψ µ µ µ µ
1 1 2 3 4
2 2 0: . . . .
+
− +
= ,�and�thus�these�are�the�cj�values�in�the �
following�equation�(+1/2,�+1/2,�−1/2,�−1/2,�respectively,�and�these�values�are�then�squared,� which�results�in�the�value�of��25):
s MS c n
error j
jj
J
ψ ’ . . . . .
=
= + + +
= ∑
2
1
36 1116 25 8
25 8
25 8
25 8
= 2 1246.
Similarly,�the�standard�errors�for�contrasts�2�and�3�are�computed�as�follows:
s MS c n j
jj
J
ψ ’ . .=
= +
= =
∑error 2
1
36 1116 1 8
1 8
3 0046
The�test�statistics�are�computed�as�follows:
t Y Y Y Y
s 1
1 2 1
1 2 2
1 2 3
1 2 4
1 2
1 2
1 211 1250 17 8750
= + + − −
= + + −
. . . .
’
( . ) ( . )
ψ
(( . ) ( . ) .
. 20 2500 24 3750
2 1246 3 6772
1 2− = −
� t
Y Y s
2 1 2 11 1250 17 8750
3 0046 2 2466=
− =
− = −. .
’
. . .
. ψ
� t
Y Y s
3 3 4 20 2500 24 3750
3 0046 1 3729=
− =
− = −. .
’
. . .
. ψ
The�result�for�contrast�1�is�that�the�combined�less�attractive�groups�have�statistically�sig- nificantly� lower� attendance,� on� average,� than� the� combined� more� attractive� groups�� The� result�for�contrast�2�is�that�the�two�less�attractive�groups�are�statistically�significantly�dif- ferent�from�one�another,�on�average��The�result�for�contrast�3�is�that�the�means�of�the�two� more�attractive�groups�are�not�statistically�significantly�different�from�one�another�
354 An Introduction to Statistical Concepts
There�is�a�practical�problem�with�this�procedure�because�(a)�the�contrasts�that�are�of� interest�to�the�researcher�may�not�necessarily�be�orthogonal,�or�(b)�the�researcher�may� not�be�interested�in�all�of�the�contrasts�of�a�particular�orthogonal�set��Another�problem� already�mentioned�occurs�when�the�design�is�unbalanced,�where�an�orthogonal�set�of� contrasts� may� be� constructed� at� the� expense� of� meaningful� contrasts�� Our� advice� is� simple:
� 1�� If�the�contrasts�you�are�interested�in�are�not�orthogonal,�then�use�another�MCP� � 2�� If� you� are� not� interested� in� all� of� the� contrasts� of� an� orthogonal� set,� then� use�
another�MCP� � 3�� If�your�design�is�not�balanced�and�the�orthogonal�contrasts�formed�are�not�mean-
ingful,�then�use�another�MCP�
In�each�case,�you�need�a�different�planned�MCP��We�recommend�using�one�of�the�following� procedures� discussed� later� in� this� chapter:� the� Dunnett,� Dunn� (Bonferroni),� or� Dunn– Sidak�procedure�
We�defined�the�POC�as�a�contrast-based�procedure��One�could�also�consider�an�alter- native� family-wise� method� where� the� αpc� level� is� divided� among� the� contrasts� in� the� set�� This� procedure� is� defined� by� αpc� =� αfw/c,� where� c� is� the� number� of� orthogonal� con- trasts�in�the�set�(i�e�,�c�=�J�−�1)��As�we�show�later,�this�borrows�a�concept�from�the�Dunn� (Bonferroni)�procedure��If�the�variances�are�not�equal�across�the�groups,�several�approxi- mate�solutions�have�been�proposed�that�take�the�individual�group�variances�into�account� (see�Kirk,�1982)�
12.2.3 planned Contrasts with Reference Group: dunnett Method
A�third�method�of�planned�comparisons�is�attributed�to�Dunnett�(1955)��It�is�designed�to� test�pairwise�contrasts�where�a�reference�group�(e�g�,�a�control�or�baseline�group)�is�com- pared�to�each�of�the�other�J�−�1�groups��Thus�a�family�of�prespecified�pairwise�contrasts�is� to�be�evaluated��The�Dunnett�method�is�a�family-wise�MCP�and�is�slightly�more�power- ful�than�the�Dunn�procedure�(another�planned�family-wise�MCP)��The�test�statistic�is�the� standard�t�except�that�the�standard�error�is�simplified�as�follows:
s MS n n
error c j
ψ ’ = +
1 1
where�c�is�the�reference�group�and�j�is�the�group�to�which�it�is�being�compared��The�test� statistic� is� compared� to� the� critical� values� ±α/2tdf(error),J−1� obtained� from� the� Dunnett� table� located�in�Table�A�7�
Using�the�example�dataset,�compare�group�1,�the�unattractive�group�(used�as�a�reference� or�baseline�group),�to�each�of�the�other�three�groups��The�contrasts�are�as�follows:
c1 c2 c3 c4
ψ1:�μ�1�−�μ�2�=�0 +1 −1 0 0 ψ2:�μ�1�−�μ�3�=�0 +1 0 −1 0 ψ3:�μ�1�−�μ�4�=�0 +1 0 0 −1
355Multiple Comparison Procedures
The� following� are� the� computations�� The� critical� values� are� as� follows:� ±α/2tdf(error),J−1� =� ±�025t28,3�≈�±2�48
The�standard�error�is�computed�as�follows�(where�nc�=�8�for�the�reference�group;�nj�=�8� for�each�of�the�other�groups;�MSerror�was�computed�in�the�previous�chapter�and�found�to� be�36�1116):
s MS n n
error c j
ψ ’ . .= +
= +
= 1 1
36 1116 1 8
1 8
3 0046
The�test�statistics�for�the�three�contrasts�(i�e�,�group�1�to�group�2,�group�1�to�group�3,�and� group�1�to�group�4)�are�computed�as�follows:
Unnattractive to slightly attractive : .. .
’ t
Y Y s
1 1 2 11 1250=
− =
ψ
−− = −
17 8750 3 0046
2 2466 .
. .
Unnattractive to moderately attractive: .. .
’ t
Y Y s
2 1 3 11 12=
− =
ψ
550 20 2500 3 0046
3 0370 −
= − .
. .
Unnattractive to very attractive : . .. .
’ t
Y Y s
3 1 4 11 1250 24=
− =
− ψ
33750 3 0046
4 4099 .
.= −
Comparing�the�test�statistics�to�the�critical�values,�we�see�that�the�second�group�(i�e�,�slightly� attractive)�is�not�statistically�significantly�different�from�the�baseline�group�(i�e�,�unattract- ive),�but�the�third�(moderately�attractive)�and�fourth�(very�attractive)�more�attractive�groups� are�significantly�different�from�the�baseline�group�
If�the�variance�of�the�reference�group�is�different�from�the�variances�of�the�other�J�−�1� groups,� then� a� modification� of� this� method� is� described� in� Dunnett� (1964)�� For� related� procedures�that�are�less�sensitive�to�unequal�group�variances,�see�Wilcox�(1987)�or�Wilcox� (1996)�(e�g�,�variation�of�the�Dunnett�T3�procedure)�
12.2.4 Other planned Contrasts: dunn (or bonferroni) and dunn–Sidak Methods
The�Dunn�(1961)�procedure�(commonly�attributed�to�Dunn�as�the�developer�is�unknown),� also�often�called�the�Bonferroni�procedure�(because�it�is�based�on�the�Bonferroni�inequal- ity),�is�a�planned�family-wise�MCP��It�is�designed�to�test�either�pairwise�or�complex�con- trasts�for�balanced�or�unbalanced�designs��Thus�this�MCP�is�very�flexible�and�may�be�used� to�test�any�planned�contrast�of�interest��The�Dunn�method�uses�the�standard�t�test�statistic� with�one�important�exception��The�α�level�is�split�up�among�the�set�of�planned�contrasts�� Typically� the� per� contrast� α� level� (denoted� as� αpc)� is� set� at� α/c,� where� c� is� the� number� of� contrasts��That�is,�αpc�=�αfw/c��According�to�this�rationale,�the�family-wise�Type�I�error�rate� (denoted�as�αfw)�will�be�maintained�at�α��For�example,�if�αfw�=��05�is�desired�and�there�are� five�contrasts�to�be�tested,�then�each�contrast�would�be�tested�at�the��01�level�of�significance� (�05/5� =� �01)�� We� are� reminded� that� α� need� not� be� distributed� equally� among� the� set� of� contrasts,�as�long�as�the�sum�of�the�individual�αpc�terms�is�equal�to�αfw�(Keppel�&�Wickens,� 2004;�Rosenthal�&�Rosnow,�1985)�
356 An Introduction to Statistical Concepts
Computationally,�the�Dunn�method�uses�the�standard�t�test�statistic,�which�is�compared� to�the�critical�values�of�±α/ctdf(error)�for�a�two-tailed�test�obtained�from�the�table�in�Table�A�8�� The�table�takes�the�number�of�contrasts�into�account�without�requiring�you�to�physically� split�up�the�α��Using�the�example�dataset�from�Chapter�11,�for�comparison�purposes,�let�us� test�the�same�set�of�three�orthogonal�contrasts�we�evaluated�with�the�POC�method��These� contrasts�are�as�follows:
c1 c2 c3 c4
ψ µ µ µ µ
1 1 2 3 4
2 2 0:
+
− +
=. . . . +1/2 +1/2 −1/2 −1/2
ψ2:�μ�1�−�μ�2�=�0 +1 −1 0 0 ψ3:�μ�3�−�μ�4�=�0 0 0 +1 −1
Following�are�the�computations,�with�the�critical�values
± = ± ≈ ±α/ ( ) . / .c df errort t05 3 28 2 539
The�standard�error�for�contrast�1�is�computed�as�follows:
s MS c n
error j
jj
J
ψ ’ . . . . .
=
= + + +
= ∑
2
1
36 1116 25 8
25 8
25 8
25 8
= 2 1246.
Similarly,�the�standard�error�for�contrasts�2�and�3�is�computed�as�follows:
s MS c n
error j
jj
J
ψ ’ . .=
= +
= =
∑ 2
1
36 1116 1 8
1 8
3 0046
The�test�statistics�are�computed�as�follows:
t Y Y Y Y
s 1
1 2 1
1 2 2
1 2 3
1 2 4
1 2
1 2
1 211 1250 17 8750
= + + − −
= + + −
. . . .
’
( . ) ( . )
ψ
(( . ) ( . ) .
. 20 2500 24 3750
2 1246 3 6772
1 2− = −
� t
Y Y s
2 1 2 11 1250 17 8750
3 0046 2 2466=
− =
− = −. .
’
. . .
. ψ
� t
Y Y s
3 3 4 20 2500 24 3750
3 0046 1 3729=
− =
− = −. .
’
. . .
. ψ
Notice�that�the�test�statistic�values�have�not�changed�from�the�POC,�but�the�critical�value� has�changed��For�this�set�of�contrasts�then,�we�see�the�same�results�as�were�obtained�via� the� POC� procedure� with� the� exception� of� contrast� 2,� which� is� now� nonsignificant� (i�e�,� only�
357Multiple Comparison Procedures
contrast� 1� is� significant)�� The� reason� for� this� difference� lies� in� the� critical� values� used,� which�were�±2�048�for�the�POC�method�and�±2�539�for�the�Dunn�method��Here�we�see�the� conservative�nature�of�the�Dunn�procedure�because�the�critical�value�is�larger�than�with� the�POC�method,�thus�making�it�a�bit�more�difficult�to�reject�H0�
The� Dunn� procedure� is� slightly� conservative� (i�e�,� not� as� powerful)� in� that� the� true� αfw� may�be�less�than�the�specified�nominal�α�level��For�example,�if�the�nominal�alpha�(speci- fied�by�the�researcher)�is��05,�then�the�true�alpha�may�be�less�than��05��Thus�when�using�the� Dunn,�you�may�be�less�likely�to�reject�the�null�hypothesis�(i�e�,�less�likely�to�find�a�statisti- cally�significant�contrast)��A�less�conservative�(i�e�,�more�powerful)�modification�is�known� as� the� Dunn–Sidak� procedure� (Dunn,� 1974;� Sidak,� 1967)� and� uses� slightly� different� criti- cal�values��For�more�information,�see�Kirk�(1982),�Wilcox�(1987),�and�Keppel�and�Wickens� (2004)��The�Bonferroni�modification�can�also�be�applied�to�other�MCPs�
12.2.5 Complex post hoc Contrasts: Scheffé and kaiser–bowden Methods
Another� early� MCP� due� to� Scheffé� (1953)� is� quite� versatile�� The� Scheffé� procedure� can� be� used� for� any� possible� type� of� comparison,� orthogonal� or� nonorthogonal,� pairwise� or� complex,�planned�or�post�hoc,�where�the�family-wise�error�rate�is�controlled��The�Scheffé� method�is�so�general�that�the�tests�are�quite�conservative�(i�e�,�less�powerful),�particularly� for�the�pairwise�contrasts��This�is�so�because�the�family�of�contrasts�for�the�Scheffé�method� consists�of�all�possible�linear�comparisons��To�control�the�Type�I�error�rate�for�such�a�large� family,� the� procedure� has� to� be� conservative� (i�e�,� making� it� less� likely� to� reject� the� null� hypothesis�if�it�is�really�true)��Thus�we�recommend�the�Scheffé�method�only�for�complex� post�hoc�comparisons�
The�Scheffé�procedure�is�the�only�MCP�that�is�necessarily�consistent�with�the�results�of�the� F�ratio�in�ANOVA��If�the�F�ratio�is�statistically�significant,�then�this�means�that�at�least�one� contrast�in�the�entire�family�of�contrasts�will�be�significant�with�the�Scheffé�method��Do�not� forget,�however,�that�this�family�can�be�quite�large�and�you�may�not�even�be�interested�in� the�contrast(s)�that�wind�up�being�significant��If�the�F�ratio�is�not�statistically�significant,�then� none�of�the�contrasts�in�the�family�will�be�significant�with�the�Scheffé�method�
The�test�statistic�for�the�Scheffé�method�is�the�standard�t�again��This�is�compared�to�the� critical�value� ( )( ), ( )J FJ df error− −1 1α �taken�from�the�F�table�in�Table�A�4��In�other�words,�the� square�root�of�the�F�critical�value�is�adjusted�by�J�−�1,�which�serves�to�increase�the�Scheffé� critical�value�and�make�the�procedure�a�more�conservative�one�
Consider�a�few�example�contrasts�with�the�Scheffé�method��Using�the�example�dataset� from�Chapter�11,�for�comparison�purposes,�we�test�the�same�set�of�three�orthogonal�con- trasts�that�were�evaluated�with�the�POC�method��These�contrasts�are�again�as�follows:
c1 c2 c3 c4
ψ µ µ µ µ
1 1 2 3 4
2 2 : 0. . . .
+
− +
= +1/2 +1/2 −1/2 −1/2
ψ2:�μ�1�−�μ�2�=�0 +1 −1 0 0 ψ3:�μ�3�−�μ�4�=�0 0 0 +1 −1
The�following�are�the�computations��The�critical�value�is�as�follows:
( ) ( ) ( )( ) ( )( . ) ., ( ) . ,J F FJ df error− = = =−1 3 3 2 95 2 971 05 3 28α
358 An Introduction to Statistical Concepts
Standard�error�for�contrast�1:
s MS c n
error j
jj
J
ψ ’ . (. / . / . / . /=
= + + +
= ∑
2
1
36 1116 25 8 25 8 25 8 25 88 2 1246) .=
Standard�error�for�contrasts�2�and�3:
s MS n n
error j j
ψ ’ ’
. .= +
= +
= 1 1
36 1116 1 8
1 8
3 0046
The�test�statistics�are�computed�as�follows:
t Y Y Y Y
s 1
1 2 1
1 2 2
1 2 3
1 2 4
1 2
1 2
1 211 1250 17 8750
= + + − −
= + + −
. . . .
’
( . ) ( . )
ψ
(( . ) ( . ) .
. 20 2500 24 3750
2 1246 3 6772
1 2− = −
� t
Y Y s
2 1 2 11 1250 17 8750
3 0046 2 2466=
− =
− = −. .
’
. . .
. ψ
� t
Y Y s
3 3 4 20 2500 24 3750
3 0046 1 3729=
− =
− = −. .
’
. . .
. ψ
Using�the�Scheffé�method,�these�results�are�precisely�the�same�as�those�obtained�via�the� Dunn�procedure��There�is�somewhat�of�a�difference�in�the�critical�values,�which�were�2�97� for�the�Scheffé�method,�2�539�for�the�Dunn�method,�and�2�048�for�the�POC�method��Here� we� see� that� the� Scheffé� procedure� is� even� more� conservative� than� the� Dunn� procedure,� thus�making�it�a�bit�more�difficult�to�reject�H0�
For�situations�where�the�group�variances�are�unequal,�a�modification�of�the�Scheffé�method� less�sensitive�to�unequal�variances�has�been�proposed�by�Brown�and�Forsythe�(1974)��Kaiser� and�Bowden�(1983)�found�that�the�Brown-Forsythe�procedure�may�cause�the�actual�α�level�to� exceed�the�nominal�α�level,�and�thus�we�recommend�the�Kaiser–Bowden�modification��For� more�information,�see�Kirk�(1982),�Wilcox�(1987),�and�Wilcox�(1996)�
12.2.6 Simple post hoc Contrasts: Tukey hSd, Tukey– kramer, Fisher lSd, and hayter Tests
Tukey’s� (1953)� honestly� significant� difference� (HSD)� test� is� one� of� the� most� popular� post� hoc�MCPs��The�HSD�test�is�a�family-wise�procedure�and�is�most�appropriate�for�consider- ing�all�pairwise�contrasts�with�equal�n’s�per�group�(i�e�,�a�balanced�design)��The�HSD�test� is� sometimes� referred� to� as� the� studentized range test� because� it� is� based� on� the� sam- pling�distribution�of�the�studentized�range�statistic�developed�by�William�Sealy�Gossett� (forced�to�use�the�pseudonym�“Student”�by�his�employer,�the�Guinness�brewery)��For�the�
359Multiple Comparison Procedures
traditional�approach,�the�first�step�in�the�analysis�is�to�rank�order�the�means�from�largest� (Y –
�1)�to�smallest�(Y –
�J)��The�test�statistic,�or�studentized�range�statistic,�is�computed�as�follows:
q Y Y s
i j j= −. . ’
’ψ
where
s MS n error
ψ ’ =
where� i�identifies�the�specific�contrast j�and�j′�designate�the�two�group�means�to�be�compared n�represents�the�number�of�observations�per�group�(equal�n’s�per�group�is�required)
The�test�statistic�is�compared�to�the�critical�value�±α�qdf(error),J,�where�dferror�is�equal�to�J(n�−�1)��The� table�for�these�critical�values�is�given�in�Table�A�9�
The� first� contrast� involves� a� test� of� the� largest� pairwise� difference� in� the� set� of� J� means� (q1)�(i�e�,�largest�vs��smallest�means)��If�these�means�are�not�significantly�different,�then�the� analysis�stops�because�no�other�pairwise�difference�could�be�significant��If�these�means�are� different,�then�we�proceed�to�test�the�second�pairwise�difference�involving�the�largest�mean� (i�e�,�q2)��Contrasts�involving�the�largest�mean�are�continued�until�a�nonsignificant�difference� is�found��Then�the�analysis�picks�up�with�the�second�largest�mean�and�compares�it�with�the� smallest�mean��Contrasts�involving�the�second�largest�mean�are�continued�until�a�nonsignif- icant�difference�is�detected��The�analysis�continues�with�the�next�largest�mean�and�the�small- est�mean,�and�so�on,�until�it�is�obvious�that�no�other�pairwise�contrast�could�be�significant�
Finally,�consider�an�example�using�the�HSD�procedure�with�the�attractiveness�data��The� following�are�the�computations��The�critical�values�are�as�follows:
± = ± ≈ ±α q qdf error J( ), . , .05 28 4 3 87
The�standard�error�is�computed�as�follows�where�n�represents�the�sample�size�per�group:
s MS n error
ψ ’ .
.= = = 36 1116
8 2 1246
The�test�statistics�are�computed�as�follows:
Very�attractive�to�unattractive:�q Y Y s
1 4 1 24 3750 11 1250
2 1246 6 2365=
− =
− =. .
’
. . .
. ψ
Very�attractive�to�slightly�attractive:�q Y Y s
2 4 2 24 3750 17 8750
2 1246 3 0594=
− =
− =. .
’
. . .
. ψ
Moderately�attractive�to�unattractive:�q Y Y s
3 3 1 20 2500 11 1250
2 1246 4 2949=
− =
− =. .
’
. . .
. ψ
Moderately�attractive�to�slightly�attractive:�q Y Y s
4 3 2 20 2500 17 8750
2 1246 1 1179=
− =
− =. .
’
. . .
. ψ
360 An Introduction to Statistical Concepts
Slightly�attractive�to�unattractive:�q Y Y s
5 2 1 17 8750 11 1250
2 1246 3 1771=
− =
− =. .
’
. . .
. ψ
Comparing�the�test�statistic�values�to�the�critical�value,�these�results�indicate�that�the�group� means�are�significantly�different�for�groups�1�(unattractive)�and�4�(very�attractive)�and�for� groups� 1� (unattractive)� and� 3� (moderately� attractive)�� Just� for� completeness,� we� examine� the�final�possible�pairwise�contrast�involving�groups�3�and�4��However,�we�already�know� from� the� results� of� previous� contrasts� that� these� means� cannot� possibly� be� significantly� different��The�test�statistic�result�for�this�contrast�is�as�follows:
Very�attractive�to�moderately�attractive:�q Y Y s
6 4 3 24 3750 20 2500
2 1246 1 9415=
− =
− =. .
’
. . .
. ψ
Occasionally� researchers� need� to� summarize� the� results� of� their� pairwise� comparisons�� Table� 12�1� shows� the� results� of� Tukey� HSD� contrasts� for� the� example� data�� For� ease� of� interpretation,�the�means�are�ordered�from�lowest�to�highest��The�first�row�consists�of�the� results�for�those�contrasts�that�involve�group�1��Thus�the�mean�for�group�1�(unattractive)�is� statistically�different�from�those�of�groups�3�(moderately�attractive)�and�4�(very�attractive)� only��None�of�the�other�pairwise�contrasts�were�shown�to�be�significant��Such�a�table�could� also�be�developed�for�other�pairwise�MCPs�
The�HSD�test�has�exact�control�of�the�family-wise�error�rate�assuming�normality,�homo- geneity,� and� equal� n’s� (better� than� Dunn� or� Dunn–Sidak)�� The� HSD� procedure� is� more� powerful�than�the�Dunn�or�Scheffé�procedure�for�testing�all�possible�pairwise�contrasts,� although� Dunn� is� more� powerful� for� less� than� all� possible� pairwise� contrasts�� The� HSD� technique�is�the�recommended�MCP�as�a�pairwise�method�in�the�equal�n’s�situation��The� HSD� test� is� reasonably� robust� to� nonnormality,� but� not� in� extreme� cases,� and� is� not� as� robust�as�the�Scheffé�MCP�
There� are� several� alternatives� to� the� HSD� for� the� unequal� n’s� case�� These� include� the� Tukey–Kramer� modification� (Kramer,� 1956;� Tukey,� 1953),� which� assumes� normality� and� homogeneity��The�Tukey–Kramer�test�statistic�is�the�same�as�the�Tukey�HSD�except�that�the� standard�error�is�computed�as�follows�(note that when requesting Tukey in SPSS, the program knows which standard error to calculate):
s MS n n
errorψ ’ = +
1 2
1 1
1 2
The�critical�value�is�determined�in�the�same�way�as�with�the�Tukey�HSD�procedure�
Table 12.1
Tukey�HSD�Contrast�Test�Statistics�and�Results
Group 1: Unattractive
Group 2: Slightly
Unattractive
Group 3: Moderately Attractive
Group 4: Very
Attractive
Group�1�(mean�=�11�1250) — 3�1771 4�2949* 6�2365* Group�2�(mean�=�17�8750) — 1�1179 3�0594 Group�3�(mean�=�20�2500) — 1�9415 Group�4�(mean�=�24�3750) —
*p�<��05;��05q28,4�=�3�87�
361Multiple Comparison Procedures
Fisher’s�(1949)�least�significant�difference�(LSD)�test,�also�known�as�the�protected�t�test,� was�the�first�MCP�developed�and�is�a�pairwise�post�hoc�procedure��It�is�a�sequential�pro- cedure�where�a�significant�ANOVA�F�is�followed�by�the�LSD�test�in�which�all�(or�perhaps� some)� pairwise� t� tests� are� examined�� The� standard� t� test� statistic� is� compared� with� the� critical�values�of�±α/2tdf(error)��The�LSD�test�has�precise�control�of�the�family-wise�error�rate� for� the� three-group� situation,� assuming� normality� and� homogeneity;� but� for� more� than� three�groups,�the�protection�deteriorates�rather�rapidly��In�that�case,�a�modification�due�to� Hayter�(1986)�is�suggested�for�more�adequate�protection��The�Hayter�test�appears�to�have� more� power� than� the� Tukey� HSD� and� excellent� control� of� family-wise� error� (Keppel� &� Wickens,�2004)�
12.2.7 Simple post hoc Contrasts for unequal Variances: Games–howell, dunnett T3 and C Tests
When�the�group�variances�are�unequal,�several�alternative�procedures�are�available��These� alternatives� include� the� Games� and� Howell� (1976),� and� Dunnett� T3� and� C� (1980)� proce- dures��According�to�Wilcox�(1996,�2003),�T3�is�recommended�for�n�<�50�and�Games–Howell� for�n�>�50,�and�C�performs�about�the�same�as�Games-Howell��For�further�details�on�these� methods,� see� Kirk� (1982),� Wilcox� (1987,� 1996,� 2003),� Hochberg� (1988),� and� Benjamini� and� Hochberg�(1995)�
12.2.8 Follow-up Tests to kruskal–Wallis
Recall�from�Chapter�11�the�nonparametric�equivalent�to�ANOVA,�the�Kruskal–Wallis�test�� Several� post� hoc� procedures� are� available� to� follow� up� a� statistically� significant� overall� Kruskal–Wallis�test��The�procedures�discussed�here�are�the�nonparametric�equivalents�to� the�Scheffé�and�Tukey�HSD�methods��One�may�form�pairwise�or�complex�contrasts�as�in� the�parametric�case��The�test�statistic�is�Z�and�computed�as�follows:
Z s
i= ψ
ψ
’
’
where�the�standard�error�in�the�denominator�is�computed�as
s N N c
n j
jj
J
ψ ’ ( )
= +
= ∑112
2
1
and�where�N�is�the�total�number�of�observations��For�the�Scheffé�method,�the�test�statistic� Z�is�compared�to�the�critical�value� α χ J −1 �obtained�from�the�χ2�table�in�Table�A�3��For�the� Tukey� HSD� procedure,� the� test� statistic� Z� is� compared� to� the� critical� value� α qdf error J( ), / 2 � obtained�from�the�table�of�critical�values�for�the�studentized�range�statistic�in�Table�A�9�
Let�us�use�the�attractiveness�data�to�illustrate��Do�not�forget�that�we�use�the�ranked�data� as�described�in�Chapter�11��The�rank�means�for�the�groups�are�as�follows:�group�1�(unat- tractive)�=�7�7500,�group�2�(slightly�attractive)�=�15�2500,�group�3�(moderately�attractive)�=� 18�7500,�and�group�4�(very�attractive)�=�24�2500��Here�we�only�examine�two�contrasts�and� then�compare�the�results�for�both�the�Scheffé�and�Tukey�HSD�methods��The�first�contrast�
362 An Introduction to Statistical Concepts
compares�the�two�low-attractiveness�groups�(i�e�,�groups�1�and�2),�whereas�the�second�con- trast�compares�the�two�low-attractiveness�groups�with�the�two�high-attractiveness�groups� (i�e�,�groups�3�and�4)��In�other�words,�we�examine�a�pairwise�contrast�and�a�complex�con- trast,�respectively��The�results�are�given�here��The�critical�values�are�as�follows:
Scheffé α χ χJ − = = =1 05 3 7 8147 2 7955. . .
Tukey α q qdf error J( ), . ,/ / . / .2 2 3 87 2 2 736505 28 4= ≈ ≈
The�standard�error�for�contrast�1�is�computed�as
s N N c
n j
jj
J
ψ ’ ( ) ( )
.= +
=
+
= =
∑112 32 33
12 1 8
1 8
4 6 2
1
9904
The�standard�error�for�contrast�2�is�calculated�as�follows:
s N N c
n j
jj
J
ψ ’ ( ) ( ) . . .
= +
=
+ + + =
∑112 32 33
12 25 8
25 8
25 8
2
1
.. .
25 8
3 3166
=
The�test�statistics�are�computed�as�follows:
� Z
Y Y s
1 1 2 7 75 15 25
4 6904 1 5990=
− =
− = −. .
’
. . .
. ψ
Z Y Y Y Y
s 2
1 2 1
1 2 2
1 2 3
1 2 4
1 2
1 2
1 27 75 15 25 18 7=
+ + − − =
+ + −. . . . ’
( . ) ( . ) ( .
ψ
55 24 25 3 3166
3 0151 1
2) ( . ) .
. −
= −
For�both�procedures,�we�find�a�statistically�significant�difference�with�the�second�contrast� but�not�with�the�first��These�results�agree�with�most�of�the�other�parametric�procedures�for� these� particular� contrasts�� That� is,� the� less� attractive� groups� are� not� significantly� different� (only�significant�with�POC),�whereas�the�two�less�attractive�groups�are�significantly�different� from�the�two�more�attractive�groups�(significant�with�all�procedures)��One�could�also�devise� nonparametric�equivalent�MCPs�for�methods�other�than�the�Scheffé�and�Tukey�procedures�
12.3 SPSS
In�our�last�section,�we�examine�what�SPSS�has�to�offer�in�terms�of�MCPs��Here�we�use�the� general� linear� model� module� (although� the� one-way� ANOVA� module� can� also� be� used)�� The�steps�for�requesting�a�one-way�ANOVA�were�presented�in�the�previous�chapter�and� will�not�be�reiterated�here��Rather,�we�will�assume�all�the�previously�mentioned�options� have�been�selected��The�last�step,�therefore,�is�selection�of�one�or�more�planned�(a�priori)�or�
363Multiple Comparison Procedures
post�hoc�MCPs��For�purposes�of�this�illustration,�the�Tukey�will�be�selected��However,�you� are�encouraged�to�examine�other�MCPs�for�this�dataset�
Step 1:�From�the�“Univariate”�dialog�box,�click�on�“Post Hoc”�to�select�various�post� hoc�MCPs�or�click�on�“Contrasts”�to�select�various�planned�MCPs�(see�screenshot�step�1)�
Clicking on “Contrasts” will allow you to conduct certain planned MCPs.
Clicking on “Post Hoc” will allow you
to select various post hoc MCPs.
MCPs
Step 2 (post hoc MCP):�Click�on�the�name�of�independent�variable�in�the�“Factor(s)”� list�box�in�the�top�left�and�move�to�the�“Post Hoc Tests for”�box�in�the�top�right�by�click- ing�on�the�arrow�key��Check�an�appropriate�MCP�for�your�situation�by�placing�a�checkmark�in� the�box�next�to�the�desired�MCP��In�this�example,�we�will�select�“Tukey�”�Click�on�“Continue” to�return�to�the�original�dialog�box��Click�on�“OK”�to�return�to�generate�the�output�
MCPs for instances when homogeneity of variance
assumption is met.
MCPs for instances when homogeneity of variance assumption
is not met.
Select the independent variable of interest from the list on the left and use the arrow to move to the “Post Hoc Tests for” box on the right.
Post hoc MCP
364 An Introduction to Statistical Concepts
Step 3a (planned MCP):�To�obtain�trend�analysis�contrasts,�click�the�“Contrasts”� button�from�the�“Univariate”�dialog�box�(see�screenshot�step�1)��From�the�“Contrasts”� dialog�box,�click�the�“Contrasts”�pulldown�and�scroll�down�to�“Polynomial.”
Planned contrast
Step 3b:�Click�“Change”�to�select�“Polynomial”�and�move�it�to�be�displayed�in�paren- theses�next�to�the�independent�variable��Recall�that�this�type�of�contrast�will�allow�testing� of� linear,� quadratic,� and� cubic� contrasts�� Other� specific� planned� contrasts� are� also� avail- able��Then�click�“Continue”�to�return�to�the�“Univariate”�dialog�box�
Planned contrast
Interpreting the output:�Annotated�results�from�the�Tukey�HSD�procedure,�as�one� example�MCP,�are�shown�in�Table�12�2��Note�that�CIs�around�a�mean�difference�of�0�are�given� to�the�right�for�each�contrast�
365Multiple Comparison Procedures
Table 12.2
Tukey�HSD�SPSS�Results�for�the�Statistics�Lab�Example
Multiple Comparisons
“Mean difference” is simply the difference between the means of the two groups compared. For example, the mean difference
of group 1 and group 2 is calculated as 11.1250 – 17.8750 = –6.7500
Number of Statistics Labs Attended
Tukey HSD
95% Confidence Interval
(I ) Level of Attractiveness
(J ) Level of Attractiveness
Mean
Difference
(I–J ) Std. Error Sig. Lower Bound
Upper Bound
Slightly attractive –6.7500 3.00465 .135 –14.9536 1.4536
Moderately attractive –9.1250* 3.00465 .025 –17.3286 –.9214
Unattractive
Very attractive –13.2500* 3.00465 .001 –21.4536 –5.0464
Unattractive 6.7500 3.00465 .135 –1.4536 14.9536
Moderately attractive –2.3750 3.00465 .858 –10.5786 5.8286
Slightly attractive
Very attractive –6.5000 3.00465 .158 –14.7036 1.7036
Unattractive 9.1250* 3.00465 .025 .9214 17.3286
Slightly attractive 2.3750 3.00465 .858 –5.8286 10.5786
Moderately attractive
Very attractive –4.1250 3.00465 .526 –12.3286 4.0786
Unattractive 13.2500* 3.00465 .001 5.0464 21.4536
Slightly attractive 6.5000 3.00465 .158 –1.7036 14.7036
Very attractive
Moderately attractive 4.1250 3.00465 .526 –4.0786 12.3286
Based on observed means.
The error term is Mean Square(error) = 36.112.
“Sig.” denotes the observed p value and provides the results of the contrasts. There are only two statistically significant contrasts. There is a statistically significant mean difference between: (1) group 1 (unattractive) and group 3 (moderately attractive); and (2) between group 1 (unattractive) and group 4 (very attractive). Note that there are only 6 unique contrast results:
The standard error calculated in SPSS uses the harmonic mean
(Tukey-Kramer modification):
SΨ΄ = MSerror 1 1 + 2 n1 n2
1
SΨ΄ =
SΨ΄ = 9.028 3.00465=
Descriptive Statistics Dependent Variable: Number of Statistics Labs Attended
Level of Attractiveness Mean Std. Deviation N
Unattractive 11.1250 5.48862 8 Slightly attractive 17.8750 5.93867 8 Moderately attractive 20.2500 7.28501 8 Very attractive 24.3750 5.09727 8 Total 18.4062 7.51283 32
Recall the means of the groups as presented in the previous chapter.
1 + 2
1 8
1 8
+ 1 8
+ 1 8
36.112 However there are redundant results presented in the table. For example, the comparison of group 1 and 2 (presented in results row 1) is the same as the comparison of group 2 and 1 (presented in results row 2).
½[J (J–1)] = ½[4 (4–1)] = ½(12) = 6.
366 An Introduction to Statistical Concepts
12.4 Template and APA-Style Write-Up
In�terms�of�an�APA-style�write-up,�the�MCP�results�for�the�Tukey�HSD�test�for�the�statistics� lab�example�are�as�follows�
Recall� that� our� graduate� research� assistant,� Marie,� was� working� on� a� research� project� for�an�independent�study�class�to�determine�if�there�was�a�mean�difference�in�the�number� of� statistics� labs� attended� based� on� the� attractiveness� of� the� lab� instructor�� Her� research� question�was�the�following:�Is there a mean difference in the number of statistics labs students attended based on the attractiveness of the lab instructor?� Marie� then� generated� a� one-way� ANOVA�as�the�test�of�inference��The�APA-style�example�paragraph�of�results�for�the�one- way�ANOVA,�prefaced�by�the�extent�to�which�the�assumptions�of�the�test�were�met,�was� presented�in�the�previous�chapter��Thus�only�the�results�of�the�MCP�(specifically�the�Tukey� HSD)�are�presented�here�
Post hoc analyses were conducted given the statistically significant omnibus ANOVA F test. Specifically, Tukey HSD tests were conducted on all possible pairwise contrasts. The following pairs of groups were found to be significantly different (p < .05): groups 1 (unat- tractive; M = 11.125, SD = 5.4886) and 3 (moderately attractive; M = 20.2500, SD = 7.2850), and groups 1 (unattractive) and 4 (very attrac- tive; M = 24.3750, SD = 5.0973). In other words, students enrolled in the least attractive instructor group attended statistically signifi- cantly fewer statistics labs than students enrolled in either of the two most attractive instructor groups.
12.5 Summary
In� this� chapter,� methods� involving� the� comparison� of� multiple� group� means� for� a� single� independent� variable� were� considered�� The� chapter� began� with� a� look� at� the� characteristics� of� multiple� comparisons� including� (a)� the� definition� of� a� contrast,� (b) planned�and post�hoc�comparisons,�(c)�contrast-based�and�family-wise�Type�I�error� rates,�and (d)�orthogonal�contrasts��Next,�we�moved�into�a�lengthy�discussion�of�recom- mended�MCPs�
Figure� 12�2� is� a� flowchart� to� assist� you� in� making� decisions� about� which� MCP� to� use�� Not�every�statistician�will�agree�with�every�decision�on�the�flowchart�as�there�is�not�total� consensus�about�which�MCP�is�appropriate�in�every�single�situation��Nonetheless,�this�is� simply�a�guide��Whether�you�use�it�in�its�present�form�or�adapt�it�for�your�own�needs,�we� hope�you�find�the�figure�to�be�useful�in�your�own�research�
At� this� point,� you� should� have� met� the� following� objectives:� (a)� be� able� to� understand� the�concepts�underlying�the�MCPs,�(b)�be�able�to�select�the�appropriate�MCP�for�a�given� research�situation,�and�(c)�be�able�to�determine�and�interpret�the�results�of�MCPs��Chapter� 13�returns�to�ANOVA�again�and�discusses�models�for�which�there�is�more�than�one�inde- pendent�variable�
367Multiple Comparison Procedures
Problems
Conceptual problems
12.1� The�Tukey�HSD�procedure�requires�equal�n’s�and�equal�means��True�or�false? 12.2� Applying�the�Dunn�procedure,�given�a�nominal�family-wise�error�rate�of��10�and�two�
contrasts,�what�is�the�per�contrast�alpha? � a�� �01 � b�� �05 � c�� �10 � d�� �20
Start
Continuous? No
Planned? Yes
Orthogonal?
No
No
No
No
No
No
No
No
No
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Yes
Control only?
Many contrasts?
Dunnett
Dunn (Bonferroni)/ Dunn–Sidak
Stop Reject F ?
Pairwise? Scheffe’
Kaiser– Bowden
Tukey HSD/ Fisher LSD/
Hayter
Equal variances?
Equal n’s?
Equal variances?
Turkey–Kramer
Games–Howell/ Dunnett T3/ Dunnett C
POC
Trend analysis
FIGuRe 12.2 Flowchart�of�recommended�MCPs�
368 An Introduction to Statistical Concepts
12.3� �Which�of�the�following�linear�combinations�of�population�means�is�not�a�legitimate� contrast?
� a�� (μ�1�+�μ�2�+�μ�3)/3�−�μ�4 � b�� μ�1�−�μ�4 � c�� (μ�1�+�μ�2)/2�−�(μ�3�+�μ�4) � d�� μ�1�−�μ�2�+�μ�3�−�μ�4 12.4� �When�a�one-factor�fixed-effects�ANOVA�results�in�a�significant�F�ratio�for�J�=�2,�one�
should�follow�the�ANOVA�with�which�one�of�the�following�procedures?
� a�� Tukey�HSD�method
� b�� Scheffé�method
� c�� Hayter�method
� d�� None�of�the�above
12.5� �If� a� family-based� error� rate� for� α� is� desired,� and� hypotheses� involving� all� pairs� of� means� are� to� be� tested,� which� method� of� multiple� comparisons� should� be� selected?
� a�� Tukey�HSD
� b�� Scheffé
� c�� Planned�orthogonal�contrasts
� d�� Trend�analysis
� e�� None�of�the�above
12.6� A�priori�comparisons�are�which�one�of�the�following?
� a�� Are�planned�in�advance�of�the�research
� b�� Often�arise�out�of�theory�and�prior�research
� c�� May�be�done�without�examining�the�F�ratio
� d�� All�of�the�above
12.7� �For� planned� contrasts� involving� the� control� group,� the� Dunn� procedure� is� most� appropriate��True�or�false?
12.8� Which�is�not�a�property�of�planned�orthogonal�contrasts?
� a�� The�contrasts�are�independent�
� b�� The�contrasts�are�post�hoc�
� c�� The�sum�of�the�cross�products�of�the�contrast�coefficients�equals�0�
� d�� If�there�are�J�groups,�there�are�J�−�1�orthogonal�contrasts�
12.9� Which�MCP�is�most�flexible�in�the�contrasts�that�can�be�tested?
� a�� Planned�orthogonal�contrasts
� b�� Newman–Keuls
� c�� Dunnett
� d�� Tukey�HSD
� e�� Scheffé
369Multiple Comparison Procedures
12.10� Post�hoc�tests�are�necessary�after�an�ANOVA�given�which�one�of�the�following? � a�� H0�is�rejected� � b�� There�are�more�than�two�groups� � c�� H0�is�rejected�and�there�are�more�than�two�groups� � d�� You�should�always�do�post�hoc�tests�after�an�ANOVA� 12.11� �Post�hoc�tests�are�done�after�ANOVA�to�determine�why�H0�was�not�rejected��True�or�
false? 12.12� �Holding�the�α�level�and�the�number�of�groups�constant,�as�the�dferror�increases,�the�
critical�value�of�the�q�decreases��True�or�false? 12.13� �The�Tukey�HSD�procedure�maintains�the�family-wise�Type�I�error�rate�at��True�or�false? 12.14� �The�Dunnett�procedure�assumes�equal�numbers�of�observations�per�group��True�or�
false? 12.15� �For�complex�post�hoc�contrasts�with�unequal�group�variances,�which�of�the�follow-
ing�MCPs�is�most�appropriate? � a�� Kaiser–Bowden � b�� Dunnett � c�� Tukey�HSD � d�� Scheffé 12.16� �The�number�of�levels�of�the�independent�variable�is�6��How�many�orthogonal�con-
trasts�can�be�tested? � a�� 1 � b�� 3 � c�� 5 � d�� 6 12.17� �A�researcher�is�interested�in�testing�the�following�contrasts�in�a�J�=�6�study:�group�
1�versus�2,�group�3�versus�4,�and�group�5�versus�6��I�assert�that�these�contrasts�are� orthogonal��Am�I�correct?
12.18� �I� assert� that� rejecting� H0� in� a� one-factor� fixed-effects� ANOVA� with� J� =� 3� indicates� that�all�three�pairs�of�group�means�are�necessarily�statistically�significantly�differ- ent�using�the�Scheffé�procedure��Am�I�correct?
12.19� �For�complex�post�hoc�contrasts�with�equal�group�variances,�which�of�the�following� MCPs�is�most�appropriate?
� a�� Planned�orthogonal�contrasts � b�� Dunnett � c�� Tukey�HSD � d�� Scheffé 12.20� �A�researcher�finds�a�statistically�significant�omnibus�F�test��For�which�one�of�the�
following�will�there�be�at�least�one�statistically�significant�MCP? � a�� Kaiser–Bowden � b�� Dunnett � c�� Tukey�HSD � d�� Scheffé
370 An Introduction to Statistical Concepts
12.21� �If�the�difference�between�two�sample�means�is�1000,�I�assert�that�H0�will�necessarily� be�rejected�with�the�Tukey�HSD��Am�I�correct?
12.22� �Suppose�all�J�=�4�of�the�sample�means�are�equal�to�100��I�assert�that�it�is�possible�to� find�a�significant�contrast�with�some�MCP��Am�I�correct?
Computational problems
12.1� �A� one-factor� fixed-effects� ANOVA� is� performed� on� data� for� 10� groups� of� unequal� sizes,�and�H0�is�rejected�at�the��01�level�of�significance��Using�the�Scheffé�procedure,� test�the�contrast�that
Y Y. .2 5 0− =
at�the��01�level�of�significance�given�the�following�information:�dfwith�=�40,�Y –
�2�=�10�8,� n2�=�8,�Y
– �5�=�15�8,�n5�=�8,�and�MSwith�=�4�
12.2� �A�one-factor�fixed-effects�ANOVA�is�performed�on�data�from�three�groups�of�equal� size�(n�=�10),�and�H0�is�rejected�at�the��01�level��The�following�values�were�computed:� MSwith� =� 40� and� the� sample� means� are� Y
– �1� =� 4�5,� Y
– �2� =� 12�5,� and� Y
– �3� =� 13�0�� Use� the�
Tukey�HSD�method�to�test�all�possible�pairwise�contrasts� 12.3� �A�one-factor�fixed-effects�ANOVA�is�performed�on�data�from�three�groups�of�equal�
size�(n�=�20),�and�H0�is�rejected�at�the��05�level��The�following�values�were�computed:� MSwith�=�60�and�the�sample�means�are�Y
– �1�=�50,�Y
– �2�=�70,�and�Y
– �3�=�85��Use�the�Tukey�
HSD�method�to�test�all�possible�pairwise�contrasts� 12.4� �Using�the�data�from�Chapter�11,�Computational�Problem�4,�conduct�a�trend�analysis�
at�the��05�level� 12.5� �Consider�the�situation�where�there�are�J�=�4�groups�of�subjects��Answer�the�follow-
ing�questions: � a�� Construct�a�set�of�orthogonal�contrasts�and�show�that�they�are�orthogonal� � b�� Is�the�following�contrast�legitimate?�Why�or�why�not?
� H 0 : 2 3 4µ µ µ µ.1 . . .( )– + +
� c�� �Using�the�same�means,�how�might�the�contrast�in�part�(b)�be�altered�to�yield�a� legitimate�contrast?
Interpretive problems
12.1� �For�the�interpretive�problem�you�selected�in�Chapter�11�(using�the�survey�1�dataset� on�the�website),�select�an�a�priori�MCP,�apply�it�using�SPSS,�and�write�an�APA-style� paragraph�describing�the�results�
12.2� �For�the�interpretive�problem�you�selected�in�Chapter�11�(using�the�survey�1�dataset� on�the�website),�select�a�post�hoc�MCP,�apply�it�using�SPSS,�and�write�an�APA-style� paragraph�describing�the�results�
371
13 Factorial Analysis of Variance: Fixed-Effects Model
Chapter Outline
13�1� Two-Factor�ANOVA�Model � 13�1�1� Characteristics�of�the�Model � 13�1�2� Layout�of�Data � 13�1�3� ANOVA�Model � 13�1�4� Main�Effects�and�Interaction�Effects � 13�1�5� Assumptions�and�Violation�of�Assumptions � 13�1�6� Partitioning�the�Sums�of�Squares � 13�1�7� ANOVA�Summary�Table � 13�1�8� Multiple�Comparison�Procedures � 13�1�9� Effect�Size�Measures,�Confidence�Intervals,�and�Power
13�1�10� Example � 13�1�11� Expected�Mean�Squares 13�2� Three-Factor�and�Higher-Order�ANOVA � 13�2�1� Characteristics�of�the�Model � 13�2�2� ANOVA�Model � 13�2�3� ANOVA�Summary�Table�and�Example � 13�2�4� Triple�Interaction 13�3� Factorial�ANOVA�With�Unequal�n’s 13�4� SPSS�and�G*Power 13�5� Template�and�APA-Style�Write-Up
Key Concepts
� 1�� Main�effects � 2�� Interaction�effects � 3�� Partitioning�the�sums�of�squares � 4�� The�ANOVA�model � 5�� Main-effects�contrasts�and�simple�and�complex�interaction�contrasts � 6�� Nonorthogonal�designs
372 An Introduction to Statistical Concepts
The�last�two�chapters�have�dealt�with�the�one-factor�analysis�of�variance�(ANOVA)�model�and� various�multiple�comparison�procedures�(MCPs)�for�that�model��In�this�chapter,�we�continue� our�discussion�of�ANOVA�models�by�extending�the�one-factor�case�to�the�two-�and�three-factor� models��This�chapter�seeks�an�answer�to�the�following�question:�What�should�we�do�if�there� are�multiple�factors�for�which�we�want�to�make�comparisons�of�the�means?�In�other�words,� the�researcher�is�interested�in�the�effect�of�two�or�more�independent�variables�or�factors�on�the� dependent�(or�criterion)�variable��This�chapter�is�most�concerned�with�two-�and�three-factor� models,�but�the�extension�to�more�than�three�factors,�when�warranted,�is�fairly�simple�
For� example,� suppose� that� a� researcher� is� interested� in� the� effects� of� textbook� choice� and�time�of�day�on�statistics�achievement��Thus,�one�independent�variable�would�be�the� textbook�selected�for�the�course,�and�the�second�independent�variable�would�be�the�time� of�day�the�course�was�offered��The�researcher�hypothesizes�that�certain�texts�may�be�more� effective� in� terms� of� achievement� than� others� and� that� student� learning� may� be� greater� at� certain� times� of� the� day�� For� the� time-of-day� variable,� one� might� expect� that� students� would�not�do�as�well�in�an�early�morning�section�or�a�late�evening�section�than�at�other� times�of�the�day��In�the�example�study,�say�that�the�researcher�is�interested�in�comparing� three�textbooks�(A,�B,�and�C)�and�three�times�of�the�day�(early�morning,�mid-afternoon,� and�evening�sections)��Students�would�be�randomly�assigned�to�sections�of�statistics�based� on�a�combination�of�textbook�and�time�of�day��One�group�of�students�might�be�assigned�to� the�section�offered�in�the�evening�using�textbook�A��These�results�would�be�of�interest�to� statistics�instructors�for�selecting�a�textbook�and�optimal�time�of�the�day�
Most�of�the�concepts�used�in�this�chapter�are�the�same�as�those�covered�in�Chapters�11� and�12��In�addition,�new�concepts�include�main�effects,�interaction�effects,�MCPs�for�main� and�interaction�effects,�and�nonorthogonal�designs��Our�objectives�are�that�by�the�end� of�this�chapter,�you�will�be�able�to�(a)�understand�the�characteristics�and�concepts�underly- ing�factorial�ANOVA,�(b)�determine�and�interpret�the�results�of�factorial�ANOVA,�and� (c)�understand�and�evaluate�the�assumptions�of�factorial�ANOVA�
13.1 Two-Factor ANOVA Model
Marie,�the�educational�research�graduate�student�that�we�have�been�following,�successfully� conducted�an�experiment�and�used�(as�we�saw�in�a�previous�chapter)�one-way�ANOVA�to� answer�her�research�question��As�we�will�see�in�this�chapter,�Marie�will�be�extending�her� analysis�to�include�an�additional�independent�variable�
As�we�learned�in�Chapter�11,�Marie�is�enrolled�in�an�independent�study�class��As�part�of�the� course�requirement,�she�was�required�to�complete�a�research�study��In�collaboration�with� the�statistics�faculty�in�her�program,�Marie�designed�an�experimental�study�to�determine�if� there�was�a�mean�difference�in�student�attendance�in�the�statistics�lab�based�on�the�attrac- tiveness�of�the�statistics�lab�instructor��Marie�had�also�included�an�additional�component�to� this�experiment—time�of�day�that�the�course�was�taken�(afternoon�or�evening)—and�she�is� now�ready�to�examine�these�data��Marie’s�research�question�is�the�following:�Is there a mean difference in the number of statistics labs attended by students based on the attractiveness of the lab instructor and time of day that the course is offered?�With�two�independent�variables,�Marie� determines�that�a�factorial�ANOVA�is�the�best�statistical�procedure�to�use�to�answer�her� question��Her�next�task�is�to�collect�and�analyze�the�data�to�address�her�research�question�
373Factorial Analysis of Variance: Fixed-Effects Model
This�section�describes�the�distinguishing�characteristics�of�the�two-factor�ANOVA�model,� the�layout�of�the�data,�the�linear�model,�main�effects�and�interactions,�assumptions�of�the� model�and�their�violation,�partitioning�the�sums�of�squares,�the�ANOVA�summary�table,� MCPs,�effect�size�measures,�confidence�intervals�(CIs),�power,�an�example,�and�expected� mean�squares�
13.1.1 Characteristics of the Model
The� first� characteristic� of� the� two-factor� ANOVA� model� should� be� obvious� by� now;� this� model�considers�the�effect�of�two�factors�or�independent�variables�on�a�dependent�variable�� Each�factor�consists�of�two�or�more�levels�(or�categories)��This�yields�what�we�call�a�facto- rial design�because�more�than�a�single�factor�is�included��We�see�then�that�the�two-factor� ANOVA�is�an�extension�of�the�one-factor�ANOVA��Why�would�a�researcher�want�to�compli- cate�things�by�considering�a�second�factor?�Three�reasons�come�to�mind��First,�the�researcher� may�have�a�genuine�interest�in�studying�the�second�factor��Rather�than�studying�each�fac- tor� separately� in� two� analyses,� the� researcher� includes� both� factors� in� the� same� analysis�� This� allows� a� test� not� only� of� the� effect� of� each� individual� factor,� known� as� main effects,� but� of� the� effect� of� both� factors� collectively�� This� latter� effect� is� known� as� an� interaction effect�and�provides�information�about�whether�the�two�factors�are�operating�independent� of�one�another�(i�e�,�no�interaction�exists)�or�whether�the�two�factors�are�operating�together� to�produce�some�additional�impact�(i�e�,�an�interaction�exists)��If�two�separate�analyses�were� conducted,�one�for�each�independent�variable,�no�information�would�be�obtained�about�the� interaction�effect��As�becomes�evident,�assuming�a�factorial�ANOVA�with�two�independent� variables,�the�researcher�will�test�three�hypotheses:�one�for�each�factor�or�main�effect�indi- vidually�and�a�third�for�the�interaction�between�the�factors��Factorial�ANOVA�models�with� more�than�two�independent�variables�will,�accordingly,�test�for�additional�main�effects�and� interactions��This�chapter�spends�considerable�time�discussing�interactions�
A�second�reason�for�including�an�additional�factor�is�an�attempt�to�reduce�the�error�(or�within- groups)�variation,�which�is�variation�that�is�unexplained�by�the�first�factor��The�use�of�a�second� factor�provides�a�more�precise�estimate�of�error�variance��For�this�reason,�a�two-factor�design�is� generally�more�powerful�than�two�one-factor�designs,�as�the�second�factor�and�the�interaction� serve�to�control�for�additional�extraneous�variability��A�third�reason�for�considering�two�factors� simultaneously�is�to�provide�greater�generalizability�of�the�results�and�to�provide�a�more�effi- cient�and�economical�use�of�observations�and�resources��Thus,�the�results�can�be�generalized�to� more�situations,�and�the�study�will�be�more�cost�efficient�in�terms�of�time�and�money�
In�addition,�for�the�two-factor�ANOVA,�every�level�of�the�first�factor�(hereafter�known� as�factor�A)�is�paired�with�every�level�of�the�second�factor�(hereafter�known�as�factor�B)�� In� other� words,� every� combination� of� factors� A� and� B� is� included� in� the� design� of� the� study,�yielding�what�is�referred�to�as�a�fully crossed design��If�some�combinations�are�not� included,�then�the�design�is�not�fully�crossed�and�may�form�some�sort�of�a�nested�design� (see�Chapter�16)��Individuals�(or�objects�or�subjects)�are�randomly�assigned�to�one�combi- nation�of�the�two�factors��In�other�words,�each�individual�responds�to�only�one�combina- tion�of�the�factors��If�individuals�respond�to�more�than�one�combination�of�the�factors,�this� would�be�some�sort�of�repeated�measures�design,�which�we�examine�in�Chapter�15��In�this� chapter,� we� only� consider� models� where� all� factors� are� fixed�� Thus,� the� overall� design� is� known�as�a�fixed-effects�model��If�one�or�both�factors�are�random,�then�the�design�is�not� a� fixed-effects� model,� which� we� discuss� in� Chapter� 15�� It� is� also� a� condition� for� factorial� ANOVA�that�the�dependent�variable�is�measured�at�least�at�the�interval�level�and�the�inde- pendent�variables�are�categorical�(either�nominal�or�ordinal)�
374 An Introduction to Statistical Concepts
In�this�section�of�the�chapter,�for�simplicity�sake,�we�impose�the�restriction�that�the�num- ber�of�observations�is�the�same�for�each�factor�combination��This�yields�what�is�known�as� an�orthogonal�design,�where�the�effects�due�to�the�factors�(separately�and�collectively)�are� independent� or� unrelated�� We� leave� the� discussion� of� the� unequal� n’s� factorial� ANOVA� until�later�in�this�chapter��In�addition,�there�must�be�at�least�two�observations�per�factor� combination�so�as�to�have�within-groups�variation�
In�summary,�the�characteristics�of�the�two-factor�ANOVA�fixed-effects�model�are�as�fol- lows:�(a)�two�independent�variables�(both�of�which�are�categorical)�each�with�two�or�more� levels,�(b)�the�levels�of�both�independent�variables�are�fixed�by�the�researcher,�(c)�subjects� are�randomly�assigned�to�only�one�combination�of�these�levels,�(d)�the�two�factors�are�fully� crossed,�and�(e)�the�dependent�variable�is�measured�at�least�at�the�interval�level��In�the�con- text�of�experimental�design,�the�two-factor�ANOVA�is�often�referred�to�as�the�completely randomized factorial design�
13.1.2 layout of data
Before�we�get�into�the�theory�and�analysis�of�the�data,�let�us�examine�one�form�in�which�the� data� can� be� placed,� known� as� the� layout� of� the� data�� We� designate� each� observation� as� Yijk,� where�the�j�subscript�tells�us�what�level�(or�category)�of�factor�A�(e�g�,�textbook)�the�observa- tion�belongs�to,�the�k�subscript�tells�us�what�level�of�factor�B�(e�g�,�time�of�day)�the�observation� belongs� to,� and� the� i� subscript� tells� us� the� observation� or� identification� number� within� that� combination�of�factor�A�and�factor�B��For�instance,�Y321�would�mean�that�this�is�the�third�obser- vation�in�the�second�level�of�factor�A�and�the�first�level�of�factor�B��The�first�subscript�ranges� over�i�=�1,�…,�n;�the�second�subscript�ranges�over�j�=�1,�…,�J;�and�the�third�subscript�ranges�over� k�=�1,�…,�K��Note�also�that�the�latter�two�subscripts�denote�the�cell�of�an�observation��Using�the� same�example,�we�are�referring�to�the�third�observation�in�the�21�cell��Thus,�there�are�J�levels�of� factor�A,�K�levels�of�factor�B,�and�n�subjects�in�each�cell,�for�a�total�of�JKn = N�observations��For� now,�we�consider�the�case�where�there�are�n�subjects�in�each�cell�in�order�to�simplify�matters;� this�is�referred�to�as�the�equal�n’s�case��Later�in�this�chapter,�we�consider�the�unequal�n’s�case�
The�layout�of�the�sample�data�is�shown�in�Table�13�1��Here�we�see�that�each�row�represents� the�observations�for�a�particular�level�of�factor�A�(textbook)�and�that�each�column�represents� the�observations�for�a�particular�level�of�factor�B�(time)��At�the�bottom�of�each�column�are�the� column�means�(Y
– ��k ),�to�the�right�of�each�row�are�the�row�means�(Y
– �j��),�and�in�the�lower�right-
hand�corner�is�the�overall�mean�(Y –
…)��We�also�need�the�cell�means�(Y –
�jk ),�which�are�shown�at� the�bottom�of�each�cell��Thus,�the�layout�is�one�form�in�which�to�think�about�the�data�
13.1.3 aNOVa Model
This�section�introduces�the�ANOVA�linear�model,�as�well�as�estimation�of�the�parameters� of�the�model��The�two-factor�ANOVA�model�is�a�form�of�the�general�linear�model�(GLM)� like� the� one-factor� ANOVA� model� of� Chapter� 11�� The� two-factor� ANOVA� fixed-effects� model�can�be�written�in�terms�of�population�parameters�as
Yijk j k jk ijk= + + + +µ α β αβ ε( )
where Yijk�is�the�observed�score�on�the�criterion�(i�e�,�dependent)�variable�for�individual�i�in�level�
j�of�factor�A�(e�g�,�text)�and�level�k�of�factor�B�(e�g�,�time)�(or�in�the�jk�cell) μ �is�the�overall�or�grand�population�mean�(i�e�,�regardless�of�cell�designation) αj�is�the�main�effect�for�level�j�of�factor�A�(row�or�text�effect)
375Factorial Analysis of Variance: Fixed-Effects Model
βk�is�the�main�effect�for�level�k�of�factor�B�(column�or�time�effect) (αβ)jk�is�the�interaction�effect�for�the�combination�of�level�j�of�factor�A�and�level�k�of�factor�B εijk�is�the�random�residual�error�for�individual�i�in�cell�jk
The� residual� error� can� be� due� to� individual� differences,� measurement� error,� and/or� other� factors�not�under�investigation�
The�population�effects�and�residual�error�can�be�computed�as�follows:
α µ µ β µ µ αβ µ µ µ µ
ε µ
j j
k k
jk jk j k
ijk ijk jkY
= − = −
= − + − = −
. .
..
. . . ..
.
( ) ( )
That�is,�the�row�effect�is�equal�to�the�difference�between�the�population�mean�of�level�j�of� factor�A�(a�particular�text)�and�the�overall�population�mean,�the�column�effect�is�equal�to� the�difference�between�the�population�mean�of�level�k�of�factor�B�(a�particular�time)�and� the�overall�population�mean,�the�interaction�effect�is�the�effect�of�being�in�a�certain�com- bination�of�the�levels�of�factor�A�and�factor�B�(a�particular�text�used�at�a�particular�time),� whereas� the� residual� error� is� equal� to� the� difference� between� an� individual’s� observed�
Table 13.1
Layout�for�the�Two-Factor�ANOVA
Level of Factor B
Level of Factor A 1 2 … K Row Mean
1 Y111 Y112 … Y11K Y –
�1�
� � … � � … � � … Yn11 Yn12 … Yn1K — — —
Y –
�11 Y –
�12 … Y –
�1K
2 Y121 Y122 … Y12K Y –
�2�
� � … � � � … � � � … � Yn21 Yn22 … Yn2K — — —
Y –
�21 Y –
�22 … Y –
�2K
� � � … � �
J Y1J1 Y1J2 … Y1JK Y –
�J�
� � … � � � … � YnJ1 YnJ2 … YnJK — — —
Y –
�J1 Y –
�J2 … Y –
�JK
Column�mean Y –
��1 Y –
��2 Y –
��K Y –
…
376 An Introduction to Statistical Concepts
score�and�the�population�mean�of�cell�jk��The�row,�column,�and�interaction�effects�can�also� be�thought�of�as�the�average�effect�of�being�a�member�of�a�particular�row�(i�e�,�a�student� who� is� assigned� to� textbook� A,� B,� or� C),� column� (i�e�,� a� student� who� attends� class� in� the� afternoon�or�evening),�or�cell�(i�e�,�a�student�assigned�to�textbook�A,�B,�or�C�who�attends� class� in� the� afternoon� or� evening),� respectively�� It� should� also� be� noted� that� the� sum� of� the row�effects�is�equal�to�0,�the�sum�of�the�column�effects�is�equal�to�0,�and�the�sum�of the� interaction� effects� is� equal� to� 0� (both� across� rows� and� across� columns)�� This� implies,� for� example,� that� if� there� are� any�nonzero� row� effects,� then�the� row� effects� will�balance� out� around�0�with�some�positive�and�some�negative�effects�
You� may� be� wondering� why� the� interaction� effect� looks� a� little� different� than� the� main� effects��We�have�given�you�the�version�that�is�solely�a�function�of�population�means��A�more� intuitively�convincing�conceptual�version�of�this�effect�is�as�follows:
( ) .αβ µ α β µjk jk j k= − − −
which�is�written�in�similar�fashion�to�the�row�and�column�effects��Here�we�see�that�the�inter- action�effect�[(αβ)jk]�is�equal�to�the�population�cell�mean�(μ�jk)�minus�the�following:�(a)�the�row� effect,�(αj);�(b)�the�column�effect,�(βk);�and�(c)�the�overall�population�mean,�(μ)��In�other�words,� the�interaction�is�solely�a�function�of�cell�means�without�regard�to,�or�controlling�for,�its�row� effect,�column�effect,�or�the�overall�mean�
To�estimate�the�parameters�of�the�model�[μ,�αj,�βk,�(αβ)jk,�and�εijk],�the�least�squares�method� of�estimation�is�used�as�the�most�appropriate�for�GLMs�(e�g�,�regression,�ANOVA)��These� sample�estimates�are�represented�by�Y
– …,�aj,�bk,�(ab)jk,�and�eijk,�respectively,�where�the�latter�
four�are�computed�as�follows,�respectively:
a Y Yj j= −� � ���
b Y Yk k= −�� ���
( ) ( ). . . .. ...ab Y Y Y Yjk jk j k= − + −
e Y Yijk ijk jk= − �
Note�that Y –
…�represents�the�overall�sample�mean Y –
�j��represents�the�sample�mean�for�level�j�of�factor�A�(a�particular�text) Y –
��k�represents�the�sample�mean�for�level�k�of�factor�B�(a�particular�time) Y –
�jk�represents�the�sample�mean�for�cell�jk�(a�particular�text�at�a�particular�time)
For�the�two-factor�ANOVA�model,�there�are�three�sets�of�hypotheses,�one�for�each�of�the� main�effects�and�one�for�the�interaction�effect��The�null�and�alternative�hypotheses,�respec- tively,�for�testing�the�main�effect�of�factor�A�(text)�are�as�follows:
H J01 1 2: . . . . . .µ µ µ= = … =
H j11 not all the are equal: . .µ
The�hypotheses�for�testing�the�main�effect�of�factor�B�(time)�are�noted�as�follows:
H K02 1 2: .. .. ..µ µ µ= = … =
H k12 not all the are equal: ..µ
377Factorial Analysis of Variance: Fixed-Effects Model
Finally,�the�hypotheses�for�testing�the�interaction�effect�(text�with�time)�are�as�follows:
H j kjk j k0 03 for all and: ( ). . . ..µ µ µ µ− − + =
H jk j k13 not all the : ( ). . . ..µ µ µ µ− − + = 0
The�null�hypotheses�can�also�be�written�in�terms�of�row,�column,�and�interaction�effects� (which�may�make�more�intuitive�sense�to�you)�as
H J0 01 1 2: α α α= = … = =
H K0 02 1 2: β β β= = … = =
H j kjk0 03 for all and: ( )αβ =
As�in�the�one-factor�model,�all�of�the�alternative�hypotheses�are�written�in�a�general�form� to� cover� the� multitude� of� possible� mean� differences� that� could� arise�� These� range� from� only�two�of�the�means�being�different�to�all�of�the�means�being�different�from�one�another�� Also,�because�of�the�way�the�alternative�hypotheses�have�been�written,�only�a�nondirec- tional�alternative�is�appropriate��If�one�of�the�null�hypotheses�is�rejected,�then�consider�an� MCP�so�as�to�determine�which�means,�or�combination�of�means,�are�significantly�different� (discussed�later)�
13.1.4 Main effects and Interaction effects
Finally� we� come� to� a� formal� discussion� of� main� effects� and� interaction� effects�� A� main effect�of�factor�A�(text)�is�defined�as�the�effect�of�factor�A,�averaged�across�the�levels�of�fac- tor�B�(time),�on�the�dependent�variable�Y�(achievement)��More�precisely,�it�represents�the� unique�effect�of�factor�A�on�the�outcome�Y,�controlling�statistically�for�factor�B��A�similar� statement�may�be�made�for�the�main�effect�of�factor�B�
As�far�as�the�concept�of�interaction�is�concerned,�things�are�a�bit�more�complex��An�interac- tion�can�be�defined�in�any�of�the�following�ways:�An�interaction�is�said�to�exist�if�(a)�certain� combinations�of�the�two�factors�produce�effects�beyond�the�effects�of�the�two�factors�when� those�two�factors�are�considered�separately;�(b)�the�mean�differences�among�the�levels�of�factor� A�are�not�constant�across,�and�thus�depend�on,�the�levels�of�factor�B;�(c)�there�is�a�joint�effect�of� factors�A�and�B�on�Y;�or�(d)�there�is�a�unique�effect�that�could�not�be�predicted�from�knowledge� of�only�the�main�effects��Let�us�mention�two�fairly�common�examples�of�interaction�effects�� The�first�is�known�as�an�aptitude-treatment�interaction�(ATI)��This�means�that�the�effective- ness�of�a�particular�treatment�depends�on�the�aptitude�of�the�individual��In�other�words,�some� treatments�are�more�effective�for�individuals�with�a�high�aptitude,�and�other�treatments�are� more� effective� for� those� with� a� low� aptitude�� A� second� example� is� an� interaction� between� treatment�and�gender��Here�some�treatments�may�be�more�effective�for�males,�and�others�may� be�more�effective�for�females��This�is�often�considered�in�gender�studies�research�
For�some�graphical�examples�of�main�and�interaction�effects,�take�a�look�at�the�various� plots�in�Figure�13�1��Each�plot�represents�the�graph�of�a�particular�set�of�cell�means�(the� mean�of�the�dependent�variable�for�a�cell—the�combination�of�a�particular�category�of�fac- tor�A�and�a�particular�category�of�factor�B),�sometimes�referred�to�as�a�profile plot��On�the� X�axis�are�the�levels�of�factor�A�(text),�the�Y�axis�provides�the�cell�means�on�the�dependent� variable�Y�(achievement),�and�the�separate�lines�in�the�body�of�the�plot�represent�the�lev- els�of�factor�B�(time)�(although�the�specific�placement�of�the�two�factors�here�is�arbitrary;�
378 An Introduction to Statistical Concepts
alternatively�factor�B�could�be�plotted�on�the�X�axis,�and�factor�A,�as�the�separate�lines)�� Profile� plots� provide� information� about� the� possible� existence� of� a� main� effect� for� A,� a� main�effect�for�B,�and/or�an�interaction�effect��A�main�effect�for�factor�A�can�be�examined� by�taking�the�means�for�each�level�of�A�and�averaging�them�across�the�levels�of�B��If�these� marginal�means�for�the�levels�of�A�are�the�same�or�nearly�so,�this�would�indicate�no�main� effect�for�factor�A��A�main�effect�for�factor�B�can�be�assessed�by�taking�the�means�for�each� level�of�B�and�averaging�them�across�the�levels�of�A��If�these�marginal�means�for�the�levels� of�B�are�the�same�or�nearly�so,�this�would�imply�no�main�effect�for�factor�B��An�interaction�
27
Y
Y
25
35
15
(c) (d)
(a) (b)
1 2 A 1 2 A
1 2 A 1 2 A
32 30
22 20
Y 35
15
YB 1 2
B 1 2
B 1 2
B 1 2
Y
Y
30
24
20
14
26
20
14
1 2 (g) (h)
(e) (f )
A 1 2 A
1 2 A 1 2 A
Y
Y
25
15
27 25
17 15
B 1 2
B 1 2
B 1 2
B 1 2
FIGuRe 13.1 Display�of�possible�two-factor�ANOVA�effects�
379Factorial Analysis of Variance: Fixed-Effects Model
effect�is�determined�by�whether�the�cell�means�for�the�levels�of�A�are�constant�across�the� levels�of�B�(or�vice�versa)��This�is�easily�viewed�in�a�profile�plot�by�checking�to�see�whether� or�not�the�lines�are�parallel��Parallel�lines�indicate�no�interaction,�whereas�nonparallel�lines� suggest�that�an�interaction�may�exist��Of�course,�the�statistical�significance�of�the�main�and� interaction�effects�is�a�matter�to�be�determined�by�the�F�test�statistics�(coming�up)��The�pro- file�plots�only�give�you�a�rough�idea�as�to�the�possible�existence�of�the�effects��For�instance,� lines� that� are� nearly� parallel� will� probably� not� show� up� as� a� significant� interaction�� It� is� suggested�that�the�plot�can�be�simplified�if�the�factor�with�the�most�levels�is�shown�on�the� X�axis��This�cuts�down�on�the�number�of�lines�drawn�
The�plots�shown�in�Figure�13�1�represent�the�eight�different�sets�of�results�possible�for� a�two-factor�design,�that�is,�from�no�effects�to�all�three�effects�being�evident��To�simplify� matters,�only�two�levels�of�each�factor�are�used��Figure�13�1a�indicates�that�there�is�no�main� effect� either� for� factor� A� or� B,� and� there� is� no� interaction� effect�� The� lines� are� horizontal� (no�A�effect),�lie�nearly�on�top�of�one�another�(no�B�effect),�and�are�parallel�(no�interaction� effect)��Figure�13�1b�suggests�the�presence�of�an�effect�due�to�factor�A�only�(the�lines�are�not� horizontal�because�the�mean�for�A1�is�greater�than�the�mean�for�A2),�but�are�nearly�on�top�of� one�another�(no�B�effect)�and�are�parallel�(no�interaction)��In�Figure�13�1c,�we�see�a�separa- tion�between�the�lines�for�the�levels�of�B�(B1�being�greater�than�B2);�thus,�a�main�effect�for�B� is�likely,�but�the�lines�are�horizontal�(no�A�effect)�and�are�parallel�(no�interaction)�
For�Figure�13�1d,�there�are�no�main�effects�(the�means�for�the�levels�of�A�are�the�same,� and�the�means�for�the�levels�of�B�are�the�same),�but�an�interaction�is�indicated�by�the�lack� of�parallel�lines��Figure�13�1e�suggests�a�main�effect�for�both�factors�as�shown�by�mean�dif- ferences�(A1�less�than�A2,�and�B1�greater�than�B2),�but�no�interaction�(the�lines�are�parallel)�� In�Figure�13�1f,�we�see�a�main�effect�for�A�(A1�less�than�A2)�and�an�interaction�effect,�but�no� main�effect�for�B�(little�separation�between�the�lines�for�factor�B)��For�Figure�13�1g,�there� appear�to�be�a�main�effect�for�B�(B1�greater�than�B2)�and�an�interaction,�but�no�main�effect� for�A��Finally,�in�Figure�13�1h,�we�see�the�likelihood�of�two�main�effects�(A1�less�than�A2,� and�B1�greater�than�B2)�and�an�interaction��Although�these�are�clearly�the�only�possible�out- comes�from�a�two-factor�design,�the�precise�pattern�will�differ�depending�on�the�obtained� cell�means��In�other�words,�if�your�study�yields�a�significant�effect�only�for�factor�A,�your� profile�plot�need�not�look�exactly�like�Figure�13�1b,�but�it�will�retain�the�same�general�pattern� and�interpretation�
In�many�statistics�texts,�a�big�deal�is�made�about�the�type�of�interaction�shown�in�the�pro- file�plot��They�make�a�distinction�between�an�ordinal�interaction�and�a�disordinal�interac- tion��An�ordinal�interaction�is�said�to�exist�when�the�lines�are�not�parallel�and�they�do�not� cross;�ordinal�here�means�the�same�relative�order�of�the�cell�means�is�maintained�across�the� levels�of�one�of�the�factors��For�example,�the�means�for�level�1�of�factor�B�are�always�greater� than�the�means�for�level�2�of�B,�regardless�of�the�level�of�factor�A��A�disordinal�interaction� is�said�to�exist�when�the�lines�are�not�parallel�and�they�do�cross��For�example,�the�mean� for�B1�is�greater�than�the�mean�for�B2�at�A1,�but�the�opposite�is�true�at�A2��Dwelling�on�the� distinction�between�the�two�types�of�interaction�is�not�recommended�as�it�can�depend�on� how�the�plot�is�drawn�(i�e�,�which�factor�is�plotted�on�the�X�axis)��That�is,�when�factor�A�is� plotted�on�the�X�axis,�a�disordinal�interaction�may�be�shown,�and�when�factor�B�is�plotted� on�the�X�axis,�an�ordinal�interaction�may�be�shown��The�purpose�of�the�profile�plot�is�to� simplify�interpretation�of�the�results;�worrying�about�the�type�of�interaction�may�merely� serve�to�confuse�that�interpretation�
Let� us� take� a� moment� to� discuss� how� to� deal� with� an� interaction� effect�� Consider� two� possible�situations,�one�where�there�is�a�significant�interaction�effect�and�one�where�there� is� no� such� effect�� If� there� is� no� significant� interaction� effect,� then� the� findings� regarding�
380 An Introduction to Statistical Concepts
the� main� effects� can� be� generalized� with� greater� confidence�� In� this� situation,� the� main� effects� are� known� as� additive effects,� and� an� additive� linear� model� with� no� interaction� term�could�actually�be�used�to�describe�the�data��For�example,�the�results�might�be�that�for� factor�A,�the�level�1�means�always�exceed�those�of�level�2�by�10�points,�across�all�levels�of� factor�B��Thus,�we�can�make�a�blanket�statement�about�the�constant�added�benefits�of�A1� over�A2,�regardless�of�the�level�of�factor�B��In�addition,�for�the�no-interaction�situation,�the� main�effects�are�statistically�independent�of�one�another;�that�is,�each�of�the�main�effects� serves�as�an�independent�predictor�of�Y�
If�there�is�a�significant�interaction�effect,�then�the�findings�regarding�the�main�effects� cannot� be� generalized� with� such� confidence�� In� this� situation,� the� main� effects� are� not� additive,� and� the� interaction� term� must� be� included� in� the� linear� model�� For� example,� the�results�might�be�that�(a)�the�mean�for�A1�is�greater�than�A2�when�considering�B1,�but� (b)�the�mean�for�A1�is�less�than�A2�when�considering�B2��Thus,�we�cannot�make�a�blanket� statement�about�the�constant�added�benefits�of�A1�over�A2,�because�it�depends�on�the�level� of�factor�B��In�addition,�for�the�interaction�situation,�the�main�effects�are�not�statistically� independent�of�one�another;�that�is,�each�of�the�main�effects�does�not�serve�as�an�indepen- dent�predictor�of�Y��In�order�to�predict�Y�well,�information�is�necessary�about�the�levels�of� factors�A�and�B��Thus,�in�the�presence�of�a�significant�interaction,�generalizations�about� the� main� effects� must� be� qualified�� A� profile� plot� should� be� examined� so� that� a� proper� graphical� interpretation� of� the� interaction� and� main� effects� can� be� made�� A� significant� interaction�serves�as�a�warning�that�one�cannot�generalize�statements�about�a�main�effect� for�A�over�all�levels�of�B��If�you�obtain�a�significant�interaction,�this�is�an�important�result�� Do�not�ignore�it�and�go�ahead�to�interpret�the�main�effects�
13.1.5 assumptions and Violation of assumptions
In�Chapter�11,�we�described�in�detail�the�assumptions�for�the�one-factor�ANOVA��In�the� two-factor�model,�the�assumptions�are�again�concerned�with�independence,�homogeneity� of�variance,�and�normality��A�summary�of�the�effects�of�their�violation�is�provided�in�Table� 13�2��The�same�methods�for�detecting�violations�described�in�Chapter�11�can�be�used�for� this�model�
There� are� only� two� different� wrinkles� for� the� two-factor� model� as� compared� to� the� one-factor� model�� First,� as� the� effect� of� heterogeneity� is� small� with� balanced� designs�
Table 13.2
Assumptions�and�Effects�of�Violations�for�the�Two-Factor�ANOVA�Design
Assumption Effect of Assumption Violation
1��Independence •��Increased�likelihood�of�a�Type�I�and/or�Type�II�error�in� the�F�statistic
•��Influences�standard�errors�of�means�and�thus� inferences�about�those�means
2��Homogeneity�of�variance •�Bias�in�SSwith •�Increased�likelihood�of�a�Type�I�and/or�Type�II�error •�Less�effect�with�balanced�or�nearly�balanced�design •�Effect�decreases�as�n�increases
3��Normality •�Minimal�effect�with�moderate�violation •�Minimal�effect�with�balanced�or�nearly�balanced�design •�Effect�decreases�as�n�increases
381Factorial Analysis of Variance: Fixed-Effects Model
(equal� n’s� per� cell)� or� nearly� balanced� designs,� and/or� with� larger� n’s,� this� is� a� reason� to� strive�for�such�a�design��Unfortunately,�there�is�very�little�research�on�this�problem,�except� the�classic�Box�(1954b)�article�for�a�no-interaction�model�with�one�observation�per�cell��There� are�limited�solutions�for�dealing�with�a�violation�of�the�homogeneity�assumption,�such�as�the� Welch�(1951)�test,�the�Johansen�(1980)�procedure,�and�variations�described�by�Wilcox�(1996,� 2003)��Transformations�are�not�usually�used,�as�they�may�destroy�an�additive�linear�model� and�create�interactions�that�did�not�previously�exist��Nonparametric�techniques�are�not�com- monly�used�with�the�two-factor�model,�although�see�the�description�of�the�Brunner,�Dette,� and�Munk�(1997)�procedure�in�Wilcox�(2003)��Second,�the�effect�of�nonnormality�seems�to�be� the�same�as�heterogeneity�(Miller,�1997)�
13.1.6 partitioning the Sums of Squares
As�pointed�out�in�Chapter�11,�partitioning�the�sums�of�squares�is�an�important�concept�in� ANOVA��We�will�illustrate�with�a�two-factor�model,�but�this�can�be�extended�to�more�than� two� factors�� Let� us� begin� with� the� total� sum� of� squares� in� Y,� denoted� here� as� SStotal�� The� term�SStotal�represents�the�amount�of�total�variation�among�all�of�the�observations�without� regard�to�row,�column,�or�cell�membership��The�next�step�is�to�partition�the�total�variation� into�variation�between�the�levels�of�factor�A�(denoted�by�SSA),�variation�between�the�levels� of�factor�B�(denoted�by�SSB),�variation�due�to�the�interaction�of�the�levels�of�factors�A�and�B� (denoted�by�SSAB),�and�variation�within�the�cells�combined�across�cells�(denoted�by�SSwith)�� In�the�two-factor�ANOVA,�then,�we�can�partition�SStotal�into
� SS SS SS SS SStotal A B AB with= + + +
Then� computational� formulas� are� used� by� statistical� software� to� actually� compute� these� sums�of�squares�
13.1.7 aNOVa Summary Table
The� next� step� is� to� assemble� the� ANOVA� summary� table�� The� purpose� of� the� summary� table�is�to�simply�summarize�ANOVA��A�general�form�of�the�summary�table�for�the�two- factor�model�is�shown�in�Table�13�3��The�first�column�lists�the�sources�of�variation�in�the� model��We�note�that�the�total�variation�is�divided�into�a�within-groups�source,�and�a�gen- eral�between-groups�source,�which�is�then�subdivided�into�sources�due�to�A,�B,�and�the�AB� interaction��This�is�in�keeping�with�the�spirit�of�the�one-factor�model,�where�total�variation� was�divided�into�a�between-groups�source�(just�one�effect�because�there�is�only�one�factor� and� no� interaction� term)� and� a� within-groups� source�� The� second� column� provides� the� computed�sums�of�squares�
Table 13.3
Two-Factor�ANOVA�Summary�Table
Source SS df MS F
A SSA J�−�1 MSA MSA/MSwith B SSB K�−�1 MSB MSB/MSwith AB SSAB (J�−�1)(K�−�1) MSAB MSAB/MSwith Within SSwith N − JK MSwith Total SStotal N�−�1
382 An Introduction to Statistical Concepts
The�third�column�gives�the�degrees�of�freedom�for�each�source��As�always,�degrees�of�free- dom�have�to�do�with�the�number�of�observations�that�are�free�to�vary�in�a�particular�context�� Because�there�are�J�levels�of�factor�A,�then�the�number�of�degrees�of�freedom�for�the�A�source� is�equal�to�J�−�1��As�there�are�J�means�and�we�know�the�overall�mean,�then�only�J�−�1�of�the� means�are�free�to�vary��This�is�the�same�rationale�we�have�been�using�throughout�this�text�� As�there�are�K�levels�of�factor�B,�there�are�K�−�1�degrees�of�freedom�for�the�B�source��For�the� AB�interaction�source,�we�take�the�product�of�the�degrees�of�freedom�for�the�main�effects�� Thus,�we�have�as�degrees�of�freedom�for�AB�the�product�(J�−�1)(K�−�1)��The�degrees�of�freedom� within� groups� are� equal� to� the� total� number� of� observations� minus� the� number� of� cells,� N − JK��Finally,�the�degrees�of�freedom�total�can�be�written�simply�as�N�−�1�
Next,�the�sum�of�squares�terms�are�weighted�by�the�appropriate�degrees�of�freedom�to� generate�the�mean�squares�terms��Thus,�for�instance,�MSA�=�SSA/dfA��Finally,�in�the�last�col- umn�of�the�ANOVA�summary�table,�we�have�the�F�values,�which�represent�the�summary� statistics�for�ANOVA��There�are�three�hypotheses�that�we�are�interested�in�testing,�one�for� each�of�the�two�main�effects�and�one�for�the�interaction�effect,�so�there�will�be�three�F�test� statistics��For�the�factorial�fixed-effects�model,�each�F�value�is�computed�by�taking�the�MS� for�the�source�that�you�are�interested�in�testing�and�dividing�it�by�MSwith��Thus,�for�each� hypothesis,�the�same�error�term�is�used�in�forming�the�F�ratio�(i�e�,�MSwith)��We�return�to�the� two-factor�model�for�cases�where�the�effects�are�not�fixed�in�Chapter�15�
Each�of�the�F�test�statistics�is�then�compared�with�the�appropriate�F�critical�value�so�as� to�make�a�decision�about�the�relevant�null�hypothesis��These�critical�values�are�found�in� the�F�table�of�Table�A�4�as�follows:�for�the�test�of�factor�A�as��αFJ−1,N−JK;�for�the�test�of�factor� B� as� αFK−1,N−JK;� and� for� the� test� of� the� interaction� as� αF(J−1)(K−1),N−JK�� Thus,� with� a� two-factor� model,�testing�two�main�effects�and�one�interaction,�there�are�three�F�tests�and�three�deci- sions�that�must�be�made��Each�significance�test�is�one-tailed�so�as�to�be�consistent�with�the� alternative�hypothesis��The�null�hypothesis�is�rejected�if�the�F�test�statistic�exceeds�the� F�critical�value�
Recall�that�these�F�tests�are�omnibus�tests�that�tell�only�if�there�is�an�overall�main�effect� or�interaction�effect��If�the�F�test�statistic�does�exceed�the�F�critical�value,�and�there�is�more� than�one�degree�of�freedom�for�the�source�being�tested,�then�it�is�not�clear�precisely�why�the� null�hypothesis�was�rejected��For�example,�if�there�are�three�levels�of�factor�A�and�the�null� hypothesis�for�A�is�rejected,�then�we�are�not�sure�where�the�mean�differences�lie�among� the�levels�of�A��In�this�case,�some�MCP�should�be�used�to�determine�where�the�mean�differ- ences�are;�this�is�the�topic�of�the�next�section�
13.1.8 Multiple Comparison procedures
In�this�section,�we�extend�the�concepts�related�to�multiple�comparison�procedures�(MCPs)� covered�in�Chapter�12�to�the�two-factor�ANOVA�model��This�model�includes�main�and�inter- action�effects;�consequently�you�can�examine�contrasts�of�both�main�and�interaction�effects�� In�general,�the�procedures�described�in�Chapter�12�can�be�applied�to�the�two-factor�situation�� Things�become�more�complicated�as�we�have�row�and�column�means�(i�e�,�marginal�means)� and�cell�means��Thus,�we�have�to�be�careful�about�which�means�are�being�considered�
Let�us�begin�with�contrasts�of�the�main�effects��If�the�effect�for�factor�A�is�significant,�and� there�are�more�than�two�levels�of�factor�A,�then�we�can�form�contrasts�that�compare�the� levels�of�factor�A�ignoring�factor�B��Here�we�would�be�comparing�the�means�for�the�levels� of�factor�A,�which�are�marginal�means�as�opposed�to�cell�means��Considering�each�factor� separately� is� strongly� advised;� considering� the� factors� simultaneously� is� to� be� avoided�� Some� statistics� texts� suggest� that� you� consider� the� design� as� a� one-factor� model� with� JK�
383Factorial Analysis of Variance: Fixed-Effects Model
levels�when�using�MCPs�to�examine�main�effects��This�is�inconsistent�with�the�design�and� the�intent�of�separating�effects,�and�is�not�recommended�
For� contrasts� involving� the� interaction,� our� recommendation� is� to� begin� with� a� complex� interaction�contrast�if�there�are�more�than�four�cells�in�the�model��Thus,�for�example,�in�a�4��4� design�that�consists�of�four�levels�of�factor�A�(method�of�instruction)�and�four�levels�of�fac- tor�B�(instructor),�one�possibility�is�to�test�both�4��2�complex�interaction�contrasts��An�example� of�one�such�contrast�is�as�follows�[where,�e�g�,�(Y
– �11�+�Y
– �21�+�Y
– �31�+�Y
– �41)�is�the�sum�of�the�cell�means�
of�each�level�of�factor�A�for�level�1�of�factor�B�and�(Y –
�12�+�Y –
�22�+�Y –
�32�+�Y –
�42)�is�the�sum�of�the�cell� means�of�each�level�of�factor�A�for�level�2�of�factor�B]:
Ψ ’ ( )
4 ( )
4 . . . . . . . .=
+ + + −
+ + +Y Y Y Y Y Y Y Y11 21 31 41 12 22 32 42
with�a�standard�error�of�the�following:
s MS c n jk
jkk
K
j
J
Ψ’ =
== ∑∑with
2
11
where�njk�is�the�number�of�observations�in�cell�jk��This�contrast�would�examine�the�inter- action� between� the� four� methods� of� instruction� and� the� first� two� instructors�� A� second� complex�interaction�contrast�could�consider�the�interaction�between�the�four�methods�of� instruction�and�the�other�two�instructors�
If�the�complex�interaction�contrast�is�significant,�then�follow�this�up�with�a�simple�inter- action� contrast� that� involves� only� four� cell� means�� This� is� a� single� degree� of� freedom� contrast�because�it�involves�only�two�levels�of�each�factor�(known�as�a�tetrad difference)�� An�example�of�such�a�contrast�is�the�following:
Ψ ’ ( ) ( ). . . .= − − −Y Y Y Y11 21 12 22
with�a�similar�standard�error�term��Using�the�same�example,�this�contrast�would�examine� the�interaction�between�the�first�two�methods�of�instruction�and�the�first�two�instructors�
Most�of�the�MCPs�described�in�Chapter�12�can�be�used�for�testing�main�effects�and�inter- action�effects�(although�there�is�some�debate�about�the�appropriate�use�of�interaction�con- trasts;�see�Boik,�1979;�Marascuilo�&�Levin,�1970,�1976)��Keppel�and�Wickens�(2004)�consider� interaction�contrasts�in�much�detail��Finally,�some�statistics�texts�suggest�the�use�of�simple� main�effects�in�testing�a�significant�interaction��These�involve�comparing,�for�example,�the� levels�of�factor�A�at�a�particular�level�of�factor�B�and�are�generally�conducted�by�further� partitioning�the�sums�of�squares��However,�the�simple�main�effects�sums�of�squares�repre- sent�a�portion�of�a�main�effect�plus�the�interaction�effect��Thus,�the�simple�main�effect�does� not�really�help�us�to�understand�the�interaction,�and�is�not�recommended�here�
13.1.9 effect Size Measures, Confidence Intervals, and power
Various� measures� of� effect� size� have� been� proposed�� Let� us� examine� two� commonly� used� measures,�which�assume�equal�variances�across�the�cells��First�is�partial�eta�squared,�η2,�which� represents�the�proportion�of�variation�in�Y�explained�by�the�effect�of�interest�(i�e�,�by�factor�A�
384 An Introduction to Statistical Concepts
or�factor�B�or�the�AB�interaction)��This�is�the�estimate�of�effect�size�that�can�be�requested�when� using�SPSS�for�factorial�ANOVA��We�determine�partial�η2�as�follows:
partial SS
SS SS ηA
A
A with
2 = +
partial SS
SS SS ηB
B
B with
2 = +
partial SS
SS SS ηAB
AB
AB with
2 = +
Another�effect�size�measure�is�the�omega�squared�statistic,�ω2��We�can�determine�ω2�as�follows:
ωA A with
total with
2 1= − −
+ SS J MS SS MS
( )
ωB B with
total with
2 1= − −
+ SS K MS SS MS
( )
ωAB AB with
total with
2 1 1= − − −
+ SS J K MS
SS MS ( )( )
Using�Cohen’s�(1988)�subjective�standards,�these�effect�sizes�can�be�interpreted�as�follows:�small� effect,�η2�or�ω2�=��01;�medium�effect,�η2�or�ω2�=��06;�and�large�effect,�η2�or�ω2�=��14��For�further�dis- cussion,�see�Keppel�(1982),�O’Grady�(1982),�Wilcox�(1987),�Cohen�(1988),�Fidler�and�Thompson� (2001),�Keppel�and�Wickens�(2004),�and�Murphy,�Myors,�and�Wolach�(2008;�with�software)�
As�mentioned�in�Chapter�11,�CIs�can�be�used�for�providing�interval�estimates�of�a�popu- lation�mean�or�mean�difference;�this�gives�us�information�about�the�accuracy�of�a�sample� estimate�� In� the� case� of� the� two-factor� model,� we� can� form� CIs� for� row� means,� column� means,�cell�means,�the�overall�mean,�as�well�as�any�possible�contrast�formed�through�an� MCP�� Note� also� that� CIs� have� been� developed� for� η2� and� ω2� (Fidler� &� Thompson,� 2001;� Smithson,�2001)�
As�also�mentioned�in�Chapter�11,�power�can�be�determined�either�in�the�planned�(a�pri- ori)� or� observed� (post� hoc)� power� context�� For� planned� power,� we� typically� use� tables� or� power�charts�(e�g�,�Cohen,�1988,�or�Murphy�et�al�,�2008)�or�software�(e�g�,�Power�and�Precision,� Ex-Sample,�G*Power,�or�Murphy�et�al��software,�2008)��These�are�particularly�useful�in�terms� of�determining�adequate�sample�sizes�when�designing�a�study��Observed�power�is�reported� by�statistics�software,�such�as�SPSS,�to�indicate�the�actual�power�in�a�given�study�
13.1.10 example
Consider�the�following�illustration�of�the�two-factor�design��Here�we�expand�on�the�exam- ple�presented�in�Chapter�11�by�adding�a�second�factor�to�the�model��Our�dependent�vari- able�will�again�be�the�number�of�times�a�student�attends�statistics�lab�during�one�semester� (or�quarter),�factor�A�is�the�attractiveness�of�the�lab�instructor�(assuming�each�instructor� is�of�the�same�gender�and�is�equally�competent),�and�factor�B�is�the�time�of�day�the�lab�is� offered��Thus,�the�researcher�is�interested�in�whether�the�attractiveness�of�the�instructor,�
385Factorial Analysis of Variance: Fixed-Effects Model
the� time� of� day,� or� the� interaction� of� attractiveness� and� time� influences� student� atten- dance� in� the� statistics� lab�� The� attractiveness�levels� are� defined� again� as� (a)� unattractive,� (b) slightly�attractive,�(c)�moderately�attractive,�and�(d)�very�attractive��The�time�of�day�lev- els�are�defined�as�(a)�afternoon�lab�and�(b)�evening�lab��Students�were�randomly�assigned� to�a�combination�of�lab�instructor�and�lab�time�at�the�beginning�of�the�semester,�and�atten- dance� was� taken� by� the� instructor�� There� were� four� students� in� each� cell� and� eight� cells� (four�levels�of�attractiveness�and�two�categories�of�time,�thus�4��2�or�eight�combinations�of� instructor�and�time)�for�a�total�of�32�observations��Students�could�attend�a�maximum�of�30� lab�sessions��Table�13�4�depicts�the�raw�data�and�sample�means�for�each�cell�(given�beneath� each�cell),�column,�row,�and�overall�
The� results� are� summarized� in� the� ANOVA� summary� table� as� shown� in� Table� 13�5�� The�F�test�statistics�are�compared�to�the�following�critical�values�obtained�from�Table� A�4�(α�=��05):��05F3,24�=�3�01�for�the�A�(i�e�,�attractiveness)�and�AB�(i�e�,�attractiveness-time� of�day)�effects,�and��05F1,24�=�4�26�for�the�B�(time�of�day)�effect��The�test�statistics�exceed� the�critical�values�for�the�A�and�B�effects�only,�so�we�can�reject�these�H0�and�conclude� that�both�the�level�of�attractiveness�and�the�time�of�day�are�related�to�mean�differences� in�statistics�lab�attendance��The�interaction�was�shown�not�to�be�a�significant�effect��If�you� would�like�to�see�an�example�of�a�two-factor�design�where�the�interaction�is�significant,� take�a�look�at�the�end�of�chapter�problems,�Computational�Problem�13�5�
Table 13.4
Data�for�the�Statistics�Lab�Example:�Number�of�Statistics�Labs� Attended,�by�Level�of�Attractiveness�and�Time�of�Day
Time of Day
Level of Attractiveness Afternoon Evening Row Mean
Unattractive 15 10 11�1250 12 8 21 7 13 3 15�2500 7�0000
Slightly�attractive 20 13 17�8750 22 9 24 18 25 12 22�7500 13�0000
Moderately�attractive 24 10 20�2500 29 12 27 21 25 14 26�2500 14�2500
Very�attractive 30 22 24�3750 26 20 29 25 28 15 28�2500 20�5000
Column mean 23�1250 13�6875 18�4063� (overall mean)
386 An Introduction to Statistical Concepts
Next�we�estimate�the�main�and�interaction�effects��The�main�effects�for�the�levels�of�A� are�estimated�to�be�the�following:
Unattractive:�a Y Y1 1 11 1250 18 4063 7 2813= − = − = −. . ... . . .
Slightly�attractive:�a2�=�Y –
�2��−�Y –
…�=�17�8750�−�18�4063�=�−�0�5313
Moderately�attractive:�a3�=�Y –
�3��−�Y –
…�=�20�2500�−�18�4063�=�1�8437
Very�attractive:�a4�=�Y –
�4��−�Y –
…�=�24�3750�−�18�4063�=�5�9687
The�main�effects�for�the�levels�of�B�(time�of�day)�are�estimated�to�be�as�follows:
Afternoon:�b1�=�Y –
��1�−�Y –
…�=�23�1250�−�18�4063�=�4�7187
Evening:�b2�=�Y –
��2�−�Y –
…�=�13�6875�−�18�4063�=�−�4�7187
Finally,�the�interaction�effects�for�the�combinations�of�the�levels�of�factors�A�(attractiveness)� and�B�(time�of�day)�are�as�follows:
( ) ( ) . ( . . .. . . .. ...ab Y Y Y Y11 11 1 1 15 2500 11 1250 23 1250 18 4= − + − = − + − 0063 0 5937) .= −
( ) ( ) . ( . . .. . . .. ...ab Y Y Y Y12 12 1 2 7 0000 11 1250 13 6875 18 40= − + − = − + − 663 0 5938) .=
( ) ( ) . ( . . .. . . .. ...ab Y Y Y Y21 21 2 1 22 7500 17 8750 23 1250 18 4= − + − = − + − 0063 0 1563) .=
( ) ( ) . ( . . .. . . .. ...ab Y Y Y Y22 22 2 2 13 0000 17 8750 13 6875 18 4= − + − = − + − 0063 0 1562) .= −
( ) ( ) . ( . . .. . . .. ...ab Y Y Y Y31 31 3 1 26 2500 20 2500 23 1250 18 4= − + − = − + − 0063 1 2813) .=
( ) ( ) . ( . . .. . . .. ...ab Y Y Y Y32 32 3 2 14 2500 20 2500 13 6875 18 4= − + − = − + − 0063 1 2813) .= −
( ) ( ) . ( . . .. . . .. ...ab Y Y Y Y41 41 4 1 28 2500 24 3750 23 1250 18 4= − + − = − + − 0063 0 8437) .= −
( ) ( ) . ( . . .. . . .. ...ab Y Y Y Y42 42 4 2 20 5000 24 3750 13 6875 18 4= − + − = − + − 0063 0 8438) .=
The�profile�plot�shown�in�Figure�13�2�graphically�depicts�these�effects��The�main�effect�for� attractiveness�(factor�A)�was�statistically�significant�and�has�more�than�two�levels,�so�let�us�
Table 13.5
Two-Factor�ANOVA�Summary�Table—Statistics� Lab�Example
Source SS df MS F
A 738�5938 3 246�1979 21�3504a
B 712�5313 1 712�5313 61�7911b
AB 21�8438 3 7�2813 0�6314a
Within 276�7500 24 11�5313 Total 1749�7188 31
a� �05F3,24�=�3�01�
b� �05F1,24�=�4�26�
387Factorial Analysis of Variance: Fixed-Effects Model
consider�one�example�of�an�MCP,�the�Tukey�HSD�test��Recall�from�Chapter�12�that�the�HSD� test�is�a�family-wise�procedure�most�appropriate�for�considering�all�pairwise�contrasts�with� a�balanced�design�(which�is�the�case�for�these�data)��The�following�are�the�computations:
Critical�value�(obtained�from�Table�A�9):
α q qdf Jwith( ) = =, . , .0 05 24 4 3 9 1
Standard�error:
s MS n
Ψ’ .
.= = =with 11 5313
8 1 2006
Test�statistics:
q Y Y
s 1
4 1 24 3750 11 1250 1 2006
11 0361= −
= −
=. . . . ’
. . .
. Ψ
q Y Y
s 2
4 2 24 3750 17 8750 1 2006
5 4140= −
= −
=. . . . ’
. . .
. Ψ
q Y Y
s 3
4 3 24 3750 20 2500 1 2006
3 4358= −
= −
=. . . . ’
. . .
. Ψ
30.00
25.00
20.00
15.00
Es tim
at ed
m ar
gi na
l m ea
ns
10.00
5.00
Unattractive Slightly attractive
Level of attractiveness
Moderately attractive
Very attractive
Estimated marginal means of number of statistics labs attended
Time of day Afternoon Evening
FIGuRe 13.2 Profile�plot�for�example�data�
388 An Introduction to Statistical Concepts
q Y Y
s 4
3 1 20 2500 11 1250 1 2006
7 6004= −
= −
=. . . . ’
. . .
. Ψ
q Y Y
s 5
3 2 20 2500 17 8750 1 2006
1 9782= −
= −
=. . . . ’
. . .
. Ψ
q Y Y
s 6
2 1 17 8750 11 1250 1 2006
5 6222= −
= −
=. . . . ’
. . .
. Ψ
Recall� that� we� compare� the� test� statistic� value� to� the� critical� value� to� make� our� hypoth- esis�testing�decision��If�the�test�statistic�value�exceeds�the�critical�value,�we�reject�the�null� hypothesis�and�conclude�that�those�means�differ��For�these�tests,�the�results�indicate�that� the�means�for�the�levels�of�factor�A�(attractiveness)�are�statistically�significantly�different� for�levels�1�and�4�(i�e�,�the�test�statistic�value�is�11�0361,�and�the�critical�value�is�3�901),�2�and� 4,�1�and�3,�and�1�and�2��Thus,�level�1�(unattractive)�is�significantly�different�from�the�other� three�levels�of�attractiveness,�and�levels�2�and�4�(slightly�unattractive�vs��very�attractive)� are�also�significantly�different��The�only�levels�that�are�not�statistically�different�are�levels�2� and�3�(q5�=�1�9782)�and�levels�3�and�4�(q3�=�3�4358)�
These� results� are� somewhat� different� than� those� found� with� the� one-factor� model� in� Chapters�11�and�12�(where�the�significantly�different�levels�were�only�1�vs��4�and�1�vs��3)�� The�MSwith�has�been�reduced�with�the�introduction�of�the�second�factor�from�36�1116�to� 11�5313�because�SSwith�has�been�reduced�from�1011�1250�to�276�7500��Although�the�SS�and� MS� for� the� attractiveness� factor� remain� unchanged,� this� resulted� in� the� F� test� statistic� being� considerably� larger� (increased� from� 6�8177� to� 21�3504),� although� observed� power� was�quite�high�in�both�models��Recall�that�this�is�one�of�the�benefits�we�mentioned�ear- lier�about�the�use�of�additional�factors�in�the�model��Also,�although�the�effect�of�factor� B� (time� of� day)� was� significant,� there� are� only� two� levels� of� time� of� day,� and,� thus,� we� need�not�carry�out�any�multiple�comparisons�(attendance�is�better�in�the�afternoon�sec- tion)��Finally,�since�the�interaction�was�not�significant,�it�is�not�necessary�to�consider�any� related�contrasts�
Finally�we�can�estimate�the�effect�size�measures��The�partial�η2’s�are�determined�to�be� the�following:
ηA A
A with
2 738 5938 738 5938 276 7500
0 7274= +
= +
= SS
SS SS .
. . .
ηB B
B with
2 712 5313 712 5313 276 7500
0 7203= +
= +
= SS
SS SS .
. . .
ηAB AB
AB with
2 21 8438 21 8438 276 7500
0 0732= +
= +
= SS
SS SS .
. . .
We�calculate�ω2�to�be�the�following:
ωA A with
total with
2 1 738 5938 3 11 5313 1749
= − −
+ =
−SS J MS SS MS
( ) . ( ) . .77188 11 5313
0 3997 +
= .
.
389Factorial Analysis of Variance: Fixed-Effects Model
ωB B with
total with
2 1 712 5313 1 11 5313 1749
= − −
+ =
−SS K MS SS MS
( ) . ( ) . .77188 11 5313
0 3980 +
= .
.
ωAB AB with
total with
2 1 1 21 8438 3 11 531= − − −
+ =
−SS J K MS SS MS
( )( ) . ( ) . 33 1749 7188 11 5313
0 . .+
=
Based� on� these� effect� size� measures,� one� would� conclude� that� there� is� a� large� effect� for� instructor� attractiveness� and� for� time� of� day,� but� no� effect� for� the� time-attractiveness� interaction�
13.1.11 expected Mean Squares
As�we�asked�in�Chapter�11�for�the�one-factor�fixed-effects�model,�for�the�two-factor�fixed- effects� model� being� considered� here,� we� again� ask� the� question,� “How do we know which source of variation to use as the error term in the denominator”?�That�is,�for�the�two-factor�fixed- effects�ANOVA�model,�how�did�we�know�to�use�MSwith�as�the�error�term�in�testing�for�the� main� effects� and� the� interaction� effect?� As� we� learned� in� Chapter� 11,� an� expected� mean� square�for�a�particular�source�of�variation�represents�the�average�mean�square�value�for� that�source�obtained�if�the�same�study�were�to�be�replicated�an�infinite�number�of�times�� For�instance,�the�expected�value�of�MSA,�denoted�by�E(MSA),�is�the�average�value�of�MSA� over�repeated�samplings�
Let� us� examine� what� the� expected� mean� square� terms� actually� look� like� for� our� two- factor� fixed-effects� model�� Consider� the� two� situations� of� (a)� all� of� the� H0� actually� being� true� and� (b)� all� of� the� H0� actually� being� false�� If� all� of� the� H0� are� actually� true,� such� that� there�really�are�no�main�effects�or�an�interaction�effect,�then�the�expected�mean�squares� are�as�follows:
E MSA( ) = 2σε
E MSB( ) = σε2
E MSAB( ) = σε2
E MSwith( ) = σε2
and�thus�using�MSwith�as�the�error�term�will�produce�F�values�around�1� If�all�of�the�H0�are�actually�false,�such�that�there�really�are�main�effects�and�an�interaction�
effect,�then�the�expected�mean�squares�are�as�follows:
E /M nK Jj j
J
A( ) = +
− =
∑σ αε2 2 1
1( )
E /MS nJ Kk k
K
B( ) = +
−
= ∑σ βε2 2
1
1( )
390 An Introduction to Statistical Concepts
E /MS n J Kjk k
K
j
J
AB( ) = +
− − ==
∑∑σ αβε2 2 11
1 1( ) ( )( )
E MSwith( ) = σε2
and�thus�using�MSwith�as�the�error�term�will�produce�F�values�greater�than�1� There�is�a�difference�in�the�main�and�interaction�effects�between�when�H0�is�actually�true�
as�compared�to�when�H0�is�actually�false�because�in�the�latter�situation,�there�is�a�second� term��The�important�parts�of�this�second�term�are�α,�β,�and�αβ,�which�represent�the�effects� for�A,�B,�and�AB,�respectively��The�larger�this�part�becomes,�the�larger�the�F�ratio�becomes�� In�comparing�the�two�situations,�we�also�see�that�E(MSwith)�is�the�same�whether�H0�is�actu- ally� true� or� false,� and� thus� represents� a� reliable� estimate� of� σε
2�� This� term� is� mean-free� because�it�does�not�depend�on�any�mean�differences�
Finally�let�us�put�all�of�this�information�together��In�general,�the�F�ratio�represents
F = +( ) /(systematic variability error variability error variabiility)
where,�for�the�two-factor�fixed-effects�model,�systematic�variability�is�variability�due�to�the� main�or�interaction�effects�(i�e�,�between�sources)�and�error�variability�is�variability�within�� The�F�ratio�is�formed�in�a�particular�way�because�we�want�to�isolate�the�systematic�vari- ability�in�the�numerator��For�this�model,�the�only�appropriate�error�term�to�use�for�each�F� ratio�is�MSwith�because�it�does�serve�to�isolate�the�systematic�variability�
13.2 Three-Factor and Higher-Order ANOVA
13.2.1 Characteristics of the Model
All� of� the� characteristics� we� discussed� for� the� two-factor� model� apply� to� the� three-fac- tor�model,�with�one�obvious�exception��There�are�three�factors�rather�than�two��This�will� result� in� three� main� effects� (one� for� each� factor,� known� as� A,� B,� and� C),� three� two-way� interactions�(known�as�AB,�AC,�and�BC),�and�one�three-way�interaction�(known�as�ABC)�� The�only�new�concept�is�the�three-way�interaction,�which�may�be�stated�as�follows:�“Is�the� AB�interaction�constant�across�all�levels�of�factor�C”?�This�may�also�be�stated�as�“AC�across� the�levels�of�B”�or�as�“BC�across�the�levels�of�A�”�These�each�have�the�same�interpretation� as�there�is�only�one�way�of�testing�the�three-way�interaction��In�short,�the�three-way�inter- action�can�be�thought�of�as�the�two-way�interaction�behaving�differently�across�the�levels� of�the�third�factor�
We� do� not� explicitly� consider� models� with� more� than� three� factors� (cf�,� Keppel� &� Wickens,� 2004;� Marascuilo� &� Serlin,� 1988;� Myers� &� Well,� 1995)�� However,� be� warned� that� such� models� do� exist� and� that� they� will� necessitate� more� main� effects,� more� two- way�interactions,�more�three-way�interactions,�as�well�as�higher-order�interactions—and� thus�more�complex�interpretations��Conceptually,�the�only�change�is�to�add�these�addi- tional�effects�to�the�model�
391Factorial Analysis of Variance: Fixed-Effects Model
13.2.2 aNOVa Model
The�model�for�the�three-factor�design�is
Yijkl j k l jk jl kl jkl ijkl= + + + + + + + +µ α β γ αβ αγ βγ αβγ ε( ) ( ) ( ) ( )
where Yijkl�is�the�observed�score�on�the�criterion�(i�e�,�dependent)�variable�for�individual�i�in�level�j�
of�factor�A,�level�k�of�factor�B,�and�level�l�of�factor�C�(or�in�the�jkl�cell) μ�is�the�overall�or�grand�population�mean�(i�e�,�regardless�of�cell�designation) αj�is�the�effect�for�level�j�of�factor�A βk�is�the�effect�for�level�k�of�factor�B γl�is�the�effect�for�level�l�of�factor�C (αβ)jk� is� the� interaction� effect� for� the� combination� of� level� j� of� factor� A� and� level� k� of�
factor�B (αγ)jl�is�the�interaction�effect�for�the�combination�of�level�j�of�factor�A�and�level�l�of�fac-
tor�C (βγ)kl�is�the�interaction�effect�for�the�combination�of�level�k�of�factor�B�and�level�l�of�factor�C (αβγ)jkl�is�the�interaction�effect�for�the�combination�of�level�j�of�factor�A,�level�k�of�factor�B,�
and�level�l�of�factor�C εijkl�is�the�random�residual�error�for�individual�i�in�cell�jkl
Given� that� there� are� three� main� effects,� three� two-way� interactions,� and� one� three-way� interaction,�there�will�be�accompanying�null�and�alternative�hypotheses�for�each�of�these� effects�� At� this� point� in� your� statistics� career,� the� hypotheses� should� be� obvious� (simply� expand�on�the�hypotheses�at�the�beginning�of�this�chapter)�
13.2.3 aNOVa Summary Table and example
The� ANOVA� summary� table� for� the� three-factor� model� is� shown� in� Table� 13�6,� with� the� usual� columns� for� sources� of� variation,� sums� of� squares,� degrees� of� freedom,� mean� squares,�and�F��A�quick�three-factor�example�dataset�and�the�resulting�ANOVA�summary� table�from�SPSS�are�shown�in�Table�13�7��Note�that�the�only�statistically�significant�effects� are�the�main�effect�for�B�and�the�AC�interaction�(p�<��01)�
Table 13.6
Three-Factor�ANOVA�Summary�Table
Source SS df MS F
A SSA J�−�1 MSA MSA/MSwith B SSB K�−�1 MSB MSB/MSwith C SSC L�−�1 MSC MSC/MSwith AB SSAB (J�−�1)(K�−�1) MSAB MSAB/MSwith AC SSAC (J�−�1)(L�−�1) MSAC MSAC/MSwith BC SSBC (K�−�1)(L�−�1) MSBC MSBC/MSwith ABC SSABC (J�−�1)(K�−�1)(L�−�1) MSABC MSABC/MSwith Within SSwith N − JKL MSwith Total SStotal N�−�1
392 An Introduction to Statistical Concepts
Table 13.7
Three-Factor�Analysis�of�Variance�Example–Raw�Data�and�SPSS�ANOVA�Summary�Table
Raw�Data: A1B1C1:�8,�10,�12,�9 A1B1C2:�23,�17,�21,�19 A1B2C1:�22,�19,�16,�24 A1B2C2:�33,�31,�27,�30 A2B1C1:�16,�19,�21,�24 A2B1C2:�6,�8,�11,�13 A2B2C1:�27,�30,�31,�33 A2B2C2:�16,�19,�21,�25
SPSS ANOVA Summary Table:
Source Type III Sum of
Squares df Mean Square F Sig.
A .031 1 .031 .004 .953 B 871.531 1 871.531 100.200 .000 C .031 1 .031 .004 .953 A * B .031 1 .031 .004 .953 A * C 830.281 1 830.281 95.457 .000 B * C .031 1 .031 .004 .953 A * B * C .281 1 .281 .032 .859 Error 208.750 24 8.698 Corrected total 1910.969 31
The row labeled “A” is the first independent variable or factor or between groups variable. The between groups mean square for factor A (.031) provides an indication of the variation in the dependent variable attributable to factor A. The degrees of freedom for the sum of squares between groups for factor A is J – 1 (df = 1 in this example indicating 2 levels for factor A). Similar interpretations are made for the other main effects and interactions.
The omnibus F test for the main effect for factor A (and computed similarly for the other main effects and interactions) is computed as
F = = = .004 MSA .031
8.698MSwith The p value for the omnibus F test of the main effect for factor A is .953. This indicates there is not a statistically significant difference in the dependent variable based on factor A, averaged across the levels of Factors B and C. In other words, there is not a unique effect of factor A on the dependent variable, controlling for factors B and C. The probability of observing these mean differences or more extreme mean differences by chance if the null hypothesis is really true (i.e., if the population means really are equal) is about 95%. We fail to reject the null hypothesis that the population means of factor A are equal. For this example, this provides evidence to suggest that the dependent variable does not differ, on average, across the levels of factor A, when controlling for factors B and C.
The row labeled “Error” is within groups. The within groups sum of squares tells us how much variation there is within the cells combined across the cells (i.e., 208.750). The degrees of freedom for the sum of squares within groups is (N – JKL) or the sample size minus the number of levels of the independent variables [i.e., 32 – (2)(2)(2) = 24]. The row labeled “corrected total” is the sum of squares total. The degrees of freedom for the total is (N – 1) or the sample size minus one.
393Factorial Analysis of Variance: Fixed-Effects Model
13.2.4 Triple Interaction
Everything�else�about�the�three-factor�design�follows�from�the�two-factor�model��The� assumptions�are�the�same,�MSwith�is�the�error�term�used�for�testing�each�of�the�hypoth- eses�in�the�fixed-effects�model,�and�the�MCPs�are�easily�utilized��The�main�new�feature� is�the�three-way�interaction��If�this�interaction�is�significant,�then�this�means�that�the� two-way� interaction� is� different� across� the� levels� of� the� third� factor�� This� result� will� need� to� be� taken� into� account� prior� to� interpreting� the� two-way� interactions� and� the� main�effects�
Although�the�inclusion�of�additional�factors�in�the�design�should�result�in�a�reduc- tion�in�MSwith,�there�is�a�price�to�pay�for�the�study�of�additional�factors��Although�the� analysis� is� simple� for� the� computer,� you� must� consider� the� possibility� of� significant� higher-order�interactions��If�you�find,�for�example,�that�the�four-way�interaction�is�sig- nificant,� how� do� you� deal� with� it?� First� you� have� to� interpret� this� interaction,� which� could� be� difficult� if� it� is� unexpected�� Then� you� may� have� difficulty� in� dealing� with� the� interpretation� of� your� other� effects�� Our� advice� is� simple�� Do� not� include� addi- tional�factors�just�because�they�sound�interesting��Only�include�those�factors�that�are� theoretically� or� empirically� important�� Then� if� a� significant� higher-order� interaction� occurs,�you�will�be�in�a�better�position�to�understand�it�because�you�will�have�already� thought� about� its� consequences�� Reporting� that� an� interaction� is� significant,� but� not� interpretable,�is�not�sound�research�(for�additional�discussion�on�this�topic,�see�Keppel� &�Wickens,�2004)�
13.3 Factorial ANOVA With Unequal n’s
Up�until�this�point�in�the�chapter,�we�have�only�considered�the�equal�n’s�or�balanced�case�� That�is,�the�model�used�was�where�the�number�of�observations�in�each�cell�was�equal��This� served�to�make�the�formulas�and�equations�easier�to�deal�with��However,�we�do�not�need� to�assume�that�the�n’s�are�equal��In�this�section,�we�discuss�ways�to�deal�with�the�unequal� n’s� (or� unbalanced)� case� for� the� two-factor� model,� although� these� notions� can� be� trans- ferred�to�higher-order�models�as�well�
When�n’s�are�unequal,�things�become�a�bit�trickier�as�the�main�effects�and�the�interaction� effect�are�not�orthogonal��In�other�words,�the�sums�of�squares�cannot�be�partitioned�into� independent�effects,�and,�thus,�the�individual�SS�do�not�necessarily�add�up�to�the�SStotal�� As�a�result,�several�computational�approaches�have�been�developed��In�the�old�days,�prior� to�the�availability�of�high-speed�computers,�the�standard�approach�was�to�use�unweighted� means� analysis�� This� is� essentially� an� analysis� of� means,� rather� than� raw� scores,� which� are�unweighted�by�cell�size��This�approach�is�only�an�approximate�procedure��Due�to�the� availability� of� quality� statistical� software,� the� unweighted� means� approach� is� no� longer� necessary�� A� rather� silly� approach,� and� one� that� we� do� not� condone,� is� to� delete� enough� data�until�you�have�an�equal�n’s�model�
There�are�three�more�modern�approaches�to�this�case��Each�of�these�approaches�really� tests�different�hypotheses�and�thus�may�result�in�different�results�and�conclusions:�(a)�the� sequential approach�(also�known�as�the�hierarchical� sums�of�squares�approach),�(b)�the� partially sequential approach�(also�known�as�the�partially�hierarchical,�or�experimental� design,� or� method� of� fitting� constants� approach),� and� (c)� the� regression approach� (also� known�as�the�marginal�means�or�unique�approach)��There�has�been�considerable�debate�
394 An Introduction to Statistical Concepts
over� the� years� about� the� relative� merits� of� each� approach� (e�g�,� Applebaum� &� Cramer,� 1974;� Carlson� &� Timm,� 1974;� Cramer� &� Applebaum,� 1980;� Overall,� Lee,� &� Hornick,� 1981;� Overall &�Spiegel,�1969;�Timm�&�Carlson,�1975)��In�the�following,�we�describe�what�each� approach�is�actually�testing�
In�the�sequential�approach,�the�effects�being�tested�are�as�follows:
α µ|
β µ α| ,
αβ µ α β| , ,
This� indicates,� for� example,� that� the� effect� for� factor� B� (β)� is� adjusted� or� controls� for� (as� denoted�by�the�vertical�line)�the�overall�mean�(μ)�and�the�main�effect�due�to�factor�A�(α)�� Thus,� each� effect� is� adjusted� for� prior� effects� in� the� sequential� order� given� (i�e�,� α,� β,� αβ)�� Here�the�α�effect�is�given�theoretical�or�practical�priority�over�the�β�effect��In�SAS�and�SPSS,� this�is�the�Type I sum of squares�method�
In�the�partially�sequential�approach,�the�effects�being�tested�are�as�follows:
α µ β| ,
β µ α| ,
αβ µ α β| , ,
There�is�difference�here�because�each�main�effect�controls�for�the�other�main�effect,�but�not� for�the�interaction�effect��In�SAS�and�SPSS,�this�is�the�Type II sum of squares�method��This�is� the�only�one�of�the�three�methods�where�the�sums�of�squares�will�add�up�to�the�total�sum� of�squares��Notice�in�the�sequential�and�partially�sequential�approaches�that�the�interac- tion�is�not�taken�into�account�in�estimating�the�main�effects,�which�is�only�fine�if�there�is� no�interaction�effect�
In�the�regression�approach,�the�effects�being�tested�are�as�follows:
α µ β αβ| , ,
β µ α αβ| , ,
αβ µ α β| , ,
In�this�approach,�each�effect�controls�for�each�of�the�other�effects��In�SAS�and�SPSS,�this� is�the�Type III sum of squares�method�(and�is�the�default�selection�in�SPSS)��Many�statisti- cians�(e�g�,�Glass�&�Hopkins,�1996;�Keppel�&�Wickens,�2004;�Mickey,�Dunn,�&�Clark,�2004),� including�the�authors�of�this�text,�recommend�exclusive�use�of�the�regression�approach� because� each� effect� is� estimated� taking� the� other� effects� into� account�� The� hypotheses� tested�in�the�sequential�and�partially�sequential�approaches�are�seldom�of�interest�and� are�difficult�to�interpret�(Carlson�&�Timm,�1974;�Kirk,�1982;�Overall�et�al�,�1981;�Timm�and� Carlson,� 1975)�� The� regression� approach� seems� to� be� conceptually� closest� to� the� tradi- tional�ANOVA�in�that�each�effect�is�estimated�controlling�for�all�other�effects��When�the� n’s� are� equal,� each� of� these� three� approaches� tests� the� same� hypotheses� and� yields� the� same�results�
395Factorial Analysis of Variance: Fixed-Effects Model
13.4 SPSS and G*Power
Next�we�consider�the�use�of�SPSS�for�the�statistics�lab�example��Instructions�for�determin- ing�the�factorial�ANOVA�using�SPSS�are�presented�first,�followed�by�additional�steps�for� examining� the� assumptions� for� factorial� ANOVA�� Finally� we� examine� a� priori� and� post� hoc�power�for�this�model�using�G*Power�
Factorial ANOVA
In� this� section,� we� take� a� look� at� SPSS� for� the� statistics� lab� example�� As� already� noted� in� Chapter�11,�SPSS�needs�the�data�to�be�in�a�specific�form�for�the�analysis�to�proceed,�which�is� different�from�the�layout�of�the�data�in�Table�13�1��For�a�two-factor�ANOVA,�the�dataset�must� consist�of�three�variables�or�columns,�one�for�the�level�of�factor�A,�one�for�the�level�of�factor�B,� and�the�third�for�the�dependent�variable��Each�row�still�represents�one�individual,�indicating� the�levels�of�factors�A�and�B�that�individual�is�a�member�of,�and�their�score�on�the�dependent� variable��As�seen�in�the�following�screenshot,�for�a�two-factor�ANOVA,�the�SPSS�data�are�in� the�form�of�two�columns�that�represent�the�group�values�(i�e�,�the�two�independent�variables)� and�one�column�that�represents�the�scores�or�values�of�the�dependent�variable�
The first independent variable is labeled “Group” where each value represents the attractiveness of the statistics lab instructor to which the student was assigned. Group 1, you recall, represented “unattractive”.Thus there were eight students randomly assigned to an “unattractive” instructor. Since each of these eight students was in the same group, each is coded with the same value (1, which represents that they were assigned to an “unattractive” instructor). The other groups (2, 3, and 4) follow this pattern as well.
The second independent variable is labeled “Time” where each value represents the time of day of the course. One represents “afternoon” and two represents “evening.”
The dependent variable is “Labs” and represents the number of statistics labs the student attended.
396 An Introduction to Statistical Concepts
Step 1:�To�conduct�a�factorial�ANOVA,�go�to�“Analyze”�in�the�top�pulldown�menu,�then� select�“General Linear Model,”�and�then�select�“Univariate.”�Following�the�screen- shot�(Step�1)�that�follows�produces�the�“Univariate”�dialog�box�
A
B C
Factorial ANOVA:
Step 1
Step 2:�Click�the�dependent�variable�(e�g�,�number�of�statistics�labs�attended)�and�move� it�into�the�“Dependent Variable”�box�by�clicking�the�arrow�button��Click�the�first�inde- pendent�variable�(e�g�,�level�of�attractiveness)�and�move�it�into�the�“Fixed Factors”�box� by�clicking�the�arrow�button��Follow�this�same�step�to�move�the�second�independent�vari- able�into�the�“Fixed Factors”�box��Next,�click�on�“Options.”
Select the dependent variable from the list on the left and use the arrow to move
to the “Dependent Variable” box on
the right. Select the
independent variables from the list on the left and use the arrow to
move to the “Fixed Factor(s)” box
on the right.
Clicking on “Contrasts” will allow you to conduct certain planned MCPs.
Clicking on “Plots” will allow you to generate profile plots.
Clicking on “Post Hoc” will allow you to generate
post hoc MCPs.
Clicking on “Save” will allow you to save various forms of residuals, among
other variables.
Clicking on “Options” will allow you to obtain a number of other statistics (e.g., descriptive statistics, effect size, power, homogeneity tests).
Factorial ANOVA:
Step 2
397Factorial Analysis of Variance: Fixed-Effects Model
Step 3:� Clicking� on�“Options”� will� provide� the� option� to� select� such� information� as� “Descriptive Statistics,” “Estimates of effect size,” “Observed power,”� “Homogeneity tests”� (i�e�,� Levene’s� test� for� equal� variances),� and�“Spread versus level plots”�(those�are�the�options�that�we�typically�utilize)��Click�on�“Continue”�to� return�to�the�original�dialog�box�
Select from the list on the left those variables
that you wish to display means for and use the arrow to move to the
“Display Means for” box on the right.
Factorial ANOVA: Step 3
Step 4:� From� the�“Univariate”� dialog� box,� click� on�“Plots”� to� obtain� a� profile� plot� of� means�� Click� the� independent� variable� (e�g�,� level� of� attractiveness� labeled� as� “Group”)� and� move�it�into�the�“Horizontal Axis”�box�by�clicking�the�arrow�button�(see�screenshot�step� 4a)��(Tip: Placing the independent variable that has the most categories or levels on the horizontal axis of the profile plots will make for easier interpretation of the graph.)�Then�click�the�second�independent� variable�(e�g�,�“Time”)�and�move�it�into�the�“Separate Lines”�box�by�clicking�the�arrow�but- ton�(see�screenshot�Step�4a)��Then�click�on�“Add”�to�move�the�variable�into�the�“Plots”�box� at�the�bottom�of�the�dialog�box�(see�screenshot�Step�4b)��Click�on�“Continue”�to�return�to�the� original�dialog�box�
398 An Introduction to Statistical Concepts
Select one independent variable from the list on the left and use the arrow to move it
to the “Horizontal Axis” box on the right.
Factorial ANOVA: Step 4a
Select the second independent variable and use the arrow to move it to the “Separate
Lines” box on the right.
�en click “Add” to move the variable
into the “Plots” box at the bottom.
Factorial ANOVA: Step 4b
Step 5:�From�the�“Univariate”�dialog�box,�click�on�“Post Hoc”�to�select�various�post� hoc�MCPs�or�click�on�“Contrasts”�to�select�various�planned�MCPs�(see�screenshot�Step�1)�� From�the�“Post Hoc Multiple Comparisons for Observed Means”�dialog�box,�click� on�the�names�of�the�independent�variables�in�the�“Factor(s)”�list�box�in�the�top�left�(e�g�,� “Group”�and�“Time”)�and�move�them�to�the�“Post Hoc Tests for”�box�in�the�top�right� by�clicking�on�the�arrow�key��Check� an� appropriate� MCP� for� your� situation� by� placing� a� checkmark�in�the�box�next�to�the�desired�MCP��In�this�example,�we�will�select�“Tukey�”�Click� on�“Continue”�to�return�to�the�original�dialog�box�
399Factorial Analysis of Variance: Fixed-Effects Model
Select the independent
variables of interest from the list on the
left and use the arrow to move to the “Post Hoc Tests for” box on the right. MCPs for instances when the
homogeneity of variance assumption is met.
Factorial ANOVA: Step 5
MCPs for instances when the homogeneity of variance assumption
is not met.
Step 6:�From�the�“Univariate”�dialog�box,�click�on�“Save”�to�select�those�elements� that�you�want�to�save�(in�our�case,�we�want�to�save�the�unstandardized�residuals�which� will�be�used�later�to�examine�the�extent�to�which�normality�and�independence�are�met)�� From�the�“Univariate”�dialog�box,�click�on�“OK”�to�return�to�generate�the�output�
Factorial ANOVA: Step 6
Interpreting the output:� Annotated� results� are� presented� in� Table� 13�8,� and� the� profile�plot�is�shown�in�Figure�13�2��Note�that�in�order�to�test�interaction�contrasts�in�SPSS,� syntax�is�required�rather�than�the�use�of�point-and-click�features�used�primarily�in�this�text� (cf�,�Page,�Braver,�&�MacKinnon,�2003)��Note�also�that�the�SPSS�ANOVA�summary�table�will� include�additional�sources�of�variation�that�we�find�not�to�be�useful�(i�e�,�corrected�model,� intercept,�total);�thus,�they�are�not�annotated�in�Table�13�8�
400 An Introduction to Statistical Concepts
Table 13.8
Selected�SPSS�Results�for�the�Statistics�Lab�Example
Descriptive Statistics Dependent Variable: Number of Statistics Labs Attended
Level of Attractiveness Time of Day Mean Std. Deviation N
Afternoon
Evening
Unattractive
Total
Afternoon
Evening
Slightly attractive
Total
Afternoon
Evening
Moderately attractive
Total
Afternoon
Evening
Very attractive
Total
Afternoon
Evening
Total
Total
15.2500
7.0000
11.1250
22.7500
13.0000
17.8750
26.2500
14.2500
20.2500
28.2500
20.5000
24.3750
23.1250
13.6875
18.4062
4.03113
2.94392
5.48862
2.21736
3.74166
5.93867
2.21736
4.78714
7.28501
1.70783
4.20317
5.09727
5.65538
6.09611
7.51283
4
4
8
4
4
8
4
4
8
4
4
8
16
16
32
Between-Subjects Factors
Value Label N
1.00
2.00
3.00
Level of attractiveness
4.00
1.00Time of day
2.00
Unattractive
Slightly attractive
Moderately attractive
Very attractive
Afternoon
Evening
8
8
8
8
16
16
The table labeled “Between- Subjects Factors” provides
sample sizes for each of the categories of the independent
variables (recall that the independent variables are the “between subjects
factors”).
The table labeled “Descriptive Statistics”
provides basic descriptive statistics (means,
standard deviations, and sample sizes) for each
cell of the design.
Levene's Test of Equality of Error Variancesa
Dependent Variable: Number of Statistics Labs Attended
F df 1 df 2 Sig.
.579 7 24 .766
Note: Tests the null hypothesis that the error variance of the dependent variable is equal across groups. a Design: Intercept + Group + Time + Group * Time.
The F test (and associated p value) for Levene’s Test for Equality of Error Variances is reviewed to determine if equal variances can be assumed.
In this case, we meet the assumption (as p is greater than α). Note that df1 is calculated as
(JK – 1) and df 2 is calculated as (N – JK).
401Factorial Analysis of Variance: Fixed-Effects Model
Table 13.8 (continued)
Selected�SPSS�Results�for�the�Statistics�Lab�Example
Tests of Between-Subjects Effects Dependent Variable: Number of Statistics Labs Attended
Source
Type III Sum of Squares df
Mean Square F Sig.
Partial Eta Squared
Noncent.
Parameter
Observed Powerb
Corrected model 1472.969a 7 210.424 18.248 .000 .842 127.737 1.000
Intercept 10841.281 1 940.165 .000 .975 940.165 1.000
Group 738.594 3 246.198
10841.281
21.350 .000 .727 64.051 1.000
Time 712.531 1 712.531 61.791 .000 .720 61.791 1.000
Group * Time 21.844 3 7.281 .631 .602 .073 1.894 .162
Error 276.750 24 11.531
Total 12591.000 32
Corrected total
R2 = = = .842 SSbetw 738.594 + 712.531 + 21.844
1749.719SStotal
1749.719 31 a R squared = .842 (adjusted R squared = .796).
b Computed using alpha = .05.
Observed power tells whether our test is powerful enough to
detect mean differences if they
really exist. Power of 1.000 indicates the
maximum probability of rejecting the null
hypothesis if it is really false (i.e., very
strong power).
R squared is listed as a footnote underneath the table. R squared is the ratio of sum of squares between (i.e., combined SS for main effects and for the interaction) divided by sum of squares total:
�e row labeled “Error” is for within groups. �e within groups sum of
squares tells us how much variation there is within the cells combined across the cells (i.e., 276.750). �e degrees of freedom for
within groups is (N – JK) or the sample size minus the independent variables [i.e.,
32 – (4)(2) = 24]. �e row labeled
“Corrected Total” is the sum of squares total. The degrees of freedom for
the total is (N – 1) or the total sample size –1.
�e omnibus F test for the main effect for “Group” (i.e., attractiveness) (and computed similarly for the other main effects and interactions) is computed as
�e p value for the omnibus F test for the main effect for attractiveness is .000. �is indicates there is a statistically significant difference in the dependent variable based on attractiveness, averaged across time of day (afternoon and evening). In other words, there is a unique effect of attractiveness on the number of stat labs attended, controlling for time of day. �e probability of observing these mean differences or more extreme mean differences by chance if the null hypothesis is really true (i.e., if the population means are really equal) is less than 1%. We reject the null hypothesis that the population means of attractiveness are equal. For our example, this provides evidence to suggest that the number of stat labs differs, on average, across the levels of attractiveness, when controlling for time of day.
F = = = 21.350 MSA 246.198
11.531MSwith
(continued)
402 An Introduction to Statistical Concepts
Table 13.8 (continued)
Selected�SPSS�Results�for�the�Statistics�Lab�Example
Dependent Variable: Number of Statistics Labs Attended 1. Grand Mean
95% Confidence Interval
Mean Std. Error Lower Bound Upper Bound
18.406 .600 17.167 19.645
2. Level of Attractiveness Dependent Variable: Number of Statistics Labs Attended
95% Confidence Interval
Level of Attractiveness Mean Std. Error Lower Bound Upper Bound
Unattractive
Slightly attractive
Moderately attractive
Very attractive
11.125
17.875
20.250
24.375
1.201
1.201
1.201
1.201
8.647
15.397
17.772
21.897
13.603
20.353
22.728
26.853
3. Time of Day Dependent Variable: Number of Statistics Labs Attended
95% Confidence Interval
Time of Day Mean Std. Error Lower Bound Upper Bound
Afternoon
Evening
23.125
13.688
.849
.849
21.373
11.935
24.877
15.440
The “Grand Mean” (in this case, 18.406) represents the overall mean, regardless of
group membership, on the dependent variable. The 95% CI represents the CI of
the grand mean.
The table labeled “Level of attractiveness”
provides descriptive statistics for each of the
categories of the first independent variable. In addition to means, the SE and 95% CI of the means
are reported.
The table labeled “Time of day” provides
descriptive statistics for each of the categories of the second independent variable. In addition to means, the SE and 95%
CI of the means are reported.
4. Level of Attractiveness * Time of Day Dependent Variable: Number of Statistics Labs Attended
95% Confidence Interval
Level of Attractiveness Time of Day Mean Std. Error Lower Bound
Upper Bound
AfternoonUnattractive
Evening
AfternoonSlightly attractive
Evening
AfternoonModerately attractive
Evening
AfternoonVery attractive
Evening
15.250
7.000
22.750
13.000
26.250
14.250
28.250
20.500
1.698
1.698
1.698
1.698
1.698
1.698
1.698
1.698
11.746
3.496
19.246
9.496
22.746
10.746
24.746
16.996
18.754
10.504
26.254
16.504
29.754
17.754
31.754
24.004
The table labeled “Level of attractiveness * Time of day” provides
descriptive statistics for each of the categories of
the first independent variable by the second independent variable
(i.e., cell means) (notice that these are the same
means reported previously). In addition
to means, the SE and 95% CI of the means are
reported.
403Factorial Analysis of Variance: Fixed-Effects Model
Table 13.8 (continued)
Selected�SPSS�Results�for�the�Statistics�Lab�Example
Number of Statistics Labs Attended Tukey HSD
95% Confidence Interval (I) Level of Attractiveness
(J) Level of Attractiveness
Mean Difference
(I – J) Std. Error
Sig.
Lower Bound Upper Bound
Slightly attractive –6.7500* 1.69788 .003 –11.4338 –2.0662
Moderately attractive –9.1250* 1.69788 .000 –13.8088 –4.4412
Unattractive
Very attractive –13.2500* 1.69788 .000 –17.9338 –8.5662
Unattractive 6.7500* 1.69788 .003 2.0662 11.4338
Moderately attractive –2.3750 1.69788 .512 –7.0588 2.3088
Slightly attractive
Very attractive –6.5000* 1.69788 .004 –11.1838 –1.8162
Unattractive 9.1250* 1.69788 .000 4.4412 13.8088
Slightly attractive 2.3750 1.69788 .512 –2.3088 7.0588
Moderately attractive
Very attractive –4.1250 1.69788 .098 –8.8088 .5588
Unattractive 13.2500* 1.69788 .000 8.5662 17.9338
Slightly attractive 6.5000* 1.69788 .004 1.8162 11.1838
Very attractive
Moderately attractive 4.1250 1.69788 .098 –.5588 8.8088
Note: Based on observed means. The error term is mean square(error) = 11.531. * The mean difference is significant at the .05 level.
“Mean difference” is simply the difference between the means of the two levels of
attractiveness being compared. For example, the mean difference of level 1 and level 2 is calculated
as11.1250 –17.8750 = –6.7500.
The standard error calculated in SPSS uses the harmonic mean (Tukey–Kramer modification):
“Sig.” denotes the observed p values and provides the results of the contrasts. There are four statistically significant mean differences between: (1) group 1 (unattractive) and group 2 (slightly attractive); (2) group 1 (unattractive) and group 3 (moderately attractive); (3) group 1 (unattractive) and group 4 (very attractive); and (4) group 2 (slightly attractive) and 4 (very attractive).
½[ J (J – 1)] = ½[4(4 – 1)] = ½(12) = 6. Thus there are redundant results presented in the table. For example, the comparison of group 1 and 2 (presented in results row 1) is the same as the comparison of group 2 and 1 (presented in results row 2).
SΨ΄ =
SΨ΄ =
SΨ΄ =
MSerror 1
11.531
2.88275 = 1.69788
1 + 2 n1
1 n2
1 + 2
1 8
1 8
+ 1 8
+ 1 8
Multiple Comparisons
Note that there are only six unique contrast results:
(continued)
404 An Introduction to Statistical Concepts
Table 13.8 (continued)
Selected�SPSS�Results�for�the�Statistics�Lab�Example
Number of Statistics Labs AttendedTukey HSDab
SubsetLevel of Attractiveness N 1 2 3
Unattractive
8 11.1250 Slightly attractive 8 1 7.8750 Moderately attractive
8 20.2500
Very attractive 8 24.3750 Sig. 1.000 .512 .098
Means for groups in homogeneous subsets are displayed.
Note: Based on observed means. �e error term is mean square(error) = 11.531. a Uses Harmonic Mean Sample Size = 8.000. b Alpha = .05.
�is table displays the means for the groups that are not statistically
significantly different. For example, in subset 2 the means for group 2 (slightly attractive) and group 3
(moderately attractive) are displayed, indicating that those group means
are “homogeneous”or not significantly different.
20.2500
Spread vs. level plots are plots of the dependent variable standard deviations (or variances) against the cell means. �ese plots can be used to determine what to do when the homogeneity of variance assumption has been violated (remember, we already have evidence of meeting the homogeneity of variance assumption). In addition to Levene’s test, homogeneity is suggested when the spread vs. level plots provide a random display of points (i.e., no systematic pattern). If the plot suggests a linear relationship between the standard deviation and mean, transforming the data by taking the log of the dependent variable values may be a solution to the heterogeneity (since the calculation of logarithms requires positive values, this assumes all the data values are positive). If there is a linear relationship between the variance and mean, transforming the data by taking the square root of the dependent variable values may be a solution to the heterogeneity (since the calculation of square roots requires positive values, this assumes all the data values are positive).
5.00 10.00 15.00 20.00 Level (mean)
Group: Group * Time
25.00 30.00
5
4
3
Sp re
ad (s
ta nd
ar d
de vi
at io
n)
2
1
Spread vs. level plot of number of statistics labs attended
25
20
15
Sp re
ad (v
ar ia
nc e)
10
5
0
5.00 10.00 15.00 20.00 25.00 30.00 Level (mean)
Groups: Group * Time
Spread vs. level plot of number on statistics labs attended
405Factorial Analysis of Variance: Fixed-Effects Model
Examining Data for Assumptions
Normality
We� will� use� the� residuals� (which� were� requested� and� created� through� the� “Save”� option�when�generating�our�factorial�ANOVA)�to�examine�the�extent�to�which�normal- ity�was�met�
�e residuals are computed by substracting the cell mean from the dependent variable value for each observation. For example, the cell mean for time 1 group 1 was 15.25. �us the residual for the �rst person is: (15 – 15.25 = –.25). As we look at our raw data, we see a new variable has been added to our dataset labeled RES_1. �is is our residual. �e residual will be used to review the assumptions of normality and independence.
Generating normality evidence:�As�alluded�to�earlier�in�the�chapter,�understand- ing� the� distributional� shape,� specifically� the� extent� to� which� normality� is� a� reasonable� assumption,�is�important��For�factorial�ANOVA,�the�distributional�shape�for�the�residuals� should�be�a�normal�distribution��We�can�again�use�“Explore”�to�examine�the�extent�to� which�the�assumption�of�normality�is�met�
The�general�steps�for�accessing�“Explore”�have�been�presented�in�previous�chapters,� and�will�not�be�repeated�here��Click�the�residual�and�move�it�into�the�“Dependent List”� box�by�clicking�on�the�arrow�button��The�procedures�for�selecting�normality�statistics�were� presented�in�Chapter�6,�and�remain�the�same�here:�Click�on�“Plots”�in�the�upper�right� corner��Place�a�checkmark�in�the�boxes�for�“Normality plots with tests”�and�also� for�“Histogram.”� Then�click�“Continue”�to�return�to�the�main�“Explore”�dialog�box�� Then�click�“OK”�to�generate�the�output�
406 An Introduction to Statistical Concepts
Select residuals from the list on the left and use the arrow to move to the “Dependent List” box on the right. Then click on “Plots.”
Generating normality
evidence
Interpreting normality evidence:� We� have� already� developed� a� good� under- standing�of�how�to�interpret�some�forms�of�evidence�of�normality�including�skewness�and� kurtosis,�histograms,�and�boxplots�
Mean
for mean 5% Trimmed mean Median Variance Std. deviation Minimum Maximum Range Interquartile range Skewness Kurtosis
95% Con�dence interval Residual for labs
Descriptives
Lower bound Upper bound
Statistic
.52819.0000 –1.0772
1.0772 –.0747 –.2500 8.927 2.98788
–5.50 6.75
12.25 3.94 .400
–.162 .809 .414
Std. Error
The� skewness� statistic� of� the� residuals� is� �400� and� kurtosis� is� −�162—both� within� the� range�of�an�absolute�value�of�2�0,�suggesting�some�evidence�of�normality�
407Factorial Analysis of Variance: Fixed-Effects Model
As�suggested�by�the�skewness�statistic,�the�histogram�of�residuals�is�slightly�positively� skewed,�but�it�approaches�a�normal�distribution�and�there�is�nothing�to�suggest�normality� may�be�an�unreasonable�assumption�
8
6
4
2
0 –5.00 –2.50 .00 2.50
Residual for labs
5.00 7.50
Fr eq
ue nc
y Histogram
Mean = –3.33E-16 Std. dev. = 2.988 N = 32
There�are�a�few�other�statistics�that�can�be�used�to�gauge�normality��The�formal�test�of� normality,�the�Shapiro–Wilk�(S–W)�test�(SW)�(Shapiro�&�Wilk,�1965),�provides�evidence�of� the�extent�to�which�our�sample�distribution�is�statistically�different�from�a�normal�distri- bution��The�output�for�the�S–W�test�is�presented�as�follows�and�suggests�that�our�sample� distribution� for� residuals� is� not� statistically� significantly� different� than� what� would� be� expected�from�a�normal�distribution�(SW�=��977,�df�=�32,�p�=��701)�
Residual for labs Statistic Statisticdf dfSig. Sig.
Tests of Normality
Shapiro–WilkKolmogorov–Smirnova
.094 32 32 .701.200 .977
a Lilliefors significance correction. *This is a lower bound of the true significance.
Quantile–quantile� (Q–Q)� plots� are� also� often� examined� to� determine� evidence� of� nor- mality�� Q–Q� plots� are� graphs� that� plot� quantiles� of� the� theoretical� normal� distribution�
408 An Introduction to Statistical Concepts
against� quantiles� of� the� sample� distribution�� Points� that� fall� on� or� close� to� the� diagonal� line�suggest�evidence�of�normality��The�Q–Q�plot�of�residuals�shown�as�follows�suggests� relative�normality�
3
2
1
0
–1Ex pe
ct ed
n or
m al
–2
–3
–6 –3 0 Observed value
3 6
Normal Q–Q plot of residual for labs
Examination�of�the�following�boxplot�suggests�a�relatively�normal�distributional�shape� of�residuals�and�no�outliers�
7.50
5.00
2.50
.00
–2.50
–5.00
Residual for labs
Considering�the�forms�of�evidence�we�have�examined,�skewness�and�kurtosis�statistics,� the�S–W�test,�the�Q–Q�plot,�and�the�boxplot,�all�suggest�normality�is�a�reasonable�assump- tion��We�can�be�reasonably�assured�that�we�have�met�the�assumption�of�normality�of�the� dependent�variable�for�each�group�of�the�independent�variable�
409Factorial Analysis of Variance: Fixed-Effects Model
Independence
The�only�assumption�we�have�not�tested�for�yet�is�independence��As�we�discussed�in�ref- erence� to� the� one-way� ANOVA,� if� subjects� have� been� randomly� assigned� to� conditions� (or� to� the� different� combinations� of� the� levels� of� the� independent� variables� in� a� factorial� ANOVA),� the� assumption� of� independence� has� been� met�� In� this� illustration,� students� were�randomly�assigned�to�instructor�and�time�of�day,�and,�thus,�the�assumption�of�inde- pendence�was�met��However,�we�often�use�independent�variables�that�do�not�allow�ran- dom�assignment,�such�as�preexisting�characteristics�such�as�education�level�(high�school� diploma,�bachelor’s,�master’s,�or�doctoral�degrees)��We�can�plot�residuals�against�levels�of� our�independent�variables�in�a�scatterplot�to�get�an�idea�of�whether�or�not�there�are�pat- terns�in�the�data�and�thereby�provide�an�indication�of�whether�we�have�met�this�assump- tion�� Given� we� have� multiple� independent� variables� in� the� factorial� ANOVA,� we� will� split�the�scatterplot�by�levels�of�one�independent�variable�(“Group”)�and�then�generate�a� bivariate�scatterplot�for�“Time”�by�residual��Remember�that�the�residual�was�added�to�the� dataset�by�saving�it�when�we�generated�the�factorial�ANOVA�model�
Please�note�that�some�researchers�do�not�believe�that�the�assumption�of�independence� can�be�tested��If�there�is�not�random�assignment�to�groups,�then�these�researchers�believe� this�assumption�has�been�violated—period��The�plot�that�we�generate�will�give�us�a�gen- eral�idea�of�patterns,�however,�in�situations�where�random�assignment�was�not�performed� or�not�possible�
Splitting the file:�The�first�step�is�to�split�our�file�by�the�levels�of�one�of�our�inde- pendent�variables�(e�g�,�“Group”)��To�do�that,�go�to�“Data”�in�the�top�pulldown�menu�and� then�select�“Split File.”
A
B
Generating independence evidence: Step 1
410 An Introduction to Statistical Concepts
Select independent variable from the list
on the left and use the arrow to move to the “Group Based on” box on the right. Then click on “Ok.”
Generating independence evidence: Step 2
Generating the scatterplot:� The� general� steps� for� generating� a� simple� scatterplot� through�“Scatter/dot”�have�been�presented�in�a�previous�chapter�(e�g�,�Chapter�10),�and� they�will�not�be�reiterated�here��From�the�“Simple Scatterplot”�dialog�screen,�click�the� residual�variable�and�move�it�into�the�“Y Axis”�box�by�clicking�on�the�arrow��Click�the�inde- pendent�variable�that�was�not�used�to�split�the�file�(e�g�,�“Time”)�and�move�it�into�the�“X Axis”� box�by�clicking�on�the�arrow��Then�click�“OK.”
411Factorial Analysis of Variance: Fixed-Effects Model
Interpreting independence evidence:�In�examining�the�scatterplots�for�evidence� of�independence,�the�points�should�fall�relatively�randomly�above�and�below�a�horizontal� line�at�0��(You�may�recall�in�Chapter�11�that�we�added�a�reference�line�to�the�graph�using� Chart�Editor��To�add�a�reference�line,�double�click�on�the�graph�in�the�output�to�activate� the� chart� editor�� Select�“Options”� in� the� top� pulldown� menu,� then�“Y axis refer- ence line.”� This� will� bring� up� the “Properties”� dialog� box�� Change� the� value� of� the�position�to�be�“0�”�Then�click�on�“Apply”�and “Close”�to�generate�the�graph�with�a� horizontal�line�at�0�)
In�this�example,�our�scatterplot�for�each�level�of�attractiveness�generally�suggests�evi- dence� of� independence� with� a� relatively� random� display� of� residuals� above� and� below� the�horizontal�line�at�0�for�each�category�of�time��Thus,�had�we�not�met�the�assumption�of� independence�through�random�assignment�of�cases�to�groups,�this�would�have�provided� evidence�that�independence�was�a�reasonable�assumption�
8.00
6.00
4.00
2.00
.00
–2.00
–4.00
Re si
du al
fo r l
ab s
1.00 1.20 1.40 1.60 Time of day
1.80 2.00
Level of attractiveness: Unattractive
6.00
4.00
2.00
.00
–2.00
–4.00
Re si
du al
fo r l
ab s
1.00 1.20 1.40 1.60
Time of day
1.80 2.00
Level of attractiveness: Slightly attractive
412 An Introduction to Statistical Concepts
8.00
6.00
4.00
2.00
.00
–2.00
–4.00
Re si
du al
fo r l
ab s
1.00 1.20 1.40 1.60
Time of day
1.80 2.00
Level of attractiveness: Moderately attractive
1.00 1.20 1.40 1.60
Time of day
1.80 2.00
–5.00
–2.50
.00
Re si
du al
fo r l
ab s 2.50
5.00
Level of attractiveness: Very attractive
413Factorial Analysis of Variance: Fixed-Effects Model
Post Hoc Power for Factorial ANOVA Using G*Power
Main effects:� When� there� are� multiple� independent� variables,� G*Power� must� be� cal- culated�for�each�main�effect�and�for�each�interaction��We�will�illustrate�the�main�effect�for� attractiveness� of� instructor,� but� note� that� computing� post� hoc� power� for� the� other� main� effect(s)�and�interaction(s)�is�similarly�obtained�
The�first�thing�that�must�be�done�when�using�G*Power�for�computing�post�hoc�power�is� to�select�the�correct�test�family��In�our�case,�we�conducted�a�factorial�ANOVA��To�find�the� factorial�ANOVA,�we�select�“Tests”�in�the�top�pulldown�menu,�then�“Means,”�and�then� “Many groups: ANOVA: Main effects and interactions (two or more inde- pendent variables).”�Once�that�selection�is�made,�the�“Test family”�automatically� changes�to�“F tests.”
A
B
C
Step 1
The�“Type of Power Analysis”�desired�then�needs�to�be�selected��To�compute�post� hoc� power,� we� need� to� select�“Post hoc: Compute achieved power—given α, sample size, and effect size.”
414 An Introduction to Statistical Concepts
The default selection for “Statistical Test” is “Correlation: Point biserial model.” Following the procedures presented in Step 1 will automatically change the statistical test to “ANOVA: Fixed effects, special, main effects and interactions” (two or more independent variables).
The default selection for “Test Family”
is “t tests.” Following the
procedures presented in Step 1 will automatically
change the test family to “F test.”
Click on “Determine” to pop out the effect size calculator box
(shown below). �is will allow you to
compute f given partial eta squared.
Once the parameters are
specified, click on “Calculate.”
The “Input Parameters” for computing post hoc power must be specified (the default
values are shown here) including:
Step 2
1. Effect size f 2. Alpha level 3. Total sample size 4. Numerator df 5. Number of groups
The�“Input Parameters”�must�then�be�specified��We�compute�the�effect�size�f�last,�so� skip� that� for� the� moment�� In� our� example,� the� alpha� level� we� used� was� �05,� and� the� total� sample�size�was�32��The�numerator df�for�attractiveness�(recall�that�we�are�computing�post�hoc� power�for�the�main�effect�of�attractiveness�here)�is�equal�to�the�number�of�categories�of�this� variable�(i�e�,�4)�minus�1;�thus,�there�are�three�degrees�of�freedom�for�attractiveness��The�num- ber of groups�is�equal�to�the�product�of�the�number�of�levels�or�categories�of�the�independent� variables�or�(J)(K)��In�this�example,�the�number�of�groups�or�cells�then�equals�(J)(K)�=�(4)(2)�=�8�
We� skipped� filling� in� the� first� parameter,� the� effect� size� f,� for� a� reason�� SPSS� only� provided� a� partial� eta� squared� effect� size�� Thus,� we� will� use� the� pop-out� effect� size� calculator�in�G*Power�to�compute�the�effect�size�f�(we�saved�this�parameter�for�last�as� the�calculation�is�based�on�the�previous�values�just�entered)��To�pop�out�the�effect�size� calculator,�click�on�“Determine”�which�is�displayed�under�“Input Parameters.”�In� the�pop-out�effect�size�calculator,�click�on�the�radio�button�for�“Direct”�and�then�enter� the� partial� eta� squared� value� for� attractiveness� that� was� calculated� in� SPSS� (i�e�,� �842)�� Clicking�on�“Calculate”�in�the�pop-out�effect�size�calculator�will�calculate�the�effect� size�f��Then�click�on�“Calculate and Transfer to Main Window”�to�transfer�the� calculated� effect� size� (i�e�,� 2�3084874)� to� the�“Input Parameters.”� Once� the� param- eters�are�specified,�click�on�“Calculate”�to�find�the�power�statistics�
415Factorial Analysis of Variance: Fixed-Effects Model
Post hoc power
Here are the post hoc power results.
The�“Output Parameters”�provide�the�relevant�statistics�given�the�input�just�speci- fied��In�this�example,�we�were�interested�in�determining�post�hoc�power�for�a�two-factor� ANOVA�with�a�computed�effect�size�f�of�2�308,�an�alpha�level�of��05,�total�sample�size�of�32,� numerator�degrees�of�freedom�of�3,�and�8�groups�or�cells�
Based�on�those�criteria,�the�post�hoc�power�for�the�main�effect�of�attractiveness�was�1�00�� In�other�words,�with�a�factorial�ANOVA,�computed�effect�size�f�of�2�308,�alpha�level�of��05,� total�sample�size�of�32,�numerator�degrees�of�freedom�of�3,�and�8�groups�(or�cells),�the�post� hoc� power� of� our� main� effect� was� 1�00—the� probability� of� rejecting� the� null� hypothesis� when�it�is�really�false�(in�this�case,�the�probability�that�the�means�of�the�dependent�vari- able�would�be�equal�for�each�level�of�the�independent�variable)�was�1�00,�which�would�be� considered�maximum�power�(sufficient�power�is�often��80�or�above)��Note�that�this�value�is� the�same�as�that�reported�in�SPSS��Keep�in�mind�that�conducting�power�analysis�a�priori� is�recommended�so�that�you�avoid�a�situation�where,�post�hoc,�you�find�that�the�sample� size�was�not�sufficient�to�reach�the�desired�level�of�power�(given�the�observed�parameters)�
416 An Introduction to Statistical Concepts
Interactions:� Calculation� of� power� for� interactions� is� conducted� similarly�� The� input� of� �075� for� partial� eta� squared� results� in� the� following� output� for� interaction� power�� The� post�hoc�power�of�the�interaction�effect�for�this�test�was��204—the�probability�of�rejecting� the�null�hypothesis�when�it�is�really�false�(in�this�case,�the�probability�that�the�means� of�the�dependent�variable�would�be�equal�for�each�cell)�was�about�20%,�which�would�be� considered�very�low�power�(sufficient�power�is�often��80�or�above)��Note�that�this�value�is� not�the�same�as�that�reported�in�SPSS�
Here are the post hoc power results for the attractiveness by time
of day interaction.
Post hoc power: Interaction
A Priori Power for Factorial ANOVA Using G*Power
For�a�priori�power,�we�can�determine�the�total�sample�size�needed�for�the�main�effects�and/ or�interactions�given�an�estimated�effect�size�f,�alpha�level,�desired�power,�numerator�degrees� of�freedom�(i�e�,�number�of�categories�of�our�independent�variable�or�interaction,�depending� on� which� a� priori� power� is� of� interest),� and� number� of� groups� or� cells� (i�e�,� the� product� of� the�number�of�levels�of�the�independent�variables)��We�follow�Cohen’s�(1988)�conventions�for� effect�size�(i�e�,�small,�f�=��10;�moderate,�f�=��25;�large,�f�=��40)��In�this�example,�had�we�estimated� a�moderate�effect�f�of��25,�alpha�of��05,�desired�power�of��80,�numerator�degrees�of�freedom�of� 3�(four�groups�in�attractiveness,�two�levels�of�time�of�day,�thus�4�−�1�×�2�−�1�=�3),�and�number� of�groups�of�8�(i�e�,�four�categories�of�attractiveness�and�two�levels�in�time�of�day�or�4�×�2�=�8),� we�would�need�a�total�sample�size�of�179�(or�about�22�or�23�individuals�per�cell)�
417Factorial Analysis of Variance: Fixed-Effects Model
A priori power:
Interaction
Here are the a priori power
results.
13.5 Template and APA-Style Write-Up
Finally� we� come� to� an� example� paragraph� of� the� results� for� the� two-factor� statistics� lab� example�� Recall� that� our� graduate� research� assistant,� Marie,� was� working� on� a� research� project� for� an� independent� study� class� to� determine� if� there� was� a� mean� difference� in� the� number� of� statistics� labs� attended� based� on� the� attractiveness� of� the� lab� instructor� (four�categories)�and�time�of�day�the�lab�was�attended�(afternoon�or�evening)��Her�research� question�was�the�following:�Is there a mean difference in the number of statistics labs students attended based on the attractiveness of the lab instructor and time of day the lab was attended?� Marie�then�generated�a�factorial�ANOVA�as�the�test�of�inference��A�template�for�writing�a� research�question�for�a�factorial�ANOVA�is�presented�as�follows;
418 An Introduction to Statistical Concepts
Is there a mean difference in [dependent variable] based on [inde- pendent variable 1] and [independent variable 2]?
This�is�illustrated�assuming�a�two-factor�model,�but�it�can�easily�be�extended�to�more�than�two� factors��As�we�noted�in�Chapter�11,�it�is�important�to�ensure�the�reader�understands�the�levels� or�groups�of�the�independent�variables��This�may�be�done�parenthetically�in�the�actual�research� question,�as�an�operational�definition,�or�specified�within�the�methods�section��In�this�example,� parenthetically�we�could�have�stated�the�following:�Is there a mean difference in the number of statis- tics labs students attend based on the attractiveness of the lab instructor (unattractive, slightly attractive, moderately attractive, very attractive) and time of day the lab was attended (afternoon or evening)?
It�may�be�helpful�to�preface�the�results�of�the�factorial�ANOVA�with�information�on� an�examination�of�the�extent�to�which�the�assumptions�were�met�(recall�there�are�three� assumptions:�normality,�homogeneity�of�variance,�and�independence)��This�assists�the� reader�in�understanding�that�you�were�thorough�in�data�screening�prior�to�conducting� the�test�of�inference:
A factorial ANOVA was conducted to determine if the mean number of statistics labs attended by students differed based on the level of attractiveness of the statistics lab instructor (unattractive, slightly attractive, moderately attractive, very attractive) and the time of day the lab was attended (afternoon or evening). The assumption of normality was tested and met via examination of the residuals. Review of the S–W test for normality (SW = .977, df = 32, p = .701) and skew- ness (.400) and kurtosis (−.162) statistics suggested that normal- ity was a reasonable assumption. The boxplot suggested a relatively normal distributional shape (with no outliers) of the residuals. The Q–Q plot and histogram suggested normality was reasonable. According to Levene’s test, the homogeneity of variance assumption was satis- fied [F(7, 24) = .579, p = .766]. Random assignment of individuals to groups helped ensure that the assumption of independence was met. Additionally, scatterplots of residuals against the levels of the inde- pendent variables were reviewed. A random display of points around 0 provided further evidence that the assumption of independence was met.
Here� is� an� APA-style� example� paragraph� of� results� for� the� factorial� ANOVA� (remember� that� this� will� be� prefaced� by� the� previous� paragraph� reporting� the� extent� to� which� the� assumptions�of�the�test�were�met):
From Table 13.8, we see that the interaction of attractiveness by time of day is not statistically significant, but there are statistically significant main effects for both attractiveness and time of day (Fattract = 21.350, df = 3, 24, p = .001; Ftime = 61.791, df = 1, 24, p = .001). Effect sizes are large for both attractiveness and time (partial η2attract = .727; partial η2time = .720), and observed power for attractiveness and time is maximal (i.e., 1.000).
Post hoc analyses were conducted given the statistically significant omnibus ANOVA F tests. The profile plot (Figure 13.2) summarizes these differences. Tukey HSD tests were conducted on all possible
419Factorial Analysis of Variance: Fixed-Effects Model
pairwise contrasts. For the main effect of attractiveness, Tukey HSD post hoc comparisons revealed that the unattractive level had sta- tistically significantly lower attendance than all the other levels of attractiveness and that the slightly attractive level had statisti- cally significantly lower attendance than the very attractive level. More specifically, the following pairs of groups were found to be significantly different (p < .05):
•� Groups 1 (unattractive; M = 11.125, SD = 5.4886) and 2 (slightly attractive; M = 17.875, SD = 5.9387)
•� Groups 1 (unattractive) and 3 (moderately attractive; M = 20.2500, SD = 7.2850)
•� Groups 1 (unattractive) and 4 (very attractive; M = 24.3750, SD = 5.0973)
•� Groups 2 (slightly attractive) and 4 (very attractive)
In other words, students enrolled in the least attractive instructor group attended statistically significantly fewer statistics labs than students enrolled in any of the three more attractive instructor groups. For the main effect of time of day, Tukey HSD post hoc comparisons
revealed that the students enrolled in the afternoon (M = 23.125, SD = 5.655) had statistically significantly higher statistics lab atten- dance than students in the evening (M = 13.688, SD = 6.096).
13.6 Summary
This�chapter�considered�methods�involving�the�comparison�of�means�for�multiple�inde- pendent� variables�� The� chapter� began� with� a� look� at� the� characteristics� of� the� factorial� ANOVA,�including�(a)�two�or�more�independent�variables�each�with�two�or�more�fixed� levels;�(b)�subjects�are�randomly�assigned�to�cells�and�then�exposed�to�only�one�combina- tion�of�the�independent�variables;�(c)�the�factors�are�fully�crossed�such�that�all�possible� combinations� of� the� factors’� levels� are� included� in� the� design;� and� (d)� the� dependent� variable�measured�at�the�interval�level�or�better��The�ANOVA�model�was�examined�and� followed�by�a�discussion�of�main�effects�and,�in�particular,�the�interaction�effect��Some� discussion�was�also�devoted�to�the�ANOVA�assumptions��The�ANOVA�summary�table� was� shown� along� with� partitioning� the� sums� of� squares�� MCPs� were� then� extended� to� factorial� models�� Then� effect� size� measures,� CIs,� power,� and� expected� mean� squares� were� considered�� Finally,� several� approaches� were� given� for� the� unequal� n’s� case� with� factorial�models��At�this�point,�you�should�have�met�the�following�objectives:�(a)�be�able� to�understand�the�characteristics�and�concepts�underlying�factorial�ANOVA,�(b)�be�able�to� determine� and� interpret� the� results� of� factorial� ANOVA,� and� (c)� be� able� to� understand� and�evaluate�the�assumptions�of�factorial�ANOVA��In�Chapter�14,�we�introduce�the�analysis� of�covariance�
420 An Introduction to Statistical Concepts
Problems
Conceptual problems
13.1� �You�are�given�a�two-factor�design�with�the�following�cell�means�(cell�11�=�25;�cell�12�=�75;� cell�21�=�50;�cell�22�=�50;�cell�31�=�75;�cell�32�=�25)��Assume�that�the�within-cell�variation�is� small��Which�one�of�the�following�conclusions�seems�most�probable?
� a�� The�row�means�are�significantly�different� � b�� The�column�means�are�significantly�different� � c�� The�interaction�is�significant� � d�� All�of�the�above� 13.2� �In�a�two-factor�ANOVA,�one�independent�variable�has�five�levels�and�the�second�has�
four�levels��If�each�cell�has�seven�observations,�what�is�dfwith? � a�� 20 � b�� 120 � c�� 139 � d�� 140 13.3� �In� a� two-factor� ANOVA,� one� independent� variable� has� three� levels� or� categories�
and�the�second�has�three�levels�or�categories��What�is�dfAB,�the�interaction�degrees�of� freedom?
� a�� 3 � b�� 4 � c�� 6 � d�� 9 13.4� �Which�of�the�following�conclusions�would�result�in�the�greatest�generalizability�of�
the�main�effect�for�factor�A�across�the�levels�of�factor�B?�The�interaction�between�the� independent�variables�A�and�B�was�…
� a�� Not�significant�at�the��25�level � b�� Significant�at�the��10�level � c�� Significant�at�the��05�level � d�� Significant�at�the��01�level � e�� Significant�at�the��001�level 13.5� �In�a�two-factor�fixed-effects�ANOVA�tested�at�an�alpha�of��05,�the�following�p�values�
were� found:� main� effect� for� factor� A,� p� =� �06;� main� effect� for� factor� B,� p� =� �09;� and� interaction�AB,�p�=��02��What�can�be�interpreted�from�these�results?
� a�� There�is�a�statistically�significant�main�effect�for�factor�A�
� b�� There�is�a�statistically�significant�main�effect�for�factor�B�
� c�� There�is�a�statistically�significant�main�effect�for�factors�A�and�B�
� d�� There�is�a�statistically�significant�interaction�effect�
421Factorial Analysis of Variance: Fixed-Effects Model
13.6� �In�a�two-factor�fixed-effects�ANOVA,�FA�=�2,�dfA�=�3,�dfB�=�6,�dfAB�=�18,�and�dfwith�=�56�� The�null�hypothesis�for�factor�A�can�be�rejected
� a�� At�the��01�level
� b�� At�the��05�level,�but�not�at�the��01�level
� c�� At�the��10�level,�but�not�at�the��05�level
� d�� None�of�the�above
13.7� In�ANOVA,�the�interaction�of�two�factors�is�certainly�present�when
� a�� The�two�factors�are�positively�correlated�
� b�� The�two�factors�are�negatively�correlated�
� c�� Row�effects�are�not�consistent�across�columns�
� d�� Main�effects�do�not�account�for�all�of�the�variation�in�Y�
� e�� Main�effects�do�account�for�all�of�the�variation�in�Y�
13.8� For�a�design�with�four�factors,�how�many�interactions�will�there�be?
� a�� 4
� b�� 8
� c�� 11
� d�� 12
� e�� 16
13.9� �Degrees� of� freedom� for� the� AB� interaction� are� equal� to� which� one� of� the� following?
� a�� dfA�−�dfB � b�� (dfA)(dfB)
� c�� dfwith�−�(dfA�+�dfB)
� d�� dftotal�−�dfwith 13.10� �A�two-factor�experiment�means�that�the�design�necessarily�includes�which�one�of�
the�following?
� a�� Two�independent�variables
� b�� Two�dependent�variables
� c�� An�interaction�between�the�independent�and�dependent�variables
� d�� Exactly�two�separate�groups�of�subjects 13.11� Two�independent�variables�are�said�to�interact�when�which�one�of�the�following�occurs? � a�� Both�variables�are�equally�influenced�by�a�third�variable� � b�� These�variables�are�differentially�affected�by�a�third�variable� � c�� Each�factor�produces�a�change�in�the�subjects’�scores� � d�� The�effect�of�one�variable�depends�on�the�second�variable� 13.12� �If� there� is� an� interaction� between� the� independent� variables� textbook� and� time� of�
day,�this�means�that�the�textbook�used�has�the�same�effect�at�different�times�of�the� day��True�or�false?
422 An Introduction to Statistical Concepts
13.13� �If�the�AB�interaction�is�significant,�then�at�least�one�of�the�two�main�effects�must�be� significant��True�or�false?
13.14� �I�assert�that�a�two-factor�experiment�(factors�A�and�B)�yields�no�more�information� than�two�one-factor�experiments�(factor�A�in�experiment�1�and�factor�B�in�experi- ment�2)��Am�I�correct?
13.15� �For�a�two-factor�fixed-effects�model,�if�the�degrees�of�freedom�for�testing�factor�A�=�2,�24,� then�I�assert�that�the�degrees�of�freedom�for�testing�factor�B�will�necessarily�be�=�2,�24�� Am�I�correct?
� �Questions�13�16�through�13�18�are�based�on�the�following�ANOVA�summary�table� (fixed-effects):
Source df MS F
A 2 45 4�5 B 1 70 7�0 AB 2 170 17�0 Within 60 10
13.16� �For� which� source� of� variation� is� the� null� hypothesis� rejected� at� the� �01� level� of� significance?
� a�� A � b�� B � c�� AB � d�� All�of�the�above 13.17� How�many�cells�are�there�in�the�design? � a�� 1 � b�� 2 � c�� 3 � d�� 5 � e�� None�of�the�above 13.18� The�total�sample�size�for�the�design�is�which�one�of�the�following? � a�� 66 � b�� 68 � c�� 70 � d�� None�of�the�above � �Questions�13�19�through�13�21�are�based�on�the�following�ANOVA�summary�table�
(fixed�effects):
Source df MS F
A 2 164 5�8 B 1 80 2�8 AB 2 68 2�4 Within 9 28
423Factorial Analysis of Variance: Fixed-Effects Model
13.19� �For� which� source� of� variation� is� the� null� hypothesis� rejected� at� the� �01� level� of� significance?
� a�� A � b�� B � c�� AB � d�� All�of�the�above 13.20� How�many�cells�are�there�in�the�design? � a�� 1 � b�� 2 � c�� 3 � d�� 6 � e�� None�of�the�above 13.21� The�total�sample�size�for�the�design�is�which�one�of�the�following? � a�� 10 � b�� 15 � c�� 20 � d�� 25
Computational problems
13.1� �Complete� the� following� ANOVA� summary� table� for� a� two-factor� fixed-effects� ANOVA,�where�there�are�two�levels�of�factor�A�(drug)�and�three�levels�of�factor�B� (dosage)��Each�cell�includes�26�students�and�α�=��05�
Source SS df MS F Critical Value Decision
A 6�15 — — — — — B 10�60 — — — — — AB 9�10 — — — — — Within — — — Total 250�85 —
13.2� �Complete� the� following� ANOVA� summary� table� for� a� two-factor� fixed-effects� ANOVA,�where�there�are�three�levels�of�factor�A�(program)�and�two�levels�of�factor� B�(gender)��Each�cell�includes�four�students�and�α�=��01�
Source SS df MS F Critical Value Decision
A 3�64 — — — — — B �57 — — — — — AB 2�07 — — — — — Within — — — Total 8�18 —
424 An Introduction to Statistical Concepts
13.3� Complete� the� following� ANOVA� summary� table� for� a� two-factor� fixed-effects� ANOVA,� where� there� are� two� levels� of� factor� A� (undergraduate� vs�� graduate)� and� two�levels�of�factor�B�(gender)��Each�cell�includes�four�students�and�α�=��05�
Source SS df MS F Critical Value Decision
A 14�06 — — — — — B 39�06 — — — — — AB 1�56 — — — — — Within — — — Total 723�43 —
13.4� Conduct�a�two-factor�fixed-effects�ANOVA�to�determine�if�there�are�any�effects�due� to�A�(task�type),�B�(task�difficulty),�or�the�AB�interaction�(α�=��01)��Conduct�Tukey�HSD� post�hoc�comparisons,�if�necessary��The�following�are�the�scores�from�the�individual� cells�of�the�model:
� A1B1:�41,�39,�25,�25,�37,�51,�39,�101 � A1B2:�46,�54,�97,�93,�51,�36,�29,�69 � A1B3:�113,�135,�109,�96,�47,�49,�68,�38 � A2B1:�86,�38,�45,�45,�60,�106,�106,�31 � A2B2:�74,�96,�101,�124,�48,�113,�139,�131 � A2B3:�152,�79,�135,�144,�52,�102,�166,�155 13.5� An�experimenter�is�interested�in�the�effects�of�strength�of�reinforcement�(factor�A),�
type� of� reinforcement� (factor� B),� and� sex� of� the� adult� administering� the� reinforce- ment�(factor�C)�on�children’s�behavior��Each�factor�consists�of�two�levels��Thirty-two� children�are�randomly�assigned�to�eight�cells�(i�e�,�four�per�cell),�one�for�each�of�the� factor�combinations��Using�the�scores�from�the�individual�cells�of�the�model�that�fol- low,�conduct�a�three-factor�fixed-effects�ANOVA�(α�=��05)��If�there�are�any�significant� interactions,�graph�and�interpret�the�interactions�
� A1B1C1:�3,�6,�3,�3 � A1B1C2:�4,�5,�4,�3 � A1B2C1:�7,�8,�7,�6 � A1B2C2:�7,�8,�9,�8 � A2B1C1:�1,�2,�2,�2 � A2B1C2:�2,�3,�4,�3 � A2B2C1:�5,�6,�5,�6 � A2B2C2:�10,�10,�9,�11 13.6� A�replication�study�dataset�of�the�example�from�this�chapter�is�given�as�follows�
(A�=�attractiveness,�B�=�time;�same�levels)��Using�the�scores�from�the�individual�cells� of�the�model�that�follow,�conduct�a�two-factor�fixed-effects�ANOVA�(α�=��05)��Are�the� results�different�as�compared�to�the�original�dataset?
� A1B1:�10,�8,�7,�3 � A1B2:�15,�12,�21,�13 � A2B1:�13,�9,�18,�12 � A2B2:�20,�22,�24,�25
425Factorial Analysis of Variance: Fixed-Effects Model
� A3B1:�24,�29,�27,�25 � A3B2:�10,�12,�21,�14 � A4B1:�30,�26,�29,�28 � A4B2:�22,�20,�25,�15
Interpretive problem
13.1� Building� on� the� interpretive� problem� from� Chapter� 11,� utilize� the� survey� 1� dataset� from�the�website��Use�SPSS�to�conduct�a�two-factor�fixed-effects�ANOVA,�including� effect�size,�where�political�view�is�factor�A�(as�in�Chapter�11,�J�=�5),�gender�is�factor�B� (a�new�factor,�K�=�2),�and�the�dependent�variable�is�the�same�one�you�used�previously� in�Chapter�11��Then�write�an�APA-style�paragraph�summarizing�the�results�
13.2� Building�on�the�interpretive�problem�from�Chapter�11,�use�the�survey�1�dataset�from� the�website��Use�SPSS�to�conduct�a�two-factor�fixed-effects�ANOVA,�including�effect� size,�where�hair�color�is�factor�A�(i�e�,�one�independent�variable)�(J�=�5),�gender�is�fac- tor�B�(a�new�factor,�K�=�2),�and�the�dependent�variable�is�a�variable�of�interest�to�you� (the�following�variables�look�interesting:�books,�TV,�exercise,�drinks,�GPA, GRE-Q,� CDs,�hair�appointment)��Then�write�an�APA-style�paragraph�describing�the�results�
427
14 Introduction to Analysis of Covariance: One- Factor Fixed-Effects Model With Single Covariate
Chapter Outline
14�1� Characteristics�of�the�Model 14�2� Layout�of�Data 14�3� ANCOVA�Model 14�4� ANCOVA�Summary�Table 14�5� Partitioning�the�Sums�of�Squares 14�6� Adjusted�Means�and�Related�Procedures 14�7� Assumptions�and�Violation�of�Assumptions � 14�7�1� Independence � 14�7�2� Homogeneity�of�Variance � 14�7�3� Normality � 14�7�4� Linearity � 14�7�5� Fixed�Independent�Variable � 14�7�6� Independence�of�the�Covariate�and�the�Independent�Variable � 14�7�7� Covariate�Measured�Without�Error � 14�7�8� Homogeneity�of�Regression�Slopes 14�8� Example 14�9� ANCOVA�Without�Randomization 14�10� More�Complex�ANCOVA�Models 14�11� Nonparametric�ANCOVA�Procedures 14�12� SPSS�and�G*Power 14�13� Template�and�APA-Style�Paragraph
Key Concepts
� 1�� Statistical�adjustment � 2�� Covariate � 3�� Adjusted�means � 4�� Homogeneity�of�regression�slopes � 5�� Independence�of�the�covariate�and�the�independent�variable
428 An Introduction to Statistical Concepts
We� have� now� considered� several� different� analysis� of� variance� (ANOVA)� models�� As� we� moved�through�Chapter�13,�we�saw�that�the�inclusion�of�additional�factors�helped�to�reduce� the� residual� or� uncontrolled� variation�� These� additional� factors� served� as� “experimental� design�controls”�in�that�their�inclusion�in�the�design�helped�to�reduce�the�uncontrolled�varia- tion��In�fact,�this�could�be�the�reason�an�additional�factor�is�included�in�a�factorial�design�
In�this�chapter,�a�new�type�of�variable,�known�as�a�covariate,�is�incorporated�into�the�analy- sis��Rather�than�serving�as�an�“experimental�design�control,”�the�covariate�serves�as�a�“statisti- cal�control”�where�uncontrolled�variation�is�reduced�statistically�in�the�analysis��Thus,�a�model� where�a�covariate�is�used�is�known�as�analysis of covariance�(ANCOVA)��We�are�most�con- cerned�with�the�one-factor�fixed-effects�model�here,�although�this�model�can�be�generalized� to�any�of�the�other�ANOVA�designs�considered�in�this�text��That�is,�any�of�the�ANOVA�models� discussed�in�the�text�can�also�include�a�covariate�and�thus�become�an�ANCOVA�model�
Most� of� the� concepts� used� in� this� chapter� have� already� been� covered� in� the� text�� In� addition,� new� concepts� include� statistical� adjustment,� covariate,� adjusted� means,� and� two� important� assumptions:� homogeneity� of� regression� slopes� and� independence� of� the� covariate�and�the�independent�variable��Our�objectives�are�that�by�the�end�of�this�chapter,� you�will�be�able�to�(a)�understand�the�characteristics�and�concepts�underlying�ANCOVA;� (b)�determine�and�interpret�the�results�of�ANCOVA,�including�adjusted�means�and�mul- tiple�comparison�procedures�(MCPs);�and�(c)�understand�and�evaluate�the�assumptions�of� ANCOVA�
14.1 Characteristics of the Model
For�the�past�few�chapters,�we�have�been�following�Marie,�the�educational�research�graduate� student�who,�as�part�of�her�independent�study�course,�conducted�an�experiment�to�examine� statistics�lab�attendance��She�has�examined�attendance�based�on�attractiveness�of�instructor� (Chapters�11�and�12)�and�based�on�attractiveness�and�time�of�day�(Chapter�13)��As�we�will�see� in�this�chapter,�Marie�will�be�continuing�to�examine�data�generated�from�a�different�experi- ment�of�students�enrolled�in�statistics�courses,�now�controlling�for�aptitude�
As�we�learned�in�previous�chapters,�Marie�is�enrolled�in�an�independent�study�class�� Her�previous�study�was�so�successful�that�Marie,�again�in�collaboration�with�the�sta- tistics�faculty�in�her�program,�has�designed�another�experimental�study�to�determine� if� there� was� a� mean� difference� in� statistics� quiz� performance� based� on� the� teaching� method�utilized�(traditional�lecture�method�or�innovative�instruction)��Twelve�students� were�randomly�assigned�to�two�different�sections�of�the�same�class��One�section�was� taught�using�traditional�lecture�methods,�and�the�second�was�taught�with�more�inno- vative�instruction�which�included,�for�example,�small-group�and�self-directed�instruc- tion�� Prior� to� random� assignment� to� sections,� participants� were� also� measured� on� aptitude�toward�statistics��Marie�is�now�ready�to�examine�these�data��Marie’s�research� question�is�the�following:�Is there a mean difference in statistics quiz scores based on teaching method, controlling for aptitude toward statistics?�With�one�independent�variable�and�one� covariate�for�which�to�control,�Marie�determines�that�an�ANCOVA�is�the�best�statistical� procedure�to�use�to�answer�her�question��Her�next�task�is�to�analyze�the�data�to�address� her�research�question�
429Introduction to Analysis of Covariance
In�this�section,�we�describe�the�distinguishing�characteristics�of�the�one-factor�fixed-effects� ANCOVA�model��However,�before�we�begin�an�extended�discussion�of�these�characteris- tics,�consider�the�following�example�(a�situation�similar�to�which�we�find�Marie)��Imagine� a�situation�where�a�statistics�professor�is�scheduled�to�teach�two�sections�of�introductory� statistics�� The� professor,� being� a� cunning� researcher,� decides� to� perform� a� little� experi- ment� where� Section� 14�1� is� taught� using� the� traditional� lecture� method� and� Section� 14�2� is�taught�with�more�innovative�methods�using�extensive�graphics,�computer�simulations,� and� computer-assisted� and� calculator-based� instruction,� as� well� as� using� mostly� small- group�and�self-directed�instruction��The�professor�is�interested�in�which�section�performs� better�in�the�course�
Before�the�study/course�begins,�the�professor�thinks�about�whether�there�are�other�vari- ables�related�to�statistics�performance�that�should�somehow�be�taken�into�account�in�the� design�� An� obvious� one� is� ability� in� quantitative� methods�� From� previous� research� and� experience,�the�professor�knows�that�ability�in�quantitative�methods�is�highly�correlated� with�performance�in�statistics�and�decides�to�give�a�measure�of�quantitative�ability�in�the� first�class�and�use�that�as�a�covariate�in�the�analysis��A�covariate�(e�g�,�quantitative�ability)� is�defined�as�a�source�of�variation�not�controlled�for�in�the�design�of�the�experiment�but� that�the�researcher�believes�to�affect�the�dependent�variable�(e�g�,�course�performance)��The� covariate�is�used�to�statistically�adjust�the�dependent�variable��For�instance,�if�Section�14�1� has�higher�quantitative�ability�than�Section�14�2�going�into�the�study,�then�it�would�be�wise� to�take�this�into�account�in�the�analysis��Otherwise�Section�14�1�might�outperform�Section� 14�2�due�to�their�higher�quantitative�ability�rather�than�due�to�the�method�of�instruction�� This�is�precisely�the�point�of�the�ANCOVA��Some�of�the�more�typical�examples�of�covari- ates�in�education�and�the�behavioral�sciences�are�pretest�(where�the�dependent�variable�is� the�posttest),�prior�achievement,�weight,�IQ,�aptitude,�age,�experience,�previous�training,� motivation,�and�grade�point�average�(GPA)�
Let�us�now�begin�with�the�characteristics�of�the�ANCOVA�model��The�first�set�of�char- acteristics� is� obvious� because� they� carry� over� from� the� one-factor� fixed-effects� ANOVA� model��There�is�a�single�independent�variable�or�factor�with�two�or�more�levels�or�catego- ries�(thus�the�independent�variable�continues�to�be�either�nominal�or�ordinal�in�measure- ment�scale)��The�levels�of�the�independent�variable�are�fixed�by�the�researcher�rather�than� randomly�sampled�from�a�population�of�levels��Once�the�levels�of�the�independent�variable� are�selected,�subjects�or�individuals�are�somehow�assigned�to�these�levels�or�groups��Each� subject�is�then�exposed�to�only�one�level�of�the�independent�variable�(although�ANCOVA� with�repeated�measures�is�also�possible,�but�is�not�discussed�here)��In�our�example,�method� of� statistics� instruction� is� the� independent� variable� with� two� levels� or� groups,� the� tradi- tional�lecture�method�and�the�cutting-edge�method�
Situations� where� the� researcher� is� able� to� randomly� assign� subjects� to� groups� are� known� as� true experimental designs�� Situations� where� the� researcher� does� not� have� control� over� which� level� a� subject� is� assigned� to� are� known� as� quasi-experimental designs�� This� lack� of� control� may� occur� for� one� of� two� reasons�� First,� the� groups� may� be�already�in�place�when�the�researcher�arrives�on�the�scene;�these�groups�are�referred� to� as� intact groups� (e�g�,� based� on� class� assignments� made� by� students� at� the� time� of� registration)��Second,�it�may�be�theoretically�impossible�for�the�researcher�to�assign�sub- jects�to�groups�(e�g�,�income�level)��Thus,�a�distinction�is�typically�made�about�whether� or� not�the�researcher�can�control�the�assignment�of�subjects�to�groups��The�distinction� between�the�use�of�ANCOVA�in�true�and�quasi-experimental�situations�has�been�quite� controversial�over�the�past�few�decades;�we�look�at�it�in�more�detail�later�in�this�chapter�� For�further�information�on�true�experimental�designs�and�quasi-experimental�designs,�
430 An Introduction to Statistical Concepts
we� suggest� you� consider� Campbell� and� Stanley� (1966),� Cook and Campbell� (1979),� and� Shadish,�Cook,�and�Campbell�(2002)��In�our�example�again,�if�assignment�of�students�to� sections�is�random,�then�we�have�a�true�experimental�design��If�assignment�of�students� to�sections�is�not�random,�perhaps�already�assigned�at�registration,�then�we�have�a�quasi- experimental�design�
One�final�item�in�the�first�set�of�characteristics�has�to�do�with�the�measurement�scales� of�the�variables��In�the�ANCOVA,�it�is�assumed�the�dependent�variable�is�measured�at�the� interval� level� or� better�� If� the� dependent� variable� is� measured� at� the� ordinal� level,� then� nonparametric�procedures�described�toward�the�end�of�this�chapter�should�be�considered�� It�is�also�assumed�that�the�covariate�is�measured�at�the�interval�level�or�better��Lastly,�as� indicated�previously,�the�independent�variable�must�be�a�grouping�or�categorical�variable�
The� remaining� characteristics� have� to� do� with� the� uniqueness� of� the� ANCOVA�� As� already� mentioned,� the� ANCOVA� is� a� form� of� statistical� control� developed� specifically� to�reduce�unexplained�error�variation��The�covariate�(sometimes�known�as�a�concomitant variable,�as�it�accompanies�or�is�associated�with�the�dependent�variable)�is�a�source�of�varia- tion�not�controlled�for�in�the�design�of�the�experiment�but�believed�to�affect�the�dependent� variable��In�a�factorial�design,�for�example,�a�factor�could�be�included�to�reduce�error�varia- tion��However,�this�represents�an�experimental�design�form�of�control�as�it�is�included�as� a�factor�in�the�model�
In�ANCOVA,�the�dependent�variable�is�adjusted�statistically�to�remove�the�effects�of�the� portion�of�uncontrolled�variation�represented�by�the�covariate��The�group�means�on�the� dependent�variable�are�adjusted�so�that�they�now�represent�groups�with�the�same�means� on�the�covariate��The�ANCOVA�is�essentially�an�ANOVA�on�these�“adjusted�means�”�This� needs�further�explanation��Consider�first�the�situation�of�the�randomized�true�experiment� where� there� are� two� groups�� Here� it� is� unlikely� that� the� two� groups� will� be� statistically� different�on�any�variable�related�to�the�dependent�measure��The�two�groups�should�have� roughly�equivalent�means�on�the�covariate,�although�5%�of�the�time,�we�would�expect�a� significant�difference�due�to�chance�at�α�=��05��Thus,�we�typically�do�not�see�preexisting� differences� between� the� two� groups� on� the� covariate� in� a� true� experiment—that� is� the� value� and� beauty� of� random� assignment,� especially� as� it� relates� to� ANCOVA�� However,� the�relationship�between�the�covariate�and�the�dependent�variable�is�important��If�these� variables�are�linearly�related�(discussed�later),�then�the�use�of�the�covariate�in�the�analysis� will�serve�to�reduce�the�unexplained�variation�in�the�model��The�greater�the�magnitude�of� the�correlation,�the�more�uncontrolled�variation�can�be�removed,�as�shown�by�a�reduction� in�mean�square�error�
Consider� next� the� situation� of� the� quasi-experiment,� that� is,� without� randomization�� Here�it�is�more�likely�that�the�two�groups�will�be�statistically�different�on�the�covariate� as�well�as�other�variables�related�to�the�dependent�variable��Thus,�there�may�indeed�be�a� preexisting�difference�between�the�two�groups�on�the�covariate��If�the�groups�do�differ� on�the�covariate�and�we�ignore�it�by�conducting�an�ANOVA,�our�ability�to�get�a�precise� estimate�of�the�group�effects�will�be�reduced�as�the�group�effect�will�be�confounded�with� the�effect�of�the�covariate��For�instance,�if�a�significant�group�difference�is�revealed�by�the� ANOVA,�we�would�not�be�certain�if�there�was�truly�a�group�effect�or�whether�the�effect� was�due�to�preexisting�group�differences�on�the�covariate,�or�some�combination�of�group� and�covariate�effects��The�ANCOVA�takes�the�covariate�mean�difference�into�account�as� well�as�the�linear�relationship�between�the�covariate�and�the�dependent�variable�
Thus,�the�covariate�is�used�to�(a)�reduce�error�variation,�(b)�take�any�preexisting�group� mean� difference� on� the� covariate� into� account,� (c)� take� into� account� the� relationship� between�the�covariate�and�the�dependent�variable,�and�(d)�yield�a�more�precise�and�less�
431Introduction to Analysis of Covariance
biased� estimate� of� the� group� effects�� If� error� variation� is� reduced,� the� ANCOVA� will� be� more� powerful� and� require� smaller� sample� sizes� than� the� ANOVA� (Keppel� &� Wickens,� 2004;�Mickey,�Dunn,�&�Clark,�2004;�Myers�&�Well,�1995)��If�error�variation�is�not�reduced,� the�ANOVA�is�more�powerful��A�more�extensive�comparison�of�ANOVA�versus�ANCOVA� is�given�in�Chapter�16��In�addition,�as�shown�later,�one�degree�of�freedom�is�lost�from�the� error�term�for�each�covariate�used��This�results�in�a�larger�critical�value�for�the�F�test�and� makes�it�a�bit�more�difficult�to�find�a�statistically�significant�F�test�statistic��This�is�the�major� cost�of�using�a�covariate��If�the�covariate�is�not�effective�in�reducing�error�variance,�then� we�are�worse�off�than�if�we�had�ignored�the�covariate��Important�references�on�ANCOVA� include�Elashoff�(1969)�and�Huitema�(1980)�
14.2 Layout of Data
Before�we�get�into�the�theory�and�subsequent�analysis�of�the�data,�let�us�examine�the�lay- out�of�the�data��We�designate�each�observation�on�the�dependent�or�criterion�variable�as�Yij,� where�the�j�subscript�tells�us�what�group�or�level�the�observation�belongs�to�and�the�i�sub- script�tells�us�the�observation�or�identification�number�within�that�group��The�first�subscript� ranges�over�i�=�1,�…,�nj,�and�the�second�subscript�ranges�over�j�=�1,�…,�J��Thus,�there�are�J�levels� of�the�independent�variable�and�nj�subjects�in�group�j��We�designate�each�observation�on�the� covariate�as�Xij,�where�the�subscripts�have�the�same�meaning�
The�layout�of�the�data�is�shown�in�Table�14�1��Here�we�see�that�each�pair�of�columns�rep- resents�the�observations�for�a�particular�group�or�level�of�the�independent�variable�on�the� dependent�variable�(i�e�,�Y)�and�the�covariate�(i�e�,�X)��At�the�bottom�of�the�pair�of�columns� for�each�group�j�are�group�means�(Y
– �j,�X
– �j)��Although�the�table�shows�there�are�n�observations�
for�each�group,�we�need�not�make�such�a�restriction,�as�this�was�done�only�for�purposes�of� simplifying�the�table�
14.3 ANCOVA Model
The�ANCOVA�model�is�a�form�of�the�general�linear�model�(GLM),�much�like�the�models� shown�in�the�last�few�chapters�of�this�text��The�one-factor�ANCOVA�fixed-effects� model� can�be�written�in�terms�of�population�parameters�as�follows:
Table 14.1
Layout�for�the�One-Factor�ANCOVA
Level of the Independent Variable
1 2 … J
Y11 X11 Y12 X12 … Y1J X1J Y21 X21 Y22 X22 … Y2J X2J … … … … … … … Yn1 Xn1 Yn2 Xn2 … YnJ XnJ
Y –
�1 X –
�1 Y –
�2 X –
�2 … Y –
�J X –
�J
432 An Introduction to Statistical Concepts
Y Xij Y j w ij X ij= + + − +µ α β µ ε( )
where Yij�is�the�observed�score�on�the�dependent�variable�for�individual�i�in�group�j μY�is�the�overall�or�grand�population�mean�(i�e�,�regardless�of�group�designation)�for�the�
dependent�variable�Y αj�is�the�group�effect�for�group�j βw�is�the�within-groups�regression�slope�from�the�regression�of�Y�on�X�(i�e�,�the�covariate) Xij�is�the�observed�score�on�the�covariate�for�individual�i�in�group�j μX�is�the�overall�or�grand�population�mean�(i�e�,�regardless�of�group�designation)�for�the�
covariate�X εij�is�the�random�residual�error�for�individual�i�in�group�j
The�residual�error�can�be�due�to�individual�differences,�measurement�error,�and/or�other� factors�not�under�investigation��As�you�would�expect,�the�least�squares�sample�estimators� for�each�of�these�parameters�are�as�follows:�Y
– �for�μY,�X
– �for�μX,�aj�for�αj,�bw�for�βw,�and�eij�for�εij��
Just�like�in�the�ANOVA,�the�sum�of�the�group�effects�is�equal�to�0��This�implies�that�if�there� are�any�nonzero�group�effects,�then�the�group�effects�will�balance�out�around�0�with�some� positive�and�some�negative�effects�
The�hypotheses�consist�of�testing�the�equality�of�the�adjusted�means�(defined�by�μ = �j� and�discussed�later)�as�follows:
H0:�μ�1�=�μ�2�=�…�=�μ�J H1:�not�all�the�μ�j�are�equal
14.4 ANCOVA Summary Table
We�turn�our�attention�to�the�familiar�summary�table,�this�time�for�the�one-factor�ANCOVA� model��A�general�form�of�the�summary�table�is�shown�in�Table�14�2��Under�the�first�column,� you�see�the�following�sources:�adjusted�between-groups�variation,�adjusted�within-groups� variation,�variation�due�to�the�covariate,�and�total�variation��The�second�column�notes�the� sums�of�squares�terms�for�each�source�(i�e�,�SSbetw(adj),�SSwith(adj),�SScov,�and�SStotal)��Recall�that� the�between�source�represents�the�independent�variable�being�systematically�studied�and� the�within�source�represents�the�error�or�residual�
The�third�column�gives�the�degrees�of�freedom�for�each�source��For�the�adjusted�between- groups� source� (i�e�,� the� independent� variable� controlling� for� the� covariate),� because� there� are�J�group�means,�the�dfbetw(adj)�is�J�−�1,�the�same�as�in�the�one-factor�ANOVA�model��For� the�adjusted�within-groups�source,�because�there�are�N�total�observations�and�J�groups,�we�
Table 14.2
One-Factor�ANCOVA�Summary�Table
Source SS df MS F
Between�adjusted SSbetw(adj) J − 1 MSbetw(adj) MSbetw(adj)/MSwith(adj) Within�adjusted�(i�e�,�error) SSwith(adj) N − J − 1 MSwith(adj) Covariate SScov 1 MScov MScov/MSwith(adj) Total SStotal N − 1
433Introduction to Analysis of Covariance
would�expect�the�degrees�of�freedom�within�to�be�N − J,�because�that�was�the�case�in�the� one-factor�ANOVA�model��However,�as�we�pointed�out�earlier�in�the�characteristics�of�the� ANCOVA�model,�a�price�is�paid�for�the�use�of�a�covariate��The�price�here�is�that�we�lose�one� degree�of�freedom�from�the�within�term�for�a�single�covariate,�so�that�dfwith(adj)�is�N − J�−�1�� For� multiple� covariates,� we� lose� one� degree� of� freedom� for� each� covariate� used� (see� later� discussion)��This�degree�of�freedom�has�gone�to�the�covariate�source�such�that�dfcov�is�equal� to 1��Finally,�for�the�total�source,�as�there�are�N�total�observations,�the�dftotal�is�the�usual�N�−�1�
The�fourth�column�gives�the�mean�squares�for�each�source�of�variation��As�always,� the�mean�squares�represent�the�sum�of�squares�weighted�by�their�respective�degrees� of�freedom��Thus,�[MSbetw(adj)�=�SSbetw(adj)/(J�−�1)],�[MSwith(adj)�=�SSwith(adj)/(N − J�−�1)],�and� [MScov�=�SScov/1]��The�last�column�in�the�ANCOVA�summary�table�is�for�the�F�values��Thus,� for�the�one-factor�fixed-effects�ANCOVA�model,�the�F�value�tests�for�differences�between� the�adjusted�means�(i�e�,�to�test�for�differences�in�the�mean�of�the�dependent�variable�based� on�the�levels�of�the�independent�variable�when�controlling�for�the�covariate)�and�is�com- puted�as�F = MSbetw(adj)/MSwith(adj)��A�second�F�value,�which�is�obviously�not�included�in�the� ANOVA�model,�is�the�test�of�the�covariate��To�be�specific,�this�F�statistic�is�actually�testing� the�hypothesis�of�H0:�βw�=�0��If�the�slope�is�equal�to�0,�then�the�covariate�and�the�dependent� variable� are� unrelated�� This� F� value� is� equal� to� F = MScov/MSwith(adj)�� If� the� F� test� for� the� covariate�is�not�statistically�significant�(and�has�a�negligible�effect�size),�the�researcher�may� want�to�consider�removing�that�covariate�from�the�model�
The�critical�value�for�the�test�of�difference�between�the�adjusted�means�is�αFJ−1,�N−J−1��The� critical�value�for�the�test�of�the�covariate�is�αF1,�N−J−1��The�null�hypotheses�in�each�case�are� rejected�if�the�F�test�statistic�exceeds�the�F�critical�value��The�critical�values�are�found�in� the�F�table�of�Table�A�4�
If�the�F�test�statistic�for�the�adjusted�means�exceeds�the�F�critical�value,�and�there�are�more� than�two�groups,�then�it�is�not�clear�exactly�how�the�means�are�different��In�this�case,�some� MCP� may� be� used� to� determine� which� means� are� different� (see� later� discussion)�� For� the� test� of� the� covariate� (i�e�,� the� within-groups� regression� slope),� we� hope� that� the� F� test� sta- tistic�does�exceed�the�F�critical�value��Otherwise�the�power�and�precision�of�the�test�of�the� adjusted�means�in�ANCOVA�will�be�lower�than�the�test�of�the�unadjusted�means�in�ANOVA� because�the�covariate�is�not�significantly�related�to�the�dependent�variable��[As�stated�previ- ously,�if�the�F�test�for�the�covariate�is�not�statistically�significant�(and�has�a�negligible�effect� size),�the�researcher�may�want�to�consider�removing�that�covariate�from�the�model�]
14.5 Partitioning the Sums of Squares
As� seen� already,� the� partitioning� of� the� sums� of� squares� is� the� backbone� of� all� GLMs,� whether�we�are�dealing�with�an�ANOVA�model,�an�ANCOVA�model,�or�a�linear�regression� model��As�always,�the�first�step�is�to�partition�the�total�variation�into�its�relevant�parts�or� sources�of�variation��As�we�have�learned�from�the�previous�section,�the�sources�of�varia- tion�for�the�one-factor�ANCOVA�model�are�adjusted�between�groups�(i�e�,�the�independent� variable),�adjusted�within�groups�(i�e�,�error),�and�the�covariate��This�is�written�as
SS SS SS SStotal betw(adj) with(adj) cov= + +
From�this�point,�the�statistical�software�is�used�to�handle�the�remaining�computations�
434 An Introduction to Statistical Concepts
14.6 Adjusted Means and Related Procedures
In�this�section,�we�formally�define�the�adjusted�mean,�briefly�examine�several�MCPs,�and� very�briefly�consider�power,�confidence�intervals�(CIs),�and�effect�size�measures�
We�have�spent�considerable�time�already�discussing�the�analysis�of�the�adjusted�means�� Now�it�is�time�to�define�them��The�adjusted�mean�is�denoted�by�Yj� ′�and�estimated�by
Y Y b X Xj j w j. . . ..( )′ = − −
Here�it�should�be�noted�that�the�adjusted�mean�is�simply�equal�to�the�unadjusted�mean� (i�e�,� Y
– �j)� minus� the� adjustment� [i�e�,� bw(X
– �j� −� X
– ��)]�� The� adjustment� is� a� function� of� the�
within-groups�regression�slope�(i�e�,�bw)�and�the�difference�between�the�group�mean�and� the�overall�mean�for�the�covariate�(i�e�,�the�difference�being�the�group�effect,�X
– �j�−�X
– ��)��
No� adjustment� will� be� made� if� (a)� bw� =� 0� (i�e�,� X� and� Y� are� unrelated),� or� (b)� the� group� means�on�the�covariate�are�all�the�same��Thus,�in�both�cases,�Y Yj j� �= ′��In�all�other�cases,� at�least�some�adjustment�will�be�made�for�some�of�the�group�means�(although�not�neces- sarily�for�all�of�the�group�means)�
You�may�be�wondering�how�this�adjustment�actually�works��Let�us�assume�the�covariate� and�the�dependent�variable�are�positively�correlated�such�that�bw�is�also�positive,�and�there� are�two�treatment�groups�with�equal�n’s�that�differ�on�the�covariate��If�group�1�has�a�higher� mean� on� both� the� covariate� and� the� dependent� variable� than� group� 2,� then� the� adjusted� means�will�be�closer�together�than�the�unadjusted�means��For�our�first�example,�we�have� the�following�conditions:
b Y Y X X Xw = = = = = =1 50 30 20 10 151 2 1 2, , , , ,. . . . ..
The�adjusted�means�are�determined�as�follows:
Y Y b X Xw. . . ..( ) ( )1 1 1 50 1 20 15 45′ = − − = − − =
Y Y b X Xw. . . ..( ) ( )2 2 2 30 1 10 15 35′ = − − = − − =
This� is� shown� graphically� in� Figure� 14�1a�� In� looking� at� the� covariate� X,� we� see� that� group�1�has�a�higher�mean�(X
– �1�=�20)�than�group�2�(X
– �2�=�10)�by�10�points��The�vertical�line�
represents� the� overall� mean� on� the� covariate� (X –
��� =� 15)�� In� looking� at� the� dependent� variable�Y,�we�see�that�group�1�has�a�higher�mean�(Y
– �1�=�50)�than�group�2�(Y
– �2�=�30)�by�20�
points��The�diagonal�lines�represent�the�regression�lines�for�each�group,�with�bw�=�1�0�� The�points�at�which�the�regression�lines�intersect�(or�cross)�the�vertical�line�(X
– ���=�15)�rep-
resent�on�the�Y�scale�the�values�of�the�adjusted�means��Here�we�see�that�the�adjusted� mean� for� group� 1�( ).Y1 45′ = � is� larger� than� the� adjusted� mean� for� group� 2� ( ).Y2 35′ = � by� 10� points�� Thus,� because� of� the� preexisting� difference� on� the� covariate,� the� adjusted� means�here�are�somewhat�closer�together�than�the�unadjusted�means�(10 points�vs��20� points,�respectively)�
435Introduction to Analysis of Covariance
If�group�1�has�a�higher�mean�on�the�covariate�and�a�lower�mean�on�the�dependent�vari- able� than� group� 2,� then� the� adjusted� means� will� be� further� apart� than� the� unadjusted� means��As�a�second�example,�we�have�the�following�slightly�different�conditions:
b Y Y X X Xw = = = = = =1 30 50 20 10 151 2 1 2, , , , ,. . . . ..
Then�the�adjusted�means�become�as�follows:
Y Y b X Xw. . . ..( ) ( )1 1 1 30 1 20 15 25′ = − − = − − =
Y Y b X Xw. . . ..( ) ( )2 2 2 50 1 10 15 55′ = − − = − − =
This� is� shown� graphically� in� Figure� 14�1b,� where� the� unadjusted� means� differ� by� 20� points� and� the� adjusted� means� differ� by� 30� points�� There� are� obviously� other� possible� situations�
Let� us� briefly� examine� MCPs� for� use� in� the� ANCOVA� situation�� Most� of� the� proce- dures� described� in� Chapter� 12� can� be� adapted� for� use� with� a� covariate,� although� a� few� procedures� are� not� mentioned� here� as� critical� values� do� not� currently� exist�� The� adapted� procedures� involve� a� different� form� of� the� standard� error� of� a� contrast�� The� contrasts�are�formed�based�on�adjusted�means,�of�course��Let�us�briefly�outline�just�a� few�procedures��Each�of�the�test�statistics�has�as�its�numerator�the�contrast,�ψ′,�such�as� ψ ′ ′ ′= −Y Y. .1 2��The�standard�errors�do�differ�somewhat�depending�on�the�specific�MCP,� just�as�they�do�in�ANOVA�
The�example�procedures�briefly�described�here�are�easily�translated�from�the�ANOVA� context�into�the�ANCOVA�context��The�Dunn�(or�the�Bonferroni)�method�is�appropriate�to� use�for�a�small�number�of�planned�contrasts�(still�utilizing�the�critical�values�from�Table� A�8)��The�Scheffé�procedure�can�be�used�for�unplanned�complex�contrasts�with�equal�group� variances�(again�based�on�the�F�table�in�Table�A�4)��The�Tukey�HSD�test�is�most�desirous�
50
Y
40
30
20
10
0 0 5 15 20 X10
X.2 –
X.1 –
Y.1 –
Y.2 –
Y.2 –
Y.1 –
Y.́1 –
Y.2́ –
X.. –
0 5 15 20 X10 X.2 –
X.1 –
X.. –
(a)
60 Y
50
40
30
20
10
0
Y.́1 –
Y.́2 –
(b)
Group 1Group 2 Group 2 Group 1
FIGuRe 14.1 Graphs�of�ANCOVA�adjustments�
436 An Introduction to Statistical Concepts
for�unplanned�pairwise�contrasts�with�equal�n’s�per�group��There�has�been�some�discus- sion�in�the�literature�about�the�appropriateness�of�this�test�in�ANCOVA��Most�statisticians� currently�argue�that�the�procedure�is�only�appropriate�when�the�covariate�is�fixed,�when�in� fact�it�is�almost�always�random��As�a�result,�the�Bryant�and�Paulson�(1976)�generalization�of� the�Tukey�procedure�has�been�developed�for�the�random�covariate�case��The�test�statistic� is�compared�to�the�critical�value�αqX,df(error),J�taken�from�Table�A�10,�where�X�is�the�number� of�covariates��If�the�group�sizes�are�unequal,�the�harmonic�mean�can�be�used�in�ANCOVA� (Huitema,�1980)��A�generalization�of�the�Tukey-Bryant�procedure�for�unequal�n’s�ANCOVA� was� developed� by� Hochberg� and� Varon-Salomon� (1984)� (also� see� Hochberg� &� Tamhane,� 1987;�Miller,�1997)�
Finally� a� very� brief� comment� about� power,� CIs,� and� effect� size� measures� for� the� one- factor�ANCOVA�model��In�short,�these�procedures�work�exactly�the�same�as�in�the�factorial� ANOVA�model,�except�that�they�are�based�on�adjusted�means�(Cohen,�1988),�and�as�we�will� see� in� SPSS,� partial� eta� squared� is� still� the� effect� size� computed�� There� really� is� nothing� more�to�say�than�that�
14.7 Assumptions and Violation of Assumptions
The� introduction� of� a� covariate� requires� several� assumptions� beyond� the� traditional� ANOVA� assumptions�� For� the� familiar� assumptions� (e�g�,� independence� of� observations,� homogeneity,�and�normality),�the�discussion�is�kept�to�a�minimum�as�these�have�already� been� described� in� Chapters� 11� and� 13�� The� new� assumptions� are� as� follows:� (a)� linear- ity,� (b)� independence� of� the� covariate� and� the� independent� variable,� (c)� the� covariate� is� measured� without� error,� and� (d)� homogeneity� of� the� regression� slopes�� In� this� section,� we� describe� each� assumption,� how� each� assumption� can� be� evaluated,� the� effects� that� a� violation� of� the� assumption� might� have,� and� how� one� might� deal� with� a� serious� viola- tion��Later�in�the�chapter,�when�we�illustrate�how�to�use�SPSS�to�generate�ANCOVA,�we� will�specifically�test�for�the�assumptions�of�independence�of�observations,�homogeneity�of� variance,�normality,�linearity,�independence�of�the�covariate�and�the�independent�variable,� and�homogeneity�of�regression�slopes�
14.7.1 Independence
As�we�learned�previously,�the�assumption�of�independence�of�observations�can�be�met� by�(a)�keeping�the�assignment�of�individuals�to�groups�(i�e�,�to�the�levels�or�categories� of� the� independent� variable)� separate� through� the� design� of� the� experiment� (specifi- cally�random�assignment—not�to�be�confused�with�random�selection),�and�(b)�keeping� the� individuals� separate� from� one� another� through� experimental� control� so� that� the� scores�on�the�dependent�variable�Y�are�independent�across�subjects�(both�within�and� across�groups)�
As�in�previous�ANOVA�models,�the�use�of�independent�random�samples�is�also�cru- cial� in� the� ANCOVA�� The� F� ratio� is� very� sensitive� to� violation� of� the� independence� assumption�in�terms�of�increased�likelihood�of�a�Type�I�and/or�Type�II�error��A�violation� of�the�independence�assumption�may�affect�the�standard�errors�of�the�sample�adjusted� means� and� thus� influence� any� inferences� made� about� those� means�� One� purpose� of�
437Introduction to Analysis of Covariance
random�assignment�of�individuals�to�groups�is�to�achieve�independence��If�each�indi- vidual� is� only� observed� once� and� individuals� are� randomly� assigned� to� groups,� then� the� independence� assumption� is� usually� met�� Random� assignment� is� important� for� valid� interpretation� of� both� the� F� test� and� MCPs�� Otherwise,� the� F� test� and� adjusted� means�may�be�biased�
The�simplest�procedure�for�assessing�independence�is�to�examine�residual�plots�by�group�� If� the� independence� assumption� is� satisfied,� then� the� residuals� should� fall� into� a� random� display�of�points��If�the�assumption�is�violated,�then�the�residuals�will�fall�into�some�type�of� cyclical�pattern��As�discussed�in�Chapter�11,�the�Durbin�and�Watson�statistic�(1950,�1951,�1971)� can�be�used�to�test�for�autocorrelation��Violations�of�the�independence�assumption�generally� occur� in� the� three� situations� we� mentioned� in� Chapter� 11:� time� series� data,� observations� within�blocks,�or�replication��For�severe�violations�of�the�independence�assumption,�there�is� no�simple�“fix,”�such�as�the�use�of�transformations�or�nonparametric�tests�(see�Scariano�&� Davenport,�1987)�
14.7.2 homogeneity of Variance
The�second�assumption�is�that�the�variances�of�each�population�are�the�same,�known� as� the�homogeneity�of�variance�assumption�� A�violation�of�this�assumption�may�lead� to� bias� in� the� SSwith� term,� as� well� as� an� increase� in� the� Type� I� error� rate,� and� possibly� an�increase�in�the�Type�II�error�rate��A�summary�of�Monte�Carlo�research�on�ANCOVA� assumption� violations� by� Harwell� (2003)� indicates� that� the� effect� of� the� violation� is� negligible� with� equal� or� nearly� equal� n’s� across� the� groups�� There� is� a� more� serious� problem�if�the�larger�n’s�are�associated�with�the�smaller�variances�(actual�or�observed� α� >� nominal� or� stated� α� selected� by� the� researcher,� which� is� a� liberal� result),� or� if� the� larger� n’s� are� associated� with� the� larger� variances� (actual� α� <� nominal� α,� which� is� a� conservative�result)�
In�a�plot�of�Y�versus�the�covariate�X�for�each�group,�the�variability�of�the�distributions� may� be� examined� for� evidence� of� the� extent� to� which� this� assumption� is� met�� Another� method�for�detecting�violation�of�the�homogeneity�assumption�is�the�use�of�formal�statisti- cal�tests�(e�g�,�Levene’s�test),�as�discussed�in�Chapter�11�and�as�we�illustrate�using�SPSS�later� in�this�chapter��Several�solutions�are�available�for�dealing�with�a�violation�of�the�homoge- neity�assumption��These�include�the�use�of�variance-stabilizing�transformations�or�other� ANCOVA� models� that� are� less� sensitive� to� unequal� variances,� such� as� nonparametric� ANCOVA�procedures�(described�at�the�end�of�this�chapter)�
14.7.3 Normality
The�third�assumption�is�that�each�of�the�populations�follows�the�normal�distribution�� Based� on� the� classic� work� by� Box� and� Anderson� (1962)� and� Atiqullah� (1964),� as� well� as� the� summarization� of� modern� Monte� Carlo� work� by� Harwell� (2003),� the� F� test� is� relatively� robust� to� nonnormal� Y� distributions,� “minimizing� the� role� of� a� normally� distributed�X”�(Harwell,�2003,�p��62)��Thus,�we�need�only�really�be�concerned�with�seri- ous� nonnormality� (although� “serious� nonnormality”� is� a� subjective� call� made� by� the� researcher)�
The� following� graphical� techniques� can� be� used� to� detect� violation� of� the� normality� assumption:� (a)� frequency� distributions� (such� as� stem-and-leaf� plots,� boxplots,� or� histo- grams)�or�(b)�normal�probability�plots��There�are�also�several�statistical�procedures�available� for�the�detection�of�nonnormality�[e�g�,�the�Shapiro–Wilk�(S–W)�test,�1965]��If�the�assumption�
438 An Introduction to Statistical Concepts
of�normality�is�violated,�transformations�can�also�be�used�to�normalize�the�data,�as�previ- ously�discussed�in�Chapter�11��In�addition,�one�can�use�one�of�the�rank�ANCOVA�procedures� previously�mentioned�
14.7.4 linearity
The�next�assumption�is�that�the�regression�of�Y�(i�e�,�the�dependent�variable)�on�X�(i�e�,� the�covariate)�is�linear��If�the�relationship�between�Y�and�X�is�not�linear,�then�use�of�the� usual�ANCOVA�procedure�is�not�appropriate,�just�as�linear�regression�(see�Chapter�17)� would�not�be�appropriate�in�cases�of�nonlinearity��In�ANCOVA�(as�well�as�in�correlation� and�linear�regression),�we�fit�a�straight�line�to�the�data�points�in�a�scatterplot��When�the� relationship�is�nonlinear,�a�straight�line�will�not�fit�the�data�particularly�well��In�addition,� the�magnitude�of�the�linear�correlation�will�be�smaller��If�the�relationship�is�not�linear,� the�estimate�of�the�group�effects�will�be�biased,�and�the�adjustments�made�in�SSwith�and� SSbetw�will�be�smaller�
Violations� of� the� linearity� assumption� can� generally� be� detected� by� looking� at� scatter- plots� of� Y� versus� X,� overall� and� for� each� group� or� category� of� the� independent� variable�� Once�a�serious�violation�of�the�linearity�assumption�has�been�detected,�there�are�two�alter- natives� that� can� be� used,� transformations� and� nonlinear� ANCOVA�� Transformations� on� one�or�both�variables�can�be�used�to�achieve�linearity�(Keppel�&�Wickens,�2004)��The�sec- ond� option� is� to� use� nonlinear� ANCOVA� methods� as� described� by� Huitema� (1980)� and� Keppel�and�Wickens�(2004)�
14.7.5 Fixed Independent Variable
The�fifth�assumption�states�that�the�levels�of�the�independent�variable�are�fixed�by�the� researcher�� This� results� in� a� fixed-effects� model� rather� than� a� random-effects� model�� As�in�the�one-factor�ANOVA�model,�the�one-factor�ANCOVA�model�is�the�same�com- putationally� in� the� fixed-� and� random-effects� cases�� The� summary� of� Monte� Carlo� research� by� Harwell� (2003)� indicates� that� the� impact� of� a� random-effect� on� the� F� test� is�minimal�
14.7.6 Independence of the Covariate and the Independent Variable
A� condition� of� the� ANCOVA� model� (although� not� an� assumption)� requires� that� the� covariate� and� the� independent� variable� be� independent�� That� is,� the� covariate� is� not� influenced�by�the�independent�or�treatment�variable��If�the�covariate�is�affected�by�the� treatment�itself,�then�the�use�of�the�covariate�in�the�analysis�either�(a)�may�remove�part� of�the�treatment�effect�or�produce�a�spurious�(inflated)�treatment�effect�or�(b)�may�alter� the�covariate�scores�as�a�result�of�the�treatment�being�administered�prior�to�obtaining� the�covariate�data��The�obvious�solution�to�this�potential�problem�is�to�obtain�the�covari- ate� scores� prior� to� the� administration� of� the� treatment�� In� other� words,� be� alert� prior� to�the�study�for�possible�covariate�candidates��There�are�many�researchers�who�argue� that,� because� of� this� assumption,� ANCOVA� is� only� appropriate� in� the� case� of� a� true� experiment� where� random� assignment� of� cases� to� groups� was� performed�� Thus,� in� a� true�experiment,�the�treatment�(i�e�,�independent�variable)�and�covariate�are�not�related� by�default�of�random�assignment,�and,�thereby,�the�assumption�of�independence�of�the�
439Introduction to Analysis of Covariance
covariate� and� independent� variable� is� met�� If� randomization� is� not� possible,� closely� matching�participants�on�the�covariate�may�also�help�to�ensure�the�assumption�is�not� violated�
Let�us�consider�an�example�where�this�condition�is�obviously�violated��A�psychologist�is� interested�in�which�of�several�hypnosis�treatments�is�most�successful�in�reducing�or�elimi- nating�cigarette�smoking��A�group�of�heavy�smokers�is�randomly�assigned�to�the�hypnosis� treatments�� After� the� treatments� have� been� completed,� the� researcher� suspects� that� some� patients�are�more�susceptible�to�hypnosis�(i�e�,�are�more�suggestible)�than�others��By�using� suggestibility�as�a�covariate�after�the�study�is�completed,�the�researcher�would�not�be�able� to�determine�whether�group�differences�were�a�result�of�hypnosis�treatment,�suggestibility,� or�some�combination��Thus,�the�measurement�of�suggestibility�after�the�hypnosis�treatments� have�been�administered�would�be�ill-advised��An�extended�discussion�of�this�condition�is� given�in�Maxwell�and�Delaney�(1990)�
Evidence�of�the�extent�to�which�this�assumption�is�met�can�be�done�by�examining�mean� differences�on�the�covariate�across�the�levels�of�the�independent�variable��If�the�indepen- dent�variable�has�only�two�levels,�an�independent�t�test�would�be�appropriate��If�the�inde- pendent�variable�has�more�than�two�categories,�a�one-way�ANOVA�would�suffice��If�the� groups� are� not� statistically� different� on� the� covariate,� then� that� lends� evidence� that� the� assumption�of�independence�of�the�covariate�and�the�independent�variable�has�been�met�� If�the�groups�are�statistically�different�on�the�covariate,�then�the�groups�are�not�likely�to� be�equivalent�
14.7.7 Covariate Measured Without error
An�assumption�that�we�have�not�yet�discussed�in�this�text�is�that�the�covariate�is�mea- sured�without�error��This�is�of�special�concern�in�education�and�the�behavioral�sciences� where� variables� are� often� measured� with� considerable� measurement� error�� In� random- ized�experiments,�bw�(i�e�,�the�within-groups�regression�slope�from�the�regression�of�the� dependent� variable,� Y,� on� the� covariate,� X)� will� be� underestimated� so� that� less� of� the� covariate� effect� is� removed� from� the� dependent� variable� (i�e�,� the� adjustments� will� be� smaller)�� In� addition,� the� reduction� in� the� unexplained� variation� will� not� be� as� great,� and� the� F� test� will� not� be� as� powerful�� The� F� test� is� generally� conservative� in� terms� of� Type�I�error�(the�actual�observed�α�will�be�less�than�the�nominal�α�which�was�selected� by�the�researcher—the�nominal�alpha�is�often��05)��However,�the�treatment�effects�will� not�be�biased��In�quasi-experimental�designs,�bw�will�also�be�underestimated�with�simi- lar�effects��However,�the�treatment�effects�may�be�seriously�biased��A�method�by�Porter� (1967)�is�suggested�for�this�situation�
There�is�considerable�discussion�about�the�effects�of�measurement�error�(e�g�,�Cohen� &�Cohen,�1983;�Huitema,�1980;�Keppel�&�Wickens,�2004;�Lord,�1960,�1967,�1969;�Mickey� et�al��2004;�Pedhazur,�1997;�Porter,�1967;�Reichardt,�1979;�Weisberg,�1979)��Obvious�viola- tions�of�this�assumption�can�be�detected�by�computing�the�reliability�of�the�covariate� prior�to�the�study�or�from�previous�research��This�is�the�minimum�that�should�be�done�� One� may� also� want� to� consider� the� validity� of� the� covariate� as� well,� where� validity� may�be�defined�as�the�extent�to�which�an�instrument�measures�what�it�was�intended�to� measure��While�this�is�the�first�mention�in�the�text�of�measurement�error,�it�is�certainly� important�that�all�measures�included�in�a�model—regardless�of�which�statistical�pro- cedure�is�being�conducted—are�measured�such�that�the�scores�provide�high�reliability� and�validity�
440 An Introduction to Statistical Concepts
14.7.8 homogeneity of Regression Slopes
The�final�assumption�puts�forth�that�the�slope�of�the�regression�line�between�the�depen- dent�variable�and�covariate�is�the�same�for�each�category�of�the�independent�variable��Here� we�assume�that�β1�=�β2�=�…�=�βJ��This�is�an�important�assumption�because�it�allows�us�to� use�bw,�the�sample�estimator�of�βw,�as�the�within-groups�regression�slope��Assuming�that� the�group�slopes�are�parallel�allows�us�to�test�for�group�intercept�differences,�which is all we are really doing when we test for differences among the adjusted means��Without�this�assumption� of� homogeneity� of� regression� slopes,� groups� can� differ� on� both� the� regression� slope� and� intercept,�and�βw�cannot�legitimately�be�used��If�the�slopes�differ,�then�the�regression�lines� interact�in�some�way��As�a�result,�the�size�of�the�group�differences�in�Y�(i�e�,�the�dependent� variable)�will�depend�on�the�value�of�X�(i�e�,�the�covariate)��For�example,�treatment�1�may�be� most�effective�on�the�dependent�variable�for�low�values�of�the�covariate,�treatment�2�may� be�most�effective�on�the�dependent�variable�for�middle�values�of�the�covariate,�and�treat- ment�3�may�be�most�effective�on�the�dependent�variable�for�high�values�of�the�covariate�� Thus,�we�do�not�have�constant�differences�on�the�dependent�variable�between�the�groups� of�the�independent�variable�across�the�values�of�the�covariate��A�straightforward�interpre- tation�is�not�possible,�which�is�the�same�situation�in�factorial�ANOVA�when�the�interaction� between�factor�A�and�factor�B�is�found�to�be�significant��Thus,�unequal�slopes�in�ANCOVA� represent�a�type�of�interaction�
There�are�other�potential�outcomes�if�this�assumption�is�violated��Without�homogeneous� regression�slopes,�the�use�of�βw�can�yield�biased�adjusted�means�and�can�affect�the�F�test�� Earlier�simulation�studies�by�Peckham�(1968)�and�Glass,�Peckham,�and�Sanders�(1972)�sug- gest�that�for�the�one-factor�fixed-effects�model,�the�effects�will�be�minimal��Later�analyti- cal�research�by�Rogosa�(1980)�suggests�that�there�is�little�effect�on�the�F�test�for�balanced� designs�with�equal�variances,�but�the�F�is�less�robust�for�mild�heterogeneity��However,�a� summary�of�modern�Monte�Carlo�work�by�Harwell�(2003)�indicates�that�the�effect�of�slope� heterogeneity�on�the�F�test�is�(a)�negligible�with�equal�n’s�and�equal�covariate�means�(ran- domized� studies),� (b)� modest� with� equal� n’s� and� unequal� covariate� means� (nonrandom- ized�studies),�and�(c)�modest�with�unequal�n’s�
A�formal�statistical�procedure�is�often�conducted�to�test�for�homogeneity�of�slopes�using� statistical� software� such� as� SPSS� (discussed� later� in� this� chapter),� although� the� eyeball� method�(i�e�,�see�if�the�slopes�look�about�the�same�by�reviewing�scatterplots�of�the�depen- dent�variable�and�covariate�for�each�category�of�the�independent�variable)�can�be�a�good� starting�point��Some�alternative�tests�for�equality�of�slopes�when�the�variances�are�unequal� are�provided�by�Tabatabai�and�Tan�(1985)�
Several�alternatives�are�available�if�the�homogeneity�of�slopes�assumption�is�violated�� The�first�is�to�use�the�concomitant�variable�not�as�a�covariate�but�as�a�blocking�variable�� This� will� work� because� this� assumption� is� not� made� for� the� randomized� block� design� (see�Chapter�16)��A�second�option,�and�not�a�very�desirable�one,�is�to�analyze�each�group� separately�with�its�own�slope�or�subsets�of�the�groups�having�equal�slopes��A�third�pos- sibility�is�to�utilize�interaction�terms�between�the�covariate�and�the�independent�variable� and�conduct�a�regression�analysis�(see�Agresti�&�Finlay,�1986)��A�fourth�option�is�to�use� the�Johnson�and�Neyman�(1936)�technique,�whose�purpose�is�to�determine�the�values�of� X�(i�e�,�the�covariate)�that�are�related�to�significant�group�differences�on�Y�(i�e�,�the�depen- dent�variable)��This�procedure�is�beyond�the�scope�of�this�text,�and�the�interested�reader� is� referred� to� Huitema� (1980)� or� Wilcox� (1987)�� A� fifth� option� is� to� use� more-modern� robust�methods�(e�g�,�Maxwell�&�Delaney,�1990;�Wilcox,�2003)�
A�summary�of�the�ANCOVA�assumptions�is�presented�in�Table�14�3�
441Introduction to Analysis of Covariance
14.8 Example
Consider�the�following�illustration�of�what�we�have�covered�in�this�chapter��Our�dependent� variable�is�the�score�on�a�statistics�quiz�(with�a�maximum�possible�score�of�6),�the�covariate�is� the�score�on�an�aptitude�test�for�statistics�taken�at�the�beginning�of�the�course�(with�a�maxi- mum�possible�score�of�10),�and�the�independent�variable�is�the�section�of�statistics�taken�(where� group�1�receives�the�traditional�lecture�method�and�group�2�receives�the�modern�innovative� method�that�includes�components�such�as�small-group�and�self-direction�instruction)��Thus,� the�researcher�is�interested�in�whether�the�method�of�instruction�influences�student�perfor- mance�in�statistics,�controlling�for�statistics�aptitude�(assume�we�have�developed�an�aptitude� measure�that�is�relatively�error-free)��Students�are�randomly�assigned�to�one�of�the�two�groups� at� the� beginning� of� the� semester� when� the� measure� of� statistics� aptitude� is� administered��
Table 14.3
Assumptions�and�Effects�of�Violations—One-Factor�ANCOVA
Assumption Effect of Assumption Violation
1��Independence •�Increased�likelihood�of�a�Type�I�and/or�Type�II�error�in�F •�Affects�standard�errors�of�means�and�inferences�about�those�means
2��Homogeneity�of�variance •�Bias�in�SSwith;�increased�likelihood�of�a�Type�I�and/or�Type�II�error •�Negligible�effect�with�equal�or�nearly�equal�n’s •��Otherwise�more�serious�problem�if�the�larger�n’s�are�associated�with�
the�smaller�variances�(increased�α)�or�larger�variances�(decreased�α)
3��Normality •��F�test�relatively�robust�to�nonnormal�Y,�minimizing�the�role�of� nonnormal�X
4��Linearity •�Reduced�magnitude�of�rXY •�Straight�line�will�not�fit�data�well •�Estimate�of�group�effects�biased •�Adjustments�made�in�SS�smaller
5��Fixed-effect •�Minimal�impact
6���Covariate�and�factor�are� independent
•�May�reduce/increase�group�effects;�may�alter�covariate�scores
7���Covariate�measured�without� error
•�True�experiment: •� bw�underestimated •� Adjustments�smaller •� Reduction�in�unexplained�variation�smaller •� F�less�powerful •� Reduced�likelihood�of�Type�I�error
•�Quasi-experiment: •� bw�underestimated •� Adjustments�smaller •� Group�effects�seriously�biased
8��Homogeneity�of�slopes •�Negligible�effect�with�equal�n’s�in�true�experiment •�Modest�effect�with�equal�n’s�in�quasi-experiment •�Modest�effect�with�unequal�n’s
442 An Introduction to Statistical Concepts
There�are�6�students�in�each�group�for�a�total�of�12��The�layout�of�the�data�is�shown�in�Table� 14�4,�where�we�see�the�data�and�sample�statistics�(means,�variances,�slopes,�and�correlations)�
The�results�are�summarized�in�the�ANCOVA�summary�table�as�shown�in�the�top�panel� of� Table� 14�5�� The� ANCOVA� test� statistics� are� compared� to� the� critical� value� �05F1,9� =� 5�12� obtained�from�Table�A�4,�using�the��05�level�of�significance��Both�test�statistics�exceed�the� critical�value,�so�we�reject�H0�in�each�case��We�conclude�that�(a)�the�quiz�score�means�do� differ�for�the�two�statistics�groups�when�adjusted�(or�controlling)�for�aptitude�in�statistics,� and� (b)� the� slope� of� the� regression� of� Y� (i�e�,� dependent� variable)� on� X� (i�e�,� covariate)� is� statistically�significantly�different�from�0�(i�e�,�the�test�of�the�covariate)��Just�to�be�complete,� the�results�for�the�ANOVA�on�Y�are�shown�in�the�bottom�panel�of�Table�14�5��We�see�that�in� the�analysis�of�the�unadjusted�means�(i�e�,�the�ANOVA),�there�is�no�significant�group�differ- ence��Thus,�the�adjustment�(i�e�,�ANCOVA�which�controlled�for�aptitude�toward�statistics)� yielded�a�different�statistical�result��The�covariate�also�“did�its�thing”�in�that�a�reduction�
Table 14.4
Data�and�Summary�Statistics�for�the�Statistics�Instruction�Example
Group 1 Group 2 Overall
Statistic Quiz (Y)
Aptitude (X)
Quiz (Y)
Aptitude (X)
Quiz (Y)
Aptitude (X)
1 4 1 1 2 3 2 3 3 5 4 2 4 6 5 4 5 7 6 5 6 9 6 7
Means 3�5000 5�6667 4�0000 3�6667 3�7500 4�6667 Variances 3�5000 4�6667 4�4000 4�6667 3�6591 5�3333 bYX 0�8143 0�8143 0�5966 rXY 0�9403 0�8386 0�7203 Adjusted� means
2�6857 4�8143
Table 14.5
One-Factor�ANCOVA�and�ANOVA�Summary� Tables—Statistics�Instruction�Example
Source SS df MS F
ANCOVA Between�adjusted 10�8127 1 10�8127 11�3734a
Within�adjusted 8�5560 9 0�9507 Covariate 20�8813 1 20�8813 21�9641a
Total 40�2500 11
ANOVA Between 0�7500 1 0�7500 0�1899b
Within 39�5000 10 3�9500 Total 40�2500 11 a�
�05F1,9�=�5�12�(critical�value)� b�
�05F1,10�=�4�96�(critical�value)�
443Introduction to Analysis of Covariance
in�MSwith�resulted�due�to�the�strong�relationship�between�the�covariate�and�the�dependent� variable�(i�e�,�rXY�=�0�7203�overall)�
Let�us�next�examine�the�group�quiz�score�means,�as�shown�in�Table�14�4��Here�we�see�that� with�the�unadjusted�quiz�score�means�(i�e�,�prior�to�controlling�for�the�covariate),�there�is�a� 0�5000�point�difference�in�favor�of�group�2�(the�innovative�teaching�method),�whereas�for�the� adjusted� quiz� score� means� (i�e�,� the� ANCOVA� results� which� controlled� for� aptitude),� there� is�a�2�1286�point�difference�in�favor�of�group�2��In�other�words,�the�adjustment�(i�e�, control- ling�for�statistics�aptitude)�in�this�case�resulted�in�a�greater�difference�between�the�adjusted� quiz�score�means�than�between�the�unadjusted�quiz�score�means��Since�there�are�only�two� groups,�an�MCP�is�unnecessary�(although�we�illustrate�this�in�the�SPSS�section)�
14.9 ANCOVA Without Randomization
As� referenced� previously� in� the� discussion� of� assumptions,� there� has� been� a� great� deal� of� discussion� and� controversy� over� the� years,� particularly� in� education� and� the� behav- ioral� sciences,� about� the� use� of� the� ANCOVA� in� situations� where� randomization� is� not� conducted��Randomization�is�defined�as�an�experiment�where�individuals�are�randomly� assigned�to�groups�(or�cells�in�a�factorial�design)��In�the�Campbell�and�Stanley�(1966)�sys- tem�of�experimental�design,�these�designs�are�known�as�true experiments��(Do�not�con- fuse� random� assignment� with� random� selection,� the� latter� of� which� deals� with� how� the� cases�are�sampled�from�the�population�)
In�certain�situations,�randomization�either�has�not�occurred�or�is�not�possible�due�to�cir- cumstances�in�the�study��The�best�example�is�the�situation�where�there�are�intact groups,� which�are�groups�that�have�been�formed�prior�to�the�researcher�arriving�on�the�scene��Either� the�researcher�chooses�not�to�randomly�assign�these�individuals�to�groups�through�a�reas- signment�(e�g�,�it�is�just�easier�to�keep�the�groups�in�their�current�form)�or�the�researcher�can- not�randomly�assign�them�(legally,�ethically,�or�otherwise)��When�randomization�does�not� occur,�the�resulting�designs�are�known�as�quasi-experimental��For�instance,�in�classroom� research,� the� researcher� is� almost� never� able� to� come� into� a� school� and� randomly� assign� students�to�groups��Once�students�are�given�their�class�assignments�at�the�beginning�of�the� year,�this�cannot�be�altered��On�occasion,�the�researcher�might�be�able�to�pull�a�few�students� out�of�several�classrooms,�randomly�assign�them�to�small�groups,�and�conduct�a�true�exper- iment��In�general,�this�is�possible�only�on�a�very�small�scale�and�for�short�periods�of�time�
Let�us�briefly�consider�the�issues�as�it�relates�to�ANCOVA,�as�not�all�statisticians�agree��In�true� experiments�(i�e�,�with�randomization),�there�is�no�cause�for�concern�(except�for�dealing�with� the�statistical�assumptions)��The�ANCOVA�is�more�powerful�and�has�greater�precision�for�true� experiments�than�for�quasi-experiments��So�if�you�have�a�choice,�go�with�a�true�experimental� situation�(which�is�a�big�if)��In�a�true�experiment,�the�probability�that�the�groups�differ�on�the� covariate� or� any� other� concomitant� variable� is� equal� to� � That� is,� the� likelihood�that�the� group�means�will�be�different�on�the�covariate�is�small,�and,�thus,�the�adjustment�in�the�group� means�may�be�small��The�payoff�is�in�the�possibility�that�the�error�term�will�be�greatly�reduced�
In� quasi-experiments,� as� it� relates� to� ANCOVA,� there� are� several� possible� causes� for� concern�� Although� this� is� the� situation� where� the� researcher� needs� the� most� help,� this� is�also�the�situation�where�less�help�is�available��Here�it�is�more�likely�that�there�will�be� statistically�significant�differences�among�the�group�means�on�the�covariate��Thus,�the� adjustment�in�the�group�means�can�be�substantial�(assuming�that�bw�is�different�from�0)��
444 An Introduction to Statistical Concepts
Because�there�are�significant�mean�differences�on�the�covariate,�any�of�the�following�may� occur:�(a)�it�is�likely�that�the�groups�may�be�different�on�other�important�characteristics� as�well,�which�have�not�been�controlled�for�either�statistically�or�experimentally;�(b)�the� homogeneity�of�regression�slopes�assumption�is�less�likely�to�be�met;�(c)�adjusting�for�the� covariate�may�remove�part�of�the�treatment�effect;�(d)�equating�groups�on�the�covariate� may�be�an�extrapolation�beyond�the�range�of�possible�values�that�occur�for�a�particular� group�(e�g�,�the�examples�by�Lord,�1967,�1969,�on�trying�to�equate�men�and�women,�or�by� Ferguson�&�Takane,�1989,�on�trying�to�equate�mice�and�elephants;�these�groups�should� not�be�equated�on�the�covariate�because�their�distributions�on�the�covariate�do�not�over- lap);�(e)�although�the�slopes�may�be�equal�for�the�range�of�Xs�obtained,�when�extrapolat- ing�beyond�the�range�of�scores,�the�slopes�may�not�be�equal;�(f)�the�standard�errors�of�the� adjusted� means� may� increase,� making� tests� of� the� adjusted� means� not� significant;� and� (g)�there�may�be�differential�growth�in�the�groups�confounding�the�results�(e�g�,�adult�vs�� child�groups)�
Although�one�should�be�cautious�about�the�use�of�ANCOVA�in�quasi-experiments,�this�is� not�to�suggest�that�ANCOVA�should�never�be�used�in�such�situations��Just�be�extra�careful� and�do�not�go�too�far�in�terms�of�interpreting�your�results��If�at�all�possible,�replicate�your� study��For�further�discussion,�see�Huitema�(1980),�or�Porter�and�Raudenbush�(1987)�
14.10 More Complex ANCOVA Models
The�one-factor�ANCOVA�model�can�be�extended�to�more-complex�models�in�the�same�way� as�we�expanded�the�one-factor�ANOVA�model��Thus,�we�can�consider�ANCOVA�designs� that� involve� any� of� the� following� characteristics:� (a)� factorial� designs� (i�e�,� having� more� than� one� factor� or� independent� variable);� (b)� fixed-,� random-,� and� mixed-effects� designs;� (c)�repeated�measures�and�split-plot�(mixed)�designs;�(d)�hierarchical�designs;�and�(e)�ran- domized� block� designs�� Conceptually� there� is� nothing� new� for� these� types� of� ANCOVA� designs,�and�you�should�have�no�trouble�getting�a�statistical�package�to�do�such�analyses�� For� further� information� on� these� designs,� see� Huitema� (1980),� Keppel� (1982),� Kirk� (1982),� Myers�and�Well�(1995),�Page,�Braver,�and�MacKinnon�(2003),�or�Keppel�and�Wickens�(2004)�� One�can�also�utilize�multiple�covariates�in�an�ANCOVA�design;�for�further�information,� see� Huitema� (1980),� Kirk� (1982),� Myers� and� Well� (1995),� Page� et� al�� (2003),� or� Keppel� and� Wickens�(2004)�
14.11 Nonparametric ANCOVA Procedures
In� situations� where� the� assumptions� of� normality,� homogeneity� of� variance,� and/or� linearity� have� been� seriously� violated,� one� alternative� is� to� consider� nonparametric� ANCOVA�procedures��Some�rank�ANCOVA�procedures�have�been�proposed�by�Quade� (1967),�Puri�and�Sen�(1969),�Conover�and�Iman�(1982),�and�Rutherford�(1992)��For�a�descrip- tion�of�such�procedures,�see�these�references�as�well�as�Huitema�(1980),�Harwell�(2003),� or�Wilcox�(2003)�
445Introduction to Analysis of Covariance
14.12 SPSS and G*Power
Next�we�consider�SPSS�for�the�statistics�instruction�example��As�noted�in�previous�chap- ters,�SPSS�needs�the�data�to�be�in�a�specific�form�for�the�analysis�to�proceed,�which�is�dif- ferent� from� the� layout� of� the� data� in� Table� 14�1�� For� a� one-factor� ANCOVA� with� a� single� covariate,�the�dataset�must�contain�three�variables�or�columns:�one�for�the�level�of�the�fac- tor�or�independent�variable,�one�for�the�covariate,�and�a�third�for�the�dependent�variable�� The�following�screenshot�presents�an�example�of�the�dataset�for�the�statistics�quiz�score� example�� Each� row� still� represents� one� individual,� displaying� the� level� of� the� factor� (or� independent�variable)�for�which�they�are�a�member,�as�well�as�their�scores�on�the�covariate� and�the�scores�for�the�dependent�variable�
The dependent variable is “quiz” and represents the statistics quiz score.
The covariate is “aptitude” measured prior to the course beginning.
The independent variable is labeled “Group” where each value represents the instructional method to which the
student was assigned (i.e., 1=traditional and 2=innovative).
Step 1:�To�conduct�an�ANCOVA,�go�to�“Analyze”�in�the�top�pulldown�menu,�then�select� “General Linear Model,”�and�then�select�“Univariate.”�Following�the�screenshot� (step�1)�that�follows�produces�the�“Univariate”�dialog�box�
A
B C
ANCOVA: Step 1
446 An Introduction to Statistical Concepts
Step 2:�From�the�“Univariate”�dialog�box�(see�screenshot�step�2),�click�the�depen- dent�variable�(e�g�,�quiz�score)�and�move�it�into�the�“Dependent Variable”�box�by� clicking�the�arrow�button��Click�the�independent�variable�(e�g�,�group)�and�move�it�into� the�“Fixed Factor(s)”� box� by� clicking� the� arrow� button�� Click� the� covariate� (e�g�,� aptitude)�and�move�it�into�the�“Covariate(s)”�box�by�clicking�the�arrow�button��Next,� click�on�“Options.”
Select the dependent variable from the list on the left and use the
arrow to move it to the “Dependent variable” box on the right.
Select the independent variable from the list on the left and use the arrow to move it to the “Fixed Factor(s)” box on the right.
Select the covariate from the list on the left and use the arrow to move it to the “Covariate(s)” box
on the right.
Clicking on “Model” will allow you to
change specifications to
the model.
Clicking on “Plots” will allow you to generate profile
plots.
Clicking on “Save” will allow you to save various forms of residuals,
among other variables.
Clicking on “Options” will allow you to
obtain a number of other statistics (e.g., descriptive statistics,
effect size, power, homogeneity tests).
ANCOVA: Step 2
Step 3:� Clicking� on� “Options”� will� provide� the� option� to� select� such� information� as� “Descriptive Statistics,” “Estimates of effect size,”� “Observed� power,”�and�“Homogeneity tests.”�While�there,�move�the�items�that�are�listed�in�the� “Factor(s) and Factor Interactions:”� box� into� the�“Display Means for:”� box�to�generate�adjusted�means��Also,�check�the�box�“Compare Main Effects,”�then� click�the�pulldown�for�“Confidence interval adjustment”�to�choose�among�the� LSD,�Bonferroni,�or�Sidak�MCPs�of�the�adjusted�means��For�this�illustration,�we�select�the� “Bonferonni.”�Notice�that�the�“Post Hoc”�option�button�from�the�main�“Univariate”� dialog�box�(see�step�2)�is�not�active;�thus,�you�are�restricted�to�the�three�MCPs�just�men- tioned�that�are�accessible�from�this�“Options”�screen��Click�on�“Continue”�to�return� to�the�original�dialog�box�
447Introduction to Analysis of Covariance
Select from the list on the left those variables that you wish to display means for and use the
arrow to move to the “Display Means for” box on the right.
Check the box to “Compare main effects,” then use the pulldown
to select “Bonferroni.”
ANCOVA: Step 3
Step 4: From�the�“Univariate”�dialog�box�(see�step�2),�click�on�“Plots”�to�obtain�a�pro- file�plot�of�means��Click�the�independent�variable�(e�g�,�statistics�course�section,�“Group”)� and�move�it�into�the�“Horizontal Axis”�box�by�clicking�the�arrow�button�(see�screen- shot�step�4a)��Then�click�on�“Add”�to�move�the�variable�into�the�“Plots”�box�at�the�bottom� of�the�dialog�box�(see�screenshot�step�4b)��Click�on�“Continue”�to�return�to�the�original� dialog�box�
Select the independent variable from the list on the
left and use the arrow to move it to the “Horizontal
Axis” box on the right.
ANCOVA: Step 4a
448 An Introduction to Statistical Concepts
Then click “Add” to move the variable
into the “Plots” box at the bottom.
ANCOVA: Step 4b
Step 5:�Finally,�in�order�to�generate�the�appropriate�sources�of�variation�and�results�as� recommended�in�this�chapter,�from�the�main�“Univariate”�dialog�box�(see�step�2),�you�need� to�click�on�the�“Model”�button��Then�select�“Type I”�from�the�“Sum of squares”�pull- down�menu��Click�on�“Continue”�to�return�to�the�original�dialog�box�
You� may� be� asking� yourself� why� we� need� to� utilize� the� Type� I� sum� of� squares,� as� up� until� this� point� in� the� text,� we� have� always� recommended� the� Type� III� (which� is� the�default�in�SPSS)��In�a�study�conducted�by�Li�and�Lomax�(2011),�the�following�were� confirmed� with� SPSS� (as� well� as� with� SAS)�� First,� when� generating� the� Type� I� sum� of� squares,�the�covariate�is�extracted�first,�then�the�treatment�is�estimated�controlling�for� the�covariate��The�Type�I�sum�of�squares�will�also�correctly�add�up�to�the�total�sum�of� squares��Second,�when�generating�the�Type�III�sum�of�squares,�each�effect�is�estimated� controlling�for�each�of�the�other�effects��In�other�words,�the�covariate�is�computed�con- trolling�for�the�treatment,�and�the�treatment�is�determined�controlling�for�the�covari- ate��The�former�is�not�of�interest�as�the�treatment�is�administered�after�the�covariate�has� been� measured;� thus,� no� such� control� is� necessary�� Also,� the� Type� III� sum� of� squares� will� not� add� up� to� the� total� sum� of� squares� as� the� covariate� sum� of� squares� will� be� different� than� when� using� Type� I�� Thus,� you� do� not� want� to� estimate� the� covariate� controlling�for�the�treatment,�and,�thus,�you�want�to�use�the�Type�I,�not�Type�III,�in�the� ANCOVA�context�
449Introduction to Analysis of Covariance
ANCOVA: Step 5
Step 6:�From�the�“Univariate”�dialog�box�(see�step�2),�click�on�“Save”�to�select�those� elements� that� you� want� to� save� (here� we� want� to� save� the� unstandardized� residuals� for� later�use�in�order�to�examine�the�extent�to�which�normality�and�independence�are�met)�� Click�on�“Continue”�to�return�to�the�original�dialog�box��From�the�“Univariate”�dialog� box,�click�on�“OK”�to�return�to�generate�the�output�
ANCOVA: Step 6
Interpreting the output:�Annotated�results�are�presented�in�Table�14�6�
450 An Introduction to Statistical Concepts
Table 14.6
Selected�SPSS�Results�for�the�Statistics�Instruction�Example
Between-subjects factors
Value label N Group 1.00 Traditional
lecture method of
instruction
6
2.00 Small group and
self-directed instruction
6
Descriptive statistics Dependent variable: Quiz score
Group Mean Std. Deviation N
3.5000 1.87083
4.0000 2.09762
Traditional lecture method of instruction Small group and self-directed instruction Total 3.7500 1.91288
6
6
12
Levene’s Test of Equality of Error Variancesa
Dependent variable: Quiz score
F df1 df2 Sig. 6.768 1 10 .026
Tests the null hypothesis that the error variance of the dependent variable is equal across groups. a Design: Intercept + aptitude + group
�e table labeled “Between-Subjects Factors” provides sample
sizes for each of the categories of the independent variable (recall that the
independent variable is the ‘between subjects factor’).
�e table labeled “Descriptive Statistics” provides basic
descriptive statistics (means, standard deviations, and sample sizes) for each
level of the independent variable.
�e F test (and associated p value) for Levene’s Test for Equality of Error Variances is reviewed to
determine if equal variances can be assumed. In this case, we meet the assumption (as p is
greater than α). Note that df 1 is degrees of freedom for the numerator (calculated as J – 1) and df 2
are the degrees of freedom for the denominator (calculated as N – J ).
451Introduction to Analysis of Covariance
Table 14.6 (continued)
Selected�SPSS�Results�for�the�Statistics�Instruction�Example
Dependent variable: Quiz s core
Source Type I Sum of
Squares df F Sig. Partial Eta Squared
Noncent. Param eter
Obs erved Powerb
Corrected model 31.693a 15.846 16.667 .787 .993 Intercept 168.750 177.483 .952 1.000 Aptitude 20.881 21.961 .709 .986 Group 10.812 11.372
.001
.000
.001
.008 .558
33.333 177.483
21.961 11.372 .850
Error .951 Total 209.000 Corrected total
2 1 1 1 9
12 11
a Squared = .787 (Adjus ted R Squared = .740) b Com puted us ing alpha = .05
Partial eta squared is one measure of effect size:
We can interpret this to say that approximately 56% of the variation in the dependent variable (in this case, statistics quiz score) is accounted for by the instructional method when controlling for aptitude.
The row labeled “GROUP” is the independent variable or between groups variable. The between groups mean square (10.812) tells how much observations vary between groups. The degrees of freedom for between groups is J –1 (or 2-1 = 1 here).
The omnibus F test is computed as
The p value for the independent variable F test is .008. This indicates there is a statistically significant difference in quiz scores based on instructional method, controlling for aptitude. The probability of observing these mean differences or more extreme mean differences by chance if the null hypothesis is really true (i.e., if the means really are equal) is substantially less than 1%. We reject the null hypothesis that all the population adjusted means are equal. The p value for the covariate F test is .001. This indicates there is a statistically significant relationship between the covariate (aptitude) and quiz score.
168.750 20.881 10.812
8.557
40.250
The row labeled “Error” is within groups. The within groups mean square tells us how much the observations within the groups vary (i.e., .951). The degrees of freedom for within groups is (N – J – 1) or the sample size minus the number of levels of the independent variable minus one covariate.
The row labeled “corrected total” is the sum of squares total. The degrees of freedom for the total is (N – 1) or the sample size minus one.
Observed power tells whether our test is powerful enough to detect mean differences if they really exist. Power of .850 indicates that the probability of rejecting the null hypothesis if it is really false is about 85%, strong power.
SStotal
SSbetw + SScovR2
R2
=
40.250 10.812 + 20.881
=
.951
10.812 = 11.37==
MSwith
MSbetwF
Tests of Between-Subjects Effects
Mean Square
R squared is listed as a footnote underneath the table. R squared is the ratio of SS between and SS covariate divided by sum of squares total:
10.812 + 8.557 10.812
= SSbetw
SSbetw + SSerror η2p = = .558
= .787
(continued)
452 An Introduction to Statistical Concepts
Table 14.6 (continued)
Selected�SPSS�Results�for�the�Statistics�Instruction�Example
Estimated Marginal Means
1. Grand Mean Dependent variable: Quiz score
Mean
Std. Error
95% Confidence Interval
Lower Bound Upper Bound
3.750a .281 3.113 4.387
a Covariates appearing in the model are evaluated at the following values: Aptitude = 4.6667.
2. Group
Estimates Dependent variable: Quiz score
Group Mean Std. Error
95% Confidence Interval
Lower Bound
Upper Bound
Traditional lecture method of instruction
2.686a .423 3.642
Small group and self-directed instruction
4.814a .423
1.729
3.858 5.771
a Covariates appearing in the model are evaluated at the following values: Aptitude = 4.6667.
�e ‘Grand Mean’ (in this case, 3.750) represents the overall mean, regardless of group membership in the independent variable. �e 95% CI represents the CI of the grand mean.
�e table labeled “Group” provides descriptive statistics for each of the categories of the independent variable, controlling for the covariate (notice that these are NOT the same means reported previously; also note the table footnote). In addition to means, the SE and 95% CI of the means are reported.
453Introduction to Analysis of Covariance
Table 14.6 (continued)
Selected�SPSS�Results�for�the�Statistics�Instruction�Example
Pairwise Comparisons Dependent variable: Quiz score
(I) Group (J) Group
Mean Di�erence
(I–J) Std. Error Sig.a
95% Con�dence interval for di�erencea
Lower Bound
Upper Bound
Traditional lecture method of instruction
Traditional lecture method of instruction
Small group and self-directed instruction
Small group and self-directed instruction
.631 .008
.631
–2.129* –701
2.129* .008
–3.556
.701 3.556
Based on estimated marginal means
*The mean difference is significant at the .05 level. a Adjustment for multiple comparisons: Bonferroni.
‘Mean di�erence’ is simply the di�erence between the adjusted group means of the two groups compared. For example, the mean di�erence of group 1 and group 2, controlling for the covariate, is calculated as 2.686–4.814 = –2.128 (rounded).
Because there are only two groups of the independent variable, the values in the table are the same (in absolute value) for row 1 as compared to row 2 (the exception is that the CI for the di�erence is switched).
‘Sig.’ denotes the observed p value and provides the results of the Bonferroni post hoc procedure. �ere is a statistically signi�cant adjusted mean di�erence between traditional instruction and innovative instruction (i.e., controlling for aptitude).
Because we had only two groups, requesting post hoc results really was not necessarily. We could have reviewed the F test and then the adjusted means to determine which group had the higher adjusted mean. �e pairwise comparison results will become more valuable when the ANCOVA includes independent variables with more than two categories.
(continued)
454 An Introduction to Statistical Concepts
Table 14.6 (continued)
Selected�SPSS�Results�for�the�Statistics�Instruction�Example
Univariate Tests Dependent variable: Quiz score
Sum of Squares df
Mean Square F
Sig. Partial Eta Squared
Noncent. Parameter
Observed Powera
Contrast 1 10.812 11.372 .008 .558 11.372 .850 Error
10.812 8.557 9 .951
The F tests the effect of Group. This test is based on the linearly independent pairwise comparisons among the estimated marginal means. a Computed using alpha = .05
The table labeled “Univariate Tests” is simply another version of the omnibus F test. In the case of one independent variable, the row labeled “Contrast” provides the same results for the independent variable as that presented in the summary table previously. �e results from this table suggest there is a statistically significant difference in adjusted mean quiz score based on instructional method when controlling for aptitude.
The profile plot is a plot of the adjusted means (i.e., controlling for the covariate) against the categories
of the independent variable. This provides visual representation of the extent to which the quiz score means
differ by instructional method when controlling for aptitude.
2.50
3.00
3.50
Es ti
m at
ed m
ar gi
na l m
ea ns
Estimated marginal means of quiz score
4.00
4.50
5.00
Group
Traditional lecture method of instruction
Small group and self-directed instruction
Covariates appearing in the model are evaluated at the following values: aptitude = 4.667
455Introduction to Analysis of Covariance
Examining Data for Assumptions
The� assumptions� that� we� will� test� for� in� our� ANCOVA� model� include� (a)� independence� of� observations,� (b)� homogeneity� of� variance� (this� was� previously� generated;� thus,� you� can� examine� Table� 14�6� for� this� assumption� as� it� will� not� be� reiterated� here),� (c)� normal- ity,� (d)� linearity,� (e)� independence� of� the� covariate� and� the� independent� variable,� and� (f)� homogeneity�of�regression�slopes��We�will�examine�the�assumptions�after�generating�the� ANCOVA�results��This�is�because�many�of�the�tests�for�assumptions�are�based�on�examina- tion�of�the�residuals,�which�were�requested�when�generating�the�ANCOVA�
Independence
If�subjects�have�been�randomly�assigned�to�conditions�(in�other�words,�the�different�lev- els� of� the� independent� variable),� the� assumption� of� independence� has� been� met�� In� this� illustration,�students�were�randomly�assigned�to�instructional�method�(i�e�,�traditional�or� innovative),�and,�thus,�the�assumption�of�independence�was�met��As�we�have�learned�in� previous�chapters,�however,�we�often�use�independent�variables�that�do�not�allow�random� assignment� (e�g�,� intact� groups)�� We� can� plot� residuals� against� levels� of� the� independent� variable� in� a� scatterplot� to� get� an� idea� of� whether� or� not� there� are� patterns� in� the� data� and� thereby� provide� an� indication� of� the� extent� to� which� we� have� met� this� assumption�� Remember�that�these�variables�were�added�to�the�dataset�by�saving�the�unstandardized� residuals�when�we�generated�the�ANCOVA�model�
Note�that�some�researchers�do�not�believe�that�the�assumption�of�independence�can�be� tested�� If� there� is� not� random� assignment� to� groups,� then� these� researchers� believe� this� assumption� has� been� violated—period�� The� plot� that� we� generate� will� give� us� a� general� idea�of�patterns,�however,�in�situations�where�random�assignment�was�not�performed�
The�general�steps�for�generating�a�simple�scatterplot�through�“Scatter/dot”�have�been� presented�in�a�previous�chapter�(e�g�,�Chapter�10),�and�they�will�not�be�reiterated�here��From� the�“Simple Scatterplot”�dialog�screen,�click�the�residual�variable�and�move�it�into�the� “Y Axis”�box�by�clicking�on�the�arrow��Click�the�independent�variable�(e�g�,�group)�and� move�it�into�the�“X Axis”�box�by�clicking�on�the�arrow��Then�click�“OK.”
Interpreting independence evidence:�In�examining�the�scatterplot�for�evidence� of�independence,�the�points�should�fall�relatively�randomly�above�and�below�the�horizon- tal�reference�line�at�0��In�this�example,�the�scatterplot�does�suggest�evidence�of�indepen- dence�with�relative�randomness�of�points�above�and�below�the�horizontal�line�at�0�
1.00
.50
.00
–.50
Re si
du al
fo r q
ui z
–1.00
–1.50
1.00 Group
Group: Traditional lecture method of instruction
456 An Introduction to Statistical Concepts
Normality
Generating normality evidence:�As�alluded�to�earlier�in�the�chapter,�understand- ing� the� distributional� shape,� specifically� the� extent� to� which� normality� is� a� reasonable� assumption,� is� important�� For� the� ANCOVA,� the� distributional� shape� for� the� residuals� should�be�a�normal�distribution��We�can�again�use�“Explore”�to�examine�the�extent�to� which�the�assumption�of�normality�is�met�
The�general�steps�for�accessing�“Explore”�have�been�presented�in�previous�chapters,�and� will�not�be�repeated�here��From�the�“Explore”�dialog�menu�(see�following�screenshot),�click� the�residual�and�move�it�into�the�“Dependent List”�box�by�clicking�on�the�arrow�button�� The�procedures�for�selecting�normality�statistics�were�presented�in�Chapter�6,�and�remain� the�same�here:�Click�on�“Plots”�in�the�upper�right�corner��Place�a�checkmark�in�the�boxes�for� “Normality plots with tests”�and�also�for�“Histogram.”�Then�click�“Continue”�to� return�to�the�main�“Explore”�dialog�box��Then�click�“OK”�to�generate�the�output�
Interpreting normality evidence:� We� have� already� developed� a� good� under- standing�of�how�to�interpret�some�forms�of�evidence�of�normality�including�skewness�and� kurtosis,�histograms,�and�boxplots��Here�we�examine�the�output�for�these�statistics�again�
The�skewness�statistic�of�the�residuals�is�−�237�and�kurtosis�is�−1�024—both�are�within� the�range�of�an�absolute�value�of�2�0,�suggesting�some�evidence�of�normality�(see�“descrip- tives”�output�as�follows)�
Residual for quiz Mean
for mean 5% Trimmed mean Median Variance Std. deviation Minimum Maximum Range
Skewness Kurtosis
95% Con�dence interval Lower bound Upper bound
Descriptives Statistic Std. Error
.25461.0000 –.5604
.5604
.0056
.1357
.778
.88200 –1.46
1.36 2.81 1.51 –.237
–1.024
Interquartile range
1.232 .637
457Introduction to Analysis of Covariance
The�histogram�of�residuals�is�not�what�most�would�consider�normal�in�shape,�and�this� is�largely�an�artifact�of�the�small�sample�size��Because�of�this,�we�will�rely�more�heavily�on� the�other�forms�of�normality�evidence�
3
2
1
Fr eq
ue nc
y
0 –1.50 –1.00 –.50 .00
Residual for quiz .50 1.00 1.50
Histogram
Mean = –5.69E–16 Std. dev. = .882 N = 12
There�are�a�few�other�statistics�that�can�be�used�to�gauge�normality��The�formal�test�of� normality,� the� S–W� test� (SW)� (Shapiro� &� Wilk,� 1965),� provides� evidence� of� the� extent� to� which� our� sample� distribution� is� statistically� different� from� a� normal� distribution�� The� output�for�the�S-W�test�is�presented�as�follows�and�suggests�that�our�sample�distribution� for�residuals�is�not�statistically�significantly�different�than�what�would�be�expected�from�a� normal�distribution�(SW�=��965,�df�=�12,�p�=��854)�
Residual for quiz
Statistic Statisticdf dfSig. Sig.
.85412.965.20012.124
Shapiro–WilkKolmogorov–Smirnova Tests of Normality
a Lilliefors significance correction. *This is a lower bound of the true significance.
Quantile–quantile� (Q–Q)� plots� are� also� often� examined� to� determine� evidence� of� nor- mality�� Q–Q� plots� are� graphs� that� plot� quantiles� of� the� theoretical� normal� distribution� against� quantiles� of� the� sample� distribution�� Points� that� fall� on� or� close� to� the� diagonal� line�suggest�evidence�of�normality��The�Q–Q�plot�of�residuals�shown�as�follows�suggests� relative�normality�
458 An Introduction to Statistical Concepts
2
1
0
Ex pe
ct ed
n or
m al
–1
–2
–2 –1 0 Observed value
1 2
Normal Q–Q plot of residual for quiz
Examination�of�the�following�boxplot�suggests�a�relatively�normal�distributional�shape� of�residuals�and�no�outliers�
1.50
1.00
.50
.00
–.50
–1.00
–1.50
Residual for quiz
Considering�the�forms�of�evidence�we�have�examined,�skewness�and�kurtosis�statistics,� histogram,�the�S-W�test,�the�Q–Q�plot,�and�the�boxplot,�all�suggest�normality�is�a�reason- able�assumption��We�can�be�reasonably�assured�we�have�met�the�assumption�of�normality� of�the�dependent�variable�for�each�group�of�the�independent�variable�
459Introduction to Analysis of Covariance
Linearity
Recall�that�the�assumption�of�linearity�means�that�the�regression�of�the�dependent�vari- able�(i�e�,�“quiz”�in�this�illustration)�on�the�covariate�(i�e�,�“aptitude”)�is�linear��Evidence�of� the�extent�to�which�this�assumption�is�met�can�be�done�by�examining�scatterplots�of�the� dependent�variable�versus�the�covariate—both�overall�and�also�for�each�category�or�group� of�the�independent�variable�
Linearity evidence: Overall.� The� general� steps� for� generating� a� simple� scatter- plot�through�“Scatter/dot”�have�been�presented�in�a�previous�chapter�(e�g�,�Chapter�10),� and�they�will�not�be�reiterated�here��To�generate�the�overall�scatterplot,�from�the�“Simple Scatterplot”�dialog�screen,�click�the�dependent�variable�and�move�it�into�the�“Y Axis”� box� by� clicking� on� the� arrow�� Click� the� covariate� (e�g�,� aptitude)� and� move� it� into� the� “X Axis”�box�by�clicking�on�the�arrow��Then�click�“OK.”
Interpreting independence of linearity (overall):�In�examining�the�scat- terplot�for�overall�evidence�of�linearity,�the�points�should�fall�relatively�linearly�(in�other� words,�we�should�not�be�seeing�a�curvilinear�or�some�other�nonlinear�relationship)��In�this� example,�our�scatterplot�suggests�we�have�evidence�of�overall�linearity�as�there�is�a�rela- tively�clear�pattern�of�points�which�suggest�a�positive�and�linear�relationship�between�the� dependent�variable�and�covariate�
6.00
5.00
4.00
3.00Q ui
z sc
or e
2.00
1.00
.00 2.00 4.00 6.00 Aptitude
8.00 10.00
R2 linear = 0.519
Linearity evidence: By group of independent variable.�To�generate�the�scat- terplot�of�the�dependent�variable�and�covariate�for�each�group�of�the�independent�variable,� we�must�first�split�the�data�file��To�do�this,�go�to “Data”�in�the�top�pulldown�menu��Then� select�“Split File.”
460 An Introduction to Statistical Concepts
A
B
Linearity evidence by group of independent variable
From� the�“Split File”� dialog� screen,� select� the� radio� button� for�“Organize out- put by groups,”�and�then�click�the�independent�variable�and�move�it�into�the�“Groups Based on”�box�by�clicking�on�the�arrow��Then�click�“OK.”
Click the radio button for “Organize
output by groups.”
Select the independent variable from the list
on the left and use the arrow to move it to the “Groups Based on”
box on the right.
After�splitting�the�file,�the�next�step�is�to�generate�the�scatterplot�of�the�dependent�variable� by�covariate��Because�we�have�split�the�file,�there�will�be�two�scatterplots�generated:�one�for� the�traditional�teaching�method�and�one�for�the�innovative�teaching�method��The�general�
461Introduction to Analysis of Covariance
steps�for�generating�a�simple�scatterplot�through�“Scatter/dot”�have�been�presented�in� a�previous�chapter�(e�g�,�Chapter�10),�and�they�will�not�be�repeated�here��Because�we�have� just�generated�the�overall�scatterplot,�the�selections�made�previously�will�remain,�and,�thus,� from�the�“Simple Scatterplot”�dialog�screen,�simply�click�“OK”�to�generate�the�output�
Interpreting evidence of linearity (by group of independent vari- able):�In�examining�the�scatterplot�for�evidence�of�linearity�by�group�of�the�independent� variable,�our�interpretation�should�remain�the�same:�the�points�should�fall�relatively�lin- early� (in� other� words,� we� should� not� see� a� curvilinear� or� some� other� nonlinear� relation- ship)��In�this�example,�our�scatterplots�suggest�we�have�evidence�of�linearity�by�group�of� the� independent� variable� as� there� is� a� relatively� clear� pattern� of� points� which� suggest� a� positive� and� linear� relationship� between� the� dependent� variable� and� covariate� for� each� group�of�the�independent�variable�
6.00
5.00
4.00
3.00Q ui
z sc
or e
2.00
1.00
3.00 4.00 5.00 6.00 Aptitude
7.00 8.00 9.00
Group: Traditional lecture method of instruction
R2 linear = 0.884
6.00
5.00
4.00
3.00Q ui
z sc
or e
2.00
1.00
1.00 2.00 3.00 4.00 Aptitude
5.00 6.00 7.00
Group: Small group and self-directed instruction
R2 linear = 0.703
462 An Introduction to Statistical Concepts
Independence of Covariate and Independent Variable
Recall� the� assumption� of� independence� of� the� covariate� and� independent� variable�� In� other�words,�the�levels�of�the�independent�variable�should�not�differ�on�the�covariate��If� subjects� have� been� randomly� assigned� to� conditions� (in� other� words,� the� different� lev- els� of� the� independent� variable),� the� assumption� of� independence� of� the� covariate� and� independent� variable� has� likely� been� met�� In� this� illustration,� students� were� randomly� assigned� to� teaching� method� (i�e�,� traditional� or� innovative),� and,� thus,� the� assumption� of�independence�of�the�covariate�and�independent�variable�was�likely�met��As�we�have� learned�in�previous�chapters,�however,�we�often�use�independent�variables�that�do�not� allow�random�assignment��Evidence�of�the�extent�to�which�this�assumption�is�met�can� be� done� by� examining� mean� differences� on� the� covariate� based� on� the� independent� variable�� If� the� independent� variable� has� only� two� levels,� an� independent� t� test� would� be� appropriate�� If� the� independent� variable� has� more� than� two� categories,� a� one-way� ANOVA�would�suffice��If�the�groups�are�not�statistically�different�on�the�covariate,�then� that�lends�evidence�that�the�assumption�of�independence�of�the�covariate�and�the�inde- pendent�variable�has�been�met�
We� have� two� levels� of� our� independent� variable;� thus,� we� will� generate� an� indepen- dent� t� test�� The� general� steps� for� generating� an� independent� t� test� have� been� presented� in� Chapter� 8,� and� they� will� not� be� reiterated� here�� From� the�“Independent Samples T Test”� dialog� screen,� click� the� covariate� (e�g�,� aptitude)� and� move� it� into� the� “Test Variable(s)”�box�by�clicking�on�the�arrow��Click�the�independent�variable�(e�g�,�group)� and� move� it� into� the� “Grouping Variable”� box� by� clicking� on� the� arrow�� Click� the� “Define Groups”� box� and� enter� “1”� for� “Group� 1”� and� “2”� for� “Group� 2�”� Then� click� “Continue”�to�return�to�the�main�“Independent Samples T Test”�dialog�screen,�and� click�on�“OK”�to�generate�the�output�
Interpreting independence of covariate and independent variable evidence:�In�examining�the�independent�t�test�results,�evidence�of�independence�of�the� covariate�and�independent�variable�is�provided�when�the�test�results�are�not�statistically� significant��In�this�example,�our�results�suggest�we�have�evidence�of�independence�of�the� covariate� and� independent� variable� as� the� results� are� not� statistically� significant,� t(10)� =� 1�604,� p� =� �140�� Thus,� we� have� likely� met� this� assumption� through� random� assignment� of�cases�to�groups,�and�this�provides�further�confirmation�that�we�have�not�violated�the� assumption�of�independence�of�the�covariate�and�independent�variable�
Independent�Samples�Test
Levene’s Test for
Equality of Variances
t-Test for Equality of Means
t df
Sig. (Two-
Tailed) Mean
Difference Std. Error Difference
95% Confidence
Interval of the Difference
F Sig. Lower Upper
Aptitude Equal� variances� assumed
Equal� variances� not�assumed
�000 1�000 1�604 10 �140 2�00000 1�24722 −�77898 4�77898
1�604 10�000 �140 2�00000 1�24722 −�77898 4�77898
463Introduction to Analysis of Covariance
Homogeneity of Regression Slopes
Step 1:�In�order�to�test�the�homogeneity�of�slopes�assumption,�you�will�need�to�rerun� the�ANCOVA�analysis��Keep�every�screen�the�same�as�before,�with one exception��Return�to� the�main�“Univariate”�dialog�box�(see�step�2)�and�click�on�“Model.”�From�the�“Model”� dialog�box,�click�on�the “Custom”�button�to�build�a�custom�model�to�include�the�inter- action� between� the� independent� and� covariate� variables�� To� do� this,� under� the�“Build Terms”�pulldown�in�the�middle�of�the�dialog�box,�select�“Main effects.”
Step 1: Generating homogeneity of
regression slopes evidence
Step 2:�Click�the�independent�variable�and�move�it�into�the�“Model”�box�by�clicking�on� the�arrow�button��Next,�click�the�covariate�and�move�it�into�the�“Model”�box�by�clicking� on�the�arrow�button��This�will�place�“Group”�and�“Aptitude”�in�the “Model”�box�on�the� right�of�the�screen�
For the main effects, select the
independent variable and covariate from
the list on the left and use the arrow to
move them to the “Model” box on the
right.
Step 2: Generating homogeneity of regression
slopes evidence
464 An Introduction to Statistical Concepts
Step 3:�Then�from�the�“Build Terms”�pulldown�menu,�select�“Interaction.”
Step 3: Generating homogeneity of
regression slopes evidence
Step 4:�Click�both�variables�at�the�same�time�(e�g�,�using�the�shift�key)�and�use�the�arrow�key�to� move�the�interaction�of�Aptitude�*�Group�into�the�“Model”�box�on�the�right��There�should�now� be�three�terms�in�the�Model�box:�the�interaction�and�two�main�effects��Then�click�“Continue”� to�return�to�the�main�“Univariate”�dialog�box��Then�click�“OK”�to�generate�the�output�
For the interaction, select both the
independent variable and covariate from
the list on the left and use the arrow to
move them to the “Model” box on the
right.
Step 4: Generating homogeneity of regression
slopes evidence
Interpreting homogeneity of regression slopes evidence:�Selected�results,� specifically�the�ANCOVA�summary�table�which�presents�the�results�for�the�homogeneity�of� slopes�test,�are�presented�as�follows��Here�the�only�thing�that�we�care�about�is�the�test�of�the� interaction,�which�we�want�to�be�nonsignificant�[and�we�find�this�to�be�the�case:�F(1,�8)�=��000,� p�=�1�000]��This�indicates�that�we�have�met�the�homogeneity�of�regression�slopes�assumption�
465Introduction to Analysis of Covariance
Dependent Variable: Quiz Score
Source
Corrected model Intercept Group Aptitude
Group*Aptitude Error Total Corrected total
a R squared = .787 (adjusted R squared = .708).
Type I Sum of Squares df F Sig.
Partial Eta Squared
Mean Square
Tests of Between-Subjects E�ects
Noncent Parameter
Observed Powerb
.955 1.000
.115
.997
.050.000
28.928 .701
157.763 29.629.005
.000
.427
.001
1.000
.787
.952
.081
.783
.000.000
28.928 .701
157.763 9.87631.693a
168.750 .750
30.943
.000 8.557
209.000 40.250
3 1 1 1
1 8
12 11
10.564 168.750
.750 30.943
.000 1.070
b Computed using alpha = .05.
Post Hoc Power for ANCOVA Using G*Power
Generating�power�analysis�for�ANCOVA�models�follows�similarly�to�that�for�ANOVA�and� factorial�ANOVA��In�particular,�if�there�is�more�than�one�independent�variable,�we�must� test�for�main�effects�and�interactions�separately��Because�we�only�have�one�independent� variable�for�our�ANCOVA�model,�our�illustration�assumes�only�one�main�effect��If�there� were�additional�independent�variables�and/or�interactions,�we�would�have�followed�these� steps�for�those�as�well�
The�first�thing�that�must�be�done�when�using�G*Power�for�computing�post�hoc�power�is�to� select�the�correct�test�family��In�our�case,�we�conducted�an�ANCOVA��To�find�ANCOVA,�we� will�select�“Tests”�in�the�top�pulldown�menu,�then “Means,”�and�then�“Many groups: ANCOVA: Main effects and interactions.”�Once�that�selection�is�made,�the�“Test family”�automatically�changes�to�“F tests.”
A
B
C
Step 1
466 An Introduction to Statistical Concepts
The�“Type of Power Analysis”�desired�then�needs�to�be�selected��To�compute�post� hoc� power,� we� need� to� select�“Post hoc: Compute achieved power—given α, sample size, and effect size.”
�e default selection for “Test
Family” is “t tests”.
Following the procedures
presented in Step 1 will automatically
change the test family to“F tests”.
�e default selection for “Statistical Test” is “Correlation: Point biserial model”. Following the procedures presented in Step 1 will automatically change the statistical test to “ANCOVA: Fixed effects, main effects and interactions”.
Click on “Determine” to pop out the effect size calculator box
(shown below). �is will allow you to
compute f given partial eta squared.
Step 2
Once the parameters are
specified, click on “Calculate”.�e “Input Parameters” for computing
post hoc power must be specified (the default values are shown here) including:
1. Effect size f 2. α level 3. Total sample size 4. Numerator df 5. Number of groups 6. Number of convariates
The�“Input Parameters”�must�then�be�specified��We�will�compute�the�effect�size�f�last,� so�we�skip�that�for�the�moment��In�our�example,�the�alpha�level�we�used�was��05,�and�the� total�sample�size�was�12��The�numerator degrees of freedom�for�group�(our�independent�vari- able)�are�equal�to�the�number�of�categories�of�this�variable�(i�e�,�2)�minus�1;�thus,�there�is� one� degree� of� freedom� for� the� numerator�� The� number of groups� equals,� in� the� case� of� an� ANCOVA� with� multiple� independent� variables,� the� product� of� the� number� of� levels� or� categories�of�the�independent�variables�or�(J)(K)��In�this�example,�we�have�only�one�inde- pendent�variable��Thus,�the�number�of�groups�when�there�is�only�one�independent�variable� is�equal�to�the�number�of�categories�of�this�independent�variable�(i�e�,�2)��The�last�param- eter�that�must�be�inputted�is�the�number�of�covariates��In�this�example,�we�have�only�one� covariate;�thus,�we�enter�1�in�this�box�
We�skipped�filling�in�the�first�parameter,�the�effect�size�f,�for�a�reason��SPSS�only�pro- vides�a�partial�eta�squared�measure�of�effect�size��Thus,�we�will�use�the�pop-out�effect� size�calculator�in�G*Power�to�compute�the�effect�size�f�(we�saved�this�parameter�for�last� as�the�calculation�is�based�on�the�previous�values�just�entered)���To�pop�out�the�effect�size�
467Introduction to Analysis of Covariance
calculator,� click� on� “Determine”� which� is� displayed� under� “Input Parameters.”� In� the� pop-out� effect� size� calculator,� click� on� the� radio� button� for� “Direct”� and� then�enter�the�partial�eta�squared�value�for�group�that�was�calculated�in�SPSS�(i�e�,��558)�� Clicking� on�“Calculate”� in� the� pop-out� effect� size� calculator� will� calculate� the� effect� size�f��Then�click�on�“Calculate and Transfer to Main window”�to�transfer�the� calculated�effect�size�(i�e�,�1�1235851)�to�the�“Input Parameters.”�Once�the�parameters� are�specified,�click�on�“Calculate”�to�find�the�power�statistics�
Post hoc power
Here are the post-hoc power results.
The�“Output Parameters”�provide�the�relevant�statistics�given�the�input�just�speci- fied��In�this�example,�we�were�interested�in�determining�post�hoc�power�for�an�ANCOVA� with� a� computed� effect� size� f� of� 1�1235851,� an� alpha� level� of� �05,� total� sample� size� of� 12,� numerator�degrees�of�freedom�of�1,�two�groups,�and�one�covariate�
Based�on�those�criteria,�the�post�hoc�power�for�the�main�effect�of�instructional�method�(i�e�,� our�only�independent�variable)�was��93��In�other�words,�with�an�ANCOVA,�computed�effect� size�f�of�1�124,�alpha�level�of��05,�total�sample�size�of�12,�numerator�degrees�of�freedom�of�1,� two�groups,�and�one�covariate,�the�post�hoc�power�of�our�main�effect�for�this�test�was��93— the�probability�of�rejecting�the�null�hypothesis�when�it�is�really�false�(in�this�case,�the�prob- ability�that�the�adjusted�means�of�the�dependent�variable�would�be�equal�for�each�level�of�the�
468 An Introduction to Statistical Concepts
independent�variable,�controlling�for�the�covariate)�was�about�93%,�which�would�be�consid- ered�more�than�sufficient�power�(sufficient�power�is�often��80�or�above)��Note�that�this�value� differs�slightly�than�that�reported�in�SPSS��Keep�in�mind�that�conducting�power�analysis�a� priori�is�recommended�so�that�you�avoid�a�situation�where,�post�hoc,�you�find�that�the�sample� size�was�not�sufficient�to�reach�the�desired�level�of�power�(given�the�observed�parameters)�
A Priori Power for ANCOVA Using G*Power
For� a� priori� power,� we� can� determine� the� total� sample� size� needed� for� the� main� effects� and/or�interactions�given�an�estimated�effect�size�f,�alpha�level,�desired�power,�numerator� degrees�of�freedom�(i�e�,�number�of�categories�of�our�independent�variable�and/or�interac- tion,�depending�on�which�a�priori�power�we�are�interested�in�and�depending�on�the�number� of�independent�variables),�number�of�groups�(i�e�,�the�number�of�categories�of�the�indepen- dent�variable�in the case of only one independent variable�OR�the�product�of�the�number�of�levels� of�the�independent�variables�in the case of multiple independent variables),�and�the�number�of� covariates��We�follow�Cohen’s�(1988)�conventions�for�effect�size�(i�e�,�small,�f�=��10;�moderate,� f =��25;�large,�f�=��40)��In�this�example,�had�we�estimated�a�moderate�effect�f�of��25,�alpha�of��05,� desired�power�of��80,�numerator�degrees�of�freedom�of�1�(two�categories�in�our�independent� variable�thus�2�−�1�=�1),�number�of�groups�of�2�(i�e�,�there�is�only�one�independent�variable,� and�there�were�two�categories),�and�one�covariate,�we�would�need�a�total�sample�size�of�9�
A priori power
Here are the a priori power
results.
469Introduction to Analysis of Covariance
14.13 Template and APA-Style Paragraph
Finally�we�come�to�an�example�paragraph�of�the�results�for�the�statistics�instruction�exam- ple��Recall�that�our�graduate�research�assistant,�Marie,�was�building�on�work�that�she�had� conducted�as�part�of�a�research�project�for�an�independent�study�class�and�had�now�con- ducted� a� second� experiment�� She� was� looking� to� see� if� there� was� a� mean� difference� in� statistics�quiz�scores�based�on�the�instructional�method�of�the�class�(two�categories:�tradi- tional�or�innovative)�while�controlling�for�aptitude��Her�research�question�was�the�follow- ing:� Is there a mean difference in statistics quiz scores based on teaching method, controlling for aptitude?�Marie�then�generated�an�ANCOVA�as�the�test�of�inference��A�template�for�writing� a�research�question�for�ANCOVA�is�presented�as�follows:
Is there a mean difference in [dependent variable] based on [inde- pendent variable], controlling for [covariate]?
This�is�illustrated�assuming�a�one-factor�(i�e�,�one�independent�variable)�model,�but�it�can� easily�be�extended�to�two�or�more�factors��As�we�noted�in�previous�chapters,�it�is�important� to�be�sure�the�reader�understands�the�levels�or�groups�of�the�independent�variables��This� may�be�done�parenthetically�in�the�actual�research�question,�as�an�operational�definition,� or� specified� within� the� methods� section�� In� this� example,� parenthetically� we� could� have� stated�the�following:�Is there a mean difference in statistics quiz scores based on teaching method (traditional vs. innovative), controlling for aptitude?
It�may�be�helpful�to�preface�the�results�of�the�ANCOVA�with�information�on�an�examina- tion�of�the�extent�to�which�the�assumptions�were�met�(recall�there�are�several�assumptions� that�we�tested:�(a)�independence�of�observations,�(b)�homogeneity�of�variance,�(c)�normal- ity,� (d)� linearity,� (e)� independence� of� the� covariate� and� the� independent� variable,� and� (f)� homogeneity�of�regression�slopes):
An ANCOVA was conducted to determine if the mean statistics quiz score differed based on the instructional method of the statistics course (traditional vs. innovative) while controlling for aptitude. Independence of observations was met by random assignment of stu- dents to instructional method. This assumption was also confirmed by review of a scatterplot of residuals against the levels of the inde- pendent variable. A random display of points around 0 provided fur- ther evidence that the assumption of independence was met. According to Levene’s test, the homogeneity of variance assumption was not satisfied [F(1, 10) = 6.768, p = .026]. However, research suggests that violation of homogeneity is minimal when the groups of the indepen- dent variable are equal in size (Harwell, 2003), as in the case of this study. The assumption of normality was tested and met via exami- nation of the residuals. Review of the S-W test for normality (SW = .965, df = 12, p = .854) and skewness (−.237) and kurtosis (−1.024) statistics suggested that normality was a reasonable assumption. The boxplot and histogram suggested a relatively normal distributional shape (with no outliers) of the residuals. The Q–Q plot suggested normality was reasonable. In general, there is evidence that nor- mality has been met. Linearity of the dependent variable with the
470 An Introduction to Statistical Concepts
covariate was examined with scatterplots, both overall and by group of the independent variable. Overall, the scatterplot of the depen- dent variable with the covariate suggested a positive linear rela- tionship. This same pattern was present for the scatterplot of the dependent variable with the covariate when disaggregated by the cat- egories of the independent variables. Independence of the covariate and independent variable was met by random assignment of students to instructional method. This assumption was also confirmed by an inde- pendent t test which examined the mean difference on the covariate (i.e., aptitude) by independent variable (i.e., teaching method). The results were not statistically significant, t(10) = 1.604, p = .140, which further confirms evidence of independence of the covariate and independent variable. There was not a mean difference in statistics aptitude based on teaching method. Homogeneity of regression slopes was suggested by similar regression lines evidenced in the scatter- plots of the dependent variable and covariates by group (reported earlier as evidence for linearity). This assumption was confirmed by a nonstatistically significant interaction of aptitude by group, F(1, 8) = .000, p = 1.000.
Here�is�an�APA-style�example�paragraph�of�results�for�the�ANCOVA�(remember�that�this� will� be� prefaced� by� the� previous� paragraph� reporting� the� extent� to� which� the� ANCOVA� assumptions�were�met):
The results of the ANCOVA suggest a statistically significant effect of the covariate, aptitude, on the dependent variable, statistics quiz score (Faptitude = 21.961; df = 1,9; p = .001). More importantly, there is a statistically significant effect for instructional method (Fgroup = 11.372; df = 1,9; p = .008), with a large effect size and strong power (partial η2group = .558, observed power = .850). The effect size suggests that about 56% of the variance in statistics quiz scores can be accounted for by teaching method when controlling for aptitude.
The� unadjusted� group� statistics� quiz� score� mean� (i�e�,� prior� to� controlling� for� aptitude)� was� larger� for� the� innovative� instruction� group� (M� =� 4�00,� SD� =� 2�10)� as� compared� to� the�traditional�lecture�method�(M�=�3�50,�SD�=�1�87)�by�only��50��However,�the�adjusted mean�for�the�innovative�instruction�group�(M�=�4�814,�SE�=��423)�as�compared�to�the�tra- ditional�lecture�method�(M�=�2�686,�SE�=��423)�was�larger�by�2�128��Thus,�the�use�of�the� covariate�resulted�in�a�large�significant�difference�between�the�instructional�groups��In� summary,�students�assigned�to�the�innovative�teaching�method�outperformed�students� in�the�traditional�lecture�method�on�the�statistics�quiz�score�when�controlling�for�sta- tistics�aptitude�
If�our�independent�variable�had�more�than�two�groups,�we�would�have�needed�to�evalu- ate�and�report�the�results�of�a�post�hoc�MCP�when�generating�SPSS�(recall�that�we�asked� for�Bonferroni�post�hoc�results)��The�following�provides�a�template�for�how�these�results� may�have�been�written,�had�our�analyses�required�them:
Follow-up tests were conducted to evaluate the pairwise differences among the adjusted means of [dependent variable] based on [indepen- dent variable]. The [post hoc procedure selected, e.g., Bonferroni]
471Introduction to Analysis of Covariance
was applied to control for the risk of increased Type I error across all pairwise comparisons. Pairwise comparisons revealed [report spe- cific results, including means and standard deviations here].
14.14 Summary
In�this�chapter,�methods�involving�the�comparison�of�adjusted�group�means�for�a�single� independent�variable�were�considered��The�chapter�began�with�a�look�at�the�unique�char- acteristics�of�the�ANCOVA,�including�(a)�statistical�control�through�the�use�of�a�covariate,� (b)�the�dependent�variable�means�adjusted�by�the�covariate,�(c)�the�covariate�used�to�reduce� error�variation,�(d)�the�relationship�between�the�covariate�and�the�dependent�variable�taken� into�account�in�the�adjustment,�and�(e)�the�covariate�measured�at�least�at�the�interval�level�� The� layout� of� the� data� was� shown,� followed� by� an� examination� of� the� ANCOVA� model,� and�the�ANCOVA�summary�table��Next�estimation�of�the�adjusted�means�was�considered� along� with� several� different� MCPs�� Some� discussion� was� also� devoted� to� the� ANCOVA� assumptions,�their�assessment,�and�how�to�deal�with�assumption�violations��We�illustrated� the� use� of� the� ANCOVA� by� looking� at� an� example�� Finally,� we� finished� off� the� chapter� by� briefly� examining� (a)� some� cautions� about� the� use� of� ANCOVA� in� situations� without� randomization,�(b)�ANCOVA�for�models�having�multiple�factors�and/or�multiple�covari- ates,� (c)� nonparametric� ANCOVA� procedures,� and� (d)� SPSS� and� G*Power�� At� this� point,� you�should�have�met�the�following�objectives:�(a)�be�able�to�understand�the�characteristics� and�concepts�underlying�ANCOVA;�(b)�be�able�to�determine�and�interpret�the�results�of� ANCOVA,�including�adjusted�means�and�MCPs;�and�(c)�be�able�to�understand�and�evalu- ate�the�assumptions�of�ANCOVA��Chapter�15�takes�us�beyond�the�fixed-effects�models�we� have�discussed�thus�far�and�considers�random-�and�mixed-effects�models�
Problems
Conceptual problems
14.1� �Malani�wants�to�determine�whether�children�whose�preschool�classroom�has�a�win- dow�differ�in�their�receptive�vocabulary�as�compared�to�children�whose�classroom� does� not� have� a� window�� At� the� beginning� of� the� school� year,� Malani� randomly� assigns� 10� children� at� Rainbow� Butterfly� Preschool� to� one� of� two� different� class- rooms:�one�classroom�has�a�window�that�looks�out�onto�a�grassy�area,�and�the�other� classroom�has�no�windows��At�the�end�of�the�school�year,�Malani�measures�children� on�their�receptive�vocabulary��Is�ANCOVA�appropriate�given�this�scenario?
14.2� �Joe� wants� to� determine� whether� the� time� to� run� the� Magic� Mountain� Marathon� (ratio�level�variable)�differs,�on�average,�for�nonprofessional�athletes�who�complete� a� 12� week� endurance� training� program� as� compared� to� those� who� complete� a� 4� week�endurance�training�program��Joe�randomly�assigns�nonprofessional�athletes� to�one�of�the�two�training�programs��In�conducting�this�experiment,�Joe�also�wants� to� control� for� the� number� of� prior� marathons� in� which� the� participant� has� run�� Is� ANCOVA�appropriate�given�this�scenario?
472 An Introduction to Statistical Concepts
14.3� �Tami� has� generated� an� ANCOVA�� In� testing� the� assumptions,� she� reviews� a� scat- terplot� of� the� residuals� for� each� category� of� the� independent� variable�� For� which� assumption�is�Tami�likely�reviewing�evidence?
� a�� Homogeneity�of�regression�slopes
� b�� Homogeneity�of�variance
� c�� Independence�of�observations
� d�� Independence�of�the�covariate�and�the�independent�variable
� e�� Linearity
14.4� �Wesley�has�generated�an�ANCOVA��In�his�model,�there�is�one�independent�vari- able� which� has� three� categories� (type� of� phone:� Blackberry,� iPhone,� and� Droid)� and� one� covariate� (amount� of� time� spent� on� desktop� or� laptop� computer)�� In� testing�the�assumptions,�he�reviews�a�one-way�ANOVA,�the�dependent�variable� being�amount�of�time�spent�on�desktop�or�laptop�computer�and�the�independent� variable�being�type�of�phone��For�which�assumption�is�Wesley�likely�reviewing� evidence?
� a�� Homogeneity�of�regression�slopes
� b�� Homogeneity�of�variance
� c�� Independence�of�observations
� d�� Independence�of�the�covariate�and�the�independent�variable
� e�� Linearity
14.5� �If� the� correlation� between� the� covariate� X� and� the� dependent� variable� Y� differs� markedly�in�the�two�treatment�groups,�it�seems�likely�that
� a�� The�assumption�of�normality�is�suspect�
� b�� The�assumption�of�homogeneity�of�slopes�is�suspect�
� c�� A�nonlinear�relation�exists�between�X�and�Y�
� d�� The�adjusted�means�for�Y�differ�significantly�
14.6� �If�for�both�the�treatment�and�control�groups�the�correlation�between�the�covariate� X�and�the�dependent�variable�Y�is�substantial�but�negative,�the�error�variation�for� ANCOVA�as�compared�to�that�for�ANOVA�is
� a�� Less
� b�� About�the�same
� c�� Greater
� d�� Unpredictably�different
14.7� �An�experiment�was�conducted�to�compare�three�different�instructional�strategies�� Fifteen�subjects�were�included�in�each�group��The�same�test�was�administered�prior� to�and�after�the�treatments��If�both�pretest�and�IQ�are�used�as�covariates,�what�are� the�degrees�of�freedom�for�the�error�term?
� a�� 2
� b�� 40
� c�� 41
� d�� 42
473Introduction to Analysis of Covariance
14.8� �The� effect� of� a� training� program� concerned� with� educating� heart� attack� patients� to�the�benefits�of�moderate�exercise�was�examined��A�group�of�recent�heart�attack� patients� was� randomly� divided� into� two� groups;� one� group� received� the� training� program� and� the� other� did� not�� The� dependent� variable� was� the� amount� of� time� taken�to�jog�three�laps,�with�the�weight�of�the�patient�after�the�program�used�as�a� covariate��Examination�of�the�data�after�the�study�revealed�that�the�covariate�means� of� the� two� groups� differed�� Which� of� the� following� assumptions� is� most� clearly� violated?
� a�� Linearity � b�� Homogeneity�of�slopes � c�� Independence�of�the�treatment�and�the�covariate � d�� Normality 14.9� In�ANCOVA,�the�covariate�is�a�variable�which�should�have�a � a�� Low,�positive�correlation�with�the�dependent�variable � b�� High,�positive�correlation�with�the�independent�variable � c�� High,�positive�correlation�with�the�dependent�variable � d�� Zero�correlation�with�the�dependent�variable 14.10� �In�ANCOVA,�how�will�the�correlation�of�0�between�the�covariate�and�the�dependent�
variable�appear? � a�� Unequal�group�means�on�the�dependent�variable � b�� Unequal�group�means�on�the�covariate � c�� Regression�of�the�dependent�variable�on�the�covariate�with�bw�=�0 � d�� Regression�of�the�dependent�variable�on�the�covariate�with�bw�=�1 14.11� Which�of�the�following�is�not�a�necessary�requirement�for�using�ANCOVA? � a�� Covariate�scores�are�not�affected�by�the�treatment� � b�� �There�is�a�linear�relationship�between�the�covariate�and�the�dependent�variable� � c�� The�covariate�variable�is�the�same�measure�as�the�dependent�variable� � d�� Regression�slopes�for�the�groups�are�similar� 14.12� Which�of�the�following�is�the�most�desirable�situation�to�use�ANCOVA? � a�� The�slope�of�the�regression�line�equals�0� � b�� �The�variance�of�the�dependent�variable�for�a�specific�covariate�score�is�relatively�
large� � c�� The�correlation�between�the�covariate�and�the�dependent�variable�is�−�95� � d�� The�correlation�between�the�covariate�and�the�dependent�variable�is��60� 14.13� �A� group� of� students� were� randomly� assigned� to� one� of� three� instructional� strat-
egies�� Data� from� the� study� indicated� an� interaction� between� slope� and� treatment� group��It�seems�likely�that
� a�� The�assumption�of�normality�is�suspect� � b�� The�assumption�of�homogeneity�of�slopes�is�suspect� � c�� A�nonlinear�relation�exists�between�X�and�Y� � d�� The�covariate�is�not�independent�of�the�treatment�
474 An Introduction to Statistical Concepts
14.14� �If�the�mean�on�the�dependent�variable�GPA�(Y)�for�persons�of�middle�social�class� (X)� is� higher� than� for� persons� of� lower� and� higher� social� classes,� one� would� expect�that
� a�� The�relationship�between�X�and�Y�is�curvilinear� � b�� The�covariate�X�contains�substantial�measurement�error� � c�� GPA�is�not�normally�distributed� � d�� Social�class�is�not�related�to�GPA� 14.15� �If�both�the�covariate�and�the�dependent�variable�are�assessed�after�the�treatment�has�
been�concluded,�and�if�both�are�affected�by�the�treatment,�the�use�of�ANCOVA�for� these�data�would�likely�result�in
� a�� An�inflated�F�ratio�for�the�treatment�effect � b�� An�exaggerated�difference�in�the�adjusted�means � c�� An�underestimate�of�the�treatment�effect � d�� An�inflated�value�of�the�slope�bw 14.16� �When� the� covariate� correlates� +�5� with� the� dependent� variable,� I� assert� that� the�
adjusted�MSwith�from�the�ANCOVA�will�be�less�than�the�MSwith�from�the�ANOVA�� Am�I�correct?
14.17� �For� each� of� two� groups,� the� correlation� between� the� covariate� and� the� dependent� variable�is�substantial,�but�negative�in�direction��I�assert�that�the�error�variance�for� ANCOVA,�as�compared�to�that�for�ANOVA,�is�greater��Am�I�correct?
14.18� In�ANCOVA,�X�is�known�as�a�factor��True�or�false? 14.19� �A� study� was� conducted� to� compare� six� types� of� diets�� Twelve� subjects� were�
included�in�each�group��Their�weights�were�taken�prior�to�and�after�treatment��If� pre-weight�is�used�as�a�covariate,�what�are�the�degrees�of�freedom�for�the�error� term?
� a�� 5 � b�� 65 � c�� 66 � d�� 71 14.20� �A� researcher� conducts� both� a� one-factor� ANOVA� and� a� one-factor� ANCOVA� on�
the�same�data��In�comparing�the�adjusted�group�means�to�the�unadjusted�group� means,�they�find�that�for�each�group,�the�adjusted�mean�is�equal�to�the�unadjusted� mean�� I� assert� that� the� researcher� must� have� made� a� computational� error�� Am� I� correct?
14.21� �The�correlation�between�the�covariate�and�the�dependent�variable�is�0��I�assert�that� ANCOVA�is�still�preferred�over�ANOVA��Am�I�correct?
14.22� �If�there�is�a�nonlinear�relationship�between�the�covariate�X�and�the�dependent�vari- able�Y,�then�it�is�very�likely�that
� a�� There�will�be�less�reduction�in�SSwith� � b�� The�group�effects�will�be�biased� � c�� The�correlation�between�X�and�Y�will�be�smaller�in�magnitude� � d�� All�of�the�above�
475Introduction to Analysis of Covariance
Computational problems
14.1� �Consider�the�ANCOVA�situation�where�the�dependent�variable�Y�is�the�posttest�of� an� achievement� test� and� the� covariate� X� is� the� pretest� of� the� same� test�� Given� the� data� that� follow,� where� there� are� three� groups,� (a)� calculate� the� adjusted� Y� values� assuming�that�bw�=�1�00,�and�(b)�determine�what�effects�the�adjustment�had�on�the� posttest�results�
Group X X –
Y Y –
40 120 1 50 50 125 125
60 130
70 140 2 75 75 150 150
80 160
90 160 3 100 100 175 175
110 190
14.2� Malani� wants� to� determine� whether� children� whose� preschool� classroom� has� a� window�differ�in�their�receptive�vocabulary�as�compared�to�children�whose�class- room� does� not� have� a� window�� At� the� beginning� of� the� school� year,� Malani� ran- domly�assigns�10�children�at�Rainbow�Butterfly�Preschool�to�one�of�two�different� classrooms:�one�classroom�which�has�a�window�that�looks�out�onto�a�grassy�area� or�another�classroom�that�has�no�windows��At�the�end�of�the�school�year,�Malani� measures� children� on� their� receptive� vocabulary�� In� the� following� are� two� inde- pendent�random�samples�(classroom�with�and�without�window)�of�paired�values� on�the�covariate�(X;�receptive�vocabulary�measured�at�beginning�of�school�year)� and�the�dependent�variable�essay�score�(Y;�receptive�vocabulary�measured�at�the� end� of� the� school� year)�� Conduct� an� ANOVA� on� Y,� an� ANCOVA� on� Y� using� X� as� a� covariate,� and� compare� the� results� (α� =� �05)�� Determine� the� unadjusted� and� adjusted�means�
Classroom with Window Classroom Without Window
X Y X Y
80 105 80 95 75 100 85 100 85 105 90 105 70 100 85 100 90 110 95 105
14.3� In� the� following� are� four� independent� random� samples� (different� methods� of� instruc- tion)�of�paired�values�on�the�covariate�IQ�(X)�and�the�dependent�variable�essay�score�(Y)�� Conduct�an�ANOVA�on�Y,�an�ANCOVA�on�Y�using�X�as�a�covariate,�and�compare�the� results�(α�=��05)��Determine�the�unadjusted�and�adjusted�means�
476 An Introduction to Statistical Concepts
Group 1 Group 2 Group 3 Group 4
X Y X Y X Y X Y
94 14 80 38 92 55 94 24 96 19 84 34 96 53 94 37
98 17 90 43 99 55 98 22 100 38 97 43 101 52 100 43 102 40 97 61 102 35 103 49 105 26 112 63 104 46 104 24 109 41 115 93 107 57 104 41 110 28 118 74 110 55 108 26 111 36 120 76 111 42 113 70 130 66 120 79 118 81 115 63
14.4� A�communications�researcher�wants�to�know�which�of�five�versions�of�commercials� for�a�new�television�show�is�most�effective�in�terms�of�viewing�likelihood��Each�com- mercial�is�viewed�by�six�students��A�one-factor�ANCOVA�was�used�to�analyze�these� data� where� the� covariate� was� amount� of� television� previously� viewed� per� week�� Complete�the�following�ANCOVA�summary�table�(α�=��05):
Source SS df MS F Critical Value Decision
Between�adjusted 96 — — — — — Within�adjusted 192 — — Covariate — — — — — — Total 328 —
Interpretive problems
14.1� The� first� interpretive� problem� in� Chapter� 11� requested� the� following:� “Using� the� survey� 1� dataset� from� the� website,� use� SPSS� to� conduct� a� one-factor� fixed-effects� ANOVA,� including� effect� size,� where� political� view� is� the� grouping� variable� (i�e�,� independent�variable)�(J�=�5)�and�the�dependent�variable�is�a�variable�of�interest�to� you�[the�following�variables�look�interesting:�books,�TV,�exercise,�drinks,�GPA,�GRE- Quantitative� (GRE-Q),� CDs,� hair� appointment]�”� Using� these� same� data,� select� an� appropriate�covariate�and�then�generate�a�one-factor�ANCOVA�(including�testing�the� assumptions�of�both�the�ANOVA�and�ANCOVA)��Compare�and�contrast�the�results� of�the�ANOVA�and�ANCOVA��Which�method�would�you�select�and�why?
14.2� The� second� interpretive� problem� in� Chapter� 11� requested� the� following:� “Using� the� survey� 1� dataset� from� the� website,� use� SPSS� to� conduct� a� one-factor� fixed-effects� ANOVA,�including�effect�size,�where�hair�color�is�the�grouping�variable�(i�e�,�indepen- dent�variable)�(J�=�5)�and�the�dependent�variable�is�a�variable�of�interest�to�you�(the�fol- lowing�variables�look�interesting:�books,�TV,�exercise,�drinks,�GPA,�GRE-Q,�CDs,�hair� appointment)�”�Using�these�same�data,�select�an�appropriate�covariate�and�then�gener- ate�a�one-factor�ANCOVA�(including�testing�the�assumptions�of�both�the�ANOVA�and� ANCOVA)�� Compare� and� contrast� the� results� of� the� ANOVA� and� ANCOVA�� Which� method�would�you�select�and�why?
477
15 Random- and Mixed-Effects Analysis of Variance Models
Chapter Outline
15�1� The�One-Factor�Random-Effects�Model 15�1�1� Characteristics�of�the�Model 15�1�2� ANOVA�Model 15�1�3� ANOVA�Summary�Table�and�Expected�Mean�Squares 15�1�4� Assumptions�and�Violation�of�Assumptions 15�1�5� Multiple�Comparison�Procedures
15�2� Two-Factor�Random-Effects�Model 15�2�1� Characteristics�of�the�Model 15�2�2� ANOVA�Model 15�2�3� ANOVA�Summary�Table�and�Expected�Mean�Squares 15�2�4� Assumptions�and�Violation�of�Assumptions 15�2�5� Multiple�Comparison�Procedures
15�3� Two-Factor�Mixed-Effects�Model 15�3�1� Characteristics�of�the�Model 15�3�2� ANOVA�Model 15�3�3� ANOVA�Summary�Table�and�Expected�Mean�Squares 15�3�4� Assumptions�and�Violation�of�Assumptions 15�3�5� Multiple�Comparison�Procedures
15�4� One-Factor�Repeated�Measures�Design 15�4�1� Characteristics�of�the�Model 15�4�2� Layout�of�Data 15�4�3� ANOVA�Model 15�4�4� Assumptions�and�Violation�of�Assumptions 15�4�5� ANOVA�Summary�Table�and�Expected�Mean�Squares 15�4�6� Multiple�Comparison�Procedures 15�4�7� Alternative�ANOVA�Procedures 15�4�8� Example
15�5� Two-Factor�Split-Plot�or�Mixed�Design 15�5�1� Characteristics�of�the�Model 15�5�2� Layout�of�Data 15�5�3� ANOVA�Model 15�5�4� Assumptions�and�Violation�of�Assumptions
478 An Introduction to Statistical Concepts
15�5�5� ANOVA�Summary�Table�and�Expected�Mean�Squares 15�5�6� Multiple�Comparison�Procedures 15�5�7� Example
15�6� SPSS�and�G*Power 15�7� Template�and�APA-Style�Write-Up
Key Concepts
� 1�� Fixed-,�random-,�and�mixed-effects�models � 2�� Repeated�measures�models � 3�� Compound�symmetry/sphericity�assumption � 4�� Friedman�repeated�measures�test�based�on�ranks � 5�� Split-plot�or�mixed�designs�(i�e�,�both�between-�and�within-subjects�factors)
In� this� chapter,� we� continue� our� discussion� of� the� analysis� of� variance� (ANOVA)� by� considering�models�in�which�there�is�a�random-effects�factor,�previously�introduced�in� Chapter�11��These�models�include�the�one-factor�and�factorial�designs,�as�well�as�repeated� measures�designs��As�becomes�evident,�repeated�measures�designs�are�used�when�there� is� at� least� one� factor� where� each� individual� is� exposed� to� all� levels� of� that� factor�� This� factor�is�referred�to�as�a�repeated factor,�for�obvious�reasons��This�chapter�is�mostly�con- cerned� with� one-� and� two-factor� random-effects� models,� the� two-factor� mixed-effects� model,�and�one-�and�two-factor�repeated�measures�designs�
It�should�be�noted�that�effect�size�measures,�power,�and�confidence�intervals�(CIs)�can�be� determined�in�the�same�fashion�for�the�models�in�this�chapter�as�for�previously�described� ANOVA�models��The�standard�effect�size�measures�already�described�are�applicable�(i�e�,� ω2� and� η2),� although� the� intraclass� correlation� coefficient,� ρI,� can� be� utilized� for� random� effects� (similarly� interpreted)�� For� additional� discussion� of� these� issues� in� the� context� of� this� chapter,� see� Cohen� (1988),� Fidler� and� Thompson� (2001),� Keppel� and� Wickens� (2004),� Murphy,�Myors,�and�Wolach�(2008),�and�Wilcox�(1996,�2003)�
Many� of� the� concepts� used� in� this� chapter� are� the� same� as� those� covered� in� Chapters� 11� through�14��In�addition,�the�following�new�concepts�are�addressed:�random-�and�mixed-effects� factors,� repeated� measures� factors,� the� compound� symmetry/sphericity� assumption,� and� mixed�designs��Our�objectives�are�that�by�the�end�of�this�chapter,�you�will�be�able�to�(a)�under- stand�the�characteristics�and�concepts�underlying�random-�and�mixed-effects�ANOVA�mod- els,�(b)�determine�and�interpret�the�results�of�random-�and�mixed-effects�ANOVA�models,�and� (c)�understand�and�evaluate�the�assumptions�of�random-�and�mixed-effects�ANOVA�models�
15.1 One-Factor Random-Effects Model
Through� the� previous� chapters,� we� have� learned� about� many� statistical� procedures� as�Marie�has�assisted�others�and�conducted�studies�of�her�own��What�is�in�store�for� Marie�now?
479Random- and Mixed-Effects Analysis of Variance Models
For�the�past�few�chapters,�we�have�followed�Marie,�a�graduate�student�enrolled�in�an� educational� research� program� who,� as� part� of� her� independent� study� course,� exam- ined� various� questions� related� to� measures� drawn� from� students� enrolled� in� statis- tics� courses�� Knowing� the� success� that� Marie� achieved� in� analysis� of� data� from� her� independent�study�course,�Marie’s�faculty�advisor�feels�confident�that�Marie�can�assist� another�faculty�member�at�the�university��Marie�is�working�with�Mark,�the�coordinator� of�the�English�program��Mark� has�conducted�an�experiment� in�which� eight�students� were�randomly�assigned�to�one�of�two�instructors��Each�student�was�then�assessed�on� writing�by�four�raters��Mark�wants�to�know�the�following:�if�there�is�a�mean�difference� in�writing�based�on�instructor,�if�there�is�a�mean�difference�in�writing�based�on�rater,� and�if�there�is�a�mean�difference�in�writing�based�on�the�rater�by�instructor�interaction�� The�research�questions�presented�to�Mark�from�Marie’s�include�the�following:
•� Is there a mean difference in writing based on instructor? •� Is there a mean difference in writing based on rater? •� Is there a mean difference in writing based on rater by instructor?
With� one� between-subjects� independent� variable� (i�e�,� instructor)� and� one� within- subjects�factor�(i�e�,�rating�on�writing�task),�Marie�determines�that�a�two-factor�split-plot� ANOVA�is�the�best�statistical�procedure�to�use�to�answer�Mark’s�question��Her�next� task�is�to�assist�Mark�in�analyzing�the�data�
This�section�describes�the�distinguishing�characteristics�of�the�one-factor�random-effects� ANOVA�model,�the�linear�model,�the�ANOVA�summary�table�and�expected�mean�squares,� assumptions�and�their�violation,�and�multiple�comparison�procedures�(MCPs)�
15.1.1 Characteristics of the Model
The�characteristics�of�the�one-factor�fixed-effects�ANOVA�model�have�already�been�covered� in�Chapter�11��These�characteristics�include�(a)�one�factor�(or�independent�variable)�with� two�or�more�levels,�(b)�all�levels�of�the�factor�of�interest�are�included�in�the�design�(i�e�,�a� fixed-effects�factor),�(c)�subjects�are�randomly�assigned�to�one�level�of�the�factor,�and�(d)�the� dependent� variable� is� measured� at� least� at� the� interval� level�� Thus,� the� overall� design� is� a�fixed-effects�model,�where�there�is�one�factor�and�the�individuals�respond�to�only�one� level�of�the�factor��If�individuals�respond�to�more�than�one�level�of�the�factor,�then�this�is�a� repeated�measures�design,�as�shown�later�in�this�chapter�
The�characteristics�of�the�one-factor�random-effects�ANOVA�model�are�the�same�with�one� obvious�exception��This�has�to�do�with�the�selection�of�the�levels�of�the�factor��In�the�fixed- effects�case,�researchers�select�all�of�the�levels�of�interest�because�they�are�only�interested� in� making� generalizations� (or� inferences)� about� those� particular� levels�� Thus,� in� replica- tions�of�this�design,�each�replicate�would�use�precisely�the�same�levels��Considering�analy- ses�that�are�conducted�on�individuals,�examples�of�factors�that�are�typically�fixed�include� SES,�gender,�specific�types�of�drug�treatment,�age�group,�weight,�or�marital�status�
In� the� random-effects� case,� researchers� randomly� select� levels� from� the� population� of� levels� because� they� are� interested� in� making� generalizations� (or� inferences)� about� the� entire� population� of� levels,� not� merely� those� that� have� been� sampled�� Thus,� in� replica- tions�of�this�design,�each�replicate�need�not�have�the�same�levels�included��The�concept�of� random�selection�of�factor�levels�from�the�population�of�levels�is�the�same�as�the�random�
480 An Introduction to Statistical Concepts
selection�of�subjects�from�the�population��Here�the�researcher�is�making�an�inference�from� the� sampled� levels� to� the� population� of� levels,� instead� of� making� an� inference� from� the� sample�of�individuals�to�the�population�of�individuals��In�a�random-effects�design�then,� a�random�sample�of�factor�levels�is�selected�in�the�same�way�as�a�random�sample�of�indi- viduals�is�selected�
For� instance,� a� researcher� interested� in� teacher� effectiveness� may� have� randomly� sampled�history�teachers�(i�e�,�the�independent�variable)�from�the�population�of�history� teachers�in�a�particular�school�district��Generalizations�can�then�be�made�about�all�his- tory� teachers� in� that� school� district� that� could� have� been� sampled�� Other� examples� of� factors�that�are�typically�random�include�randomly selected�classrooms,�types�of�medica- tion,�observers�or�raters,�time�(seconds,�minutes,�hours,�days,�weeks,�etc�),�animals,�stu- dents,�or�schools��It�should�be�noted�that�in�educational�settings,�the�random�selection� of�schools,�classes,�teachers,�and/or�students�is�not�often�possible�as�that�decision�is�not� under� the� researcher’s� control�� Here� we� would� need� to� consider� such� factors� as� fixed� rather�than�random�effects�
15.1.2 aNOVa Model
The� one-factor� ANOVA� random-effects� model� is� written� in� terms� of� population� param- eters�as
Y aij j ij= + +µ ε
where Yij�is�the�observed�score�on�the�dependent�variable�for�individual�i�in�level�j�of�factor�A μ�is�the�overall�or�grand�population�mean aj�is�the�random�effect�for�level�j�of�factor�A εij�is�the�random�residual�error�for�individual�i�in�level�j
The�residual�error�can�be�due�to�individual�differences,�measurement�error,�and/or�other� factors�not�under�investigation��Note�that�we�use�aj�to�designate�the�random�effects�to�dif- ferentiate�them�from�αj�in�the�fixed-effects�model�
Because�the�random-effects�model�consists�of�only�a�sample�of�the�effects�from�the�popu- lation,� the� sum� of� the� sampled� effects� is� not� necessarily� 0�� For� instance,� we� may� select� a� sample�having�only�positive�effects�(e�g�,�all�very�effective�teachers)��If�the�entire�popula- tion�of�effects�were�examined,�then�the�sum�of�those�effects�would�indeed�be�0�
For�the�one-factor�random-effects�ANOVA�model,�the�hypotheses�for�testing�the�effect� of� factor� A� are� written� in� terms� of� equality� of� the� variances� among� the� means� of� the� random�levels,�as�follows�(i�e�,�the�means�for�each�level�are�about�the�same,�and,�thus,� the�variability�among�those�means�is�about�0)��It�should�be�noted�that�the�sign�for�the� alternative�hypothesis�is�“greater�than,”�reflecting�the�fact�that�the�variance�cannot�be� negative:
H a0 0: σ 2 =
H a1 2: σ > 0
481Random- and Mixed-Effects Analysis of Variance Models
Recall�for�the�one-factor�fixed-effects�ANOVA�model�that�the�hypotheses�for�testing�the�effect� of�factor�A�are�written�in�terms�of�equality�of�the�means�of�the�groups�(as�presented�here):
H J0 : . . .µ µ µ 1 2= = … =
H j1 .: not all the µ are equal
This�reflects�the�difference�in�the�inferences�made�in�the�random-�and�fixed-effects�models��In� the�fixed-effects�case,�the�null�hypothesis�is�about�specific�population�means;�in�the�random- effects�case,�the�null�hypothesis�is�about�variation�among�the�entire�population�of�means�� As�becomes�evident,�the�difference�in�the�models�is�reflected�in�the�MCPs�
15.1.3 aNOVa Summary Table and expected Mean Squares
Here�there�are�very�few�differences�between�the�one-factor�random-effects�and�one-factor� fixed-effects� models�� The� sources� of� variation� are� still� A� (or� between),� within,� and� total�� The�sums�of�squares,�degrees�of�freedom,�mean�squares,�F�test�statistic,�and�critical�value� are�determined�in�the�same�way�as�in�the�fixed-effects�case��Obviously�then,�the�ANOVA� summary�table�looks�the�same�as�well��Using�the�example�from�Chapter�11,�assuming�the� model�is�now�a�random-effects�model,�we�obtain�a�test�statistic�F�=�6�8177,�which�is�again� significant�at�the��05�level�
As� in� Chapters� 11� and� 13,� the� formation� of� a� proper� F� ratio� is� related� to� the� expected� mean�squares��If�H0�is�actually�true,�then�the�expected mean squares�are�as�follows:
E AMS( ) = σε2
E withMS( ) = σε2
and�thus�the�ratio�of�expected�mean�squares�is�as�follows:
E E
1A with
MS MS ( )
( ) =
where the�expected�value�of�F�is�E(F)�=�dfwith/(dfwith�−�2) σε
2�is�the�population�variance�of�the�residual�errors
If�H0�is�actually�false,�then�the�expected�mean�squares�are�as�follows:
E AMS n a( ) = +σ σε2 2
E withMS( ) = σε2
and�thus�the�ratio�of�the�expected�mean�squares�is�as�follows:
E E
1A with
MS MS ( )
( ) >
where� E(F)� >� dfwith/(dfwith� −� 2)� and� σa 2� is� the� population� variance� of� the� levels� of� factor� A��
Thus,�the�important�part�of�E(MSA)�is�the�magnitude�of�the�second�term,�n aσ 2�
482 An Introduction to Statistical Concepts
As�in�previous�ANOVA�models,�the�proper�F�ratio�should�be�formed�as�follows:
F = +( )/(systematic variability error variability error variabiility)
For� the� one-factor� random-effects� model,� the� only� appropriate� F� ratio� is� MSA/MSwith� because�it�does�serve�to�isolate�the�systematic�variability�(i�e�,�the�variability�between�the� levels�or�groups�in�factor�A,�the�independent�variable)��That�is,�the�within�term�must�be� utilized�as�the�error�term�in�the�F�ratio�
15.1.4 assumptions and Violation of assumptions
In� Chapter� 11,� we� described� the� assumptions� for� the� one-factor� fixed-effects� model�� The� assumptions� are� nearly� the� same� for� the� one-factor� random-effects� model,� and� we� need� not� devote� much� attention� to� them� here�� In� short,� the� assumptions� are� again� concerned� with�the�distribution�of�the�dependent�variable�scores,�specifically�that�scores�are�random� and�independent,�coming�from�normally�distributed�populations�with�equal�population� variances��The�effect�of�assumption�violations�and�how�to�deal�with�them�have�been�thor- oughly� discussed� in� Chapter� 11� (although� see� Wilcox,� 1996,� 2003,� for� alternative� proce- dures�when�variances�are�unequal)�
Additional� assumptions� must� be� made� for� the� random-effects� model�� These� assump- tions�deal�with�the�effects�for�the�levels�of�the�independent�variable,�the�aj��First,�here�are� a�few�words�about�the�aj��The�random�group�effects�aj�are�computed,�in�the�population,�by� the�following:
aj j= −. ..µ µ
For�example,�a3�represents�the�effect�for�being�a�member�of�group�3��If�the�overall�mean�μ��� is�60�and�the�mean�of�group�3�(i�e�,�μ�3)�is�100,�then�the�group�effect�would�be
a3 3 1 6 4= − = − =. ..µ µ 00 0 0
In�other�words,�the�effect�for�being�a�member�of�group�3�is�an�increase�of�40�points�over� the�overall�mean�
The�assumptions�are�that�the�aj�group�effects�are�randomly�and�independently�sampled� from� the�normally�distributed� population� of� group� effects,� with� a�population� mean�of�0� and�a�population�variance�of�σ2a��Stated�another�way,�there�is�a�population�of�group�effects� out� there� from� which� we� are� taking� a� random� sample�� For� example,� with� teacher� as� the� factor�of�interest,�we�are�interested�in�examining�the�effectiveness�of�teachers�as�measured� by� academic� performance� of� students� in� their� class�� We� take� a� random� sample� of� teach- ers� from� the� population� of� second-grade� teachers�� For� these� teachers,� we� measure� their� effectiveness� in� the� classroom� via� student� performance� and� generate� an� effect� for� each� teacher�(i�e�,�the�aj)��These�effects�indicate�the�extent�to�which�a�particular�teacher�is�more�or� less�effective�than�the�population�average�of�teachers��Their�effects�are�known�as�random� effects�as�the�teachers�are�randomly�selected��In�selecting�teachers,�each�teacher�is�selected� independently�of�all�other�teachers�to�prevent�a�biased�sample�
483Random- and Mixed-Effects Analysis of Variance Models
The� effects� of� the� violation� of� the� assumptions� about� the� aj� are� the� same� as� with� the� dependent�variable�scores��The�F�test�is�quite�robust�to�nonnormality�of�the�aj�terms�and� unequal�variances�of�the�aj�terms��However,�the�F�test�is�quite�sensitive�to�nonindepen- dence� among� the� aj� terms,� with� no� known� solutions�� A� summary� of� the� assumptions� and�the�effects�of�their�violation�for�the�one-factor�random-effects�model�is�presented�in� Table�15�1�
15.1.5 Multiple Comparison procedures
Let�us�think�for�a�moment�about�the�use�of�MCPs�for�the�random-effects�model��In�general,� the�researcher�is�not�usually�interested�in�making�inferences�about�just�the�levels�of�A�that� were�sampled��Thus,�estimation�of�the�aj�terms�does�not�provide�us�with�any�information� about�the�aj�terms�that�were�not�sampled��Also,�the�aj�terms�cannot�be�summarized�by�their� mean,�as�they�do�not�necessarily�sum�to�0�for�the�levels�sampled,�only�for�the�population� of�levels�
15.2 Two-Factor Random-Effects Model
In� this� section,� we� describe� the� distinguishing� characteristics� of� the� two-factor� random- effects�ANOVA�model,�the�linear�model,�the�ANOVA�summary�table�and�expected�mean� squares,�assumptions�of�the�model�and�their�violation,�and�MCPs�
15.2.1 Characteristics of the Model
The� characteristics� of� the� one-factor� random-effects� ANOVA� model� have� already� been� covered�in�this�chapter,�and�of�the�two-factor�fixed-effects�model,�in�Chapter�13��Here�we� extend� and� combine� these� characteristics� to� form� the� two-factor� random-effects� model�� These�characteristics�include�(a)�two�factors�(or�independent�variables)�each�with�two�or� more�levels,�(b)�the�levels�of�each�of�the�factors�are�randomly�sampled�from�the�population� of�levels�(i�e�,�two�random-effects�factors),�(c)�subjects�are�randomly�assigned�to�one�combi- nation�of�the�levels�of�the�two�factors,�and�(d)�the�dependent�variable�is�measured�at�least� at�the�interval�level��Thus,�the�overall�design�is�a�random-effects�model,�with�two�factors,�and� the�individuals�respond�to�only�one�combination�of�the�levels�of�the�two�factors�(note�that� this�is�not�a�popular�model�in�education�and�the�behavioral�sciences;�in�factorial�designs,�
Table 15.1
Assumptions�and�Effects�of�Violations:�One-Factor�Random-Effects�Model
Assumption Effect of Assumption Violation
Independence •��Increased�likelihood�of�a�Type�I�and/or�Type�II�error�in�F •��Affects�standard�errors�of�means�and�inferences�about�those�means
Homogeneity�of�variance •��Bias�in�SSwith;�increased�likelihood�of�a�Type�I�and/or�Type�II�error •��Small�effect�with�equal�or�nearly�equal�n’s;�otherwise�effect�decreases�
as�n�increases
Normality •�Minimal�effect�with�equal�or�nearly�equal�n’s
484 An Introduction to Statistical Concepts
we�typically�see�a�random-effects�factor�with�a�fixed-effects�factor)��If�individuals�respond� to�more�than�one�combination�of�the�levels�of�the�two�factors,�then�this�is�a�repeated�mea- sures�design�(discussed�later�in�this�chapter)�
15.2.2 aNOVa Model
The� two-factor� ANOVA� random-effects� model� is� written� in� terms� of� population� param- eters�as
Y a b abijk j k jk ijk= + + + +µ ε( )
where Yijk�is�the�observed�score�on�the�dependent�variable�for�individual�i�in�level�j�of�factor�A�
and�level�k�of�factor�B�(or�in�the�jk�cell) μ�is�the�overall�or�grand�population�mean�(i�e�,�regardless�of�cell�designation) aj�is�the�random�effect�for�level�j�of�factor�A�(row�effect) bk�is�the�random�effect�for�level�k�of�factor�B�(column�effect) (ab)jk�is�the�interaction�random�effect�for�the�combination�of�level�j�of�factor�A�and�level�k�
of�factor�B εijk�is�the�random�residual�error�for�individual�i�in�cell�jk
The�residual�error�can�be�due�to�individual�differences,�measurement�error,�and/or�other� factors�not�under�investigation��Note�that�we�use�aj,�bk,�and�(ab)jk�to�designate�the�random� effects� to� differentiate� them� from� the� αj,� βk,� and� (αβ)jk� in� the� fixed-effects� model�� Finally,� there�is�no�requirement�that�the�sum�of�the�main�or�interaction�effects�is�equal�to�0�as�only� a�sample�of�these�effects�are�taken�from�the�population�of�effects�
There�are�three�sets�of�hypotheses,�one�for�each�of�the�two�main�effects�and�one�for�the� interaction�effect��The�null�and�alternative�hypotheses,�respectively,�for�testing�the�main� effect�of�factor�A�(i�e�,�independent�variable�A)�follows��The�null�hypothesis�tests�whether� the�variance�among�the�means�for�the�random�effect�of�independent�variable�A�is�equal� to�0�(i�e�,�the�means�for�each�level�of�factor�A�are�about�the�same;�thus,�the�variability�among� those�means�is�about�0)��It�should�be�noted�that�the�sign�for�the�alternative�hypothesis�is� “greater�than,”�reflecting�the�fact�that�the�variance�cannot�be�negative:
H a0 01 2: σ =
H a11 2: σ > 0
The�hypotheses�for�testing�the�main�effect�of�factor�B�(i�e�,�independent�variable�B)�similarly� test�whether�the�variance�among�the�means�for�the�random�effect�of�independent�variable�B� is�equal�to�0�(i�e�,�the�means�for�each�level�of�factor�B�are�about�the�same,�and,�thus,�the�vari- ability� among� those� means� is� about� 0)�� It� should� be� noted� that� the� sign� for� the� alternative� hypothesis�is�“greater�than,”�reflecting�the�fact�that�the�variance�cannot�be�negative:
H b02 2: σ = 0
H b12 2: >σ 0
485Random- and Mixed-Effects Analysis of Variance Models
Finally,� the� hypotheses� for� testing� the� interaction� effect� are� presented� next�� In� this� case,� the�null�hypothesis�tests�whether�the�variance�among�the�means�for�the�interaction�of�the� random�effects�of�factors�A�and�B�is�equal�to�0�(i�e�,�the�means�for�each�AB�cell�are�about� the�same,�and,�thus,�the�variability�among�those�means�is�about�0)��It�should�be�noted�that� the�sign�for�the�alternative�hypothesis�is�“greater�than,”�reflecting�the�fact�that�the�variance� cannot�be�negative:
H ab03 2: 0σ =
H ab13 2: σ > 0
These� hypotheses� again� reflect� the� difference� in� the� inferences� made� in� the� random-� and� fixed-effects� models�� In� the� fixed-effects� case,� the� null� hypotheses� are� about� means,� whereas� in� the� random-effects� case,� the� null� hypotheses� are� about� variation� among� the� means�
15.2.3 aNOVa Summary Table and expected Mean Squares
Here� there� are� very� few� differences� between� the� two-factor� fixed-effects� and� random- effects�models��The�sources�of�variation�are�still�A,�B,�AB,�within,�and�total��The�sums�of� squares,�degrees�of�freedom,�and�mean�squares�are�determined�the�same�as�in�the�fixed- effects�case��However,�the�F�test�statistics�are�different�due�to�the�expected�mean�squares,� as�are�the�critical�values�used��The�F�test�statistics�are�formed�for�the�test�of�factor�A�(i�e�,�the� main�effect�for�independent�variable�A)�as�follows:
F MS MS
= A AB
for� the� test� of� factor� B� (i�e�,� the� main� effect� for� independent� variable� B)� as� presented� here:
F MS MS
= B AB
and�for�the�test�of�the�AB�interaction�as�indicated:
F MS MS
= AB with
Recall�that�in�the�fixed-effects�model,�the�MSwith�was�used�as�the�error�term�for�all�three� hypotheses��However,�in�the�random-effects�model,�the�MSwith�is�used�as�the�error�term�only� for�the�test�of�the�interaction��The�MSAB�is�used�as�the�error�term�for�the�tests�of�both�main� effects��The�critical�values�used�are�those�based�on�the�degrees�of�freedom�for�the�numera- tor�and�denominator�of�each�hypothesis�tested��Thus,�using�the�example�from�Chapter�13,�
486 An Introduction to Statistical Concepts
assuming�that�the�model�is�now�a�random-effects�model,�we�obtain�the�following�as�our� test�statistic�for�the�test�of�factor�A�(i�e�,�the�main�effect�for�independent�variable�A):
F MS MS
A = = = A
AB
246 1979 7 2813
33 8124 .
. .
for�the�test�of�factor�B,�the�test�statistic�is�computed�as�follows:
F MS MS
B = = = B
AB
712 5313 7 2813
97 8577 .
. .
and�for�the�test�of�the�AB�interaction,�we�find�the�following:
F MS MS
AB = = = AB
with
7 2813 11 5313
0 6314 . .
.
The�critical�value�for�the�test�of�factor�A�is�found�in�the�F�table�of�Table�A�4�as�αFJ−1,�(J−1)(K−1),� which�for�the�example�is��05F3,3�=�9�28,�and�is�significant�at�the��05�level��The�critical�value�for� the�test�of�factor�B�is�found�in�the�F�table�as�αFK−1,(J−1)�(K−1),�which�for�the�example�is��05F1,3�=�10�13,� and�is�significant�at�the��05�level��The�critical�value�for�the�test�of�the�interaction�is�found�in� the�F�table�as�αF(J−1)�(K−1),N−JK,�which�for�the�example�is��05F3,24�=�3�01,�and�is�not�significant�at�the� �05�level��It�just�so�happens�for�the�example�data�that�the�results�for�the�random-�and�fixed- effects�models�are�the�same��This�will�not�always�be�the�case�
The�formation�of�the�proper�F�ratios�is�again�related�to�the�expected�mean�squares��Recall� that�our�hypotheses�for�the�two-factor�random-effects�model�are�based�on�variation�among� the�means�of�the�random�effects�(rather�than�the�means�as�seen�in�the�fixed-effects�case)��If� H0�is�actually�true�(i�e�,�there�is�no�variation�among�the�means�of�the�random�effects),�then� the�expected mean squares�are�as�follows:
E 2( )MSA = σε
E 2( )MSB = σε
E 2( )MSAB = σε
E 2( )MSwith = σ ε
where�σε 2�is�the�population�variance�of�the�residual�errors�
If�H0�is�actually�false�(i�e�,�there�is�variation�among�the�means�of�the�random�effects),�then� the�expected�mean�squares�are�as�follows:
E( )MS n Knab aA = + +σ σ σε 2 2 2
E 2( )MS n Jnab bB = + +σ σ σε 2 2
E 2( )MS n abAB = +σ σε 2
E 2( )MSwith = σε
where�σa 2,�σb
2,�and�σab 2 �are�the�population�variances�of�A,�B,�and�AB,�respectively�
487Random- and Mixed-Effects Analysis of Variance Models
As�in�previous�ANOVA�models,�the�proper�F�ratio�should�be�formed�as�follows:
F = +( ) /(systematic variability error variability error variabiility)
For�the�two-factor�random-effects�model,�the�appropriate�error�term�for�the�main�effects�is� MSAB�and�the�appropriate�error�term�for�the�interaction�effect�is�MSwith�
15.2.4 assumptions and Violation of assumptions
Previously� we� described� the� assumptions� for� the� one-factor� random-effects� model�� The� assumptions� are� nearly� the� same� for� the� two-factor� random-effects� model,� and� we� need� not�devote�much�attention�to�them�here��As�before,�the�assumptions�are�concerned�with�the� distribution�of�the�dependent�variable�scores,�and�of�the�random-effects�(sampled�levels�of� the�independent�variables,�the�aj,�bk,�and�their�interaction�(ab)jk)��However,�there�are�a�few� new�wrinkles��Little�is�known�about�the�effect�of�unequal�variances�(i�e�,�heteroscedastic- ity)�or�dependence�(i�e�,�violation�of�the�assumption�of�independence)�for�this�random-effects� model,� although� we� expect� the� effects� to� be� the� same� as� for� the� fixed-effects� model�� For� violation�of�the�normality�assumption,�effects�are�known�to�be�substantial��A�summary�of� the�assumptions�and�the�effects�of�their�violation�for�the�two-factor�random-effects�model� is�presented�in�Table�15�2�
15.2.5 Multiple Comparison procedures
The�story�of�multiple�comparisons�for�the�two-factor�random-effects�model�is�the�same�as�that�for� the�one-factor�random-effects�model��In�general,�the�researcher�is�not�usually�interested�in�mak- ing�inferences�about�just�the�levels�of�A,�B,�or�AB�that�were�sampled,�and�thus�performing�MCPs� in�a�two-factor�random-effects�model�is�a�moot�point��Thus,�estimation�of�the�aj,�bk,�or�(ab)jk�terms� does�not�provide�us�with�any�information�about�the�aj,�bk,�or�(ab)jk�terms�that�were�not�sampled�� Also,�the�aj,�bk,�or�(ab)jk�terms�cannot�be�summarized�by�their�means�as�they�will�not�necessarily� sum�to�0�for�the�levels�sampled,�only�for�the�population�of�levels�
Table 15.2
Assumptions�and�Effects�of�Violations:�Two-Factor�Random-Effects�Model
Assumption Effect of Assumption Violation
Independence Little�is�known�about�the�effects�of�dependence;�however,�based�on�the� fixed-effects�model,�we�might�expect�the�following:
•�Increased�likelihood�of�a�Type�I�and/or�Type�II�error�in�F •��Affects�standard�errors�of�means�and�inferences�about�those�means
Homogeneity�of�variance �Little�is�known�about�the�effects�of�heteroscedasticity;�however,�based� on�the�fixed-effects�model,�we�might�expect�the�following:
•�Bias�in�SSwith •�Increased�likelihood�of�a�Type�I�and/or�Type�II�error •�Small�effect�with�equal�or�nearly�equal�n’s •�Otherwise�effect�decreases�as�n�increases
Normality •�Minimal�effect�with�equal�or�nearly�equal�n’s •�Otherwise�substantial�effects
488 An Introduction to Statistical Concepts
15.3 Two-Factor Mixed-Effects Model
This� section� describes� the� distinguishing� characteristics� of� the� two-factor� mixed-effects� ANOVA�model,�the�linear�model,�the�ANOVA�summary�table�and�expected�mean�squares,� assumptions�of�the�model�and�their�violation,�and�MCPs�
15.3.1 Characteristics of the Model
The� characteristics� of� the� two-factor� random-effects� ANOVA� model� have� already� been� covered�in�the�preceding�section,�and�of�the�two-factor�fixed-effects�model,�in�Chapter�13�� Here�we�combine�these�characteristics�to�form�the�two-factor�mixed-effects�model��These� characteristics� include� (a)� two� factors� (or� independent� variables)� each� with� two� or� more� levels,�(b)�the�levels�for�one�of�the�factors�are�randomly�sampled�from�the�population�of�lev- els�(i�e�,�the�random-effects�factor)�and�all�of�the�levels�of�interest�for�the�second�factor�are� included�in�the�design�(i�e�,�the�fixed-effects�factor),�(c)�subjects�are�randomly�selected�and� assigned�to�one�combination�of�the�levels�of�the�two�factors,�and�(d)�the�dependent�variable� is�measured�at�least�at�the�interval�level��Thus,�the�overall�design�is�a�mixed-effects�model,� with� one� fixed-effects� factor� and� one� random-effects� factor,� and� individuals� respond� to� only�one�combination�of�the�levels�of�the�two�factors��If�individuals�respond�to�more�than� one�combination,�then�this�is�a�repeated�measures�design�
15.3.2 aNOVa Model
There�are�actually�two�variations�of�the�two-factor�mixed-effects�model,�one�where�fac- tor� A� is� fixed� and� factor� B� is� random� and� the� other� where� factor� A� is� random� and� factor�B�is�fixed��The�labeling�of�a�factor�as�A�or�B�is�arbitrary,�so�we�only�consider�the� former�variation�where�A�is�fixed�and�B�is�random��For�the�latter�variation,�merely�switch� the�labels�of�the�factors��The�two-factor�ANOVA�mixed-effects�model�is�written�in�terms� of�population�parameters�as
Y b bijk j k jk ijk= + + + +µ α α ε( )
where Yijk�is�the�observed�score�on�the�dependent�variable�for�individual�i�in�level�j�of�factor�A�
and�level�k�of�factor�B�(or�in�the�jk�cell) μ�is�the�overall�or�grand�population�mean�(i�e�,�regardless�of�cell�designation) αj�is�the�fixed�effect�for�level�j�of�factor�A�(row�effect) bk�is�the�random�effect�for�level�k�of�factor�B�(column�effect) (αb)jk�is�the�interaction�mixed�effect�for�the�combination�of�level�j�of�factor�A�and�level�k�
of�factor�B εijk�is�the�random�residual�error�for�individual�i�in�cell�jk
The�residual�error�can�be�due�to�individual�differences,�measurement�error,�and/or�other� factors�not�under�investigation��Note�that�we�use�bk�and�(αb)jk�to�designate�the random� and� mixed� effects,� respectively,� to� differentiate� them� from� βk� and� (αβ)jk� in� the� fixed- effects�model�
489Random- and Mixed-Effects Analysis of Variance Models
As�shown�in�Figure�15�1,�due�to�the�nature�of�the�mixed-effects�model,�only�some�of�the� columns� are� randomly� selected� for� inclusion� in� the� design�� Each� cell� of� the� design� will� include� row� (α),� column� (b),� and� interaction� (αb)� effects�� With� an� equal� n’s� model,� if� we� sum�these�effects�for�a�given�column,�then�the�effects�will�sum�to�0��However,�if�we�sum� these�effects�for�a�given�row,�then�the�effects�will�not�sum�to�0,�as�some�columns�were�not� sampled�
The� null� and� alternative� hypotheses,� respectively,� for� testing� the� effect� of� factor� A� are� presented�as�follows��These�hypotheses�reflect�testing�the�equality�of�means�of�the�levels� of�independent�variable�A�(the�fixed�effect):
H J01 . . . . .: .µ µ µ 1 2= = … =
H j11 . .: not all the are equalµ
The�hypotheses�for�testing�the�effect�of�factor�B,�the�random�effect,�follow��The�null�hypoth- esis� tests� whether� the� variance� among� the� means� for� the� random� effect� of� independent� variable�B�is�equal�to�0�(i�e�,�the�means�for�each�level�of�factor�B�are�about�the�same,�and,� thus,� the� variability� among� those� means� is� about� 0)�� It� should� be� noted� that� the� sign� for� the�alternative�hypothesis�is�“greater�than,”�reflecting�the�fact�that�the�variance�cannot�be� negative:
H b02 : 0 2σ =
H b12 2: 0σ >
Finally,� the� hypotheses� for� testing� the� interaction� effect� are� presented� next�� In� this� case,� the�null�hypothesis�tests�whether�the�variance�among�the�means�for�the�interaction�of�the� random�effects�of�factors�A�and�B�is�equal�to�0�(i�e�,�the�means�for�each�AB�cell�are�about�the� same,�and,�thus,�the�variability�among�those�means�is�about�0)��It�should�be�noted�that� the�sign�for�the�alternative�hypothesis�is�“greater�than,”�reflecting�the�fact�that�the�variance� cannot�be�negative:
H b0 03 2: σα =
H b13 2: 0σα >
b1 b2 b3 b4 b5 b6 α1 α2 α3 α4
FIGuRe 15.1 Conditions� for� the� two-factor� mixed-effects� model:� although� all� four� levels� of� factor� A� are� selected� by� the� researcher�(A�is�fixed),�only�three�of�the�six�levels�of�factor�B�are�selected�(B�is�random)��If�the�levels�of�B�selected� are�1,�3,�and�6,�then�the�design�will�only�consist�of�the�shaded�cells��In�each�cell�of�the�design�are�row,�column,� and�cell�effects��If�we�sum�these�effects�for�a�given�column,�then�the�effects�will�sum�to�0��If�we�sum�these�effects� for�a�given�row,�then�the�effects�will�not�sum�to�0�(due�to�missing�cells)�
490 An Introduction to Statistical Concepts
These�hypotheses�reflect�the�difference�in�the�inferences�made�in�the�mixed-effects�model�� Here�we�see�that�the�hypotheses�about�the�fixed-effect�A�(i�e�,�the�main�effect�for�indepen- dent�variable�A)�are�about�means,�whereas�the�hypotheses�involving�the�random-effect�B� (i�e�,�the�main�effect�of�B�and�the�interaction�effect�AB)�are�about�variation among the means� as�these�involve�a�random�effect�
15.3.3 aNOVa Summary Table and expected Mean Squares
There�are�very�few�differences�between�the�two-factor�fixed-effects,�random-effects,�and� mixed-effects� models�� The� sources� of� variation� for� the� mixed-effects� model� are� again� A� (the�fixed� effect),� B�(the�random�effect),� AB�(the�interaction�effect),� within,� and� total��The� sums� of� squares,� degrees� of� freedom,� and� mean� squares� are� determined� the� same� as� in� the�fixed-effects�case��However,�the�F�test�statistics�are�different�in�each�of�these�models,�as� well�as�the�critical�values�used��The�F�test�statistics�are�formed�for�the�test�of�factor�A,�the� fixed�effect,�as�seen�here:
F MS MS
A = A
AB
for�the�test�of�factor�B,�the�random�effect,�is�computed�as�follows:
F MS MS
B = B
with
and�for�the�test�of�the�AB�interaction,�the�mixed�effect,�as�indicated�here:
F MS MS
AB = AB
with
Recall� that� in� the� fixed-effects� model,� the� MSwith� is� used� as� the� error� term� for� all� three� hypotheses�� However,� in� the� random-effects� model,� the� MSwith� is� used� as� the� error� term� only�for�the�test�of�the�interaction,�and�the�MSAB�is�used�as�the�error�term�for�the�tests�of� both�main�effects��Finally,�in�the�mixed-effects�model,�the�MSwith�is�used�as�the�error�term� for�the�test�of�factor�B�(the�random�effect)�and�the�interaction�(i�e�,�AB),�whereas�the�MSAB� is�used�as�the�error�term�for�the�test�of�factor�A�(the�fixed�effect)��The�critical�values�used� are� those� based� on� the� degrees� of� freedom� for� the� numerator� and� denominator� of� each� hypothesis�tested�
Thus,� using� the� example� from� Chapter� 13,� let� us� assume� the� model� is� now� a� mixed- effects�model�where�factor�A,�the�fixed�effect,�is�the�level�of�attractiveness�(four�catego- ries)��Factor�B,�the�random�effect,�is�time�of�day�(two�randomly�selected�categories)��We� obtain�as�our�test�statistic�for�the�test�of�factor�A,�the�fixed�effect�of�level�of�attractiveness,� as�follows:
F MS MS
A = = = A
AB
246 1979 7 2813
33 8124 .
. .
491Random- and Mixed-Effects Analysis of Variance Models
for� the� test� of� factor�B,� the� random� effect� of� time�of� day,� the� test� statistic� is� computed� as� follows:
F MS MS
B = = = B
with
712 5313 11 5313
61 7911 .
. .
and�for�the�test�of�the�AB�(fixed�by�random�effect,�levels�of�attractiveness�by�time�of�day)� interaction,�we�find�a�test�statistic�as�follows:
F MS MS
AB = = = AB
with
7 2813 11 5313
0 6314 . .
.
The�critical�value�for�the�test�of�factor�A�(the�fixed�effect,�level�of�attractiveness)�is�found� in�the�F�table�as�αFJ−1,�(J−1)(K−1),�which�for�the�example�is��05F3,3�=�9�28,�and�is�statistically�sig- nificant�at�the��05�level��The�critical�value�for�the�test�of�factor�B�(the�random�effect,�time� of�day)�is�found�in�the�F�table�as� αFK−1,N−JK,�which�for�the�example�is� �05F1,24�=�4�26,�and�is� significant�at�the��05�level��The�critical�value�for�the�test�of�the�interaction�between�level�of� attractiveness�and�time�of�day�is�found�in�the�F�table�as�αF(J−1)(K−1),�N−JK,�which�for�the�exam- ple�is��05F3,24�=�3�01,�and�is�not�significant�at�the��05�level��It�just�so�happens�for�the example� data�that�the�results�for�the�mixed-,�random-,�and�fixed-effects�models�are�the�same��This� is�not�always�the�case�
The�formation�of�the�proper�F�ratio�is�again�related�to�the�expected�mean�squares��If�H0�is�actu- ally�true�(i�e�,�the�variance�among�the�means�is�0),�then�the�expected mean squares�are�as�follows:
E 2MSA( ) = σε
� E 2MSB( ) = σε
� E 2MSAB( ) = σε
� E 2MSwith( ) = σε
where�σε 2�is�the�population�variance�of�the�residual�errors�
If�H0�is�actually�false�(the�variance�among�the�means�is�not�equal�to�0),�then�the�expected� mean�squares�are�as�follows:
E 2MS n Kn Jb j j
J
A( ) = + + −
= ∑σ σ αε α2 2
1
1/( )
� E 2MS Jn bB( ) = +σ σε 2
� E 2MS n bAB( ) = +σ σε α2
� E MSwith( ) = σε2
where�all�terms�have�been�previously�defined�
492 An Introduction to Statistical Concepts
As�in�previous�ANOVA�models,�the�proper�F�ratio�should�be�formed�as�follows:
F = +( ) /(systematic variability error variability error variabiility)
For�the�two-factor�mixed-effects�model,�MSAB�must�be�used�as�the�error�term�for�the�test�of�A,� and�MSwith�must�be�used�as�the�error�term�for�the�test�of�B�and�for�the�interaction�test�
15.3.4 assumptions and Violation of assumptions
Previously� we� described� the� assumptions� for� the� two-factor� random-effects� model�� The� assumptions�are�nearly�the�same�for�the�two-factor�mixed-effects�model,�and�we�need�not� devote�much�attention�to�them�here��As�before,�the�assumptions�are�concerned�with�the� distribution� of� the� dependent� variable� scores� and� of� the� random� effects�� However,� note� that�not�much�is�known�about�the�effects�of�dependence�or�heteroscedasticity�for�random� effects,� although� we�expect�the�effects� are�the�same�as�for�the�fixed-effects� case��A�sum- mary�of�the�assumptions�and�the�effects�of�their�violation�for�the�two-factor�mixed-effects� model�is�presented�in�Table�15�3�
15.3.5 Multiple Comparison procedures
For� multiple� comparisons� in� the� two-factor� mixed-effects� model,� the� researcher� is� not� usually�interested�in�making�inferences�about�just�the�levels�of�the�random-effects�factor� (i�e�,� B)� or� the� interaction� (i�e�,� AB)� that� were� randomly� sampled�� Thus,� estimation� of� the� bk�or�(αb)jk�terms�does�not�provide�us�with�any�information�about�the�bk�or�(αb)jk�terms�not� sampled��Also,�the�bk�or�(αb)jk�terms�cannot�be�summarized�by�their�means�as�they�will�not� necessarily� sum� to� 0� for� the� levels� sampled,� only� for� the� population� of� levels�� However,� inferences�about�the�fixed-factor�A�can�be�made�in�the�same�way�they�were�made�for�the� two-factor� fixed-effects� model�� We� have� already� used� the� example� data� to� look� at� some� MCPs�in�Chapter�13�
Table 15.3
Assumptions�and�Effects�of�Violations:�Two-Factor�Mixed-Effects�Model
Assumption Effect of Assumption Violation
Independence Little�is�known�about�the�effects�of�dependence;�however,�based�on�the�fixed-effects�model,� we�might�expect�the�following:
•�Increased�likelihood�of�a�Type�I�and/or�Type�II�error�in�F •�Affects�standard�errors�of�means�and�inferences�about�those�means
Homogeneity� of�variance
Little�is�known�about�the�effects�of�heteroscedasticity;�however,�based�on�the�fixed-effects� model,�we�might�expect�the�following:
•�Bias�in�SSwith •�Increased�likelihood�of�a�Type�I�and/or�Type�II�error •�Small�effect�with�equal�or�nearly�equal�n’s •�Otherwise�effect�decreases�as�n�increases
Normality •�Minimal�effect�with�equal�or�nearly�equal�n’s •�Otherwise�substantial�effects
493Random- and Mixed-Effects Analysis of Variance Models
This�concludes�our�discussion�of�random-�and�mixed-effects�models�for�the�one-�and�two- factor�designs��For�three-factor�designs,�see�Keppel�(1982)�or�Keppel�and�Wickens�(2004)��In� the�major�statistical�software,�the�analysis�of�random�effects�can�be�treated�as�follows:�in�SAS� PROC�general�linear�model�(GLM),�use�the�RANDOM�statement�to�designate�random�effects;� in�SPSS�GLM,�random�effects�can�also�be�designated,�either�in�the�point-and-click�mode�(by� using�the�“Random�Factor(s)”�box)�or�in�the�syntax�mode�to�designate�random�effects�
15.4 One-Factor Repeated Measures Design
In� this� section,� we� describe� the� distinguishing� characteristics� of� the� one-factor� repeated� measures� ANOVA� model,� the� layout� of� the� data,� the� linear� model,� assumptions� of� the� model�and�their�violation,�the�ANOVA�summary�table�and�expected�mean�squares,�MCPs,� alternative�ANOVA�procedures,�and�an�example�
15.4.1 Characteristics of the Model
The� one-factor� repeated� measures� model� is� the� logical� extension� to� the� dependent� t� test�� Although�in�the�dependent�t�test�there�are�only�two�measurements�for�each�subject�(e�g�,�the� same�individuals�measured�prior�to�an�intervention�and�then�again�after�an�intervention),� in� the� one-factor� repeated� measures� model,� two� or more� measurements� can� be� examined�� The�characteristics�of�the�one-factor�repeated�measures�ANOVA�model�are�somewhat�simi- lar�to�the�one-factor�fixed-effects�model,�yet�there�are�a�number�of�obvious�exceptions��The� first�unique�characteristic�has�to�do�with�the�fact�that�each�subject�responds�to�each�level�of� factor�A��This�is�in�contrast�to�the�nonrepeated�case�where�each�subject�is�exposed�to�only� one�level�of�factor�A��This�design�is�often�referred�to�as�a�within-subjects design,�as�each� subject�responds�to�each�level�of�factor�A��Thus,�subjects�serve�as�their�own�controls�such�that� individual�differences�are�taken�into�account��This�was�not�the�case�in�any�of�the�previously� discussed�ANOVA�models��As�a�result,�subjects’�scores�are�not�independent�across�the�levels� of�factor�A��Compare�this�design�to�the�one-factor�fixed-effects�model�where�total�variation� was�decomposed�into�variation�due�to�A�(or�between)�and�due�to�the�residual�(or�within)�� In�the�one-factor�repeated�measures�design,�residual�variation�is�further�decomposed�into� variation�due�to�subjects�and�variation�due�to�the�interaction�between�A�and�subjects��The� reduction� in� the� residual� sum� of� squares� yields� a� more� powerful� design� as� well� as� more� precision�in�estimating�the�effects�of�A�and�thus�is�more�economical�in�that�less�subjects�are� necessary�than�in�previously�discussed�models�(Murphy,�Myors,�&�Wolach,�2008)�
The�one-factor�repeated�measures�design�is�also�a�mixed�model��The�subjects�factor�is�a� random�effect,�whereas�the�A�factor�is�almost�always�a�fixed�effect��For�example,�if�time�is�the� fixed�effect,�then�the�researcher�can�examine�phenomena�over�time��Finally,�the�one-factor� repeated� measures� design� is� similar� in� some� ways� to� the� two-factor� mixed-effects� design� except�with�one�subject�per�cell��In�other�words,�the�one-factor�repeated�measures�design�is� really�a�special�case�of�the�two-factor�mixed-effects�design�with�n�=�1�per�cell��Unequal�n’s� can�only�happen�when�subjects�miss�the�administration�of�one�or�more�levels�of�factor�A�
On�the�down�side,�the�repeated�measures�design�includes�some�risk�of�carryover�effects�from� one�level�of�A�to�another�because�each�subject�responds�to�all�levels�of�A��As�examples�of�the� carryover�effect,�subjects’�performance�may�be�altered�due�to�fatigue�(decreased�performance),�
494 An Introduction to Statistical Concepts
practice� (increased� performance),� or� sensitization� (increased� performance)� effects�� These� effects�may�be�minimized�by�(a)�counterbalancing�the�order�of�administration�of�the�levels�of� A�so�that�each�subject�does�not�receive�the�same�order�of�the�levels�of�A�(this�can�also�minimize� problems�with�the�compound�symmetry�assumption;�see�subsequent�discussion),�(b)�allowing� some�time�to�pass�between�the�administration�of�the�levels�of�A,�or�(c)�matching�or�blocking� similar�subjects�with�the�assumption�of�subjects�within�a�block�being�randomly�assigned�to�a� level�of�A��This�last�method�is�a�type�of�randomized�block�design�(see�Chapter�16)�
15.4.2 layout of data
The� layout� of� the� data� for� the� one-factor� repeated� measures� model� is� shown� in� Table� 15�4�� Here�we�see�the�columns�designated�as�the�levels�of�factor�A�and�the�rows�as�the�subject�� Thus,�the�columns�or�“levels”�of�factor�A�represent�the�different�measurements��An�example� is�measuring�children�on�reading�performance�before,�immediately�after,�and�6�months�after� they�participate�in�a�reading�intervention��Row,�column,�and�overall�means�are�also�shown� in�Table�15�4,�although�the�subject�means�are�seldom�of�any�utility�(and�thus�are�not�reported� in�research�studies)��Here�you�see�that�the�layout�of�the�data�looks�the�same�as�the�two-factor� model,�although�there�is�only�one�observation�per�cell�
15.4.3 aNOVa Model
The�one-factor�repeated�measures�ANOVA�model�is�written�in�terms�of�population�param- eters�as
Y s sij j i ij ij= + + + +µ α α ε( )
where Yij�is�the�observed�score�on�the�dependent�variable�for�individual�i�responding�to�level�j�
of�factor�A μ�is�the�overall�or�grand�population�mean αj�is�the�fixed�effect�for�level�j�of�factor�A si�is�the�random�effect�for�subject�i�of�the�subject�factor (sα)ij�is�the�interaction�between�subject�i�and�level�j εij�is�the�random�residual�error�for�individual�i�in�level�j
The�residual�error�can�be�due�to�measurement�error�and/or�other�factors�not�under�inves- tigation��From�the�model,�you�can�see�this�is�similar�to�the�two-factor�model�only�with�one�
Table 15.4
Layout�for�the�One-Factor�Repeated�Measures� ANOVA
Level of Factor A (Repeated Factor)
Level of Factor S 1 2 … J Row Mean
1 Y11 Y12 … Y1J Y –
1�
2 Y21 Y22 … Y2J Y –
2�
… … … … … … n Yn1 Yn2 YnJ Y
– n�
Column�mean Y –
�1 Y –
�2 … Y –
�J Y –
��
495Random- and Mixed-Effects Analysis of Variance Models
observation�per�cell��Also,�the�fixed�effect�is�denoted�by�α�and�the�random�effect�by�s;�thus,� we�have�a�mixed-effects�model��Lastly,�for�the�equal�n’s�model,�the�effects�for�α�and�sα�sum� to�0�for�each�subject�(or�row)�
The�hypotheses�for�testing�the�effect�of�factor�A�are�as�follows��The�null�hypothesis�indi- cates�that�the�means�for�each�measurement�are�the�same:
H J01 1 2: . . .µ µ µ= = … =
H j11: not all the are equal.µ
The� hypotheses� are� written� in� terms� of� means� because� factor� A� is� a� fixed� effect� (i�e�,� all� sampled�cases�have�been�measured)�
15.4.4 assumptions and Violation of assumptions
Previously� we� described� the� assumptions� for� the� two-factor� mixed-effects� model�� The� assumptions� are� nearly� the� same� for� the� one-factor� repeated� measures� model� (since� it� is� similar�to�the�two-factor�mixed-effects�model)�and�are�again�mainly�concerned�with�the� distribution�of�the�dependent�variable�scores�and�of�the�random�effects�
A� new� assumption� is� known� as� compound symmetry� and� states� that� the� covariances� between� the� scores� of� the� subjects� across� the� levels� of� the� repeated� factor� A� are� constant�� In�other�words,�the�covariances�for�all�pairs�of�levels�of�the�fixed�factor�are�the�same�across� the�population�of�random�effects�(i�e�,�the�subjects)��The�analysis�of�variance�(ANOVA)�is�not� particularly�robust�to�a�violation�of�this�assumption��In�particular,�the�assumption�is�often� violated�when�factor�A�is�time,�as�the�relationship�between�adjacent�levels�of�A�is�stronger� than� when� the� levels� are� farther� apart�� For� example,� consider� the� previous� illustration� of� children� measured� in� reading� performance� before,� after,� and� 6� months� after� intervention�� The�means�of�the�pre-�and�immediate�post-reading�performance�will�likely�be�more�similar� than� the� means� of� the� pre-� and� 6� months� post-reading� performance�� If� the� assumption� is� violated,�three�alternative�procedures�are�available��The�first�is�to�limit�the�levels�of�factor�A� (i�e�,�the�repeated�measures�factor)�either�to�those�that�meet�the�assumption,�or�to�limit�the� number�of�repeated�measures�to�2�(in�which�case,�there�would�be�only�one�covariance�and� thus�nothing�to�assume)��The�second�and�more�plausible�alternative�is�to�use�adjusted�F�tests�� These�are�reported�shortly��The�third�is�to�use�multivariate�analysis�of�variance�(MANOVA),� which�makes�no�compound�symmetry�assumption,�but�is�slightly�less�powerful��For�readers� interested�in�MANOVA,�there�are�a�number�of�excellent�multivariate�textbooks�that�can�be� referred�to�(e�g�,�Hair,�Black,�Babin,�Anderson,�&�Tatham,�2006;�Tabachnick�&�Fidell,�2007)�
Huynh�and�Feldt�(1970)�showed�that�the�compound�symmetry�assumption�is�a�sufficient� but�not�necessary�condition�for�the�validity�of�the�F�test��Thus,�the�F�test�may�also�be�valid� under�less�stringent�conditions��The�necessary�and�sufficient�condition�for�the�validity�of� the�F�test�is�known�as�sphericity��This�assumes�that�the�variance�of�the�difference�scores� for�each�pair�of�factor�levels�is�the�same�(e�g�,�with�J�=�3�levels,�the�variance�of�the�difference� score�between�levels�1�and�2�is�the�same�as�the�variance�of�the�difference�score�between� levels�1�and�3,�which�is�the�same�as�the�variance�of�the�difference�score�between�levels�2� and�3;�thus,�another�type�of�homogeneity�of�variance�assumption)��Further�discussion�of� sphericity�is�beyond�the�scope�of�this�text�(see�Keppel,�1982;�Kirk,�1982;�or�Myers�&�Well,� 1995)��A�summary�of�the�assumptions�and�the�effects�of�their�violation�for�the�one-factor� repeated�measures�design�is�presented�in�Table�15�5�
496 An Introduction to Statistical Concepts
15.4.5 aNOVa Summary Table and expected Mean Squares
The� sources� of� variation� for� this� model� are� similar� to� those� for� the� two-factor� model,� except� that� there� is� no� within-cell� variation�� The� ANOVA� summary� table� is� shown� in� Table�15�6,�where�we�see�the�following�sources�of�variation:�A�(i�e�,�the�repeated�measure),� subjects� (denoted� by� S),� the� SA� interaction,� and� total�� The� test� of� subject� differences� is� of�no�real�interest��Quite�naturally,�we�expect�there�to�be�variation�among�the�subjects�� From�the�table,�we�see�that�although�three�mean�square�terms�can�be�computed,�only�one� F�ratio�results�for�the�test�of�factor�A;�thus,�the�subjects�effect�cannot�be�tested�anyway� as�there�is�no�appropriate�error�term��This�is�subsequently�shown�through�the�expected� mean�squares�
Next� we� need� to� consider� the� sums� of� squares� for� the� one-factor� repeated� measures� model��If�we�take�the�total�sum�of�squares�and�decompose�it,�we�have
SS SS SS SStotal A S SA= + +
These�three�terms�can�then�be�computed�by�statistical�software��The�degrees�of�freedom,� mean�squares,�and�F�ratio�are�determined�as�shown�in�Table�15�6�
Table 15.5
Assumptions�and�Effects�of�Violations:�One-Factor�Repeated�Measures�Model
Assumption Effect of Assumption Violation
Independence Little�is�known�about�the�effects�of�dependence;�however,�based�on�the� fixed-effects�model,�we�might�expect�the�following:
•�Increased�likelihood�of�a�Type�I�and/or�Type�II�error�in�F •�Affects�standard�errors�of�means�and�inferences�about�those�means
Homogeneity�of�variance Little�is�known�about�the�effects�of�heteroscedasticity;�however,�based�on�the� fixed-effects�model,�we�might�expect�the�following:
•�Bias�in�SSSA •�Increased�likelihood�of�a�Type�I�and/or�Type�II�error •�Small�effect�with�equal�or�nearly�equal�n’s •�Otherwise�effect�decreases�as�n�increases
Normality •�Minimal�effect�with�equal�or�nearly�equal�n’s •�Otherwise�substantial�effects
Sphericity •�F�not�particularly�robust •��Consider�usual�F�test,�Geisser–Greenhouse�conservative�F�test,�and�adjusted�
(Huynh–Feldt)�F�test,�if�necessary
Table 15.6
One-Factor�Repeated�Measures�ANOVA� Summary�Table
Source SS df MS F
A SSA J�−�1 MSA MSA/MSSA S SSS n�−�1 MSS SA SSSA (J�−�1)(n�−�1) MSSA Total SStotal N�−�1
497Random- and Mixed-Effects Analysis of Variance Models
The�formation�of�the�proper�F�ratio�is�again�related�to�the�expected�mean�squares��If�H0� is�actually�true�(in�other�words,�the�means�are�the�same�for�each�of�the�measures),�then�the� expected mean squares�are�as�follows:
E A 2MS( ) = σε
E S 2MS( ) = σε
E SA 2MS( ) = σε
where�σε 2�is�the�population�variance�of�the�residual�errors�
If�H0�is�actually�false�(i�e�,�the�means�are�not�the�same�for�each�of�the�measures),�then�the� expected�mean�squares�are�as�follows:
E A 2MS n Js j
j
J
( ) = + + −
= ∑σ σ αε α2 2
1
1/( )
� E S
2MS J s( ) = +σ σε 2
� E SA
2MS s( ) = +σ σε α2
where�σs 2�and�σ αs
2 �represent�variability�due�to�subjects�and�to�the�interaction�of�factor�A�and� subjects,�respectively,�and�other�terms�are�as�before�
As�in�previous�ANOVA�models,�the�proper�F�ratio�should�be�formed�as�follows:
F = +( )/(systematic variability error variability error variabiility)
For�the�one-factor�repeated�measures�model,�MSSA�must�be�used�as�the�error�term�for�the� test�of�A,�and�there�is�no�appropriate�error�term�for�the�test�of�S�or�the�test�of�SA�(although� that� is� fine� as� we� are� not� really� interested� in� those� tests� anyway� since� they� refer� to� the� individual�cases)�
As�noted�earlier�in�the�discussion�of�assumptions�for�this�model,�the�F�test�is�not�very� robust� to� violation� of� the� compound� symmetry� assumption�� This� assumption� is� often� violated� in� education� and� the� behavioral� sciences;� consequently,� statisticians� have� spent� considerable�time�studying�this�problem��Research�suggests�that�the�following�sequential� procedure�be�used�in�the�test�of�factor�A��First,�do�the�usual�F�test�that�is�quite�liberal�in� terms�of�rejecting�H0�too�often��If�H0�is�not�rejected,�then�stop��If�H0�is�rejected,�then�continue� with�step�2,�which�is�to�use�the�Geisser�and�Greenhouse�(1958)�conservative�F�test��For�the� model�being�considered�here,�the�degrees�of�freedom�for�the�F�critical�value�are�adjusted� to�be�1�and�n�−�1��If�H0�is�rejected,�then�stop��This�would�indicate�that�both�the�liberal�and� conservative�tests�reached�the�same�conclusion�to�reject�H0��If�H0�is�not�rejected,�then�the� two� tests� did� not� reach� the� same� conclusion,� and� a� further� test� (a� tiebreaker)� should� be� undertaken��Thus,�in�step�3,�an�adjusted�F�test�is�conducted��The�adjustment�is�known�as� Box’s�(1954b)�correction�(usually�referred�to�as�the�Huynh�and�Feldt�[1970]�procedure)��Here�
498 An Introduction to Statistical Concepts
the�numerator�degrees�of�freedom�are�(J�−�1)ε,�and�the�denominator�degrees�of�freedom�are� (J�−�1)(n�−�1)ε,�where�ε�is�a�correction�factor�(not�to�be�confused�with�the�residual�term�ε)�� The�correction�factor�is�quite�complex�and�is�not�shown�here�(see�Keppel�&�Wickens,�2004;� Myers,�1979;�Myers�&�Well,�1995;�or�Wilcox,�1987)��Most�major�statistical�software�conducts� the�Geisser–Greenhouse�and�Huynh–Feldt� tests��The�Huynh–Feldt� test�is�recommended� due�to�greater�power�(Keppel�&�Wickens,�2004;�Myers�&�Well,�1995);�thus,�when�available,� you�can�simply�use�the�Huynh–Feldt�procedure�rather�than�the�previously�recommended� sequence�
15.4.6 Multiple Comparison procedures
If�the�null�hypothesis�for�repeated�factor�(i�e�,�factor�A)�is�rejected�and�there�are�more�than� two�levels�of�the�factor,�then�the�researcher�may�be�interested�in�which�means�or�combina- tions�of�means�are�different�(in�other�words,�which�measurement�means�differ�from�one� other)��This�could�be�assessed,�as�we�have�seen�in�previous�chapters,�by�the�use�of�some� MCP��In�general,�most�of�the�MCPs�outlined�in�Chapter�12�can�be�used�in�the�one-factor� repeated�measures�model�(see�additional�discussion�in�Keppel�&�Wickens,�2004;�Mickey,� Dunn,�&�Clark,�2004)�
It�has�been�shown�that�these�MCPs�are�seriously�affected�by�a�violation�of�the�compound� symmetry� assumption�� In� this� situation,� two� alternatives� are� recommended�� The� first� alternative�is,�rather�than�using�the�same�error�term�for�each�contrast�(i�e�,�MSSA),�to�use�a� separate�error�term�for�each�contrast�tested��Then�many�of�the�MCPs�previously�covered�in� Chapter�12�can�be�used��This�complicates�matters�considerably�(see�Keppel,�1982;�Keppel�&� Wickens,�2004;�or�Kirk,�1982)��A�second�alternative,�recommended�by�Maxwell�(1980)�and� Wilcox�(1987),�involves�the�use�of�multiple�dependent�t�tests�where�the�α�level�is�adjusted� much�like�the�Bonferroni�procedure��Maxwell�concluded�that�this�procedure�is�better�than� many�of�the�other�MCPs��For�other�similar�procedures,�see�Hochberg�and�Tamhane�(1987)�
15.4.7 alternative aNOVa procedures
There� are� several� alternative� procedures� to� the� one-factor� repeated� measures� ANOVA� model��These�include�the�Friedman�(1937)�test,�as�well�as�others,�such�as�the�Agresti�and� Pendergast�(1986)�test��The�Friedman�test,�like�the�Kruskal–Wallis�test,�is�a�nonparametric� procedure�based�on�ranks��However,�the�Kruskal–Wallis�test�cannot�be�used�in�a�repeated� measures� model� as� it� assumes� that� the� individual� scores� are� independent�� This� is� obvi- ously� not� the� case� in� the� one-factor� repeated� measures� model� where� each� individual� is� exposed�to�all�levels�of�factor�A�
Let�us�outline�how�the�Friedman�test�is�conducted��First,�scores�are�ranked�within�sub- ject��For�instance,�if�there�are�J�=�4�levels�of�factor�A,�then�the�scores�for�each�subject�would� be�ranked�from�1�to�4��From�this,�one�can�compute�a�mean�ranking�for�each�level�of�fac- tor� A�� The� null� hypothesis� essentially� becomes� a� test� of� whether� the� mean� rankings� for� the�levels�of�A�are�equal��The�test�statistic�is�a�χ2�statistic��In�the�case�of�tied�ranks,�either� the�available�ranks�can�be�averaged,�or�a�correction�factor�can�be�used�as�done�with�the� Kruskal–Wallis�test�(see�Chapter�11)��The�test�statistic�is�compared�to�the�critical�value�of� αχ
2 J�−1�(see�Table�A�3)��The�null�hypothesis�that�the�mean�rankings�are�the�same�for�the��levels�
of�factor�A�will�be�rejected�if�the�test�statistic�exceeds�the�critical�value� You� may� also� recall� from� the� Kruskal–Wallis� test� the� problem� with� small� n’s� in� terms�
of� the� test� statistic� not� being� precisely� distributed� as� χ2�� The� same� problem� exists� with� the� Friedman� test� when� J� <� 6� and� n� <� 6,� so� we� suggest� you� consult� the� table� of� critical�
499Random- and Mixed-Effects Analysis of Variance Models
values� in� Marascuilo� and� McSweeney� (1977,� Table� A-22,� p�� 521)�� The� Friedman� test,� like� the� Kruskal–Wallis� test,� assumes� that� the� population� distributions� have� the� same� shape� (although�not�necessarily�normal)�and�variability�and�that�the�dependent�measure�is�con- tinuous��For�a�discussion�of�other�alternative�nonparametric�procedures,�see�Agresti�and� Pendergast�(1986),�Myers�and�Well�(1995),�and�Wilcox�(1987,�1996,�2003)��For�information�on� more�advanced�within-subjects� ANOVA� models,� see�Cotton�(1998),� Keppel� and�Wickens� (2004),�and�Myers�and�Well�(1995)�
Various�MCPs�can�be�used�for�the�Friedman�test��For�the�most�part,�these�MCPs�are�ana- logs�to�their�parametric�equivalents��In�the�case�of�planned�(or�a�priori)�pairwise�compari- sons,�one�may�use�multiple�matched-pair�Wilcoxon�tests�(i�e�,�a�form�of�the�Kruskal–Wallis� test�for�two�groups)�in�a�Bonferroni�form�(i�e�,�taking�the�number�of�contrasts�into�account� through�an�adjustment�of�the�α�level;�for�example,�if�there�are�six�contrasts�with�an�alpha� of� �05,� the� adjusted� alpha� would� be� �05/6,� or� �008)�� For� post� hoc� comparisons,� numerous� parametric� analogs� are� available�� For� additional� discussion� on� MCPs� for� this� model,� see� Marascuilo�and�McSweeney�(1977)�
15.4.8 example
Let�us�consider�an�example�to�illustrate�the�procedures�used�for�this�model��The�data�are� shown�in�Table�15�7,�where�there�are�eight�subjects,�each�of�whom�has�been�evaluated�by� four� raters� on� a� task� of� writing� assessment�� First,� let� us� take� a� look� at� the� results� for� the� parametric�ANOVA�model,�as�shown�in�Table�15�8��The�F�test�statistic�is�compared�to�the� usual�F�test�critical�value�of��05F3,21�=�3�07,�which�is�significant��For�the�Geisser–Greenhouse� conservative� procedure,� the� test� statistic� is� compared� to� the� critical� value� of� �05F1,7� =� 5�59,� which� is�also�significant��The�two�procedures�both�yield�a�statistically�significant� result;� thus,�we�need�not�be�concerned�with�a�violation�of�the�compound�symmetry�assumption�� As�an�example�MCP,�the�Bonferroni�procedure�determined�that�all�pairs�of�raters�are�sig- nificantly�different�from�one�another,�except�for�rater�1�versus�rater�2�
Finally,�let�us�take�a�look�at�the�Friedman�test��The�test�statistic�is�χ2�=�22�9500��This�test� statistic�is�compared�to�the�critical�value��05χ23�=�7�8147,�which�is�significant��Thus,�the�con- clusions�for�the�parametric�ANOVA�and�nonparametric�Friedman�tests�are�the�same�here�� This�will�not�always�be�the�case,�particularly�when�ANOVA�assumptions�are�violated�
Table 15.7
Data�for�the�Writing�Assessment�Example�One-Factor�Design:� Raw Scores�and�Rank�Scores�on�the�Writing�Assessment�Task� by Subject�and�Rater
Rater 1 Rater 2 Rater 3 Rater 4
Subject Raw Rank Raw Rank Raw Rank Raw Rank
1 3 1 4 2 7 3 8 4 2 6 2 5 1 8 3 9 4 3 3 1 4 2 7 3 9 4 4 3 1 4 2 6 3 8 4 5 1 1 2 2 5 3 10 4 6 2 1 3 2 6 3 10 4 7 2 1 4 2 5 3 9 4 8 2 1 3 2 6 3 10 4
500 An Introduction to Statistical Concepts
15.5 Two-Factor Split-Plot or Mixed Design
In� this� section,� we� describe� the� distinguishing� characteristics� of� the� two-factor� split- plot� or� mixed� ANOVA� design,� the� layout� of� the� data,� the� linear� model,� assumptions� and their� violation,� the� ANOVA� summary� table� and� expected� mean� squares,� MCPs,� and�an�example�
15.5.1 Characteristics of the Model
The�characteristics�of�the�two-factor�split-plot�or�mixed�ANOVA�design�are�a�combina- tion�of�the�characteristics�of�the�one-factor�repeated�measures�and�the�two-factor�fixed- effects�models��It�is�unique�because�there�are�two�factors,�only�one�of�which�is�repeated�� For�this�reason,�the�design�is�often�called�a�mixed design��Thus,�one�of�the�factors�is�a� between-subjects�factor,�the�other�is�a�within-subjects�factor,�and�the�result�is�known�as�a� split-plot design�(from�agricultural�research)��Each�subject�then�responds�to�every�level� of�the�repeated�factor�but�to�only�one�level�of�the�nonrepeated�factor��Subjects�then�serve� as�their�own�controls�for�the�repeated�factor�but�not�for�the�nonrepeated�factor��The�other� characteristics� carry� over� from� the� one-factor� repeated� measures� model� and� the� two- factor�model�
15.5.2 layout of data
The�layout�of�the�data�for�the�two-factor�split-plot�or�mixed�design�is�shown�in�Table�15�9�� Here� we� see� the� rows� designated� as� the� levels� of� factor� A,� the� between-subjects� or� non- repeated�factor,�and�the�columns�as�the�levels�of�factor�B,�the�within-subjects�or�repeated� factor��Within�each�factor�level�combination�or�cell�are�the�subjects��Notice�that�the�same� subjects�appear�at�all�levels�of�factor�B�(the�within-subjects�factor,�the�repeated�measure)� but�only�at�one�level�of�factor�A�(the�between-subjects�factor)��Row,�column,�cell,�and�over- all�means�are�also�shown��Here�you�see�that�the�layout�of�the�data�looks�much�the�same�as� the�two-factor�model�
Table 15.8
One-Factor�Repeated�Measures�ANOVA� Summary�Table�for�the�Writing�Assessment� Example
Source SS df MS F
Within�subjects Rater�(A) 198�125 3 66�042 73�477a
Error�(SA) 18�875 21 �899
Between�subjects Error�(S) 14�875 7 2�125
Total 231�875 31
a� �05F3,21�=�3�07�
501Random- and Mixed-Effects Analysis of Variance Models
15.5.3 aNOVa Model
The�two-factor�split-plot�model�can�be�written�in�terms�of�population�parameters�as
Y s sijk j i j k jk ki j ijk= + + + + + +( )µ α β αβ β ε( ) ( ) ( )
where Yijk�is�the�observed�score�on�the�dependent�variable�for�individual�i�in�level�j�of�factor�A�
(the�between-subjects�factor)�and�level�k�of�factor�B�(i�e�,�the�jk�cell,�the�within-subjects� factor�or�repeated�measure)
μ�is�the�overall�or�grand�population�mean�(i�e�,�regardless�of�cell�designation) αj�is�the�effect�for�level�j�of�factor�A�(row�effect�for�the�nonrepeated�factor)
Table 15.9
Layout�for�the�Two-Factor�Split-Plot�or�Mixed�ANOVA
Level of Factor A (Nonrepeated Factor)
Level of Factor A (Repeated Factor)
1 2 … K Row Mean
1 Y111 Y112 … Y11K � � … � � � … � Y
– �1�
� � … � Yn11 Yn12 … Yn1K — — — Y –
�11 Y –
�12 Y –
�1K
2 Y121 Y122 … Y12K � � … � � � … � Y
– �2�
� � … � Yn21 Yn22 … Yn2K — — — Y –
�21 Y –
�22 … Y –
�2K
� � � … � � � � � … � � � � � … � �
J Y1J1 Y1J2 … Y1JK � � … � � � … � Y
– �J�
� � … � YnJ1 YnJ2 … YnJK — — — Y –
�J1 Y –
�J2 … Y –
�JK
Column�Mean Y –
��1 Y –
��2 Y –
��K Y –
…
Note:� Each� subject� is� measured� at� all� levels� of� factor� B,� but� at� only� one�level�of�factor�A�
502 An Introduction to Statistical Concepts
si(j)�is�the�effect�of�subject�i�that�is�nested�within�level�j�of�factor�A�(i�e�,�i(j)�denotes�that� i�is�nested�within�j)
βk�is�the�effect�for�level�k�of�factor�B�(column�effect�for�the�repeated�factor) (αβ)jk�is�the�interaction�effect�for�the�combination�of�level�j�of�factor�A�and�level�k�of�factor�B (βs)ki(j)� is� the� interaction� effect� for� the� combination� of� level� k� of� factor� B� (the� within-
subjects� factor,� the� repeated� measure)� and� subject� i� that� is� nested� within� level� j� of� factor�A�(the�between-subjects�factor)
εijk�is�the�random�residual�error�for�individual�i�in�cell�jk
We�use�the�terminology�“subjects�are�nested�within�factor�A”�to�indicate�that�a�particular� subject�si�is�only�exposed�to�one�level�of�factor�A�(the�between-subjects�factor),�level�j��This� observation� is� then� denoted� in� the� subjects� effect� by� si(j)� and� in� the� interaction� effect� by� (βs)ki(j)��This�is�due�to�the�fact�that�not�all�possible�combinations�of�subject�with�the�levels� of�factor�A�are�included�in�the�model��A�more�extended�discussion�of�designs�with�nested� factors� is� given� in� Chapter� 16�� The� residual� error� can� be� due� to� individual� differences,� �measurement�error,�and/or�other�factors�not�under�investigation��We�assume�for�now�that� A�and�B�are�fixed-effects�factors�and�that�S�is�a�random-effects�factor�
It�should�be�mentioned�that�for�the�equal�n’s�model,�the�sum�of�the�row�effects,�the�sum�of� the�column�effects,�and�the�sum�of�the�interaction�effects�are�all�equal�to�0,�both�across�rows� and� across� columns�� This� implies,� for� example,� that� if� there� are� any� nonzero� row� effects,� then�the�row�effects�will�balance�out�around�0�with�some�positive�and�some�negative�effects�
The�hypotheses�to�be�tested�here�are�exactly�the�same�as�in�the�nonrepeated�two-factor� ANOVA� model� (see� Chapter� 13)�� For� the� two-factor� ANOVA� model,� there� are� three� sets� of�hypotheses,�one�for�each�of�the�main�effects�and�one�for�the�interaction�effect��The�null� and�alternative�hypotheses,�respectively,�for�testing�the�main�effect�of�factor�A�(between- subjects�factor)�are�as�follows:
H J01 1 2: . . . . . .µ µ µ= = … =
H j11 not all the are equal: . .µ
The� hypotheses� for� testing� the� main� effect� of� factor� B� (within-subjects� factor,� i�e�,� the� repeated�measure)�are�noted�as�follows:
H K02 1 2: .. .. ..µ µ µ= = … =
H k12 : not all the are equal..µ
Finally,�the�hypotheses�for�testing�the�interaction�effect�(between�by�within�factors)�are�as� follows:
H j kjk j k03: ( ) 0 for all and. . . ..µ µ µ µ− − + =
H jk j k13 : not all the ( ). . . ..µ µ µ µ− − + = 0
If� one� of� the� null� hypotheses� is� rejected,� then� the� researcher� may� want� to� consider� an� MCP�so�as�to�determine�which�means�or�combination�of�means�are�significantly�differ- ent�(discussed�later�in�this�chapter)�
503Random- and Mixed-Effects Analysis of Variance Models
15.5.4 assumptions and Violation of assumptions
Previously�we�described�the�assumptions�for�the�different�two-factor�models�and�the�one- factor� repeated� measures� model�� The� assumptions� for� the� two-factor� split-plot� or� mixed� design�are�actually�a�combination�of�these�two�sets�of�assumptions�
The� assumptions� can� be� divided� into� two� sets� of� assumptions,� one� for� the� between- subjects� factor� and� one� for� the� within-subjects� (or� repeated� measures)� factor�� For� the� between-subjects�factor,�we�have�the�usual�assumptions�of�population�scores�being�ran- dom,�independent,�and�normally�distributed�with�equal�variances��For�the�within-sub- jects�factor�(i�e�,�the�repeated�measure),�the�assumption�is�the�already�familiar�compound� symmetry�assumption��For�this�design,�the�assumption�involves�the�population�covari- ances�for�all�pairs�of�the�levels�of�the�within-subjects�factor�(i�e�,�k�and�k’)�being�equal,�at� each�level�of�the�between-subjects�factor�(for�all�levels�j)��To�deal�with�this�assumption,� we� look� at� alternative� F� tests� in� the� next� section�� A� summary� of� the� assumptions� and� the� effects� of� their� violation� for� the� two-factor� split-plot� or� mixed� design� is� presented� in Table�15�10�
15.5.5 aNOVa Summary Table and expected Mean Squares
The�ANOVA�summary�table�is�shown�in�Table�15�11,�where�we�see�the�following�sources�of� variation:�A,�S,�B,�AB,�BS,�and�total��The�table�is�divided�into�within-subjects�sources�and� between-subjects�sources��The�between-subjects�sources�are�A�and�S,�where�S�will�be�used� as� the� error� term� for� the� test� of� factor� A�� The� within-subjects� sources� are� B,� AB,� and� BS,� where�BS�will�be�used�as�the�error�term�for�the�test�of�factor�B�and�of�the�AB�interaction�� This�will�become�clear�when�we�examine�the�expected�mean�squares�shortly�
Next�we�need�to�consider�the�sums�of�squares�for�the�two-factor�mixed�design��Taking� the�total�sum�of�squares�and�decomposing�it�yields
SS SS SS SS SS SStotal A S B AB BS= + + + +
We�leave�the�computation�of�these�five�terms�for�statistical�software��The�degrees�of�freedom,� mean�squares,�and�F�ratios�are�computed�as�shown�in�Table�15�11�
Table 15.10
Assumptions�and�Effects�of�Violations:�Two-Factor�Split-Plot�or�Mixed�Model
Assumption Effect of Assumption Violation
Independence •�Increased�likelihood�of�a�Type�I�and/or�Type�II�error�in�F •�Affects�standard�errors�of�means�and�inferences�about�those�means
Homogeneity�of�variance •�Bias�in�error�terms •�Increased�likelihood�of�a�Type�I�and/or�Type�II�error •�Small�effect�with�equal�or�nearly�equal�n’s •�Otherwise�effect�decreases�as�n�increases
Normality •�Minimal�effect�with�equal�or�nearly�equal�n’s •�Otherwise�substantial�effects
Sphericity •�F�not�particularly�robust •��Consider�usual�F�test,�Geisser–Greenhouse�conservative�F�test,�and�adjusted�
(Huynh–Feldt)�F�test,�if�necessary
504 An Introduction to Statistical Concepts
The�formation�of�the�proper�F�ratio�is�again�related�to�the�expected�mean�squares��If�H0�is� actually�true�(i�e�,�the�means�are�really�equal),�then�the�expected mean squares�are�as�follows:
E 2MSA( ) = σε
E 2MSS( ) = σε
E 2MSB( ) = σε
� E 2MSAB( ) = σε
� E 2MSBS( ) = σε
where�σε 2�is�the�population�variance�of�the�residual�errors�
If�H0�is�actually�false�(i�e�,�the�means�are�really�not�equal),�then�the�expected�mean�squares� are�as�follows:
E 2MS K nK Js j j
J
A( ) = + + −
= ∑σ σε 2 2
1
1α /( )
� E 2MS K sS( ) = +σ σε 2
�
E 2MS nJ Ks k k
K
B( ) = + + −
=
∑σ σ βε β2 2 1
1/( )
�
E 2MS n J Ks jk k
K
j
J
AB( ) = + + − −
== ∑∑σ σ αβε β2 2
11
1 1( ) /( )( )
Table 15.11
Two-Factor�Split-Plot�or�Mixed�Model�ANOVA� Summary�Table
Source SS df MS F
Between�subjects
A SSA J�−�1 MSA MSA/MSS S SSS J(n�−�1) MSS
Within�subjects
B SSB K�−�1 MSB MSB/MSBS AB SSAB (J�−�1)�(K�−�1) MSAB MSAB/MSBS BS SSBS (K�−�1)�J(n�−�1) MSBS Total SStotal N�−�1
505Random- and Mixed-Effects Analysis of Variance Models
E 2MS sBS( ) = +σ σε β2
where�σβ s 2 � represents� variability� due� to� the� interaction� of� factor� B� (the� within-subjects� or�
repeated�measures�factor)�and�subjects,�and�the�other�terms�are�as�before� As�in�previous�ANOVA�models,�the�proper�F�ratio�should�be�formed�as�follows:
F = +( ) /(systematic variability error variability error variabiility)
For� the� two-factor� split-plot� design,� the� error� term� for� the� proper� test� of� factor� A� (the� between-subjects� factor)� is� the� S� term,� whereas� the� error� term� for� the� proper� tests� of� factor�B�(the�within-subjects�or�repeated�measures�factor)�and�the�AB�interaction�is�the� BS�interaction��For�models�where�factors�A�and�B�are�not�both�fixed-effects�factors,�see� Keppel�(1982)�
As�the�compound�symmetry�assumption�is�often�violated,�we�again�suggest�the�follow- ing�sequential�procedure�to�test�for�B�(the�repeated�measure)�and�for�AB�(the�within-�by� between-subjects� factor� interaction)�� First,� do� the� usual� F� test,� which� is� quite� liberal� in� terms�of�rejecting�H0�too�often��If�H0�is�not�rejected,�then�stop��If�H0�is�rejected,�then�con- tinue�with�step�2,�which�is�to�use�the�Geisser�and�Greenhouse�(1958)�conservative�F�test�� For� the� model� under� consideration� here,� the� degrees� of� freedom� for� the� F� critical� values� are�adjusted�to�be�1�and�J(n�−�1)�for�the�test�of�B,�and�J�−�1�and�J(n�−�1)�for�the�test�of�the� AB�interaction��There�is�no�conservative�test�necessary�for�factor�A,�the�between-subjects� or�nonrepeated�factor,�as�the�assumption�does�not�apply;�thus,�the�usual�test�is�all�that�is� necessary�for�the�test�of�A��If�H0�for�B�and/or�AB�is�rejected,�then�stop��This�would�indicate� that�both�the�liberal�and�conservative�tests�reached�the�same�conclusion�to�reject�H0��If�H0� is�not�rejected,�then�the�two�tests�did�not�yield�the�same�conclusion,�and�an�adjusted�F�test� is�conducted��The�adjustment�is�known�as�Box’s�(1954b)�correction�(or�the�Huynh�and�Feldt� [1970]� procedure)�� Most� major� statistical� software� conducts� the� Geisser–Greenhouse� and� Huynh–Feldt�tests�
15.5.6 Multiple Comparison procedures
Consider�the�situation�where�the�null�hypothesis�for�any�of�the�three�hypotheses�is�rejected� (i�e�,� for� A,� B,� and/or� AB)�� If� there� is� more� than� one� degree� of� freedom� in� the� numerator� for� any� of� these� hypotheses,� then� the� researcher� may� be� interested� in� which� means� or� combinations� of� means� are� different�� This� could� be� assessed� again� by� the� use� of� some� MCP��Thus,�the�procedures�outlined�in�Chapter�13�(i�e�,�for�main�effects�and�for�simple�and� complex� interaction� contrasts)� for� the� regular� two-factor� ANOVA� model� can� be� adapted� to�this�model�
However,� it� has� been� shown� that� the� MCPs� involving� the� repeated� factor� are� seri- ously� affected� by� a� violation� of� the� compound� symmetry� assumption�� In� this� situa- tion,�two�alternatives�are�recommended��The�first�alternative�is,�rather�than�using�the� same�error�term�for�each�contrast�involving�the�repeated�factor�(i�e�,�MSB�or�MSAB),�to� use�a�separate�error�term�for�each�contrast�tested��Then�many�of�the�MCPs�previously� covered�in�Chapter�12�can�be�used��This�complicates�matters�considerably�(see�Keppel,� 1982;� Keppel� &� Wickens,� 2004;� or� Kirk,� 1982)�� The� second� and� simpler� alternative� is�
506 An Introduction to Statistical Concepts
suggested� by� Shavelson� (1988)�� He� recommended� that� the� appropriate� error� terms� be� used�in�MCPs�involving�the�main�effects,�but�for�interaction�contrasts,�both�error�terms� be�pooled�(or�added)�together�(this�procedure�is�conservative�yet�simpler�than�the�first� alternative)�
15.5.7 example
Consider� now� an� example� problem� to� illustrate� the� two-factor� mixed� design�� Here� we� expand�on�the�example�presented�earlier�in�this�chapter�by�adding�a�second�factor�to�the� model��The�data�are�shown�in�Table�15�12,�where�there�are�eight�subjects,�each�of�whom� has� been� evaluated� by� four� raters� on� a� task� of� writing� assessment� (rater� is� the� within- subjects�factor�as�each�individual�has�been�evaluated�by�four�raters)��Ratings�on�the�writ- ing�assessment�can�range�from�1�(lowest�rating)�to�10�(highest�rating)��Each�student�was� also�randomly�assigned�to�one�of�two�instructors��Thus,�factor�A�represents�the�instruc- tors�of�English�composition,�where�we�see�that�four�subjects�are�randomly�assigned�to� level� 1� of� factor� A� (i�e�,� instructor� 1)� and� the� remaining� four� to� level� 2� of� factor� A� (i�e�,� instructor�2)��Thus,�factor�B�(i�e�,�rater)�is�repeated�(the�within-subjects�factor),�and�factor� A� (i�e�,� instructor)� is� not� repeated� (the� between-subjects� factor)�� The� ANOVA� summary� table�is�shown�in�Table�15�13�
The� test� statistics� are� compared� to� the� following� usual� F� test� critical� values:� for� factor� A� (the� between-subjects� factor� that� tests� mean� differences� based� on� instruc- tor),��05F1,6�=�5�99,�which�is�not�statistically�significant;�for�factor�B�(the�within-subjects� factor� that� tests� mean� differences� based� on� repeated� ratings),� �05F3,18� =� 3�16,� which� is� significant;� and� for� AB,� �05F3,18� =� 3�16,� which� is� also� statistically� significant�� For� the� Geisser–Greenhouse� conservative� procedure,� the� test� statistics� are� compared� to� the� following� critical� values:� for� factor� A� (i�e�,� instructor),� no� conservative� procedure� is� necessary;� for� factor� B� (i�e�,� repeated� measure� rater),� �05F1,6� =� 5�99,� which� is� also� sig- nificant;� and� for� the� interaction� AB� (instructor� by� rater),� �05F1,6� =� 5�99,� which� is� also� significant�� The� usual� and� Geisser–Greenhouse� procedures� both� yield� a� statistically� significant� result� for� factor� B� (rater)� and� for� the� interaction� AB� (instructor� by� rater);�
Table 15.12
Data�for�the�Writing�Assessment�Example�Two-Factor� Design:�Raw�Scores�on�the�Writing�Assessment�Task� by Instructor�and�Rater
Factor A (Nonrepeated Factor) Factor B (Repeated Factor)
Instructor Subject Rater 1 Rater 2 Rater 3 Rater 4
1 1 3 4 7 8 2 6 5 8 9 3 3 4 7 9 4 3 4 6 8
2 5 1 2 5 10 6 2 3 6 10 7 2 4 5 9 8 2 3 6 10
507Random- and Mixed-Effects Analysis of Variance Models
thus,�we�need�not�be�concerned�with�a�violation�of�the�sphericity�assumption��A�pro- file�plot�of�the�interaction�is�shown�in�Figure�15�2�
There�is�a�significant�AB�(i�e�,�instructor�by�rater)�interaction,�so�we�should�follow�this�up� with�simple�interaction�contrasts,�each�involving�only�four�cell�means��As�an�example�of� an�MCP,�consider�the�contrast
ψ’ = − − −
= − − −( ) ( ) ( . . ) ( . .. . . .Y Y Y Y11 21 14 24
4 3 7500 1 7500 8 5000 9 7500))
.8125 4
=
Table 15.13
Two-Factor�Split-Plot�ANOVA�Summary�Table� for the�Writing�Assessment�Example
Source SS df MS F
Between�subjects
Instructor�(A) 6�125 1 6�125 4�200b
Error�(S) 8�750 6 1�458
Within�subjects
Rater�(B) 198�125 3 66�042 190�200a
Instructor��rater 12�625 3 4�208 12�120a
Error�(BS) 6�250 18 �347
Total 231�875 31 a�
�05F3,18�=�3�16� b�
�05F1,6�=�5�99�
Es tim
at ed
m ar
gi na
l m ea
ns
Estimated marginal means of MEASURE_1
2.00
4.00
6.00
8.00
10.00
.00
Rater 1 2 3 4
Instructor
Instructor 2 Instructor 1
FIGuRe 15.2 Profile�plot�for�example�writing�data�
508 An Introduction to Statistical Concepts
with�a�standard�error�computed�as�follows:
se MS
c
n
jk
k
K
j
J
jk ψ’ BS=
= + +==
∑∑ 2 11 0 3472
1 16 1 16 .
( / / 11 16 1 16 4
0 1473 / / )
. +
=
Using�the�Scheffé�procedure,�we�formulate�the�following�as�the�test�statistic:
t se
= = = ψ
ψ
’
’
0 8125 0 1473
5 5160 . .
.
This�is�compared�with�the�critical�value�presented�here:
( )( ) ( ) ( . ) .( )( ),( ) ( ) . ,J K F FJ K K J n− − = = =− − − −1 1 3 3 3 16 3 01 1 1 1 05 3 18α 7790
Thus,�we�may�conclude�that�the�tetrad�interaction�difference�between�the�first�and�second� levels�of�factor�A�(instructor)�and�the�first�and�fourth�levels�of�factor�B�(rater,�the�repeated� measure)�is�significant��In�other�words,�rater�1�finds�better�writing�among�the�students�of� instructor�1�than�instructor�2,�whereas�rater�4�finds�better�writing�among�the�students�of� instructor�2�than�instructor�1�
Although� we� have� only� considered� the� basic� repeated� measures� designs� here,� more� complex�repeated�measures�designs�also�exist��For�further�information,�see�Myers�(1979),� Keppel�(1982),�Kirk�(1982),�Myers�and�Well�(1995),�Glass�and�Hopkins�(1996),�Cotton�(1998),� Keppel�and�Wickens�(2004),�as�well�as�alternative�ANOVA�procedures�described�by�Wilcox� (2003)�and�McCulloch�(2005)��To�analyze�repeated�measures�designs�in�SAS,�use�the�GLM� procedure� with� the� REPEATED� statement�� In� SPSS� GLM,� use� the� repeated� measures� program�
15.6 SPSS and G*Power
Next�we�consider�SPSS�for�the�models�presented�in�this�chapter��Note�that�all�of�the�designs� in�this�chapter�are�discussed�in�the�SPSS�context�by�Page,�Braver,�and�MacKinnon�(2003)�� This�is�followed�by�an�illustration�of�the�use�of�G*Power�for�post�hoc�and�a�priori�power� analysis�for�the�two-factor�split-plot�ANOVA�
One-Factor Random-Effects ANOVA
To�conduct�a�one-factor�random-effects�ANOVA�analysis,�there�are�only�two�differences� from�the�one-factor�fixed-effects�ANOVA�(Chapter�11)��Otherwise,�the�form�of�the�data� and�the�conduct�of�the�analyses�are�exactly�the�same��In�terms�of�the�form�of�the�data,�one� column�or�variable�indicates�the�levels�or�categories�of�the�independent�variable�(i�e�,�the�
509Random- and Mixed-Effects Analysis of Variance Models
random�factor),�and�the�second�is�for�the�dependent�variable��Each�row�then�represents� one�individual,�indicating�the�level�or�group�that�individual�is�a�member�of�(1,�2,�3,�or�4�in� our�example;�recall�that�for�the�one-factor�random-effects�ANOVA,�these�categories�are� randomly�selected�from�the�population�of�categories),�and�their�score�on�the�dependent� variable��Thus,�we�wind�up�with�two�long�columns�of�group�values�and�scores�as�shown� in�the�following�screenshot��We�will�use�the�data�from�Chapter�11�to�illustrate,�this�time� assuming�the�independent�variable�is�a�random�factor�rather�than�fixed�
The form of the data for the one-factor random effects
ANOVA follows that of the one-factor fixed effects
ANOVA. The “independent variable” (which is now a
random rather than fixed effect) is labeled “Group”
where each value represents the category to which the
student was randomly assigned. The categories of
the random factor were randomly selected from the
population of categories.
The “dependent variable” is “Labs” and represents the number of statistics labs the
student attended.
Step 1:� To� conduct� a� one-factor� random-effects� ANOVA,� go� to� “Analyze”� in� the� top� pulldown� menu,� then� select� “General Linear Model,”� and� then� select� “Univariate.”� Following� the� screenshot� (step� 1)� as� follows� produces� the� “Univariate”�dialog�box�
510 An Introduction to Statistical Concepts
One-factor random
effects ANOVA:
Step 1
C B
A
Step 2:�Click�the�dependent�variable�(e�g�,�number�of�statistics�labs�attended)�and�move� it� into� the�“Dependent Variable”� box� by� clicking� the� arrow� button�� Click� the� inde- pendent�variable�(e�g�,�level�of�attractiveness;�this�is�the�random-effects�factor)�and�move� it�into�the�“Random Factors”�box�by�clicking�the�arrow�button��On�this�“Univariate”� dialog�screen,�you�will�notice�that�while�the�“Post hoc” option�button�is�active,�clicking� on�“Post hoc”�will�produce�a�dialog�box�with�no�active�options�as�we�are�now�dealing� with�a�random�factor�rather�than�fixed�factor��Post�hoc�MCPs�are�only�available�from�the� “Options”�screen�as�we�will�see�in�the�following�screenshots�
Univariate
Clicking on “Plots” will allow
you to generate profile plots.
Clicking on “Save” will allow you to
save various forms of residuals, among other
variables.
Clicking on “Options” will allow you to obtain a number of other statistics (e.g., descriptive statistics,
effect size, power, homogeneity tests, and
multiple comparison procedures).
Select the dependent variable from the list on the
left and use the arrow to move to the “Dependent Variable”
box on the right. Select the random factor from the list on the left and use the arrow to move to the “Random Factor(s)”
box on the right.
One-factor random
effects ANOVA:
Step 2
511Random- and Mixed-Effects Analysis of Variance Models
Step 3:� Clicking� on� “Options”� provides� the� option� to� select� such� information� as� “Descriptive Statistics,” “Estimates of effect size,” “Observed power,”� and�“Homogeneity tests”� (i�e�,� Levene’s� test� for� equal� variances)�� Click� on� “Continue” to�return�to�the�original�dialog�box��Note that if you are interested in an MCP, post hoc MCPs are only available from the�“Options”�screen��To�select�a�post�hoc�procedure,� click�on�“Compare main effects”�and�use�the�toggle�menu�to�reveal�the�Tukey LSD, Bonferroni,�and�Sidak�procedures�� However,� we�have�already�mentioned�that�MCPs� are�not�generally�of�interest�for�this�model�
While post hoc MCPs are usually not of interest in random
effects models, if you wish to conduct a post hoc test, that selection
must be made from this screen using the “Compare main
effects” option then selecting one of the three MCPs that are available
from the toggle menu under “Confidence interval adjustment” (i.e., LSD,
Bonferroni, or Sidak).
One-factor random effects ANOVA: Step 3
Select from the list on the left those variables that you wish to display means for and use the arrow to move to the
“Display means for” box on the right.
Step 4:�From�the�“Univariate”�dialog�box,�click�on�“Plots”�to�obtain�a�profile�plot� of� means�� Click� the� random� factor� (e�g�,� level� of� attractiveness� labeled� as� “Group”)� and� move�it�into�the�“Horizontal Axis”�box�by�clicking�the�arrow�button�(see�screenshot� step�4a)��Then�click�on�“Add”�to�move�the�variable�into�the�“Plots”�box�at�the�bottom� of�the�dialog�box�(see�screenshot�step�4b)��Click�on�“Continue”�to�return�to�the�original� dialog�box�
512 An Introduction to Statistical Concepts
One-factor random
effects ANOVA: Step 4a
Select the random factor from the list on the left and
use the arrow to move to the “Horizontal Axis” box on
the right.
One-factor random
effects ANOVA: Step 4b
�en click “Add” to move the variable
into the “Plots” box at the bottom.
513Random- and Mixed-Effects Analysis of Variance Models
Step 5:�From�the�“Univariate”�dialog�box�(see�screenshot�step�2),�click�on�“Save”�to� select�those�elements�that�you�want�to�save��In�our�case,�we�want�to�save�the�unstandard- ized�residuals�which�will�be�used�later�to�examine�the�extent�to�which�normality�and�inde- pendence�are�met��Thus,�place�a�checkmark�in�the�box�next�to�“Unstandardized.”�Click� “Continue”�to�return�to�the�main�“Univariate”�dialog�box��From�the�“Univariate”� dialog�box,�click�on “Ok”�to�return�to�generate�the�output�
One-factor random
effects ANOVA: Step 5
Two-Factor Random-Effects ANOVA
To�run�a�two-factor�random-effects�ANOVA�model,�there�are�the�same�two�differences� from� the� two-factor� fixed-effects� ANOVA� (covered� in� Chapter� 13)�� First,� on� the� GLM� screen�(shown�in�the�following�screenshot),�click�both�factor�names�into�the�“Random Factor(s)”�box�rather�than�the�“Fixed Factor(s)”�box��Second,�the�same�situation� exists� with� MCPs:� if� you� are� interested� in� an� MCP,� post� hoc� MCPs� are� only� available� from�the�“Options”�screen��However,�we�have�already�mentioned�that�MCPs�are�not� generally� of� interest� for� this� model�� For� brevity,� the� subsequent� screenshots� are� not� presented�
514 An Introduction to Statistical Concepts
Two-factor random-
effects ANOVA
Select the dependent variable from the list on the
left and use the arrow to move it to the “Dependent Variable” box on
the right. Select the random factors from the
list on the left and use the arrow to move them to the
“Random Factor(s)” box on
the right.
Clicking on “Plots” will allow you to generate profile
plots.
Clicking on “Save” will allow you to
save various forms of residuals, among other
variables.
Clicking on “Options” will allow you to obtain a
number of other statistics (e.g., descriptive statistics,
effect size, power, homogeneity tests, and
multiple comparison procedures).
Two-Factor Mixed-Effects ANOVA
To�conduct�a�two-factor�mixed-effects�ANOVA,�there�are�three�differences�from�the�two- factor�fixed-effects�ANOVA�when�using�SPSS�to�analyze�the�model��The�first�is�that�both� a�random-�and�a�fixed-effects�factor�must�be�defined�(see�screenshot�step�2�that�follows)�� The�second�difference�is�that�post�hoc�MCPs�for�the�fixed-effects�factor�are�available�from� either�the�“Post Hoc”�or�“Options”�screens,�while�for�the�random-effects�factor,�they� are�only�available�from�the�“Options”�screen��The�third�difference�is�related�to�the�out- put�provided�by�SPSS��Unfortunately�the�F�statistic�for�any�main�effect�that�is�random�in�a� mixed-effects�model�is�computed�incorrectly�in�SPSS�because�the�wrong�error�term�is�used� when�implementing�the�SPSS�point-and-click�mode��As�described�in�Lomax�and�Surman� (2007)� and� extended� by� Li� and� Lomax� (2011),� you� need� to� (a)� compute� the� F� statistics� by� hand�from�the�MS�values�(which�are�correct),�(b)�use�SPSS�syntax�where�the�user�indicates� the�proper� error� terms,�or�(c)�use�a�different�software�package� (e�g�,�SAS,�where�the�user� also�provides�the�proper�error�terms)��These�options�are�not�presented�here��Rather,�read- ers�are�referred�to�the�appropriate�references��For�the�purpose�of�this�illustration,�we�will� use�the�statistics�lab�data��The�dependent�variable�remains�the�same—the�number�of�sta- tistics�labs�attended��The�level�of�attractiveness�will�be�a�fixed�factor,�and�the�time�of�day� will�be�a�random�factor�
Step 1:�To�conduct�a�one-factor�fixed-effects�ANOVA,�go�to�“Analyze”�in�the�top�pull- down�menu,�then�select�“General Linear Model,”�and�then�select�“Univariate.”� Following� screenshot� step� 1� for� the� one-factor� random-effects� ANOVA� presented� previ- ously�produces�the�“Univariate”�dialog�box�
515Random- and Mixed-Effects Analysis of Variance Models
Step 2:� Per� screenshot� step� 2� that� follows,� click� the� dependent� variable� (e�g�,� number� of� statistics� labs� attended)� and� move� it� into� the� “Dependent Variable”� box� by� clicking� the� arrow� button�� Click� the� fixed� factor� (e�g�,� level� of� attractiveness)� and� move� it� into� the� “Fixed Factors”�box�by�clicking�the�arrow�button��Click�the�random�factor�(e�g�,�time�of� day)�and�move�it�into�the�“Random Factors”�box�by�clicking�the�arrow�button��Next,�click� on�“Options.”�Please�note�that�post�hoc�MCPs�for�the�fixed-effects�factor�(in�this�case,�level� of�attractiveness)�are�available�from�either�the�“Post Hoc”�or�“Options”�screens,�while� for�the�random-effects�factor,�they�are�only�available�from�the�“Options” screen��Because� these�steps�have�been�presented�in�previous�screenshots�(e�g�,�Chapter�12�for�MCPs�and�the� one-factor�random-effects�previously�shown�in�this�chapter),�they�are�not�repeated�here�
Two-factor mixed-
effects ANOVA: Step 2
Select the dependent variable from the list on the
left and use the arrow to move it to the “Dependent Variable” box on
the right. Select the random
factor (or fixed factor) from the
list on the left and use the arrow to
move it to the “Random
Factor(s)” (or “Fixed Factor(s)”)
box on the right.
Clicking on “Plots” will allow you to generate profile
plots.
Clicking on “Save” will allow you to
save various forms of residuals, among other
variables.
Clicking on “Options” will allow you to obtain a
number of other statistics (e.g., descriptive statistics,
effect size, power, homogeneity tests, and
multiple comparison procedures).
One-Factor Repeated Measures ANOVA
In�order�to�run�a�one-factor�repeated�measures�ANOVA�model,�the�data�have�to�be�in�the�form� suggested�by�the�following�screenshot��Each�row�represents�one�person�in�our�sample��All�of� the�scores�for�each�subject�must�be�in�one�row�of�the�dataset,�and�each�level�of�the�repeated�fac- tor�is�a�separate�variable�(represented�by�the�columns)��For�example,�if�there�are�four�raters�who� assess�each�student’s�essay,�there�will�be�variables�for�each�rater�(e�g�,�rater�1�through�rater�4;� example�dataset�on�the�website)��In�this�illustration,�we�have�both�raw�scores�and�ranked�data� for�each�of�the�four�raters��When�using�ANOVA�for�repeated�measures,�we�will�apply�the�raw� scores��The�ranked�scores�will�only�be�of�value�when�computing�the�nonparametric�version�of� ANOVA�(i�e�,�the�Friedman�test)�which�will�be�covered�later�in�this�chapter�
516 An Introduction to Statistical Concepts
For the repeated measures ANOVA, each row represents one person in our sample. Each column
represents one level of the repeated measures factor.
For this illustration, four raters assessed the writing essay of each person in the sample, thus there are four columns that represent the raw scores of each of the raters (Rater1_raw, Rater2_raw, etc.) and four scores that represent the ranked scores of each of the raters
(Rater1_rank, Rater2_rank, etc.).
Step 1:�To�conduct�a�one-factor�repeated�measures�ANOVA,�go�to�“Analyze”�in�the�top� pulldown� menu,� then� select�“General Linear Model,”� and� then� select�“Repeated Measures.”� Following� the� screenshot� (step� 1)� as� follows� produces� the� “Repeated Measures”�dialog�box�
One-factor repeated
measures ANOVA:
Step 1
C B
A
Step 2:�The�“Repeated Measures Define Factor(s)”�dialog�box�will�appear�(see� screenshot�step�2)��In�the�box�under�“Within-Subject Factor Name,”�enter�the�name� you�wish�to�call�the�repeated�factor��For�this�illustration,�we�will�label�the�repeated�measure� “Rater.”�It�is�necessary�to�define�a�name�for�the�repeated�factor�as�there�is�no�single�vari- able�representing�this�factor�(recall�that�the�columns�in�the�dataset�represent�the�repeated� measures);�in�the�dataset,�there�is�one�variable�for�each�level�of�the�factor�(in�other�words,� one�variable�for�each�different�rater�or�measurement)��Again,�in�our�example,�there�are�four� levels�of�raters�(i�e�,�four�raters)�and�thus�four�variables��Thus,�we�name�the�within-subjects�
517Random- and Mixed-Effects Analysis of Variance Models
factor�“Rater.”�The�“Number of Levels”�indicates�the�number�of�measurements�of� the�repeated�measure��In�this�example,�there�were�four�raters,�and,�thus,�the�“Number of Levels”�of�the�factor�is�4�(e�g�,�4)�
One-factor repeated
measures ANOVA:
Step 2
Clicking on “Add” will move these choices into this
area.
Step 3:�After�we�have�defined�the�“Within-Subject Factor Name”�and�the�“Number of Levels,”�click�on�“Add”�to�move�this�information�into�the�middle�box��In�screenshot� step� 3,� we� see� our� newly� defined� repeated� measures� factor� (i�e�,� Rater)� with� “4”� indi- cating�that�there�are�four�levels:�Rater(4)��Finally,�click�on�“Define”�to�open�the�main� “Repeated Measures”�dialog�box�
One-factor repeated
measures ANOVA:
Step 3
Now the choices are shown in the box.
518 An Introduction to Statistical Concepts
Step 4a:�From�the�“Repeated Measures”�dialog�box�(see�screenshot�step�4a),�we�see�a�head- ing�called�“Within-Subjects Variables”�with�the�newly�defined�factor�rater�in�parenthe- ses��In�this�illustration,�the�values�of�1�through�4�represent�each�one�of�the�four�raters�that�we� just�defined�through�screenshot�step�3��Preceding�each�of�the�levels�of�the�repeated�factor�are� lines�with�question�marks��This�is�the�software’s�way�of�asking�us�to�define�which�variable� from�the�list�on�the�left�represents�the�first�measurement�(or�the�first�rater�in�our�illustration)�
One-factor repeated
measures ANOVA:
Step 4a
Clicking on “Plots” will allow you to generate profile
plots.
Clicking on “Save” will allow you to
save various forms of residuals, among other
variables.
Clicking on “Options” will allow you to obtain a
number of other statistics (e.g., descriptive statistics,
effect size, power, homogeneity tests, and
multiple comparison procedures).
Step 4b:�Move�the�appropriate�variables�from�the�variable�list�on�the�left�into�the�“Within- Subjects Variables”�box�on�the�right��It�is�important�to�make�sure�that�the�first�measure- ment�is�matched�up�with�“1,”�the�second�measurement�is�matched�with�“2,”�and�so�forth�so� that�the�correct�order�of�repeated�measures�is�defined��This�is�especially�critical�when�there�is� some�temporal�order�to�the�repeated�measures�(e�g�,�pre-,�post-,�3�months�after�post-)�
One-factor repeated
measures ANOVA:
Step 4b
519Random- and Mixed-Effects Analysis of Variance Models
Step 5:�From�the�“Univariate”�dialog�box�(see�screenshot�step�4a),�clicking�on�“Options”� will� provide� the� option� to� select� such� information� as� “Descriptive Statistics,” “Estimates of effect size,” “Observed power,”�and�“Homogeneity tests.”�For� the�one-factor�repeated�measures�ANOVA,�the�“Options”�dialog�box�is�the�proper�place�to� obtain�post�hoc�MCPs�including�the�Tukey�LSD,�Bonferroni,�and�Sidak�procedures��Click�on� “Continue”�to�return�to�the�original�dialog�box�
One-factor repeated
measures ANOVA:
Step 5
Select from the list on the left those variables that you wish to display means for and use the arrow to move them to
the “Display Means for” box on the right.
If you wish to conduct a post hoc test to
determine where there are mean differences between the repeated
measures, that selection must be
made from this screen using the “Compare
main effects” option, then selecting one of the three MCPs that
are available from the toggle menu under
“Confidence interval adjustment” (i.e., LSD,
Bonferroni, or Sidak).
Step 6:�From�the�“Univariate”�dialog�box�(see�screenshot�step�4a),�click�on�“Plots”�to� obtain�a�profile�plot�of�means��Click�the�repeated�measure�factor�(e�g�,�“Rater”)�and�move� it�into�the�“Horizontal Axis”�box�by�clicking�the�arrow�button�(see�screenshot�step�6a)�� Then�click�on�“Add”�to�move�the�variable�into�the�“Plots”�box�at�the�bottom�of�the�dialog� box�(see�screenshot�step�6b)��Click�on�“Continue”�to�return�to�the�original�dialog�box�
One-factor repeated
measures ANOVA:
Step 6a
Select the repeated measures factor from the list on the left and
use the arrow to move it to the “Horizontal Axis” box on the right.
520 An Introduction to Statistical Concepts
One-factor repeated
measures ANOVA:
Step 6b
�en click “Add” to move the variable
into the “Plots” box at the bottom.
Step 7:�From�the�“Univariate”�dialog�box�(see�screenshot�step�4a),�click�on�“Save”�to� select�those�elements�that�you�want�to�save�(in�our�case,�we�want�to�save�the�unstandard- ized� residuals� which� will� be� used� later� to� examine� the� extent� to� which� normality� and� independence�are�met)��To�do�this,�place�a�checkmark�next�to�“Unstandardized.”�Click� “Continue”�to�return�to�the�main�“Univariate”�dialog�box�and�then�click�on�“Ok”�to� return�to�generate�the�output�
One-factor repeated
measures ANOVA:
Step 7
Interpreting the output:�Annotated�results�are�presented�in�Table�15�14�
521Random- and Mixed-Effects Analysis of Variance Models
Table 15.14
One-Factor�Repeated�Measures�ANOVA�SPSS�Results�for�the�Writing�Assessment�Example
Descriptive Statistics
Mean Std. Deviation N Rater1_raw 8
The table labeled “Descriptive Statistics” provides basic descriptive
statistics (means, standard deviations, and sample sizes) for
each group of the repeated measure.
Rater2_raw 8 Rater3_raw 8 Rater4_raw
2.7500 3.6250 6.2500 9.1250
1.48805 .91613
1.03510 .83452 8
Multivariate Testsa
Effect Value F Hypothesis
df Error df Sig. Partial
Eta Squared Noncent. Parameter
Observed Powerb
Pillai's trace
3.000 5.000 .967 145.949 1.000
Wilks’ lambda
3.000 5.000 .967 145.949 1.000
Hotelling's trace
3.000 5.000 .967 145.949 1.000
Rater
Roy's largest
root
.967
.033
29.190
29.190
48.650c
48.650c
48.650c
48.650c 3.000 5.000
.000
.000
.000
.000 .967 145.949 1.000
c Exact statistic. b Computed using alpha = .05. a Design: intercept.
Within-subjects design: rater.
The table labeled “Multivariate Tests” provides results for the multivariate test of mean differences between the repeated measures. Multivariate tests are provided when there are three or more levels of the within-subjects factor. These results are generally more conser- vative than the univariate results (in other words, you may be less likely to find statistically significant multivariate results as compared to univariate results). Note that the multivariate tests do not require meeting the assumption of sphericity. Thus if the assumption of sphericity is met, reporting univariate results is recommended. If results for the multivariate tests are reported, of the four test results, Wilks’ lambda is reco- mmended. In this example, all four multivariate criteria produce the same results—specifically that there is a statistically significant multivariate mean difference (as noted by p less than α.)
Mauchly's Test of SphericityaMeasure: MEASURE_1
EpsilonbWithin- Subjects Effect Mauchly's W
Approx. Chi-
Square df Sig. Greenhouse Geisser– Huynh–
Feldt Lower Bound
Rater .155 10.679 5 .062 .476 .564 .333
Tests the null hypothesis that the error covariance matrix of the orthonormalized transformed dependent variables is proportional to an identity matrix.
b May be used to adjust the degrees of freedom for the averaged tests of significance. Corrected tests are displayed in the tests of within-subjects effects table.
a Design: intercept.
Within-subjects design: rater.
“Mauchly’s Test of Sphericity” can be reviewed to determine if the assumption of sphericity is met. If the p value is larger than α (as in this illustration), we have met the assumption of sphericity.
“Epsilon” is a gauge of differences in the variances of the repeated measures and is used to adjust the degrees of freedom when sphericity is violated. �e closer the epsilon value is to 1.0, the more homogenous are the variances. Complete heterogeneity of variances is speci- fied by the “Lower bound” and is computed as 1/(K – 1) where K is the number of within subjects factors. For this example, with four raters, the lower bound is 1/(4 – 1) or .333.
(continued)
522 An Introduction to Statistical Concepts
Table 15.14 (continued)
One-Factor�Repeated�Measures�ANOVA�SPSS�Results�for�the�Writing�Assessment�Example
Tests of within-Subjects EffectsMeasure: MEASURE_1
Source
Type III Sum of Squares df
Mean Square F Sig.
Partial Eta Squared
Noncent. Parameter
Observed Powera
Sphericity assumed
Greenhouse Geisser–
Huynh–Feldt
Rater
Lower-bound
198.125
198.125
198.125 198.125
3
1.428
1.691 1.000
66.042
138.760
117.163 198.125
73.477
73.477
73.477 73.477
.000
.000
.000
.000
.913
.913
.913
.913
220.430
104.912
124.250 73.477
1.000
1.000
1.000 1.000
Sphericity assumed
18.875 21 .899
Greenhouse Geisser– 18.875 9.995 1.888
Huynh–Feldt 18.875 11.837 1.595
Error (rater)
Lower-bound 18.875 7.000 2.696 a Computed using alpha = .05.
Since we met the assumption of
sphericity, we use the results from the
row labeled “sphericity assumed.”
Error sum of squares indicates
how much variability is
unexplained across the conditions of
the repeated measures.
(J – 1) (N – 1) = (4 – 1) (8 – 1) =
21
Rater df is computed as
(J – 1) = 4 – 1 = 3
Had we violated the assumption of sphericity, we would have wanted to
use a different set of results (e.g., Geisser–Greenhouse, Huynh–Feldt,
Lower-bound). Notice that in all four sets of results, the sum of squares is the same value, however the degrees
of freedom differs for each. The F ratio is computed the same for each
(i.e., MSrater/MSerror). Of the three results that can be used when
sphericity is violated, the Lower- bound is the most conservative,
followed by Geisser–Greenhouse (use when epsilon is ≤ .75) and then Huynh–Feldt (use when
.75 < epsilon < 1.0).
Comparing p to α, we find a statistically
significant difference in the mean ratings. This is an omnibus test. We will
look at our MCP to determine which mean
ratings differ.
Partial eta squared is one measure of effect size:
198.125 198.125 + 18.875
η2 =
η2 = = .913
SSbetw SSbetw + SSerror
We can interpret this to say that approximately 91% of the variation in the rating is accounted for by the differences in the raters.
Observed power tells whether our test is powerful enough to
detect mean differences if they really exist.
Power of 1.000 indicates maximum power, the
probability of rejecting the null hypothesis if it
is really false is 1.00.
Error df is computed as
523Random- and Mixed-Effects Analysis of Variance Models
Table 15.14 (continued)
One-Factor�Repeated�Measures�ANOVA�SPSS�Results�for�the�Writing�Assessment�Example
Tests of within-Subjects ContrastsMeasure: MEASURE_1
Source Rater
Type III Sum of Squares df
Mean Square F Sig.
Partial Eta Squared
Noncent. Parameter
Observed Powera
Linear Quadratic
Rater
Cubic
103.685 18.667
2.032
.000
.003
.197
.937
.727
.225
103.685 18.667
2.032
1.000 .957 .235
Linear
Quadratic
Error
(rater)
Cubic
189.225 8.000
.900 12.775
3.000
3.100
1 1 1 7
7
7
189.225 8.000
.900 1.825
.429
.443 a Computed using alpha = .05.
Tests of between-Subjects Effects Measure: MEASURE_1 Transformed Variable: Average
Source
Type III Sum of Squares df Mean Square F Sig.
Partial Eta Squared
Noncent. Parameter
Observed Powera
Intercept 445.235 .000 .985 445.235 1.000 Error
946.125 14.875
1 7
946.125 2.125
a Computed using alpha = .05.
The output from the “Tests of within-Subjects Contrasts” will not be used. Polynomial contrasts do not make sense for the rater factor.
The output from the “Tests of between-Subjects Effects” will not be used as there is no between-subjects factor.
Estimated Marginal Means 1. Grand Mean Measure: MEASURE_1
95% Confidence Interval Mean Std. Error Lower Bound Upper Bound
5.438 .258 4.828 6.047
2. Rater EstimatesMeasure: MEASURE_1
95% Confidence Interval Rater Mean Std. Error Lower Bound Upper Bound
1 2 3 4
2.750 3.625 6.250 9.125
.526
.324
.366
.295
1.506 2.859 5.385 8.427
3.994 4.391 7.115 9.823
The “Grand Mean” (in this case, 5.438) represents the overall mean, regardless of the rater. The 95% CI represents the CI
of the grand mean.
The table labeled “Rater” provides descriptive statistics for each of the four raters. In addition to means, the SE and 95% CI of the means
are reported.
(continued)
524 An Introduction to Statistical Concepts
Table 15.14 (continued)
One-Factor�Repeated�Measures�ANOVA�SPSS�Results�for�the�Writing�Assessment�Example
Pairwise ComparisonsMeasure: MEASURE_1
95% Confidence Interval for Differencea
(I) Rater (J) Rater
Mean Difference
(I–J) Std. Error Sig.a Lower Bound Upper Bound
2 –.875 –1.948 .198
3 –3.500* –4.472 –2.528
1
4 –6.375* –8.940 –3.810
1 .875 –.198 1.948
3 –2.625 *
–3.581 –1.669
2
4 –5.500 *
–7.561 –3.439
1 3.500* 2.528 4.472 2 2.625* 1.669 3.581
3
4 –2.875* –4.871 –.879
1 6.375* 3.810 8.940
2 5.500* 3.439 7.561
4
3 2.875*
.295
.267
.706
.295
.263
.567
.267
.263
.549
.706
.567
.549
.126
.000
.000
.126
.000
.000
.000
.000
.007
.000
.000
.007 .879 4.871
Based on estimated marginal means. a Adjustment for multiple comparisons: Bonferroni. *The mean difference is significant at the .05 level.
“Mean Difference” is simply the difference between the means of the two raters being compared. For example, the mean difference of rater 1 and rater 2 is calculated as 2.750 – 3.625 = –.875.
“Sig.” denotes the observed p value and provides the results of the Bonferroni post hoc procedure. �ere is a statistically significant mean difference between:
1. Rater 1 and rater 3 2. Rater 1 and rater 4 3. Rater 2 and rater 3 4. Rater 2 and rater 4 5. Rater 3 and rater 4
�e only groups for which there is not a statistically significant mean difference is between raters 1 and 2.
Note there are redundant results presented in the table. The comparison of rater 1 and 2 (presented in results for rater 1) is the same as the comparison of rater 2 and 1 (presented in results for rater 2) and so forth.
Friedman Test: Nonparametric One-Factor Repeated Measures ANOVA
Step 1:� The� nonparametric� version� of� the� repeated� measures� ANOVA� is� the� Friedman� test��To�compute�the�Friedman�test,�go�to�“Analyze”�in�the�top�pulldown�menu�and�then� select�“Nonparametric Tests,”�then�“Legacy Dialogs,”�and�then�finally�“K Related Samples.”�Following�the�screenshot�(step�1)�as�follows�produces�the�“Tests for Several Related Samples”�dialog�box�
525Random- and Mixed-Effects Analysis of Variance Models
A
B
C
D
Friedman’s test: Step 1
Step 2:�Recall�that�the�Friedman�test�operates�using�ranked�data,�not�continuous�raw�scores� as�with�the�repeated�measures�ANOVA;�thus,�we�will�work�with�the�ranked�variables�in�our� dataset�for�this�test��From�the�“Tests for Several Related Samples”�dialog�box,�click� the�variables�representing�the�ranked levels�of�the�repeated�factor�into�the�“Test Variables”� box�by�using�the�arrow�key�in�the�middle�of�the�dialog�box��Under�“Test Type”�at�the�bottom� left,�check “Friedman.”�Then�click�on�“Ok”�to�return�to�generate�the�output�
Select the ranked repeated measures from the list on the
left and use the arrow to move
them to the “Test Variables” box on
the right.
Friedman’s test: Step 1
Interpreting the output:�Annotated�results�are�presented�in�Table�15�15�
526 An Introduction to Statistical Concepts
Table 15.15
Friedman’s�Test�SPSS�Results�for�the�Writing�Assessment�Example
Ranks
Mean Rank
Rater1_rank
Rater2_rank
Rater3_rank
Rater4_rank
1.13
1.88
3.00
4.00
Test Statisticsa
N 8 Chi-Square 22.950 df 3 Asymp. Sig. .000
a Friedman test.
The table labeled “Ranks” provides the average rank for each of the
repeated measures levels.
The table labeled “Test Statistics” provides the results for the hypothesis
test of the difference in the mean ranks. Since p is less than α, this tells us there is a statistically significant difference in
the mean ranks of the raters.
Two-Factor Split-Plot ANOVA
To�conduct�the�two-factor�split-plot�ANOVA,�the�dataset�must�include�variables�for�each� level�of�the�repeated�factor�(as�in�the�one-factor�repeated�measures�ANOVA)�and�another� variable�for�the�nonrepeated�factor��Here�our�repeated�measures�or�within-subjects�factor� is�reflected�in�the�raw�scores�of�the�four�raters,�and�the�nonrepeated�or�between-subjects� factor�is�the�instructor�
The repeated measures or within-subjects factor is
labeled “Rater” where there are four different raters, each
reflected in the score they assigned to each of the eight
participants. (We will use the raw scores of the raters for the two-factor
split-plot ANOVA.)
The nonrepeated or between- subjects factor is labeled
“Instructor” where each value represents the instructor to
which the students were randomly assigned. Four students were randomly
assigned to instructor 1 and four were randomly assigned to
instructor 2.
527Random- and Mixed-Effects Analysis of Variance Models
Step 1:�To�conduct�a�two-factor�split-plot�ANOVA,�go�to “Analyze”�in�the�top�pulldown� menu,�then�select�“General Linear Model,”�and�then�select�“Repeated Measures.”� This�will�produce�the�“Repeated Measures”�dialog�box��This�step�has�been�presented� previously�(see�screenshot�step�1�for�the�one-factor�repeated�measures�design)�and�will�not� be�reiterated�here�
Step 2: The�“Repeated Measures Define Factor(s)”�dialog�box�will�appear�(see� screenshot�step�2�for�the�one-factor�repeated�measures�design�presented�previously)��In� the� box� under�“Within-Subjects Factor Name,”� enter� the� name� you� wish� to� call� the�repeated�factor��For�this�example,�we�label�the�repeated�factor “Rater.”�It�is�neces- sary�to�define�a�name�for�the�repeated�factor�as�there�is�no�single�variable�representing� this� factor� (recall� that� the� columns� in� the� dataset� represent� the� repeated� measures);� in� the�dataset,�there�is�one�variable�for�each�level�of�the�factor�(in�other�words,�one�variable� for� each� different� rater� or� measurement)�� Again,� in� our� example,� there� are� four� levels� of�rater�(i�e�,�four�raters)�and�thus�four�variables��Let�us�name�the�within-subjects�factor “Rater.”� The�“Number of Levels”� indicates� the� number� of� measurements� of� the� repeated�factor��Here�there�were�four�raters,�and,�thus,�the�“Number of Levels”�of�the� factor�is�4�
Step 3:�After�defining�the�“Within-Subjects Factor Name”�and�the�“Number of Levels,”�then�click�on�“Add”�to�move�this�information�into�the�middle�box��In�screen- shot�step�3�for�the�one-factor�repeated�measures�design�presented�previously,�we�see�our� newly�defined�repeated�factor�(i�e�,�Rater)�with�“4”�indicating�it�was�measured�by�four� raters:� Rater(4).� Finally,� click� on� “Define”� to� open� the� main� “Repeated� Measures”� dialog�box�
Step 4a:�From�the�“Repeated Measures”�dialog�box�(see�screenshot�steps�4a�and�b�for� the� one-factor� repeated� measures� design� presented� previously),� we� see� a� heading� called� “Within-Subjects Variables”� with� the� newly� defined� factor� rater� in� parentheses�� Here�the�values�of�1�through�4�represent�each�one�of�the�four�raters��Preceding�each�of�the� levels�of�the�repeated�factor�are�lines�with�question�marks��This�is�the�software’s�way�of� asking�us�to�define�which� variable� represents�the�first�measurement�(or�the�first�rater�in� our�illustration)�
Step 4b:� Move� the� appropriate� variables� from� the� variable� list� on� the� left� into� the� “Within-Subjects Variables”�box�on�the�right��It�is�important�to�make�sure�that�the� first�measurement�is�matched�up�with�“1,”�the�second�measurement�is�matched�with�“2,” and�so�forth�so�that�the�correct�order�of�repeated�measures�is�defined�
Step 5:� Once� the� “Within-Subjects Variables”� are� defined,� the� next� step� is� to� define�the�between-subjects�or�nonrepeated�factor,�as�we�see�in�screenshot�step�5�that�fol- lows��Move�the�appropriate�variable�from�the�variable�list�on�the�left�into�the�“Between- Subjects Factors”�box�on�the�right��From�this�point,�the�options�and�selections�work� as�we�have�seen�when�conducting�other�ANOVA�models�
528 An Introduction to Statistical Concepts
Clicking on “Plots” will allow you to generate profile
plots.
Clicking on “Save” will allow you to
save various forms of residuals, among other
variables.
Clicking on “Options” will allow you to
obtain a number of other statistics (e.g., descriptive statistics,
effect size, power, homogeneity tests).
Select the nonrepeated factor from the list on the
left and use the arrow to move it to
the “Between- Subjects
Factors(s)” box on the right.
Two-factor split-plot ANOVA:
Step 5
Step 6:�From�the�“Repeated Measures”�dialog�box,�clicking�on�“Options”�will�pro- vide�the�option�to�select�such�information�as�“Descriptive Statistics,” “Estimates of effect size,” “Observed power,”�and�“Homogeneity tests”�(see�screenshot� step�6)��For�the�two-factor�split-plot�ANOVA,�the�“Options”�dialog�box�is�the�proper�place� to�obtain�post�hoc�MCPs�for�the�repeated measure��Post�hoc�procedures�include�the�Tukey�LSD,� Bonferroni,�and�Sidak�procedures��Click�on�“Continue”�to�return�to�the�original�dialog�box�
If you wish to conduct a post hoc test to
determine where there are mean differences between the repeated
measures, that selection must be
made from this screen using the “Compare
main effects” option, then select one of the three MCPs that are
available from the toggle menu under
“Confidence interval adjustment” (i.e., LSD,
Bonferroni, or Sidak).
Select from the list on the left those variables
that you wish to display means for and use the arrow to move them to the “Display means for” box on the right.
Two-factor split-plot ANOVA: Step 6
529Random- and Mixed-Effects Analysis of Variance Models
Step 7:� Click� on� the� name� of� the� nonrepeated� or� between-subjects� factor� in� the “Factor(s)” list�box�in�the�top�left�and�move�it�to�the�“Post Hoc Tests for”�box�in� the�top�right�by�clicking�on�the�arrow�key��Check�an�appropriate�MCP�for�your�situation�by� placing�a�checkmark�in�the�box�next�to�the�desired�MCP��In�this�example,�we�select�Tukey� (see�screenshot�step�7)��Click�on�“Continue”�to�return�to�the�original�dialog�box�
MCPs for instances when the homogeneity of
variance assumption is met.
MCPs for instances when the homogeneity of variance assumption
is not met.
Select the fixed factor of interest from the list on the left and use the
arrow to move it to the “Post Hoc Tests for” box on the right.
Two-factor split-plot ANOVA: Step 7
Step 8:� From� the�“Repeated Measures”� dialog� box,� click� on “Plots”� to� obtain� a� profile�plot�of�means��Click�one�independent�variable�(e�g�,�“Rater”)�and�move�it�into�the� “Horizontal Axis”�box�by�clicking�the�arrow�button��Then�click�the�other�independent� variable� (e�g�,� instructor)� and� move� it� into� the�“Separate Lines”� box� by� clicking� the� arrow�button��Then�click�on�“Add”�to�move�this�into�the�“Plots”�box�at�the�bottom�of� the�dialog�box�(see�screenshot�steps�8a�and�b)��Click�on�“Continue”�to�return�to�the�original� dialog�box��(Tip: Placing the factor that has the most categories or levels on the horizontal axis of the profile plot will make for easier interpretation of the graph. In this case, there were four raters and two instructors; thus, we placed “rater” on the horizontal axis�)
Select the factor with the most levels from the list on the left and use the arrow to
move to the “Horizontal Axis” box on the right. Repeat these steps to move the other
factor into the box for “Separate Lines.”
Two-factor split-plot ANOVA: Step 8a
530 An Introduction to Statistical Concepts
Then click “Add” to move the variables
into the “Plots” box at the bottom.
Two-factor split-plot ANOVA: Step 8b
Step 9:�From�the�“Repeated Measures” dialog�box,�click�on�“Save”�to�select�those� elements�that�you�want�to�save�(here�we�want�to�save�the�unstandardized�residuals�which� will�be�used�later�to�examine�the�extent�to�which�normality�and�independence�are�met)��To� do�this,�place�a�checkmark�next�to�“Unstandardized.”�Click�“Continue”�to�return�to� the�main�“Repeated Measures”�dialog�box��From�the�“Repeated Measures”�dialog� box,�click�on “Ok”�to�generate�the�output�
Two-factor split-plot ANOVA: Step 9
Interpreting the output:�Annotated�results�are�presented�in�Table�15�16�
531Random- and Mixed-Effects Analysis of Variance Models
Table 15.16
Two-Factor�Split-Plot�ANOVA�SPSS�Results�for�the�Writing�Assessment�Example
Within-Subjects Factors
Measure: MEASURE_1
Rater Dependent
Variable
1 Rater1_raw
2 Rater2_raw
3 Rater3_raw
4 Rater4_raw
Between-Subjects Factors
Value Label N
1.00 Instructor 1 4Instructor
2.00 Instructor 2 4
Descriptive Statistics
Instructor Mean Std. Deviation N
Instructor 1 3.7500 1.50000 4
Instructor 2 1.7500 .50000 4
Rater 1 raw score
Total 2.7500 1.48805 8
Instructor 1 4.2500 .50000 4
Instructor 2 3.0000 .81650 4
Rater 2 raw score
Total 3.6250 .91613 8
Instructor 1 7.0000 .81650 4
Instructor 2 5.5000 .57735 4
Rater 3 raw score
Total 6.2500 1.03510 8
Instructor 1 8.5000 .57735 4
Instructor 2 9.7500 .50000 4
Rater 4 raw score
Total 9.1250 .83452 8
The table labeled “Within-Subjects Factors” lists the variable names for
levels of the repeated factor.
The table labeled “Between- Subjects Factors” lists the names and sample sizes for the levels of the nonrepeated factor.
The table labeled “Descriptive Statistics” lists the means, standard deviations, and sample sizes for each of the between-subjects factors (i.e., instructors) by each of the repeated measures (i.e., raters).
(continued)
532 An Introduction to Statistical Concepts
Table 15.16 (continued)
Two-Factor�Split-Plot�ANOVA�SPSS�Results�for�the�Writing�Assessment�Example
Multivariate Testsa
Effect Value F Hypothesis
df Error
df Sig.
Partial Eta
Squared
Noncent. Parameter
Observed Powerb
Pillai's trace .983 74.892c 3.000 4.000 .001 .983 224.677 1.000
Wilks' lambda .017 74.892c 3.000 4.000 .001 .983 224.677 1.000
Hotelling's trace 56.169 74.892c 3.000 4.000 .001 .983 224.677 1.000
Rater
Roy's largest root 56.169 74.892c 3.000 4.000 .001 .983 224.677 1.000 Pillai's trace .899 11.925c 3.000 4.000 .018 .899 35.774 .860
Wilks' lambda .101 11.925c 3.000 4.000 .018 .899 35.774 .860
Hotelling's trace 8.944 11.925c 3.000 4.000 .018 .899 35.774 .860
Rater* instructor
Roy's largest root 8.944 11.925c 3.000 4.000 .018 .899 35.774 .860
c Exact statistic. b Computed using alpha = .05. a Design: intercept + instructor
Within-subjects design: rater.
The table labeled “Multivariate Tests” provides results for the multivariate test of mean differences for the repeated measures factor (i.e., “Rater”), and for the between-by within-subjects interaction (i.e., “Rater*Instructor”). Multivariate tests are provided when there are three or more levels of the within-subjects factor. These results are generally more conservative than the univariate results (in other words, you may be less likely to find statistically significant multivariate results as compared to univariate results). Note that the multivariate tests do not require meeting the assumption of sphericity. Thus if the assumption of sphericity is met, reporting univariate results is recommended. If results for the multivariate tests are reported, of the four test criteria, Wilks’ lambda is recommended. In this example, all four multivariate criteria produce the same results—specifically that there is a statistically significant multivariate mean difference for the repeated measures factor and a statistically significant between- by within-subjects interaction (as noted by p less than α).
Mauchly's Test of SphericityaMeasure: MEASURE_1
Epsilonb
Within Subject Effects Mauchly's W
Approx. Chi-Square df Sig.
Geisser– Greenhouse
Huynh–Feldt Lower bound
Rater .429 4.001 5 .557 .706 1.000 .333
Tests the null hypothesis that the error covariance matrix of the orthonormalized transformed dependent variables is proportional to an identity matrix.
b May be used to adjust the degrees of freedom for the averaged tests of significance. Corrected tests are displayed in the Tests of within-Subjects Effects table.
a Design: intercept + instructor
“Mauchly’s Test of Sphericity” can be reviewed to determine if the assumption of sphericity is met. If the p value is larger than α (as in this illustration), we have met the assumption of sphericity.
“Epsilon” is a gauge of differences in the variances of the repeated measures. �e closer the epsilon value is to 1.0, the more homogenous are the variances. Complete heterogeneity of variances is specified by the “Lower bound” and is computed as 1/(K–1) where K is the number of within-subjects levels. For this example, with four raters, the lower bound is 1/(4–1) or .333.
Within-subjects design: rater.
533Random- and Mixed-Effects Analysis of Variance Models
Table 15.16 (continued)
Two-Factor�Split-Plot�ANOVA�SPSS�Results�for�the�Writing�Assessment�Example
Measure: MEAS URE_1
Source
Type III Sum of Squares df
Mean Square F Sig.
Partial Eta
Squared Noncent. Parameter
Observed Powera
Sphericity assumed
3 66.042 190.200 .000 .969 570.600 1.000
Geisser– Greenhouse
93.515 190.200 .000 .969 402.966 1.000
Huynh–Feldt 66.042 190.200 .000 .969 570.600 1.000
Rater
Lower bound 198.125 190.200 .000 .969 190.200 1.000
Sphericity assumed
3 4.208 12.120 .000 .669 36.360 .998
Geisser– Greenhouse
5.959 12.120 .001 .669 25.678 .983
Huynh–Feldt 4.208 12.120 .000 .669 36.360 .998
Rater* instructor
Lower bound 12.625 12.120 .013 .669 12.120 .825
Sphericity assumed
18 .347
Geisser– Greenhouse
.492
.347
Error (rater)
Lower bound
198.125
198.125
198.125 198.125
12.625
12.625
12.625 12.625
6.250
6.250
6.250
6.250 1.042
Since we met the assumption of sphericity, we use the results from
the row labeled “sphericity assumed.”
Error sum of squares indicates
how much variability is unexplained
across the conditions of the
repeated measures.
Within*Between interaction df is
computed as (K – 1)( J – 1) = (4 – 1)(2 – 1) = 3
Error df is computed as
(J)(K – 1)(n – 1) = 2(4 – 1)(4 – 1) = 18
Rater df is computed as
(K – 1) = 4 – 1 = 3
Had we violated the assumption of sphericity, we would have wanted
to use a different set of results (e.g., Geisser–Greenhouse, Huynh–
Feldt, Lower bound). Notice that in all four sets of results, the sum
of squares is the same value, however the degrees of freedom differs for each. The F ratio is computed the same for each.
Of the three results that can be used when sphericity is violated,
the Lower bound is the most conservative, followed by
Geisser-Greenhouse and then Huynh-Feldt.
Comparing p to α, we find a statistically
significant difference in the raters and a
statistically significant rater by instructor
interaction. �ese are omnibus tests. We will
look at our MCPs to determine which raters
differ and which differ by instructor.
Partial eta squared is one measure of effect size:
η2 SSbetw
SSbetw + SSerror =
η2 198.125
= .969 198.125 + 6.250
=
We can interpret this to say that approximately 97% of the variation in the ratings is accounted for by the differences in the raters.
Observed power tells whether our test is powerful enough to
detect mean differences if they really exist.
Power of 1.000 indicates maximum power, the
probability of rejecting the null hypothesis if it is really false is 1.00. Power of .998 is only
slightly below maximum power of 1.00; this is
extremely strong power.
The table labeled “Tests of within- Subjects Effects” provides results for the
univariate test of mean differences for the within-subjects factor (i.e., “rater”) and
within-between subjects interaction (i.e., “rater*instructor”).
Tests of within-Subjects Effects
2.119
3.000 1.000
2.1 19
1.000
6.000
3.000
12.712
18.000
a Computed using alpha = .05.
Huynh–Feldt
(continued)
534 An Introduction to Statistical Concepts
Table 15.16 (continued)
Two-Factor�Split-Plot�ANOVA�SPSS�Results�for�the�Writing�Assessment�Example
Tests of within-Subjects ContrastsMeasure: MEASURE_1
Source Rater
Type III Sum of Squares df
Mean Square F Sig.
Partial Eta Squared
Noncent. Parameter
Observed Powera
Linear 189.225 1 189.225 302.760 .000 .981 302.760 1.000
Quadratic 8.000 1 8.000 48.000 .000 .889 48.000 1.000
Rater
Cubic .900 1 .900 3.600 .107 .375 3.600 .359 Linear 9.025 1 9.025 14.440 .009 .706 14.440 .883 Quadratic 2.000 1 2.000 12.000 .013 .667 12.000 .821
Rater* instructor
Cubic 1.600 1 1.600 6.400 .045 .516 6.400 .563 Linear 3.750 6 .625 Quadratic 1.000 6 .167
Error(rater)
Cubic 1.500 6 .250
a Computed using alpha = .05.
Levene's Test of Equality of Error Variancesa
F df 1 df 2 Sig.
Rater 1 raw score 3.600 1 6
Rater 2 raw score .158 1 6
Rater 3 raw score .000 1 6
Rater 4 raw score 1.000 1 6
.107
.705
1.000
.356
Tests the null hypothesis that the error variance of the dependent variable is equal across groups.
The F test (and associated p values) for Levene’s Test for
Equality of Error Variances is reviewed to determine if equal variances can be assumed. In
this case, we meet the assumption (as p is greater
than α).
Note that df 1 is degrees of freedom for the numerator (calculated as J – 1 and df 2
are the degrees of freedom for the denominator
(calculated as N – J ).
The output from the “Tests of within-Subjects
Contrasts” will not be used as polynomial contrasts
do not make sense here.
a Design: intercept + instructor Within-subjects design: rater.
535Random- and Mixed-Effects Analysis of Variance Models
Table 15.16 (continued)
Two-Factor�Split-Plot�ANOVA�SPSS�Results�for�the�Writing�Assessment�Example
Type III Sum of
Squares df Mean
Square F Sig. Partial Eta Squared
Noncent. Parameter
Observed Powera
Intercept 946.125 1 946.125 648.771 .000 .991 648.771 1.000 Instructor 6.125 1 6.125 4.200 .086 .412 4.200 .407
Error 8.750 6 1.458
a Computed using alpha = .05
Estimated Marginal Means 1. Grand Mean Measure: MEASURE_1
95% Confidence Interval Mean Std. Error Lower Bound Upper Bound
5.438 .213 4.915 5.960
2. Rater Estimates
Measure: MEASURE_1
95% Confidence Interval Rater Mean Std. Error Lower Bound Upper Bound
1 2.750 .395 1.783 3.717 2 3.625 .239 3.039 4.211 3 6.250 .250 5.638 6.862 4 9.125 .191 8.658 9.592
The “Grand Mean” (in this case, 5.438) represents the overall mean, regardless of
the rater or instructor. The 95% CI represents the CI of the grand mean.
The table labeled “Rater” provides descriptive statistics for each of the four raters. In addition to means, the SE and 95% CI of the means
are reported.
The table labeled “Tests of between-Subjects
Effects” provides results for the univariate test of mean
differences for the between-subjects factor
(i.e., “instructor”).
Instructor df is computed as
(J – 1)= 2 – 1=1
Comparing p to α, we do not find a statistically significant difference in
the mean ratings by instructor. These are
omnibus tests. We look at MCPs to determine
which mean ratings differ by instructor.
Partial eta squared is one measure of effect size:
We can interpret this to say that approximately 41% of the variation in the ratings is accounted for by the differences in the instructors.
Observed power tells whether our test is powerful enough to detect mean differences if they really exist. Power of .407 indicates low
power; the probability of rejecting the null hypothesis if it is really false is about .41.
η2 SSbetw
SSbetw + SSerror =
η2 6.125
= .412 6.125 + 8.750
=
Tests of between-Subjects Effects Measure: MEASURE_1
Source
Transformed Variable: Average
(continued)
536 An Introduction to Statistical Concepts
Table 15.16 (continued)
Two-Factor�Split-Plot�ANOVA�SPSS�Results�for�the�Writing�Assessment�Example
Pairwise Comparisons Measure: MEASURE_1
95% Confidence Interval for Differencea
(I) Rater (J) Rater Mean Difference
(I – J) Std. Error Sig.a Lower Bound Upper Bound
2 –.875 .280 .122 –1.955 .205
3 –3.500* .270 .000 –4.543 –2.457
1
4 –6.375* .375 .000 –7.824 –4.926
1 .875 .280 .122 –.205 1.955
3 –2.625* .280 .000 –3.705 –1.545
2
4 –5.500* .339 .000 –6.808 –4.192
1 3.500* .270 .000 2.457 4.543
2 2.625* .280 .000 1.545 3.705
3
4 –2.875* .191 .000 –3.613 –2.137
1 6.375* .375 .000 4.926 7.824
2 5.500* .339 .000 4.192 6.808
4
3 2.875* .191 .000 2.137 3.613
Based on estimated marginal means. a Adjustment for multiple comparisons: Bonferroni. *The mean difference is significant at the .05 level.
“Mean Difference” is simply the difference between the means of the two raters being compared. For example, the mean difference of
rater 1 and rater 2 is calculated as 2.750 – 3.625 = –.875.
“Sig.” denotes the observed p value and provides the results of the Bonferroni post hoc procedure. There is a statistically significant mean difference in ratings of writing between: 1. Rater 1 and rater 3 2. Rater 1 and rater 4 3. Rater 2 and rater 3 4. Rater 2 and rater 4 5. Rater 3 and rater 4 The only groups for which there is not a statistically significant mean difference is raters 1 and 2. Note there are redundant results presented in the table. �e comparison of rater 1 and 2 (presented in results for rater 1) is the same as the comparison of rater 2 and 1 (presented in results for rater 2) and so forth.
537Random- and Mixed-Effects Analysis of Variance Models
Table 15.16 (continued)
Two-Factor�Split-Plot�ANOVA�SPSS�Results�for�the�Writing�Assessment�Example
3. Instructor Estimates
Measure: MEASURE_1
95% Confidence Interval Instructor Mean Std. Error Lower Bound Upper Bound
5.875 .302 5.136Instructor 1 Instructor 2 5.000 .302 4.261
6.614 5.739
Pairwise ComparisonsMeasure: MEASURE_1
95% Confidence Interval for Differencea
(I) Instructor (J) Instructor Mean Difference
(I – J) Std. Error Sig.a Lower Bound Upper Bound
Instructor 1 Instructor 2 .427 Instructor 2 Instructor 1
.875 –.875 .427
–.170 –1.920
1.920 .170
Based on estimated marginal means. a Adjustment for multiple comparisons: Bonferroni.
The table for “Instructor” provides descriptive statistics for each of the levels of our
between-subjects factor. In addition to means, the
SE and 95% CI of the means are reported.
“Mean difference” is simply the difference between the means of the two categories of our between-subjects factor. For example, the mean difference of instructor 1 and instructor 2 is calculated
as 5.875 – 5.000 = .875
.086
.086
“Sig.” denotes the observed p value and provides the results of the Bonferroni post hoc procedure. There is not a statistically significant mean
difference in ratings between instructor 1 and 2. Note there are redundant results presented in the table. The comparison
of instructor 1 and 2 (presented in the first row) is the same as the comparison of instructor 2 and 1 (presented in the second row).
Univariate TestsMeasure: MEASURE_1
Sum of Squares df
Mean Square F Sig.
Partial Eta
Squared Noncent. Parameter
Observed Powera
Contrast 1.531 1 1.531 4.200 .086 .412 4.200 .407 Error 2.188 6 .365
The F tests the effect of instructor. This test is based on the linearly independent pairwise comparisons among the estimated marginal means. a Computed using alpha = .05.
The contrast output from the “Univariate Tests” will not be used here.
(continued)
538 An Introduction to Statistical Concepts
Table 15.16 (continued)
Two-Factor�Split-Plot�ANOVA�SPSS�Results�for�the�Writing�Assessment�Example
4. Instructor *Rater Measure: MEASURE_1
95% Confidence Interval
Instructor Rater Mean Std. Error Lower Bound Upper Bound
1
2 3
Instructor 1
4
1
2 3
Instructor 2
4
3.750
4.250 7.000 8.500
1.750
3.000 5.500 9.750
.559
.339
.354
.270
.559
.339
.354
.270
2.382
3.422 6.135 7.839
.382
2.172 4.635 9.089
5.118
5.078 7.865 9.161
3.118
3.828 6.365
10.411
The table for “Instructor*Rater”
provides descriptive statistics for each of the combinations of instructor by rater
(or cell). In addition to means, the SE and
95% CI of the means are reported.
The “Profile Plot” is a graph of the means for each combination of instructor by
rater (or cell). We see the ratings follow a similar
pattern. Three of the four raters provided a lower
mean rating for writing for instructor 2 (as compared to
instructor 1).
10.00
8.00
6.00
4.00
2.00
.00
1 2 3 Rater
4
Es tim
at ed
m ar
gi na
l m ea
ns
Estimated marginal means of MEASURE_1
Instructor Instructor 1 Instructor 2
Examining Data for Assumptions for Two-Factor Split-Plot ANOVA
Normality
We�use�the�residuals�(which�we�requested�and�created�through�the�“Save”�option�when� generating� our� two-factor� split-plot� ANOVA)� to� examine� the� extent� to� which� normality� was�met�
539Random- and Mixed-Effects Analysis of Variance Models
�e residuals are computed by subtracting the cell mean from each observation. For example, the mean rating on writing for students assigned to instructor 1 and rated by rater 1 was 3.75. Person 1 was rated a “3” on writing by rater 1. �us the residual for person 1 is
We see four new variables have been added to the dataset labeled RES_1, RES_2, and so forth. �ese are the residual used to review the normality assumption.
3.00 – 3.75 = –.75.
Generating normality evidence:�As�mentioned�in�previous�chapters,�understand- ing� the� distributional� shape,� specifically� the� extent� to� which� normality� is� a� reasonable� assumption,� is� important�� For� the� two-factor� mixed� design� ANOVA,� the� distributional� shape�for�the�residuals�should�be�a�normal�distribution��Because�we�have�multiple�residu- als�to�reflect�the�multiple�measurements,�we�need�to�examine�normality�for�each�residual�� For�brevity,�we�provide�SPSS�excerpts�only�for�“RES_1,”�which�reflects�the�residual�for�time�1;� however,�we�will�narratively�discuss�all�of�the�residuals�
As�in�previous�chapters,�we�can�again�use�“Explore”�to�examine�the�extent�to�which� the�assumption�of�normality�is�met��The�steps�for�accessing�“Explore”�have�already�been� presented,�and,�thus,�we�only�provide�a�basic�overview�of�the�process��Click�the�residual� and�move�it�into�the�“Dependent List”�box�by�clicking�on�the�arrow�button��The�proce- dures�for�selecting�normality�statistics�are�as�follows:�Click�on�“Plots”�in�the�upper�right� corner��Place�a�checkmark�in�the�boxes�for�“Normality plots with tests”�and�also� for�“Histogram.”�Then�click�“Continue”�to�return�to�the�main�“Explore”�dialog�box�� Finally�click�“Ok”�to�generate�the�output�
Generating normality evidence
Select residuals from the list on the left and
use the arrow to move to the
“Dependent List” box on the right.
�en click on “Plots.”
540 An Introduction to Statistical Concepts
Interpreting normality evidence:� We� have� already� developed� a� good� under- standing�of�how�to�interpret�some�forms�of�evidence�of�normality�including�skewness�and� kurtosis,�histograms,�and�boxplots��Next�we�see�the�output�for�this�evidence�
Mean
5% Trimmed mean Median Variance Std. deviation Minimum Maximum Range Interquartile range Skewness Kurtosis
95% Confidence interval for mean
Lower bound Upper bound
Residual for Rater1_raw
Statistic Std. Error
Descriptives
.0000 –.8654
.8654 –.0833 –.2500 1.071 1.03510 –.75 2.25 3.00 1.00
1.675 3.136 1.481
.752
.36596
The�skewness�statistic�of�the�residuals�for�rater�1�is�1�675�and�kurtosis�is�3�136—skewness� being� within� the� range� of� an� absolute� value� of� 2�0,� suggesting� some� evidence� of� normality�� However,� kurtosis� suggests� some� nonnormality�� For� the� other� three� residuals,� all� skewness� and�kurtosis�statistics�(not�shown�here)�are�within�an�absolute�value�of�2�0,�suggesting�evidence� of�normality��As�suggested�by�the�skewness�statistic,�the�histogram�of�residuals�is�positively� skewed,�and�the�histogram�also�provides�a�visual�display�of�the�leptokurtic�distribution�
4
3
2
Fr eq
ue nc
y
1
0 –1.00 .00 1.00 2.00
Residual for rater1_raw
Histogram
Mean = –5.55E – 17 Std. dev. = 1.035 N = 8
541Random- and Mixed-Effects Analysis of Variance Models
There�are�a�few�other�statistics�that�can�be�used�to�gauge�normality��The�formal�test�of� normality,�the�Shapiro–Wilk�(S–W)�test�(SW)�(Shapiro�&�Wilk,�1965),�provides�evidence� of� the� extent� to� which� the� sample� distribution� is� statistically� different� from� a� normal� distribution��The�output�for�the�S–W�test�is�presented�in�the�following�and�suggests�that� our�sample�distributions�for�three�of�the�four�residuals�(specifically�residuals�for�raters�2,�3,� and� 4)� are� not� statistically� significantly� different� than� what� would� be� expected� from� a� normal�distribution,�as�those�p�values�are�less�than�α��However,�the�distribution�for�the� residual�for�rater�1�is�statistically�significantly�different�than�a�normal�distribution�(SW�=��745,� df�=�8,�p�=��007)�
Statistic df dfSig. Sig.Statistic
Tests of Normality
Kolmogorov–Smirnova
8 8 8 8 8
8 8 8
.018
.200*
.150
.065 .745 .913 .965 .828
.280
.250
.152
.316
.007
.374
.857
.057
Shapiro–Wilk
Residual for Rater1_raw Residual for Rater2_raw Residual for Rater3_raw Residual for Rater4_raw
a Lilliefors significance correction. *This is a lower bound of the true significance.
Quantile–quantile� (Q–Q)� plots� are� also� often� examined� to� determine� evidence� of� nor- mality�� These� graphs� plot� quantiles� of� the� theoretical� normal� distribution� against� quan- tiles� of� the� sample� distribution�� Points� that� fall� on� or� close� to� the� diagonal� line� suggest� evidence�of�normality��The�Q–Q�plot�of�residuals�shown�in�the�following�suggests�some� nonnormality�
3
2
1
Ex pe
ct ed
n or
m al
0
–1
–2 –1 0 1 Observed value
2 3
Normal Q–Q plot of residual for Rater1_raw
This case, which falls far from the
diagonal, suggests some nonnormality.
Examination�of�the�following�boxplot�also�suggests�a�nonnormal�distributional�shape�of� residuals�with�one�outlier�
542 An Introduction to Statistical Concepts
3
2
1
0
–1
Residual for Rater1_raw
2
For�three�of�the�four�residuals�(residuals�for�raters�2,�3,�and�4),�the�forms�of�evidence�we� have� examined—skewness� and� kurtosis� statistics,� the� S–W� test,� the� Q–Q� plot,� and� the� boxplot—all�suggest�normality�is�a�reasonable�assumption��We�can�be�reasonably�assured� we�have�met�the�assumption�of�normality�for�residuals�for�raters�2,�3,�and�4��However,�all� forms�of�evidence�suggest�nonnormality�for�the�residual�for�rater�1�
Independence
The� only� assumption� we� have� not� tested� for� yet� is� independence�� As� we� discussed� in� reference� to� the� one-way� ANOVA,� if� subjects� have� been� randomly� assigned� to� condi- tions�(in�other�words,�the�different�levels�of�the�between-subjects�factor),�the�assumption� of� independence� has� been� met�� In� this� illustration,� students� were� randomly� assigned� to� instructor,� and,� thus,� the� assumption� of� independence� was� met�� However,� we� often� use�between-subjects�factors�that�do�not�allow�random�assignment,�such�as�preexisting� characteristics� (e�g�,� gender� or� education� level)�� We� can� plot� residuals� against� levels� of� our� between-subjects� factor� using� a� scatterplot� to� get� an� idea� of� whether� or� not� there� are� patterns� in� the� data� and� thereby� provide� an� indication� of� whether� we� have� met� this�assumption��In�this�illustration,�we�only�have�one�between-subjects�factor��If�there� were�multiple�between-subjects�factors,�we�would�split�the�scatterplot�by�levels�of�one� between-subjects�factor�and�then�generate�a�bivariate�scatterplot�for�the�other�between- subjects�factor�by�residual�(as�we�did�with�factorial�ANOVA)��Remember�that�the�resid- ual� was� added� to� the� dataset� by� saving� it� when� we� generated� the� two-factor� split-plot� ANOVA�model�
Please� note� that� some� researchers� do� not� believe� that� the� assumption� of� indepen- dence�can�be�tested��If�there�is�not�random�assignment�to�groups,�then�these�researchers�
543Random- and Mixed-Effects Analysis of Variance Models
believe�this�assumption�has�been�violated—period��The�plot�that�we�generate�will�give� us�a�general�idea�of�patterns,�however,�in�situations�where�random�assignment�was�not� performed�
Generating the scatterplot:�The�general�steps�for�generating�a�simple�scatterplot� through�“Scatter/dot”�have�been�presented�in�a�previous�chapter�(e�g�,�Chapter�10),� and�will�not�be�reiterated�here��From�the�“Simple Scatterplot”�dialog�screen,�click� the� residual� variable� and� move� it� into� the� “Y Axis”� box� by� clicking� on� the� arrow�� Click� the� between-subjects� factor� (e�g�,� “Instructor”)� and� move� it� into� the�“X Axis”� box�by�clicking�on�the�arrow��Then�click�“Ok.”�Repeat�these�steps�for�each�of�the�four� residuals�
Simple Scatterplot
Rater 1 raw score [...
Rater 2 raw score [...
Rater 3 raw score [...
Rater 4 raw score [...
Residual for Rater2... Residual for Rater3... Residual for Rater4...
Rater 2 ranked scor...
Rater 2 ranked scor...
Rater 3 ranked scor...
Rater 4 ranked scor...
544 An Introduction to Statistical Concepts
Interpreting independence evidence:�In�examining�the�scatterplots�for�evidence� of�independence,�the�points�should�fall�relatively�randomly�above�and�below�a�horizontal� line�at�0��(You�may�recall�in�Chapter�11�that�we�added�a�reference�line�to�the�graph�using� Chart�Editor��To�add�a�reference�line,�double�click�on�the�graph�in�the�output�to�activate�the� chart�editor��Select “Options”�in�the�top�pulldown�menu,�then�“Y axis reference line.”�This�will�bring�up�the�“Properties”�dialog�box��Change�the�value�of�the�position� to�be�“0�”�Then�click�on “Apply”�and�“Close”�to�generate�the�graph�with�a�horizontal� line�at�0�)
Here� our� scatterplot� for� each� residual� generally� suggests� evidence� of� independence� with�a�relatively�random�display�of�residuals�above�and�below�the�horizontal�line�at�0�for� each�category�of�time�(note�that�only�the�scatterplot�of�the�residual�for�rater�3�by�instruc- tor� is� presented)�� If� we� had� not� met� the� assumption� of� independence� through� random� assignment�of�cases�to�groups,�this�provides�evidence�that�independence�was�a�reason- able�assumption�
1.00
.50
.00
Re si
du al
fo r R
at er
3_ ra
w
–.50
–1.00
1.00 1.20 1.40 1.60 Instructor
1.80 2.00
Post Hoc Power for Two-Factor Split-Plot ANOVA Using G*Power
Generating�power�analyses�for�two-factor�split-plot�ANOVA�models�follows�similarly�to� that� for� ANOVA,� factorial� ANOVA,� and� ANCOVA�� In� particular,� if� there� is� more� than� one�independent�variable,�we�must�test�for�main�effects�and�interactions�separately��The� first�thing�that�must�be�done�when�using�G*Power�for�computing�post�hoc�power�is�to� select�the�correct�test�family��In�our�case,�we�conducted�a�two-factor�split-plot�ANOVA�� Because�we�have�both�between,�within,�and�interaction�terms,�the�type�of�statistical�test� selected�depends�on�which�part�of�the�model�power�is�to�be�estimated��In�this�illustra- tion,� let� us� first� determine� power� for� the� within-between� subjects� interaction�� To� find�
545Random- and Mixed-Effects Analysis of Variance Models
this� design,� we� select� “Tests”� in� the� top� pulldown� menu,� then� “Means,”� and� then� “ANOVA: Repeated measures, within-between interactions.”� Once� that� selection� is� made,� the� “Test family”� automatically� changes� to� “F Tests.”� (Note� that�had�we�wanted�to�determine�power�for�the�between-subjects�main�effect,�we�would� have�selected�“ANOVA: Repeated measures, between factors.”�For�the�within- subjects�main�effect,�we�would�have�selected�“ANOVA: Repeated measures, within factors.”)
A
B
C
Step 1
The�“Type of Power Analysis”�desired�needs�to�be�selected��To�compute�post�hoc� power,�select�“Post hoc: Compute achieved power—given α, sample size, and effect size.”
546 An Introduction to Statistical Concepts
The default selection for “Test
Family” is “t tests.” Following the
procedures presented in Step 1 will automatically
change the test family to “F tests.”
The default selection for “Statistical Test” is
“Correlation: Point biserial model.” Following the procedures presented in Step 1 will automatically change the statistical test
to “ANOVA: Repeated measures, within- between interaction.”
Click on “Determine” to pop out the effect size calculator box
(shown below). This will allow you to
compute f given partial eta squared.
Once the parameters are
specified, click on “Calculate.”
The “Input Parameters” for computing post hoc power must be specified (the default values are shown here) including:
Step 2
1. Effect size f 2. Alpha level 3. Total sample size 4. Number of groups 5. Number of measurements 6. Correlation among repeated measures 7. Nonsphericity correction
The� “Input Parameters”� must� then� be� specified�� We� will� compute� the� effect� size� f� last,� so� we� skip� that� for� the� moment�� In� our� example,� the� alpha� level� we� used� was� �05,� and�the�total�sample�size�was�8��The�number of groups,�in�the�case�of�a�two-factor�split-plot� ANOVA�with�one�nonrepeated�factor�having�two�categories,�equals�2��The�next�parameter� is�the�number�of�measurements��This�refers�to�the�number�of�levels�of�the�repeated�factor,� which�in�this�illustration�is�4��Next,�we�have�to�input�the�correlation�among�repeated�mea- sures��We�will�estimate�this�parameter�as�the�average�correlation�among�all�bivariate�cor- relations�of�the�repeated�measures��For�our�raters,�the�Pearson�correlation�coefficients�were� as�follows:�r12�=��865,�r13�=��881,�r14�=�−�431,�r23�=��716,�r24�=�−�677,�and�r34�=�−�372,�and,�thus,� the�average�correlation�was��657�(in�absolute�value�terms)��The�last�parameter�to�define�is� the�nonsphericity�correction�epsilon,�ε��Epsilon�ranges�from�0�to�1,�with�0�indicating�the� assumption�is�violated�completely�and�1�being�perfect�sphericity��Acceptable�sphericity�is� approximately��75�or�higher��One�option�is�to�input�an�acceptable�level�of�sphericity;�thus,� we�input��75�here��Alternatively,�we�could�input�the�epsilon�values�obtained�for�the�usual,� Geisser–Greenhouse,�and�Huynh–Feldt�F�tests�
We�skipped�filling�in�the�first�parameter,�the�effect�size�f,�until�all�of�the�previous�values� were�input��This�is�because�SPSS�only�provides�a�partial�eta�squared�effect�size��We�use�the� pop-out�effect�size�calculator�in�G*Power�to�compute�the�effect�size�f��To�pop�out�the�effect� size�calculator,�click�on�“Determine,”�which�is�displayed�under�“Input Parameters.”� In� the� pop-out� effect� size� calculator,� click� on� the� radio� button� for� “Direct”� and� then� enter� the� partial� eta� squared� value� that� was� calculated� in� SPSS� (i�e�,� �899)�� Clicking� on�
547Random- and Mixed-Effects Analysis of Variance Models
“Calculate”�in�the�pop-out�effect�size�calculator�will�calculate�the�effect�size�f��Then�click� on�“Calculate and Transfer to Main Window”�to�transfer�the�calculated�effect�size� (i�e�,�2�9834527)�to�the�“Input Parameters.”�Once�the�parameters�are�specified,�click�on� “Calculate”�to�find�the�power�statistics�
Here are the post hoc power results.
Step 3
The�“Output Parameters”�provide�the�relevant�statistics�given�the�input�just�specified�� In�this�example,�we�were�interested�in�determining�post�hoc�power�for�the�within-between� interaction� in� a� two-factor� split-plot� ANOVA� with� a� computed� effect� size� f� of� 2�9834527,� an� alpha�level�of��05,�total�sample�size�of�8,�two�groups,�four�measurements,�an�average�correla- tion�among�repeated�measures�of��657,�and�epsilon�sphericity�correction�of��75��Based�on�those� criteria,�the�post�hoc�power�of�our�within-between�interaction�effect�for�this�test�was�1�000— the�probability�of�rejecting�the�null�hypothesis�when�it�is�really�false�(in�this�case,�the�probabil- ity�that�the�means�of�the�dependent�variable�would�be�equal�for�each�level�of�the�independent� variable)�was�at�the�maximum�(i�e�,�100%)�(sufficient�power�is�often��80�or�above)��Note�that�this� is�the�same�value�as�that�reported�in�SPSS��Keep�in�mind�that�conducting�power�analysis�a� priori�is�recommended�so�that�you�avoid�a�situation�where,�post�hoc,�you�find�that�the�sample� size�was�not�sufficient�to�reach�the�desired�level�of�power�(given�the�observed�parameters)�
A Priori Power for Two-Factor Split-Plot ANOVA Using G*Power
For� a� priori� power,� we� can� determine� the� total� sample� size� needed� for� the� main� effects� and/or interactions� given� an� estimated� effect� size� f,� alpha� level,� desired� power,� number� of�
548 An Introduction to Statistical Concepts
groups�(i�e�,�the�number�of�categories�of�the�independent�variable in the case of only one inde- pendent variable�OR�the�product�of�the�number�of�levels�of�the�independent�variables�in the case of multiple independent variables),�number�of�measurements,�correlation�among�repeated�mea- sures,�and�nonsphericity�correction�epsilon��We�follow�Cohen’s�(1988)�convention�for�effect�size� (i�e�,�small�f�=��10;�moderate�f�=��25;�large�f�=��40)��In�this�example,�had�we�wanted�to�determine� a�priori�power�for�a�within-between�interaction�and�had�estimated�a�moderate�effect�f�of��25,� alpha�of��05,�desired�power�of��80,�number�of�groups�was�2�(i�e�,�we�have�only�one�independent� variable,�and�there�were�two�categories),�four�measurements,�a�moderate�correlation�among� repeated�measures�of��50,�and�a�nonsphericity�correction�epsilon�of��75,�we�would�need�a�total� sample�size�of�30�(i�e�,�15�cases�per�group�given�two�levels�to�our�independent�variable)��Here� are�the�post�hoc�power�results�for�the�attractiveness�by�time�of�day�interaction�
Post hoc power
15.7 Template and APA-Style Write-Up
Finally,�here�is�an�example�paragraph�just�for�the�results�of�the�two-factor�split-plot�design� (feel�free�to�write�similar�paragraphs�for�the�other�models�in�this�chapter)��Recall�that�our� graduate�research�assistant,�Marie,�was�assisting�the�coordinator�of�the�English�program,� Mark��Mark�wanted�to�know�the�following:�if�there�is�a�mean�difference�in�writing�based� on�instructor,�if�there�is�a�mean�difference�in�writing�based�on�rater,�and�if�there�is�a�mean�
549Random- and Mixed-Effects Analysis of Variance Models
difference� in� writing� based� on� rater� by� instructor�� The� research� questions� presented� to� Mark�from�Marie’s�work�include�the�following:
•� Is there a mean difference in writing based on instructor? •� Is there a mean difference in writing based on rater? •� Is there a mean difference in writing based on rater by instructor?
Marie�then�assisted�Mark�in�generating�a�two-factor�split-plot�ANOVA�as�the�test�of�infer- ence,�and�a�template�for�writing�the�research�questions�for�this�design�is�presented�as�fol- lows��As�we�noted�in�previous�chapters,�it�is�important�to�ensure�the�reader�understands� the�levels�or�groups�of�the�factor(s)��This�may�be�done�parenthetically�in�the�actual�research� question,�as�an�operational�definition,�or�specified�within�the�methods�section:
•� Is there a mean difference in [dependent variable] based on [between-subjects factor]?
•� Is there a mean difference in [dependent variable] based on [within-subjects factor]?
•� Is there a mean difference in [dependent variable] based on [between-subjects factor] by [within-subjects factor]?
It�may�be�helpful�to�preface�the�results�of�the�two-factor�split-plot�ANOVA�with�informa- tion�on�an�examination�of�the�extent�to�which�the�assumptions�were�met�(recall�there�are� several�assumptions�that�we�tested)��For�the�between-subjects�factor�(i�e�,�the�nonrepeated� factor),� assumptions� include� (a)� independence� of� observations,� (b)� homogeneity� of� vari- ance,�and�(c)�normality��For�the�within-subjects�factor�(i�e�,�the�repeated�factor),�we�examine� the�assumption�of�sphericity�
A two-factor split-plot (one within-subjects factor and one between- subjects factor) ANOVA was conducted. The within-subjects factor was rater on a writing assessment task (four independent raters), and the between-subjects factor was instructor (two instructors). The null hypotheses tested include the following: (1) the mean writing scores were equal for each of the four different raters, (2) the mean writ- ing scores for each instructor were equal, and (3) the mean writing scores by rater given instructor were equal.
There were no missing data and no univariate outliers. The assump- tion of sphericity was met (χ2 = 4.001, Mauchly’s W = .429, df = 5, p = .557); therefore, the results reported reflect univariate results. The sphericity assumption was further upheld in that the same results were obtained for the usual, Geisser–Greenhouse, and Huynh–Feldt F tests. The assumption of homogeneity of variance was met for the writing scores of all raters [rater 1, F(1, 6) = 3.600, p = .107; rater 2, F(1, 6) = .158, p = .705; rater 3, F(1, 6) = .000, p = 1.000; and rater 4, F(1, 6) = 1.000, p = .356].
The assumption of normality was tested via examination of the residu- als. Review of the S–W test for normality (SWrater1 = .745, df = 8, p = .007; SWrater2 = .913, df = 8, p = .374; SWrater3 = .965, df = 8, p = .857; SWrater4 = .828, df = 8, p = .057), and skewness (rater 1 = 1.675; rater
550 An Introduction to Statistical Concepts
2 = .290; rater 3 = .000; rater 4 = −.571) and kurtosis (rater 1 = 3.136; rater 2 = .272; rater 3 = −.700; rater 4 = −1.729) statistics suggest that normality was a reasonable assumption for raters 2, 3, and 4, but nonnormality was suggested for rater 1. The boxplot suggested a rela- tively normal distributional shape (with no outliers) of the residuals for raters 2 through 4. The boxplot of the residuals for rater 1 sug- gested nonnormality with one outlier. The Q–Q plots suggested normal- ity was reasonable for the residuals of raters 2, 3, and 4, but suggested nonnormality for rater 1. Thus, while there was nonnormality suggested by the residuals for rater 1, the two-factor split-plot ANOVA is robust to violations of normality with equal sample sizes of groups as is evident in this design.
Random assignment of individuals to instructors helped ensure that the assumption of independence was met. Additionally, a scatterplot of residuals against the levels of the between-subjects factors was reviewed. A relatively random display of points around 0 provided further evidence that the assumption of independence was met.
Here�is�an�APA-style�example�paragraph�of�results�for�the�two-factor�split-plot�ANOVA� (remember� that� this� will� be� prefaced� by� the� previous� paragraph� reporting� the� extent� to� which�the�assumptions�of�the�test�were�met)�
From Table 15.16, the results for the univariate ANOVA indicate the following:
1. A statistically significant within-subjects main effect for rater (Frater = 190.200, df = 3,18, p = .001) (rater 1, M = 2.750, SE = .395; rater 2, M = 3.625, SE = .239; rater 3, M = 6.250, SE = .250; rater 4, M = 9.125, SE = .191)
2. A statistically significant within-between subjects interac- tion effect between rater and instructor (Frater × instructor = 12.120, df = 3,18, p = .001) (for brevity, we have not included the means and standard errors here; however, you may want to include those in the narrative or in tabular form)
3. A nonstatistically significant between-subjects main effect for instructor (Finstructor = 4.200, df = 1,6, p = .086) (instructor 1, M = 5.875, SE = .302; instructor 2, M = 5.000, SE = .302)
Effect sizes were rather large for the significant effects (partial η2rater = .969, power = 1.000; partial η2rater × instructor = .669, power = .998) with more than sufficient observed power, but less so for the non- significant effect (partial η2instructor = .412, power = .407) which had less than desired power.
The statistically significant main effect for the within-subjects factor suggests that there are mean differences in writing scores by rater. The raters were quite inconsistent in that Bonferroni MCPs revealed statistically significant differences among all pairs of raters except for rater 1 versus rater 2. The nonstatistically
551Random- and Mixed-Effects Analysis of Variance Models
significant main effect for the between-subjects factor suggests that there are not differences, on average, in writing scores per instructor. In examining CIs of the interaction for the between- within factor (i.e., instructor by rater), nonoverlapping CIs sug- gest statistically significant differences. We see that the patterns evident for the within-subjects factors echo here as well. For both instructor 1 and instructor 2, there are statistically significant differences among all pairs of raters except for rater 1 versus rater 2. From the profile plot in Figure 15.2, we see that while rater 4 found the students of instructor 2 to have better essays, the other raters liked the essays written by the students of instructor 1. It is suggested that a more detailed plan for evaluating essays, including rater training, be implemented in the future.
15.8 Summary
In� this� chapter,� methods� involving� the� comparison� of� means� for� random-� and� mixed- effects�models�were�considered��Five�different�models�were�examined;�these�included�the� one-factor� random-effects� model,� the� two-factor� random-� and� mixed-effects� models,� the one-factor� repeated� measures� model,� and� the� two-factor� split-plot� or� mixed� design�� Included�for�each�design�were�the�usual�topics�of�model�characteristics,�the�linear�model,� assumptions�of�the�model�and�the�effects�of�their�violation,�the�ANOVA�summary�table� and�expected�mean�squares,�and�MCPs��Also�included�for�particular�designs�was�a�discus- sion�of�the�compound�symmetry�assumption�and�alternative�ANOVA�procedures�
At�this�point,�you�should�have�met�the�following�objectives:�(a)�be�able�to�understand� the�characteristics�and�concepts�underlying�random-�and�mixed-effects�ANOVA�models,� (b)�be�able�to�determine�and�interpret�the�results�of�random-�and�mixed-effects�ANOVA� models,� and� (c)� be� able� to� understand� and� evaluate� the� assumptions� of� random-� and� mixed-effects� ANOVA� models�� In� Chapter� 16,� we� continue� our� extended� tour� of� the� ANOVA�by�looking�at�hierarchical�designs�that�involve�one�factor�nested�within�another� factor� (i�e�,� nested� or� hierarchical� designs),� and� randomized� block� designs,� which� we� have�very�briefly�introduced�in�this�chapter�
Problems
Conceptual problems
15.1� When�an�ANOVA�design�includes�a�random�factor�that�is�crossed�with�a�fixed�factor,� the�design�illustrates�which�type�of�model?
� a�� Fixed � b�� Mixed � c�� Random � d�� Crossed
552 An Introduction to Statistical Concepts
15.2� The�denominator�of�the�F�ratio�used�to�test�the�interaction�in�a�two-factor�ANOVA�is� MSwith�in�which�one�of�the�following?
� a�� Fixed-effects�model � b�� Random-effects�model � c�� Mixed-effects�model � d�� All�of�the�above 15.3� A� course� consists� of� five� units,� the� order� of� presentation� of� which� is� varied� (coun-
terbalanced)�� A� researcher� used� a� 5� � 2� ANOVA� design� with� order� (five� different� randomly� selected� orders)� and� gender� serving� as� factors�� Which� ANOVA� model� is� illustrated�by�this�design?
� a�� Fixed-effects�model � b�� Random-effects�model � c�� Mixed-effects�model � d�� Nested�model 15.4� A�researcher�conducts�a�study�where�children�are�measured�on�frequency�of�sharing�
at�three�different�times�over�the�course�of�the�academic�year��Which�ANOVA�model� is�most�appropriate�for�analysis�of�these�data?
� a�� One-factor�random-effects�model � b�� Two-factor�random-effects�model � c�� Two-factor�mixed-effects�model � d�� One-factor�repeated�measures�design � e�� Two-factor�split-plot�design 15.5� A�health-care�researcher�wants�to�make�generalizations�about�the�number�of�patients�
served�by�after�hour�clinics�in�her�region��She�randomly�samples�clinics�and�collects� data�on�the�number�of�patients�served��Which�ANOVA�model�is�most�appropriate�for� analysis�of�these�data?
� a�� One-factor�random-effects�model � b�� Two-factor�random-effects�model � c�� Two-factor�mixed-effects�model � d�� One-factor�repeated�measures�design � e�� Two-factor�split-plot�design 15.6� �A� preschool� teacher� randomly� assigns� children� to� classrooms—some� with� win-
dows�and�some�without�windows��She�wants�to�know�if�there�is�a�mean�difference� in� receptive� vocabulary� based� on� type� of� classroom� (with� and� without� windows)� and�whether�this�varies�by�classroom�teacher��Which�ANOVA�model�is�most�appro- priate�for�analysis�of�these�data?
� a�� One-factor�random-effects�model � b�� Two-factor�random-effects�model � c�� Two-factor�mixed-effects�model � d�� One-factor�repeated�measures�design � e�� Two-factor�split-plot�design
553Random- and Mixed-Effects Analysis of Variance Models
15.7� �If�a�given�set�of�data�were�analyzed�with�both�a�one-factor�fixed-effects�model�and� a�one-factor�random-effects�model,�the�F�ratio�for�the�random-effects�model�will�be� greater�than�the�F�ratio�for�the�fixed-effects�model��True�or�false?
15.8� �A�repeated�measures�design�is�necessarily�an�example�of�the�random-effects�model�� True�or�false?
15.9� �Suppose� researchers� A� and� B� perform� a� two-factor� ANOVA� on� the� same� data,� but�that�A�assumes�a�fixed-effects�model�and�B�assumes�a�random-effects�model�� I assert�that�if�A�finds�the�interaction�significant�at�the��05�level,�B�will�also�find�the� interaction�significant�at�the��05�level��Am�I�correct?
15.10� �I�assert�that�MSwith�should�always�be�used�as�the�denominator�for�all�F�ratios�in�any� two-factor�ANOVA��Am�I�correct?
15.11� �I�assert�that�in�a�one-factor�repeated�measures�ANOVA�and�a�two-factor�split-plot� ANOVA,�the�SStotal�will�be�exactly�the�same�when�using�the�same�data��Am�I�correct?
15.12� �Football� players� are� each� exposed� to� all� three� different� counterbalanced� coaching� strategies,�one�per�month��This�is�an�example�of�which�type�of�model?
� a�� One-factor�fixed-effects�ANOVA�model � b�� One-factor�repeated-measures�ANOVA�model � c�� One-factor�random-effects�ANOVA�model � d�� One-factor�fixed-effects�ANCOVA�model 15.13� A�two-factor�split-plot�design�involves�which�of�the�following? � a�� Two�repeated�factors � b�� Two�nonrepeated�factors � c�� One�repeated�factor�and�one�nonrepeated�factor � d�� Farmers�splitting�up�their�land�into�plots 15.14� �The�interaction�between�factors�L�and�M�can�be�assessed�only�if�which�one�of�the�
following�occurs? � a�� Both�factors�are�crossed� � b�� Both�factors�are�random� � c�� Both�factors�are�fixed� � d�� Factor�L�is�a�repeated�factor� 15.15� A�student�factor�is�almost�always�random��True�or�false? 15.16� �In� a� two-factor� split-plot� design,� there� are� two� interaction� terms�� Hypotheses� can�
actually�be�tested�for�how�many�of�those�interactions? � a�� 0 � b�� 1 � c�� 2 � d�� Cannot�be�determined 15.17� �In�a�one-factor�repeated�measures�ANOVA�design,�the�F�test�is�quite�robust�to�viola-
tion�of�the�sphericity�assumption,�and,�thus,�we�never�need�to�worry�about�it��True� or�false?
554 An Introduction to Statistical Concepts
Computational problems
15.1� Complete�the�following�ANOVA�summary�table�for�a�two-factor�model,�where�there� are�three�levels�of�factor�A�(fixed�method�effect)�and�two�levels�of�factor�B�(random� teacher�effect)��Each�cell�of�the�design�includes�four�students�(α�=��01)�
Source SS df MS F Critical Value Decision
A 3�64 — — — — — B �57 — — — — — AB 2�07 — — — — — Within — — — Total 8�18 —
15.2� A�researcher�tested�whether�aerobics�increased�the�fitness�level�of�eight�undergradu- ate� students� participating� over� a� 4-month� period�� Students� were� measured� at� the� end� of� each� month� using� a� 10-point� fitness� measure� (10� being� most� fit)�� The� data� are�shown�here��Conduct�an�ANOVA�to�determine�the�effectiveness�of�the�program,� using�α�=��05��Use�the�Bonferroni�method�to�detect�exactly�where�the�differences�are� among�the�time�points�(if�they�are�different)�
Subject Time 1 Time 2 Time 3 Time 4
1 3 4 6 9 2 4 7 5 10 3 5 7 7 8 4 1 3 5 7 5 3 4 7 9 6 2 5 6 7 7 1 4 6 9 8 2 4 5 6
15.3� Using�the�same�data�as�in�Computational�Problem�2,�conduct�a�two-factor�split-plot� ANOVA,�where�the�first�four�subjects�participate�in�a�step�aerobics�program�and�the� last�four�subjects�participate�in�a�spinning�program�(α�=�05)�
15.4� To�examine�changes�in�teaching�self-efficacy,�10�teachers�were�measured�on�their� self-efficacy� toward� teaching� at� the� beginning� of� their� teaching� career� and� at� the� end�of�their�1st�and�3rd�years�of�teaching��The�teaching�self-efficacy�scale�ranged� from�0�to�100�with�higher�scores�reflecting�greater�teaching�self-efficacy��The�data� are� shown� here�� Conduct� a� one-factor� repeated� measures� ANOVA� to� determine� mean�differences�across�time,�using�α�=��05��Use�the�Bonferroni�method�to�detect�if� and/or�where�the�differences�are�among�the�time�points�
555Random- and Mixed-Effects Analysis of Variance Models
Subject Beginning Year 1 End Year 1 End Year 3
1 35 50 45 2 50 75 82 3 42 51 56 4 70 72 71 5 65 50 81 6 92 42 69 7 80 82 88 8 78 76 79 9 85 60 83
10 64 71 89
15.5� Using�the�same�data�as�in�Computational�Problem�4,�conduct�a�two-factor�split-plot� ANOVA,�where�the�first�five�subjects�participate�in�a�mentoring�program�and�the�last� five�subjects�do�not�participate�in�a�mentoring�program�(α�=�05)�
15.6� As�a�statistical�consultant,�a�researcher�comes�to�you�with�the�following�partial�SPSS� output� (sphericity� assumed)�� In� a� two-factor� split-plot� ANOVA� design,� rater� is� the� repeated�(or�within-subjects)�factor,�gender�of�the�rater�is�the�nonrepeated�(or�between- subjects)�factor,�and�the�dependent�variable�is�history�exam�scores��(a)�Are�the�effects� significant�(which�you�must�determine,�as�significance�is�missing,�using�α�=��05)?�(b)� What�are�the�implications�of�these�results�in�terms�of�rating�the�history�exam?
Tests�of�Within-Subjects�Effects
Source Type III SS df MS F
Rater 298�38 3 99�46 30�47 Rater*gender 184�38 3 61�46 18�83 Error�(rater) 58�75 18 3�26
Tests�of�Between-Subjects�Effects
Source Type III SS df MS F
Gender 153�13 1 153�13 20�76 Error 44�25 6 7�38
Interpretive problems
15.1� In� Chapter� 13,� you� built� on� the� interpretive� problem� from� Chapter� 11� utilizing� the� survey� 1� dataset� from� the� website�� SPSS� was� used� to� conduct� a� two-factor� fixed- effects�ANOVA,�including�effect�size,�where�political�view�was�factor�A�(as�in�Chapter� 11,�J�=�5),�gender�is�factor�B�(a�new�factor,�K�=�2),�and�the�dependent�variable�was�the�
556 An Introduction to Statistical Concepts
same�one�you�used�previously�in�Chapter�11��Now,�in�addition�to�the�two-factor�fixed- effects� ANOVA,� conduct� a� random-effects� and� mixed-effects� designs�� Determine� whether�the�nature�of�the�factors�makes�any�difference�in�the�results�
15.2� In�Chapter�13,�you�built�on�the�interpretive�problem�from�Chapter�11�utilizing�the� survey� 1� dataset� from� the� website�� SPSS� was� used� to� conduct� a� two-factor� fixed- effects�ANOVA,�including�effect�size,�where�hair�color�was�factor�A�(i�e�,�one�inde- pendent�variable)�(J�=�5),�gender�was�factor�B�(a�new�factor,�K�=�2),�and�the�dependent� variable�was�a�variable�of�interest�to�you�(the�following�variables�look�interesting:� books,� TV,� exercise,� drinks,� GPA,� GRE-Q,� CDs,� hair� appointment)�� Now,� in� addi- tion�to�the�two-factor�fixed-effects�ANOVA,�conduct�a�random-effects�and�mixed- effects�designs��Determine�whether�the�nature�of�the�factors�makes�any�difference� in�the�results�
557
16 Hierarchical and Randomized Block Analysis of Variance Models
Chapter Outline
16�1� Two-Factor�Hierarchical�Model 16�1�1� Characteristics�of�the�Model 16�1�2� Layout�of�Data 16�1�3� ANOVA�Model 16�1�4� ANOVA�Summary�Table�and�Expected�Mean�Squares 16�1�5� Multiple�Comparison�Procedures 16�1�6� Example
16�2� Two-Factor�Randomized�Block�Design�for�n�=�1 16�2�1� Characteristics�of�the�Model 16�2�2� Layout�of�Data 16�2�3� ANOVA�Model 16�2�4� Assumptions�and�Violation�of�Assumptions 16�2�5� ANOVA�Summary�Table�and�Expected�Mean�Squares 16�2�6� Multiple�Comparison�Procedures 16�2�7� Methods�of�Block�Formation 16�2�8� Example
16�3� Two-Factor�Randomized�Block�Design�for�n�>�1 16�4� Friedman�Test 16�5� Comparison�of�Various�ANOVA�Models 16�6� SPSS 16�7� Template�and�APA-Style�Write-Up
Key Concepts
� 1�� Crossed�designs�and�nested�designs � 2�� Confounding � 3�� Randomized�block�designs � 4�� Methods�of�blocking
558 An Introduction to Statistical Concepts
In�the�last�several�chapters,�our�discussion�has�dealt�with�different�analysis�of�variance� (ANOVA)�models��In�this�chapter,�we�complete�our�discussion�of�ANOVA�by�consider- ing� models� in� which� there� are� multiple� factors,� but� where� at� least� one� of� the� factors� is� either� a� hierarchical� (or� nested)� factor� or� a� blocking� factor�� As� we� define� these� models,� we�shall�see�that�this�results�in�a�hierarchical�(or�nested)�design�and�a�blocking�design,� respectively�� In� this� chapter,� we� are� mostly� concerned� with� the� two-factor� hierarchical� (or� nested)� model� and� the� two-factor� randomized� block� model,� although� these� models� can�be�generalized�to�designs�with�more�than�two�factors��Most�of�the�concepts�used�in� this�chapter�are�the�same�as�those�covered�in�previous�chapters��In�addition,�new�con- cepts�include�crossed�and�nested�factors,�confounding,�blocking�factors,�and�methods�of� blocking��Our�objectives�are�that�by�the�end�of�this�chapter,�you�will�be�able�to�(a)�under- stand� the� characteristics� and� concepts� underlying� hierarchical� and� randomized� block� ANOVA� models,� (b)� determine� and� interpret� the� results� of� hierarchical� and� random- ized�block�ANOVA�models,�(c)�understand�and�evaluate�the�assumptions�of�hierarchical� and�randomized�block�ANOVA�models,�and�(d)�compare�different�ANOVA�models�and� select�an�appropriate�model�
16.1 Two-Factor Hierarchical Model
Throughout� the� text,� we� have� followed� Marie,� a� graduate� student� enrolled� in� an� educa- tional�research�program,�on�her�statistical�analysis�adventures��In�this�chapter,�we�see�her� embarking�on�a�new�journey�
Seeing�the�success�that�Marie�has�had�with�more�complex�statistical�analysis,�Marie’s� faculty�advisor�has�provided�Marie�with�another�challenging�task��This�time,�Marie� will� be� working� with� a� reading� faculty� member� (JoAnn)� at� their� university�� JoAnn� has� conducted� an� experiment� in� which� children� were� randomly� assigned� to� one� of� two�reading�approaches�(basal�or�whole�language)�and�one�of�four�different�teachers�� There�were�24�children�who�participated;�thus,�there�were�six�children�in�each�read- ing� approach-teacher� combination�� Each� student� was� assessed� on� reading� compre- hension�at�the�conclusion�of�the�study��JoAnn�wants�to�know�the�following:�if�there� is�a�mean�difference�in�reading�based�on�approach�to�reading�and�if�there�is�a�mean� difference�in�reading�between�teachers��Marie�suggests�the�following�research�ques- tions�to�JoAnn:
•� Is there a mean difference in reading based on approach to reading? •� Is there a mean difference in reading based on teacher?
With�one�between-subjects�independent�variable�(i�e�,�approach�to�reading)�and�one�hier- archical� or� nested� factor� (i�e�,� teacher),� Marie� determines� that� a� two-factor� hierarchical� ANOVA�is�the�best�statistical�procedure�to�use�to�answer�JoAnn’s�question��Her�next�task� is�to�assist�JoAnn�in�analyzing�the�data�
In�this�section,�we�describe�the�distinguishing�characteristics�of�the�two-factor�hierarchi- cal�ANOVA�model,�the�layout�of�the�data,�the�linear�model,�the�ANOVA�summary�table� and�expected�mean�squares,�and�multiple�comparison�procedures�(MCPs)�
559Hierarchical and Randomized Block Analysis of Variance Models
16.1.1 Characteristics of the Model
The� characteristics� of� the� two-factor� fixed-,� random-,� and� mixed-effects� models� have� already�been�covered�in�Chapters�13�and�15��Here�we�consider�a�special�form�of�the�two- factor�model�where�one�factor�is�nested�within�another�factor��The�best�introduction�to�this� model�is�via�an�example��Suppose�you�are�interested�in�which�of�several�different�major� teaching�pedagogies�(e�g�,�worksheet,�math�manipulative,�and�computer-based�approaches)� results�in�the�highest�level�of�achievement�in�mathematics�among�second-grade�students�� Thus,�math�achievement�is�the�dependent�variable,�and�teaching�pedagogy�is�one�factor�� A�second�factor�is�teacher��That�is,�you�may�also�believe�that�some�teachers�are�more�effec- tive�than�others,�which�results�in�different�levels�of�student�achievement��However,�each� teacher� has� only� one� class� of� students� and� thus� only� one� major� teaching� pedagogy�� In� other�words,�all�combinations�of�the�pedagogy�and�teacher�factors�are�not�possible��This� design� is� known� as� a� nested design,� hierarchical design,� or� multilevel model� because� the�teacher�factor�is�nested�within�the�pedagogy�factor��This�is�in�contrast�to�a�two-factor� crossed design�where�all�possible�combinations�of�the�two�factors�are�included��The�two- factor�designs�described�in�Chapters�13�and�15�were�all�crossed�designs�
Let�us�give�a�more�precise�definition�of�crossed�and�nested�designs��A�two-factor�com- pletely�crossed�design�(or�complete factorial design)�is�one�where�every�level�of�factor�A� occurs�in�combination�with�every�level�of�factor�B��A�two-factor�nested�design�(or�incom- plete factorial design)�of�factor�B�being�nested�within�factor�A�is�one�where�the�levels�of� factor� B� occur� for� only� one� level� of� factor� A�� We� denote� this� particular� nested� design� as� B(A),�which�is�read�as�factor�B�being�nested�within�factor�A�(in�other�references,�you�may� see�this�written�as�B:A�or�as�B|A)��To�return�to�our�example,�the�teacher�factor�(factor�B)�is� nested�within�the�method�factor�(factor�A),�as�each�teacher�utilizes�only�one�major�teaching� pedagogy��The�outcome�measured�is�student�performance��Thus,�a�researcher�may�select� a�nested�design�to�examine�the�extent�to�which�student�performance�in�mathematics�dif- fers�given�that�teachers�are�nested�within�teaching�pedagogy��The�researcher�is�likely�most� interested�in�the�treatment�(e�g�,�teaching�pedagogy),�but�recognizes�that�the�context�(i�e�,� the�classroom�teacher)�may�contribute�to�differences�in�the�outcome,�and�can�model�this� statistically�through�a�hierarchical�ANOVA�
These�models�are�shown�graphically�in�Figure�16�1��In�Figure�16�1a,�a�completely�crossed� or�complete�factorial�design�is�shown�where�there�are�two�levels�of�factor�A�and�six�levels�of� factor�B��Thus,�there�are�12�possible�factor�combinations�that�would�all�be�included�in�a�com- pletely�crossed�design��The�shaded�region�indicates�the�combinations�that�might�be�included� in�a�nested�or�incomplete�factorial�design�where�factor�B�(e�g�,�teacher)�is�nested�within�fac- tor� A� (e�g�,� teaching� pedagogy)�� Although� the� number� of� levels� of� each� factor� remains� the� same,�factor�B�now�has�only�three�levels�within�each�level�of�factor�A��For�A1,�we�see�only� B1,�B2,�and�B3,�whereas�for�A2,�we�see�only�B4,�B5,�and�B6��Thus,�only�6�of�the�possible�12�fac- tor�combinations�are�included�in�the�nested�design��For�example,�level�1�of�factor�B�occurs� only�in�combination�with�level�1�of�factor�A��In�summary,�Figure�16�1a�shows�that�the�nested� or� incomplete� factorial� design� consists� of� only� a� portion� of� the� completely� crossed� design� (the� shaded� regions)�� In� Figure� 16�1b,� we� see� the� nested� design� depicted� in� its� more� tradi- tional�form��Here�you�see�that�the�six�factor�combinations�not�included�are�not�even�shown� (e�g�,�A1�with�B4)��Other�examples�of�the�two-factor�nested�design�are�as�follows:�(a)�school�is� nested�within�school�district,�(b)�faculty�member�is�nested�within�department,�(c)�individual� is�nested�within�neighborhood,�and�(d)�county�is�nested�within�state�
Thus,�with�this�design,�one�factor�is�nested�within�another�factor,�rather�than�the�two�fac- tors�being�crossed��As�is�shown�in�more�detail�later�in�this�chapter,�the�nesting�characteristic�
560 An Introduction to Statistical Concepts
has�some�interesting�and�distinct�outcomes��For�now,�some�brief�mention�should�be�made�of� these�outcomes��Nesting�is�a�particular�type�of�confounding�among�the�factors�being�investi- gated,�where�the�AB�interaction�is�part�of�the�B�effect�(or�is�confounded�with�B)�and�therefore� cannot�be�investigated��(Going�back�to�the�previous�example,�this�means�that�the�teacher�by� teaching�pedagogy�interaction�effect�is�confounded�with�the�teacher�main�effect,�and�thus� teasing�apart�those�effects�is�not�possible�)�In�the�ANOVA�model�and�the�ANOVA�summary� table,�there�will�not�be�an�interaction�term�or�source�of�variation��This�is�due�to�the�fact�that� each�level�of�factor�B�(the�nested�factor,�such�as�the�teacher)�occurs�in�combination�with�only� one�level�of�factor�A�(the�nonnested�factor,�such�as�the�teaching�pedagogy)��We�cannot�com- pare�for�a�particular�level�of�B�(e�g�,�the�classroom�teacher)�all�levels�of�factor�A�(e�g�,�teaching� pedagogy),�as�a�certain�level�of�B�only�occurs�with�one�level�of�A�
Confounding�may�occur�for�two�reasons��First,�the�confounding�may�be�intentional�due� to�practical�reasons,�such�as�a�reduction�in�the�number�of�individuals�to�be�observed��Fewer� individuals�would�be�necessary�in�a�nested�design,�as�compared�to�a�crossed�design,�due� to�the�fact�that�there�are�fewer�cells�in�the�model��Second,�the�confounding�may�be�abso- lutely�necessary�because�crossing�may�not�be�possible��For�example,�school�is�nested�within� school�district�because�a�particular�school�can�only�be�a�member�of�one�school�district��The� nested�factor�(here�factor�B)�may�be�a�nuisance�variable�that�the�researcher�wants�to�take� into�account�in�terms�of�explaining�or�predicting�the�dependent�variable�Y��An�error�com- monly� made� is� to� ignore� the� nuisance� variable� B� and� go� ahead� with� a� one-factor� design� using�only�factor�A��This�design�may�result�in�a�biased�test�of�factor�A�such�that�the�F�ratio� is�inflated��Thus,�H0�would�be�rejected�more�often�than�it�should�be,�serving�to�increase�the� actual�α�level�over�that�specified�by�the�researcher�and�thereby�increase�the�likelihood�of�a� Type�I�error��The�F�test�is�then�too�liberal�
Let�us�make�two�further�points�about�this�first�characteristic��First,�in�the�one-factor�design� discussed� in� Chapter� 11,� we� have� already� seen� nesting� going� on� in� a� different� way�� Here� subjects� were� nested� within� factor� A� because� each� subject� only� responded� to� one� level� of� factor�A��It�was�only�when�we�got�to�repeated�measures�designs�in�Chapter�15�that�individu- als�were�allowed�to�respond�to�more�than�one�level�of�a�factor��For�the�repeated�measures� design,�we�actually�had�a�completely�crossed�design�of�subjects�by�factor�A��Second,�Glass� and�Hopkins�(1996)�give�a�nice�conceptual�example�of�a�nested�design�with�teachers�being� nested�within�schools,�where�each�school�is�like�a�nest�having�multiple�eggs�or�teachers�
B1 B2 B3 B4 B5 B6 A1 A2
(a)
A1 A2 B1 B2 B3 B4 B5 B6
(b)
FIGuRe 16.1 Two-factor�completely�crossed�versus�nested�designs��(a)�The�completely crossed design:�The�shaded�region�indi- cates�the�cells�that�would�be�included�in�a�nested�design�where�factor�B�is�nested�within�factor�A��In�the�nested� design,�factor�A�has�two�levels,�and�factor�B�has�three�levels�within�each�level�of�factor�A��You�see�that�only�6�of� the�12�possible�cells�are�filled�in�the�nested�design��(b)�The�same�nested�design�in�traditional form:�The�shaded� region�indicates�the�cells�included�in�the�nested�design�(i�e�,�the�same�six�as�shown�in�the�first�part)�
561Hierarchical and Randomized Block Analysis of Variance Models
The� remaining� characteristics� should� be� familiar�� These� include� the� following:� (a)� two� factors�(or�independent�variables)� that�are�nominal�or�ordinal�in�scale,�each�with�two�or� more�levels;�(b)�the�levels�of�each�of�the�factors�may�be�either�randomly�sampled�from�the� population�of�levels�or�fixed�by�the�researcher�(i�e�,�the�model�may�be�fixed,�mixed,�or�ran- dom);�(c)�subjects�are�randomly�assigned�to�only�one�combination�of�the�levels�of�the�two� factors;�and�(d)�the�dependent�variable�is�measured�at�least�at�the�interval�level��If�individu- als� respond� to�more� than� one� combination� of� the� levels� of� the� two� factors,�then�this� is� a� repeated�measures�design�(see�Chapter�15)�
For�simplicity,�we�again�assume�the�design�is�balanced��For�the�two-factor�nested�design,�a� design�is�balanced�if�(a)�the�number�of�observations�within�each�factor�combination�(or�cell)� is�the�same�(in�other�words,�the�sample�size�for�each�cell�of�the�design�is�the�same),�and�(b)� the�number�of�levels�of�the�nested�factor�within�each�level�of�the�other�factor�is�the�same��The� first�portion�of�this�statement�should�be�quite�familiar�from�factorial�designs,�so�no�further� explanation�is�necessary��The�second�portion�of�this�statement�is�unique�to�this�design�and� requires�a�brief�explanation��As�an�example,�say�factor�B�is�nested�within�factor�A�and�factor� A�has�two�levels��On�the�one�hand,�factor�B�may�have�the�same�number�of�levels�for�each�level� of�factor�A��This�occurs�if�there�are�three�levels�of�factor�B�under�level�1�of�factor�A�(i�e�,�A1)�and� also�three�levels�of�factor�B�under�level�2�of�factor�A�(i�e�,�A2)��On�the�other�hand,�factor�B�may� not�have�the�same�number�of�levels�for�each�level�of�factor�A��This�occurs�if�there�are�three� levels�of�factor�B�under�A1�and�only�two�levels�of�factor�B�under�A2��If�the�design�is�unbal- anced,�see�the�discussion�in�Kirk�(1982)�and�Dunn�and�Clark�(1987),�although�most�statistical� software�can�seamlessly�deal�with�this�type�of�unbalanced�design�
16.1.2 layout of data
The� layout� of� the� data� for� the� two-factor� nested� design� is� shown� in� Table� 16�1�� To� sim- plify�matters,�we�have�limited�the�number�of�levels�of�the�factors�to�two�levels�of�factor�A� (e�g�,�teaching�pedagogy)�and�three�levels�of�factor�B�(e�g�,�teacher)��This�only�serves�as�an� example�layout�because�many�other�possibilities�obviously�exist��Here�we�see�the�major�set� of�columns�designated�as�the�levels�of�factor�A,�the�nonnested�factor�(e�g�,�teaching�peda- gogy),�and�for�each�level�of�A,�the�minor�set�of�columns�are�the�levels�of�factor�B,�the�nested� factor�(e�g�,�teacher)��Within�each�factor�level�combination�or�cell�are�the�subjects��Means�are� shown�for�each�cell,�for�the�levels�of�factor�A,�and�overall��Note�that�the�means�for�the�levels� of�factor�B�need�not�be�shown,�as�they�are�the�same�as�the�cell�means��For�instance,�Y
– �11�is�
the�same�as�Y –
��1�(not�shown)�as�B1�only�occurs�once��This�is�another�result�of�the�nesting�
Table 16.1
Layout�for�the�Two-Factor�Nested�Design
A1 A2
B1 B2 B3 B4 B5 B6
Y111 Y112 Y113 Y124 Y125 Y126 � � � � � � � � � � � � � � � � � �
Yn11 Yn12 Yn13 Yn24 Yn25 Yn26
Cell�means Y –
�11 Y –
�12 Y –
�13 Y –
�24 Y –
�25 Y –
�26
A�means Y –
�1� Y –
�2�
Overall�mean Y –
…
562 An Introduction to Statistical Concepts
16.1.3 aNOVa Model
The� nested� factor� is� almost� always� random� (Glass� &� Hopkins,� 1996;� Keppel� &� Wickens,� 2004;�Mickey,�Dunn,�&�Clark,�2004;�Page,�Braver,�&�MacKinnon,�2003)��In�other�words,�the� levels�of�the�nested�factor�are�a�random�sample�of�the�population�of�levels��For�example,� in�the�case�of�teachers�nested�within�teaching�pedagogy,�it�is�often�the�case�that�a�random� sample� of� the� teachers� is� selected� rather� than� specific� teachers� (which� would� be� a� fixed- effects�factor)��Thus,�the�nested�factor�(i�e�,�the�teacher�factor)�is�a�random�factor��As�a�result,� the�two-factor�nested�ANOVA�is�often�a�mixed-effects�model�where�the�nonnested�factor�is� fixed�(i�e�,�all�the�levels�of�interest�for�the�nonnested�factor�are�included�in�the�model)�and� the�nested�factor�is�random��The�two-factor�mixed-effects�nested�ANOVA�model�is�written� in�terms�of�population�parameters�as
Y bijk j k j ijk= + + +µ α ε( )
where Yijk�is�the�observed�score�on�the�dependent�variable�for�individual�i�in�level�j�of�factor�A�
and�level�k�of�factor�B�(or�in�the�jk�cell) μ�is�the�overall�or�grand�population�mean�(i�e�,�regardless�of�cell�designation) αj�is�the�fixed�effect�for�level�j�of�factor�A bk(j)�is�the�random�effect�for�level�k�of�factor�B εijk�is�the�random�residual�error�for�individual�i�in�cell�jk
Notice�that�there�is�no�interaction�term�in�the�model�and�also�that�the�effect�for�factor�B�is� denoted�by�bk(j)��This tells us that factor B is nested within factor A��The�residual�error�can�be� due�to�individual�differences,�measurement�error,�and/or�other�factors�not�under�investi- gation��We�consider�the�fixed-,�mixed-,�and�random-effects�cases�later�in�this�chapter�
For� the� two-factor� mixed-effects� nested� ANOVA� model,� there� are� only� two� sets� of� hypotheses,�one�for�each�of�the�main�effects,�because�there�is�no�interaction�effect��The�null� and� alternative� hypotheses,� respectively,� for� testing� the� effect� of� factor� A� are� as� follows�� The�null�hypothesis�is�similar�to�what�we�have�seen�in�previous�chapters�for�fixed-effects� factors�and�written�as�the�means�of�the�levels�of�factor�A�are�the�same:
H J01 1 2: . . . . . .µ µ µ= = =�
H j11 not all the are equal: . .µ
The�hypotheses�for�testing�the�effect�of�factor�B,�because�this�is�a�random-effects�factor,�are� written�as�the�variation�among�the�means,�and�are�presented�as�follows:
H b0 02 2: σ =
H b12 2: σ > 0
These� hypotheses� reflect� the� inferences� made� in� the� fixed-,� mixed-,� and� random-effects� models�(as�fully�described�in�Chapter�15)��For�fixed�main�effects,�the�null�hypotheses�are� about� means,� whereas� for� random� main� effects,� the� null� hypotheses� are� about� variation�
563Hierarchical and Randomized Block Analysis of Variance Models
among�the�means��As�we�already�know,�the�difference�in�the�models�is�also�reflected�in� the�MCPs��As�before,�we�do�need�to�pay�particular�attention�to�whether�the�model�is�fixed,� mixed,� or� random�� The� assumptions� about� the� two-factor� nested� model� are� exactly� the� same�as�with�the�two-factor�crossed�model�(discussed�in�Chapters�13�and�15),�and,�thus,�we� need�not�provide�any�additional�discussion�other�than�to�remind�you�of�the�assumptions� regarding�normality,�homogeneity�of�variance,�and�independence�(of�observations�within� cells)��In�addition,�procedures�for�determining�power,�confidence�intervals�(CIs),�and�effect� size�are�the�same�as�with�the�two-factor�crossed�model�
16.1.4 aNOVa Summary Table and expected Mean Squares
The� computations� of� the� two-factor� mixed-effects� nested� model� are� somewhat� similar� to�those�of�the�two-factor�mixed-effects�crossed�model��The�main�difference�lies�in�the� fact�that�there�is�no�interaction�term��The�ANOVA�summary�table�is�shown�in�Table�16�2,� where�we�see�the�following�sources�of�variation:�A,�B(A),�within�cells,�and�total��There� we�see�that�only�two�F�ratios�can�be�formed,�one�for�each�of�the�two�main�effects,�because� no�interaction�term�is�estimated�(recall�that�this�is�because�not�all�possible�combinations� of�A�and�B�occur)�
If�we�take�the�total�sum�of�squares�and�decompose�it,�we�have�the�following:
SS SS SS SStotal A B A with= + +( )
We�leave�the�computations�involving�these�terms�to�the�statistical�software��The�degrees� of�freedom,�mean�squares,�and�F�ratios�are�determined�as�shown�in�Table�16�2,�assuming�a� mixed-effects�model��The�critical�value�for�the�test�of�factor�A�is�αFJ−1,�J�(K(j)−1)�and�for�the�test� of�factor�B�is�αFJ(K(j)−1),�JK(j)�(n−1)��Let�us�explain�something�about�the�degrees�of�freedom��The� degrees�of�freedom�for�B(A)�are�equal�to�J(K(j)�−�1)��This�means�that�for�a�design�with�two� levels�of�factor�A�(e�g�,�teaching�pedagogy)�and�three�levels�of�factor�B�(e�g�,�teacher)�within� each�level�of�A�(for�a�total�of�six�levels�of�B),�the�degrees�of�freedom�are�equal�to�2(3�−�1)�=�4�� This is�not�the�same�as�the�degrees�of�freedom�for�a�completely�crossed�design�where�dfB� would�be�5�(i�e�,�6�−�1�=�5)��The�degrees�of�freedom�for�within�are�equal�to�JK(j)(n�−�1)��For� this�same�design�with�n�=�10,�then�the�degrees�of�freedom�within�are�equal�to�(2)(3)(10�−�1)�=�54� (i�e�,�six�cells�with�nine�degrees�of�freedom�per�cell)�
The�appropriate�error�terms�for�each�of�the�fixed-,�random-,�and�mixed-effects�models�are� described�in�the�following�two�paragraphs��For�the�fixed-effects�model,�both�F�ratios�use� the�within�source�as�the�error�term��For�the�random-effects�model,�the�appropriate�error� term�for�the�test�of�A�is�MSB(A)�and�for�the�test�of�B�is�MSwith��For�the�mixed-effects�model� where� A� is� fixed� and� B� is� random,� the� appropriate� error� term� for� the� test� of� A� is� MSB(A),�
Table 16.2
Two-Factor�Nested�Design�ANOVA�Summary� Table:�Mixed-Effects�Model
Source SS df MS F
A SSA J�−�1 MSA MSA/MSB(A) B(A) SSB(A) J(K(j)�−�1) MSB(A) MSB(A)/MSwith Within SSwith JK(j)(n�−�1) MSwith
Total SStotal N�−�1
564 An Introduction to Statistical Concepts
and� for� the� test� of� B,� is� MSwith�� As� already� mentioned,� this� is� the� predominant� model� in� education� and� the� behavioral� sciences�� Finally,� for� the� mixed-effects� model� where� A� is� random�and�B�is�fixed,�both�F�ratios�use�the�within�source�as�the�error�term��These�are�now� described�by�the�expected�mean�squares�
The�formation�of�the�proper�F�ratios�is�again�related�to�the�expected�mean�squares��If�H0� is�actually�true,�then�the�expected mean squares�are�as�follows:
E( )AMS = σε 2
E( )B AMS ( ) = σε 2
E( )withMS = σε 2
If�H0�is�actually�false,�then�the�expected�mean�squares�for�the�fixed-effects case�are�as�follows:
E( ) /AMS nK Jj j j
J
= + −
= ∑σ αε2 2
1
1( ) ( )
E( ) /B AMS n J Kk j k
K
j
J
j( ) ( ) ( )( )= + −
== ∑∑σ βε2 2
11
1
E( )withMS = σε 2
Thus,�the�appropriate�F�ratios�both�involve�using�the�within�source�as�the�error�term� If�H0�is�actually�false,�then�the�expected�mean�squares�for�the�random-effects case�are�as�
follows:
E( )AMS n nKb a j a= + +σ σ σε 2 2 2
( ) ( )
E( )B AMS n b a( ) ( )= +σ σε 2 2
E( )withMS = σε 2
Thus,�the�appropriate�error�term�for�the�test�of�A�is�MSB(A),�and�the�appropriate�error�term� for�the�test�of�B�is�MSwith�
If�H0�is�actually�false,�then�the�expected�mean�squares�for�the�mixed-effects case where A is fixed and B is random�are�as�follows:
E( )AMS n nK Jb a j j j
J
= + + −
= ∑σ σ αε2 2 2
1
1( ) ( ) /( )
565Hierarchical and Randomized Block Analysis of Variance Models
E( )B AMS n b a( ) ( )= +σ σε 2 2
E( )withMS = σε 2
Thus,�the�appropriate�error�term�for�the�test�of�A�is�MSB(A),�and�the�appropriate�error�term� for�the�test�of�B�is�MSwith�
Finally,� if� H0� is� actually� false,� then� the� expected� mean� squares� for� the� mixed-effects case where A is random and B is fixed�are�as�follows:
E( )AMS nK j a= +σ σε 2 2
( )
E( )B AMS n J Kk j k
K
j
J
j( ) ( ) ( )/ ( )= + −
== ∑∑σ βε2 2
11
1
E( )withMS = σε 2
Thus,�the�appropriate�F�ratios�both�involve�using�the�within�source�as�the�error�term�
16.1.5 Multiple Comparison procedures
This�section�considers�MCPs�for�the�two-factor�nested�design��First�of�all,�the�researcher� is�usually�not�interested�in�making�inferences�about�random�effects��Second,�for�MCPs� based� on� the� levels� of� factor� A� (the� nonnested� factor),� there� is� nothing� new� to� report�� Third,� for� MCPs� based� on� the� levels� of� factor� B� (the� nested� factor),� this� is� a� different� situation��The�researcher�is�not�usually�as�interested�in�MCPs�about�the�nested�factor�as� compared�to�the�nonnested�factor�because�inferences�about�the�levels�of�factor�B�are�not� even�generalizable�across�the�levels�of�factor�A,�due�to�the�nesting��If�you�are�nonethe- less�interested�in�MCPs�for�factor�B,�by�necessity�you�have�to�look�within�a�level�of�A�to� formulate�a�contrast��Otherwise�MCPs�are�conducted�as�before��For�more�complex�nested� designs,�see�Myers�(1979),�Kirk�(1982),�Dunn�and�Clark�(1987),�Myers�and�Well�(1995),�or� Keppel�and�Wickens�(2004)�
16.1.6 example
Let�us�consider�an�example�to�illustrate�the�procedures�in�this�section��The�data�are�shown� in� Table� 16�3�� Factor� A� is� approach� to� the� teaching� of� reading� (basal� vs�� whole� language� approaches),�and�factor�B�is�teacher��Thus,�there�are�two�teachers�using�the�basal�approach� and� two� different� teachers� using� the� whole� language� approach�� The� researcher� is� inter- ested� in� the� effects� these� factors� have� on� student’s� reading� comprehension� in� the� first� grade��Thus,�the�dependent�variable�is�a�measure�of�reading�comprehension��Six�students� are�randomly�assigned�to�each�approach-teacher�combination�for�small-group�instruction�� This� particular� example� is� a� mixed� model,� where� factor� A� (teaching� method)� is� a� fixed� effect�and�factor�B�(teacher)�is�a�random�effect��The�results�are�shown�in�the�ANOVA�sum- mary�table�of�Table�16�4�
566 An Introduction to Statistical Concepts
From�Table�A�4,�the�critical�value�for�the�test�of�factor�A�is�αFJ−1,�J(K(j)−1)�=��05F1,2�=�18�51,�and� the�critical�value�for�the�test�of�factor�B�is� αFJ�(K(j)−1),�JK(j)�(n−1)�=� �05F2,20�=�3�49��Thus,�there�is�a� statistically� significant� difference� between� the� two� approaches� to� reading� instruction� at� the� �05� level� of� significance,� and� there� is� no� significant� difference� between� the� teachers�� When�we�look�at�the�means�for�the�levels�of�factor�A,�we�see�that�the�mean�comprehension� score�for�the�whole�language�approach�(Y
– �2��=�10�8333)�is�greater�than�the�mean�for�the�basal�
approach�(Y –
�1��=�3�3333)��Because�there�were�only�two�levels�of�the�reading�approach�tested� (whole�language�and�basal),�no�post�hoc�multiple�comparisons�are�really�necessary��Rather� the� mean� reading� comprehension� scores� for� each� approach� can� be� merely� examined� to� determine�which�mean�was�statistically�significantly�larger�
16.2 Two-Factor Randomized Block Design for n = 1
In� this� section,� we� describe� the� distinguishing� characteristics� of� the� two-factor� random- ized� block� ANOVA� model� for� one� observation� per� cell,� the� layout� of� the� data,� the� linear� model,�assumptions�and�their�violation,�the�ANOVA�summary�table�and�expected�mean� squares,�MCPs,�and�methods�of�block�formation�
Table 16.4
Two-Factor�Nested�Design�ANOVA� Summary�Table:�Teaching�Reading�Example
Source SS df MS F
A 337�5000 1 337�5000 59�5585a
B(A) 11�3333 2 5�6667 0�9524b
Within 119�0000 20 5�9500
Total 467�8333 23
a� �05F1,2�=�18�51�
b� �05F2,20�=�3�49�
Table 16.3
Data�for�the�Teaching�Reading�Example:�Two-Factor� Nested Design
Reading Approaches
A1 (Basal) A2 (Whole Language)
Teacher B1 Teacher B2 Teacher B3 Teacher B4
1 1 7 8 1 3 8 9 2 3 8 11 4 4 10 13 4 6 12 14 5 6 15 15
Cell�means 2�8333 3�8333 10�0000 11�6667 A�means 3�3333 10�8333 Overall�mean 7�0833
567Hierarchical and Randomized Block Analysis of Variance Models
16.2.1 Characteristics of the Model
The� characteristics� of� the� two-factor� randomized� block� ANOVA� model� are� quite� similar� to�those�of�the�regular�two-factor�ANOVA�model,�as�well�as�sharing�a�few�characteristics� with� the� one-factor� repeated� measures� ANOVA� design�� There� is� one� obvious� exception,� which�has�to�do�with�the�nature�of�the�factors�being�used��Here�there�will�be�two�factors,� each�with�at�least�two�levels��One�factor�is�known�as�the�treatment factor�and�is�referred� to� here� as� factor� A� (a� treatment� factor� is� technically� what� we� have� been� considering� in� Chapters�11�through�15)��The�second�factor�is�known�as�the�blocking factor�and�is�referred� to�here�as�factor�B��A�blocking�factor�is�a�new�concept�and�requires�some�discussion�
Take� an� ordinary� one-factor� ANOVA� design,� where� the� single� factor� is� a� treatment� fac- tor�(e�g�,�method�of�exercising)�and�the�researcher�is�interested�in�its�effect�on�some�depen- dent�variable�(e�g�,�percentage�of�body�fat)��Despite�individuals�being�randomly�assigned�to� a� treatment� group,� the� groups� may� be� different� due� to� a� nuisance� variable� operating� in� a� nonrandom�way��For�instance,�group�1�may�consist�of�mostly�older�adults�and�group�2�may� consist�of�mostly�younger�adults��Thus,�it�is�likely�that�group�2�will�be�favored�over�group�1� because�age,�the�nuisance�variable,�has�not�been�properly�balanced�out�across�the�groups�by� the�randomization�process�
One� way� to� deal� with� this� problem� is� to� control� the� effect� of� the� nuisance� variable� by� incorporating�it�into�the�design�of�the�study��Including�the�blocking�or�nuisance�variable� as� a� factor� in� the� design� should� result� in� a� reduction� in� residual� variation� (due� to� some� additional� portion� of� individual� differences� being� explained)� and� an� increase� in� power� (Glass� &� Hopkins,� 1996;� Keppel� &� Wickens,� 2004)�� The� blocking� factor� is� selected� based� on� the� strength� of� its� relationship� to� the� dependent� variable,� where� an� unrelated� block- ing�variable�would�not�reduce�residual�variation��It�would�be�reasonable�to�expect,�then,� that�variability�among�individuals�within�a�block�(e�g�,�within�younger�adults)�should�be� less�than�variability�among�individuals�between�blocks�(e�g�,�between�younger�and�older� adults)��Thus,�each�block�represents�the�formation�of�a�matched�set�of�individuals,�that�is,� matched�on�the�blocking�variable,�but�not�necessarily�matched�on�any�other�nuisance�vari- able�� Using� our� example,� we� expect� that� in� general,� adults� within� a� particular� age� block� (i�e�,�the�older�or�younger�blocks)�will�be�more�similar�in�terms�of�variables�related�to�body� fat�than�adults�across�blocks�
Let�us�consider�several�examples�of�blocking�factors��Some�blocking�factors�are�naturally� occurring�blocks�such�as�siblings,�friends,�neighbors,�plots�of�land,�and�time��Other�block- ing�factors�are�not�naturally�occurring�but�can�be�formulated�by�the�researcher��Examples� of� this� type� include� grade� point� average� (GPA),� age,� weight,� aptitude� test� scores,� intelli- gence�test�scores,�socioeconomic�status,�and�school�or�district�size��Note�that�the�examples� of�blocking�factors�here�represent�a�variety�of�measurement�scales�(categorical�as�well�as� continuous)��Later�we�will�discuss�how�to�deal�with�the�blocking�factor�based�on�its�mea- surement�scale�
Let� us� make� some� summary� statements� about� characteristics� of� blocking� designs�� First,�designs�that�include�one�or�more�blocking�factors�are�known�as�randomized block designs,�also�known�as�matching�designs�or�treatment�by�block�designs��The�researcher’s� main� interest� is� in� the� treatment� factor�� The� purpose� of� the� blocking� factor� is� to� reduce� residual�variation��Thus,�the�researcher�is�not�as�much�interested�in�the�test�of�the�blocking� factor�(possibly�not�at�all)�as�compared�to�the�treatment�factor��Thus,�there�is�at�least�one� blocking�factor�and�one�treatment�factor,�each�with�two�or�more�levels��Second,�each�sub- ject�falls�into�only�one�block�in�the�design�and�is�subsequently�randomly�assigned�to�one� level�of�the�treatment�factor�within�that�block��Thus,�subjects�within�a�block�serve�as�their�
568 An Introduction to Statistical Concepts
own�controls�such�that�some�portion�of�their�individual�differences�is�taken�into�account�� As� a� result,� the� scores� of� subjects� are� not� independent� within� a� particular� block�� Third,� for�purposes�of�this�section,�we�assume�there�is�only�one�subject�for�each�treatment-block� level�combination��As�a�result,�the�model�does�not�include�an�interaction�term��Later�in�this� chapter,�we�consider�the�multiple�observations�case,�where�there�is�an�interaction�term�in� the�model��Finally,�the�dependent�variable�is�measured�at�least�at�the�interval�level�
16.2.2 layout of data
The�layout�of�the�data�for�the�two-factor�randomized�block�model�is�shown�in�Table�16�5�� Here�we�see�the�columns�designated�as�the�levels�of�the�blocking�factor�B�and�the�rows�as� the�levels�of�the�treatment�factor�A��Row,�block,�and�overall�means�are�also�shown��Here� you�see�that�the�layout�of�the�data�looks�the�same�as�the�two-factor�model,�but�with�a�single� observation�per�cell�
16.2.3 aNOVa Model
The�two-factor�fixed-effects�randomized�block�ANOVA�model�is�written�in�terms�of�popu- lation�parameters�as
Yjk j k jk= + + +µ α β ε
where Yjk�is�the�observed�score�on�the�dependent�variable�for�the�individual�responding�to�level�
j�of�factor�A�and�level�k�of�block�B μ�is�the�overall�or�grand�population�mean αj�is�the�fixed�effect�for�level�j�of�factor�A βk�is�the�fixed�effect�for�level�k�of�the�block�B εjk�is�the�random�residual�error�for�the�individual�in�cell�jk
The�residual�error�can�be�due�to�measurement�error,�individual�differences,�and/or�other� factors�not�under�investigation��You�can�see�this�is�similar�to�the�two-factor�fully�crossed� model�with�one�observation�per�cell�(i�e�,�i�=�1,�making�the�i�subscript�unnecessary)�and� with�no�interaction�term�included��Also,�the�effects�are�denoted�by�α�and�β�given�we�have� a� fixed-effects� model�� Note� that� the� row� and� column� effects� both� sum� to� 0� in� the� fixed- effects�model�
Table 16.5
Layout�for�the�Two-Factor�Randomized�Block�Design
Level of Factor A
Level of Factor B
Row Mean1 2 … K
1 Y11 Y12 … Y1K Y –
1�
2 Y21 Y22 … Y2K Y –
2�
� � � … � � � � � … � � � � � … � � J YJ1 YJ2 YJK Y
– J�
Block�mean Y –
�1 Y –
�2 … Y –
�K Y –
���(overall mean)
569Hierarchical and Randomized Block Analysis of Variance Models
The�hypotheses�for�testing�the�effect�of�factor�A�are�as�follows,�where�the�null�indicates� that�the�means�of�the�levels�of�factor�A�are�equal:
H J01 1 2: . . .µ µ µ= = =�
H j11 not all the are equal: .µ
For�testing�the�effect�of�factor�B�(the�blocking�factor),�the�hypotheses�are�presented�here,� where�the�null�hypothesis�is�that�the�means�of�the�levels�of�the�blocking�factor�are�equal:
H K02 1 2: . . .µ µ µ= = =�
H k12 not all the are equal: .µ
The�factors�are�both�fixed,�so�the�hypotheses�are�written�in�terms�of�means�
16.2.4 assumptions and Violation of assumptions
In� Chapter� 15,� we� described� the� assumptions� for� the� one-factor� repeated� measures� ANOVA�model��The�assumptions�are�nearly�the�same�for�the�two-factor�randomized�block� model,�and�we�need�not�devote�much�attention�to�them�here��As�before,�the�assumptions� are�mainly�concerned�with�independence,�normality,�and�homogeneity�of�variance�of�the� population�scores�on�the�dependent�variable�
Another� assumption� is� compound symmetry� and� is� necessary� because� the� observa- tions� within� a� block� are� not� independent�� The� assumption� states� that� the� population� covariances� for� all� pairs� of� the� levels� of� the� treatment� factor� A� (i�e�,� j� and� j′)� are� equal�� ANOVA�is�not�particularly�robust�to�a�violation�of�this�assumption��If�the�assumption�is� violated,�three�alternative�procedures�are�available��The�first�is�to�limit�the�levels�of�factor� A,�either�to�those�that�meet�the�assumption�or�to�two�levels�(in�which�case,�there�is�only� one� covariance)�� The� second,� and� more� plausible,� alternative� is� to� use� adjusted� F� tests�� These�are�reported�shortly��The�third�is�to�use�multivariate�ANOVA,�which�has�no�com- pound� symmetry� assumption� but� is� slightly� less� powerful�� This� method� is� beyond� the� scope�of�this�text�
Huynh�and�Feldt�(1970)�showed�that�the�compound�symmetry�assumption�is�a�sufficient� but�unnecessary�condition�for�the�test�of�treatment�factor�A�to�be�F�distributed��Thus,�the�F�test� may�also�be�valid�under�less�stringent�conditions��The�necessary�and�sufficient�condition� for�the�validity�of�the�F�test�of�A�is�known�as�sphericity��This�assumes�that�the�variance�of� the�difference�scores�for�each�pair�of�factor�levels�is�the�same��Further�discussion�of�sphe- ricity�is�beyond�the�scope�of�this�text�(see�Keppel,�1982;�or�Kirk,�1982),�although�we�have� previously�discussed�sphericity�for�repeated�measures�designs�in�Chapter�15�
A� final� assumption� purports� that� there� is� no� interaction� between� the� treatment� and� blocking� factors�� This� is� obviously� an� assumption� of� the� model� because� no� interaction� term�is�included��Such�a�model�is�often�referred�to�as�an�additive model��As�was�men- tioned� previously,� in� this� model,� the� interaction� is� confounded� with� the� error� term�� Violation� of� the� additivity� assumption� results� in� the� test� of� factor� A� to� be� negatively� biased;�thus,�there�is�an�increased�probability�of�committing�a�Type�II�error��As�a�result,� if�H0�is�rejected,�then�we�are�confident�that�H0�is�really�false��If�H0�is�not�rejected,�then�
570 An Introduction to Statistical Concepts
our�interpretation�is�ambiguous�as�H0�may�or�may�not�be�really�true�(due�to�an�increased� probability�of�a�Type�II�error)��Here�you�would�not�know�whether�H0�was�true�or�not,�as� there�might�really�be�a�difference,�but�the�test�may�not�be�powerful�enough�to�detect�it�� Also,�the�power�of�the�test�of�factor�A�is�reduced�by�a�violation�of�the�additivity�assump- tion��The�assumption�may�be�tested�by�Tukey’s�(1949)�test�of�additivity�(see�Hays,�1988;� Kirk,�1982;�Timm,�2002),�which�generates�an�F�test�statistic�that�is�compared�to�the�critical� value�of�αF1,�[(J−1)�(K−1)−1]��If�the�test�is�not�statistically�significant,�then�the�model�is�addi- tive� and� the� assumption� has� been� met�� If� the� test� is� significant,� then� the� model� is� not� additive�and�the�assumption�has�not�been�met��A�summary�of�the�assumptions�and�the� effects�of�their�violation�for�this�model�is�presented�in�Table�16�6�
16.2.5 aNOVa Summary Table and expected Mean Squares
The�sources�of�variation�for�this�model�are�similar�to�those�of�the�regular�two-factor�model,� except�that�there�is�no�interaction�term��The�ANOVA�summary�table�is�shown�in�Table�16�7,� where� we� see� the� following� sources� of� variation:� A� (treatments),� B� (blocks),� residual,� and� total��The�test�of�block�differences�is�usually�of�no�real�interest��In�general,�we�expect�there� to�be�differences�between�the�blocks��From�the�table,�we�see�that�two�F�ratios�can�be�formed�
If�we�take�the�total�sum�of�squares�and�decompose�it,�we�have
SS SS SS SStotal A B res= + +
The� remaining� computations� are� determined� by� the� statistical� software�� The� degrees� of� freedom,�mean�squares,�and�F�ratios�are�also�shown�in�Table�16�7�
Table 16.6
Assumptions�and�Effects�of�Violations:�Two-Factor�Randomized�Block�ANOVA
Assumption Effect of Assumption Violation
Independence •�Increased�likelihood�of�a�Type�I�and/or�Type�II�error�in�F •��Affects�standard�errors�of�means�and�inferences�about�
those�means
Homogeneity�of�variance •�Small�effect�with�equal�or�nearly�equal�n’s •�Otherwise�effect�decreases�as�n�increases
Normality •�Minimal�effect�with�equal�or�nearly�equal�n’s
Sphericity •�Fairly�serious�effect
No�interaction�between�treatment�and�blocks •��Increased�likelihood�of�a�Type�II�error�for�the�test�of�factor�A� and�thus�reduced�power
Table 16.7
Two-Factor�Randomized�Block�Design�ANOVA� Summary�Table
Source SS df MS F
A SSA J�−�1 MSA MSA/MSres B SSB K�−�1 MSB MSB/MSres Residual SSres (J�−�1)�(K�−�1) MSres
Total SStotal N�−�1
571Hierarchical and Randomized Block Analysis of Variance Models
Earlier�in�our�discussion�of�the�two-factor�randomized�block�design,�we�mentioned�that� the�F�test�is�not�very�robust�to�violation�of�the�sphericity�assumption��We�again�recommend� the�following�sequential�procedure�be�used�in�the�test�of�factor�A��First,�perform�the�usual� F�test,�which�is�quite�liberal�in�terms�of�rejecting�H0�too�often,�where�the�degrees�of�free- dom�are�J�−�1�and�(J�−�1)(K�−�1)��If�H0�is�not�rejected,�then�stop��If�H0�is�rejected,�then�continue� with�step�2,�which�is�to�use�the�Geisser�and�Greenhouse�(1958)�conservative�F�test��For�the� model�we�are�considering�here,�the�degrees�of�freedom�for�the�F�critical�value�are�adjusted� to�be�1�and�K�−�1��If�H0�is�rejected,�then�stop��This�would�indicate�that�both�the�liberal�and� conservative� tests� reached� the� same�conclusion,� that�is,�to�reject�H0��If�H0�is� not�rejected,� then�the�two�tests�did�not�reach�the�same�conclusion,�and�a�further�test�should�be�under- taken��Thus,�in�step�3,�an�adjusted�F�test�is�conducted��The�adjustment�is�known�as�Box’s� (1954b)� correction� [the� Huynh� and� Feldt� (1970)� procedure]�� Here� the� degrees� of� freedom� are�equal�to�(J�−�1)ε�and�(J�−�1)(K�−�1)ε,�where�ε�is�the�correction�factor�(see�Kirk,�1982)��It�is� now�fairly�standard�for�the�major�statistical�software�to�conduct�the�Geisser-Greenhouse� and�Huynh-Feldt�tests�
Based�on�the�expected�mean�squares�(not�shown�here�for�simplicity),�the�residual�is� the�proper� error� term� for� the� fixed-,� random-,� and� mixed-effects� models�� Thus,� MSres� is� the� proper� error� term� for� every� version� of� this� model�� One� may� also� be� interested� in� an�assessment�of�the�effect�size�for�the�treatment�factor�A;�note�that�the�effect�size�of�the� blocking� factor� B� is� usually� not� of� interest�� As� in� previously� presented� ANOVA� models,� effect� size� measures� such� as� ω2� and� η2� should� be� considered�� Finally,� the� procedures� for� determining�CIs�and�power�are�the�same�as�in�previous�models�
16.2.6 Multiple Comparison procedures
If�the�null�hypothesis�for�either�the�A�(treatment)�or�B�(blocking)�factor�is�rejected�and�there� are�more�than�two�levels�of�the�factor�for�which�statistical�significance�was�found,�then�the� researcher�may�be�interested�in�which�means�or�combinations�of�means�are�different��This� could�be�assessed,�as�put�forth�in�previous�chapters,�by�the�use�of�some�MCP��In�general,� the�use�of�MCPs�outlined�in�Chapter�12�is�unchanged�as�long�as�the�sphericity�assumption� is�met��If�the�assumption�is�not�met,�then�MSres�is�not�the�appropriate�error�term,�and�the� alternatives�recommended�in�Chapter�15�should�be�considered�(see�Boik,�1981;�Kirk,�1982;� or�Maxwell,�1980)�
16.2.7 Methods of block Formation
There�are�different�methods�available�for�the�formation�of�blocks�depending�on�the�nature� of�the�blocking�variable��As�we�see,�the�methods�have�to�do�with�whether�the�blocking�fac- tor�is�an�ordinal�or�an�interval/ratio�variable�and�whether�the�blocking�factor�is�a�fixed�or� random�effect��This�discussion�borrows�heavily�from�the�work�of�Pingel�(1969)�in�defining� five�such�methods��The�first�method�is�the�predefined value blocking method,�where�the� blocking�factor�is�an�ordinal�variable��Here�the�researcher�specifies�K�different�population� values�of�the�blocking�variable��For�each�of�these�values�(i�e�,�a�fixed�effect),�individuals�are� randomly�assigned�to�the�levels�of�the�treatment�factor��Thus,�individuals�within�a�block� have� the� same� value� on� the� blocking� variable�� For� example,� if� class� rank� is� the� blocking� variable,�the�levels�might�be�the�top�third,�middle�third,�and�bottom�third�of�the�class�
The�second�method�is�the�predefined range blocking method,�where�the�blocking�factor� is�an�interval�or�ratio�variable��Here�the�researcher�specifies�K�mutually�exclusive�ranges� in�the�population�distribution�of�the�blocking�variable,�where�the�probability�of�obtaining�
572 An Introduction to Statistical Concepts
a�value�of�the�blocking�variable�in�each�range�may�be�specified�as�1/K��For�each�of�these� ranges�(i�e�,�a�fixed�effect),�individuals�are�randomly�assigned�to�the�levels�of�the�treatment� factor��Thus,�individuals�within�a�block�are�in�the�same�range�on�the�blocking�variable��For� example,�if�the�Graduate�Record�Exam-Verbal�(GRE-V)�score�is�the�blocking�variable,�the� levels�might�be�200–400,�401–600,�and�601–800�
The�third�method�is�the�sampled value blocking method,�where�the�blocking�variable�is�an� ordinal�variable��Here�the�researcher�randomly�samples�K�population�values�of�the�blocking� variable�(i�e�,�a�random�effect)��For�each�of�these�values,�individuals�are�randomly�assigned�to� the�levels�of�the�treatment�factor��Thus,�individuals�within�a�block�have�the�same�value�on�the� blocking�variable��For�example,�if�class�rank�is�again�the�blocking�variable,�only�this�time�mea- sured�in�10ths,�the�researcher�might�randomly�select�3�levels�from�the�population�of�10�levels�
The�fourth�method�is�the�sampled range blocking method,�where�the�blocking�variable�is� an�interval�or�ratio�variable��Here�the�researcher�randomly�samples�N�individuals�from�the� population,�such�that�N = JK,�where�K�is�the�number�of�blocks�desired�(i�e�,�a�fixed�effect)�and� J�is�the�number�of�treatment�groups��These�individuals�are�ranked�according�to�their�values� on� the� blocking� variable� from� 1� to� N�� The� first� block� consists� of� those� individuals� ranked� from�1�to�J,�the�second�block�of�those�ranked�from�J�+�1�to�2J,�and�so�on��Finally�individuals� within�a�block�are�randomly�assigned�to�the�J�treatment�groups��For�example,�consider�the� GRE-V�score�again�as�the�blocking�variable,�where�there�are�J�=�4�treatment�groups,�K�=�10� blocks,� and� thus� N = JK� =� 40� individuals�� The� top� four� ranked� individuals� on� the� GRE-V� exam� would� constitute� the� first� block,� and� they� would� be� randomly� assigned� to� the� four� groups��The�next�four�ranked�individuals�would�constitute�the�second�block,�and�so�on�
The� fifth� method� is� the� post hoc blocking method�� Here� the� researcher� has� already� designed�the�study�and�collected�the�data,�without�the�benefit�of�a�blocking�variable��After� the�fact,�a�blocking�variable�is�identified�and�incorporated�into�the�analysis��It�is�possible�to� implement�any�of�the�four�preceding�procedures�on�a�post�hoc�basis�
Based�on�the�research�of�Pingel�(1969),�some�statements�can�be�made�about�the�precision� of�these�blocking�methods�in�terms�of�a�reduction�in�residual�variability�as�well�as�better� estimation� of� the� treatment� effect�� In� general,� for� an� ordinal� blocking� variable,� the� pre- defined�value�blocking�method�is�more�precise�than�the�sampled�value�blocking�method�� Likewise,�for�an�interval�or�ratio�blocking�variable,�the�predefined�range�blocking�method� is�more�precise�than�the�sampled�range�blocking�method��Finally,�the�post�hoc�blocking� method�is�the�least�precise�of�the�methods�discussed��For�discussion�of�selecting�an�opti- mal�number�of�blocks,�we�suggest�you�consider�Feldt�(1958;�highly�recommended),�as�well� as�Myers�(1979),�Myers�and�Well�(1995),�and�Keppel�and�Wickens�(2004)��These�researchers� make�the�following�recommendations�about�the�optimal�number�of�blocks�(where�rxy�is�the� correlation�between�the�blocking�factor�X,�in�a�randomized�block�design,�and�the�depen- dent�variable�Y):�if�rxy�=��2,�then�use�five�blocks;�if�rxy�=��4,�then�use�four�blocks;�if�rxy�=��6,� then�use�three�blocks;�and�if�rxy�=��8,�then�use�two�blocks�
16.2.8 example
Let�us�consider�an�example�to�illustrate�the�procedures�in�this�section��The�data�are�shown�in� Table�16�8��The�blocking�factor�is�age�(i�e�,�20,�30,�40,�and�50�years�of�age),�the�treatment�factor� is�number�of�workouts�per�week�(i�e�,�1,�2,�3,�and�4),�and�the�dependent�variable�is�amount�of� weight�lost�during�the�1st�month��Presume�we�have�a�fixed-effects�model��Table�16�9�contains� the�resultant�ANOVA�summary�table�
The�test�statistics�are�both�compared�to�the�usual�F�test�critical�value�of��05F3,9�=�3�86�(from� Table�A�4),�so�that�both�main�effects�tests�are�statistically�significant��The�Geisser-Greenhouse�
573Hierarchical and Randomized Block Analysis of Variance Models
conservative�procedure�is�necessary�for�the�test�of�factor�A;�here�the�test�statistic�is�com- pared� to� the� critical� value� of� �05F1,3� =� 10�13,� which� is� also� significant�� The� two� procedures� both�yield�a�statistically�significant�result,�so�we�need�not�be�concerned�with�a�violation�of� the�sphericity�assumption�for�the�test�of�A��In�summary,�the�effects�of�amount�of�exercise� undertaken� and� age� on� amount� of� weight� lost� are� both� statistically� significant� at� the� �05� level�of�significance�
Next�we�need�to�test�the�additivity�assumption�using�Tukey’s�(1949)�test�of�additivity��The�F� test�statistic�is�equal�to�0�1010,�which�is�compared�to�the�critical�value�of��05F1,8�=�5�32�from�Table� A�4��The�test�is�nonsignificant,�so�the�model�is�additive�and�the�assumption�has�been�met�
As�an�example�of�a�MCP,�the�Tukey�HSD�procedure�is�used�to�test�for�the�equivalence�of� exercising�once�a�week�(j�=�1)�and�four�times�a�week�(j�=�4),�where�the�contrast�is�written�as� Y –
4��−�Y –
1���The�mean�amounts�of�weight�lost�for�these�groups�are�1�5000�for�the�once�a�week�pro- gram�and�7�7500�for�the�four�times�a�week�program��The�standard�error�is�computed�as�follows:
s MS J
ψ ’ res= = =
0 3958 4
. 0.3146
and�the�studentized�range�statistic�is�as�follows:
q Y Y s
= −
= −
=4 1 7 75 1 50
0 3146 19 8665. .
. . .
. ψ ’
The�critical�value�is�αq9,4�=�4�415�(from�Table�A�9)��The�test�statistic�exceeds�the�critical�value;� thus,� we� conclude� that� the� mean� amounts� of� weight� lost� for� groups� 1� (exercise� once� per� week)�and�4�(exercise�four�times�per�week)�are�statistically�significantly�different�at�the��05� level�(i�e�,�more�frequent�exercise�helps�one�to�lose�more�weight)�
Table 16.9
Two-Factor�Randomized�Block�Design� ANOVA�Summary�Table:�Exercise�Example
Source SS df MS F
A 21�6875 3 7�2292 18�2648a
B 110�1875 3 36�7292 92�7974a
Residual 3�5625 9 0�3958
Total 135�4375 15
a� �05F3,9�=�3�86�
Table 16.8
Data�for�the�Exercise�Example:�Two-Factor�Randomized�Block�Design
Age
Exercise Program 20 30 40 50 Row Means
1/week 3 2 1 0 1�5000 2/week 6 5 4 2 4�2500 3/week 10 8 7 6 7�7500 4/week 9 7 8 7 7�7500 Block�means 7�0000 5�5000 5�0000 3�7500 5�3125�(overall mean)
574 An Introduction to Statistical Concepts
16.3 Two-Factor Randomized Block Design for n > 1
For�two-factor�randomized�block�designs�with�more�than�one�observation�per�cell,�there� is�little�that�we�have�not�already�covered��First,�the�characteristics�are�exactly�the�same� as� with� the� n� =� 1� model,� with� the� obvious� exception� that� when� n� >� 1,� an� interaction� term�exists��Second,�the�layout�of�the�data,�the�model,�the�ANOVA�summary�table,�and� the�MCPs�are�the�same�as�in�the�regular�two-factor�model��Third,�the�assumptions�are� the�same�as�with�the�n�=�1�model,�except�the�assumption�of�additivity�is�not�necessary� because�an�interaction�term�exists��The�sphericity�assumption�is�required�for�those�tests� using� MSAB� as� the� error� term�� We� do� not� mean� to� minimize� the� importance� of� this� popular� model;� however,� there� really� is� no� additional� information� to� provide� beyond� what�we�have�already�presented��For�a�discussion�of�other�randomized�block�designs,� see�Kirk�(1982)�
16.4 Friedman Test
There�is�a�nonparametric�equivalent�to�the�two-factor�randomized�block�ANOVA�model��The� test�was�developed�by�Friedman�(1937)�and�is�based�on�mean�ranks��For�the�case�of�n�=�1,�the� procedure� is� precisely� the� same� as� the� Friedman� test� for� the� one-factor� repeated� measures� model�(see�Chapter�15)��For�the�case�of�n�>�1,�the�procedure�is�slightly�different��First,�all�of�the� scores�within�each�block�are�ranked�for�that�block��For�instance,�if�there�are�J�=�4�levels�of�factor� A�and�n�=�10�individuals�per�cell,�then�each�block’s�scores�would�be�ranked�from�1�to�40��From� this,�a�mean�ranking�can�be�determined�for�each�level�of�factor�A��The�null�hypothesis�tests� whether�the�mean�rankings�for�each�of�the�levels�of�A�are�equal��The�test�statistic�is�a�χ2,�which� is�compared�to�the�critical�value�of�α χ2J−1�(see�Table�A�3),�where�the�null�hypothesis�is�rejected�if� the�test�statistic�exceeds�the�critical�value�
In� the� case� of� tied� ranks,� either� the� available� ranks� can� be� averaged,� or� a� correction� factor� can� be� used� (see� Chapter� 15)�� You� may� also� recall� the� problem� with� small� n’s� in� terms�of�the�test�statistic�not�being�precisely�distributed�as�a�χ2��For�situations�where�J�<� 6�and�n�<�6,�consult�the�table�of�critical�values�in�Marascuilo�and�McSweeney�(1977,�Table� A-22,�p��521)��The�Friedman�test�assumes�that�the�population�distributions�have�the�same� shape�(although�not�necessarily�normal)�and�the�same�variability�and�that�the�dependent� measure� is� continuous�� For� alternative� nonparametric� procedures,� see� the� discussion� in� Chapter�15�
Various�MCPs�can�be�used�for�the�nonparametric�two-factor�randomized�block�model�� For�the�most�part,�these�MCPs�are�analogs�to�their�parametric�equivalents��In�the�case� of�planned�pairwise�comparisons,�one�may�use�multiple�matched-pair�Wilcoxon�tests� in�a�Bonferroni�form�(i�e�,�taking�the�number�of�contrasts�into�account�by�splitting�up� the�α�level)��Due�to�the�nature�of�planned�comparisons,�these�are�more�powerful�than� the� Friedman� test�� For� post� hoc� comparisons,� two� example� MCPs� are� the� Tukey� HSD� analog� for� pairwise� contrasts� and� the� Scheffé� analog� for� complex� contrasts�� For� addi- tional�discussion�about�the�use�of�MCPs�for�this�model,�see�Marascuilo�and�McSweeney� (1977)�� For� an� example� of� the� Friedman� test,� return� to� Chapter� 15�� Finally,� note� that� MCPs�are�not�usually�conducted�on�the�blocking�factor�as�they�are�rarely�of�interest�to� the�applied�researcher�
575Hierarchical and Randomized Block Analysis of Variance Models
16.5 Comparison of Various ANOVA Models
How�do�some�of�the�ANOVA�models�we�have�considered�compare�in�terms�of�power�and� precision?�Recall�again�that�power�is�defined�as�the�probability�of�rejecting�H0�when�H0�is� false,�and�precision�is�defined�as�a�measure�of�our�ability�to�obtain�good�estimates�of�the� treatment�effects��The�classic�literature�on�this�topic�revolves�around�the�correlation�between� the�dependent�variable�Y�and�the�concomitant�variable�X�(i�e�,�rxy),�where�the�concomitant� variable� can� be� either� a� covariate� or� a� blocking� factor�� First� let� us� compare� the� one-factor� ANOVA�and�one-factor�ANCOVA�models��If�rxy,�the�correlation�between�the�covariate�X�and� the�dependent�variable�Y,�is�not�statistically�significantly�different�from�0,�then�the�amount� of�unexplained�variation�will�be�the�same�in�the�two�models��Thus,�no�statistical�adjustment� will�be�made�on�the�group�means��In�this�situation,�the�ANOVA�model�is�more�powerful,� as�we�lose�one�degree�of�freedom�for�each�covariate�used�in�the�ANCOVA�model��If�rxy�is� significantly�different�from�0,�then�the�amount�of�unexplained�variation�will�be�smaller�in� the�ANCOVA�model�as�compared�to�the�ANOVA�model��Here�the�ANCOVA�model�is�more� powerful�and�is�more�precise�as�compared�to�the�ANOVA�model��Second,�compare�the�one- factor�ANOVA�and�two-factor�randomized�block�designs��If�rxy,�the�correlation�between�the� blocking� factor� X� and� the� dependent� variable� Y,� is� not� statistically� significantly� different� from�0,�then�the�blocking�factor�will�not�account�for�much�variability�in�the�dependent�vari- able��One�rule�of�thumb�states�that�if�rxy�<��2,�then�ignore�the�concomitant�variable�(whether� it�is�a�covariate�or�a�blocking�factor),�and�use�the�one-factor�ANOVA��Otherwise,�take�the� concomitant�variable�into�account�somehow,�either�as�a�covariate�or�blocking�factor�
How�should�we�take�the�concomitant�variable�into�account�if�it�correlates�with�the�depen- dent� variable� at� greater� than� �20� (i�e�,� rxy� >� �2)?� The� two� best� possibilities�are� the� analysis� of� covariance� design� (ANCOVA,� Chapter� 14)� and� the� randomized� block� ANOVA� design� (discussed�in�this�chapter)��That�is,�the�concomitant�variable�can�be�used�either�as�a�covari- ate�through�a�statistical�form�of�control�(i�e�,�ANCOVA)�or�as�a�blocking�factor�through�an� experimental�design�form�of�control�(i�e�,�randomized�block�ANOVA)��As�suggested�by�the� classic�work�of�Feldt�(1958),�if��2�<�rxy�<��4,�then�use�the�concomitant�variable�as�a�blocking� factor�in�a�randomized�block�design�as�it�is�the�most�powerful�and�precise�design��If�rxy�>��6,� then�use�the�concomitant�variable�as�a�covariate�in�an�ANCOVA�design�as�it�is�the�most� powerful� and� precise� design�� If� �4� <� rxy� <� �6,� then� the� randomized� block� and� ANCOVA� designs�are�about�equal�in�terms�of�power�and�precision�
However,� Maxwell,� Delaney,� and� Dill� (1984)� showed� that� the� correlation� between� the� covariate�and�dependent�variable�should�not�be�the�ultimate�criterion�in�deciding�whether� to� use� an� ANCOVA� or� a� randomized� block� design�� These� designs� differ� in� the� following� two�ways:�(a)�whether�the�concomitant�variable�is�treated�as�continuous�(ANCOVA)�or�cat- egorical� (randomized� block)� and� (b)� whether� individuals� are� assigned� to� groups� based� on� the�concomitant�variable�(randomized�blocks)�or�without�regard�to�the�concomitant�variable� (ANCOVA)��Thus,�the�Feldt�(1958)�comparison�of�these�particular�models�is�not�a�fair�one�in� that�the�models�differ�in�these�two�ways��The�ANCOVA�model�makes�full�use�of�the�informa- tion�contained�in�the�concomitant�variable,�whereas�in�the�randomized�block�model,�some� information�is�lost�due�to�the�categorization��In�examining�nine�different�models,�Maxwell� and�colleagues�suggest�that�rxy�should�not�be�the�sole�factor�in�the�choice�of�a�design�(given� that�rxy�is�at�least��3),�but�that�two�other�factors�be�considered��The�first�factor�is�whether�scores� on�the�concomitant�variable�are�available�prior�to�the�assignment�of�individuals�to�groups�� If�so,�power�will�be�increased�by�assigning�individuals�to�groups�based�on�the�concomitant� variable�(i�e�,�blocking)��The�second�factor�is�whether�X�(the�concomitant�variable)�and�Y�(the�
576 An Introduction to Statistical Concepts
dependent�variable)�are�linearly�related��If�so,�the�use�of�ANCOVA�with�a�continuous�con- comitant�variable�is�more�powerful�because�linearity�is�an�assumption�of�the�model�(Keppel� &�Wickens,�2004;�Myers�&�Well,�1995)��If�not,�either�the�concomitant�variable�should�be�used� as�a�blocking�variable,�or�some�sort�of�nonlinear�ANCOVA�model�should�be�used�
There�are�a�few�other�decision�criteria�you�may�want�to�consider�in�choosing�between� the�randomized�block�and�ANCOVA�designs��First,�in�some�situations,�blocking�may�be� difficult�to�carry�out��For�instance,�we�may�not�be�able�to�find�enough�homogeneous�indi- viduals�to�constitute�a�block��If�the�blocks�formed�are�not�very�homogeneous,�this�defeats� the�whole�purpose�of�blocking��Second,�the�interaction�of�the�independent�variable�and�the� concomitant�variable�may�be�an�important�effect�to�study��In�this�case,�use�the�randomized� block�design�with�multiple�individuals�per�cell��If�the�interaction�is�significant,�this�violates� the�assumption�of�homogeneity�of�regression�slopes�in�the�analysis�of�covariance�design,� but�does�not�violate�any�assumption�in�the�randomized�block�design�with�n�>�1��Third,�it� should�be�obvious�by�now�that�the�assumptions�of�the�ANCOVA�design�are�much�more� restrictive�than�in�the�randomized�block�design��Thus,�when�important�assumptions�are� likely�to�be�seriously�violated,�the�randomized�block�design�is�preferable�
There�are�other�alternative�designs�for�incorporating�the�concomitant�variable�as�a�pre- test,�such�as�an�ANOVA�on�gain�(the�difference�between�posttest�and�pretest),�or�a�mixed� (split-plot)� design� where� the� pretest� and� posttest� measures� are� treated� as� the� levels� of� a� repeated�factor��Based�on�the�research�of�Huck�and�McLean�(1975)�and�Jennings�(1988),�the� ANCOVA�model�is�generally�preferred�over�these�other�two�models��For�further�discus- sion,�see�Reichardt�(1979),�Huitema�(1980),�or�Kirk�(1982)�
16.6 SPSS
In�this�section,�we�examine�SPSS�for�the�models�presented�in�this�chapter��We�begin�with� the�two-factor�hierarchical�ANOVA�and�then�follow�with�the�two-factor�randomized�block� ANOVA�
Two-Factor Hierarchical ANOVA
To�conduct�a�two-factor�hierarchical�(or�nested)�ANOVA,�there�are�a�few�differences�from� other�ANOVA�models�we�have�considered�in�this�text��We�will�illustrate�computation�of� the�model�that�follows�the�point-and-click�method�as�we�have�done�in�previous�chapters�� It� is� important� to� note,� however,� that� while� SPSS� offers� limited� capability� for� estimating� hierarchical�ANOVA�models,�the�most�recent�versions�of�SPSS�offer�increasing�ability�to� generate�multilevel�regression�models,�and�readers�interested�in�more�complex�regression� models�are�referred�to�Heck,�Thomas,�and�Tabata�(2010)�
In�terms�of�the�form�of�the�data,�one�column�or�variable�indicates�the�levels�or�catego- ries�of�the�independent�variable�(i�e�,�the�fixed�factor),�one�column�indicates�the�levels�of� the�nested�factor,�and�the�one�variable�represents�the�outcome�or�the�dependent�variable�� Each�row�represents�one�individual,�indicating�the�level�or�group�of�the�nonnested�factor� (basal�or�whole�language,�in�our�example),�the�level�or�group�of�the�nested�factor�(teach- ers�1,�2,�3,�or�4),�and�their�score�on�the�dependent�variable��Thus,�we�have�three�columns� which�represent�the�nonnested�factor,�the�nested�factor,�and�the�scores,�as�shown�in�the� following�screenshot�
577Hierarchical and Randomized Block Analysis of Variance Models
�e nested factor is labeled “Teacher” where each value represents the child’s
classroom teacher.
�e dependent variable is “Score” and repersents the reading score.
The form of the data for the two-factor hierarchial ANOVA follows similarly to
previous ANOVA models. The non-nested factor is labeled “Approach” where
each value represents the reading approach to which they were assigned.
Step 1:�To�conduct�a�two-factor�hierarchical�ANOVA,�go�to�“Analyze”�in�the�top�pulldown� menu,�then�select�“General Linear Model,”�and�then�select�“Univariate.”�Following� the�screenshot�(step�1)�as�follows�produces�the�“Univariate”�dialog�box�
Two-factor hierarchical ANOVA:
Step 1 A
B C
578 An Introduction to Statistical Concepts
Step 2:� Click� the� dependent� variable� (e�g�,� reading� score)� and� move� it� into� the� “Dependent Variable”�box�by�clicking�the�arrow�button��Click�the�nonnested�fac- tor� (e�g�,� reading� approach;� this� is� a� fixed-effects� factor)� and� move� it� into� the�“Fixed Factors”�box�by�clicking�the�arrow�button��Click�the�nested�variable�(e�g�,�teacher;�this� is� a� random-effects� factor)� and� move� it� into� the�“Random Factors”� box� by� clicking� the�arrow�button�
Two-factor hierarchical ANOVA:
Step 2 Clicking on “Model”
will allow you to define the nested
factor.
Clicking on “Plots” will allow you to generate profile
plots.
Clicking on “Save” will allow you to
save various forms of residuals, among other
variables.
Clicking on “Options” will allow you to obtain a
number of other statistics (e.g., descriptive statistics,
effect size, power, homogeneity tests, and
multiple comparison procedures).
Select the nested factor from the list on the left and use the arrow to move it to the
“Random Factor(s)” box on the right.
Select the non-nested factor from the list on the left and
use the arrow to move it to the “Fixed Factor(s)” box on
the right.
Select the dependent variable from the list on the
left and use the arrow to move it to the “Dependent Variable” box on the right.
Univariate
Step 3a:� From� the� main� “Univariate”� dialog� box� (see� screenshot� step� 2),� click� on� “Model”� to� enact� the� “Univariate Model”� dialog� box�� From� the� “Univariate Model”�dialog�box,�click�the�“Custom”�radio�button�located�in�the�top�left�(see�screen- shot�step�3a)��We�will�now�define�a�main effect�for�reading�approach�(see�screenshot�step� 3a)��To�do�this,�click�the�“Build Terms”�toggle�menu�in�the�center�of�the�page�and�select� “Main�Effect.”� Click� the� nonnested� factor� (in� this� illustration,� “Approach”)� from� the� “Factors & Covariates”�list�on�the�left�and�move�to�the�“Model”�box�on�the�right�by� clicking�the�arrow�
579Hierarchical and Randomized Block Analysis of Variance Models
Two-factor hierarchical ANOVA:
Step 3a
Click the toggle menu for “Build Terms” to select
“Main Effects.” Select the non-
nested variable from the list on the left and use the arrow to move it to the
“Model” box on the right.
Step 3b:� We� will� now� define� an� interaction effect� for� reading� approach� by� teacher� (see� screenshot�step�3b)��To�do�this,�click�the�“Build Terms”�toggle�menu�in�the�center�of�the� page� and� select�“Interaction.”� Click� both� the� nonnested� factor� (e�g�,� “Approach”)� and� nested�factor�(e�g�,�“Teacher”)�from�the�“Factors & Covariates”�list�on�the�left�and�move� them�to�the�“Model”�box�on�the�right�by�clicking�the�arrow��The�interaction�term�is�neces- sary�to�trick�SPSS�into�computing�the�main�effect�of�B(A)�for�the�nested�factor�(which�SPSS� calls� “method*teacher,”� but� is� actually� “teacher”)� and� thus� generate� the� proper� ANOVA� summary�table��Thus,�the�model�should�not�include�a�main�effect�term�for�“Teacher�”
Two-factor hierarchical ANOVA:
Step 3b
Click the toggle menu for “Build Terms”
to select “Interaction.”
Select both the non-nested and nested
factors from the list on the left and use the arrow to move
them to the “Model” box on the right.
580 An Introduction to Statistical Concepts
Step 4:�From�the�“Univariate”�dialog�box�(see�screenshot�step�2),�clicking�on�“Post hoc” will�provide�the�option�to�select�post�hoc�MCPs�for�the�nonnested�factor��From�the�“Post Hoc Multiple Comparisons for Observed Means”�dialog�box,�click�on�the�name�of�the�non- nested�factor�in�the�“Factor(s)”�list�box�in�the�top�left�and�move�it�to�the�“Post Hoc Tests for”�box�in�the�top�right�by�clicking�on�the�arrow�key��Check�an�appropriate�MCP�for�your� situation�by�placing�a�checkmark�in�the�box�next�to�the�desired�MCP��In�this�example,�we�select� “Tukey�”�Click�on�“Continue”�to�return�to�the�original�dialog�box�
Two-factor hierarchical ANOVA:
Step 4
Select the non-nested factor of interest from the list on the left and
use the arrow to move it to the “Post Hoc Tests for” box on the right. MCPs for instances when
homogeneity of variance assumption is met.
MCPs for instances when homogeneity of variance assumption
is not met.
Step 5:�Clicking�on�“Options”�from�the�main�“Univariate”�dialog�box�(see�screenshot� step�2)�will�provide�the�option�to�select�such�information�as�“Descriptive Statistics,” “Estimates of effect size,” “Observed power,”�and�“Homogeneity tests”� (i�e�,�Levene’s�test)��Click�on�“Continue”�to�return�to�the�original�dialog�box��Note that if you are interested in an MCP for the nested factor�(although generally not of interest for this model),� post hoc MCPs are only available from the “Options” screen�� To� select� a� post� hoc� procedure,� click�on�“Compare main effects”�and�use�the�toggle�menu�to�reveal�the�Tukey LSD,� Bonferroni,�and�Sidak�procedures��However,� we�have�already� mentioned�that�MCPs� are�not�generally�of�interest�for�the�nested�factor�
It� is� important� to� note� that� Li� and� Lomax� (2011)� found� that� the� standard� errors� of� the� MCPs�for�the�nonnested�factor�in�SPSS�point-and-click�(PAC)�mode�are�not�correct��More� specifically,�SPSS�PAC�uses�MSwithin�as�the�error�term�in�computing�the�MCP�standard�error� rather�than�MSB(A)�as�the�error�term��There�is�no�way�to�generate�the�correct�results�solely� with� SPSS� PAC,� unless� hand� computations� using� the� correct� error� term� are� utilized� or� other�software�programs�(e�g�,�SPSS�syntax)�are�also�involved�
581Hierarchical and Randomized Block Analysis of Variance Models
Two-factor hierarchical ANOVA:
Step 5
Select from the list on the left those variables that you wish to display means for and use the arrow to
move them to the “Display Means for” box on the right.
While post hoc MCPs are usually not of
interest in random effects models, if you
wish to conduct a post hoc test, that selection
must be made from this screen using the “Compare main
effects” option, then select one of the three MCPs that are available from the
toggle menu under “Confidence interval adjustment” (i.e., LSD
Bonferroni, or Sidak).
Step 6:�From�the�“Univariate”�dialog�box�(screenshot�step�2),�click�on�“Save”�to�select� those�elements�you�want�to�save��Here�we�want�to�save�the�unstandardized�residuals�to� be�used�to�examine�the�extent�to�which�normality�and�independence�are�met��Thus,�place� a�checkmark�in�the�box�next�to�“Unstandardized.”�Click�“Continue”�to�return�to�the� main�“Univariate”� dialog� box�� From� the�“Univariate”� dialog� box,� click� on�“OK”� to� generate�the�output�
Two-factor hierarchical ANOVA:
Step 6
Interpreting the output:�Annotated�results�are�presented�in�Table�16�10�
582 An Introduction to Statistical Concepts
Table 16.10
Two-Factor�Hierarchical�ANOVA�SPSS�Results�for�the�Approaches�to�Reading�Example
Between-Subjects Factors
Value Label N
1.00 Basal 12Approach to reading
2.00 Whole language
12
1.00 Teacher B1 6 2.00 Teacher B2 6
3.00 Teacher B3 6
Teacher
4.00 Teacher B4 6
Descriptive Statistics
Dependent Variable: Reading Score
Approach to Reading Teacher Mean Std. Deviation N
Teacher B1 2.8333 1.72240 6
Teacher B2 3.8333 1.94079 6
Basal
Total 3.3333 1.82574 12 Teacher B3 10.0000 3.03315 6 Teacher B4 11.6667 2.80476 6
Whole language
Total 10.8333 2.91807 12 Teacher B1 2.8333 1.72240 6 Teacher B2 3.8333 1.94079 6
Teacher B3 10.0000 3.03315 6
Teacher B4 11.6667 2.80476 6
Total
Total 7.0833 4.51005 24
Levene's Test of Equality of Error Variancesa
Dependent Variable: Reading Score
F df1 df 2 Sig.
1.042 3 20 .396
�e table labeled “Descriptive Statistics” provides basic
descriptive statistics (means, standard
deviations, and sample sizes) for each
non-nested factor and nested factor
combination (or cell).
�e table labeled “Between- Subjects Factors” lists the
variable names and sample sizes for the non-nested factor (i.e. “Approach to reading”) and the nested factor
(i.e., “Teacher”).
�e F test (and associated p value) for Levene’s Test for Equality of Error Variances is reviewed to determine if equal variances can be assumed. In this case, we meet the assumption
(as p is greater than α). a Tests the null hypothesis that the error variance of the dependent variable is equal across groups.
583Hierarchical and Randomized Block Analysis of Variance Models
Table 16.10 (continued)
Two-Factor�Hierarchical�ANOVA�SPSS�Results�for�the�Approaches�to�Reading�Example
Tests of Between-Subjects Effects
Dependent Variable: Reading Score
Source
Type III Sum of Squares df
Mean Square F Sig.
Partial Eta
Squared Noncent. Parameter
Observed Powera
Hypothesis 1 .005 .991 1.000Intercept
Error 2 Hypothesis 1 .016 .968 .948Approach Error 2 Hypothesis 2
212.500
59.559
.952 .403 .087
212.500
59.559
1.905 .192Approach*
Teacher Error
1204.167
11.333 337.500
11.333 11.333
119.000 20
1204.167
5.667b
337.500
5.667b
5.667
5.950c
a Computed using alpha = .05. b MS(Approach * Teacher). c MS(Error).
Estimated Marginal Means 1. Grand Mean Dependent Variable: Reading Score
95% Confidence Interval Mean Std. Error Lower Bound Upper Bound
7.083a .498 6.045 8.122 a Based on modified population marginal mean.
Observed power tells whether our test is powerful enough to detect mean differences if they really exist. power of .948 is strong. The
probability of rejecting the null hypothesis, if it is really false, is about 95%.
Comparing p to α, we find a statistically
significant difference in approach to reading.
�is is an omnibus test. We will look at our MCPs to determine which mean
ratings differ.
Partial eta squared is one measure of effect size:
We can interpret this to say that approximately 97% of the variation in reading score is accounted for by the differences in reading approach.
337.500+11.333 337.500 =.968=
=
�e “Grand Mean” (in this case, 7.083) represents the overall reading score
mean, regardless of the reading approach or teacher. �e 95% CI represents the
CI of the grand mean.
SSapproach SSapproach+ SSapproach_error
η2p
η2p
(continued)
584 An Introduction to Statistical Concepts
Table 16.10 (continued)
Two-Factor�Hierarchical�ANOVA�SPSS�Results�for�the�Approaches�to�Reading�Example
2. Approach to Reading
Estimates Dependent Variable: Reading Score
95% Confidence Interval
Approach to Reading Mean Std. Error Lower Bound Upper Bound
Basal .704 1.864 4.802 Whole language
3.333a
10.833a .704 9.364 12.302 a Based on modified population marginal mean.
Pairwise Comparisons
Dependent Variable: Reading Score
95% Confidence Interval for Differencec
(I) Approach to Reading
(J) Approach to Reading
Mean Difference
(I – J) Std. Error Sig.c Lower Bound
Upper Bound
Basal Whole language
–7.500*,a,b .996 .000 –9.577 –5.423
Whole language
Basal 7.500*,a,b .996 .000 5.423 9.577
Based on estimated marginal means.
*The mean difference is significant at the .05 level.
a An estimate of the modified population marginal mean (I ). b An estimate of the modified population marginal mean (J ). c Adjustment for multiple comparisons: Bonferroni.
Univariate Tests
Dependent Variable: Reading Score
Sum of Squares df
Mean Square F Sig.
Partial eta Squared
Noncent. Parameter
Observed Powera
Contrast 1 56.723 .000 .739 56.723 1.000 Error
337.500 119.000 20
337.500 5.950
a Computed using alpha = .05.
The table for “Approach to
Reading” provides descriptive statistics
for each of the reading approaches.
In addition to means, the SE and 95% CI of
the means are reported.
“Mean Difference’’ is simply the difference between
the means of the two categories of our reading approach factor. For example, the mean difference of basal
reading and whole language is calculated as 3.333–
10.833 = –7.500.
“Sig.’’ is the observed p value for the results of the
Bonferroni post hoc MCP. There is a statistically
significant mean difference in reading scores between basal reading and whole language (p < .001). Note
the redundant results in the table. The comparison of basal and whole language (row 1) is the same as the
comparison of whole language and basal (row 2).
The error term represents the within cells source of variation.
The F tests the effect of approach to reading. This test is based on the linearly independent pairwise comparisons among the estimated marginal means.
585Hierarchical and Randomized Block Analysis of Variance Models
Table 16.10 (continued)
Two-Factor�Hierarchical�ANOVA�SPSS�Results�for�the�Approaches�to�Reading�Example
3. Approach to Reading * Teacher Dependent Variable: Reading Score
95% Confidence Interval Approach to Reading Teacher Mean Std. Error
Lower Bound
Upper Bound
Teacher B1 2.833 .996
Teacher B2 3.833 .996
Teacher B3 .a .
Basal
Teacher B4 .a . Teacher B1 .a .
Teacher B2 .a .
Teacher B3 10.000 .996
Whole language
Teacher B4 11.667 .996
.756 1.756
.
.
.
. 7.923 9.589
4.911
5.911
.
.
.
.
12.077
13.744
a
a This level combination of factors is not observed, thus the corresponding population marginal mean is not estimable.
The table for “Approach to Reading * Teacher”
provides descriptive statistics for each of the
approach-teacher combinations. In addition to
means, the SE and 95% CI of the means are reported.
Note the footnote in reference to the missing
mean values. This is because this is not a
completely crossed design (i.e., the teachers taught
only one reading approach).
Examining Assumptions for Two-Factor Hierarchical ANOVA
Normality
We�will�use�the�residuals�(which�were�requested�and�created�through�the�“Save”�option� mentioned�earlier)�to�examine�the�extent�to�which�normality�was�met�
The residuals are computed by subtracting the cell mean from each observation. For example, the
mean reading score for students assigned to teacher 1 who received the basal approach to reading was
2.833. The first student scored 1 on reading comprehension. Thus the residual for the first
person is 1.00 – 2.83 = –1.83. As we look at the raw data, we see one new variable has been added to our dataset labeled RES_1. These are the residuals and will be used to review the assumption of normality.
586 An Introduction to Statistical Concepts
Generating normality evidence:�As�described�in�earlier�ANOVA�chapters,�under- standing�the�distributional�shape,�specifically�whether�normality�is�a�reasonable�assump- tion,� is� important�� For� the� two-factor� hierarchical� ANOVA,� the� residuals� should� be� normally�distributed�
As�in�previous�chapters,�we�use�“Explore”�to�examine�whether�the�assumption�of�nor- mality�is�met��The�general�steps�for�accessing “Explore”�have�been�presented�in�previous� chapters�and�will�not�be�repeated�here��Click�the�residual�and�move�it�into�the�“Dependent List”�box�by�clicking�on�the�arrow�button��The�procedures�for�selecting�normality�statis- tics�were�presented�in�Chapter�6�and�remain�the�same�here:�Click�on�“Plots”�in�the�upper� right�corner��Place�a�checkmark�in�the�boxes�for�“Normality plots with tests”�and� also�for�“Histogram.”�Then�click�“Continue”�to�return�to�the�main�“Explore”�dialog� box�and�click�“OK”�to�generate�the�output�
Generating normality evidence
Select residuals from the list on the left and
use the arrow to move it to the
“Dependent List” box on the right. �en
click on “Plots.”
Interpreting normality evidence:�By�this�point,�we�have�had�a�substantial�amount� of�practice�in�interpreting�quite�a�range�of�normality�statistics�and�interpret�them�again�in� reference�to�the�hierarchical�ANOVA�model�assumption�of�normality�
587Hierarchical and Randomized Block Analysis of Variance Models
Descriptives
Mean
95% Confidence interval for mean
Lower bound Upper bound
5% Trimmed mean
Median
Variance Std. deviation Minimum
Maximum
Range Interquartile range
Skewness Kurtosis
.0000
–.9605 .9605
–.0648
–.3333
5.174 2.27462
–3.67
5.00
8.67 4.08
.284 –.693
Statistic Std. Error
.46431
.472
.918
Residual for score
The�skewness�statistic�of�the�residuals�is��284�and�kurtosis�is�−�693—both�being�within� the�range�of�an�absolute�value�of�2�0,�suggesting�some�evidence�of�normality�
As� suggested� by� the� skewness� statistic,� the� histogram� of� residuals� is� slightly� positively� skewed,�and�the�histogram�also�provides�a�visual�display�of�the�slightly�platykurtic�distribution�
Histogram
5
4
3
2
1
0 –4.00 –2.00 .00
Residual for score
Fr eq
ue nc
y
2.00 4.00 6.00
Mean = 8.33E – 17 Std. dev. = 2.275 N = 24
There�are�a�few�other�statistics�that�can�be�used�to�gauge�normality��The�formal�test�of� normality,�the�Shapiro–Wilk�(S–W)�test�(SW)�(Shapiro�&�Wilk,�1965),�provides�evidence�of� the�extent�to�which�our�sample�distribution�is�statistically�different�from�a�normal�distri- bution��The�output�for�the�S–W�test�is�presented�as�follows�and�suggests�that�our�sample�
588 An Introduction to Statistical Concepts
distribution�for�the�residual�is�not�statistically�significantly�different�than�what�would�be� expected�from�a�normal�distribution�as�the�p�value�is�greater�than�α�
Tests of Normality
Kolmogorov–Smirnova Shapiro–Wilk
Residual for score a Lilliefors significance correction. *This is alower bound of the true significance.
.123 24 .200* .960 24 .442
Statistic Statisticdf dfSig. Sig.
Quantile–quantile� (Q–Q)� plots� are� also� often� examined� to� determine� evidence� of� nor- mality,�where�quantiles�of�the�theoretical�normal�distribution�are�plotted�against�quantiles� of�the�sample�distribution��Points�that�fall�on�or�close�to�the�diagonal�line�suggest�evidence� of�normality��The�Q–Q�plot�of�residuals�shown�in�the�following�suggests�relative�normality�
–5.0
–2
–1
0
1
2
3
–2.5 0.0 Observed value
Ex pe
ct ed
n or
m al
Normal Q–Q plot of residual for score
2.5 5.0
Examination� of� the� following� boxplot� also� suggests� a� relatively� normal� distributional� shape�of�residuals�with�no�outliers�
Residual for score
–4.00
–2.00
.00
2.00
4.00
6.00
589Hierarchical and Randomized Block Analysis of Variance Models
Considering�the�forms�of�evidence�we�have�examined,�skewness�and�kurtosis�statistics,� the�S–W�test,�histogram,�the�Q–Q�plot,�and�the�boxplot,�all�suggest�normality�is�a�reason- able�assumption��We�can�be�reasonably�assured�we�have�met�the�assumption�of�normality�
Independence
The� last� assumption� to� test� for� is� independence�� As� we� have� seen� this� tested� in� other� designs,�we�do�not�consider�it�further�here�
Two-Factor Fixed-Effects Randomized Block ANOVA for n = 1
To� run� a� two-factor� fixed-effects� randomized� block� ANOVA� for� n� =� 1,� there� a� few� dif- ferences� from� the� regular� two-factor� fixed-effects� ANOVA� that� we� see� later� as� we� build� the� model� in� SPSS�� Additionally,� the� test� of� additivity� is� not� available� in� SPSS,� nor� are� the�adjusted�F�tests�(i�e�,�the�Geisser–Greenhouse�and�Huynh–Feldt�procedures)��All�other� ANOVA�procedures�that�you�are�familiar�with�will�operate�as�before�
In�terms�of�the�form�of�the�data,�it�looks�just�as�we�saw�with�the�two-factor�fixed-effects� ANOVA�with�the�exception�that�now�we�have�one�treatment�factor�and�one�blocking�vari- able��The�dataset�must�therefore�consist�of�three�variables�or�columns,�one�for�the�level�of� the� treatment� factor,� second� for� the� level� of� the� blocking� factor,� and� the� third� for� the� dependent�variable��Each�row�still�represents�one�individual,�indicating�the�levels�of�the� treatment� and� blocking� factors� to� which� the� individual� is� a� member,� and� their� score� on� the�dependent�variable��As�seen�in�the�following�screenshot,�for�a�two-factor�fixed-effects� randomized�block�ANOVA,�the�SPSS�data�are�in�the�form�of�two�columns�that�represent�the� group� values� (i�e�,� the� treatment� and� blocking� factors)� and� one� column� that� represents� the�scores�on�the�dependent�variable�
�e treatment factor is labeled “Program” where each value
represents the exercise program in which the individual participated
(e.g., 1 represents “1/week”). Thus there were four people assigned to
exercise once per week. �e blocking factor is labeled “Age” where 1 represents 20 years of age, 2 represents 30, 3 represents 40, and 4
represents 50. The dependent variable is
“Weightloss” and represents the amount of weight lost.
We see that one person from each of the four age groups was assigned to each exercise program. �e other
exercise programs (2, 3, and 4) follow this pattern as well.
Step 1:�To�conduct�a�two-factor�randomized�block�ANOVA�for�n�=�1,�go�to�“Analyze”� in� the� top� pulldown� menu,� then� select� “General Linear Model,”� and� then� select� “Univariate.”�Following�the�screenshot�(step�1)�as�follows�produces�the�“Univariate”� dialog�box�
590 An Introduction to Statistical Concepts
Two-factor randomized block ANOVA:
Step 1
A
B C
Step 2:�Click�the�dependent�variable�(e�g�,�weight�loss)�and�move�it�into�the�“Dependent Variable”�box�by�clicking�the�arrow�button��Click�the�treatment�factor�and�the�blocking� factor�and�move�them�into�the�“Fixed Factors”�box�by�clicking�the�arrow�button�
Two-factor randomized block ANOVA: Step 2
Clicking on “Model” will allow you to
define the blocking factor.
Clicking on “Plots” will allow you to generate profile
plots.
Clicking on “Save” will allow you to
save various forms of residuals, among other
variables.
Select the dependent variable from the list on the
left and use the arrow to move it to the “Dependent Variable” box on the right.
Select the treatment and blocking factors from the list
on the left and use the arrow to them move to the “Fixed Factor(s)” box
on the right. Clicking on “Options” will
allow you to obtain a number of other statistics (e.g., descriptive statistics,
effect size, power, and multiple comparsion
procedures).
Step 3:� From� the� main� “Univariate”� dialog� box� (see� screenshot� step� 2),� click� on� “Model”�to�enact�the�“Univariate Model”�dialog�box��From�the�“Univariate Model”� dialog�box,�click�the�“Custom”�radio�button�(see�screenshot�step�3)��We�will�now�define� the�effects�necessary�for�this�model,�a�main�effect�for�both�exercise�program�and�for�age�� We�will�not�define�an�interaction��To�do�this,�click�the�“Build Terms”�toggle�menu�in�the� center�of�the�page�and�select�“Main effect.”�Click�the�treatment�factor�(i�e�,�“Program”)� and�the�blocking�factor�(i�e�,�“Age”)�from�the�“Factors & Covariates”�list�on�the�left� and�move�them�to�the�“Model”�box�on�the�right�by�clicking�the�arrow��Thus,�the�model� should�not�include�an�interaction�effect�for�“Program�*�Age�”
591Hierarchical and Randomized Block Analysis of Variance Models
Two-factor randomized block ANOVA: Step 3
Click the toggle menu for “Build Terms” to select “Main effects.” Select the treatment and blocking factors from the list on the
left and use the arrow to move them to the “Model” box
on the right.
Step 4:�From�the�“Univariate”�dialog�box�(see�screenshot�step�2),�clicking�on�“Post Hoc”� will� provide� the� option� to� select� post� hoc� MCPs� for� both� factors�� From� the�“Post Hoc Multiple Comparisons for Observed Means”�dialog�box,�click�on�the�name� of�the�factors�(i�e�,�“Program”�and�“Age”)�in�the�“Factor(s)”�list�box�in�the�top�left�and� move�to�the�“Post Hoc Tests for”�box�in�the�top�right�by�clicking�on�the�arrow�key�� Check�an�appropriate�MCP�for�your�situation�by�placing�a�checkmark�in�the�box�next�to� the�desired�MCP��In�this�example,�we�select�“Tukey�”�Click�on�“Continue”�to�return�to�the� original�dialog�box�
Two-factor randomized block ANOVA: Step 4
Select the treatment and blocking factors from the list on the left and use the arrow to move them to the “Post Hoc Tests for” box on the right.
MCPs for instances when the homogeneity of variance assumption
is not met.
MCPs for instances when the homogeneity of
variance assumption is met.
592 An Introduction to Statistical Concepts
Step 5:�Clicking�on “Options”�from�the�main�“Univariate”�dialog�box�(see�screen- shot� step� 2)� will� provide� the� option� to� select� such� information� as “Descriptive Statistics,” “Estimates of effect size,”�and�“Observed power.”�Click�on� “Continue”�to�return�to�the�original�dialog�box�
Two-factor randomized block ANOVA: Step 5
Select from the list on the left those variables that you wish to display means for and use the arrow to move them to the “Display Means for” box on the right.
Step 6:� From� the� “Univariate”� dialog� box,� click� on� “Plots”� to� obtain� a� profile� plot� of� means�� Click� the� treatment� factor� (e�g�,� “Program”)� and� move� it� into� the�“Horizontal Axis” box�by�clicking�the�arrow�button��Click�the�blocking�factor�(e�g�,�“Age”)�and�move�it� into�the�“Separate Lines”�box�by�clicking�the�arrow�button�(see�screenshot�step�6a)��Then� click�on�“Add”�to�move�this�arrangement�into�the�“Plots”�box�at�the�bottom�of�the�dialog� box�(see�screenshot�step�6b)��Click�on�“Continue”�to�return�to�the�original�dialog�box�
Two-factor randomized block ANOVA: Step 6a
Select the treatment factor from the list on the left and use the arrow to move it to
the “Horizontal Axis” box on the right. select the blocking factor and move it to
the “Separate Lines” box on the right.
593Hierarchical and Randomized Block Analysis of Variance Models
Two-factor randomized block ANOVA: Step 6b
Then click “Add” to move the arrangement
into the “Plots” box at the Bottom.
Step 7:� From� the�“Univariate”� dialog� box� (see� screenshot� step� 2),� click� on�“Save”� to� select�those�elements�you�want�to�save��Here�we�save�the�unstandardized�residuals�to�use� later� to� examine� the� extent� to� which� normality� and� independence� are� met�� Thus,� place� a� checkmark� in� the� box� next� to� “Unstandardized.”� Click� “Continue”� to� return� to� the� main�“Univariate”�dialog�box��From�the�“Univariate”�dialog�box,�click�on�“OK”�to�return� and�generate�the�output�
Two-factor randomized block ANOVA: Step 7
Interpreting the output:�Annotated�results�are�presented�in�Table�16�11�
594 An Introduction to Statistical Concepts
Table 16.11
Two-Factor�Randomized�Block�ANOVA�SPSS�Results�for�the�Exercise�Program�Example
Between-Subjects Factors
Value Label N
1.00 1/week 2.00 2/week 3.00 3/week
Exercise program
4.00 4/week 1.00 20 years old 2.00 30 years old 3.00 40 years old
Age
4.00 50 years old
4
4
4
4 4
4 4 4
Descriptive Statistics Dependent Variable: Weight Loss
Exercise Program Age Mean Std. Deviation N
20 years old 30 years old 40 years old 50 years old
1/week
Total
20 years old 30 years old 40 years old 50 years old
2/week
Total
20 years old 30 years old 40 years old 50 years old
3/week
Total
20 years old 30 years old 40 years old 50 years old
4/week
Total
1
1 1 1 1 4
1 1 1 1 4
1 1 1
4
1 1 1 1 4
�e table labeled “Between- Subjects Factors” lists the variable names and sample sizes for the levels of treatment factor
(i.e., “Exercise program”) and the blocking factor (i.e., “Age”).
�e table labeled “Descriptive Statistics” provides basic
descriptive statistics (means, standard
deviations, and sample sizes) for each
treatment factor- blocking factor
combination. Because there was only one individual per age
group in each exercise program, there is no
within cells variation to calculate (and thus
missing values for the standard deviation).
20 years old
30 years old
40 years old
50 years old
Total
Total
3.0000 2.0000 1.0000
.0000 1.5000
6.0000 5.0000 4.0000 2.0000 4.2500
10.0000 8.0000 7.0000 6.0000 7.7500
9.0000 7.0000 8.0000 7.0000 7.7500
7.0000
5.5000
5.0000
3.7500
5.3125
.
.
.
. 1.29099
.
.
.
. 1.70783
.
.
.
. 1.70783
.
.
.
. .95743
3.16228
2.64575
3.16228
3.30404
3.00486
4
4
4
4
16
595Hierarchical and Randomized Block Analysis of Variance Models
Table 16.11 (continued)
Two-Factor�Randomized�Block�ANOVA�SPSS�Results�for�the�Exercise�Program�Example
Tests of Between-Subjects Effects
Dependent Variable: Weight Loss
Source
Type III Sum of Squares df
Mean Square F Sig.
Partial Eta
Squared Noncent.
Parameter Observed
Powerb
131.875a 6 55.526 .000 .974 333.158 1.000
451.563 1 1140.789 .000 .992 1140.789 1.000 110.187 3 92.789 .000 .969 278.368 1.000
21.688 3 18.263 .000 .859 54.789 .999
3.563 9
21.979
451.563 36.729
7.229
.396
587.000 16
Corrected Model Intercept Program Age
Error
Total Corrected Total
135.438 15
a R Squared = .974 (Adjusted R Squared = .956). b Computed using alpha = .05.
Observed power tells whether our test is powerful enough to detect mean differences if
they really exist. Power of 1.00 indicates maximum power, the probability of rejecting the
null hypothesis if it is really false is about 1.
Comparing p to α, we find a statistically significant difference in weight loss based for both exercise program and age group.�ese are
omnibus tests.We will look at post hoc tests to determine which
exercise programs and age groups statistically differ on weight loss.
Partial eta squared is one measure of effect size:
We can interpret this to say that approximately 97% of the variation in weight loss is accounted for by the exercise program.
SSprogram SSprogram + SSerror
110.187 + 3.563 110.187
= .969
η2 =
η2 =
Estimated Marginal Means 1. Grand Mean
Dependent Variable: Weight Loss
95% Confidence Interval
Mean Std. Error Lower Bound Upper Bound
5.313 .157 4.957 5.668
The “Grand Mean’’ (in this case, 5.313) represents the overall mean, regardless of
the exercise program or age. The 95% CI represents the CI of the
grand mean.
(continued)
596 An Introduction to Statistical Concepts
Table 16.11 (continued)
Two-Factor�Randomized�Block�ANOVA�SPSS�Results�for�the�Exercise�Program�Example
Post Hoc Tests Exercise Program
Weight Loss Tukey HSD
95% Confidence Interval
(I) Exercise Program
(J) Exercise Program
Mean Difference
(I – J ) Std.
Error Sig. Lower Bound
Upper Bound
.44488 .001 –1.3612
.44488 .000 –4.8612
.44488 .000 –4.8612
.44488 .001 4.1388
.44488 .000 –2.1112
.44488 .000 –2.1112
.44488 .000 7.6388
.44488 .000 4.8888
.44488 1.000 1.3888
.44488 .000 7.6388
.44488 .000 4.8888
1/week
2/week
3/week
4/week
2/week
3/week
4/week
1/week
3/week
4/week
1/week
2/week
4/week
1/week
2/week
3/week
–2.7500*
–6.2500*
–6.2500*
2.7500*
–3.5000*
–3.5000*
6.2500*
3.5000*
.0000
6.2500*
3.5000*
.0000 .44488 1.000
–4.1388
–7.6388
–7.6388
1.3612
–4.8888
–4.8888
4.8612
2.1112
–1.3888
4.8612
2.1112
–1.3888 1.3888
Based on observed means. The error term is mean square(error) = .396. *The mean difference is significant at the .05 level.
“Mean Difference” is simply the difference between the means of the categories of our program factor. For example, the mean difference of exercising once per week and exercising twice per week is calculated as 1.500 – 4.250 = –2.750.
“Sig.” denotes the observed p value and provides the results of the Tukey post hoc procedure. There is a statistically significant mean difference in weight loss for all exercise programs except for exercising 3 vs. 4 times per week ( p = 1.000). Note there are redundant results presented in the table. The comparison of exercising 1/week vs. 2/week (row 1) is the same as the comparison of 2/week vs. 1/week (row 4).
Multiple Comparisons
2. Exercise Program Dependent Variable: Weight Loss
95% Confidence Interval Exercise Program Mean Std. Error Lower Bound Upper Bound
1/week 2/week 3/week 4/week
1.500 4.250 7.750 7.750
.315
.315
.315
.315
.788 3.538 7.038 7.038
2.212 4.962 8.462 8.462
3. Age Dependent Variable: Weight Loss
95% Confidence Interval
Age Mean Std. Error Lower Bound Upper Bound
20 years old 7.000 7.712 30 years old 5.500 6.212 40 years old 5.000 5.712 50 years old 3.750
.315
.315
.315
.315
6.288 4.788 4.288 3.038 4.462
The table for “Exercise Program” provides descriptive statistics for each of the programs. In addition to means, the SE and 95% CI of the means are reported.
The table for “Age” provides descriptive statistics for each of the age groups. In addition to means, the SE and 95% CI of the means are reported.
597Hierarchical and Randomized Block Analysis of Variance Models
Table 16.11 (continued)
Two-Factor�Randomized�Block�ANOVA�SPSS�Results�for�the�Exercise�Program�Example
Homogeneous Subsets Weight Loss Tukey HSDa,b
Subset Exercise Program N 1 2 3
4 4 4 4
1/week 2/week 3/week 4/week Sig.
1.5000
1.000
4.2500
1.000
7.7500 7.7500 1.000
Means for groups in homogeneous subsets are displayed. Based on observed means. �e error term is mean square(error) = .396. a Uses harmonic mean sample size = 4.000. b Alpha = .05.
“Homogenous Subsets”
provides a visual representation of the MCP. For each subset,
the means that are printed are homogeneous, or not significantly different.
For example, in subset 1 the mean weight loss for exercising
once per week (regardless of age group) is 1.50. �is is
statistically significantly different than the mean weight loss for
exercising two, three, or four times per week (as reflected by empty
cells in row 1). Similar interpretations are made for
contrasts involving exercising two, three, and four times per week.
Age
Multiple Comparisons Weight Loss Tukey HSD
95% Confidence Interval (I) Age (J) Age
Mean Difference (I – J) Std. Error Sig. Lower Bound Upper Bound
30 years old .44488 .034
40 years old .44488 .007
20 years old
50 years old .44488 .000 20 years old .44488 .034 40 years old .44488 .685
30 years old
50 years old .44488 .015 20 years old .44488 .007 30 years old .44488 .685
40 years old
50 years old .44488 .080 20 years old .44488 .000
30 years old .44488 .015
50 years old
40 years old
1.5000*
2.0000*
3.2500* –1.5000*
.5000 1.7500*
–2.0000 –.5000 1.2500
–3.2500*
–1.7500*
–1.2500 .44488 .080
.1112
.6112
1.8612 –2.8888 –.8888
.3612 –3.3888 –1.8888 –.1388
–4.6388
–3.1388
–2.6388
2.8888
3.3888
4.6388 –.1112 1.8888 3.1388 –.6112
.8888 2.6388
–1.8612
–.3612
.1388
Based on observed means. �e error term is mean square(error) = .396. *�e mean difference is significant at the .05 level.
“Mean difference” is simply the difference between the means of the age groups (i.e., the blocking factor). For example, the mean
weight loss difference of 20 – 30 year olds is calculated as 7.000 – 5.500 = 1.5000.
“Sig.” denotes the observed p value and provides the results of the Tukey post hoc procedure.�ere is a statistically significant mean difference in weight loss for: • 20 and 30 year olds (p = .034) • 20 and 40 year olds (p = .007) • 20 and 50 year olds (p < .001) • 30 and 50 year olds (p = .015) Note there are redundant results presented in the table. �e comparison of 20–3 0 year olds is the same as the comparison of 30–20 year olds, and so forth.
(continued)
598 An Introduction to Statistical Concepts
Table 16.11 (continued)
Two-Factor�Randomized�Block�ANOVA�SPSS�Results�for�the�Exercise�Program�Example
Homogeneous Subsets
Weight Loss Tukey HSDa,b
Subset Age N 1 2 3
4 3.7500 4 5.0000 5.0000 4 5.5000 4 7.0000
50 years old 40 years old 30 years old 20 years old Sig. .080 .685 1.000
Means for groups in homogeneous subsets are displayed. Based on observed means.
�e error term is mean square(error) = .396. a Uses harmonic mean sample size = 4.000. b Alpha = .05.
“Homogenous Subsets” provides a visual representation
of the MCP. For each subset, the means that are printed are
homogeneous, or not significantly different. For
example, in subset 1 the mean weight loss for 50 year olds
(regardless of exercise program) is 3.750. �is is statistically
significantly different than the mean weight loss for individuals
in the 30 and 20 year old age groups (as they are not printed
in subset 1).
Estimated marginal means of weight loss
Es tim
at ed
m ar
gi na
l m ea
ns
Exercise program
Age
The “profile plot’’ is a graph of the mean weight
loss by exercise program and age. We see that, across all age
groups, the greatest weight loss was for individuals who
exercised either 3 or 4 weeks.
10.00
8.00
6.00
4.00
2.00
.00
–2.00
1/week 2/week 3/week 4/week
20 years old 30 years old 40 years old 50 years old
Examining Assumptions for Two-Factor Randomized Block ANOVA
Normality
We�use�the�residuals�(which�were�requested�and�created�through�the�“Save”�option�when� generating�our�model)�to�examine�the�extent�to�which�normality�was�met�
599Hierarchical and Randomized Block Analysis of Variance Models
Generating normality evidence:�As�shown�in�previous�ANOVA�chapters,�under- standing�the�distributional�shape,�specifically�the�extent�to�which�normality�is�a�reason- able�assumption,�is�important��For�the�two-factor�randomized�block�ANOVA,�the�residuals� should�be�normally�distributed��Because�the�steps�for�generating�normality�evidence�were� presented� previously� in� the� chapter� for� the� two-factor� hierarchical� ANOVA� model,� they� will�not�be�reiterated�here�
Interpreting normality evidence:�By�this�point,�we�have�had�a�substantial�amount� of� practice� in� interpreting� quite� a� range� of� normality� statistics�� Here� we� interpret� them� again,�only�now�in�reference�to�the�two-factor�randomized�block�ANOVA�model�
Descriptives
Residual for weight loss Mean 95% Con dence interval
5% Trimmed mean Median Variance Std. deviation Minimum Maximum Range Interquartile range Skewness Kurtosis
Std. ErrorStatistic
Lower bound Upper bound
.0000 –.2597
.2597
.0069
.0625 .238
.48734 –.94
.81 1.75
–.154 –.496
.12183
.564 1.091
.87
for mean
The�skewness�statistic�of�the�residuals�is�−�154�and�kurtosis�is�−�496—both�being�within� the�range�of�an�absolute�value�of�2�0,�suggesting�some�evidence�of�normality�
As� suggested� by� the� skewness� statistic,� the� histogram� of� residuals� is� slightly� negatively� skewed,�and�the�histogram�also�provides�a�visual�display�of�the�slightly�platykurtic�distribution�
Histogram
Mean = –2.36E – 16 Std. dev. = .487 N = 16
0 –1.00 –.50 .00 .50 1.00
Residual for weight loss
Fr eq
ue nc
y
1
2
3
4
5
600 An Introduction to Statistical Concepts
There�are�a�few�other�statistics�that�can�be�used�to�gauge�normality��The�formal�test�of� normality,� the� S–W� test� (SW)� (Shapiro� &� Wilk,� 1965),� provides� evidence� of� the� extent� to� which� our� sample� distribution� is� statistically� different� from� a� normal� distribution�� The� output�for�the�S–W�test�is�presented�as�follows�and�suggests�that�our�sample�distribution� for� the� residuals� is� not� statistically� significantly� different� than� what� would� be� expected� from�a�normal�distribution�as�the�p�value�is�greater�than�α�
Tests of Normality
Kolmogorov–Smirnova Shapiro–Wilk
Residual for weight loss
a Lilliefors significance correction.
Statistic Statistic df Sig.df Sig.
.136 16 .200* .965 16 .757
*This is a lower bound of the true significance.
Q–Q�plots�are�also�often�examined�to�determine�evidence�of�normality�where�quantiles� of�the�theoretical�normal�distribution�are�plotted�against�quantiles�of�the�sample�distribu- tion�� Points� that� fall� on� or� close� to� the� diagonal� line� suggest� evidence� of� normality�� The� Q–Q�plot�of�residuals�shown�in�the�following�suggests�relative�normality�
2
1
0
–1
–2
–3
–1.5 –1.0 –0.5 0.0 0.5 1.0 Observed value
Normal Q–Q plot of residual for weight loss
Ex pe
ct ed
n or
m al
Examination� of� the� following� boxplot� also� suggests� a� relatively� normal� distributional� shape�of�residuals�with�no�outliers�
601Hierarchical and Randomized Block Analysis of Variance Models
Residual for weight loss
1.00
.50
.00
–.50
–1.00
Considering�the�forms�of�evidence�we�have�examined,�skewness�and�kurtosis�statis- tics,�the�S–W�test,�histogram,�the�Q–Q�plot,�and�the�boxplot,�all�suggest�normality�is�a� reasonable�assumption�� We�can�be�reasonably�assured�we�have�met�the�assumption�of� normality�
Independence
The�only�assumption�we�have�not�tested�for�yet�is�independence��As�we�discussed�in�ref- erence�to�the�one-way�ANOVA,�if�subjects�have�been�randomly�assigned�to�conditions�(in� other�words,�the�different�levels�of�the�treatment�factor�in�a�two-factor�randomized�block� ANOVA),�the�assumption�of�independence�has�likely�been�met��In�our�example,�individu- als�were�randomly�assigned�to�exercise�program,�and,�thus,�the�assumption�of�indepen- dence�was�met��However,�we�often�use�independent�variables�that�do�not�allow�random� assignment��We�can�plot�residuals�against�levels�of�our�treatment�factor�using�a�scatterplot� to�see�whether�or�not�there�are�patterns�in�the�data�and�thereby�provide�an�indication�of� whether�we�have�met�this�assumption�
Please� note� that� some� researchers� do� not� believe� that� the� assumption� of� indepen- dence�can�be�tested��If�there�is�not�random�assignment�to�groups,�then�these�researchers� believe�this�assumption�has�been�violated—period��The�plot�that�we�generate�will�give� us�a�general�idea�of�patterns,�however,�in�situations�where�random�assignment�was�not� performed�
Generating the scatterplot:�The�general�steps�for�generating�a�simple�scatterplot� through� “Scatter/Dot”� have� been� presented� in� previous� chapters� (e�g�,� Chapter� 10),� and�they�will�not�be�reiterated�here��From�the�“Simple Scatterplot”�dialog�screen,� click�the�residual�variable�and�move�it�into�the�“Y Axis”�box�by�clicking�on�the�arrow�� Click� the� independent� variable� that� we� wish� to� display� (e�g�,� “Exercise� Program”)� and� move�it�into�the�“X Axis”�box�by�clicking�on�the�arrow��Then�click�“OK.”
602 An Introduction to Statistical Concepts
Interpreting independence evidence:�In�examining�the�scatterplot�for�evidence� of�independence,�the�points�should�fall�relatively�randomly�above�and�below�a�horizon- tal�line�at�0��(You�may�recall�in�Chapter�11�that�we�added�a�reference�line�to�the�graph� using� Chart� Editor�� To� add� a� reference� line,� double� click� on� the� graph� in� the� output� to� activate� the� chart� editor�� Select�“Options”� in� the� top� pulldown� menu,� then�“Y axis reference line.” This� will� bring� up� the�“Properties”� dialog� box�� Change� the� value� of� the� position� to� be� “0�”� Then� click� on� “Apply”� and� “Close”� to� generate� the� graph�with�a�horizontal�line�at�0�)
In�this�example,�our�scatterplot�for�exercise�program�by�residual�generally�suggests�evi- dence�of�independence�with�a�relatively�random�display�of�residuals�above�and�below�the� horizontal�line�at�0��Thus,�had�we�not�met�the�assumption�of�independence�through�ran- dom�assignment�of�cases�to�groups,�this�would�have�provided�evidence�that�independence� was�a�reasonable�assumption�
603Hierarchical and Randomized Block Analysis of Variance Models
1.00
.50
.00
Re si
du al
fo r w
ei gh
t l os
s
–.50
–1.00
1.00 1.50 2.00 2.50 3.00 Exercise program
3.50 4.00
Two-Factor Fixed-Effects Randomized Block ANOVA n > 1
To� run� a� two-factor� randomized� block� ANOVA� for� n� >� 1,� the� procedures� are� exactly� the� same� as� with� the� regular� two-factor� ANOVA�� However,� the� adjusted� F� tests� are� not� available�
Friedman Test
Lastly,�the�Friedman�test�can�be�run�as�previously�described�in�Chapter�15�
Post Hoc Power for Two-Factor Randomized Block ANOVA Using G*Power
G*Power�provides�power�calculations�for�the�two-factor�randomized�block�ANOVA�model�� In�G*Power,�just�treat�this�design�as�if�it�were�a�regular�two-factor�ANOVA�model�
16.7 Template and APA-Style Write-Up
Finally,� here� is� an� example� paragraph� just� for� the� results� of� the� two-factor� hierarchical� ANOVA�design�(feel�free�to�write�a�similar�paragraph�for�the�two-factor�randomized�block� ANOVA�example)��Recall�that�our�graduate�research�assistant,�Marie,�was�assisting�a�read- ing�faculty�member,�JoAnn��JoAnn�wanted�to�know�the�following:�if�there�is�a�mean�dif- ference�in�reading�based�on�the�approach�to�reading�and�if�there�is�a�mean�difference�in�
604 An Introduction to Statistical Concepts
reading�based�on�teacher��The�research�questions�presented�to�JoAnn�from�Marie�include� the�following:
•� Is there a mean difference in reading based on approach to reading? •� Is there a mean difference in reading based on teacher?
Marie�then�assisted�JoAnn�in�generating�a�two-factor�hierarchical�ANOVA�as�the�test�of� inference,�and�a�template�for�writing�the�research�questions�for�this�design�is�presented� as�follows��As�we�noted�in�previous�chapters,�it�is�important�to�ensure�the�reader�under- stands�the�levels�of�the�factor(s)��This�may�be�done�parenthetically�in�the�actual�research� question,�as�an�operational�definition,�or�specified�within�the�methods�section:
•� Is there a mean difference in [dependent variable] based on [nonnested factor]?
•� Is there a mean difference in [dependent variable] based on [nested factor]?
It�may�be�helpful�to�preface�the�results�of�the�two-factor�hierarchical�ANOVA�with�infor- mation�on�an�examination�of�the�extent�to�which�the�assumptions�were�met��The�assump- tions�include�(a)�homogeneity�of�variance�and�(b)�normality�
A two-factor hierarchical ANOVA was conducted. The nonrepeated factor was approach to reading (basal or whole language) and the nested factor was teacher (four teachers). The null hypotheses tested included the following: (1) the mean reading score was equal for each of the reading approaches, and (2) the mean reading score for each teacher was equal.
The data were screened for missingness and violation of assump- tions prior to analysis. There were no missing data. The assumption of homogeneity of variance was met (F(3, 20) = 1.042, p = .396). The assumption of normality was tested via examination of the residuals. Review of the S–W test (SW = .960, df = 24, p = .442) and skewness (.284) and kurtosis (−.693) statistics suggested that normality was a reasonable assumption. The boxplot displayed a relatively normal dis- tributional shape (with no outliers) of the residuals. The Q–Q plot and histogram suggested normality was tenable.
Here�is�an�APA-style�example�paragraph�of�results�for�the�two-factor�hierarchical�ANOVA� (remember� that� this� will� be� prefaced� by� the� previous� paragraph� reporting� the� extent� to� which�the�assumptions�of�the�test�were�met)�
From Table 15.10, the results for the two-factor hierarchical ANOVA indicate the following:
1. A statistically significant main effect for approach to reading (Fapproach = 59.559, df = 1, 2, p = .016)
2. A nonstatistically significant main effect for teacher (Fteacher = .952, df = 2, 20, p = .403)
605Hierarchical and Randomized Block Analysis of Variance Models
Effect size was rather large for the effect of approach to read- ing (partial η2approach = .968), with high observed power (.948), but expectedly less so for the nonsignificant teacher effect (par- tial η2teacher = .087, power = .192). The results of this study pro- vide evidence to suggest that reading comprehension scores are significantly higher for students taught by the whole language method (M = 10.833, SE = .704) as compared to the basal method (M = 3.333, SE = .704). The results also suggest that mean scores for reading are comparable for children regardless of the teacher who instructed them.
16.8 Summary
In�this�chapter,�models�involving�nested�and�blocking�factors�for�the�two-factor�case�were� considered��Three�different�models�were�examined;�these�included�the�two-factor�hierar- chical�design,�the�two-factor�randomized�block�design�with�one�observation�per�cell,�and� the�two-factor�randomized�block�design�with�multiple�observations�per�cell��Included�for� each�design�were�the�usual�topics�of�model�characteristics,�the�layout�of�the�data,�the�linear� model,�assumptions�of�the�model�and�dealing�with�their�violation,�the�ANOVA�summary� table� and� expected� mean� squares,� and� MCPs�� Also� included� for� particular� designs� was� a� discussion� of� the� compound� symmetry/sphericity� assumption� and� the� Friedman� test� based� on� ranks�� We� concluded� with� a� comparison� of� various� ANOVA� models� on� preci- sion� and� power�� At� this� point,� you� should� have� met� the� following� objectives:� (a)� be� able� to� understand� the� characteristics� and� concepts� underlying� hierarchical� and� randomized� block�ANOVA�models,�(b)�be�able�to�determine�and�interpret�the�results�of�hierarchical�and� randomized� block� ANOVA� models,� (c)� be� able� to� understand� and� evaluate� the� assump- tions�of�hierarchical�and�randomized�block�ANOVA�models,�and�(d)�be�able�to�compare� different� ANOVA� models� and� select� an� appropriate� model�� This� chapter� concludes� our� extended� discussion� of� ANOVA� models�� In� the� remaining� three� chapters� of� the� text,� we� discuss� regression� models� where� the� dependent� variable� is� predicted� by� one� or� more� independent�variables�or�predictors�
Problems
Conceptual problems
16.1� A�researcher�wants�to�know�if�the�number�of�professional�development�courses�that� a� teacher� completes� differs� based� on� the� format� that� the� professional� development� is�offered�(online,�mixed�mode,�face-to-face)��The�researcher�randomly�samples�100� teachers� employed� in� the� district�� Believing� that� years� of� teaching� experience� may� be�a�concomitant�variable,�the�researcher�ranks�the�teachers�on�years�of�experience� and� places� them� in� categories� that� represent� 5-year� intervals�� The� researcher� then� randomly�selects�4�years�of�experience�blocks��The�teachers�within�those�blocks�are� then�randomly�assigned�to�professional�development�format��Which�of�the�following� methods�of�blocking�is�employed�here?
606 An Introduction to Statistical Concepts
� a�� Predefined�value�blocking � b�� Predefined�range�blocking � c�� Sampled�value�blocking � d�� Sampled�range�blocking 16.2� To�study�the�effectiveness�of�three�spelling�methods,�45�subjects�are�randomly�selected�
from�the�fourth�graders�in�a�particular�elementary�school��Based�on�the�order�of�their� IQ�scores,�subjects�are�grouped�into�IQ�groups�(low�=�75–99,�average�=�100–115,�high�=� 116–130),�15�in�each�group��Subjects�in�each�group�are�randomly�assigned�to�one�of�the� three� methods� of� spelling,� five� each�� Which� of� the� following� methods� of� blocking� is� employed�here?
� a�� Predefined�value�blocking � b�� Predefined�range�blocking � c�� Sampled�value�blocking � d�� Sampled�range�blocking 16.3� A�researcher�is�examining�preschoolers’�knowledge�of�number�identification��Fifty�
preschoolers� are� grouped� based� on� socioeconomic� status� (low,� moderate,� high)�� Within� each� SES� group,� students� are� randomly� assigned� to� one� of� two� treatment� groups:� one� which� incorporates� numbers� through� individual,� small� group,� and� whole�group�work�with�manipulatives,�music,�and�art;�and�a�second�which�incorpo- rates�numbers�through�whole�group�study�only��Which�of�the�following�methods�of� blocking�is�employed�here?
� a�� Predefined�value�blocking � b�� Predefined�range�blocking � c�� Sampled�value�blocking � d�� Sampled�range�blocking 16.4� If�three�teachers�employ�method�A�and�three�other�teachers�employ�method�B,�then�
which�one�of�the�following�is�suggested? � a�� Teachers�are�nested�within�method� � b�� Teachers�are�crossed�with�methods� � c�� Methods�are�nested�within�teacher� � d�� Cannot�be�determined� 16.5� The�interaction�of�factors�A�and�B�can�be�assessed�only�if�which�one�of�the�following�
occurs? � a�� Both�factors�are�fixed� � b�� Both�factors�are�random� � c�� Factor�A�is�nested�within�factor�B� � d�� Factors�A�and�B�are�crossed� 16.6� In�a�two-factor�design,�factor�A�is�nested�within�factor�B�for�which�one�of�the�following? � a�� At�each�level�of�A,�each�level�of�B�appears� � b�� At�each�level�of�A,�unique�levels�of�B�appear� � c�� At�each�level�of�B,�unique�levels�of�A�appear� � d�� Cannot�be�determined�
607Hierarchical and Randomized Block Analysis of Variance Models
16.7� Five� teachers� use� an� experimental� method� of� teaching� statistics,� and� five� other� teachers�use�the�traditional�method��If�factor�M�is�method�of�teaching,�and�factor�T� is�teacher,�this�design�can�be�denoted�by�which�one�of�the�following?
� a�� T(M) � b�� T��M � c�� M��T � d�� M(T) 16.8� If�factor�C�is�nested�within�factors�A�and�B,�this�is�denoted�as�AB(C)��True�or�false? 16.9� A�design�in�which�all�levels�of�each�factor�are�found�in�combination�with�each�level�
of�every�other�factor�is�necessarily�a�nested�design��True�or�false? 16.10� To� determine� if� counseling� method� E� is� uniformly� superior� to� method� C� for� the�
population� of� counselors,� from� which� random� samples� are� taken� to� conduct� a� study,�one�needs�a�nested�design�with�a�mixed�model��True�or�false?
16.11� I�assert�that�the�predefined�value�method�of�block�formation�is�more�effective�than� the�sampled�value�method�in�reducing�unexplained�variability��Am�I�correct?
16.12� For�the�interaction�to�be�tested�in�a�two-factor�randomized�block�design,�it�is�required� that�which�one�of�the�following�occurs?
� a�� Both�factors�be�fixed � b�� Both�factors�be�random � c�� n�=�1 � d�� n�>�1 16.13� Five� medical� professors� use� a� computer-based� method� of� teaching� and� five� other�
medical� professors� use� a� lecture-based� method� of� teaching�� A� researcher� is� inter- ested�in�student�outcomes�for�those�enrolled�in�classes�taught�by�these�instructional� methods��This�is�an�example�of�which�type�of�design?
� a�� Completely�crossed�design � b�� Repeated�measures�design � c�� Hierarchical�design � d�� Randomized�block�design 16.14� In� a� randomized� block� study,� the� correlation� between� the� blocking� factor� and� the�
dependent�variable�is��35��I�assert�that�the�residual�variation�will�be�smaller�when� using�the�blocking�variable�than�without��Am�I�correct?
16.15� A�researcher�is�interested�in�examining�the�number�of�suspensions�of�high�school� students� based� on� random� assignment� participation� in� a� series� of� self-awareness� workshops��The�researcher�believes�that�age�may�be�a�concomitant�variable��Applying� a� two-factor� randomized� block� ANOVA� design� to� the� data,� age� is� an� appropriate� blocking�factor?
16.16� In�a�two-factor�hierarchical�design�with�two�levels�of�factor�A�and�three�levels�of� factor�B�nested�within�each�level�of�A,�how�many�F�ratios�can�be�tested?
� a�� 1 � b�� 2 � c�� 3 � d�� Cannot�be�determined
608 An Introduction to Statistical Concepts
16.17� If�the�correlation�between�the�concomitant�variable�and�dependent�variable�is�−�80,� which�of�the�following�designs�is�recommended?
� a�� ANCOVA � b�� One-factor�ANOVA � c�� Randomized�block�ANOVA � d�� All�of�the�above 16.18� IQ�must�be�used�as�a�treatment�factor��True�or�false? 16.19� Which�of�the�following�blocking�methods�best�estimates�the�treatment�effects? � a�� Predefined�value�blocking � b�� Post�hoc�predefined�value�blocking � c�� Sampled�value�blocking � d�� Sampled�range�blocking
Computational problems
16.1� An�experiment�was�conducted�to�compare�three�types�of�behavior�modification�(1,�2,� and�3)�using�age�as�a�blocking�variable�(4-,�6-,�and�8-year-old�children)��The�mean�scores� on�the�dependent�variable,�number�of�instances�of�disruptive�behavior,�are�listed�here� for�each�cell��The�intention�of�the�treatments�is�to�minimize�the�number�of�disruptions�
Type of Behavior Modification
Age
4 Years 6 Years 8 Years
1 20 40 40 2 50 30 20 3 50 40 30
Use�these�cell�means�to�graph�the�interaction�between�type�of�behavior�modification� and�age�
� a�� Is�there�an�interaction�between�type�of�behavior�modification�and�age? � b�� What�kind�of�recommendation�would�you�make�to�teachers? 16.2� An�experiment�was�conducted�to�compare�four�different�preschool�curricula�that�
were�adopted�in�four�different�classrooms��Reading�readiness�proficiency�was�used� as�a�blocking�variable�(below�proficient,�at�proficient,�above�proficient)��The�mean� scores� on� the� dependent� variable,� letter� recognition,� are� listed� here� for� each� cell�� The�intention�of�the�treatment�(i�e�,�the�curriculum)�is�to�increase�letter�recognition�
Curriculum
Reading Readiness Proficiency
Below At Above
1 12 20 22 2 20 24 18 3 16 16 20 4 15 18 25
609Hierarchical and Randomized Block Analysis of Variance Models
Use�these�cell�means�to�graph�the�interaction�between�curriculum�and�reading�readi- ness�proficiency�
� a�� Is� there� an� interaction� between� type� of� curriculum� and� reading� readiness� proficiency?
� b�� What�kind�of�recommendation�would�you�make�to�teachers? 16.3� An� experimenter� tested� three� types� of� perfume� (or� aftershave)� (tame,� sexy,� and�
musk)� when� worn� by� light-haired� and� dark-haired� women� (or� men)�� Thus,� hair� color� is� a� blocking� variable�� The� dependent� measure� was� attractiveness� defined� as�the�number�of�times�during�a�2-week�period�that�other�persons�complimented� a� subject� on� their� perfume� (or� aftershave)�� There� were� five� subjects� in� each� cell�� Complete� the� ANOVA� summary� table� below,� assuming� a� fixed-effects� model,� where�α�=��05�
Source SS df MS F Critical Value Decision
Perfume�(A) 200 — — — — — Hair�color�(B) 100 — — — — — Interaction� (AB)
20 — — — — —
Within 240 — — Total — —
16.4� An�experiment�was�conducted�to�determine�if�there�was�a�mean�difference�in�weight� for�women�based�on�type�of�aerobics�exercise�program�participated�(low�impact�vs�� high� impact)��Body�mass�index�(BMI)�was�used�as�a�blocking�variable� to�represent� below,� at,� or� above� recommended� BMI�� The� data� are� shown� as� follows�� Conduct� a� two-factor�randomized�block�ANOVA�(α�=��05)�and�Bonferroni�MCPs�using�SPSS�to� determine�the�results�of�the�study�
Subject Exercise Program BMI Weight
1 1 1 100 2 1 2 135 3 1 3 200 4 1 1 95 5 1 2 140 6 1 3 180 7 2 1 120 8 2 2 152 9 2 3 176
10 2 1 128 11 2 2 142 12 2 3 220
610 An Introduction to Statistical Concepts
16.5� A�mathematics�professor�wants�to�know�which�of�three�approaches�to�teaching�cal- culus�resulted�in�the�best�test�performance�(Sections�16�1,�16�2,�or�16�3)��Scores�on�the� GRE-Quantitative�(GRE-Q)�portion�were�used�as�a�blocking�variable�(block�1:�200– 400;�block�2:�401–600;�block�3:�601–800)��The�data�are�shown�as�follows��Conduct�a� two-factor�randomized�block�ANOVA�(α�=��05)�and�Bonferroni�MCPs�using�SPSS�to� determine�the�results�of�the�study�
Subject Section GRE-Q Test
Score
1 1 1 90 2 1 2 93 3 1 3 100 4 2 1 88 5 2 2 90 6 2 3 97 7 3 1 79 8 3 2 85 9 3 3 92
Interpretive problems
16.1� The�following�is�the�first�one-factor�ANOVA�interpretive�problem�you�developed�in� Chapter�11:�Using the survey 1 dataset from the website, use SPSS to conduct a one-factor fixed-effects ANOVA, including effect size, where political view is the grouping variable�(i.e., independent variable)�(J = 5)�and the dependent variable is a variable of interest to you�(the following variables look interesting: books, TV, exercise, drinks, GPA, GRE-Q, CDs, hair appointment)��Then write an APA-style paragraph describing the results�
Take� the� one-factor� ANOVA� interpretive� problem� you� developed� in� Chapter� 11�� What� are� some� reasonable� blocking� variables� to� consider?� Which� type� of� blocking� would�be�best�in�your�situation?�Select�this�blocking�variable�from�the�same�dataset� and�conduct�a�two-factor�randomized�block�ANOVA��Compare�these�results�with�the� one-factor�ANOVA�results�(without�the�blocking�factor)�to�determine�how�useful�the� blocking�variable�was�in�terms�of�reducing�residual�variability�
16.2� The�following�is�the�second�one-factor�ANOVA�interpretive�problem�you�devel- oped�in�Chapter�11:�Using the survey 1 dataset from the website, use SPSS to conduct a one-factor fixed-effects ANOVA, including effect size, where hair color is the grouping variable� (i.e., independent variable)� (J = 5)� and the dependent variable is a variable of interest to you�(the following variables look interesting: books, TV, exercise, drinks, GPA, GRE-Q, CDs, hair appointment)�� Then write an APA-style paragraph describing the results�
Take� this� one-factor� ANOVA� interpretive� problem� you� developed� in� Chapter� 11�� What� are� some� reasonable� blocking� variables� to� consider?� Which� type� of� blocking� would�be�best�in�your�situation?�Select�this�blocking�variable�from�the�same�dataset� and�conduct�a�two-factor�randomized�ANOVA��Compare�these�results�with�the�one- factor� ANOVA� results� (without� the� blocking� factor)� to� determine� how� useful� the� blocking�variable�was�in�terms�of�reducing�residual�variability�
611
17 Simple Linear Regression
Chapter Outline
17�1� �Concepts�of�Simple�Linear�Regression 17�2� �Population�Simple�Linear�Regression�Model 17�3� �Sample�Simple�Linear�Regression�Model
17�3�1� �Unstandardized�Regression�Model 17�3�2� �Standardized�Regression�Model 17�3�3� �Prediction�Errors 17�3�4� �Least�Squares�Criterion 17�3�5� �Proportion�of�Predictable�Variation�(Coefficient�of�Determination) 17�3�6� �Significance�Tests�and�Confidence�Intervals 17�3�7� �Assumptions�and�Violation�of�Assumptions
17�4� �SPSS 17�5� �G*Power 17�6� �Template�and�APA-Style�Write-Up
Key Concepts
� 1�� Slope�and�intercept�of�a�straight�line � 2�� Regression�model � 3�� Prediction�errors/residuals � 4�� Standardized�and�unstandardized�regression�coefficients � 5�� Proportion�of�variation�accounted�for;�coefficient�of�determination
In� Chapter� 10,� we� considered� various� bivariate� measures� of� association�� Specifically,� the� chapter� dealt� with� the� topics� of� scatterplot,� covariance,� types� of� correlation� coefficients,� and�their�resulting�inferential�tests��Thus,�the�chapter�was�concerned�with�addressing�the� question�of�the�extent�to�which�two�variables�are�associated�or�related��In�this�chapter,� we�extend�our�discussion�of�two�variables�to�address�the�question�of�the�extent�to�which� one�variable�can�be�used�to�predict�or�explain�another�variable�
Beginning�in�Chapter�11,�we�examined�various�analysis�of�variance�(ANOVA)�models�� It�should�be�mentioned�again�that�ANOVA�and�regression�are�both�forms�of�the�same�gen- eral�linear�model�(GLM),�where�the�relationship�between�one�or�more�independent�variables�
612 An Introduction to Statistical Concepts
and�one�dependent�variable�is�evaluated��The�major�difference�between�the�two�procedures� is�that�in�ANOVA,�the�independent�variables�are�discrete�variables�(i�e�,�nominal�or�ordinal),� while�in�regression,�the�independent�variables�are�continuous�variables�(i�e�,�interval�or�ratio;� however,�we�will�see�later�how�we�can�apply�dichotomous�variables�in�regression�models)�� Otherwise�there�is�considerable�overlap�of�these�two�procedures�in�terms�of�concepts�and� their� implementation�� Note� that� a� continuous� variable� can� be� transformed� into� a� discrete� variable��For�example,�the�Graduate�Record�Exam-Quantitative�(GRE_Q)�exam�is�a�continu- ous�variable�scaled�from�200�to�800�(albeit�in�10-point�score�increments)��It�could�be�made�into� a�discrete�variable,�such�as�low�(200–400),�average�(401–600),�and�high�(601–800)�
When� considering� the� relationship� between� two� variables� (say� X� and� Y),� the� researcher� usually�determines�some�measure�of�relationship�between�those�variables,�such�as�a�correla- tion�coefficient�(e�g�,�rXY,�the�Pearson�product–moment�correlation�coefficient),�as�we�did�in� Chapter�10��Another�way�of�looking�at�how�two�variables�may�be�related�is�through�regression� analysis,�in�terms�of�prediction�or�explanation��That�is,�we�evaluate�the�ability�of�one�variable� to�predict�or�explain�a�second�variable��Here�we�adopt�the�usual�notation�where�X�is�defined�as� the�independent�or�predictor variable,�and�Y�as�the�dependent�or�criterion variable�
For�example,�an�admissions�officer�might�want�to�use�GRE�scores�to�predict�graduate- level� grade� point� averages� (GPAs)� to� make� admission� decisions� for� a� sample� of� appli- cants� to� a� university� or� college�� The� research� question� of� interest� is� how� well� does� the� GRE�(the�independent�or�predictor�variable)�predict�or�explain�performance�in�graduate� school�(the�dependent�or�criterion�variable)?�This�is�an�example�of�simple�linear�regres- sion�where�only�a�single�predictor�variable�is�included�in�the�analysis��The�utility�of�the� GRE�in�predicting�GPA�requires�that�these�variables�have�a�correlation�different�from�0�� Otherwise� the� GRE� will� not� be� very� useful� in� predicting� GPA�� For� education� and� the� behavioral� sciences,� the� use� of� a� single� predictor� does� not� usually� result� in� reasonable� prediction�or�explanation��Thus,�Chapter�18�considers�the�case�of�multiple�predictor�vari- ables�through�multiple�linear�regression�analysis�
In�this�chapter,�we�consider�the�concepts�of�slope,�intercept,�regression�model,�unstan- dardized� and� standardized� regression� coefficients,� residuals,� proportion� of� variation� accounted�for,�tests�of�significance,�and�statistical�assumptions��Our�objectives�are�that�by� the�end�of�this�chapter,�you�will�be�able�to�(a)�understand�the�concepts�underlying�simple� linear�regression,�(b)�determine�and�interpret�the�results�of�simple�linear�regression,�and� (c)�understand�and�evaluate�the�assumptions�of�simple�linear�regression�
17.1 Concepts of Simple Linear Regression
In�this�chapter,�we�continue�to�follow�Marie�on�yet�another�statistical�analysis�adventure�
Marie� has� developed� excellent� rapport� with� the� faculty� at� her� institution� as� she� has� assisted� them� in� statistical� analysis�� Marie� will� now� be� working� with� Randall,� an� associate�dean�in�the�Graduate�Student�Services�office��Randall�wants�to�know�if�the� required� entrance� exam� for� graduate� school� (specifically� the� GRE_Q)� can� be� used� to� predict� midterm� grades�� Marie� suggests� the� following� research� question� to� Randall:� Can midterm exam scores be predicted from the GRE_Q?�Marie�determines�that�a�simple� linear�regression�is�the�best�statistical�procedure�to�use�to�answer�Randall’s�question�� Her�next�task�is�to�assist�Randall�in�analyzing�the�data�
613Simple Linear Regression
Let� us� consider� the� basic� concepts� involved� in� simple� linear� regression�� Many� years� ago� when�you�had�algebra,�you�learned�about�an�equation�used�to�describe�a�straight�line,
Y bX a= +
Here�the�predictor�variable�X�is�used�to�predict�the�criterion�variable�Y��The�slope�of�the� line�is�denoted�by�b�and�indicates�the�number�of�Y�units�the�line�changes�for�a�one-unit� change�in�X��You�may�find�it�easier�to�think�about�the�slope�as�measuring�tilt�or�steepness�� The�Y-intercept�is�denoted�by�a�and�is�the�point�at�which�the�line�intersects�or�crosses�the� Y�axis��To�be�more�specific,�a�is�the�value�of�Y�when�X�is�equal�to�0��Hereafter�we�use�the�term� intercept�rather�than�Y-intercept�to�keep�it�simple�
Consider�the�plot�of�the�straight�line�Y�=�0�5X�+�1�0�as�shown�in�Figure�17�1��Here�we�see� that�the�line�clearly�intersects�the�Y�axis�at�Y�=�1�0;�thus,�the�intercept�is�equal�to�1��The�slope� of�a�line�is�defined,�more�specifically,�as�the�change�in�Y�(numerator)�divided�by�the�change� in�X�(denominator)�
b Y X
Y Y X X
= = − −
∆ ∆
2 1
2 1
For�instance,�take�two�points�shown�in�Figure�17�1,�(X1,�Y1)�and�(X2,�Y2),�that�fall�on�the� straight� line� with� coordinates� (0,� 1)� and� (4,� 3),� respectively�� We� compute� the� slope� for� those�two�points�to�be�(3�−�1)/(4�−�0)�=�0�5��If�we�were�to�select�any�other�two�points�that� fall�on�the�straight�line,�then�the�slope�for�those�two�points�would�also�be�equal�to�0�5�� That�is,�regardless�of�the�two�points�on�the�line�that�we�select,�the�slope�will�always�be� the�same,�constant�value�of�0�5��This�is�true�because�we�only�need�two�points�to�define� a�particular�straight�line��That�is,�with�the�points�(0,�1)�and�(4,�3),�we�can�draw�only�one� straight�line�that�passes�through�both�of�those�points,�and�that�line�has�a�slope�of�0�5�and� an�intercept�of�1�0�
Let�us�take�the�concepts�of�slope,�intercept,�and�straight�line�and�apply�them�in�the�con- text�of�correlation�so�that�we�can�study�the�relationship�between�the�variables�X�and�Y��
3.00
2.50
2.00
1.50
1.00
Y
0.00 1.00 2.00 X
3.00 4.00 FIGuRe 17.1 Plot�of�line:�Y�=�0�5X�+�1�0�
614 An Introduction to Statistical Concepts
If�the�slope�of�the�line�is�a�positive�value�(e�g�,�Figure�17�1),�as�X�increases�Y�also�increases,� then�the�correlation�will�be�positive��If�the�slope�of�the�line�is�0,�such�that�the�line�is�paral- lel�or�horizontal�to�the�X�axis,�as�X�increases�Y�remains�constant,�then�the�correlation�will� be� 0�� If� the� slope� of� the� line� is� a� negative� value,� as� X� increases� Y� decreases� (i�e�,� the� line� decreases� from� left� to� right),� then� the� correlation� will� be� negative�� Thus,� the� sign� of� the� slope�corresponds�to�the�sign�of�the�correlation�
17.2 Population Simple Linear Regression Model
Let�us�take�these�concepts�and�apply�them�to�simple�linear�regression��Consider�the�situ- ation�where�we�have�the�entire�population�of�individual’s�scores�on�both�variables�X�(the� independent�variable,�such�as�GRE)�and�Y�(the�dependent�variable,�such�as�GPA)��We�define� the�linear�regression�model�as�the�equation�for�a�straight�line��This�yields�an�equation�for� the�regression�of�Y�the�criterion,�given�X�the�predictor,�often�stated�as�the�regression�of�Y� on�X,�although�more�easily�understood�as�Y�being�predicted�by�X�
The�population regression model�for�Y�being�predicted�by�X�is
Y Xi YX i YX i= + +β α ε
where Y�is�the�criterion�variable X�is�the�predictor�variable βYX�is�the�population�slope�for�Y�predicted�by�X αYX�is�the�population�intercept�for�Y�predicted�by�X εi�are�the�population�residuals�or�errors�of�prediction�(the�part�of�Yi�not�predicted�from�Xi) i�represents�an�index�for�a�particular�case�(an�individual�or�object;�in�other�words,�the�
unit�of�analysis�that�has�been�measured)
The�index�i�can�take�on�values�from�1�to�N,�where�N�is�the�size�of�the�population,�written� as�i�=�1,…,�N�
The�population prediction model�is
Y Xi YX i YX′ = +β α
where� Y′i� is� the� predicted� value� of� Y� for� a� specific� value� of� X�� That� is,� Yi� is� the� actual or observed score�obtained�by�individual�i,�while�Y′i�is�the�predicted score�based�on�their�X�score� for�that�same�individual�(in�other�words,�you�are�using�the�value�of�X�to�predict�what�Y� will�be)��Thus,�we�see�that�the�population�prediction�error�is�defined�as�follows:
εi i iY Y= − ′
There� is� only� one� difference� between� the� regression� and� prediction� models�� The� regres- sion�model�explicitly�includes�prediction�error�as�εi,�whereas�the�prediction�model�includes� prediction�error�implicitly�as�part�of�the�predicted�score�Y′i�(i�e�,�there�is�some�error�in�the� predicted�values)�
615Simple Linear Regression
Consider�for�a�moment�a�practical�application�of�the�difference�between�the�regression� and� prediction� models�� Frequently� a� researcher� will� develop� a� regression� model� for� a� population� where� X� and� Y� are� both� known,� and� then� use� the� prediction� model� to� actu- ally� predict� Y� when� only� X� is� known� (i�e�,� Y� will� not� be� known� until� later)�� Using� the� GRE�example,�the�admissions�officer�first�develops�a�regression�model�for�a�population�of� students�currently�attending�the�university�so�as�to�have�a�current�measure�of�GPA��This� yields� the� slope� and� intercept�� Then� the� prediction� model� is� used� to� predict� future� GPA� and� to� help� make� admission� decisions� for� next� year’s� population� of� applicants� based� on� their�GRE�scores�
A�simple�method�for�determining�the�population�slope�(βYX)�and�intercept�(αYX)�is�com- puted�as
β ρ σ σ
YX XY Y
X =
and
α µ β µYX Y YX X= −
where σY�and�σX�are�the�population�standard�deviations�for�Y�and�X�respectively ρXY�is�the�population�correlation�between�X�and�Y�(simply�the�Pearson�correlation�coef-
ficient,�rho) μY�and�μX�are�the�population�means�for�Y�and�X�respectively
Note�that�the�previously�used�mathematical�method�for�determining�the�slope�and�inter- cept�of�a�straight�line�is�not�appropriate�in�regression�analysis�with�real�data�
17.3 Sample Simple Linear Regression Model
17.3.1 unstandardized Regression Model
Let�us�return�to�the�real�world�of�sample�statistics�and�consider�the�sample�simple�linear� regression�model��As�usual,�Greek�letters�refer�to�population�parameters,�and�English�letters� refer�to�sample�statistics��The�sample regression model�for�predicting�Y�from�X�is�computed� as�follows:
Y b X a ei YX i YX i= + +
where Y�and�X�are�as�before�(i�e�,�the�dependent�and�independent�variables,�respectively) bYX�is�the�sample�slope�for�Y�predicted�by�X aYX�is�the�sample�intercept�for�Y�predicted�by�X ei�are�sample�residuals�or�errors�of�prediction�(the�part�of�Yi�not�predictable�from�Xi) i�represents�an�index�for�a�case�(an�individual�or�object)
616 An Introduction to Statistical Concepts
The�index�i�can�take�on�values�from�1�to�n,�where�n�is�the�size�of�the�sample,�and�is�written� as�i�=�1,…,�n�
The�sample prediction model�is�computed�as�follows:
Y b X ai YX i YX′ = +
where�Y′i�is�the�predicted�value�of�Y�for�a�specific�value�of�X��We�define�the�sample�predic- tion�error�as�the�difference�between�the�actual score�obtained�by�individual�i�(i�e�,�Yi)�and�the� predicted score�based�on�the�X�score�for�that�individual�(i�e�,�Y′i)��In�other�words,�the�residual� is�that�part�of�Y�that�is�not�predicted�by�X��The�goal�of�the�prediction�model�is�to�include� an�independent�variable�X�that�minimizes�the�residual;�this�means�that�the�independent� variable�does�a�nice�job�of�predicting�the�outcome��Computationally,�the�residual�(or�error)� is�computed�as�follows:
e Y Yi i i= − ′
The� difference� between� the� regression� and� prediction� models� is� the� same� as� previously� discussed,�except�now�we�are�dealing�with�a�sample�rather�than�a�population�
The�sample�slope�(bYX)�and�intercept�(aYX)�can�be�determined�by
b r s s
YX XY Y
X =
and
a Y b XYX YX= −
where sY�and�sX�are�the�sample�standard�deviations�for�Y�and�X�respectively rXY�is�the�sample�correlation�between�X�and�Y�(again�the�Pearson�correlation�coefficient,�rho) Y – �and�X
– �are�the�sample�means�for�Y�and�X,�respectively
The�sample�slope�(bYX)�is�referred�to�alternately�as�(a)�the�expected�or�predicted�change�in� Y�for�a�one-unit�change�in�X�and�(b)�the�unstandardized�or�raw�regression�coefficient��The� sample�intercept�(aYX)�is�referred�to�alternately�as�(a)�the�point�at�which�the�regression�line� intersects�(or�crosses)�the�Y�axis�and�(b)�the�value�of�Y�when�X�is�0�
Consider�now�the�analysis�of�a�realistic�example�to�be�followed�throughout�this�chap- ter��Let�us�use�the�GRE_Q�subtest�to�predict�midterm�scores�of�an�introductory�statistics� course��The�GRE_Q�has�a�possible�range�of�20–80�points�(if�we�remove�the�unnecessary� last�digit�of�zero),�and�the�statistics�midterm�has�a�possible�range�of�0–50�points��Given�the� sample� of� 10� statistics� students� shown� in� Table� 17�1,� let� us� work� through� a� simple� linear� regression�analysis��The�observation�numbers�(i�=�1,…,�10),�and�values�for�the�GRE_Q�(the� independent�variable,�X)�and�midterm�(the�dependent�variable,�Y)�variables�are�given�in� the�first�three�columns�of�the�table,�respectively��The�other�columns�are�discussed�as�we� go�along�
617Simple Linear Regression
The�sample�statistics�for�the�GRE_Q�(the�independent�variable)�are�X – �=�55�5�and�sX�=�13�1339,�
for�the�statistics�midterm�(the�dependent�variable)�are�Y – �=�38�and�sY�=�7�5130,�and�the�correlation�
rXY�is�0�9177��The�sample�slope�(bYX)�and�intercept�(aYX)�are�computed�as�follows:
b r s s
YX XY Y
X = = =0 9177
7 5130 13 1339
0 5250. . .
.
and
a Y b XYX YX= − = − =38 0 5250 55 5 8 8625. ( . ) .
Let� us� interpret� the� slope� and� intercept� values�� A� slope� of� 0�5250� means� that� if� your� score� on� the� GRE_Q� is� increased� by� one� point,� then� your� predicted� score� on� the� sta- tistics� midterm� (i�e�,� the� dependent� variable)� will� be� increased� by� 0�5250� points� or� about�half�a�point��An�intercept�of�8�8625�means�that�if�your�score�on�the�GRE_Q�is�0� (although�not�possible�as�you�receive�200�points�just�for�showing�up),�then�your�score� on� the� statistics� midterm� is� 8�8625�� The� sample� simple� linear� regression� model,� given� these�values,�becomes
Y b X a e X ei YX i YX i i i= + + = + +. .525 8 86250
If�your�score�on�the�GRE_Q�is�63,�then�your�predicted�score�on�the�statistics�midterm�is�the� following:
Y i′ = + =. . .525 (63) 8 8625 41 93750
Thus,�based�on�the�prediction�model�developed,�your�predicted�score�on�the�midterm�is� approximately�42;�however,�as�becomes�evident,�predictions�are�generally�not�perfect�
Table 17.1
Statistics�Midterm�Example�Regression�Data
Student GRE_Q (X) Midterm (Y) Residual (e) Predicted
Midterm (Y′)
1 37 32 3�7125 28�2875 2 45 36 3�5125 32�4875 3 43 27 −4�4375 31�4375 4 50 34 −1�1125 35�1125 5 65 45 2�0125 42�9875 6 72 49 2�3375 46�6625 7 61 42 1�1125 40�8875 8 57 38 −0�7875 38�7875 9 48 30 −4�0625 34�0625
10 77 47 −2�2875 49�2875
618 An Introduction to Statistical Concepts
17.3.2 Standardized Regression Model
Up�until�now,�the�computations�in�simple�linear�regression�have�involved�the�use�of�raw� scores��For�this�reason,�we�call�this�the�unstandardized regression model��The�slope�estimate� is�an�unstandardized�or�raw�regression�slope�because�it�is�the�predicted�change�in�Y�raw� score�units�for�a�one�raw�score�unit�change�in�X��We�can�also�express�regression�in�stan- dard�z�score�units�for�both�X�and�Y�as
z X X X s
i i
X ( ) =
−
and
z Y Y Y s
i i
Y ( ) =
−
In�both�cases,�the�numerator�is�the�difference�between�the�observed�score�and�the�mean,� and�the�denominator�is�the�standard�deviation�(and�dividing�by�the�standard�deviation,� standardizes�the�value)��The�means�and�variances�of�both�standardized�variables�(i�e�,�zX� and�zY)�are�0�and�1,�respectively�
The�sample�standardized�linear�prediction�model�becomes�the�following,�where�z Yi( )′ �is� the�standardized�predicted�value�of�Y:
z Y b z X r z Xi YX i XY i( ) * ( ) ( )′ = =
Thus,� the� standardized� regression� slope,�bYX* ,� sometimes� referred� to� as� a� beta weight,� is� equal�to�rXY��No�intercept�term�is�necessary�in�the�prediction�model�as�the�mean�of�the�z� scores�for�both�X�and�Y�is�0�(i�e�,�a z b zYX Y YX X* *= − = 0)��In�summary,�the standardized slope is equal to the correlation coefficient,�and�the standardized intercept is equal to 0�
For�our�statistics�midterm�example,�the�sample�standardized�linear�prediction�model�is
z Y z Xi i( ) . ( )′ = 9177
The�slope�of��9177�would�be�interpreted�as�the�expected�increase�in�the�statistics�midterm� in� z� score� (i�e�,� standardized� score)� units� for� a� one� z� score� (i�e�,� standardized� score)� unit� increase� in� the� GRE_Q�� A� one� z� score� unit� increase� is� also� the� same� as� a� one� standard� deviation�increase�because�the�standard�deviation�of�z�is�equal�to�1�(recall�from�Chapter�4� that�the�mean�of�a�standardized�z�score�is�0�with�a�standard�deviation�of�1)�
When� should� you� consider� use� of� the� standardized� versus� unstandardized� regres- sion� analyses?� According� to� Pedhazur� (1997),� the� standardized� regression� slope� b*� is� not� very� stable� from� sample� to� sample�� For� example,� at� Ivy-Covered� University,� the� standardized� regression� slope� b*� would� vary� across� different� graduating� classes� (or� samples),�whereas�the�unstandardized�regression�slope�b�would�be�much�more�consis- tent�across�classes��Thus,�in�simple�regression,�most�researchers�prefer�the�use�of�b��We� see�later�that�the�standardized�regression�slope�b*�has�some�utility�in�multiple�regres- sion�analysis�
619Simple Linear Regression
17.3.3 prediction errors
Previously�we�mentioned�that�perfect�prediction�of�Y�from�X�is�extremely�unlikely,�only� occurring�with�a�perfect�correlation�between�X�and�Y�(i�e�,�rXY�=�±1�0)��When�developing�the� regression�model,�the�values�of�the�outcome,�Y,�are�known��Once�the�slope�and�intercept� have� been� estimated,� we� can� then� use� the� prediction� model� to� predict� the� outcome� (Y)� from�the�independent�variable�(X)�when�the�values�of�Y�are�unknown��We�have�already� defined�the�predicted�values�of�Y�as�Y′��In�other�words,�a�predicted�value�Y′�can�be�com- puted�by�plugging�the�obtained�value�for�X�into�the�prediction�model��It�can�be�shown�that� Y′i�=�Yi�for�all�i�only�when�there�is�perfect�prediction��However,�this�is�extremely�unlikely�in� reality,�particularly�in�simple�linear�regression�using�a�single�predictor�
We�can�determine�a�value�of�Y′�for�each�of�the�i�cases�(individuals�or�objects)�from�the� prediction�model��In�comparing�the�actual�Y�values�to�the�predicted�Y�values,�we�obtain� the� residuals� as� the� difference� between� the� observed� (Yi)� and� predicted� values� (Y′i),� com- puted�as�follows:
e Y Yi i i= − ′
for�all�i�=�1,…,�n�individuals�or�objects�in�the�sample��The�residuals,�ei,�are�also�known� as�errors of estimate,�or�prediction errors,�and�are�that�portion�of�Yi�that�is�not�predict- able�from�Xi��The�residual�terms�are�random�values�that�are�unique�to�each�individual� or�object�
The�residuals�and�predicted�values�for�the�statistics�midterm�example�are�shown�in�the� last�two�columns�of�Table�17�1,�respectively��Consider�observation�2,�where�the�observed� GRE_Q�score�is�45�and�the�observed�midterm�score�is�36��The�predicted�midterm�score� is�32�4875�and�the�residual�is�+3�5125��This�indicates�that�person�2�had�a�higher�observed� midterm�score�than�was�predicted�using�the�GRE_Q�as�a�predictor��We�see�that�a�posi- tive�residual�indicates�the�observed�criterion�score�is�larger�than�the�predicted�criterion� score,�whereas�a�negative�residual�(such�as�in�observation�3)�indicates�the�observed�crite- rion�score�is�smaller�than�the�predicted�criterion�score��For�observation�3,�the�observed� GRE_Q� score� is� 43,� the� observed� midterm� score� is� 27,� the� predicted� midterm� score� is� 31�4375,�and,�thus,�the�residual�is�−4�4375��Person�2�scored�higher�on�the�midterm�than�we� predicted,�and�person�3�scored�lower�on�the�midterm�than�we�predicted�
The�regression�example�is�shown�graphically�in�the�scatterplot�of�Figure�17�2,�where� the� straight� diagonal� line� represents� the� regression� line�� Individuals� falling� above� the� regression� line� have� positive� residuals� (e�g�,� observation� 1)� (in� other� words,� the� differ- ence�between�the�observed�score,�represented�as�open�circle�1�on�the�graph,�is�greater�in� value�than�the�predicted�value,�which�is�represented�by�the�regression�line),�and�indi- viduals�falling�below�the�regression�line�have�negative�residuals�(e�g�,�observation�3)�(in� other�words,�the�difference�between�the�observed�score,�represented�as�open�circle�3�on� the�graph,�is�less�in�value�than�the�predicted�value,�which�is�represented�by�the�regres- sion�line)��The�residual�is,�very�simply,�the�vertical�distance�between�the�observed�score� [represented�by�the�open�circles�or�“dots”�in�the�scatterplot�(Figure�17�2)]�and�the�regres- sion�line��In�the�residual�column�of�Table�17�1,�we�see�that�half�of�the�residuals�are�posi- tive�and�half�negative,�and�in�Figure�17�2,�that�half�of�the�points�fall�above�the�regression� line�and�half�below�the�regression�line��It�can�be�shown�that�the�mean�of�the�residuals�is� always�0�(i�e�,�e–�=�0),�as�the�sum�of�the�residuals�is�always�0��This�results�from�the�fact�that� the�mean�of�the�observed�criterion�scores�is�equal�to�the�mean�of�the�predicted�criterion� scores�(i�e�,�Y
– �=�Y
–′�38�for�the�example�data)�
620 An Introduction to Statistical Concepts
17.3.4 least Squares Criterion
How� was� one� particular� method� selected� for� determining� the� slope� and� intercept?� Obviously,�some�standard�procedure�has�to�be�used��Thus,�there�are�statistical�criteria�that� help�us�decide�which�method�to�use�in�determining�the�slope�and�intercept��The�criterion� usually� used� in� linear� regression� analysis� (and� in� all� GLMs,� for� that� matter)� is� the� least squares criterion��According�to�the�least�squares�criterion,�the�sum�of�the�squared�predic- tion�errors�or�residuals�is�smallest��That�is,�we�want�to�find�that�regression�line,�defined�by� a�particular�slope�and�intercept,�which�results�in�the�smallest�sum�of�the�squared�residuals� (recall�that�the�residual�is�the�difference�between�the�observed�and�predicted�values�for�the� outcome)��Since�the�residual�is�the�vertical�difference�between�the�observed�and�predicted� value,� the� regression� line� is� simply� the� line� that� minimizes� that� vertical� distance�� Given� the�value�that�we�place�on�the�accuracy�of�prediction,�this�is�the�most�logical�choice�of�a� method�for�estimating�the�slope�and�intercept�
In�summary�then,�the�least�squares�criterion�gives�us�a�particular�slope�and�intercept,� and�thus�a�particular�regression�line,�such�that�the�sum�of�the�squared�residuals�is�small- est�� We� often� refer� to� this� particular� method� for� determining� the� slope� and� intercept� as� least squares estimation� because� b� and� a� represent� sample� estimates� of� the� population� parameters�β�and�α�obtained�using�the�least�squares�criterion�
17.3.5 proportion of predictable Variation (Coefficient of determination)
How�well�is�the�criterion�variable�Y�predicted�by�the�predictor�variable�X?�For�our�example,� we�want�to�know�how�well�the�statistics�midterm�scores�are�predicted�by�the�GRE_Q��Let�us� consider�two�possible�situations�with�respect�to�this�example��First,�if�the�GRE_Q�is�found�to� be�a�really�good�predictor�of�statistics�midterm�scores,�then�instructors�could�use�the�GRE_Q� information�to�individualize�their�instruction�to�the�skill�level�of�each�student�or�class��They� could,� for� example,� provide� special� instruction� to� those� students� with� low� GRE_Q� scores,� or� in� general,� adjust� the� level� of� instruction� to� fit� the� quantitative� skills� of� their� students��
50.00
45.00
40.00
35.00
M id
te rm
e xa
m sc
or e
30.00 1
3 25.00
30.00 40.00 50.00 60.00 GRE_Q
70.00 80.00
Imagine a point on the regression line directly below (or above) each open
dot in the scatterplot. The vertical distance
from the observed score (i.e., the
open dot) and the regression line is
the residual.
This closed dot represents the predicted value for the dependent
variable. Although not shown, each observed value (i.e., each open dot) has a predicted value just like this closed dot on the regression line.
FIGuRe 17.2 Scatterplot�for�midterm�example�
621Simple Linear Regression
Second,�if�the�GRE_Q�is�not�found�to�be�a�very�good�predictor�of�statistics�midterm�scores,� then�instructors�would�not�find�very�much�use�for�the�GRE_Q�in�terms�of�their�preparation� for� the� statistics� course�� They� could� search� for� some� other� more� useful� predictor,� such� as� prior�grades�in�quantitatively�oriented�courses�or�the�number�of�years�since�the�student�had� taken�algebra��In�other�words,�if�a�predictor�is�not�found�to�be�particularly�useful�in�predict- ing�the�criterion�variable,�then�other�relevant�predictors�should�be�considered�
How�do�we�determine�the�utility�of�a�predictor�variable?�The�simplest�method�involves�par- titioning�the�total�sum�of�squares�in�Y,�which�we�denote�as�SStotal�(sometimes�written�as�SSY)�� This�process�is�much�like�partitioning�the�sum�of�squares�in�ANOVA�
In�simple�linear�regression,�we�can�partition�SStotal�into
SS SS SStotal reg res= +
( ) ( ) ( )Y Y Y Y Y Y i
n
i
n
i
n
− = ′ − + − ′ = = =
∑ ∑ ∑2 1
2
1
2
1
where SStotal�is�the�total�sum�of�squares�in�Y SSreg�is�the�sum�of�squares�of�the�regression�of�Y�predicted�by�X�(sometimes�written�as�
SSY′)�(and�represented�in�the�equation�as� ( ) 2
1
′ − =
∑ Y Y i
n
)
SSres� is� the� sum� of� squares� of� the� residuals� (and� represented� in� the� equation� as�
( )Y Y i
n
− ′ =
∑ 2 1
),�and�the�sums�are�taken�over�all�observations�from�i�=�1,…,�n
Thus,�SStotal�represents�the�total�variation�in�the�observed�Y�scores,�SSreg�the�variation�in�Y� predicted�by�X,�and�SSres�the�variation�in�Y�not�predicted�by�X�
The�equation�for�SSreg�uses�information�about�the�difference�between�the�predicted�value�
of�Y�and�the�mean�of�Y:� ( ) .′ − =
∑ Y Y i
n 2
1
�Thus,�the�SSreg�is�essentially�examining�how�much�bet-
ter�the�line�of�best�fit�(i�e�,�the�predicted�value�of�Y)�is�as�compared�to�the�mean�of�Y�(recall� that�a�slope�of�0�is�a�horizontal�line,�which�is�the�mean�of�Y)��The�equation�for�SSres�uses� information�about�the�difference�between�the�observed�value�of�Y�and�the�predicted�value�
of�Y:� ( ) .Y Y i
n
− ′ =
∑ 2 1
�Thus,�the�SSres�is�providing�an�indication�of�how�“off”�or�inaccurate�the�
model�is��The�closer�SSres�is�to�0,�the�better�the�model�fit�(as�more�variability�of�the�depen- dent�variable�is�being�explained�by�the�model;�in�other�words,�the�independent�variables� are�doing�a�good�job�of�prediction�when�the�SSres�is�smaller)��Since�r SS SSXY reg total
2 = ,�we�can� write�SStotal,�SSreg,�and�SSres�as�follows:
SS
n Y Y
n total
i
n
i
n
=
−
= = ∑ ∑2
1 1
2
SS r SSreg XY total= 2
SS r SSres XY total= −( )1 2
622 An Introduction to Statistical Concepts
where�rXY 2 �is�the�squared�sample�correlation�between�X�and�Y,�commonly�referred�to�as�the�
coefficient of determination��The�coefficient�of�determination�in�simple�linear�regression� is� not� only� the� squared� simple� bivariate� Pearson� correlation� between� X� and� Y� but� also�
r SS SS
XY reg
total
2 = ,�which�tells�us�that�it�is�the�proportion�of�the�total�variation�of�the�dependent�
variable�(i�e�,�the�denominator)�that�has�been�explained�by�the�regression�model�(i�e�,�the� numerator)�
There�is�no�objective�gold�standard�as�to�how�large�the�coefficient�of�determination�needs� to�be�in�order�to�say�a�meaningful�proportion�of�variation�has�been�predicted��The�coefficient� is�determined,�not�just�by�the�quality�of�the�one�predictor�variable�included�in�the�model,� but�also�by�the�quality�of�relevant�predictor�variables�not�included�in�the�model�and�by�the� amount�of�total�variation�in�Y��However,�the�coefficient�of�determination�can�be�used�both�as� a�measure�of�effect�size�and�as�a�test�of�significance�(described�in�the�next�section)��According� to�the�subjective�standards�of�Cohen�(1988),�a�small�effect�size�is�defined�as�r�=��10�or r2�=��01,� a�medium�effect�size�as�r�=��30�or r2�=��09,�and�a�large�effect�size�as�r�=��50�or r2�=��25��For�addi- tional�information�on�effect�size�measures�in�regression,�we�suggest�you�consider�Steiger�and� Fouladi�(1992),�Mendoza�and�Stafford�(2001),�and�Smithson�(2001;�which�also�includes�some� discussion�of�power)�
With� the� sample� data� of� predicting� midterm� statistics� scores� from� the� GRE_Q,� let� us� determine�the�sums�of�squares��We�can�write�SStotal�as�follows:
SS
n Y Y
n total
i
n
i
n
=
−
= −
== =
∑ ∑2 1 1
2
210 14 948 380 10
508 00 ( , ) ( )
. 000
We�already�know�that�rXY�=��9177,�so�squaring�it,�we�obtain�rXY 2 .= 8422��Next�we�can�deter-
mine�SSreg�and�SSres�as�follows:
SS r SSreg XY total= = = 2 8422 508 0000 427 8376. ( . ) .
SS r SSres XY total= − = − =( ) ( . )( . ) .1 1 8422 508 0000 80 1624 2
Given�the�squared�correlation�between�X�and�Y�(rXY 2 = .8422),�the�GRE_Q�predicts�approxi-
mately�84%�of�the�variation�in�the�midterm�statistics�exam,�which�is�clearly�a�large�effect� size��Significance�tests�are�discussed�in�the�next�section�
17.3.6 Significance Tests and Confidence Intervals
This� section� describes� four� procedures� used� in� the� simple� linear� regression� context�� The�first�two�are�tests�of�statistical�significance�that�generally�involve�testing�whether� or�not�X�is�a�significant�predictor�of�Y��Then�we�consider�two�confidence�interval�(CI)� techniques�
623Simple Linear Regression
17.3.6.1 Test of Significance of rXY 2
The�first�test�is�the�test�of�the�significance�of�rXY 2 �(alternatively�known�as�the�test�of�the�propor-
tion�of�variation�in�Y�predicted�or�explained�by�X)��It�is�important�that�rXY 2 �be�different�from�0�
in�order�to�have�reasonable�prediction��The�null�and�alternative�hypotheses,�respectively,�are� as�follows,�where�the�null�indicates�that�the�correlation�between�X�and�Y�will�be�0:
H XY0 2: 0ρ =
H XY1: ρ 2 0>
This�test�is�based�on�the�following�test�statistic:
F r m
r n m =
− − −
2
21 1 /
( ) /( )
where F�indicates�that�this�is�an�F�statistic r2�is�the�coefficient�of�determination 1�−�r2�is�the�proportion�of�variation�in�Y�that�is�not�predicted�by�X m�is�the�number�of�predictors�(which�in�the�case�of�simple�linear�regression�is�always�1) n�is�the�sample�size
The�F�test�statistic�is�compared�to�the�F�critical�value,�always�a�one-tailed�test�(given�that� a� squared� value� cannot� be� negative),� and� at� the� designated� level� of� significance� α,� with� degrees�of�freedom�equal�to�m�(i�e�,�the�number�of�independent�variables)�and�(n − m�−�1),� as�taken�from�the�F�table�in�Table�A�4��That�is,�the�tabled�critical�value�is�αFm,�(n − m�−�1)�
For�the�statistics�midterm�example,�we�determine�the�test�statistic�to�be�the�following:
F r m
r n m =
− − − =
− − − =
2
21 1 8422 1
1 8422 10 1 1 42 6971
/ ( )/( )
. / ( . )/( )
.
From�Table�A�4,�the�critical�value,�at�the��05�level�of�significance,�with�degrees�of�freedom� of�1�(i�e�,�one�predictor)�and�8�(i�e�,�n − m�−�1�=�10�−�1�−�1�=�8),�is��05F1,8�=�5�32��The�test�statistic� exceeds�the�critical�value;�thus,�we�reject�H0�and�conclude�that�ρXY
2 �is�not�equal�to�0�at�the� �05�level�of�significance�(i�e�,�GRE_Q�does�predict�a�significant�proportion�of�the�variation� on�the�midterm�exam)�
17.3.6.2 Test of Significance of bYX
The�second�test�is�the�test�of�the�significance�of�the�slope�or�regression�coefficient,�bYX��In� other�words,�is�the�unstandardized�regression�coefficient�statistically�significantly�differ- ent� from� 0?� This� is� actually� the� same� as� the� test� of� b*,� the� standardized� regression� coef- ficient,�so�we�need�not�develop�a�separate�test�for�b*��The�null�and�alternative�hypotheses,� respectively,�are�as�follows:
H YX0 0: β =
H YX1 0: β ≠
624 An Introduction to Statistical Concepts
To� test� whether� the� regression� coefficient� is� equal� to�0,� we� need�a� standard� error� for� the� slope�b��However,�first�we�need�to�develop�some�new�concepts��The�first�new�concept�is�the� variance error of estimate��Although�this�is�the�correct�term,�it�is�easier�to�consider�this�as� the�variance of the residuals��The�variance�error�of�estimate,�or�variance�of�the�residuals,� is�defined�as
s e df SS df MSres i res res res res 2 2/ /= = =Σ
where�the�summation�is�taken�from�i�=�1,…,�n�and�dfres�=�(n − m�−�1)�(or�n�−�2�if�there�is� only�a�single�predictor)��Two�degrees�of�freedom�are�lost�because�we�have�to�estimate�the� population�slope�and�intercept,�β�and�α,�from�the�sample�data��The�variance�error�of�esti- mate�indicates�the�amount�of�variation�among�the�residuals��If�there�are�some�extremely� large�residuals,�this�will�result�in�a�relatively�large�value�of�s2res,�indicating�poor�prediction� overall��If�the�residuals�are�generally�small,�this�will�result�in�a�comparatively�small�value� of�s2res,�indicating�good�prediction�overall�
The�next�new�concept�is�the�standard error of estimate�(sometimes�known�as�the�root� mean�square�error)��The�standard�error�of�estimate�is�simply�the�positive�square�root�of�the� variance�error�of�estimate�and�thus�is�the�standard�deviation�of�the�residuals�or�errors�of� estimate��We�denote�the�standard�error�of�estimate�as�sres�
The�final�new�concept�is�the�standard error of�b��We�denote�the�standard�error�of�b�as�sb� and�define�it�as
s s
n X X n
s SS
b res res
X
= − ( )
=
∑ ∑2 2
where� the� summation� is� taken� over� i� =� 1,…,� n�� We� want� sb� to� be� small� to� reject� H0,� so� we�need�sres�to�be�small�and�SSX�to�be�large��In�other�words,�we�want�there�to�be�a�large� spread�of�scores�in�X��If�the�variability�in�X�is�small,�it�is�difficult�for�X�to�be�a�significant� predictor�of�Y�
Now�we�can�put�these�concepts�together�into�a�test�statistic�to�test�the�significance�of�the� slope�b��As�in�many�significance�tests,�the�test�statistic�is�formed�by�the�ratio�of�a�parameter� estimate�divided�by�its�respective�standard�error��A�ratio�of�the�parameter�estimate�of�the� slope�b�to�its�standard�error�sb�is�formed�as�follows:
t b sb
=
The�test�statistic�t�is�compared�to�the�critical�values�of�t�(in�Table�A�2),�a�two-tailed�test�for�a� nondirectional�H1,�at�the�designated�level�of�significance�α,�and�with�degrees�of�freedom�of� (n − m�−�1)��That�is,�the�tabled�critical�values�are�±(α/2)�t(n − m�−�1)�for�a�two-tailed�test�
In�addition,�all�other�things�being�equal�(i�e�,�same�data,�same�degrees�of�freedom,�same� level� of� significance),� both� of� these� significance� tests� (i�e�,� the� test� of� significance� of� the� squared� bivariate� correlation� between� X� and� Y� and� the� test� of� significance� of� the� slope)� will�yield�the�exact�same�result��That�is,�if�X�is�a�significant�predictor�of�Y,�then�H0�will�be�
625Simple Linear Regression
rejected�in�both�tests��If�X is not�a�significant�predictor�of�Y,�then�H0�will�not�be�rejected�for� either�test��In�simple�linear�regression,�each�of�these�tests�is�a�method�for�testing�the�same� general�hypothesis�and�logically�should�lead�the�researcher�to�the�exact�same�conclusion�� Thus,�there�is�no�need�to�implement�both�tests�
We�can�also�form�a�CI�around�the�slope�b��As�in�most�CI�procedures,�it�follows�the�form� of�the�sample�estimate�plus�or�minus�the�tabled�critical�value�multiplied�by�the�standard� error��The�CI�around�b�is�formed�as�follows:
CI ( ) 2 1b b t sn m b= ± − −( / ) ( )α
Recall�that�the�null�hypothesis�was�written�as�H0:�β�=�0��Therefore,�if�the�CI�contains�0,�then� β�is�not�significantly�different�from�0�at�the�specified�α�level��This�is�interpreted�to�mean� that�in�(1�−�α)%�of�the�sample�CIs�that�would�be�formed�from�multiple�samples,�β�will�be� included��This�procedure�assumes�homogeneity�of�variance�(discussed�later�in�this�chap- ter);�for�alternative�procedures,�see�Wilcox�(1996,�2003)�
Now� we� can� determine� the� second� test� statistic� for� the� midterm� statistics� example�� We�specify�H0:�β�=�0�(i�e�,�the�null�hypothesis�is�that�the�slope�is�equal�to�0;�visually�a� slope�of�0�is�a�horizontal�line)�and�conduct�a�two-tailed�test��First�the�variance�error�of� estimate�is
s e df SS df MSres i res res res res 2 2 / / 8 1578/8 1 197= = = = =Σ 0 0 0. .
The� standard� error� of� estimate,� sres,� is� 10 0197 3 1654. .= �� Next� the� standard� error� of� b� is� computed�as�follows:
s s
n X X n
s SS
b res res
X
= − ( )
= = =
∑ ∑2 2
3 1654 1552 5000
0803 .
. .
Finally,�we�determine�the�test�statistic�to�be�as�follows:
t b sb
= = = . .
. 5250 0803
6 5380
To� evaluate� the� null� hypothesis,� we� compare� this� test� statistic� to� its� critical� values� ±�025t8� =� ±2�306�� The� test� statistic� exceeds� the� critical� value,� so� H0� is� rejected� in� favor� of� H1��We�conclude�that�the�slope�is�indeed�significantly�different�from�0,�at�the��05�level�of� significance�
Finally�let�us�determine�the�CI�for�the�slope�b�as�follows:
CI ( )
525 2 3 6( 8
2 1 25 8b b t s b t sn m b b= ± = ±
= ±
− −( / ) ( ) . ( )
. . .
α 0
0 0 0 0 0 033) 3398 71 2= ( . , . )0 0 0
The�interval�does�not�contain�0,�the�value�specified�in�H0;�thus,�we�conclude�that�the�slope� β�is�significantly�different�from�0,�at�the��05�level�of�significance�
626 An Introduction to Statistical Concepts
17.3.6.3 Confidence Interval for the Predicted Mean Value of Y
The�third�procedure�is�to�develop�a�CI�for�the�predicted�mean�value�of�Y,�denoted�by�Y0′,�for� a�specific�value�of�X0��Alternatively,�Y0′�is�referred�to�as�the�conditional�mean�of�Y�given�X0� (more�about�conditional�distributions�in�the�next�section)��In�other�words,�for�a�particular� predictor�score�X0,�how�confident�can�we�be�in�the�predicted�mean�for�Y?
The�standard�error�of�Y0′�is
s Y s n X X SSres X( ) ( / ) [( ) ]0 0 21′ = + − /
In�looking�at�this�equation,�the�further�X0�is�from�X – ,�the�larger�the�standard�error��Thus,�the�
standard�error�depends�on�the�particular�value�of�X0�selected��In�other�words,�we�expect� to�make�our�best�predictions�at�the�center�of�the�distribution�of�X�scores�and�to�make�our� poorest�predictions�for�extreme�values�of�X��Thus,�the�closer�the�value�of�the�predictor�is�to� the�center�of�the�distribution�of�the�X�scores,�the�better�the�prediction�will�be�
A�CI�around�Y0′�is�formed�as�follows:
CI 2 2( ) ( )( / ) ( )Y Y t s Yn0 0 0′ ′ ′= ± −α
Our�interpretation�is�that�in�(1�−�α)%�of�the�sample�CIs�that�would�be�formed�from�multiple� samples,�the�population�mean�value�of�Y�for�a�given�value�of�X�will�be�included�
Let�us�consider�an�example�of�this�CI�procedure�with�the�midterm�statistics�data��If�we� take�a�GRE_Q�score�of�50,�the�predicted�score�on�the�statistics�midterm�is�35�1125��A�CI�for� the�predicted�mean�value�of�35�1125�is�as�follows:
s Y s n X X SSres X( ) ( ) [( ) ] . ( ) [( )0 0 2 21 3 1654 1 10 50 55 1552′ = + − = + −/ / / / .. ] .5000 1 0786=
CI
35 1125 2
2 2 25 8( ) ( ) ( )
. (
( / ) ( ) .Y Y t s Y Y t s Yn0 0 0 0 0 0′ ′ ′ ′ ′= ± = ±
= ±
−α
.. ) . ( . , . )3 6 (1 786) 32 6252 37 59980 0 =
In�Figure�17�3,�the�CI�around�Y0′�given�X0�is�plotted�as�the�pair�of�curved�lines�closest�to�the� regression�line��Here�we�see�graphically�that�the�width�of�the�CI�increases�the�further�we� move�from�X
– �(where�X
– �=�55�5000)�
17.3.6.4 Prediction Interval for Individual Values of Y
The� fourth� and� final� procedure� is� to� develop� a� prediction� interval� (PI)� for� an� individual� predicted�value�of�Y′0�at�a�specific�individual�value�of�X0��That�is,�the�predictor�score�for�a� particular�individual�is�known,�but�the�criterion�score�for�that�individual�has�not�yet�been� observed��This�is�in�contrast�to�the�CI�just�discussed�where�the�individual�Y�scores�have� already�been�observed��Thus,�the�CI�deals�with�the�mean�of�the�predicted�values,�while�the� PI�deals�with�an�individual�predicted�value�not�yet�observed�
The�standard�error�of�Y′0�is
s Y s n X X SSres X( ) ( / ) [( ) ]0 0 21 1′ = + + − /
627Simple Linear Regression
The�standard�error�of�Y′0�is�similar�to�the�standard�error�of�Y0′�with�the�addition�of�1�to�the� equation��Thus,�the�standard�error�of�Y ′0�will�always�be�greater�than�the�standard�error�of�Y0′� as�there�is�more�uncertainty�about�individual�values�than�about�the�mean��The�further�X0� is�from�X
– ,�the�larger�the�standard�error��Thus,�the�standard�error�again�depends�on�the�par-
ticular�value�of�X,�where�we�have�more�confidence�in�predictions�for�values�of�X�close�to�X – �
The�PI�around�Y′0�is�formed�as�follows:
PI( ) 2 2Y Y t s Yn′ = ′ ± ′−0 0 0( / ) ( ) ( )α
Our�interpretation�is�that�in�(1�−�α)%�of�the�sample�PIs�that�would�be�formed�from�multiple� samples,�the�new�observation�Y0�for�a�given�value�of�X�will�be�included�
Consider� an� example� of� this� PI� procedure� with� the� midterm� statistics� data�� If� we� take� a�GRE_Q�score�of�50,�the�predicted�score�on�the�statistics�midterm�is�35�1125��A�PI�for�the� predicted�individual�value�of�35�1125�is�as�follows:
s Y s n X X SSres X( ) ( / ) [( ) / ] . ( ) [( )0 0 2 21 1 3 1654 1 1 10 50 55′ = + + − = + + −/ /11552 5000 3 3441. ] . .=
PI ( )
35 1125
2 2 25 8Y Y t s Y Y t s Yn′ = ′ ± ′ = ′ ± ′
= ±
−0 0 0 0 0 0( / ) ( ) .( ) ( )
. (
α
22 3 6 3 3441 27 4 1 42 824. )( . ) ( . , . )0 0 0 0=
In� Figure� 17�3,� the� PI� around� Y′0� given� X0� is� plotted� as� the� pair� of� curved� lines� furthest� from�the�regression�line��Here�we�see�graphically�that�the�PI�is�always�wider�than�its�cor- responding�CI�
17.3.7 assumptions and Violation of assumptions
In�this�section,�we�consider�the�following�assumptions�involved�in�simple�linear�regres- sion:�(a)�independence,�(b)�homogeneity,�(c)�normality,�(d)�linearity,�and�(e)�fixed�X��Some� discussion�is�also�devoted�to�the�effects�of�assumption�violations�and�how�to�detect�them�
50.00
45.00
40.00
M id
te rm
e xa
m sc
or e
35.00
30.00
25.00
30.00 40.00 50.00 60.00 GRE_Q
70.00 80.00
FIGuRe 17.3 CIs�for�midterm�example:�the�curved�lines� closest� to� the� regression� line� are� for� the� 95%�CI;�the�curved�lines�furthest�from�the� regression�line�are�for�the�95%�PI�
628 An Introduction to Statistical Concepts
17.3.7.1 Independence
The�first�assumption�is�concerned�with�independence�of�the�observations��We�should�be� familiar�with�this�assumption�from�previous�chapters�(e�g�,�ANOVA)��In�regression�analy- sis,� another� way� to� think� about� this� assumption� is� that� the� errors� in� prediction� or� the� residuals�(i�e�,�ei)�are�assumed�to�be�random�and�independent��That�is,�there�is�no�system- atic�pattern�about�the�errors,�and�each�error�is�independent�of�the�other�errors��An�example� of� a� systematic� pattern� would� be� where� for� small� values� of� X� the� residuals� tended� to� be� small,�whereas�for�large�values�of�X,�the�residuals�tended�to�be�large��Thus,�there�would�be� a�relationship�between�the�independent�variable� X�and�the�residual�e��Dependent�errors� occur�when�the�error�for�one�individual�depends�on�or�is�related�to�the�error�for�another� individual�as�a�result�of�some�predictor�not�being�included�in�the�model��For�our�midterm� statistics�example,�students�similar�in�age�might�have�similar�residuals�because�age�was� not�included�as�a�predictor�in�the�model�
Note�that�there�are�several�different�types�of�residuals��The�ei�is�known�as�raw residuals� for�the�same�reason�that�Xi�and�Yi�are�called�raw�scores,�all�being�in�their�original�scale�� The�raw�residuals�are�on�the�same�raw�score�scale�as�Y�but�with�a�mean�of�0�and�a�vari- ance�of�s2res��Some�researchers�dislike�raw�residuals�as�their�scale�depends�on�the�scale�of�Y,� and,� therefore,� they� must� temper� their� interpretation� of� the� residual� values�� Several� dif- ferent�types�of�standardized residuals�have�been�developed,�including�the�original�form� of�standardized�residual�ei /sres��These�values�are�measured�along�the�z�score�scale�with�a� mean�of�0�and�a�variance�of�1,�and�approximately�95%�of�the�values�are�within�±2�units�of�0�� Later�in�our�illustration�of�SPSS,�we�will�use�studentized residuals�for�diagnostic�checks�� Studentized�residuals�are�a�type�of�standardized�residual�that�are�more�sensitive�to�detect- ing�outliers��Some�researchers�prefer�these�or�other�variants�of�standardized�residuals�over� raw�residuals�because�they�find�it�easier�to�detect�large�residuals��However,�if�you�really� think�about�it,�one�can�easily�look�at�the�middle�95%�of�the�raw�residuals�by�just�consider- ing� the� range� of� ±2� standard� errors� (i�e�,� ±2sres)� around� 0�� Readers� interested� in� learning� more� about� other� types� of� standardized� residuals� are� referred� to� a� number� of� excellent� resources� (see� Atkinson,� 1985;� Cook� &� Weisberg,� 1982;� Dunn� &� Clark,� 1987;� Kleinbaum,� Kupper,�Muller,�&�Nizam,�1998;�Weisberg,�1985)�
The�simplest�procedure�for�assessing�this�assumption�is�to�examine�a�scatterplot�(Y�vs��X)� or�a�residual�plot�(e�g�,�e�vs��X)��If�the�independence�assumption�is�satisfied,�there�should�be� a�random�display�of�points��If�the�assumption�is�violated,�the�plot�will�display�some�type� of�pattern;�for�example,�the�negative�residuals�tend�to�cluster�together,�and�positive�residu- als� tend� to� cluster� together�� As� we� know� from� ANOVA,� violation� of� the� independence� assumption�generally�occurs�in�the�following�three�situations:�(a)�when�the�observations� are�collected�over�time�(the�independent�variable�is�a�measure�of�time;�consider�using�the� Durbin�and�Watson�test�[1950,�1951,�1971]);�(b)�when�observations�are�made�within�blocks,� such�that�the�observations�within�a�particular�block�are�more�similar�than�observations�in� different�blocks;�or�(c)�when�observation�involves�replication��Lack�of�independence�affects� the�estimated�standard�errors,�being�under-�or�overestimated��For�serious�violations,�one� could�consider�using�generalized�or�weighted�least�squares�as�the�method�of�estimation�
17.3.7.2 Homogeneity
The� second� assumption� is� homogeneity of variance,� which� should� also� be� a� familiar� assumption�(e�g�,�ANOVA)��This�assumption�must�be�reframed�a�bit�in�the�regression�context� by�examining�the�concept�of�a�conditional distribution��In�regression�analysis,�a�conditional�
629Simple Linear Regression
distribution�is�defined�as�the�distribution�of�Y�for�a�particular�value�of�X��For�instance,�in� the�midterm�statistics�example,�we�could�consider�the�conditional�distribution�of�midterm� scores�when�GRE_Q�=�50;�in�other�words,�what�the�distribution�of�Y�looks�like�for�X�=�50��We� call�this�a�conditional�distribution�because�it�represents�the�distribution�of�Y�conditional�on�a� particular�value�of�X�(sometimes�denoted�as�Y|X,�read�as�Y�given�X)��Alternatively�we�could� examine�the�conditional�distribution�of�the�prediction�errors,�that�is,�the�distribution�of�the� prediction�errors�conditional�on�a�particular�value�of�X�(i�e�,�e|X,�read�as�e�given�X)��Thus,�the� homogeneity�assumption�is�that�the�conditional�distributions�have�a�constant�variance�for� all�values�of�X�
In�a�plot�of�the�Y�scores�or�the�residuals�versus�X,�the�consistency�of�the�variance�of�the�con- ditional�distributions�can�be�examined��A�common�violation�of�this�assumption�occurs�when� the�conditional�residual�variance�increases�as�X�increases��Here�the�residual�plot�is�cone-�or� fan-shaped,�where�the�cone�opens�toward�the�right��An�example�of�this�violation�would�be� where�weight�is�predicted�by�age,�as�weight�is�more�easily�predicted�for�young�children�than� it�is�for�adults��Thus,�residuals�would�tend�to�be�larger�for�adults�than�for�children�
If�the�homogeneity�assumption�is�violated,�estimates�of�the�standard�errors�are�larger,� and�although�the�regression�coefficients�remain�unbiased,�the�validity�of�the�significance� tests�is�affected��In�fact�with�larger�standard�errors,�it�is�more�difficult�to�reject�H0,�therefore� resulting� in� a� larger� number� of� Type� II� errors�� Minor� violations� of� this� assumption� will� have�a�small�net�effect;�more�serious�violations�occur�when�the�variances�are�greatly�dif- ferent��In�addition,�nonconstant�variances�may�also�result�in�the�conditional�distributions� being�nonnormal�in�shape�
If�the�homogeneity�assumption�is�seriously�violated,�the�simplest�solution�is�to�use�some� sort� of� transformation,� known� as� variance stabilizing transformations� (e�g�,� Weisberg,� 1985)�� Commonly� used� transformations� are� the� log� or� square� root� of� Y� (e�g�,� Kleinbaum� et� al�,� 1998)�� These� transformations� can� also� often� improve� on� the� nonnormality� of� the� conditional�distributions��However,�this�complicates�things�in�terms�of�dealing�with�trans- formed�variables�rather�than�the�original�variables��A�better�solution�is�to�use�generalized� or�weighted�least�squares�(e�g�,�Weisberg,�1985)��A�third�solution�is�to�use�a�form�of�robust� estimation�(e�g�,�Carroll�&�Ruppert,�1982;�Kleinbaum�et�al�,�1998;�Wilcox,�1996,�2003)�
17.3.7.3 Normality
The�third�assumption�of�normality�should�also�be�a�familiar�one��In�regression,�the�nor- mality�assumption�is�that�the�conditional�distributions�of�either�Y�or�the�prediction�errors� (i�e�,�residuals)�are�normal�in�shape��That�is,�for�all�values�of�X,�the�scores�on�Y�or�the�pre- diction� errors� are� normally� distributed�� Oftentimes� nonnormal� distributions� are� largely� a�function�of�one�or�a�few�extreme�observations,�known�as�outliers��Extreme�values�may� cause�nonnormality�and�seriously�affect�the�regression�results��The�regression�estimates� are� quite� sensitive� to� outlying� observations� such� that� the� precision� of� the� estimates� is� affected,� particularly� the� slope�� Also� the� coefficient� of� determination� can� be� affected�� In� general,� the� regression� line� will� be� pulled� toward� the� outlier,� because� the� least� squares� principle�always�attempts�to�find�the�line�that�best�fits�all�of�the�points�
Various�rules�of�thumb�are�used�to�crudely�detect�outliers�from�a�residual�plot�or�scatter- plot��A�commonly�used�rule�is�to�define�an�outlier�as�an�observation�more�than�two�or�three� standard�errors�from�the�mean�(i�e�,�a�large�distance�from�the�mean)��The�outlier�observation� may�be�a�result�of�(a)�a�simple�recording�or�data�entry�error,�(b)�an�error�in�observation,�(c) an� improperly�functioning�instrument,�(d)�inappropriate�use�of�administration�instructions,�or� (e)�a�true�outlier��If�the�outlier�is�the�result�of�an�error,�correct�the�error�if�possible�and�redo�the�
630 An Introduction to Statistical Concepts
regression�analysis��If�the�error�cannot�be�corrected,�then�the�observation�could�be�deleted��If� the�outlier�represents�an�accurate�observation,�then�this�observation�may�contain�important� theoretical� information,� and� one� would� be� more� hesitant� to� delete� it� (or� perhaps� seek� out� similar�observations)�
A� simple� procedure� to� use� for� single� case� outliers� (i�e�,� just� one� outlier)� is� to� perform� two�regression�analyses,�both�with�and�without�the�outlier�being�included��A�comparison� of� the� regression� results� will� provide� some� indication� of� the� effects� of� the� outlier�� Other� methods�for�detecting�and�dealing�with�outliers�are�available,�but�are�not�described�here� (e�g�,� Andrews� &� Pregibon,� 1978;� Barnett� &� Lewis,� 1978;� Beckman� &� Cook,� 1983;� Cook,� 1977;�Hawkins,�1980;�Kleinbaum�et�al�,�1998;�Mickey,�Dunn,�&�Clark,�2004;�Pedhazur,�1997;� Rousseeuw�&�Leroy,�1987;�Wilcox,�1996,�2003)�
How�does�one�go�about�detecting�violation�of�the�normality�assumption?�There�are�two� commonly�used�procedures��The�simplest�procedure�involves�checking�for�symmetry�in�a� histogram,�frequency�distribution,�boxplot,�or�skewness�and�kurtosis�statistics��Although� nonzero kurtosis� (i�e�,� a� distribution� that� is� either� flat,� platykurtic,� or� has� a� sharp� peak,� leptokurtic)�will�have�minimal�effect�on�the�regression�estimates,�nonzero skewness�(i�e�,� a� distribution� that� is� not� symmetrical� with� either� a� positive� or� negative� skew)� will� have� much�more�impact�on�these�estimates��Thus,�finding�asymmetrical�distributions�is�a�must�� One� rule� of� thumb� is� to� be� concerned� if� the� skewness� value� is� larger� than� 1�5� or� 2�0� in� magnitude��For�the�midterm�statistics�example,�the�skewness�value�for�the�raw�residuals�is� −0�2692��Thus,�there�is�evidence�of�normality�in�this�illustration�
Another� useful� graphical� technique� is� the� normal� probability� plot� [or� quantile–quan- tile� (Q–Q)� plot]�� With� normally� distributed� data� or� residuals,� the� points� on� the� normal� probability�plot�will�fall�along�a�straight�diagonal�line,�whereas�nonnormal�data�will�not�� There�is�a�difficulty�with�this�plot�because�there�is�no�criterion�with�which�to�judge�devia- tion�from�linearity��A�normal�probability�plot�of�the�raw�residuals�for�the�midterm�statis- tics�example�is�shown�in�Figure�17�4��Together�the�skewness�and�normal�probability�plot� results�indicate�that�the�normality�assumption�is�satisfied��It�is�recommended�that�skew- ness�and/or�the�normal�probability�plot�be�considered�at�a�minimum�
There� are� also� several� statistical� procedures� available� for� the� detection� of� nonnormal- ity�(e�g�,�Andrews,�1971;�Belsley,�Kuh,�&�Welsch,�1980;�Ruppert�&�Carroll,�1980;�Wu,�1985)�� In�addition,�various�transformations�are�available�to�transform�a�nonnormal�distribution�
FIGuRe 17.4 Normal� probability� plot� for� midterm� example�
1.0
0.8
0.6
0.4
Ex pe
ct ed
c um
ul at
iv e
pr ob
ab ili
ty
0.2
0.0 0.0 0.2 0.4 0.6
Observed cumulative probability 0.8 1.0
Dependent variable: midterm exam score
631Simple Linear Regression
into�a�normal�distribution��The�most�commonly�used�transformations�to�correct�for�non- normality� in� regression� analysis� are� to� transform� the� dependent� variable� using� the� log� (to�correct�for�positive�skew)�or�the�square�root�(to�correct�for�positive�or�negative�skew)�� However,� again� there� is� the� problem� of� dealing� with� transformed� variables� measured� along�some�other�scale�than�that�of�the�original�variables�
17.3.7.4 Linearity
The�fourth�assumption�is�linearity��This�assumption�simply�indicates�that�there�is�a�lin- ear�relationship�between�X�and�Y,�which�is�also�assumed�for�most�types�of�correlations�� Consider�the�scatterplot�and�regression�line�in�Figure�17�5�where�X�and�Y�are�not�linearly� related��Here�X�and�Y�form�a�perfect�curvilinear�relationship�as�all�of�the�points�fall�pre- cisely�on�a�curve��However,�fitting�a�straight�line�to�these�points�will�result�in�a�slope�of 0� not� useful� at� all� for� predicting� Y� from� X� (as� the� predicted� score� for� all� cases� will� be� the� mean�of�Y)��For�example,�age�and�performance�are�not�linearly�related�
If�the�relationship�between�X�and�Y�is�linear,�then�the�sample�slope�and�intercept�will� be�unbiased�estimators�of�the�population�slope� and�intercept,� respectively��The�linearity� assumption� is� important� because,� regardless� of� the� value� of� Xi,� we� always� expect� Yi� to� increase� by� bYX� units� for� a� one-unit� increase� in� Xi�� If� a� nonlinear� relationship� exists,� this� means�that�the�expected�increase�in�Yi�depends�on�the�value�of�Xi��Strictly�speaking,�lin- earity�in�a�model�refers�to�there�being�linearity�in�the�parameters�of�the�model�(i�e�,�slope� β�and�intercept�α)�
Detecting�violation�of�the�linearity�assumption�can�often�be�done�by�looking�at�the�scat- terplot�of�Y�versus�X��If�the�linearity�assumption�is�met,�we�expect�to�see�no�systematic�pat- tern�of�points��While�this�plot�is�often�satisfactory�in�simple�linear�regression,�less�obvious� violations� are� more� easily� detected� in� a� residual� plot�� If� the� linearity� assumption� is� met,� we� expect� to� see� a� horizontal� band� of� residuals� mainly� contained� within� ±2� or� ±3sres� (or� standard�errors)�across�the�values�of�X��If�the�assumption�is�violated,�we�expect�to�see�a� systematic�pattern�between�e�and�X��Therefore,�we�recommend�you�examine�both�the�scat- terplot�and�the�residual�plot��A�residual�plot�for�the�midterm�statistics�example�is�shown�in� Figure�17�6��Even�with�a�very�small�sample,�we�see�a�fairly�random�display�of�residuals�and� therefore�feel�fairly�confident�that�the�linearity�assumption�has�been�satisfied�
5.00
4.00
3.00Y
2.00
1.00
1.00 2.00 3.00 X
4.00 5.00 FIGuRe 17.5 Nonlinear�regression�example�
632 An Introduction to Statistical Concepts
�If�a�serious�violation�of�the�linearity�assumption�has�been�detected,�how�should�we�deal� with�it?�There�are�two�alternative�procedures�that�the�researcher�can�utilize,�transforma- tions�or�nonlinear models��The�first�option�is�to�transform�either�one�or�both�of�the�vari- ables�to�achieve�linearity��That�is,�the�researcher�selects�a�transformation�that�subsequently� results� in� a� linear� relationship� between� the� transformed� variables�� Then� the� method� of� least�squares�can�be�used�to�perform�a�linear�regression�analysis�on�the�transformed�vari- ables�� However,� when� dealing� with� transformed� variables� measured� along� a� different� scale,� results� need� to� be� described� in� terms� of� the� transformed� rather� than� the� original� variables��A�better�option�is�to�use�a�nonlinear�model�to�examine�the�relationship�between� the�variables�in�their�original�scale�(see�Wilcox,�1996,�2003;�also�discussed�in�Chapter�18)�
17.3.7.5 Fixed X
The�fifth�and�final�assumption�is�that�the�values�of�X�are�fixed��That�is,�X�is�a�fixed�variable� rather�than�a�random�variable��This�results�in�the�regression�model�being�valid�only�for� those�particular�values�of�X�that�were�actually�observed�and�used�in�the�analysis��Thus,� the�same�values�of�X�would�be�used�in�replications�or�repeated�samples��You�may�recall�a� similar�concept�in�the�fixed-effects�ANOVA�models�previously�considered�
Strictly�speaking,�the�regression�model�and�its�parameter�estimates�are�only�valid�for� those�values�of�X�actually�sampled��The�use�of�a�prediction�model,�based�on�one�sample� of� individuals,� to� predict� Y� for� another� sample� of� individuals� may� also� be� suspect�� Depending�on�the�circumstances,�the�new�sample�of�individuals�may�actually�call�for�a� different�set�of�parameter�estimates��Two�obvious�situations�that�come�to�mind�are�the� extrapolation�and�interpolation�of�values�of�X��In�general,�we�may�not�want�to�make� predictions�about�individuals�having�X�scores�(i�e�,�scores�on�the�independent�variable)� that�are�outside�of�the�range�of�values�used�in�developing�the�prediction�model;�this�is� defined�as�extrapolating�beyond�the�sample�predictor�data��We�cannot�assume�that�the� function� defined� by� the� prediction� model� is� the� same� outside� of� the� values� of� X� that� were�initially�sampled��The�prediction�errors�for�the�new�nonsampled�X�values�would� be�expected�to�be�larger�than�those�for�the�sampled�X�values�because�there�are�no�sup- portive�prediction�data�for�the�former�
FIGuRe 17.6 Residual�plot�for�midterm�example�
30.00 40.00 50.00 60.00 GRE_Q
70.00 80.00
–6.00000
–4.00000
–2.00000
0.00000
U ns
ta nd
ar di
ze d
re si
du al
2.00000
4.00000
633Simple Linear Regression
On�the�other�hand,�we�are�not�quite�as�concerned�in�making�predictions�about�individu- als�having�X�scores�within�the�range�of�values�used�in�developing�the�prediction�model;� this� is� defined� as� interpolating� within� the� range� of� the� sample� predictor� data�� We� would� feel�somewhat�more�comfortable�in�assuming�that�the�function�defined�by�the�prediction� model�is�the�same�for�other�new�values�of�X�within�the�range�of�those�initially�sampled�� For�the�most�part,�the�fixed�X�assumption�is�satisfied�if�the�new�observations�behave�like� those� in� the� prediction� sample�� In� the� interpolation� situation,� we� expect� the� prediction� errors� to� be� somewhat� smaller� as� compared� to� the� extrapolation� situation� because� there� are�at�least�some�similar�supportive�prediction�data�for�the�former��It�has�been�shown�that� when� other� assumptions� are� met,� regression� analysis� performs� just� as� well� when� X� is� a� random�variable�(e�g�,�Glass�&�Hopkins,�1996;�Myers�&�Well,�1995;�Pedhazur,�1997)��There� is�no�corresponding�assumption�about�the�nature�of�Y�
In� our� midterm� statistics� example,� we� have� more� confidence� in� our� prediction� for� a� GRE_Q�value�of�52�(which�did�not�occur�in�the�sample,�but�falls�within�the�range�of�sam- pled�values)�than�in�a�value�of�20�(which�also�did�not�occur,�but�is�much�smaller�than�the� smallest�value�sampled,�37)��In�fact,�this�is�precisely�the�rationale�underlying�the�PI�previ- ously�developed,�where�the�width�of�the�interval�increased�as�an�individual’s�score�on�the� predictor�(Xi)�moved�away�from�the�predictor�mean�(X
– )�
A�summary�of�the�assumptions�and�the�effects�of�their�violation�for�simple�linear�regres- sion�is�presented�in�Table�17�2�
17.3.7.6 Summary
The�simplest�procedure�for�assessing�assumptions�is�to�plot�the�residuals�and�see�what�the� plot�tells�you��Take�the�midterm�statistics�problem�as�an�example��Although�sample�size�is� quite�small�in�terms�of�looking�at�conditional�distributions,�it�would�appear�that�all�of�our� assumptions�have�been�satisfied��All�of�the�residuals�are�within�two�standard�errors�of�0,� and�there�does�not�seem�to�be�any�systematic�pattern�in�the�residuals��The�distribution�of� the�residuals�is�nearly�symmetrical,�and�the�normal�probability�plot�looks�good��The�scat- terplot�also�strongly�suggests�a�linear�relationship�
Table 17.2
Assumptions�and�Violation�of�Assumptions:�Simple�Linear�Regression
Assumption Effect of Assumption Violation
Independence •�Influences�standard�errors�of�the�model
Homogeneity •�Bias�in�s2res •�May�inflate�standard�errors�and�thus�increase�likelihood�of�a�Type�II�error •�May�result�in�nonnormal�conditional�distributions
Normality •�Less�precise�slope,�intercept,�and�R2
Linearity •�Bias�in�slope�and�intercept •�Expected�change�in�Y�is�not�a�constant�and�depends�on�value�of�X •�Reduced�magnitude�of�coefficient�of�determination
Values�of�X�fixed •��Extrapolating�beyond�the�range�of�X:�prediction�errors�larger,�may�also�bias�slope� and�intercept
•��Interpolating�within�the�range�of�X:�smaller�effects�than�when�extrapolating;�if� other�assumptions�met,�negligible�effect
634 An Introduction to Statistical Concepts
17.4 SPSS
Next�we�consider�SPSS�for�the�simple�linear�regression�model��Before�we�conduct�the�anal- ysis,�let�us�review�the�data��With�one�independent�variable�and�one�dependent�variable,�the� dataset�must�consist�of�two�variables�or�columns,�one�for�the�independent�variable�and�one� for�the�dependent�variable��Each�row�still�represents�one�individual,�with�the�value�of�the� independent�variable�for�that�particular�case�and�their�score�on�the�dependent�variable��In� the�following�screenshot,�we�see�the�SPSS�dataset�is�in�the�form�of�two�columns�represent- ing�one�independent�variable�(GRE_Q)�and�one�dependent�variable�(midterm�exam�score)�
�e independent variable is labeled
“GRE_Q” where each value represents the
student’s score on the GRE_Q.
�e dependent variable is “Midterm”
and represents the score on the midterm
exam.
Step 1:� To� conduct� a� simple� linear� regression,� go� to�“Analyze”� in� the� top� pulldown� menu,�then�select�“Regression,”�and�then�select�“Linear.”�Following�the�screenshot� (step�1)�as�follows�produces�the�“Linear Regression”�dialog�box�
A
B C
Simple linear regression: Step 1
635Simple Linear Regression
Step 2:� Click� the� dependent� variable� (e�g�,� “Midterm”)� and� move� it� into� the� “Dependent” box�by�clicking�the�arrow�button��Click�the�independent�variable�and� move�it�into�the�“Independent(s)”�box�by�clicking�the�arrow�button�(see�screenshot� step�2)�
Clicking on “Statistics” will
allow you to select various regression
coefficients and residuals.
Clicking on “Plots” will allow you to
select various residual plots.
Clicking on “Save” will allow you to
save various predicted values,
residuals, and other statistics
useful for diagnostics.
Select the dependent variable from the list on the
left and use the arrow to move it to the “Dependent”
box on the right.
Select the independent
variable from the list on the left and use the arrow to
move it to the “Independent(s)”
box on the right.
Simple linear regression: Step 2
Step 3:�From�the�“Linear Regression”�dialog�box�(see�screenshot�step�2),�clicking�on� “Statistics”�will�provide�the�option�to�select�various�regression�coefficients�and�residu- als��From�the�“Statistics”�dialog�box�(see�screenshot�step�3),�place�a�checkmark�in�the� box�next�to�the�following:�(1) estimates, (2)�confidence intervals,�(3)�model fit,� (4)�descriptives,� (5)�Durbin–Watson, and� (6)�casewise diagnostics.� Click� on� “Continue”�to�return�to�the�original�dialog�box�
636 An Introduction to Statistical Concepts
Simple linear regression: Step 3
Step 4:�From�the�“Linear Regression”�dialog�box�(see�screenshot�step�2),�click- ing� on� “Plots”� will� provide� the� option� to� select� various� residual� plots�� From� the� “Plots”�dialog�box,�place�a�checkmark�in�the�box�next�to�the�following:�(1) histo- gram� and� (2)�normal probability plot�� Click� on�“Continue”� to� return� to� the� original�dialog�box�
Simple linear regression: Step 4
637Simple Linear Regression
Step 5:� From� the�“Linear Regression”� dialog� box� (see� screenshot� step� 2),� clicking� on�“Save”� will� provide� the� option� to� save� various� predicted� values,� residuals,� and� sta- tistics� that� can� be� used� for� diagnostic� examination�� From� the�“Save”� dialog� box� under� the�heading�of�Predicted Values,�place�a�checkmark�in�the�box�next�to�the�following:� unstandardized.�Under�the�heading�of�Residuals,�place�a�checkmark�in�the�box�next� to� the� following:� (1)�unstandardized and� (2)�studentized. Under� the� heading� of� Distances, place�a�checkmark�in�the�box�next�to�the�following:�(1)�Mahalanobis�and� (2)�Cook’s. Under�the�heading�of�Influence Statistics,�place�a�checkmark�in�the� box� next� to� the� following:� (1)�DFBETA(s)� and� (2)�Standardized DFBETA(s).� Click� on� “Continue”�to�return�to�the�original�dialog�box��From�the�“Linear Regression”�dialog� box,�click�on�“OK”�to�return�to�generate�the�output�
Simple linear regression: Step 5
Interpreting the output: Annotated� results� are� presented� in� Table� 17�3�� In� Chapters� 18� and� 19,� we� see� other� regression� modules� in� SPSS� which� allow� you� to� consider,� for� example,� generalized� or� weighted� least� squares� regression,� nonlinear� regression,�and�logistic�regression��Additional�information�on�regression�analysis�in� SPSS�is�provided�in�texts�such�as�Morgan�and�Griego�(1998)�and�Meyers,�Gamst,�and� Guarino�(2006)�
638 An Introduction to Statistical Concepts
Table 17.3
Selected�SPSS�Results�for�the�Midterm�Example
Descriptive Statistics
Mean Std. Deviation N
Midterm exam score 38.0000 7.51295 10 GRE_Q 55.5000 13.13393 10
Correlations
Midterm Exam Score GRE_Q
Pearson correlation
Sig. (One-tailed)
N
Midterm exam score
GRE_Q Midterm exam score GRE_Q Midterm exam score GRE_Q
Variables Entered/Removeda
Model Variables Entered Variables Removed Method
1 GRE_Qb Enter a Dependent variable: midterm exam score. b All requested variables entered.
�e table labeled “Descriptive Statistics” provides basic descriptive statistics (means, standard deviations, and sample sizes) for the independent and dependent variables.
�e table labeled “Correlations” provides the correlation coefficient value (r = .918), p value (<.001), and sample size (N = 10) for the simple bivariate Pearson correlation between the independent and dependent variables. �ere is a statistically significant bivariate correlation between GRE_Q and midterm exam score.
“Variables Entered/ Removed” lists the independent variables included in the model and the method by which they were entered (i.e., “Enter”).
10 10
1.000
.918
.000
.
.918
1.000 .000
10 10
.
Model Summarya
Model R R Square Adjusted R
Square Std. Error of the Estimate
Durbin– Watson
1 .918b .842 .822 3.16540 1.287
“Adjusted R Square” is an estimate of how well the model would fit other data from the same population and is calculated as:
If an additional independent variable were entered in the model, an increase in adjusted R2 indicates the new variable is adding value to the model. Negative adjusted R2 values can occur and indicate the model fits the data VERY poorly.
R in simple linear regression is the simple bivariate
Pearson correlation between X and Y.
R2 in simple linear regression is the squared simple bivariate Pearson
correlation between X and Y. It represents the
proportion of variance in the dependent variable that is explained by the independent variable.
Durbin–Watson is a test of independence
of the residuals. Ranging from 0 to 4, values of 2 indicate uncorrelated errors. Values less than 1 or
greater than 3 indicate a likely
assumption violation.
R2adj = 1 – (1 – R2) n – 1
n – m – 1
b Predictors: (constant), GRE_Q. a Dependent variable: midterm exam score.
639Simple Linear Regression
Table 17.3 (continued)
Selected�SPSS�Results�for�the�Midterm�Example
ANOVAa
Model Sum of Squares df Mean Square F Sig. Regression 1
42.700 .000bResidual 8 427.842
10.020 1
Total
427.842 80.158
508.000 9
b Predictors: (constant), GRE-Q.
Total sum of squares is partitioned into SS regression and SS residual. When the regression SS equals 0, this indicates that the independent variable has provided no information in terms of explaining the dependent variable.
�e F statistic is computed as
The p value (.000) indicates we reject the null hypothesis. The prediction equation provides a better fit to the data than estimating the predicted value of Y to be equal to the mean of Y.
a Dependent variable: midterm exam score.
F = MSreg MSres
= 427.842 10.020
Coefficientsa
t b SEb
= = = .080
6.535.525
Unstandardized Coefficients
Standardized Coefficients
95.0% Confidence Interval for B
Model B Std. Error Beta t Sig.
Lower Bound
Upper Bound
(Constant)1 GRE_Q
8.865 .525
4.570 .080 .918
1.940 6.535
.088
.000 –1.673
.340 19.402
.710 a Dependent variable: midterm exam score.
a Dependent variable: midterm exam score.
Residuals Statisticsa
Minimum Maximum Mean Std. Deviation N Predicted value 10 Std. predicted value 10 Standard error of predicted value
10
Adjusted predicted value 10 Residual 10 Std. residual 10 Stud. residual 10 Deleted residual 10 Stud. deleted residual 10 Mahal. distance 10 Cook's distance 10 Centered leverage value
28.2882 –1.409
1.008
26.5379 –4.43800 –1.402 –1.568 –5.55197 –1.763
.013
.004
.001
49.2866 1.637 1.996
50.7968 3.71176 1.173 1.422 5.46209 1.539 2.680
.477
.298
38.0000 .000
1.380
37.9612 .00000 .000 .006 .03876
–.009 .900 .159 .100
6.89478 1.000
.333
7.24166 2.98436
.943 1.071 3.87616 1.135
.893
.157
.099 10
The “constant” is the intercept and tells us that if GRE_Q (the independent variable) was zero, the midterm exam score (the dependent variable) would be 8.865. �e “GRE_Q” is the slope and tells us that for a one point increase in GRE_Q, the midterm exam score will increase by about one half of one point.
�e test statistic, t, is calculated as the unstandardized coefficient divided by its standard error. �us for the slope, the test statistic is:
�e p value for the intercept (the “constant”) ( p = .088) indicates that the intercept is not statistically significantly different from 0 (this finding is usually of less interest than the slope). �e p value for GRE_Q (the independent variable) ( p = .000) indicates that the slope is statistically significantly different from 0.
“Residuals statistics” and related graphs (histogram and Q–Q plot, not shown here) will be examined in our discussionof assumptions.
640 An Introduction to Statistical Concepts
Examining Data for Assumptions in Simple Linear Regression
As� you� may� recall,� there� were� a� number� of� assumptions� associated� with� simple� linear� regression��These�included�the�following:�(a)�independence,�(b)�homogeneity�of�variance,� (c)� linearity,� and� (d)� normality�� Although� fixed� values� of� X� are� assumed,� this� is� not� an� assumption�that�can�be�tested�but�is�instead�related�to�the�use�of�the�results�(i�e�,�extrapola- tion�and�interpolation)�
Before�we�begin�to�examine�assumptions,�let�us�review�the�values�that�we�requested�to� be�saved�to�our�data�file�(see�dataset�screenshot�that�follows)�
� 1��PRE _ 1�are�the�unstandardized�predicted�values�(i�e�,�Y′i )�
� 2��RES _ 1� are� the� unstandardized� residuals,� simply� the� difference� between� the� observed� and� predicted� values�� For� student� 1,� for� example,� the� observed� value for�the�midterm�(i�e�,�the�dependent�variable)�was�32,�and�the�predicted� value�was�28�28824��Thus,�the�unstandardized�residual�is�simply�32�−�28�28824,� or�3�71176�
� 3��SRE _ 1� are� the� studentized� residuals,� a� type� of� standardized� residual� that� is� more� sensitive� to� outliers� as� compared� to� standardized� residuals�� Studentized� residuals�are�computed�as�the�unstandardized�residual�divided�by�an�estimate� of� the� standard� deviation� with� that� case� removed�� As� a� rule� of� thumb,� studen- tized� residuals� with� an� absolute� value� greater� than� 3� are� considered� outliers� (Stevens,�1984)�
� 4��MAH _ 1� are� Mahalanobis� distance� values� that� can� be� helpful� in� detecting� out- liers�� These� values� can� be� reviewed� to� determine� cases� that� are� exerting� lever- age�� Barnett� and� Lewis� (1994)� produced� a� table� of� critical� values� for� evaluating� Mahalanobis�distance��Squared�Mahalanobis�distances�divided�by�the�number�of� variables� (D2/df )� which� are� greater� than� 2�5� (for� small� samples)� or� 3–4� (for� large� samples)�are�suggestive�of�outliers�(Hair,�Black,�Babin,�Anderson,�&�Tatham,�2006)�� Later,� we� will� follow� another� convention� for� examining� these� values� using� the� chi-square�distribution�
� 5��COO _ 1�are�Cook’s�distance�values�and�provide�an�indication�of�influence�of�indi- vidual�cases��As�a�rule�of�thumb,�Cook’s�values�greater�than�1�0�suggest�that�case� is�potentially�problematic�
� 6��DFB0 _ 1�and�DFB1 _ 1�are�unstandardized�DFBETA�values�for�the�intercept�and� slope,�respectively��These�values�provide�estimates�of�the�intercept�and�slope�when� the�case�is�removed�
� 7��SDB0 _ 1� and�SDB1 _ 1� are� standardized� DFBETA� values� for� the� intercept� and� slope,� respectively,� and� are� easier� to� interpret� as� compared� to� their� unstandard- ized�counterparts��Standardized�DFBETA�values�greater�than�an�absolute�value�of� 2�suggest�that�the�case�may�be�exerting�undue�influence�on�the�parameters�of�the� model�(i�e�,�the�slope�and�intercept)�
641Simple Linear Regression
As we look at our raw data, we see nine new variables have been added to our dataset. These are our predicted values, residuals, and other diagnostic statistics. The residuals will
be used as diagnostics to review the extent to which our data meet the assumptions of simple linear regression.
1 2 3 4 5 6 7
Independence
We� now� plot� the� studentized� residuals� (which� were� requested� and� created� through� the� “Save”�option�mentioned�earlier)�against�the�values�of�X�to�examine�the�extent�to�which�inde- pendence�was�met��The�general�steps�for�generating�a�simple�scatterplot�through�“Scatter/ dot”�have�been�presented�in�a�previous�chapter�(e�g�,�Chapter�10),�and�they�will�not�be�reiter- ated�here��From�the�“Simple Scatterplot”�dialog�screen,�click�the�studentized�residual� variable�and�move�it�into�the�“Y Axis”�box�by�clicking�on�the�arrow��Click�the�independent� variable�X�and�move�it�into�the�“X Axis”�box�by�clicking�on�the�arrow��Then�click�“OK.”
Interpreting independence evidence:�If�the�assumption�of�independence�is�met,� the�points�should�fall�randomly�within�a�band�of�−2�0�to�+2�0��Here�we�have�evidence�of� independence,�especially�given�the�small�sample�size,�as�all�points�are�within�an�absolute� value�of�2�0�and�fall�relatively�randomly�
642 An Introduction to Statistical Concepts
2.00000
1.00000
.00000
St ud
en tiz
ed re
si du
al
–1.00000
–2.00000
30.00 40.00 50.00 60.00 GRE_Q
70.00 80.00
Homogeneity of Variance
We� can� use� the� same� plot� of� studentized� residuals� against� X� values� (used� earlier� for� inde- pendence)�to�examine�the�extent�to�which�homogeneity�was�met��Recall�that�homogeneity�is� when�the�dependent�variable�has�the�same�variance�for�all�values�of�the�independent�variable�� Evidence�of�meeting�the�assumption�of�homogeneity�is�a�plot�where�the�spread�of�residuals� appears�fairly�constant�over�the�range�of�X�values�(i�e�,�a�random�display�of�points)��If�the�spread� of�the�residuals�increases�or�decreases�across�the�plot�from�left�to�right,�this�may�indicate�that� the�assumption�of�homogeneity�has�been�violated��Here�we�have�evidence�of�homogeneity�
Linearity
Since�we�have�only�one�independent�variable,�a�simple�bivariate�scatterplot�of�the�depen- dent�variable�(on�the�Y�axis)�and�the�independent�variable�(on�the�X�axis)�will�provide�a� visual�indication�of�the�extent�to�which�linearity�is�reasonable��As�those�steps�have�been� presented�previously�in�the�discussion�of�independence,�they�will�not�be�repeated�here�� For�this�scatterplot,�there�is�a�general�positive�linear�relationship�between�the�variables�
50.00
45.00
40.00
35.00
M id
te rm
e xa
m sc
or e
30.00
25.00
30.00 40.00 50.00 60.00 GRE_Q
70.00 80.00
643Simple Linear Regression
Additionally,� the� plot� of� studentized� residuals� against� X� values� (used� earlier� for� inde- pendence)�can�be�used�to�examine�the�extent�to�which�linearity�was�met��We�highly�rec- ommend� examining� this� residual� plot� as� it� is� more� sensitive� to� detecting� independence� violations�� Here� a� random� display� of� points� within� an� absolute� value� of� 2� or� 3� suggests� further�evident�of�linearity�
Normality
Generating normality evidence:�Understanding�the�distributional�shape,�specifi- cally� the� extent� to� which� normality� is� a� reasonable� assumption,� is� important� in� simple� linear� regression� just� as� it� was� in� ANOVA� models�� We� again� examine� residuals� for� nor- mality,�following�the�same�steps�as�with�the�previous�ANOVA�designs��We�also�use�vari- ous�diagnostics�to�examine�our�data�for�influential�cases��Let�us�begin�by�examining�the� unstandardized� residuals� for� normality�� For� simple� linear� regression,� the� distributional� shape�of�the�unstandardized�residuals�should�be�a�normal�distribution��Because�the�steps� for�generating�normality�evidence�were�presented�previously�in�the�chapters�for�ANOVA� models,�they�will�not�be�provided�here�
Interpreting normality evidence:�By�now,�we�have�had�a�substantial�amount�of� practice� in� interpreting� quite� a� range� of� normality� statistics�� We� interpret� them� again� in� reference�to�the�assumption�of�normality�for�the�unstandardized�residuals�in�simple�linear� regression�
Mean 95% Confidence interval
5% Trimmed mean Median Variance Std. deviation Minimum Maximum Range Interquartile range Skewness Kurtosis
Lower bound Upper bound
Statistic Std. Error .0000000
–2.1348848 2.1348848
.0403471
.1626409 8.906
2.98436314 –4.43800
3.71176 8.14976 5.36232
–.269 –1.369 1.334
.687
.94373849
Descriptives
Unstandardized residual
for mean
The�skewness�statistic�of�the�residuals�is�−�269�and�kurtosis�is�−1�369—both�being�within� the�range�of�an�absolute�value�of�2�0,�suggesting�some�evidence�of�normality�
While�we�have�a�very�small�sample�size,�the�histogram�reflects�the�skewness�and�kur- tosis�statistics�
644 An Introduction to Statistical Concepts
4
3
2
Fr eq
ue nc
y
1
0 –6.00000 –4.00000 –2.00000
Unstandardized residual .00000 2.00000 4.00000
Histogram
Mean = 1.11E – 15 Std. dev. = 2.98436 N = 10
There�are�a�few�other�statistics�that�can�be�used�to�gauge�normality��The�formal�test� of� normality,� the� Shapiro–Wilk� (S–W)� test� (SW)� (Shapiro� &� Wilk,� 1965),� provides� evi- dence� of� the� extent� to� which� our� sample� distribution� is� statistically� different� from� a� normal�distribution��The�output�for�the�S–W�test�is�presented�as�follows�and�suggests� that� our� sample� distribution� for� the� residual� is� not� statistically� significantly� different� than�what�would�be�expected�from�a�normal�distribution�as�the�p�value�is�greater�than� α�(p�=��416)�
Tests of Normality
Statistic Statisticdf dfSig. Sig.
Shapiro–Wilk
.41610.927.200*10.150Unstandardized residual
a Lilliefors significance correction. *This is a lower bound of the true significance.
Kolmogorov–Smirnova
Q–Q�plots�are�also�often�examined�to�determine�evidence�of�normality��Q–Q�plots�graph� quantiles�of�the�theoretical�normal�distribution�against�quantiles�of�the�sample�distribu- tion�� Points� that� fall� on� or� close� to� the� diagonal� line� suggest� evidence� of� normality�� The� Q–Q�plot�of�residuals�shown�as�follows�suggests�relative�normality�
645Simple Linear Regression
2
1
0
Ex pe
ct ed
n or
m al
–1
–2
–5.0 –2.5 0.0 Observed value
2.5 5.0
Normal Q–Q plot of unstandardized residual
Examination� of� the� following� boxplot� also� suggests� a� relatively� normal� distributional� shape�of�residuals�with�no�outliers�
4.00000
2.00000
.00000
–2.00000
–4.00000
–6.00000
Unstandardized residual
Considering�the�forms�of�evidence�we�have�examined,�skewness�and�kurtosis�statistics,� the�S–W�test,�histogram,�the�Q–Q�plot,�and�the�boxplot,�all�suggest�normality�is�a�reason- able�assumption��We�can�be�reasonably�assured�we�have�met�the�assumption�of�normality� of�the�residuals�
Screening Data for Influential Points
Casewise diagnostics:� Recall� that� we� requested� a� number� of� statistics� to� help� us� in�diagnostics�and�screening�our�data��One�that�we�requested�was�for�“Casewise�diag- nostics�”�If�there�were�any�cases�with�large�values�for�the�standardized�residual�(more�
646 An Introduction to Statistical Concepts
than� three� standard� deviations),� there� would� have� been� information� in� our� output� to� indicate�the�case�number�and�values�of�the�standardized�residual,�predicted�value,�and� unstandardized�residual��This�information�is�useful�for�more�closely�examining�case(s)� with�extreme�standardized�residuals�
Cook’s distance: Cook’s� distance� provides� an� overall� measure� for� the� influence� of� individual� cases�� Values� greater� than� one� suggest� that� the� case� may� be� problematic� in� terms�of�undue�influence�on�the�model��In�examining�the�residual�statistics�provided�in�the� following�output,�we�see�that�the�maximum�value�for�Cook’s�distance�is��477,�well�under� the�point�at�which�we�should�be�concerned�
Predicted value Std. predicted value Standard error of predicted value
Adjusted predicted value Residual Std. residual
Stud. residual Deleted residual
Stud. deleted residual Mahal. distance Cook’s distance Centered leverage value
a Dependent variable: midterm exam score.
Residuals Statisticsa
Minimum Maximum Mean Std. Deviation N
28.2882 49.2866
1.637
1.996
50.7968 3.71176 1.173 1.422 5.46209 1.539 2.680 .477 .298 .100
.159 .900
–.009 .03876 .006 .000 .00000
37.9612
1.380
.000
38.0000 6.89478
1.000
.333
7.24166 2.98436
.943 1.071 3.87616 1.135
.893
.157
.099 10 10 10 10 10 10 10 10 10
10
10
10
–1.409
1.008
26.5379 –4.43800 –1.402 –1.568 –5.55197 –1.763
.013 .004 .001
Mahalanobis distances:� Mahalanobis� distances� are� measures� of� the� distance� from� each�case�to�the�mean�of�the�independent�variable�for�the�remaining�cases��We�can�use�the� value� of� Mahalanobis� distance� as� a� test� statistic� value� using� the� chi-square� distribution�� With� only� one� independent� variable� and� one� dependent� variable,� we� have� two� degrees� of� freedom�� Given� an� alpha� level� of� �05,� the� chi-square� critical� value� is� 5�99�� Thus,� any� Mahalanobis�distance�greater�than�5�99�suggests�that�case�is�an�outlier��With�a�maximum� distance�of�2�680�(see�previous�table),�there�is�no�evidence�to�suggest�there�are�outliers�in� our�data�
DFBETA:�We�also�asked�to�save�DFBETA�values��These�values�provide�another�indication� of�the�influence�of�cases��The�DFBETA�provides�information�on�the�change�in�the�predicted� value�when�the�case�is�deleted�from�the�model��For�standardized�DFBETA�values,�values� greater� than� an� absolute� value� of� 2�0� should� be� examined� more� closely�� Looking� at� the� minimum�(−�87682)�and�maximum�(�62542)�DFBETA�values�for�the�slope�(i�e�,�GRE_Q),�we� do�not�have�any�cases�that�suggest�undue�influence�
647Simple Linear Regression
N Minimum
Descriptive Statistics
Maximum Mean Std. Deviation
DFBETA GRE_Q Standardized DFBETA GRE_Q Valid N (listwise) 10
10 10 –.06509 .04470
.62542 –.0275752 .47302980 .03608593–.0021866
–.87682
17.5 G*Power
A� priori� and� post� hoc� power� could� again� be� determined� using� the� specialized� software� described� previously� in� this� text� (e�g�,� G*Power);� alternatively,� you� can� consult� a� priori� power� tables� (e�g�,� Cohen,� 1988)�� As� an� illustration,� we� use� G*Power� to� compute� the� post� hoc�power�of�our�test�
Post Hoc Power for Simple Linear Regression Using G*Power
The�first�thing�that�must�be�done�when�using�G*Power�to�compute�post�hoc�power�is�to� select�the�correct�test�family��Here�we�conducted�simple�linear�regression��To�find�regres- sion,� select� “Tests”� in� the� top� pulldown� menu,� then� “Correlation and regres- sion,”�and�then�“Linear bivariate regression: One group, size of slope.”� Once�that�selection�is�made,�the�“Test family”�automatically�changes�to�“t tests.”
A
B
C
Step 1
The�“Type of Power Analysis”�desired�then�needs�to�be�selected��To�compute�post� hoc�power,�select�“Post hoc: Compute achieved power—given α, sample size, and effect size.”
648 An Introduction to Statistical Concepts
Click on “Determine” to pop out the effect size calculator box
(shown below). This will allow you to
compute the effect size, “Slope H1.”
Once the parameters are
specified, click on “Calculate.”
The default selection for “Test Family” is
“t tests” and this is the appropriate test
family for linear regression.
Change the statistical test to “Linear bivariate regression: One group, size of slope.”
The “Input Parameters” for computing post hoc power must be specified including: 1. number of tails (i.e., directionality of the test) 2. effect size, slope H1 3. α level 4. total sample size 5. Slope H0 (i.e., null) 6. standard deviation of X (estimated from sample) 7. standard deviation of Y (estimated from sample)
Step 2
The�“Input Parameters”�must�then�be�specified��In�our�example,�we�conducted�a�two- tailed�test��We�will�compute�the�effect�size,�Slope H1,�last,�so�we�skip�that�for�the�moment�� The�alpha�level�we�used�was��05,�and�the�total�sample�size�was�10��The�Slope H0�is�the�slope� specified�in�the�null�hypothesis—thus�a�value�of�0��The�last�two�parameters�to�be�specified� are�for�the�standard�deviation�of�X,�the�independent�variable,�and�the�standard�deviation� of�Y,�the�dependent�variable�
We�skipped�filling�in�the�second�parameter,�the�effect�size,�Slope H1,�for�a�reason��We�will�use� the�pop-out�effect�size�calculator�in�G*Power�to�compute�the�effect�size�Slope H1��To�pop�out� the�effect�size�calculator,�click�on�“Determine”�displayed�under�“Input Parameters.” In�the�pop-out�effect�size�calculator,�click�the�toggle�menu�to�select�ρ,�σ_x,�σ_y�=>�slope��Input� the�values�for�the�correlation�coefficient�of�X�and�Y,�the�standard�deviation�of�X,�and�the�stan- dard�deviation�of�Y��Click�on�“Calculate”�in�the�pop-out�effect�size�calculator�to�compute� the�effect�size�Slope H1��Then�click�on�“Calculate and Transfer to Main Window”� to�transfer�the�calculated�effect�size�(i�e�,�1�604822)�to�the�“Input Parameters.”�Once�the� parameters�are�specified,�click�on�“Calculate”�to�find�the�power�statistics�
649Simple Linear Regression
Post hoc power
Here are the post-hoc power results.
The�“Output Parameters”�provide�the�relevant�statistics�given�the�input�just�specified�� Here�we�were�interested�in�determining�post�hoc�power�for�simple�linear�regression�with�a� two-tailed�test,�a�computed�effect�size�Slope H1�of�1�6048220,�an�alpha�level�of��05,�total�sample� size�of�10,�a�hypothesized�null�slope�of�0,�a�standard�deviation�of�X�of�7�51295,�and�a�standard� deviation�of�Y�of�13�13393��Based�on�those�criteria,�the�post�hoc�power�for�the�simple�linear� regression�was��9999926��In�other�words,�for�these�conditions�the�post�hoc�power�of�our�sim- ple�linear�regression�was�nearly�1�00—the�probability�of�rejecting�the�null�hypothesis�when�it� is�really�false�(in�this�case,�the�probability�that�the�slope�is�0)�was�around�the�maximum�(i�e�,� 1�00)�(sufficient�power�is�often��80�or�above)��Keep�in�mind�that�conducting�power�analysis�a� priori�is�recommended�so�that�you�avoid�a�situation�where,�post�hoc,�you�find�that�the�sample� size�was�not�sufficient�to�reach�the�desired�level�of�power�(given�the�observed�parameters)�
A Priori Power for Simple Linear Regression Using G*Power
For�a�priori�power,�we�can�determine�the�total�sample�size�needed�for�simple�linear�regres- sion�given�the�directionality�of�the�test,�an�estimated�effect�size�Slope H1,�α�level,�desired� power,�slope�for�the�null�hypothesis�(i�e�,�0),�and�the�standard�deviations�of�X�and�Y��We� follow�Cohen’s�(1988)�conventions�for�effect�size�(i�e�,�small�r�=��10;�moderate�r�=��30;�large� r�=��50)��In�this�example,�had�we�wanted�to�determine�a�priori�power�and�had�estimated�a� moderate�effect�r�of��30,�α�of��05,�desired�power�of��80,�null�slope�of�0,�and�standard�devia- tion�of�5�for�both�the�X�and�Y,�we�would�need�a�total�sample�size�of�82�
650 An Introduction to Statistical Concepts
A Priori power
Here are the post-hoc power results.
17.6 Template and APA-Style Write-Up
Finally,�here�is�an�example�paragraph�for�the�results�of�the�simple�linear�regression�analy- sis�� Recall� that� our� graduate� research� assistant,� Marie,� was� assisting� the� associate� dean� in�Graduate�Student�Services,�Randall��Randall�wanted�to�know�if�midterm�exam�scores� could�be�predicted�by�the�quantitative�subtest�of�the�required�graduate�entrance�exam,�the� GRE_Q��The�research�question�presented�to�Randall�from�Marie�included�the�following:� Can midterm exam scores be predicted from the GRE_Q?
Marie� then� assisted� Randall� in� generating� a� simple� linear� regression� model� as� the� test� of� inference��A�template�for�writing�the�research�question�for�this�design�is�presented�as�follows:
• Can [dependent variable] be predicted from [independent variable]?
It� may� be� helpful� to� preface� the� results� of� the� simple� linear� regression� with� information� on� an� examination� of� the� extent� to� which� the� assumptions� were� met�� The� assumptions� include� (a)� independence,� (b)� homogeneity� of� variance,� (c)� normality,� (d)� linearity,� and� (e) fixed�values�of�X�
A simple linear regression analysis was conducted to determine if midterm exam scores (dependent variable) could be predicted from GRE _ Q scores (independent variable). The null hypothesis tested was
651Simple Linear Regression
that the regression coefficient (i.e., the slope) was equal to 0. The data were screened for missingness and violation of assumptions prior to analysis. There were no missing data.
Linearity: The scatterplot of the independent variable (GRE _ Q) and the dependent variable (midterm exam scores) indicates that the assump- tion of linearity is reasonable—as GRE _ Q increases, midterm exam scores generally increase as well. With a random display of points falling within an absolute value of 2, a scatterplot of unstandardized residuals against values of the independent variable provided further evidence of linearity.
Normality: The assumption of normality was tested via examination of the unstandardized residuals. Review of the S–W test for nor- mality (SW = .927, df = 10, p = .416) and skewness (−.269) and kur- tosis (−1.369) statistics suggested that normality was a reasonable assumption. The boxplot suggested a relatively normal distribu- tional shape (with no outliers) of the residuals. The Q–Q plot and histogram suggested normality was reasonable.
Independence: A relatively random display of points in the scatterplot of studentized residuals against values of the independent variable provided evidence of independence. The Durbin–Watson statistic was computed to evaluate independence of errors and was 1.287, which is considered acceptable. This suggests that the assumption of indepen- dent errors has been met.
Homogeneity of variance: A relatively random display of points, where the spread of residuals appears fairly constant over the range of values of the independent variable (in the scatterplot of studentized residuals against values of the independent variable) provided evidence of homogeneity of variance.
Here�is�an�APA-style�example�paragraph�of�results�for�the�simple�linear�regression�analysis� (remember� that� this� will� be� prefaced� by� the� previous� paragraph� reporting� the� extent� to� which�the�assumptions�of�the�test�were�met)�
The results of the simple linear regression suggest that a signifi- cant proportion of the total variation in midterm scores was pre- dicted by GRE _ Q. In other words, a student’s score on the GRE _ Q is a good predictor of their midterm exam grade, F(1, 8) = 42.700, p < .001. Additionally, we find the following: (a) the unstandardized slope (.525) and standardized slope (.918) are statistically signifi- cantly different from 0 (t = 6.535, df = 8, p < .001); with every one point increase in the GRE _ Q, midterm exam scores will increase by approximately one half of one point; (b) the CI around the unstan- dardized slope does not include 0 (.340, .710), further confirm- ing that GRE _ Q is a statistically significant predictor of midterm scores; and (c) the intercept (or average midterm exam score when
652 An Introduction to Statistical Concepts
GRE _ Q is 0) was 8.865. Multiple R squared indicates that approxi- mately 84% of the variation in midterm scores was predicted by GRE _ Q scores. According to Cohen (1988), this suggests a large effect.
17.7 Summary
In�this�chapter,�the�method�of�simple�linear�regression�was�described��First�we�discussed� the�basic�concepts�of�regression�such�as�the�slope�and�intercept��Next,�a�formal�introduc- tion� to� the� population� simple� linear� regression� model� was� given�� These� concepts� were� then�extended�to�the�sample�situation�where�a�more�detailed�discussion�was�given��In�the� sample�context,�we�considered�unstandardized�and�standardized�regression�coefficients,� errors�in�prediction,�the�least�squares�criterion,�the�coefficient�of�determination,�tests�of�sig- nificance,�and�a�discussion�of�statistical�assumptions��At�this�point,�you�should�have�met� the�following�objectives:�(a)�be�able�to�understand�the�concepts�underlying�simple�linear� regression,�(b)�be�able�to�determine�and�interpret�the�results�of�simple�linear�regression,� and� (c)� be� able� to� understand� and� evaluate� the� assumptions� of� simple� linear� regression�� Chapter�18�follows�up�with�a�description�of�multiple�regression�analysis,�where�regression� models�are�developed�based�on�two�or�more�predictors�
Problems
Conceptual problems
17.1� A�regression�intercept�represents�which�one�of�the�following? � a�� The�slope�of�the�line � b�� The�amount�of�change�in�Y�given�a�one-unit�change�in�X � c�� The�value�of�Y�when�X�is�equal�to�0 � d�� The�strength�of�the�relationship�between�X�and�Y 17.2� The�regression�line�for�predicting�final�exam�grades�in�history�from�midterm�scores�
in�the�same�course�is�found�to�be�Y′�=��61�X�+�3�12��If�the�value�of�X�increases�from�74� to�75,�the�value�of�Y�will�do�which�one�of�the�following?
� a�� Increase�by��61�points � b�� Increase�by�1�00�points � c�� Increase�by�3�12�points � d�� Decrease�by��61�points 17.3� The�regression�line�for�predicting�salary�of�principals�from�cumulative�GPA�in�gradu-
ate�school�is�found�to�be�Y′�=�35,000�X�+�37,000��What�does�the�value�of�37,000�represent? � a�� Average�cumulative�GPA � b�� The�criterion�value � c�� The�mean�salary�of�principals�when�cumulative�GPA�is�0 � d�� The�standardized�regression�coefficient�given�an�intercept�of�0
653Simple Linear Regression
17.4� The�regression�line�for�predicting�salary�of�principals�from�cumulative�GPA�in�gradu- ate�school�is�found�to�be�Y′�=�35,000X�+�37,000��What�does�the�value�of�35,000�represent?
� a�� The�amount�of�change�in�Y�given�a�one-unit�change�in�X � b�� The�correlation�between�X�and�Y � c�� The�intercept�value � d�� The�value�of�Y�when�X�is�equal�to�0 17.5� You� are� given� that� μX� =� 14,� σ2X� =� 36,� μY� =� 14,� σ2Y� =� 49,� and� Y� =� 14� is� the� prediction�
equation� for� predicting� Y� from� X�� Which� of� the� following� is� the� variance� of� the� predicted�values�of�Y′?
� a�� 0 � b�� 14 � c�� 36 � d�� 49 17.6� In�regression�analysis,�the�prediction�of�Y�is�most�accurate�for�which�of�the�following�
correlations�between�X�and�Y? � a�� −�90 � b�� −�30 � c�� +�20 � d�� +�80 17.7� If�the�relationship�between�two�variables�is�linear,�then�which�one�of�the�following�is�
correct? � a�� All�of�the�points�must�fall�on�a�curved�line� � b�� The�relationship�is�best�represented�by�a�curved�line� � c�� All�of�the�points�must�fall�on�a�straight�line� � d�� The�relationship�is�best�represented�by�a�straight�line� 17.8� If�both�X�and�Y�are�measured�on�a�z�score�scale,�the�regression�line�will�have�a�slope�
of�which�one�of�the�following? � a�� 0�00 � b�� +1�or�−1 � c�� rXY � d�� sY/sX 17.9� �If�the�simple�linear�regression�equation�for�predicting�Y�from�X�is�Y′�=�25,�then�the�
correlation�between�X�and�Y�is�which�one�of�the�following? � a�� 0�00 � b�� 0�25 � c�� 0�50 � d�� 1�00 17.10� Which�one�of�the�following�is�correct�for�the�unstandardized�regression�slope? � a�� It�may�never�be�negative� � b�� It�may�never�be�greater�than�+1�00� � c�� It�may�never�be�greater�than�the�correlation�coefficient�rXY� � d�� None�of�the�above�
654 An Introduction to Statistical Concepts
17.11� �If�two�individuals�have�the�same�score�on�the�predictor,�their�residual�scores�will�be� which�one�of�the�following?
� a�� Be�necessarily�equal
� b�� Depend�only�on�their�observed�scores�on�Y
� c�� Depend�only�on�their�predicted�scores�on�Y
� d�� �Depend� only� on� the� number� of� individuals� that� have� the� same� predicted� score
17.12� �If�rXY�=��6,�the�proportion�of�variation�in�Y�that�is�not�predictable�from�X�is�which�one� of�the�following?
� a�� �36
� b�� �40
� c�� �60
� d�� �64
17.13� Homogeneity�assumes�which�one�of�the�following?
� a�� The�range�of�Y�is�the�same�as�the�range�of�X�
� b�� The�X�and�Y�distributions�have�the�same�mean�values�
� c�� The�variability�of�the�X�and�the�Y�distributions�is�the�same�
� d�� The�conditional�variability�of�Y�is�the�same�for�all�values�of�X�
17.14� �Which�one�of�the�following�is�suggested�to�examine�the�extent�to�which�homogeneity� of�variance�has�been�met?
� a�� Scatterplot�of�Mahalanobis�distances�against�standardized�residuals
� b�� Scatterplot�of�studentized�residuals�against�unstandardized�predicted�values
� c�� Simple�bivariate�correlation�between�X�and�Y
� d�� S–W�test�results�for�the�unstandardized�residuals
17.15� �Which�one�of�the�following�is�suggested�to�examine�the�extent�to�which�normality� has�been�met?
� a�� Scatterplot�of�Mahalanobis�distances�against�standardized�residuals
� b�� Scatterplot�of�studentized�residuals�against�unstandardized�predicted�values
� c�� Simple�bivariate�correlation�between�X�and�Y
� d�� S–W�test�results�for�the�unstandardized�residuals
17.16� The�linear�regression�slope�bYX�represents�which�one�of�the�following?
� a�� Amount�of�change�in�X�expected�from�a�one-unit�change�in�Y
� b�� Amount�of�change�in�Y�expected�from�a�one-unit�change�in�X
� c�� Correlation�between�X�and�Y
� d�� Error�of�estimate�of�Y�from�X
17.17� �If� the� correlation� between� X� and� Y� is� 0,� then� the� best� prediction� of� Y� that� can� be� made�is�the�mean�of�Y��True�or�false?
17.18� �If�X�and�Y�are�highly�nonlinear,�linear�regression�is�more�useful�than�the�situation� where�X�and�Y�are�highly�linear��True�or�false?
655Simple Linear Regression
17.19� �If� the� pretest� (X)� and� the� posttest� (Y)� are� positively� correlated,� and� your� friend� receives�a�pretest�score�below�the�mean,�then�the�regression�equation�would�predict� that�your�friend�would�have�a�posttest�score�that�is�above�the�mean��True�or�false?
17.20� �Two� variables� are� linearly� related� so� that� given� X,� Y� can� be� predicted� without� error��I�assert�that�rXY�must�be�equal�to�either�+1�0�or�−1�0��Am�I�correct?
17.21� �I� assert� that� the� simple� regression� model� is� structured� so� that� at� least� two� of� the� actual�data�points�will�necessarily�fall�on�the�regression�line��Am�I�correct?
Computational problems
17.1� You�are�given�the�following�pairs�of�scores�on�X�(number�of�hours�studied)�and� Y�(quiz�score)�
X Y
4 5 4 6 3 4 7 8 2 4
� a�� Find�the�linear�regression�model�for�predicting�Y�from�X� � b�� �Use� the� prediction� model� obtained� to� predict� the� value� of� Y� for� a� new� person�
who�has�a�value�of�6�for�X� 17.2� You�are�given�the�following�pairs�of�scores�on�X�(preschool�social�skills)�and�Y�(receptive�
vocabulary�at�the�end�of�kindergarten)�
X Y
25 60 30 45 42 56 45 58 36 42 50 38 38 35 47 45 32 47 28 57 31 56
� a�� Find�the�linear�regression�model�for�predicting�Y�from�X� � b�� Use�the�prediction�model�obtained�to�predict�the�value�of�Y�for�a�new�child�who�
has�a�value�of�48�for�X� 17.3� The�prediction�equation�for�predicting�Y�(pain�indicator)�from�X�(drug�dosage)�is�
Y�=�2�5�X�+�18��What�is�the�observed�mean�for�Y�if�μX�=�40�and�σ2X�=�81?
656 An Introduction to Statistical Concepts
17.4� You� are� given� the� following� pairs� of� scores� on� X� (number� of� years� working)� and� Y (number�of�raises)�
X Y
2 2 2 1 1 1 1 1 3 5 4 4 5 7 5 6 7 7 6 8 4 3 3 3 6 6 6 6 8 10 9 9
10 6 9 6 4 9 4 10
Perform�the�following�computations�using�α�=��05� � a�� The�regression�equation�of�Y�predicted�by�X� � b�� Test�of�the�significance�of�X�as�a�predictor� � c�� Plot�Y�versus�X� � d�� Compute�the�residuals� � e�� Plot�residuals�versus�X�
Interpretive problems
17.1� �With�the�class�survey�1�dataset�on�the�website,�your�task�is�to�use�SPSS�to�find�a�suit- able�single�predictor�of�current�GPA��In�other�words,�select�several�potential�predic- tors� that� seem� reasonable,� and� conduct� a� simple� linear� regression� analysis� for� each� of�those�predictors�individually��Which�of�those�is�the�best�predictor�of�current�GPA?� What�is�the�interpretation�of�the�effect�size?�Write�up�the�results�following�APA�
17.2� With� the� class� survey� 1� dataset� on� the� website,� your� task� is� to� use� SPSS� to� find� a� suitable�single�predictor�of�the�number�of�hours�exercised�per�week��In�other�words,� select�several�potential�predictors�that�seem�reasonable,�and�conduct�a�simple�linear� regression� analysis� for� each� of� those� predictors� individually�� Which� of� those� is� the� best�predictor�of�the�number�of�hours�of�exercise?�What�is�the�interpretation�of�the� effect�size?�Write�up�the�results�following�APA�
657
18 Multiple Regression
Chapter Outline
18�1� Partial�and�Semipartial�Correlations 18�1�1� Partial�Correlation 18�1�2� Semipartial�(Part)�Correlation
18�2� Multiple�Linear�Regression 18�2�1� Unstandardized�Regression�Model 18�2�2� Standardized�Regression�Model 18�2�3� Coefficient�of�Multiple�Determination�and�Multiple�Correlation 18�2�4� Significance�Tests 18�2�5� Assumptions
18�3� Methods�of�Entering�Predictors 18�3�1� Backward�Elimination 18�3�2� Forward�Selection 18�3�3� Stepwise�Selection 18�3�4� All�Possible�Subsets�Regression 18�3�5� Hierarchical�Regression 18�3�6� Commentary�on�Sequential�Regression�Procedures
18�4� Nonlinear�Relationships 18�5� Interactions 18�6� Categorical�Predictors 18�7� SPSS 18�8� G*Power 18�9� Template�and�APA-Style�Write-Up
Key Concepts
� 1�� Partial�and�semipartial�(part)�correlations � 2�� Standardized�and�unstandardized�regression�coefficients � 3�� Coefficient�of�multiple�determination�and�multiple�correlation
658 An Introduction to Statistical Concepts
In� Chapter� 17,� our� concern� was� with� the� prediction� or� explanation� of� a� dependent� or� cri- terion� variable� (Y)� by� a� single� independent� or� predictor� variable� (X)�� However,� given� the� types� of� phenomena� we� typically� deal� with� in� education� and� the� behavioral� sciences,� the� use�of�a�single�predictor�variable�is�quite�restrictive��In�other�words,�given�the�complexity�of� most�human,�organizational,�and�animal�behaviors,�one�predictor�is�usually�not�sufficient� in� terms� of� understanding� the� criterion�� In� order� to� account� for� a� sufficient� proportion� of� variability�in�the�criterion,�more�than�one�predictor�is�necessary��This�leads�us�to�analyze�the� data�via�multiple�regression�analysis�where�two�or�more�predictors�are�used�to�predict�or� explain�the�criterion�variable��Here�we�adopt�the�usual�notation�where�the�X’s�are�defined�as� the�independent�or�predictor�variables,�and�Y�as�the�dependent�or�criterion�variable�
For�example,�our�admissions�officer�might�want�to�use�more�than�just�Graduate�Record� Exam�(GRE)�scores�to�predict�graduate-level�grade�point�averages�(GPAs)�to�make�admis- sions�decisions�for�a�sample�of�applicants�to�your�favorite�local�university�or�college��Other� potentially�useful�predictors�might�be�undergraduate�grade�point�averages�(UGPAs),�rec- ommendation� letters,� writing� samples,� and/or� an� evaluation� from� a� personal� interview�� The�research�question�of�interest�would�now�be,�how�well�do�the�GRE,�UGPAs,�recommen- dations,�writing�samples,�and/or�interview�scores�(the�independent�or�predictor�variables)� predict� performance� in� graduate� school� (the� dependent� or� criterion� variable)?� This� is� an� example� of� a� situation� where� multiple� regression� analysis� using� multiple� predictor� vari- ables�might�be�the�method�of�choice�
Most� of� the� concepts� used� in� simple� linear� regression� from� Chapter� 17� carry� over� to� multiple� regression� analysis�� This� chapter� considers� the� concepts� of� partial,� semipartial,� and�multiple�correlations,�standardized�and�unstandardized�regression�coefficients,�and� the�coefficient�of�multiple�determination,�as�well�as�introduces�a�number�of�other�types�of� regression�models��Our�objectives�are�that�by�the�end�of�this�chapter,�you�will�be�able�to�(a)� determine�and�interpret�the�results�of�partial�and�semipartial�correlations,�(b)�understand� the�concepts�underlying�multiple�linear�regression,�(c)�determine�and�interpret�the�results� of� multiple� linear� regression,� (d)� understand� and� evaluate� the� assumptions� of� multiple� linear�regression,�and�(e)�have�a�basic�understanding�of�other�types�of�regression�models�
18.1 Partial and Semipartial Correlations
Marie�has�developed�into�quite�a�statistics�guru��We�see�in�this�chapter�that�her�statistical� prowess�has�garnered�her�repeat�business�
As�you�may�recall�from�the�previous�chapter,�Randall,�an�associate�dean�in�the�Graduate� Student�Services�office,�was�assisted�by�Marie�in�determining�if�the�GRE-Quantitative� (GRE-Q)� can� be� used� to� predict� midterm� grades�� Having� had� such� a� good� expe- rience� in� working� with� Marie,� Randall� has� requested� that� Jennifer,� the� assistant� dean� in� the� Graduate� Student� Services� office,� seek� advice� from� Marie� on� a� special� project�� Jennifer�is�interested�in�estimating�the�extent�to�which�GGPA�can�be�predicted�by�scores� on�the�overall�GRE�total�and�UGPA��Marie�suggests�the�following�research�question�to� Jennifer:�Can GGPA be predicted by scores on the overall GRE total and UGPA?�Marie�deter- mines�that�a�multiple�linear�regression�is�the�appropriate�statistical�procedure�to�use�to� answer�Jennifer’s�question��Marie�then�proceeds�to�assist�Jennifer�in�analyzing�the�data�
659Multiple Regression
Prior�to�a�discussion�of�regression�analysis,�we�need�to�consider�two�related�concepts�in� correlational� analysis,� partial� and� semipartial� correlations�� Multiple� regression� analysis� involves�the�use�of�two�or�more�predictor�variables�and�one�criterion�variable;�thus,�there� are�at�a�minimum�three�variables�involved�in�the�analysis��If�we�think�about�these�vari- ables�in�the�context�of�the�Pearson�correlation,�we�have�a�problem�because�this�correlation� can�only�be�used�to�relate�two�variables�at�a�time��How�do�we�incorporate�additional�vari- ables�into�a�correlational�analysis?�The�answer�is�through�partial�and�semipartial�correla- tions,�and�later�in�this�chapter,�multiple�correlations�
18.1.1 partial Correlation
First� we� discuss� the� concept� of� partial correlation�� The� simplest� situation� consists� of� three�variables,�which�we�label�X1,�X2,�and�X3��Here�an�example�of�a�partial�correlation� would�be�the�correlation�between�X1�and�X2�where�X3�is�held�constant�(i�e�,�controlled�or� partialled�out)��That�is,�the�influence�of�X3�is�removed�from�both�X1�and�X2�(both�have� been� adjusted� for� X3)�� Thus,� the� partial� correlation� here� represents� the� linear� relation- ship�between�X1�and�X2�independent�of�the�linear�influence�of�X3��This�particular�partial� correlation� is� denoted� by� r12�3,� where� the� X’s� are� not� shown� for� simplicity� and� the� dot� indicates�that�the�variables�preceding�it�are�to�be�correlated�and�the�variable(s)�following� it�are�to�be�partialled�out��We�compute�r12�3�as�follows:
r r r r
r r 12 3
12 13 23
13 2
23 21 1
. ( )( )
= −
− −
Let�us�take�an�example�of�a�situation�where�a�partial�correlation�might�be�computed��Say� a� researcher� is� interested� in� the� relationship� between� height� (X1)� and� weight� (X2)�� The� sample�consists�of�individuals�ranging�in�age�(X3)�from�6�months�to�65�years��The�sample� correlations�are�for�height�(X1)�and�weight�(X2),�r12�=��7;�height�(X1)�and�age�(X3),�r13�=��1;�and� weight�(X2)�and�age�(X3),�r23�=��6��We�compute�the�correlation�between�height�and�weight,� controlling�for�age,�r12�3,�as�follows:
r r r r
r r 12 3
12 13 23
13 2
23 21 1
. ( )( )
.7 (.1)(.6) (1 .01)(1 .36)
= −
− − =
− − −
== .8040
We�see�here�that�the�bivariate�correlation�between�height�and�weight,�ignoring�age�(r12�=��7),� is� smaller� than� the� partial� correlation� between� height� and� weight� controlling� for� age� (r12�3�=��8040)��That�is,�the�relationship�between�height�and�weight�is�stronger�when�age� is�held�constant�(i�e�,�for�a�particular�age)�than�it�is�across�all�ages��Although�we�often� talk�about�holding�a�particular�variable�constant,�in�reality�variables�such�as�age�cannot� be�held�constant�artificially�
Some�rather�interesting�partial�correlation�results�can�occur�in�particular�situations��At� one�extreme,�if�both�the�correlation�between�height�(X1)�and�age�(X3),�r13,�and�weight�(X2)� and� age� (X3),� r23,� equal� 0,� then� the� correlation� between� height� (X1)� and� weight� (X2)� will� equal�the�partial�correlation�between�height�and�weight�controlling�for�age,�r12�=�r12�3��That� is,�if�the�variable�being�partialled�out�is�uncorrelated�with�each�of�the�other�two�variables,� then� the� partialling� process� will� logically� not� have� any� effect�� At� the� other� extreme,� if� either� r13� or� r23� equals� 1,� then� r12�3� cannot� be� calculated� as� the� denominator� is� equal� to� 0�
660 An Introduction to Statistical Concepts
(in other� words,� at�least� one� of� the�terms� in� the�denominator�is� equal� to�0�which� results� in� the� product� of� the� two� terms� in� the� denominator� equaling� 0� and� thus� a� denominator� of 0—and�you�cannot�divide�by�0)��Thus,�in�this�situation�(where�either�r13�or�r23�is�perfectly� correlated� at� 1�0),� the� partial� correlation� (i�e�,� r12�3,� partial� correlation� between� height� and� weight�controlling�for�age)�is�not�defined��Later�in�this�chapter,�we�refer�to�this�as�perfect� collinearity,�which�is�a�serious�problem��In�between�these�extremes,�it�is�possible�for�the� partial� correlation�to�be�greater�than�or�less�than�its�corresponding�bivariate�correlation� (including�a�change�in�sign)�and�even�for�the�partial�correlation�to�be�equal�to�0�when�its� bivariate�correlation�is�not��For�significance�tests�of�partial�and�semipartial�correlations,�we� refer�you�to�your�favorite�statistical�software�
18.1.2 Semipartial (part) Correlation
Next�the�concept�of�semipartial correlation�(also�called�a�part correlation)�is�discussed�� The� simplest� situation� consists� again� of� three� variables,� which� we� label� X1,� X2,� and� X3�� Here�an�example�of�a�semipartial�correlation�would�be�the�correlation�between�X1�and�X2� where�X3�is�removed�from�X2�only��That�is,�the�influence�of�X3�is�removed�from�X2�only�� Thus,� the� semipartial� correlation� here� represents� the� linear� relationship� between� X1� and�X2�after�that�portion�of�X2�that�can�be�linearly�predicted�from�X3�has�been�removed� from�X2��This�particular�semipartial�correlation�is�denoted�by�r1(2�3),�where�the�X’s�are�not� shown� for� simplicity� and� within� the� parentheses,� the� dot� indicates� that� the� variable(s)� following�it�are�to�be�removed�from�the�variable�preceding�it��Another�use�of�the�semi- partial�correlation�is�when�we�want�to�examine�the�predictive�power�in�the�prediction� of�Y�from�X1�after�removing�X2�from�the�prediction��A�method�for�computing�r1(2�3)�is� as�follows:
r r r r
r 1 2 3
12 13 23
23 21
( . ) ( )
= − −
Let�us�take�an�example�of�a�situation�where�a�semipartial�correlation�might�be�computed�� Say�a�researcher�is�interested�in�the�relationship�between�GPA�(X1)�and�GRE�scores�(X2)��The� researcher�would�like�to�remove�the�influence�of�intelligence�(IQ:�X3)�from�GRE�scores�but� not�from�GPA��The�simple�bivariate�correlation�between�GPA�and�GRE�is�r12�=��5;�between� GPA� and� IQ� is� r13� =� �3;� and� between� GRE� and� IQ� is� r23� =� �7�� We� compute� the� semipartial� correlation�that�removes�the�influence�of�intelligence�(IQ:�X3)�from�GRE�scores�(X2)�but�not� from�GPA�(X1)�(i�e�,�r1(2�3))�as�follows:
r r r r
r 1 2 3
12 13 23
23 21
5 3 7 1 49
4061( . ) ( )
. (. )(. ) .
.= − −
= −
− =
Thus,� the� bivariate� correlation� between� GPA� (X1)� and� GRE� scores� (X2)� ignoring� IQ� (X3)� (r12� =� �50)� is� larger� than� the� semipartial� correlation� between� GPA� and� GRE� controlling� for�IQ� in� GRE�(r1(2�3)� =��4061)�� As�was� the�case� with� partial�correlations,� various� values� of� a�semipartial�correlation�can�be�obtained�depending�on�the�combination�of�the�bivariate� correlations��For�more�information�on�partial�and�semipartial�correlations,�see�Hays�(1988),� Glass�and�Hopkins�(1996),�or�Pedhazur�(1997)�
661Multiple Regression
Now�that�we�have�considered�the�correlational�relationships�among�two�or�more�vari- ables� (i�e�,� partial� and� semipartial� correlations),� let� us� move� on� to� an� examination� of� the� multiple�regression�model�where�there�are�two�or�more�predictor�variables�
18.2 Multiple Linear Regression
Let�us�take�the�concepts�we�have�learned�in�this�and�the�previous�chapter�and�place�them� into�the�context�of�multiple�linear�regression��For�purposes�of�brevity,�we�do�not�consider� the�population�situation�because�the�sample�situation�is�invoked�99�44%�of�the�time��In�this� section,�we�discuss�the�unstandardized�and�standardized�multiple�regression�models,�the� coefficient�of�multiple�determination,�multiple�correlation,�tests�of�significance,�and�statis- tical�assumptions�
18.2.1 unstandardized Regression Model
The�sample�multiple�linear�regression�model�for�predicting�Y�from�m�predictors�X1,2,…,�m�is
Y b X b X b X a ei i i m mi i= + + + + +1 1 2 2 �
where Y�is�the�criterion�variable�(also�known�as�the�dependent�variable) Xk’s�are�the�predictor�(or�independent)�variables�where�k�=�1,…,�m bk�is�the�sample�partial�slope�of�the�regression�line�for�Y�as�predicted�by�Xk a�is�the�sample�intercept�of�the�regression�line�for�Y�as�predicted�by�the�set�of�Xk’s ei� represents� the� residuals� or� errors� of� prediction� (the� part� of� Y� not� predictable� from�
the�Xk’s) i represents�an�index�for�an�individual�or�object��The�index�i�can�take�on�values�from�1�to�n�
where�n�is�the�size�of�the�sample�(i�e�,�i�=�1,…,�n)
The�term�partial slope�is�used�because�it�represents�the�slope�of�Y�for�a�particular�Xk�in� which�we�have�partialled�out�the�influence�of�the�other�Xk’s,�much�as�we�did�with�the�par- tial�correlation�
The�sample�prediction�model�is
Y b X b X b X ai i i m mi′ = + + + +1 1 2 2 �
where�Y′i�is�the�predicted�value�of�Y�for�specific�values�of�the�Xk’s,�and�the�other�terms�are� as� before�� The� difference� between� the� regression� and� prediction� models� is� the� same� as� in�Chapter�17��We�can�compute�residuals,�the�ei,�for�each�of�the�i�individuals�or�objects�by� comparing�the�actual�Y�values�with�the�predicted�Y�values�as
e Y Yi i i= − ’
for�all�i�=�1,…,�n�individuals�or�objects�in�the�sample�
662 An Introduction to Statistical Concepts
Determining�the�sample�partial�slopes�and�the�intercept�in�the�multiple�predictor�case� is� rather� complicated�� To� keep� it� simple,� we� use� a� two-predictor� model� for� illustrative� purposes��Generally�we�rely�on�statistical�software�for�implementing�multiple�regression� analysis��For�the�two-predictor�case,�the�sample�partial�slopes�(b1�and�b2)�and�the�intercept� (a)�can�be�determined�as�follows:
b r r r s
r s Y Y Y
1 1 2 12
12 2
11 =
− −
( ) ( )
b r r r s
r s
a Y b X b X
Y Y Y 2
2 1 12
12 2
2
1 1 2 2
1 =
− −
= − −
( ) ( )
The�sample�partial�slope�b1�is�referred�to�alternately�as�(a)�the�expected�or�predicted�change� in�Y�for�a�one-unit�change�in�X1�with�X2�held�constant�(or�for�individuals�with�the�same� score�on�X2)�and�(b)�the�unstandardized�or�raw�regression�coefficient�for�X1��Similar�state- ments�may�be�made�for�b2��Note�the�similarity�of�the�partial�slope�equation�to�the�semipar- tial�correlation��The�sample�intercept�is�referred�to�as�the�value�of�the�dependent�variable�Y� when�the�values�of�the�independent�variables�X1�and�X2�are�both�0�
An�alternative�method�for�computing�the�sample�partial�slopes�that�involves�the�use�of� a�partial�correlation�is�as�follows:
b r s r
s r Y
Y Y 1 1 2
2 2
1 12 2
1
1 =
− −
.
b r s r
s r Y
Y Y 2 2 1
1 2
2 12 2
1
1 =
− −
.
What�statistical�criterion�is�used�to�arrive�at�the�particular�values�for�the�partial�slopes�and� intercept?�The�criterion�usually�used�in�multiple�linear�regression�analysis�[and�in�all�general� linear�models�(GLM)�for�that�matter]�is�the�least�squares�criterion��The�least�squares�criterion� arrives�at�those�values�for�the�partial�slopes�and�intercept�such�that�the�sum�of�the�squared� prediction� errors� or� residuals� is� smallest�� That� is,� we� want� to� find� that� regression� model,� defined�by�a�particular�set�of�partial�slopes�and�an�intercept,�which�has�the�smallest�sum�of� the�squared�residuals��We�often�refer�to�this�particular�method�for�calculating�the�slope�and� intercept�as�least�squares�estimation�because�a�and�the�bk’s�represent�sample�estimates�of�the� population�parameters�α�and�the�βk’s,�which�are�obtained�using�the�least�squares�criterion�� Recall�from�simple�linear�regression�that�the�residual�is�simply�the�vertical�distance�from�the� observed�value�of�Y�to�the�predicted�value�of�Y,�and�the�line�of�best�fit�minimizes�this�dis- tance��This�concept�still�applies�to�multiple�linear�regression�with�the�exception�that�we�are� now�in�a�three-dimensional�(or�more)�plane�given�there�are�multiple�independent�variables�
Consider�now�the�analysis�of�a�realistic�example�we�will�follow�in�this�chapter��We�use� the� GRE� Quantitative� +� Verbal� Total� (GRETOT)� and� undergraduate� grade� point� average� (UGPA)�to�predict�graduate�grade�point�average�(GGPA)��GRETOT�has�a�possible�range�of� 40–160�points�(if�we�remove�the�unnecessary�last�digit�of�0),�and�GPA�is�defined�as�having� a�possible�range�of�0�00–4�00�points��Given�the�sample�of�11�statistics�students�as�shown�in� Table�18�1,�let�us�work�through�a�multiple�linear�regression�analysis�
663Multiple Regression
As� sample� statistics,� we� compute� for� GRETOT� (X1� or� subscript� 1)� that� the� mean� is� X –
1�=�112�7273�and�the�variance�is�s12�=�266�8182,�for�UGPA�(X2�or�subscript�2)�that�the�mean�is� X –
2�=�3�1091�and�the�variance�is�s22�=�0�1609,�and�for�GGPA�(Y),�a�mean�of�Y – �=�3�5000�and�vari-
ance�of�sY 2�=�0�1100��In�addition,�we�compute�the�bivariate�correlation�between�the�depen-
dent�variable�(GGPA)�and�GRE�total,�rY1�=��7845;�between�the�dependent�variable�(GGPA)� and�UGPA,�rY2�=��7516;�and�between�GRE�total�and�UGPA,�r12�=��3011��The�sample�partial� slopes�(b1�and�b2)�and�intercept�(a)�are�determined�as�follows:
b r r r s
r s Y Y Y
1 1 2 12
12 2
11 7845 7516 3011 3317
1 =
− −
= − −
( ) ( )
[. (. )(. )]. ( .. ) .
. 3011 16 3346
01252 =
b r r r s
r s Y Y Y
2 2 1 12
12 2
21 7516 7845 3011 3317
1 =
− −
= −
− ( )
( ) [. (. )(. )].
( .. ). .
3011 4011 46872 =
a Y b X b X= − − = − − =1 1 2 2 3 5000 0125 112 7273 4687 3 1091 63. (. )( . ) (. )( . ) . 337
Let� us� interpret� the� partial� slope� and� intercept� values�� A� partial� slope� of� �0125� for� GRETOT�would� mean�that�if�your�score�on�the�GRETOT�was�increased�by�one�point,� then�your�GGPA�would�be�increased�by��0125�points,�controlling�for�UGPA��Likewise,� a�partial�slope�of��4687�for�UGPA�would�mean�that�if�your�UGPA�was�increased�by�one� point,� then� your� GGPA� would� be� increased� by� �4687� points,� controlling� for� GRETOT�� An�intercept�of��6337�would�mean�that�if�your�scores�on�the�GRETOT�and�UGPA�were� both�0,�then�your�GGPA�would�be��6337��However,�it�is�impossible�to�obtain�a�GRETOT� score�of�0�because�you�receive�40�points�for�putting�your�name�on�the�answer�sheet��In� a�similar�way,�an�undergraduate�student�could�not�obtain�a�UGPA�of�0�and�be�admit- ted�to�graduate�school��This�is�not�to�say�that�the�regression�equation�is�incorrect�but� just� to� point� out� how� the� interpretation� of� “GRETOT� and� UGPA� were� both� 0”� is� a� bit� meaningless�in�context�
To�put�all�of�this�together�then,�the�sample�multiple�linear�regression�model�is
Y b X b X a e X X ei i i i i i i= + + + = + + +1 1 2 2 1 2125 4687 6337. . .0
Table 18.1
GRE–GPA�Example�Data
Student GRE Total (X1) UGPA (X2) GGPA (Y)
1 145 3�2 4�0 2 120 3�7 3�9 3 125 3�6 3�8 4 130 2�9 3�7 5 110 3�5 3�6 6 100 3�3 3�5 7 95 3�0 3�4 8 115 2�7 3�3 9 105 3�1 3�2
10 90 2�8 3�1 11 105 2�4 3�0
664 An Introduction to Statistical Concepts
If�your�score�on�the�GRETOT�was�130�and�your�UGPA�was�3�5,�then�your�predicted�score� on�the�GGPA�would�be�computed�as�follows:
Yi′ = + + =. . . . .0 0 000125 (13 ) 4687(3 5 ) 6337 3 8992
Based� on� the� prediction� equation,� we� predict� your� GGPA� to� be� around� 3�9;� however,� as� we�saw�in�Chapter�17,�predictions�are�usually�somewhat�less�than�perfect,�even�with�two� predictors�
18.2.2 Standardized Regression Model
Up� until� this� point� in� the� chapter,� everything� in� multiple� linear� regression� analysis� has� involved�the�use�of�raw�scores��For�this�reason,�we�referred�to�the�model�as�the�unstandard- ized�regression�model��Often�we�may�want�to�express�the�regression�in�terms�of�standard� z�score�units�rather�than�in�raw�score�units�(as�in�Chapter�17)��The�means�and�variances�of� the�standardized�variables�(e�g�,�z1,�z2,�zY)�are�0�and�1,�respectively��The�sample�standard- ized�linear�prediction�model�becomes�the�following:
z Y b z b z b zi i i m mi( ) * * ... *′ = + + +1 1 2 2
where�bk*�represents�a�sample�standardized�partial�slope�(sometimes�called�beta�weights)� and�the�other�terms�are�as�before��As�was�the�case�in�simple�linear�regression,�no�intercept� term�is�necessary�in�the�standardized�prediction�model�as�the�mean�of�the�z�scores�for�all� variables� is� 0�� (Recall� that� the� intercept� is� the� value� of� the� dependent� variable� when� the� scores�on�the�independent�variables�are�all�0��Thus,�in�a�standardized�prediction�model,� the�dependent�variable�will�equal�0�when�the�values�of�the�independent�variables�are�equal� to�their�means—i�e�,�0�)�The�sample�standardized�partial�slopes�are,�in�general,�computed� by�the�following�equation:
b b s s
k k k
Y
* =
For�the�two-predictor�case,�the�standardized�partial�slopes�can�be�calculated�by
b b s sY
1 1 1* =
or
b r r r
r Y Y
1 1 2 12
12 21
* ( )
= − −
and
b b s sY
2 2 2* =
665Multiple Regression
or
b r r r
r Y Y
2 2 1 12
12 21
* ( )
= − −
If� the� two� predictors� are� uncorrelated� (i�e�,� r12� =� 0),� then� the� standardized� partial� slopes� are� equal�to�the�simple�bivariate�correlations�between�the�dependent�variable�and�the�independent� variables�(i�e�,�b rY1* = 1�and�b rY2* = 2)�because�the�rest�of�the�equation�goes�away��For�example,
b r r r
r r r
rY Y Y Y Y1 1 2 12
12 2
1 2 1
1 0
1 0 *
( ) ( )
( ) =
− −
= −
− =
For�our�GGPA�example,�the�standardized�partial�slopes�are�equal�to
b b s sY
1 1 1 0125 16 3346 3317 6156* . ( . . ) .= = =/
b b s sY
2 2 2 4687 4011 3317 5668* . (. . ) .= = =/
The�prediction�model�is�then
z Y z zi i i( ) . .′ = +6156 56681 2
The�standardized�partial�slope�of��6156�for�GRETOT�would�be�interpreted�as�the�expected� increase�in�GGPA�in�z�score�units�for�a�one�z�score�unit�increase�in�the�GRETOT,�controlling� for�UGPA��A�similar�statement�may�be�made�for�the�standardized�partial�slope�of�UGPA�� The�bk*�can�also�be�interpreted�as�the�expected�standard�deviation�change�in�the�dependent� variable�Y�associated�with�a�one�standard�deviation�change�in�the�independent�variable�Xk� when�the�other�Xk’s�are�held�constant�
When� would� you� want� to� use� the� standardized� versus� unstandardized� regression� analy- ses?�According�to�Pedhazur�(1997),�bk*�is�sample�specific�and�is�not�very�stable�across�different� samples� due� to� the� variance� of� Xk� changing� (as� the� variance� of� Xk� increases,� the� value� of�bk*� also� increases,� all� else� being� equal)�� For� example,� at� Ivy-Covered� University,�bk*� would� vary� across�different�graduating�classes�(or�samples)�while�bk�would�be�much�more�consistent�across� classes��Thus,�most�researchers�prefer�the�use�of�bk�to�compare�the�influence�of�a�particular�pre- dictor�variable�across�different�samples�and/or�populations��Pedhazur�also�states�that�the�bk*� is�of�“limited�value”�(p��321),�but�could�be�reported�along�with�the�bk��As�Pedhazur�and�others� have�reported,�the�bk*�can�be�deceptive�in�determining�the�relative�importance�of�the�predic- tors�as�they�are�affected�by�the�variances�and�covariances�of�both�the�included�predictors�and� the�predictors�not�included�in�the�model��Thus,�we�recommend�the�bk�for�general�purpose�use�
18.2.3 Coefficient of Multiple determination and Multiple Correlation
An�obvious�question�now�is,�how�well�is�the�criterion�variable�predicted�or�explained�by� the�set�of�predictor�variables?�For�our�example,�we�are�interested�in�how�well�the�GGPAs� (the� dependent� variable)� are� predicted� by� the� GRE� total� scores� and� the� UGPAs�� In� other� words,�what�is�the�utility�of�the�set�of�predictor�variables?
666 An Introduction to Statistical Concepts
The�simplest�method�involves�the�partitioning�of�the�familiar�total�sum�of�squares� in Y,�which�we�denote�as�SStotal��In�multiple�linear�regression�analysis,�we�can�write�SStotal� as�follows:
SS n Y Y ntotal i i= − / 2 2[ ( ) ]Σ Σ
or
SS n stotal Y= −( )1 2
where�we�sum�over�Y�from�i�=�1,…,�n��Next�we�can�conceptually�partition�SStotal�as
SS SS SStotal reg res= +
Σ Σ Σ( ) ( ) ( )Y Y Y Y Y Yi i i i− = − + − 2 2 2′ ′
where SSreg�is�the�regression�sum�of�squares�due�to�the�prediction�of�Y�from�the�Xk’s�(often�writ-
ten�as�SSY′) SSres�is�the�sum�of�squares�due�to�the�residuals
Before�we�consider�computation�of�SSreg�and�SSres,�let�us�look�at�the�coefficient�of�multiple� determination��Recall�from�Chapter�17�the�coefficient�of�determination,�rXY
2 ��Now�consider� the�multiple�predictor�version�of�rXY
2 ,�here�denoted�as�RY m. ,...,1 2 ��The�subscript�tells�us�that�
Y�is�the�criterion�(or�dependent)�variable�and�that�X1,…,�m�are�the�predictor�(or�independent)� variables��The�simplest�procedure�for�computing�R2�is�as�follows:
R b r b r b rY m Y Y m Ym. ,..., * * *
1 2
1 2= + + +1 2 �
The�coefficient�of�multiple�determination�tells�us�the�proportion�of�total�variation�in�the� dependent� variable� Y� that� is� predicted� from� the� set� of� predictor� variables� (i�e�,� X1,…,m’s)�� Often�we�see�the�coefficient�in�terms�of�SS�as
R SS SSY m reg total. ,...,1 2 = /
Thus,� one� method� for� computing� the� sums� of� squares� regression� and� residual,� SSreg� and� SSres,�is�from�the�coefficient�of�multiple�determination,�R2,�as�follows:
SS R SSreg total= 2
SS R SS SS SSres total total reg= − = −( )1 2
As�discussed�in�Chapter�17,�there�is�no�objective�gold�standard�as�to�how�large�the�coef- ficient�of�determination�needs�to�be�in�order�to�say�a�meaningful�proportion�of�varia- tion� has� been� predicted�� The� coefficient� is� determined� not� just� by� the� quality� of� the�
667Multiple Regression
predictor�variables�included�in�the�model�but�also�by�the�quality�of�relevant�predictor� variables�not�included�in�the�model,�as�well�as�by�the�amount�of�total�variation�in�the� dependent�variable�Y��However,�the�coefficient�of�determination�can�be�used�as�a�mea- sure�of�effect�size��According�to�the�subjective�standard�of�Cohen�(1988),�a�small�effect� size�is�defined�as�R2�=��10,�a�medium�effect�size�as�R2�=��30,�and�a�large�effect�size�as�R2� =��50��For�additional�information�on�effect�size�measures�in�regression,�we�suggest�you� consider�Steiger�and�Fouladi�(1992),�Mendoza�and�Stafford�(2001),�and�Smithson�(2001;� which�also�includes�some�discussion�of�power)��Note�also�that�RY�1,�…,�m�is�referred�to�as� the�multiple correlation coefficient�so�as�not�to�confuse�it�with�a�simple�bivariate�correla- tion�coefficient�
With� the� example� of� predicting� GGPA� from� GRETOT� and� UGPA,� let� us� examine� the� partitioning�of�the�total�sum�of�squares�SStotal�as�follows:
SS n stotal Y= − = =( 1) 1 11 1 1 2 ( ) . .0 00 000
Next,�we�can�determine�the�multiple�correlation�coefficient�R2�as
R b r b r b rY m Y Y m Ym. ,..., * * * . . . .1 2
1 2= + + … + = +1 2 6156( 7845) 5668( 75166) 9 89= . 0
We�can�also�partition�SStotal�into�SSreg�and�SSres,�where
SS R SSreg total= = = 2 9 89(1 1 ) 9998. . .0 000 0
SS R SSres total= − = − =( ) ( . ) . .1 1 9 89 1 1 1 2 2 0 000 00
Finally,�let�us�summarize�these�results�for�the�example�data��We�found�that�the�coefficient� of�multiple�determination�(R2)�was�equal�to��9089��Thus,�the�GRE�total�score�and�the�UGPA� predict�around�91%�of�the�variation�in�the�GGPA��This�would�be�quite�satisfactory�for�the� college� admissions� officer� in� that� there� is� little� variation� left� to� be� explained,� although� this� result� is� quite� unlikely� in� actual� research� in� education� and� the� behavioral� sciences�� Obviously�there�is�a�large�effect�size�here�
It� should� be� noted� that� R2� is� sensitive� to� sample� size� and� to� the� number� of� predic- tor�variables��As�sample�size�and/or�the�number�of�predictor�variables�increase,�R2�will� increase� as� well�� R� is� a� biased� estimate� of� the� population� multiple� correlation� due� to� sampling�error�in�the�bivariate�correlations�and�in�the�standard�deviations�of�X�and�Y�� Because�R�systematically�overestimates�the�population�multiple�correlation,�an�adjusted� coefficient� of� multiple� determination� has� been� devised�� The� adjusted� R2(Radj
2 )� is� calcu- lated�as�follows:
R R n
n m adj 2 21 1
1 1
= − − −
− −
( )
Thus,� Radj 2 � adjusts� for� sample� size� and� for� the� number� of� predictors� in� the� model;� this�
allows� us� to� compare� models� fitted� to� the� same� set� of� data� with� different� numbers� of� predictors�or�with�different�samples�of�data��The�difference�between�R2�and�Radj
2 �is�called� shrinkage�
668 An Introduction to Statistical Concepts
When�n�is�small�relative�to�m,�the�amount�of�bias�can�be�large�as�R2�can�be�expected�to� be�large�by�chance�alone��In�this�case,�the�adjustment�will�be�quite�large,�as�it�should�be��In� addition,�with�small�samples,�the�regression�coefficients�(i�e�,�the�bk’s)�may�not�be�very�good� estimates�of�the�population�values��When�n�is�large�relative�to�m,�bias�will�be�minimized� and�generalizations�are�likely�to�be�better�about�the�population�values�
With�a�large�number�of�predictors,�power�is�reduced,�and�there�is�an�increased�like- lihood� of� a� Type� I� error� across� the� total� number� of� significance� tests� (i�e�,� one� for� each� predictor�and�overall,�as�we�show�in�the�next�section)��In�multiple�regression,�power�is�a� function�of�sample�size,�the�number�of�predictors,�the�level�of�significance,�and�the�size� of� the� population� effect� (i�e�,� for� a� given� predictor,� or� overall)�� To� determine� how� large� a� sample� you� need� relative� to� the� number� of� predictors,� we� suggest� that� you� consult� power�tables�(e�g�,�Cohen,�1988)�or�power�software�(e�g�,�Murphy�&�Myors,�2004;�Power� and�Precision;�G*Power)��Simple�advice�is�to�design�your�research�such�that�the�ratio�of� n�to�m�is�large�
For�the�example�data,�we�determine�the�adjusted�multiple�coefficient�of�determination� Radj
2 �to�be
R R n
n m adj 2 21 1
1 1
1 1 9089 11 1
11 2 1 = − −
− − −
= − −
− − −
=( ) ( . ) .88861
which,�in�this�case,�indicates�a�very�small�adjustment�in�comparison�to�R2�
18.2.4 Significance Tests
Here�we�describe�two�procedures�used�in�multiple�linear�regression�analysis��These�involve� testing�the�significance�of�the�overall�regression�model�and�of�each�individual�partial�slope� (or�regression�coefficient)�
18.2.4.1 Test of Significance of Overall Regression Model
The�first�test�is�the�test�of�significance�of�the�overall�regression�model,�or�alternatively�the� test�of�significance�of�the�coefficient�of�multiple�determination��This�is�a�test�of�all�of�the� bk’s� simultaneously,� an� examination� of� overall� model� fit� of� the� independent� variables� in� aggregate��The�null�and�alternative�hypotheses,�respectively,�are�as�follows:
H k0 0: β β β1 2= = = =�
H k1 not all the: β = 0
If�H0�is�rejected,�then�one�or�more�of�the�individual�regression�coefficients�(i�e�,�the�bk)�is�sta- tistically�significantly�different�from�0�(if�the�assumptions�are�satisfied,�as�discussed�later)�� If�H0�is�not�rejected,�then�none�of�the�individual�regression�coefficients�will�be�significantly� different�from�0�
The�test�is�based�on�the�following�test�statistic:
F R m
R n m =
− − −
2
21 1 /
( )/( )
669Multiple Regression
where F�indicates�that�this�is�an�F�statistic m�is�the�number�of�predictors�or�independent�variables n�is�the�sample�size
The�F�test�statistic�is�compared�to�the�F�critical�value,�always�a�one-tailed�test�(by�default,� this�value�can�never�be�negative�given�the�terms�in�the�equation,�so�this�will�always�be�a� nondirectional� test)� and� at� the� designated� level� of� significance,� with� degrees� of� freedom� being�m�and�(n − m�−�1),�as�taken�from�the�F�table�in�Table�A�4��That�is,�the�tabled�critical� value�is�αFm,(n−m−1)��The�test�statistic�can�also�be�written�in�equivalent�form�as
F SS df SS df
MS MS
reg reg
res res
reg
res = =
/ /
where� the� degrees� of� freedom� regression� equals� the� number� of� independent� variables,� dfreg�=�m,�and�degrees�of�freedom�residual�equals�the�difference�between�the�sample�size,� number�of�independent�variables,�and�1,�dfres�=�(n − m�−�1)�
For�the�GGPA�example,�we�compute�the�overall�F�test�statistic�as�the�following:
F R m
R n m =
− − − =
− − − =
2
21 1 9089 2
1 9089 11 2 1 39 9078
/ ( )/( )
. / ( . )/( )
.
or�as
F SS df SS df
reg reg
res res = = =
/ /
/ /
0 9998 2 1002 8
39 9122 .
. .
The�critical�value,�at�the��05�level�of�significance,�is��05F2,8�=�4�46��The�test�statistic�exceeds� the�critical�value,�so�we�reject�H0�and�conclude�that�all�of�the�partial�slopes�are�not�equal� to�0�at�the��05�level�of�significance�(the�two�F�test�statistics�differ�slightly�due�to�rounding� error)�
18.2.4.2 Test of Significance of bk
The�second�test�is�the�test�of�the�statistical�significance�of�each�individual�partial�slope�or� regression�coefficient,�bk��That�is,�are�the�individual�unstandardized�regression�coefficients� statistically�significantly�different�from�0?�This�is�actually�the�same�as�the�test�of�bk*,�so�we� need�not�develop�a�separate�test�for�bk*��The�null�and�alternative�hypotheses,�respectively,� are�as�follows:
H k0 0: β =
H k1 0: β ≠
where�βk�is�the�population�partial�slope�for�Xk�
670 An Introduction to Statistical Concepts
In�multiple�regression,�it�is�necessary�to�compute�a�standard�error�for�each�regression�coef- ficient�bk��Recall�from�Chapter�17�the�variance�error�of�estimate�concept��The�variance�error�of� estimate�is�similarly�defined�for�multiple�linear�regression�and�computed�as�follows:
s SS df
MSres res
res res
2 = =
where�dfres�=�(n − m�−�1)��Degrees�of�freedom�are�lost�as�we�have�to�estimate�the�population� partial�slopes� and�intercept,� the�βk’s� and�α,� respectively,� from� the� sample� data��The�vari- ance�error�of�estimate�indicates�the�amount�of�variation�among�the�residuals��The�standard� error�of�estimate�is�simply�the�positive�square�root�of�the�variance�error�of�estimate�and�is� the�standard�deviation�of�the�residuals�or�errors�of�estimate��We�call�it�the�standard error of estimate,�denoted�as�sres�
Finally,�we�need�to�compute�a�standard�error�for�each�bk��Denote�the�standard�error�of�bk� as�s(bk)�and�define�it�as
s b s
n s R k
res
k k
( ) ( ) ( )
= − −1 12 2
where sk
2�is�the�sample�variance�for�predictor�Xk Rk
2�is�the�squared�multiple�correlation�between�Xk�and�the�remaining�Xk’s Rk
2�represents�the�overlap�between�that�predictor�(Xk)�and�the�remaining�predictors
In�the�case�of�two�predictors,�the�squared�multiple�correlation,�Rk 2,�is�equal�to�the�simple�
bivariate�correlation�between�the�two�independent�variables,�r12 2 �
The�test�statistic�for�testing�the�significance�of�the�regression�coefficients,�bk’s,�is�as�follows:
t b s b
k
k =
( )
The�test�statistic�t�is�compared�to�the�critical�values�of�t,�a�two-tailed�test�for�a�nondirec- tional�H1,�at�the�designated�level�of�significance,�and�with�degrees�of�freedom�(n − m�−�1),� as�taken�from�the�t�table�in�Table�A�2��Thus,�the�tabled�critical�values�are�±(α/2)�t(n−m−1)�for�a� two-tailed�test�
We�can�also�form�a�confidence�interval�(CI)�around�bk�as�follows:
CI( ) 2 1b b t s bk k n m k= ± − −( / ) ( ) ( )α
Recall�that�the�null�hypothesis�tested�is�H0:�βk�=�0��Therefore,�if�the�CI�contains�0,�then�the� regression�coefficient�bk�is�not�statistically�significantly�different�from�0�at�the�specified�α� level��This�is�interpreted�to�mean�that�in�(1�−�α)%�of�the�sample�CIs�that�would�be�formed� from�multiple�samples,�βk�will�be�included�
Let� us� compute� the� second� test� statistic� for� the� GGPA� example�� We� specify� the� null� hypothesis�to�be�βk�=�0�(i�e�,�the�slope�is�0)�and�conduct�two-tailed�tests��First�the�variance� error�of�estimate�is
s SS df
res res
res
2 1002 8
0125= = = .
.
671Multiple Regression
The�standard�error�of�estimate,�sres,�is��1118��Next�the�standard�errors�of�the�bk�are�found�to�be
s b s
n s r res( )
( ) ( )
.
( ) . ( . ) .1
1 2
12 2 21 1
1118
10 266 8182 1 3011 00=
− − =
− = 223
s b s
n s r res( )
( ) ( )
.
( ) . ( . ) .2
2 2
12 2 21 1
1118
10 0 1609 1 3011 0924=
− − =
− =
Finally�we�find�the�t�test�statistics�to�be�computed�as�follows:
t b s b1 1 1/ 125/ 23 5 4348= = =( ) . . .0 00
t b s b2 2 2/ 4687/ 924 5 725= = =( ) . . .0 0
To�evaluate�the�null�hypotheses,�we�compare�these�test�statistics�to�the�critical�values�of� ±�025�t8�=�±2�306��Both�test�statistics�exceed�the�critical�value;�consequently�H0�is�rejected�in� favor�of�H1�for�both�predictors��We�conclude�that�both�partial�slopes�are�indeed�statistically� significantly�different�from�0�at�the��05�level�of�significance�
Finally,�let�us�compute�the�CIs�for�the�bk’s�as�follows:
CI( ) 125 2 3 6(1 1 2 1 1 1 25 8 1b b t s b b t s bn m= ± = ± = ±− −( / ) ( ) .( ) ( ) . . .α 0 0 0 000 00 023) 72 178= (. , . )
CI( ) 4687 2 3 6(2 2 ( 2 1 2 2 25 8 2b b t s b b t s bn m= ± = ± = ±− −α/ ) ( ) .( ) ( ) . . .0 0 09924) 2556 6818= (. , . )
The�intervals�do�not�contain�0,�the�value�specified�in�H0;�thus,�we�again�conclude�that�both� bk’s�are�significantly�different�from�0�at�the��05�level�of�significance�
18.2.4.3 Other Tests
One�can�also�form�CIs�for�the�predicted�mean�of�Y�and�the�prediction�intervals�for�indi- vidual�values�of�Y,�as�we�described�in�Chapter�17�
18.2.5 assumptions
A�considerable�amount�of�space�in�Chapter�17�was�dedicated�to�the�assumptions�of�simple� linear�regression��For�the�most�part,�the�assumptions�of�multiple�linear�regression�analysis� are�the�same,�and,�thus,�we�need�not�devote�as�much�space�here��The�assumptions�are�con- cerned�with�(a)�independence,�(b)�homogeneity,�(c)�normality,�(d)�linearity,�(e)�fixed�X,�and� (f)�noncollinearity��This�section�also�mentions�those�techniques�appropriate�for�evaluating� each�assumption�
18.2.5.1 Independence
The� first� assumption� is� concerned� with� independence� of� the� observations�� The� simplest� procedure� for� assessing� independence� is� to� examine� residual� plots� of� e� versus� the� pre- dicted�values�of�the�dependent�variable�Y′�and�of�e�versus�each�independent�variable�Xk�
672 An Introduction to Statistical Concepts
(alternatively,�one�can�look�at�plots�of�observed�values�of�the�dependent�variable�Y�versus� predicted� values� of� the� dependent� variable� Y′� and� of� observed� values� of� the� dependent� variable�Y�versus�each�independent�variable�Xk)��If�the�independence�assumption�is�satis- fied,�the�residuals�should�fall�into�a�random�display�of�points��If�the�assumption�is�violated,� the�residuals�will�fall�into�some�sort�of�pattern��Lack�of�independence�affects�the�estimated� standard� errors� of� the� model�� For� serious� violations,� one� could� consider� generalized� or� weighted�least�squares�as�the�method�of�estimation�(e�g�,�Myers,�1986;�Weisberg,�1985),�or� some�type�of�transformation��The�residual�plots�shown�in�Figure�18�1�do�not�suggest�any� independence�problems�for�the�GGPA�example,�where�Figure�18�1a�represents�the�residual� e�versus�the�predicted�value�of�the�dependent�variable�Y′,�Figure�18�1b�represents�e�versus� GRETOT,�and�Figure�18�1c�represents�e�versus�UGPA�
18.2.5.2 Homogeneity
The�second�assumption�is�homogeneity of variance,�where�the�conditional�distributions� have�the�same�constant�variance�for�all�values�of�X��In�the�residual�plots,�the�consistency� of� the� variance� of� the� conditional� distributions� may� be� examined�� If� the� homogeneity� assumption�is�violated,�estimates�of�the�standard�errors�are�larger,�and�the�conditional�dis- tributions�may�also�be�nonnormal��As�described�in�Chapter�17,�solutions�include�variance- stabilizing�transformations�(such�as�the�square�root�or�log�of�Y),�generalized�or�weighted� least�squares�(e�g�,�Myers,�1986;�Weisberg,�1985),�or�robust�regression�(Kleinbaum,�Kupper,� Muller,�&�Nizam,�1998;�Myers,�1986;�Wilcox,�1996,�2003;�Wu,�1985)��Due�to�the�small�sample� size,�homogeneity�cannot�really�be�assessed�for�the�example�data�
18.2.5.3 Normality
The�third�assumption�is�that�the�conditional�distributions�of�the�scores�on�Y,�or�the�pre- diction�errors,�are�normal�in�shape��Violation�of�the�normality�assumption�may�be�the� result�of�outliers��The�simplest�outlier�detection�procedure�is�to�look�for�observations�that� are� more� than� two� standard� errors� from� the� mean�� Other� procedures� were� previously� described�in�Chapter�17��Several�methods�for�dealing�with�outliers�are�available,�such�as� conducting�regression�analyses�with�and�without�suspected�outliers,�robust�regression� (Kleinbaum� et� al�,� 1998;� Myers,� 1986;� Wilcox,� 1996,� 2003;� Wu,� 1985),� and� nonparametric� regression�(Miller,�1997;�Rousseeuw�&�Leroy,�1987;�Wu,�1985)��The�following�can�be�used� to� detect� normality� violations:� frequency� distributions,� normal� probability� [quantile– quantile� (Q–Q)]� plots,� and� skewness� statistics�� For� the� example� data,� the� normal� prob- ability�plot�is�shown�in�Figure�18�2,�and�even�with�a�small�sample�looks�good��Violation� can� lead� to� imprecision� in� the� partial� slopes� and� in� the� coefficient� of� determination�� There�are�also�several�statistical�procedures�available�for�the�detection�of�nonnormality� (e�g�,�Andrews,�1971;�Belsley,�Kuh,�&�Welsch,�1980;�D’Agostino,�1971;�Ruppert�&�Carroll,� 1980;�Shapiro�&�Wilk,�1965;�Wu,�1985);�transformations�can�also�be�used�to�normalize�the� data��Review�Chapter�17�for�more�details�
18.2.5.4 Linearity
The�fourth�assumption�is�linearity,�that�there�is�a�linear�relationship�between�the�observed� scores� on� the� dependent� variable� Y� and� the� values� of� the� independent� variables,� Xk’s�� If� satisfied,�then�the�sample�partial�slopes�and�intercept�are�unbiased�estimators�of�the�pop- ulation� partial� slopes� and� intercept,� respectively�� The� linearity� assumption� is� important�
673Multiple Regression
Unstandardized predicted value(a)
St ud
en tiz
ed re
si du
al
2.00000
1.00000
.00000
–1.00000
–2.00000
3.00000 3.20000 3.40000 3.60000 3.80000 4.00000
2.00000
1.00000
.00000
–1.00000
–2.00000
90.00 100.00 110.00 120.00 130.00 140.00 150.00 GRE total score(b)
St ud
en tiz
ed re
si du
al
Undergraduate grade point average(c)
St ud
en tiz
ed re
si du
al
2.00000
1.00000
.00000
–1.00000
–2.00000
2.50 2.75 3.00 3.25 3.50 3.75
FIGuRe 18.1 Residual�plots�for�GRE–GPA�example:�(a),�(b),�and�(c)�
674 An Introduction to Statistical Concepts
because�regardless�of�the�value�of�Xk,�we�always�expect�Y�to�increase�by�bk�units�for�a�one- unit� increase� in� Xk,� controlling� for� the� other� Xk’s�� If� a� nonlinear� relationship� exists,� this� means� that� the� expected� increase� in� Y� depends� on� the� value� of� Xk;� that� is,� the� expected� increase�is�not�a�constant�value��Strictly�speaking,�linearity�in�a�model�refers�to�there�being� linearity�in�the�parameters�of�the�model�(i�e�,�α�and�the�βk’s)�
Violation�of�the�linearity�assumption�can�be�detected�through�residual�plots��The�residu- als� should� be� located� within� a� band� of� ±2sres� (or� standard� errors),� indicating� no� system- atic�pattern�of�points,�as�previously�discussed�in�Chapter�17��Residual�plots�for�the�GGPA� example�are�shown�in�Figure�18�1��Even�with�a�very�small�sample,�we�see�a�fairly�random� pattern�of�residuals,�and�therefore�feel�fairly�confident�that�the�linearity�assumption�has� been� satisfied�� Note� also� that� there� are� other� types� of� residual� plots� developed� espe- cially� for� multiple� regression� analysis,� such� as� the� added� variable� and� partial� residual� plots� (Larsen� &� McCleary,� 1972;� Mansfield� &� Conerly,� 1987;� Weisberg,� 1985)�� Procedures� to�deal�with�nonlinearity�include�transformations�(of�one�or�more�of�the�Xk’s�and/or�of�Y� as�described�in�Chapter�17)�and�other�regression�models�(discussed�later�in�this�chapter)�
18.2.5.5 Fixed X
The� fifth� assumption� is� that� the� values� of� Xk� are� fixed,� where� the� independent� variables,� Xk’s,�are�fixed�variables�rather�than�random�variables��This�results�in�the�regression�model� being�valid�only�for�those�particular�values�of�Xk�that�were�actually�observed�and�used�in� the�analysis��Thus,�the�same�values�of�Xk�would�be�used�in�replications�or�repeated�samples�
Strictly� speaking,� the� regression� model� and� its� parameter� estimates� are� only� valid� for� those�values�of�Xk�actually�sampled��The�use�of�a�prediction�model�developed�to�predict� the�dependent�variable�Y,�based�on�one�sample�of�individuals,�may�be�suspect�for�another� sample� of� individuals�� Depending� on� the� circumstances,� the� new� sample� of� individuals� may�actually�call�for�a�different�set�of�parameter�estimates��Expanding�on�our�discussion� in�Chapter�17,�generally�we�may�not�want�to�make�predictions�about�individuals�having� combinations�of�Xk�scores�outside�of�the�range�of�values�used�in�developing�the�prediction�
Observed value
Ex pe
ct ed
n or
m al
Normal Q–Q plot of unstandardized residual
–0.3
–3
–2
–1
0
1
2
–0.2 –0.1 0.0000 0.1 0.2
FIGuRe 18.2 Normal�probability�plot�for�GRE–GPA�example�
675Multiple Regression
model;�this�is�defined�as�extrapolating�beyond�the�sample�predictor�data��On�the�other�hand,� we�may�not�be�quite�as�concerned�in�making�predictions�about�individuals�having�combi- nations�of�Xk�scores�within�the�range�of�values�used�in�developing�the�prediction�model;� this�is�defined�as�interpolating�within�the�range�of�the�sample�predictor�data�
It�has�been�shown�that�when�other�assumptions�are�met,�regression�analysis�performs� just�as�well�when�X�is�a�random�variable�(e�g�,�Glass�&�Hopkins,�1996;�Myers�&�Well,�1995;� Pedhazur,�1997;�Wonnacott�&�Wonnacott,�1981)��There�is�no�such�assumption�about�Y�
18.2.5.6 Noncollinearity
The�final�assumption�is�unique�to�multiple�linear�regression�analysis,�being�unnecessary� in�simple�linear�regression��A�violation�of�this�assumption�is�known�as�collinearity�where� there�is�a�very�strong�linear�relationship�between�two�or�more�of�the�predictors��The�pres- ence�of�severe�collinearity�is�problematic�in�several�respects��First,�it�will�lead�to�instability� of�the�regression�coefficients�across�samples,�where�the�estimates�will�bounce�around�quite� a�bit�in�terms�of�magnitude�and�even�occasionally�result�in�changes�in�sign�(perhaps�oppo- site�of�expectation)��This�occurs�because�the�standard�errors�of�the�regression�coefficients� become� larger,� thus� making� it� more� difficult� to� achieve� statistical� significance�� Another� result� that� may� occur� involves� an� overall� regression� that� is� significant,� but� none� of� the� individual�predictors�are�significant��Collinearity�will�also�restrict�the�utility�and�general- izability�of�the�estimated�regression�model�
Recall�from�earlier�in�the�chapter�the�notion�of�partial�regression�coefficients,�where�the� other�predictors�were�held�constant��In�the�presence�of�severe�collinearity,�the�other�predic- tors�cannot�really�be�held�constant�because�they�are�so�highly�intercorrelated��Collinearity� may�be�indicated�when�there�are�large�changes�in�estimated�coefficients�due�to�(a)�a�vari- able�being�added�or�deleted�and/or�(b)�an�observation�being�added�or�deleted�(Chatterjee�&� Price,�1977)��Collinearity�is�also�likely�when�a�composite�variable�as�well�as�its�component� variables� are� used� as� predictors� in� the� same� regression� model� (e�g�,� including� GRETOT,� GRE-Quantitative,�and�GRE-Verbal�as�predictors)�
How� do� we� detect� violations� of� this� assumption?� The� simplest� procedure� is� to� conduct� a�series�of�special�regression�analyses,�one�for�each�X,�where�that�predictor�is�predicted�by� all�of�the�remaining�X’s�(i�e�,�the�criterion�variable�is�not�involved)��If�any�of�the�resultant�Rk
2� values�are�close�to�1�(greater�than��9�is�a�good�rule�of�thumb),�then�there�may�be�a�collinearity� problem��However,�the�large�R2�value�may�also�be�due�to�small�sample�size;�thus,�more�data� would�be�useful��For�the�example�data,�R12
2 .091= �and�therefore�collinearity�is�not�a�concern� Also,�if�the�number�of�predictors�is�greater�than�or�equal�to�n,�then�perfect�collinearity�is�
a�possibility��Another�statistical�method�for�detecting�collinearity�is�to�compute�a�variance� inflation�factor�(VIF)�for�each�predictor,�which�is�equal�to�1/(1 2− Rk)��The�VIF�is�defined�as� the�inflation�that�occurs�for�each�regression�coefficient�above�the�ideal�situation�of�uncorre- lated�predictors��Many�suggest�that�the�largest�VIF�should�be�less�than�10�in�order�to�satisfy� this�assumption�(Myers,�1990;�Stevens,�2009;�Wetherill,�1986)�
There�are�several�possible�methods�for�dealing�with�a�collinearity�problem��First,�one�can� remove�one�or�more�of�the�correlated�predictors��Second,�ridge�regression�techniques�can�be� used�(e�g�,�Hoerl�&�Kennard,�1970a,�1970b;�Marquardt�&�Snee,�1975;�Myers,�1986;�Wetherill,� 1986)��Third,�principal�component�scores�resulting�from�principal�component�analysis�can� be�utilized�rather�than�raw�scores�on�each�variable�(e�g�,�Kleinbaum�et�al�,�1998;�Myers,�1986;� Weisberg,� 1985;� Wetherill,� 1986)�� Fourth,� transformations� of� the� variables� can� be� used� to� remove�or�reduce�the�extent�of�the�problem��The�final�solution,�and�probably�our�last�choice,� is�to�use�simple�linear�regression,�as�collinearity�cannot�exist�with�a�single�predictor�
676 An Introduction to Statistical Concepts
18.2.5.7 Summary
For�the�GGPA�example,�although�sample�size�is�quite�small�in�terms�of�looking�at�condi- tional�distributions,�it�would�appear�that�all�of�our�assumptions�have�been�satisfied��All�of� the�residuals�are�within�two�standard�errors�of�0,�and�there�does�not�seem�to�be�any�sys- tematic�pattern�in�the�residuals��The�distribution�of�the�residuals�is�nearly�symmetric,�and� the�normal�probability�plot�looks�good��A�summary�of�the�assumptions�and�the�effects�of� their�violation�for�multiple�linear�regression�analysis�is�presented�in�Table�18�2�
18.3 Methods of Entering Predictors
The�multiple�predictor�model�which�we�have�considered�thus�far�can�be�viewed�as�simulta- neous regression��That�is,�all�of�the�predictors�to�be�used�are�entered�(or�selected)�simultane- ously,�such�that�all�of�the�regression�parameters�are�estimated�simultaneously;�here�the�set� of�predictors�has�been�selected�a�priori��In�computing�these�regression�models,�we�have�used� the�default�setting�in�SPSS�of�the�method�of�entry�as�“Enter,”�which�enters�the�set�of�indepen- dent�variables�in�aggregate��There�are�other�methods�of�entering�the�independent�variables� where�the�predictor�variables�are�entered�(or�selected)�systematically;�here�the�set�of�predic- tors�has�not�been�selected�a�priori��This�class�of�models�is�referred�to�as�sequential regression� (also�known�as�variable selection procedures)��This�section�introduces�a�brief�description� of�the�following�sequential�regression�procedures:�backward�elimination,�forward�selection,� stepwise�selection,�all�possible�subsets�regression,�and�hierarchical�regression�
18.3.1 backward elimination
First�consider�the�backward�elimination�procedure��Here�variables�are�eliminated�from�the� model�based�on�their�minimal�contribution�to�the�prediction�of�the�criterion�variable��In�the�
Table 18.2
Assumptions�and�Violation�of�Assumptions:�Multiple�Linear�Regression�Analysis
Assumption Effect of Assumption Violation
Independence •�Influences�standard�errors�of�the�model
Homogeneity •�Bias�in�s2res •�May�inflate�standard�errors�and�thus�increase�likelihood�of�a�Type�II�error •�May�result�in�nonnormal�conditional�distributions
Normality •�Less�precise�slopes,�intercept,�and�R2
Linearity •�Bias�in�slope�and�intercept •�Expected�change�in�Y�is�not�a�constant�and�depends�on�value�of�X
Fixed�X�values •��Extrapolating�beyond�the�range�of�X�combinations:�prediction�errors�larger,�may� also�bias�slopes�and�intercept
•��Interpolating�within�the�range�of�X�combinations:�smaller�effects�than�earlier;� if other�assumptions�met,�negligible�effect
Noncollinearity�of�X’s •��Regression�coefficients�can�be�quite�unstable�across�samples�(as�standard�errors� are�larger)
•�R2�may�be�significant,�yet�none�of�the�predictors�are�significant •�Restricted�generalizability�of�the�model
677Multiple Regression
first� stage� of� the� analysis,� all� potential� predictors� are� included� in� the� model�� In� the� second� stage,� that� predictor� is� deleted� from� the� model� that� makes� the� smallest� contribution� to� the� prediction�of�the�dependent�variable��This�can�be�done�by�eliminating�that�variable�having�the� smallest�t�or�F�statistic�such�that�it�is�making�the�smallest�contribution�to�Radj
2 ��In�subsequent� stages,�that�predictor�is�deleted�that�makes�the�next�smallest�contribution�to�the�prediction�of� the�outcome�Y��The�analysis�continues�until�each�of�the�remaining�predictors�in�the�model�is�a� significant�predictor�of�Y��This�could�be�determined�by�comparing�the�t�or�F�statistics�for�each� predictor�to�the�critical�value,�at�a�preselected�level�of�significance��Some�computer�programs� use�as�a�stopping�rule�the�maximum�F-to-remove�criterion,�where�the�procedure�is�stopped� when�all�of�the�selected�predictors’�F�values�are�greater�than�the�specified�F�criterion��Another� stopping� rule� is� where� the� researcher� stops� at� a� predetermined� number� of� predictors� (see� Hocking,�1976;�Thompson,�1978)��In�SPSS,�this�is�the�backward�method�of�entering�predictors�
18.3.2 Forward Selection
In�the�forward�selection�procedure,�variables�are�added�or�selected�into�the�model�based�on� their� maximal� contribution� to� the� prediction� of� the� criterion� variable�� Initially,� none� of� the� potential�predictors�are�included�in�the�model��In�the�first�stage,�the�predictor�is�added�to�the� model�that�makes�the�largest�contribution�to�the�prediction�of�the�dependent�variable��This�can� be�done�by�selecting�that�variable�having�the�largest�t�or�F�statistic�such�that�it�is�making�the� largest�contribution�to�Radj
2 ��In�subsequent�stages,�the�predictor�is�selected�that�makes�the�next� largest�contribution�to�the�prediction�of�Y��The�analysis�continues�until�each�of�the�selected�pre- dictors�in�the�model�is�a�significant�predictor�of�the�outcome�Y,�whereas�none�of�the�unselected� predictors�is�a�significant�predictor��This�could�be�determined�by�comparing�the�t�or�F�statistics� for�each�predictor�to�the�critical�value,�at�a�preselected�level�of�significance��Some�computer�pro- grams�use�as�a�stopping�rule�the�minimum�F-to-enter�criterion,�where�the�procedure�is�stopped� when�all�of�the�unselected�predictors’�F�values�are�less�than�the�specified�F�criterion��For�the� same�set�of�data�and�at�the�same�level�of�significance,�the�backward�elimination�and�forward� selection�procedures�may�not�necessarily�result�in�the�exact�same�final�model�due�to�the�differ- ences�in�how�variables�are�selected��In�SPSS,�this�is�the�forward�method�of�entering�predictors�
18.3.3 Stepwise Selection
The�stepwise�selection�procedure�is�a�modification�of�the�forward�selection�procedure�with� one�important�difference��Predictors�that�have�been�selected�into�the�model�can,�at�a�later�step,� be�deleted�from�the�model;�thus,�the�modification�conceptually�involves�a�backward�elimina- tion�mechanism��This�situation�can�occur�for�a�predictor�when�a�significant�contribution�at�an� earlier�step�later�becomes�a�nonsignificant�contribution�given�the�set�of�other�predictors�in�the� model��Thus,�a�predictor�loses�its�significance�due�to�new�predictors�being�added�to�the�model�
The�stepwise�selection�procedure�is�as�follows��Initially,�none�of�the�potential�predictors�are� included�in�the�model��In�the�first�step,�that�predictor�is�added�to�the�model�that�makes�the� largest�contribution�to�the�explanation�of�the�dependent�variable��This�can�be�done�by�select- ing�that�variable�having�the�largest�t�or�F�statistic�such�that�it�is�making�the�largest�contribution� to�Radj
2 ��In�subsequent�stages,�the�predictor�is�selected�that�makes�the�next�largest�contribution� to�the�prediction�of�Y��Those�predictors�that�have�entered�at�earlier�stages�are�also�checked�to� see�if�their�contribution�remains�significant��If�not,�then�that�predictor�is�eliminated�from�the� model��The�analysis�continues�until�each�of�the�predictors�remaining�in�the�model�is�a�sig- nificant�predictor�of�Y,�while�none�of�the�other�predictors�is�a�significant�predictor��This�could� be�determined�by�comparing�the�t�or�F�statistics�for�each�predictor�to�the�critical�value,�at�a� specified�level�of�significance��Some�computer�programs�use�as�stopping�rules�the�minimum�
678 An Introduction to Statistical Concepts
F-to-enter�and�maximum�F-to-remove�criteria,�where�the�F-to-enter�value�selected�is�usually� equal�to�or�slightly�greater�than�the�F-to-remove�value�selected�(to�prevent�a�predictor�from� continuously�being�entered�and�removed)��For�the�same�set�of�data�and�at�the�same�level�of� significance,�the�backward�elimination,�forward�selection,�and�stepwise�selection�procedures� may�not�necessarily�result�in�the�exact�same�final�model,�due�to�differences�in�how�variables� are�selected��In�SPSS,�this�is�the�stepwise�method�of�entering�predictors�
18.3.4 all possible Subsets Regression
Another� sequential� regression� procedure� is� known� as� all� possible� subsets� regression�� Let� us� say,� for� example,� that� there� are� five� potential� predictors�� In� this� procedure,� all� possible� one-,�two-,�three-,�and�four-variable�models�are�analyzed�(with�five�predictors,�there�is�only� a�single�five-predictor�model)��Thus,�there�will�be�5�one-predictor�models,�10�two-predictor� models,�10�three-predictor�models,�and�5�four-predictor�models��The�best�k�predictor�model� can� be� selected� as� the� model� that� yields� the� largest� Radj
2 �� For� example,� the� best� 3-predictor� model�would�be�that�model�of�the�10�estimated�that�yields�the�largest�Radj
2 ��With�today’s�pow- erful�computers,�this�procedure�is�easier�and�more�cost�efficient�than�in�the�past��However,� the�researcher�is�not�advised�to�consider�this�procedure,�or�for�that�matter,�any�of�the�other� sequential�regression�procedures,�when�the�number�of�potential�predictors�is�large��Here�the� researcher�is�allowing�number�crunching�to�take�precedence�over�thoughtful�analysis��Also,� the�number�of�models�will�be�equal�to�2m,�so�that�for�10�predictors,�there�are�1024�possible� subsets��Obviously,�examining�that�number�of�models�is�not�a�thoughtful�analysis�
18.3.5 hierarchical Regression
In�hierarchical�regression,�the�researcher�specifies�a�priori�a�sequence�for�the�individual�predic- tor�variables�(not�to�be�confused�with�hierarchical�linear�models,�which�is�a�regression�approach� for�analyzing�nested�data�collected�at�multiple�levels,�such�as�child,�classroom,�and�school)��The� analysis�proceeds�in�a�forward�selection,�backward�elimination,�or�stepwise�selection�mode� according�to�a�researcher-specified,�theoretically�based�sequence,�rather�than�an�unspecified,� statistically�based�sequence��This�variable�selection�method�is�different�from�those�previously� discussed�in�that�the�researcher�determines�the�order�of�entry�from�a�careful�consideration�of� the�available�theory�and�research,�instead�of�the�software�dictating�the�sequence�
A� type� of� hierarchical� regression� is� known� as� setwise regression� (also� called� block- wise,�chunkwise,�or�forced stepwise regression)��Here�the�researcher�specifies�a�priori�a� sequence�for�sets�of�predictor�variables��This�procedure�is�similar�to�hierarchical�regression� in�that�the�researcher�determines�the�order�of�entry�of�the�predictors��The�difference�is�that� the�setwise�method�uses�sets�of�predictor�variables�at�each�stage�rather�than�one�individual� predictor�variable�at�a�time��The�sets�of�variables�are�determined�by�the�researcher�so�that� variables�within�a�set�share�some�common�theoretical�ground�(e�g�,�home�background�vari- ables�in�one�set�and�aptitude�variables�in�another�set)��Variables�within�a�set�are�selected� according�to�one�of�the�sequential�regression�procedures��The�variables�selected�for�a�par- ticular�set�are�then�entered�in�the�specified�theoretically�based�sequence��In�SPSS,�this�is� conducted�by�entering�predictors�in�blocks�and�selecting�their�desired�method�of�entering� variables�in�each�block�(e�g�,�simultaneously,�forward,�backward,�stepwise)�
18.3.6 Commentary on Sequential Regression procedures
Let� us� make� some� comments� and� recommendations� about� the� sequential� regression� pro- cedures�� First,� numerous� statisticians� have� noted� problems� with� stepwise� methods� (i�e�,�
679Multiple Regression
backward�elimination,�forward�selection,�and�stepwise�selection)�(e�g�,�Derksen�&�Keselman,� 1992;� Huberty,� 1989;� Mickey,� Dunn,� &� Clark,� 2004;� Miller,� 1984,� 1990;� Wilcox,� 2003)�� These� problems� include� the� following:� (a)� selecting� noise� rather� than� important� predictors;� (b)� highly�inflated�R2�and�Radj
2 �values;�(c)�CIs�for�partial�slopes�that�are�too�narrow;�(d)�p�values� that�are�not�trustworthy;�(e)�important�predictors�being�barely�edged�out�of�the�model,�mak- ing�it�possible�to�miss�the�true�model;�and�(f)�potentially�heavy�capitalization�on�chance�given� the�number�of�models�analyzed��Second,�theoretically�based�regression�models�have�become� the�norm�in�many�disciples�(and�the�stepwise�methods�of�entry�are�driven�by�mathematics� of�the�models�rather�than�theory)��Thus,�hierarchical�regression�either�has�or�will�dominate� the�landscape�of�the�sequential�regression�procedures��Thus,�we�strongly�encourage�you�to� consider�more�extended�discussions�of�hierarchical�regression�(e�g�,�Bernstein,�1988;�Cohen�&� Cohen,�1983;�Pedhazur,�1997;�Schafer,�1991;�Tabachnick�&�Fidell,�2007)�
If�you�are�working�in�an�area�of�inquiry�where�research�evidence�is�scarce�or�nonexistent,� then� you� are� conducting� exploratory� research�� Thus,� you� are� probably� trying� to� simply� identify� the� key� variables�� Here� hierarchical� regression� is� not� appropriate,� as� a� theoreti- cally� driven� sequence� cannot� be� developed� and� there� is� no� theory� to� guide� its� develop- ment��Here�we�recommend�the�use�of�all�possible�subsets�regression�(e�g�,�Kleinbaum�et�al�� 1998)��For�additional�information�on�the�sequential�regression�procedures,�see�Cohen�and� Cohen�(1983),�Weisberg�(1985),�Miller�(1990),�Pedhazur�(1997),�and�Kleinbaum�et�al��(1998)�
18.4 Nonlinear Relationships
Here�we�continue�our�discussion�on�how�to�deal�with�nonlinearity�from�Chapter�17��We� formally�introduce�several�multiple�regression�models�for�when�the�criterion�variable�does� not�have�a�linear�relationship�with�the�predictor�variables�
First�consider�polynomial�regression�models��In�polynomial�models,�powers�of�the�pre- dictor�variables�(e�g�,�squared,�cubed)�are�used��In�general,�a�sample�polynomial�regression� model�that�includes�one�predictor�is�as�follows:
Y b X b X b X a em m= + + … + + +1 2
2
where�the�independent�variable�X�is�taken�from�the�first�power�through�the�mth�power,� and� the� i� subscript� for� observations� has� been� deleted� to� simplify� matters�� If� the� model� consists�only�of�X�taken�to�the�first�power,�then�this�is�a�simple linear regression model� (or�first-degree polynomial;�this�is�a�straight�line�and�what�we�have�studied�to�this�point)�� A�second-degree polynomial�includes�X�taken�to�the�second�power�(or�quadratic model;� this�is�a�curve�with�one�bend�in�it�rather�than�a�straight�line)��A�third-degree polynomial� includes�X�taken�to�the�third�power�(or�cubic model;�this�is�a�curve�with�two�bends�in�it)�
A�polynomial�model�with�multiple�predictors�can�also�be�utilized��An�example�of�a�sec- ond-degree�polynomial�model�with�two�predictors�is�illustrated�in�the�following�equation:
Y b X b X b X b X a e= + + + + +1 1 2 1 3 2 4 2 22
It�is�important�to�note�that�whenever�a�higher-order�polynomial�is�included�in�a�model� (e�g�,�quadratic,�cubic,�and�more),�the�first-order�polynomial�must�also�be�included�in�the�
680 An Introduction to Statistical Concepts
model�� In� other� words,� it� is� not� appropriate� to� include� a� quadratic� term� X2� without� also� including� the� first-order� polynomial� X�� For� more� information� on� polynomial� regression� models,� see� Weisberg� (1985),� Bates� and� Watts� (1988),� Seber� and� Wild� (1989),� Pedhazur� (1997),�and�Kleinbaum�et�al��(1998)��Alternatively,�one�might�transform�the�criterion�vari- able�and/or�the�predictor�variables�to�obtain�a�more�linear�form,�as�previously�discussed�
18.5 Interactions
Another�type�of�model�involves�the�use�of�an�interaction�term,�as�previously�discussed�in� factorial�ANOVA�(Chapter�13)��These�can�be�implemented�in�any�type�of�regression�model�� We�can�write�a�simple�two-predictor�interaction-type�model�as
Y b X b X b X X a e= + + + +1 1 2 2 3 1 2
where�X1X2�represents�the�interaction�of�predictor�variables�1�and�2��An�interaction�can�be� defined� as� occurring� when� the� relationship� between� Y� and� X1� depends� on� the� level� of� X2�� In other�words,�X2�is�a�moderator variable��For�example,�suppose�one�were�to�use�years�of� education�and�age�to�predict�political�attitude��The�relationship�between�education�and�atti- tude� might� be� moderated� by� age�� In� other� words,� the� relationship� between� education� and� attitude�may�be�different�for�older�versus�younger�individuals��If�age�were�a�moderator,�we� would�expect�there�to�be�an�interaction�between�age�and�education�in�a�regression�model�� Note�that�if�the�predictors�are�very�highly�correlated,�collinearity�is�likely��For�more�informa- tion�on�interaction�models,�see�Cohen�and�Cohen�(1983),�Berry�and�Feldman�(1985),�Kleinbaum� et�al��(1998),�Weinberg�and�Abramowitz�(2002),�and�Meyers,�Gamst,�and�Guarino�(2006)�
18.6 Categorical Predictors
So� far,� we� have� only� considered� continuous� predictors—independent� variables� that� are� interval� or� ratio� in� scale�� There� may� be� times,� however,� that� you� wish� to� use� a� categorical� predictor—an�independent�variable�that�is�nominal�or�ordinal�in�scale��For�example,�gender,� grade� level� (e�g�,� freshman,� sophomore,� junior,� senior),� and� highest� education� earned� (less� than�high�school,�high�school�graduate,�etc�)�are�all�categorical�variables�that�may�be�very� interesting�and�theoretically�appropriate�to�include�in�either�a�simple�or�multiple�regression� model��Given�their�scale�(i�e�,�nominal�or�ordinal),�however,�we�must�recode�the�values�prior� to�analysis�so�that�they�are�on�a�scale�of�0�and�1��This�is�called�“dummy�coding”�as�this�type� of� recoding� makes� the� model� work�� For� example,� males� might� be� coded� as� 0� and� females� coded�as�1��When�there�are�more�than�two�categories�to�the�categorical�predictor,�multiple� dummy� coded� variables� must� be� created—specifically 1 minus the number of levels or catego- ries of the categorical variable��Thus,�in�the�case�of�grade�level�where�there�are�four�categories� (freshman,�sophomore,�junior,�senior),�three�of�the�four�categories�would�be�dummy�coded� and� included� in� the� regression� model� as� predictors�� The� category� that� is� “left� out”� is� the� reference�category,�or�that�category�to�which�all�other�levels�are�compared��The�easiest�way� to�understand�this�is�perhaps�to�examine�the�data��In�the�screenshot�that�follows,�the�first� column�represents�grade�level�where�1 =�freshman,�2�=�sophomore,�3�=�junior,�and�4�=�senior�� Dummy�coding�three�of�the�four�grade�levels,�with�“senior”�as�the�reference�category,�will� result�in�three�additional�columns�(columns�2,�3,�and�4�in�the�screenshot)�
681Multiple Regression
1 2 3 4
In�terms�of�generating�the�analysis�and�the�point�and�click�use�of�SPSS�to�compute�the�regres- sion�model,�nothing�changes��The�steps�are�the�same�regardless�of�whether�the�predictors�are� continuous�or�categorical��Now�let�us�discuss�why�dummy�coding�works�in�this�situation��You� may�recall�from�Chapter�10�our�discussion�of�point�biserial�correlations��The�point�biserial�cor- relation�is�a�variant�of�the�Pearson�product–moment�correlation,�and�we�can�use�the�Pearson� as�a�variant�of�the�point�biserial��Thus,�while�we�will�not�have�a�linear�relationship�between�a� continuous�outcome�and�a�binary�variable,�the�mathematics�that�underlie�the�model�will�hold�
Consider�an�example�output�for�predicting�GPA�based�on�grade�level,�where�“senior”�is� the�reference�category��We�see�that�the�intercept�(i�e�,�“constant”)�is�statistically�significant�as� is�“freshman�”�The�interpretation�of�the�intercept�remains�the�same�regardless�of�the�scale�of� the�predictors��The�intercept�represents�GPA�(the�dependent�variable)�when�all�the�predictors� are�0��In�this�case,�this�means�that�GPA�is�3�267�for�seniors�(the�reference�category)��The�only� statistically� significant� predictor� is� “freshman�”� This� is� interpreted� to� say� that� mean� GPA� decreases�by��800�points�for�freshmen�as compared to seniors��The�nonstatistically�significant� regression�coefficients�for�“sophomore”�and�“junior”�indicate�that�mean�GPA�is�similar�for� these�grade�levels�as�compared�to�seniors��The�interpretation�for�dummy�variable�predictors� is�always�in�reference�to�the�category�that�was�“left�out�”�In�this�case,�that�was�“seniors�”
Coefficientsa
Unstandardized Coefficients
BModel 1 (Constant)
Freshman Sophomore Junior
3.267 –.800
.233
.200
.183
.258
.258
.258
17.892 –3.098
.904
.775
.000
.015
.393
.461
–.704 .205 .176
a Dependent variable: GPA.
Std. Error Beta Sig.
Standardized Coefficients
t
It�is�important�to�note�that�even�though�“sophomore”�and�“junior”�were�not�statistically� significant,� they� should� be� retained� in� the� model� as� they� represent� (along� with� “fresh- man”)�a�group��Dropping�one�or�more�dummy�coded�indicator�variables�that�represent�a� group�will�change�the�reference�category��For�example,�if�“sophomore”�and�“junior”�were� dropped�from�the�model,�the�interpretation�would�then�become�the�mean�GPA�for�fresh- men�as compared to all other grade levels��Thus,�careful�thought�needs�to�be�put�into�dropping� one�or�more�indicators�that�are�part�of�a�set�
682 An Introduction to Statistical Concepts
18.7 SPSS
Next� we� consider� SPSS� for� the� multiple� linear� regression� model�� Before� we� conduct� the� analysis,�let�us�review�the�data��With�one�dependent�variable�and�two�independent�vari- ables,� the� dataset� must� consist� of� three� variables� or� columns,� one� for� each� independent� variable�and�one�for�the�dependent�variable��Each�row�still�represents�one�individual,�indi- cating�the�value�of�the�independent�variables�for�that�particular�case�and�their�score�on�the� dependent�variable��As�seen�in�the�following�screenshot,�for�a�multiple�linear�regression� analysis�therefore,�the�SPSS�data�are�in�the�form�of�three�columns�that�represent�the�two� independent�variables�(GRE�total�score�and�UGPA)�and�one�dependent�variable�(GGPA)�
�e independent variables are labeled “GRE Total” and “UGPA” where each value
represents the student’s total score on the GRE and their undergraduate GPA. �e dependent variable
is “GGPA” and represents their graduate GPA.
Step 1:� To� conduct� a� simple� linear� regression,� go� to�“Analyze”� in� the� top� pulldown� menu,�then�select�“Regression,”�and�then�select�“Linear.”�Following�the�screenshot� (step�1)�as�follows�produces�the�“Linear Regression”�dialog�box�
Multiple linear regression: Step 1
A
B
C
Step 2:�Click�the�dependent�variable�(e�g�,�“GGPA”)�and�move�it�into�the�“Dependent” box�by�clicking�the�arrow�button��Click�the�independent�variables�and�move�them�into�the� “Independent(s)”�box�by�clicking�the�arrow�button�(see�screenshot�Step�2)�
683Multiple Regression
Multiple linear regression:Step 2
Clicking on “Statistics” will
allow you to select various regression
coefficients and residuals.
Clicking on “Plots” will allow you to
select various residuals plots.
Clicking on “Save” will allow you to
save various predicted values,
residuals, and other statistics
useful for diagnostics.
Clicking on “Next” will allow you to
define the blocks when
entering variables in
sets.
Clicking on “Enter” will allow you to select different
types of methods of entering the variables (e.g.,
stepwise, forward). “Enter” is the default and all predictors are
entered as one set.
Select the independent
variables from the list on the left and use the arrow to
move them to the “Independent(s)”
box on the right.
Select the dependent variable from the list on the
left and use the arrow to move it to the “Dependent”
box on the right.
Step 3:�From�the�“Linear Regression”�dialog�box�(see�screenshot�Step�2),�clicking�on� “Statistics”�will�provide�the�option�to�select�various�regression�coefficients�and�residu- als��From�the�“Statistics”�dialog�box�(see�screenshot�Step�3),�place�a�checkmark�in�the�box� next�to�the�following:�(a)�estimates,�(b) CIs,�(c)�model fit,�(d)�R squared change, (e)�descriptives, (f) part and partial correlations, (g)�collinearity diag- nostics, (h)�Durbin–Watson, and (i)�casewise diagnostics��For�this�example,�we� apply�an�α�level�of��05;�thus,�we�will�leave�the�default�CI�percentage�at�95��If�we�were�using�a� different�α,�the�CI�would�be�the�complement�of�alpha�(e�g�,�α�=��01,�then�CI�=�1�−��01�=�99)��We� will�also�leave�the�default�of�“three standard deviations”�for�defining�outliers�for�the� casewise�diagnostics��Click�on�“Continue”�to�return�to�the�original�dialog�box�
Multiple linear regression: Step 3
684 An Introduction to Statistical Concepts
Step 4:�From�the�“Linear Regression”�dialog�box�(see�screenshot�Step�2),�clicking� on�“Plots”� will� provide� the� option� to� select� various� residual� plots�� From� the “Plots”� dialog�box,�place�a�checkmark�in�the�box�next�to�the�following:�(a)�histogram,�(b)�normal probability plot,�and�(c)�produce all partial plots.�Click�on�“Continue”�to� return�to�the�original�dialog�box�
Mutiple linear regression: Step 4
Step 5:�From�the�“Linear Regression”�dialog�box�(see�screenshot�Step�2),�clicking�on� “Save”�will�provide�the�option�to�save�various�predicted�values,�residuals,�and�statistics�that� can�be�used�for�diagnostic�examination��From�the “Save”�dialog�box�under�the�heading�of� Predicted Values,�place�a�checkmark�in�the�box�next�to�the�following:�unstandard- ized.� Under� the� heading� of� Residuals,� place� a� checkmark� in� the� box� next� to� the� fol- lowing:�(a)�unstandardized�and�(b)�studentized. Under�the�heading�of�Distances,� place�a�checkmark�in�the�box�next�to�the�following:�(a)�Mahalanobis,�(b)�Cook’s,�and�(c)� leverage values��Under�the�heading�of�Influence Statistics,�place�a�checkmark�in� the�box�next�to�the�following:�standardized DfBeta(s).�Click�on�“Continue”�to�return� to�the�original�dialog�box��From�the�“Linear Regression”�dialog�box,�click�on�“OK”�to� return�and�generate�the�output�
Mutiple linear regression: Step 5
Interpreting the output:�Annotated�results�are�shown�in�Table�18�3�
685Multiple Regression
Table 18.3
SPSS�Results�for�the�Multiple�Regression�GRE–GPA�Example
Descriptive Statistics
Mean Std. Deviation N Graduate grade point average 11 GRE total score 11 Undergraduate grade point average
3.5000 112.7273
3.1091
.33166 16.33457
.40113 11
Correlations
Graduate Grade Point
Average
GRE Total Score
Undergraduate Grade Point
Average
Graduate grade point average GRE total score
Pearson correlation
Undergraduate grade point average
Graduate grade point average GRE total score
Sig. (1-tailed)
Undergraduate grade point average Graduate grade point average 11 GRE total score
N
Undergraduate grade point average
.784 1.000
.301
.002 .
.184
11 11
.752
.301 1.000
.004
.184 .
11 11 11
The table labeled “Descriptive Statistics” provides basic descriptive statistics (means,
standard deviations, and sample sizes) for the
independent and dependent variables.
�e table labeled “Correlations”
provides the: Pearson correlation coefficient values,
p values,
and sample size
for the simple bivariate Pearson
correlation between the independent and dependent
variables.
1.000 .784 .752
. .002 .004
11 11 11
�e correlation between graduate GPA and GRE-total (p = .002) and the correlation between graduate GPA and
undergraduate GPA (p = .004) are statistically significant.
Variables Entered/Removedb
Model Variables Entered Variables Removed Method
1 Undergraduate grade point average, GRE total score
Enter
a All requested variables entered. b Dependent variable: Graduate grade point average.
“Variables Entered/ Removed”
lists the independent variables included in the model and the
method they were entered (i.e., “Enter”).
(continued)
686 An Introduction to Statistical Concepts
Table 18.3 (continued)
SPSS�Results�for�the�Multiple�Regression�GRE–GPA�Example
Model Summaryb
Change Statistics
Model R R Square Adjusted R Square
Std. Error of the Estimate
R Square Change F Change df 1 df 2
Sig. F Change
Durbin– Watson
1 .953a .908 .885 .11272 .908 39.291 2 8 .000 2.116 a Predictors: (Constant), undergraduate grade point average, GRE total score. b Dependent variable: Graduate grade point average.
“Adjusted R square” adjusts for the number of independent variables and sample size. Shrinkage is the
difference between R2 and adjusted R2. When sample size is small, given the number of independent variables, the difference
between R2 and adjusted R2 will be large to compensate for a large amount of bias. If an additional independent
variable were entered in the model, an increase in adjusted R2 indicates the new variable is adding value to the model.
Negative adjusted R2 values can occur and indicate the model fits the data VERY poorly.
R is the multiple correlation coefficient.
R2 is the squared multiple correlation coefficient
(aka, coefficient of determination). It
represents the proportion of variance in the
dependent variable that is explained by the
independent variables.
Durbin–Watson is a test for
independence of residuals.
Ranging from 0 to 4, values of
2 indicate uncorrelated errors; values less than 1 or greater than 3
indicate a likely violation of this
assumption.
Change statistics are used when methods
other than simultaneous entry (e.g., hierarchical, forward, backward) are
used to enter the predictors in the model.
In those cases, more than one row will be
presented here. A p value less than α would
indicate the additional variables are explaining
additional variation.
Adjusted R2 is interpreted as the percentage of variation in the dependent variable that is explained after adjusting
for sample size and the number of predictors.
ANOVAb
Model Sum of Squares df Mean Square F Sig.
Regression .998 2 .499 39.291 .000a
Residual .102 8 .013
1
Total 1.100 10 a Predictors: (Constant), undergraduate grade point average, GRE total score. b Dependent variable: Graduate grade point average.
Total SS is partitioned into SS regression and SS residual.
Regression sum of squares indicates variability explained by the
regression model. Residual sum of squares indicates variability not
explained by the regression model.
The F statistic tests the overall regression model (i.e., that the
population multiple correlation coefficient is zero).
The p value (.000) indicates we reject the null
hypothesis. The probability of finding a
sample value of multiple R2 of .908 or larger when
the true population multiple correlation
coefficient is zero is less than 1%.
687Multiple Regression
Table 18.3 (continued)
SPSS�Results�for�the�Multiple�Regression�GRE–GPA�Example
Coefficientsa
t = = .093 .469
5.043
Unstandardized Coefficients
Standardized Coefficients
95.0% Confidence Interval for B
Correlations CollinearityStatistics
Model B Std. Error Beta t Sig.
Lower Bound
Upper Bound
Zero- Order Partial Part Tolerance VIF
(Constant)
GRE total score
.585 1.100
1
UGPA
.638
.012
.469
.327
.002
.093
.614
.567
1.954
5.447
5.030
.087
.001
.001
–.115
.007
.254
1.391
.018
.684
.784
.752
.887
.872 .541
.909
.909 1.100
a Dependent variable: Graduate grade point average.
�e “Constant” is the intercept and the unstandardized coefficient tells us that
if the predictors were zero, graduate GPA (the dependent variable) would
be .638. �e “GRE-Total” and “UGPA” are the slopes. For every one point
increase in GRE-total, graduate GPA will increase by about 1/10 of one
point (holding constant undergraduate GPA). For every one point increase in
undergraduate GPA, graduate GPA will increase by about ½ of one point (holding constant GRE-total).
�e test statistic, t, is calculated as the
unstandardized coefficient divided by its standard error.
�us the slope for undergraduate GPA is
calculated as (difference due to rounding):
The p value for the intercept (the “constant”) ( p= .087)
indicates that the intercept is not statistically significantly different from zero (this finding is usually of less interest than the slopes). The p values for GRE-total and
undergraduate GPA (the independent variables) ( p = .001)
indicate that the slopes are statistically significantly
different from zero.
Zero-order correlations are the simple bivariate Pearson correlations between
the dependent variable and the independent variables.
�e partial correlation of .887 is the correlation
between GRE-total and graduate GPA (dependent variable) when the linear effect of undergraduate GPA has been removed
from both GRE-total and graduate GPA. Squaring this indicates that 78.7%
of the variation in graduate GPA that is not
explained by undergraduate GPA is
explained by GRE-total.
�e part correlation of .585, when
squared (i.e., .342) indicates that GRE-
total explains an additional 34% of the variance in graduate GPA over and above
the variance in graduate GPA which
is explained by undergraduate GPA.
Collinearity statistics to be reviewed
under assumptions.
(continued)
688 An Introduction to Statistical Concepts
Table 18.3 (continued)
SPSS�Results�for�the�Multiple�Regression�GRE–GPA�Example
Collinearity Diagnosticsa
Variance Proportions
Model Dimension Eigenvalue Condition Index (Constant) GRE Total Score
Undergraduate Grade Point
Average
1 2.981 1.000 .00 .00 .00
2 .012 15.727 .03 .86 .40 1
3 .007 20.537 .97 .13 .60 a Dependent variable: Graduate grade point average.
Residuals Statisticsa
Minimum Maximum Mean Std. Deviation N 11 11 11
11 11 11 11 11 11 11 11
value
Residual
Predicted value Std. predicted value Standard error of predicted
Adjusted predicted value
Std. residual Stud. residual Deleted residual Stud. deleted residual Mahal. distance Cook's distance Centered leverage value
3.0714 –1.357
.038
3.0599 –.19943
–1.769 –1.881
–.22531 –2.355
.240
.012
.024
3.9448 1.408
.079
3.9117
.17207 1.527 1.716
.21754 2.020 4.053
.260
.405
3.5000 .000 .058
3.4954 .00000
.000
.017 .00458
.000 1.818
.092
.182
.31597 1.000
.011
.30917
.10082 .894
1.008 .12935
1.145 1.048
.081
.105 11 a Dependent variable: Graduate grade point average.
“Residual statistics” and related graphs (histogram and Q–Q plot of standardized residuals, not presented
here) will be examined in our discussion of assumptions.
“Collinearity diagnostics” will be examined in our discussion of assumptions.
689Multiple Regression
Table 18.3 (continued)
SPSS�Results�for�the�Multiple�Regression�GRE–GPA�Example
Histogram Dependent variable: Graduate grade point average
Mean = 3.61E–16 Std. dev. = 0.894 N = 11
Regression standardized residual
Fr eq
ue nc
y 3
2
1
0 –2 –1 0 1 2
Normal p–p plot of regression standardized residual Dependent variable: Graduate grade point average
0.0 0.2 0.4 0.6 0.8 1.0 0.0
0.2
0.4
0.6
0.8
1.0
Ex pe
ct ed
c um
p ro
b
Observed cum prob
Partial regression plot Dependent variable: Graduate grade point average
GRE total score –20.00 –10.00 .00 10.00 20.00 30.00 40.00
–.40
–.20
.00
.20
.40
.60
G ra
du at
e gr
ad e
po in
t a ve
ra ge
(continued)
690 An Introduction to Statistical Concepts
Table 18.3 (continued)
SPSS�Results�for�the�Multiple�Regression�GRE–GPA�Example
Partial regression plot Dependent variable: Graduate grade point average
Undergraduate grade point average –.75
–.40
–.20
.00
.20
–.50 –.25 .00 .25 .50
G ra
du at
e gr
ad e
po in
t a ve
ra ge
Examining Data for Assumptions for Multiple Linear Regression
As�you� may� recall,�there�were� a�number�of�assumptions�associated�with�multiple�linear� regression�� These� included� (a)� independence,� (b)� homogeneity� of� variance,� (c)� linear- ity,�(d)�normality,�and�(e)�multicollinearity��Although�fixed�values�of�X�were�discussed�in� assumptions,�this�is�not�an�assumption�that�will�be�tested�but�is�instead�related�to�the�use� of�the�results�(i�e�,�extrapolation�and�interpolation)�
Before�we�begin�to�examine�assumptions,�let�us�review�the�values�that�we�requested�to� be�saved�to�our�dataset�(see�dataset�screenshot�that�follows)�
� 1��PRE _ 1�represents�the�unstandardized�predicted�values�(i�e�,�Y′i)� � 2��RES _ 1� represents� the� unstandardized� residuals,� simply� the� difference�
between� the� observed� and� predicted� values�� For� student� 1,� for� example,� the� observed�value�for�the�GGPA�(i�e�,�the�dependent�variable)�was�4,�and�the�pre- dicted�value�was�3�94483��Thus,�the�unstandardized�residual�is�simply�4�−�3�94483,� or��05517�
� 3��SRE _ 1� represents� the� studentized� residuals,� a� type� of� standardized� resid- ual� that� is� more� sensitive� to� outliers� as� compared� to� standardized� residuals�� Studentized� residuals� are� computed� as� the� unstandardized� residual� divided� by� an� estimate� of� the� standard� deviation� with� that� case� removed�� As� a� rule� of� thumb,�studentized�residuals�with�an�absolute�value�greater�than�3�are�consid- ered�outliers�(Stevens,�1984)�
� 4��MAH _ 1� represents� Mahalanobis� distance� values� which� measure� how� far� that� particular� case� is� from� the� average� of� the� independent� variable� and� thus� can� be�
691Multiple Regression
helpful�in�detecting�outliers��These�values�can�be�reviewed�to�determine�cases�that� are�exerting�leverage��Barnett�and�Lewis�(1978)�produced�a�table�of�critical�values� for�evaluating�Mahalanobis�distance��Squared�Mahalanobis�distances�divided�by� the�number�of�variables�(D2/df )�which�are�greater�than�2�5�(for�small�samples)�or� 3–4�(for�large�samples)�are�suggestive�of�outliers�(Hair,�Black,�Babin,�Anderson,�&� Tatham,� 2006)�� Later,� we� follow� another� convention� for� examining� these� values� using�the�chi-square�distribution�
� 5� COO _ 1�represents�Cook’s�distance�values�and�provide�an�indication�of�influence� of�individual�cases��As�a�rule�of�thumb,�Cook’s�values�greater�than�1�suggest�that� case�is�potentially�problematic�
� 6��LEV _ 1�represents�leverage�values,�a�measure�of�distance�from�a�respective�case� to�the�average�of�the�predictor�
� 7��SDB0 _ 1, SDB1 _ 1,� and� SDB2 _ 1� are� standardized� DFBETA� values� for� the�intercept�and�slopes,�respectively,�and�are�easier�to�interpret�as�compared� to� their� unstandardized� counterparts�� Standardized� DFBETA� values� greater� than�an�absolute�value�of�2�suggest�that�the�case�may�be�exerting�undue�influ- ence� on� the� calculation� of� the� parameters� in� the� model� (i�e�,� the� slopes� and� intercept)�
1 2
As we look at the raw data, we see nine new variables have been added to our dataset. These are our predicted values, residuals, and other diagnostic
statistics. The residuals will be used to for diagnostics to review the extent to which our data meet the assumptions of multiple linear regression.
3 4 5 6 7 7 7
Independence
Here� we� will� plot� the� following:� (a)� studentized� residuals� (which� were� requested� and� created� through� the� “Save” option� when� generating� our� model)� against� unstandard- ized�predicted�values�and�(b)�studentized�residuals�against�each�independent�variable�to� examine� the� extent� to� which� independence� was� met�� The� general� steps� for� generating� a� simple� scatterplot� through� “Scatter/dot”� have� been� presented� in� a� previous� chapter� (e�g�,�Chapter�10),�and�they�will�not�be�reiterated�here��From�the�“Simple Scatterplot”� dialog�screen,�click�the�studentized�residual�variable�and�move�it�into�the�“Y Axis”�box� by�clicking�on�the�arrow��Click�the�unstandardized�predicted�values�and�move�them�into� the “X Axis”�box�by�clicking�on�the�arrow��Then�click�“Ok.”�Repeat�these�steps�to�plot� the�studentized�residual�to�each�independent�variable�
692 An Introduction to Statistical Concepts
If�the�assumption�of�independence�is�met,�the�points�should�fall�randomly�within�a�band� of�−2�0�to�+2�0��In�this�illustration�(see�Figure�18�1),�we�have�evidence�of�independence�as� all�points�for�all�graphs�are�within�an�absolute�value�of�2�0�and�fall�relatively�randomly�
Homogeneity of Variance
We�can�use�the�same�plots�that�were�used�to�examine�independence��To�examine�the�extent� to�which�homogeneity�was�met,�we�plot�(a)�studentized�residuals�(which�were�requested� and�created�through�the�“Save”�option�when�generating�our�model)�against�unstandard- ized� predicted� values� and� (b)� studentized� residuals� against� each� independent� variable�� Recall� that� homogeneity� is� when� the� dependent� variable� has� the� same� variance� for� all� values�of�the�independent�variable�
Evidence�of�meeting�the�assumption�of�homogeneity�is�a�plot�where�the�spread�of�residu- als�appears�fairly�constant�over�the�range�of�unstandardized�predicted�values�(i�e�,�a�ran- dom�display�of�points)�and�observed�values�of�the�independent�variables��If�the�display�of� residuals�increases�or�decreases�across�the�plot,�then�there�may�be�an�indication�that�the� assumption�of�homogeneity�has�been�violated��Here�we�see�evidence�of�homogeneity�
Linearity
Since�we�have�more�than�one�independent�variable,�we�have�to�take�a�different�approach�to� examining� linearity� than� what� was� done� with� simple� linear� regression�� However,� we� can� use�the�same�information�gleaned�from�our�examination�of�independence�and�homogeneity� for�reviewing�the�assumption�of�linearity��As�those�steps�have�been�presented�previously�in� the�discussion�of�independence,�they�will�not�be�repeated�here��From�the�scatterplot,�there�is� a�general�positive�linear�relationship�between�the�variables,�and,�thus,�we�have�evidence�of� linearity��We�can�also�review�the�partial�regression�plots�that�we�asked�for�when�generating� the� regression� model�� A� separate� partial� regression� plot� is� provided� for� each� independent� variable,� where� we� are� looking� for� linearity� (rather� than� some� type� of� polynomial)�� Even� with�a�small�sample�size,�the�partial�regression�plots�suggest�evidence�of�linearity�
693Multiple Regression
Normality
Generating normality evidence:�Understanding�the�distributional�shape,�specifi- cally�the�extent�to�which�normality�is�a�reasonable�assumption,�is�important�in�multiple� linear�regression�just�as�it�was�in�simple�linear�regression��We�will�examine�residuals�for� normality,� following� the� same� steps� as� with� the� previous� procedures�� We� will� also� use� various�diagnostics�to�examine�our�data�for�influential�cases��Let�us�begin�by�examining� the�unstandardized�residuals�for�normality��Just�as�we�saw�with�simple�linear�regression,� for� multiple� linear� regression,� the� distributional� shape� of� the� unstandardized� residuals� should�be�normal��Because�the�steps�for�generating�normality�evidence�were�presented�in� previous�chapters,�they�will�not�be�repeated�here� Interpreting normality evidence:�By�this�point,�we�are�well�versed�in�interpret- ing�quite�a�range�of�normality�statistics�and�will�do�the�same�for�multiple�linear�regression�
Descriptives
MeanUnstandardized residual Lower bound
Statistic Std. Error .03039717.0000000
–.0677291 .0677291 .0015202 .0281190
.010 .10081601
–.19943 .17207 .37150
.14051 –.336
.484 .661
1.279
Upper boundfor mean
Median
95% Confidence interval
5% Trimmed mean
Variance Std. deviation Minimum Maximum Range Interquartile range Skewness Kurtosis
The�skewness�statistic�of�the�residuals�is�−�336�and�kurtosis�is��484—both�being�within�the� range�of�an�absolute�value�of�2�0,�suggesting�some�evidence�of�normality��Given�the�very�small� sample�size,�the�following�histogram�reflects�as�normal�a�distribution�as�might�be�expected�
Histogram
5 Mean = 3.82E–17 Std. dev. = .10082 N=11
4
3
2
1
0 –.20000 –.10000 –.00000
Unstandardized residual
Fr eq
ue nc
y
.10000 .20000
694 An Introduction to Statistical Concepts
There�are�a�few�other�statistics�that�can�be�used�to�gauge�normality��The�formal�test� of� normality,� the� Shapiro–Wilk� (S–W)� test� (SW)� (Shapiro� &� Wilk,� 1965),� provides� evi- dence� of� the� extent� to� which� our� sample� distribution� is� statistically� different� from� a� normal�distribution��The�output�for�the�S–W�test�is�presented�as�follows�and�suggests� that� our� sample� distribution� for� the� residual� is� not� statistically� significantly� different� than�what�would�be�expected�from�a�normal�distribution�as�the�p�value�is�greater�than� α�(p�=��918)�
Tests of Normality
Kolmogorov–Smirnova
Unstandardized residual
a Lilliefors significance correction. *This is a lower bound of the true significance.
.155 Statistic df dfSig. Sig.
11 .200* .973 11 .918 Statistic
Shapiro–Wilk
Q–Q� plots� are� also� often� examined� to� determine� evidence� of� normality�� Q–Q� plots� graph� quantiles� of� the� theoretical� normal� distribution� against� quantiles� of� the� sample� distribution��Points�that�fall�on�or�close�to�the�diagonal�line�suggest�evidence�of�normal- ity��The�Q–Q�plot�of�residuals�(see�Figure�18�2)�suggests�relative�normality��Examination� of�the�following�boxplot�also�suggests�a�relatively�normal�distribution�of�residuals�with� no�outliers�
.20000
.10000
.00000
–.10000
–.20000
Unstandardized residual
Considering�the�forms�of�evidence�we�have�examined,�skewness�and�kurtosis�statistics,� the�S–W�test,�histogram,�the�Q–Q�plot,�and�the�boxplot,�all�suggest�normality�is�a�reason- able�assumption�
695Multiple Regression
Screening Data for Influential Points
Casewise diagnostics:�Recall�that�we�requested�a�number�of�statistics�to�help�in�diag- nostics��One�that�we�requested�was�for�“Casewise�diagnostics�”�If�we�had�any�cases�with� large� values� for� the� standardized� residual� (outside� three� standard� deviations),� informa- tion� would� have� been� included� in� our� output� to� indicate� the� case� number,� value� of� the� standardized� residual,� predicted� value,� and� unstandardized� residual�� This� information� can�be�used�to�more�closely�examine�case(s)�with�the�extreme�values�on�the�standardized� residuals�
Cook’s distance:� Cook’s� distance� provides� an� overall� measure� for� the� influence� of� individual�cases��Values�greater�than�1�suggest�that�the�case�may�be�problematic�in�terms� of�undue�influence�on�the�model��Examining�the�residual�statistics�in�our�output�(see�fol- lowing�table),�we�see�that�the�maximum�value�for�Cook’s�distance�is��260,�well�under�the� point�at�which�we�should�be�concerned�
Residuals Statisticsa
Minimum Maximum Mean Std. Deviation N
11 11 11
11 11 11 11 11 11 11 11 11
.31597 1.000
.011
.30917
.10082 .894
1.008 .12935
1.145 1.048
.081
.105
3.5000 .000 .058
3.4954 .00000
.000
.017 .00458
.000 1.818
.092
.182
3.9448 1.408
.079
3.9117 .17207
1.527 1.716
.21754 2.020
4.053 .260 .405
3.0714 –1.357
.038
3.0599 –.19943
–1.769 –1.881
–.22531 –2.355
.240
.012
.024
Predicted value Std. predicted value Standard error of predicted value Adjusted predicted value Residual Std. residual Stud. residual Deleted residual Stud deleted residual
Mahal distance Cook’s distance Centered leverage value
a Dependent variable. Graduate grade point average.
Mahalanobis distances:� Mahalanobis� distances� are� measures� of� the� distance� from� each� case� to� the� mean� of� the� independent� variable� for� the� remaining� cases�� We� can� use� the� value� of� Mahalanobis� distance� as� a� test� statistic� value� with� the� chi-square� distribu- tion��With�two�independent�variables�and�one�dependent�variable,�we�have�three�degrees� of� freedom�� Given� an� alpha� level� of� �05,� the� chi-square� critical� value� is� 7�82�� Thus,� any� Mahalanobis�distance�greater�than�7�82�suggests�that�case�is�an�outlier��With�a�maximum� of�4�053�(see�previous�table),�there�is�no�evidence�to�suggest�there�are�outliers�in�our�data�
Centered leverage values:�Centered�leverage�values�less�than��20�suggest�there�are� no�problems�with�cases�that�are�exerting�undue�influence��Values�greater�than��5�indicate� problems�
696 An Introduction to Statistical Concepts
DFBETA:�We�also�asked�to�save�DFBETA�values��These�values�provide�another�indication� of� the� influence� of� cases�� DFBETA� provides� information� on� the� change� in� the� predicted� value�when�the�case�is�deleted�from�the�model��For�standardized�DFBETA�values,�values� greater�than�an�absolute�value�of�2�0�should�be�examined�more�closely��Looking�at�the�min- imum�and�maximum�DFBETA�values,�there�are�no�cases�suggestive�of�undue�influence�
Descriptive Statistics
N Minimum Maximum
11
11
11
11
–.51278
–.75577
–.32176
.63170
.59269
.55938
Standardized DFBETA
Standardized DFBETA
GRE total
Standardized DFBETA
UGPA
Valid N (listwise)
intercept
Diagnostic plots:�There�are�a�number�of�diagnostic�plots�that�can�be�generated�from� the�values�we�saved��For�example,�a�plot�of�Cook’s�distance�against�centered�leverage�val- ues�provides�a�way�to�identify�influential�cases�(i�e�,�cases�with�leverage�of��50�or�above�and� Cook’s�distance�of�1�0�or�greater)��Here�there�are�no�cases�that�suggest�undue�influence�
Centered leverage value
C oo
k’ s d
is ta
nc e
.00000 .10000 .20000 .30000 .40000 .50000
.00000
.10000
.05000
.15000
.20000
.25000
.30000
Multicollinearity
Generating multicollinearity evidence:� Multicollinearity,� as� you� recall,� refers� to�strong�correlations�between�the�independent�variables��Detecting�multicollinearity�can� be� done� by� reviewing� the� VIF� and� tolerance� statistics�� From� the� following� table,� we� see� tolerance�and�VIF�values��Tolerance�is�calculated�as�(1�−�R2),�and�values�close�to�0�(a�rule�of�
697Multiple Regression
thumb�is��10�or�less)�suggest�potential�multicollinearity�problems��Why?�A�tolerance�of� �10�suggests�that�90%�(or�more)�of�the�variance�in�one�of�the�independent�variables�can�be� explained�by�another�independent�variable��VIF�is�the�“variance�inflation�factor”�and�is�the�
reciprocal�of � tolerance�where�VIF
tolerance =
1 ��VIF�values�greater�than�10�(which�correspond
� to�a�tolerance�of��10)�suggest�potential�multicollinearity�
Collinearity Statistics
Tolerance VIF
.909 1.100
.909 1.100
Collinearity� diagnostics� (see� the� following� SPSS� output)� can� also� be� reviewed�� “Dimension� 1”� refers� to� the� intercept;� however,� we� are� interested� in� reviewing� data� for� “dimensions�2�and�3�”�Multiple�eigenvalues�close�to�0�indicate�independent�variables�that� have�strong�intercorrelations��The�condition�index�is�calculated�as�the�square�root�of�the�
ratio�of�the�largest�eigenvalue�to�each�preceding�eigenvalue�(e�g�,� 2.981 .012
15.76= )��Condition�
indices�greater�than�15�suggest�there�is�a�possible�problem�with�multicollinearity,�and�val- ues�greater�than�30�indicate�a�substantial�multicollinearity�problem��In�this�case,�both�the� eigenvalues�and�condition�indices�suggest�possible�problems�with�multicollinearity�
Model 1 1
2
3
2.981 .012
.007
1.000 15.727
20.537
.00
.03
.97
.00
.86
.13
.00
.40
.60
Dimension Eigenvalue
a Dependent variable: Graduate grade point average.
Condition Index (Constant) GRE Total Score
Undergraduate Grade Point
Average
Variance Proportions
Collinearity Diagnosticsa
Multicollinearity� can� also� be� examined� by� computing� regression� models� where� each� independent�variable�is�considered�the�outcome�and�is�predicted�by�the�remaining�indepen- dent�variables�(the�dependent�variable�is�not�included�in�these�models)��Because�the�steps� for�conducting�regression�have�already�been�presented,�they�will�not�be�repeated�again�� Click�one�of�the�independent�variables�(e�g�,�“UGPA”)�and�move�it�into�the�“Dependent”� box�by�clicking�the�arrow�button��Click�the�remaining�independent�variable(s)�and�move� those�into�the�“Independent(s)”�box�by�clicking�the�arrow�button�
Interpreting multicollinearity evidence:� If� any� of� the� resultant� Rk 2� values�
are� close� to� 1� (greater� than� �9� is� a� good� rule� of� thumb),� then� there� may� be� a� collinearity� problem��For�the�example�data,�R12
2 091= . �and�therefore�collinearity�is�not�a�concern��Note� that� in� multiple� regression� situations� where� there� are� two� independent� variables� (as� in� this� example� with� GRE� total� and� UGPA),� only� one� regression� needs� to� be� conducted� to� check�for�multicollinearity�as�the�results�for�regressing�UGPA�on�GRE�total�are�the�same� as�regressing�GRE�total�on�UGPA�
698 An Introduction to Statistical Concepts
R R SquareModel R Square
1 .301a .091 –.010 16.41926
Adjusted Std. Error of the Estimate
18.8 G*Power
A� priori� and� post� hoc� power� could� again� be� determined� using� the� specialized� software� described�previously�in�this�text�(e�g�,�G*Power),�or�you�can�consult�a�priori�power�tables�(e�g�,� Cohen,�1988)��As�an�illustration,�we�use�G*Power�to�compute�the�post�hoc�power�of�our�test�
Post Hoc Power for Multiple Linear Regression Using G*Power
The�first�thing�that�must�be�done�when�using�G*Power�for�computing�post�hoc�power�is� to� select� the� correct� test� family�� In� our� case,� we� conducted� a� multiple� linear� regression�� To� find� regression,� we� select�“Tests”� in� the� top� pulldown� menu,� then�“Correlation and regression,”�and�then�“Linear multiple regression: Fixed model, R2 deviation from zero.”�This�will�allow�us�to�determine�power�for�the�hypothesis�that� the�overall�multiple�R2�is�equal�to�0�(i�e�,�power�for�the�overall�regression�model)��Once�that� selection�is�made,�the�“Test family”�automatically�changes�to�“F test.”
Step 1 A
B
C
699Multiple Regression
The�“Type of Power Analysis”�desired�needs�to�be�selected��To�compute�post�hoc� power,�select�“Post hoc: Compute achieved power—given α, sample size, and effect size.”
Step 2
Click on “Determine” to pop out the effect size calculator box
(shown below). �is will allow you to
compute the effect size, “f2,” given the
squared multiple correlation.
�e default selection for “Statistical Test” is
“Correlation: Point biserial model.” Following the procedures presented in Step 1 will automatically change the statistical test to
“Linear multiple regression: Fixed model, R2 deviation from zero”.
�e default selection for “Test Family” is
“t tests” and this will change to “F tests”
when the linear multiple regression is
selected.
Once the parameters are
specified, click on “Calculate.”
The “Input Parameters” for computing post hoc power
must be specified including: 1. Effect size f 2 2. α Level 3. Total sample size 4. Number of predictors
The�“Input Parameters”�must�then�be�specified��We�compute�the�effect�size,�f 2,�last� and�so�we�skip�that�for�the�moment��The�α�level�we�used�was��05,�the�total�sample�size�was� 11,�and�there�were�two�independent�variables��Next�we�use�the�pop-out�effect�size�calcula- tor�in�G*Power� to�compute� the�effect� size�f 2��To�do�this,� click� on�“Determine”� which� is� displayed�under�“Input Parameters.”�In�the�pop-out�effect�size�calculator,�input�the� value�for�the�squared�multiple�correlation��Click�on�“Calculate”�to�compute�the�effect� size�f 2��Then�click�on�“Calculate and Transfer to Main Window”�to�transfer�the� calculated�effect�size�(i�e�,�9�8695652)�to�the�“Input Parameters.”�Once�the�parameters� are�specified,�click�on�“Calculate”�to�find�the�power�statistics�
700 An Introduction to Statistical Concepts
Post hoc power
Here are the post-hoc power results.
The�“Output Parameters”�provide�the�relevant�statistics�given�the�input�just�speci- fied��Here�we�were�interested�in�determining�post�hoc�power�for�a�multiple�linear�regres- sion�with�a�computed�effect�size�f 2�of�9�8695652,�an�alpha�level�of��05,�total�sample�size�of�11,� and�two�predictors��Based�on�those�criteria,�the�post�hoc�power�for�the�overall�multiple�lin- ear�regression�model�was�1�0000��In�other�words,�given�the�input�parameters,�the�probabil- ity�of�rejecting�the�null�hypothesis�when�it�is�really�false�(in�this�case,�the�probability�that� the�multiple�correlation�coefficient�is�0)�was�at�the�maximum�(i�e�,�1�00)�(sufficient�power�is� often��80�or�above)��Do�not�forget�that�conducting�power�analysis�a�priori�is�recommended� so�that�you�avoid�a�situation�where,�post�hoc,�you�find�that�the�sample�size�was�not�suf- ficient� to� reach� the� desired� level� of� power� (given� the� observed� parameters)�� Conducting� power� for� change� in� R2� and� for� the� slopes� can� be� conducted� similarly� by� selecting� the� test�family�of�“Linear multiple regression: Fixed model, R2 increase” or “Linear multiple regression: Fixed model, single regression coeffi- cient,” respectively�
701Multiple Regression
A priori power
Here are the post-hoc power results.
A Priori Power for Multiple Linear Regression Using G*Power
For� a� priori� power,� we� can� determine� the� total� sample� size� needed� for� multiple� linear� regression�given�the�estimated�effect�size�f 2,�α�level,�desired�power,�and�number�of�pre- dictors��We�follow�Cohen’s�(1988)�conventions�for�effect�size�(i�e�,�small�r2�=��02;�moderate� r2�=��15;�large�r2�=��35)��If�we�had�estimated�a�moderate�effect�r2�of��15,�alpha�of��05,�observed� power�of��80,�and�two�independent�variables,�we�would�need�a�total�sample�size�of�58�
18.9 Template and APA-Style Write-Up
Finally,�here�is�an�example�paragraph�for�the�results�of�the�multiple�linear�regression�anal- ysis��Recall�that�our�graduate�research�assistant,�Marie,�was�assisting�the�assistant�dean�in� Graduate�Student�Services,�Jennifer��Jennifer�wanted�to�know�if�GGPA�could�be�predicted� by�the�total�score�on�the�required�graduate�entrance�exam�(GRE�total)�and�by�UGPA��The� research�question�presented�to�Jennifer�from�Marie�included�the�following:�Can GGPA be predicted from the GRE total and UGPA?
Marie�then�assisted�Jennifer�in�generating�a�multiple�linear�regression�as�the�test�of�infer- ence,�and�a�template�for�writing�the�research�question�for�this�design�is�presented�as�follows:
•� Can [dependent variable] be predicted from [list independent variables]?
702 An Introduction to Statistical Concepts
It�may�be�helpful�to�preface�the�results�of�the�multiple�linear�regression�with�information� on� an� examination� of� the� extent� to� which� the� assumptions� were� met�� The� assumptions� include�(a)�independence,�(b)�homogeneity�of�variance,�(c)�normality,�(d)�linearity,�(e)�non- collinearity,�and�(f)�values�of�X�are�fixed��Because�the�last�assumption�(fixed�X)�is�based�on� interpretation,�it�will�not�be�discussed�here�
A multiple linear regression model was conducted to determine if GGPA (dependent variable) could be predicted from GRE total scores and UGPA (independent variables). The null hypotheses tested were that the multiple R2 was equal to 0 and that the regression coefficients (i.e., the slopes) were equal to 0. The data were screened for miss- ingness and violation of assumptions prior to analysis. There were no missing data.
Linearity: Review of the partial scatterplot of the independent vari- ables (GRE total and UGPA) and the dependent variable (GGPA scores) indicates linearity is a reasonable assumption. Additionally, with a random display of points falling within an absolute value of 2, a scatterplot of unstandardized residuals to predicted values provided further evidence of linearity.
Normality: The assumption of normality was tested via examination of the unstandardized residuals. Review of the S–W test for normal- ity (SW = .973, df = 11, p = .918) and skewness (−.336) and kurtosis (.484) statistics suggested that normality was a reasonable assump- tion. The boxplot suggested a relatively normal distributional shape (with no outliers) of the residuals. The Q–Q plot and histogram sug- gested normality was reasonable. Examination of casewise diagnos- tics, including Mahalanobis distance, Cook’s distance, DfBeta values, and centered leverage values, suggested there were no cases exerting undue influence on the model.
Independence: A relatively random display of points in the scat- terplots of studentized residuals against values of the indepen- dent variables and studentized residuals against predicted values provided evidence of independence. The Durbin–Watson statistic was computed to evaluate independence of errors and was 2.116, which is considered acceptable. This suggests that the assumption of indepen- dent errors has been met.
Homogeneity of variance: A relatively random display of points, where the spread of residuals appears fairly constant over the range of values of the independent variables (in the scatterplots of studen- tized residuals against predicted values and studentized residuals against values of the independent variables) provided evidence of homogeneity of variance.
Multicollinearity: Tolerance was greater than .10 (.909), and the variance inflation factor was less than 10 (1.100), suggesting that multicollinearity was not an issue. However, the eigenvalues for the predictors were close to 0 (.012 and .007). A review of GRE total
703Multiple Regression
regressed on UGPA, however, produced a multiple R squared of .091, which suggests noncollinearity. In aggregate, therefore, the evidence suggests that multicollinearity is not an issue.
Here� is� an� APA-style� example� paragraph� of� results� for� the� multiple� linear� regression� (remember� that� this� will� be� prefaced� by� the� previous� paragraph� reporting� the� extent� to� which�the�assumptions�of�the�test�were�met)�
The results of the multiple linear regression suggest that a sig- nificant proportion of the total variation in GGPA was predicted by GRE total and UGPA, F(2, 8) = 39.291, p < .001. Additionally, we find the following:
1. For GRE total, the unstandardized partial slope (.012) and standardized partial slope (.614) are statistically signifi- cantly different from 0 (t = 5.447, df = 8, p < .001); with every one-point increase in the GRE total, GGPA will increase by approximately 1/100 of one point when controlling for UGPA.
2. For UGPA, the unstandardized partial slope (.469) and standard- ized partial slope (.567) are statistically significantly dif- ferent from 0 (t = 5.030, df = 8, p < .001); with every one-point increase in UGPA, GGPA will increase by approximately one-half of one point when controlling for GRE total.
3. The CI around the unstandardized partial slopes do not include 0 (GRE total, .007, .018; UGPA, .254, .684), further confirming that these variables are statistically significant predictors of GGPA. Thus, GRETOT and UGPA were shown to be statistically sig- nificant predictors of GGPA, both individually and collectively.
4. The intercept (or average GGPA when GRE total and UGPA is 0) was .638, not statistically significantly different from 0 (t = 1.954, df = 8, p = .087).
5. Multiple R2 indicates that approximately 91% of the variation in GGPA was predicted by GRE total scores and UGPA. Interpreted according to Cohen (1988), this suggests a large effect.
6. Estimated power to predict multiple R2 is at the maximum, 1.00.
We�note�that�the�more�advanced�regression�models�described�in�this�chapter�can�all�be�con- ducted�using�SPSS��For�further�information�on�regression�analysis�with�SPSS,�see�Morgan� and�Griego�(1998),�Weinberg�and�Abramowitz�(2002),�and�Meyers�et�al��(2006)�
18.10 Summary
In� this� chapter,� methods� involving� multiple� predictors� in� the� regression� context� were� considered�� The� chapter� began� with� a� look� at� partial� and� semipartial� correlations�� Next,� a� lengthy� discussion� of� multiple� linear� regression� analysis� was� conducted�� Here� we�
704 An Introduction to Statistical Concepts
extended� many� of� the� basic� concepts� of� simple� linear� regression� to� the� multiple� predic- tor� context�� In� addition,� several� new� concepts� were� introduced,� including� the� coefficient� of�multiple�determination,�the�multiple�correlation,�and�tests�of�the�individual�regression� coefficients��Finally�we�examined�a�number�of�other�regression�models,�such�as�forward� selection,�backward�elimination,�stepwise�selection,�all�possible�subsets�regression,�hier- archical�regression,�and�nonlinear�regression��At�this�point,�you�should�have�met�the�fol- lowing�objectives:�(a)�be�able�to�determine�and�interpret�the�results�of�part�and�semipartial� correlations,�(b)�be�able�to�understand�the�concepts�underlying�multiple�linear�regression,� (c)�be�able�to�determine�and�interpret�the�results�of�multiple�linear�regression,�(d)�be�able�to� understand�and�evaluate�the�assumptions�of�multiple�linear�regression,�and�(e)�be�able�to�have� a�basic�understanding�of�other�types�of�regression�models��In�Chapter�19,�we�conclude�the� text�by�considering�logistic�regression�analysis�
Problems
Conceptual problems
18.1� �The�correlation�of�salary�and�cumulative�GPA�controlling�for�socioeconomic�status� is�an�example�of�which�one�of�the�following?
� a�� Bivariate�correlation � b�� Partial�correlation � c�� Regression�correlation � d�� Semipartial�correlation 18.2� �Variable�1�is�to�be�predicted�from�a�combination�of�variable�2�and�one�of�variables�3,�
4,�5,�and�6��The�correlations�of�importance�are�as�follows: � r13�=��8� � r23�=��2 � r14�=��6� � r24�=��5 � r15�=��6� � r25�=��2 � r16�=��8� � r26�=��5 � Which�of�the�following�multiple�correlation�coefficients�will�have�the�largest�value? � a�� r1�23 � b�� r1�24 � c�� r1�25 � d�� r1�26 18.3� �The�most�accurate�predictions�are�made�when�the�standard�error�of�estimate�equals�
which�one�of�the�following? � a�� Y
–
� b�� sY � c�� 0 � d�� 1 18.4� The�intercept�can�take�on�a�positive�value�only��True�or�false?
705Multiple Regression
18.5� ��Adding�an�additional�predictor�to�a�regression�equation�will�necessarily�result�in� an�increase�in�R2��True�or�false?
18.6� �The�best�prediction�in�multiple�regression�analysis�will�result�when�each�predictor� has�a�high�correlation�with�the�other�predictor�variables�and�a�high�correlation�with� the�dependent�variable��True�or�false?
18.7� Consider�the�following�two�situations: � Situation�1� rY1�=��6� rY2�=��5� r12�=��0 � Situation�2� rY1�=��6� rY2�=��5� r12�=��2
I�assert�that�the�value�of�R2�will�be�greater�in�situation�2��Am�I�correct? 18.8� �Values�of�variables�X1,�X2,�and�X3�are�available�for�a�sample�of�50�students��The�value�
of� r12� =� �6�� I� assert� that� if� the� partial� correlation� r12�3� were� calculated,� it� would� be� larger�than��6��Am�I�correct?
18.9� �A�researcher�is�building�a�regression�model��There�is�theory�to�suggest�that�science� ability�can�be�predicted�by�literacy�skills�when�controlling�for�child�characteristics� (e�g�,�age�and�socioeconomic�status)��Which�one�of�the�following�variable�selection� procedures�is�suggested?
� a�� Backward�elimination � b�� Forward�selection � c�� Hierarchical�regression � d�� Stepwise�selection 18.10� �I�assert�that�the�forward�selection,�backward�elimination,�and�stepwise�regression�
methods� will� always� arrive� at� the� same� final� model,� given� the� same� dataset� and� level�of�significance?�Am�I�correct?
18.11� �I�assert�the�Radj 2 �will�always�be�larger�for�the�model�with�the�most�predictors��Am�I�
correct? 18.12� �In�a�two-predictor�regression�model,�if�the�correlation�among�the�predictors�is��95�
and�VIF�is�20,�then�we�should�be�concerned�about�collinearity��True�or�false?
Computational problems
18.1� �You�are�given�the�following�data,�where�X1�(hours�of�professional�development)�and� X2�(aptitude�test�scores)�are�used�to�predict�Y�(annual�salary�in�thousands):
Y X1 X2
40 100 10 50 200 20 50 300 10 70 400 30 65 500 20 65 600 20 80 700 30
Determine�the�following�values:�intercept,�b1,�b2,�SSres,�SSreg,�F,�s2res,�s(b1),�s(b2),�t1,�t2�
706 An Introduction to Statistical Concepts
18.2� �You�are�given�the�following�data,�where�X1�(final�percentage�in�science�class)�and� X2�(number�of�absences)�are�used�to�predict�Y�(standardized�science�test�score�in� third�grade):
Y X1 X2
300 65 7 480 98 0 350 70 3 420 80 2 400 82 0 335 70 3 370 75 4 390 80 1 485 99 0 415 95 2 375 88 3
Determine�the�following�values:�intercept,�b1,�b2,�SSres,�SSreg,�F,�s2res,�s(b1),�s(b2),�t1,�t2� 18.3� �Complete�the�missing�information�for�this�regression�model�(df�=�23)�
Y′ = 25.1 + 1.2X1 + 1.0X2 − .50X3
(2�1) (1�5) (1�3) (�06) Standard�errors (11�9) (�) (�) (�) t�ratios
(�) (�) (�) Significant�at��05?
18.4� �Consider�a�sample�of�elementary�school�children��Given�that�r(strength,�weight)�=��6,� r(strength,�age)�=��7,�and�r(weight,�age)�=��8,�what�is�the�first-order�partial�correlation� coefficient�between�strength�and�weight�holding�age�constant?
18.5� �For�a�sample�of�100�adults,�you�are�given�that�r12�=��55,�r13�=��80,�and�r23�=��70��What�is� the�value�of�r1(2�3)?
18.6� �A� researcher� would� like� to� predict� salary� from� a� set� of� four� predictor� variables� for� a� sample� of� 45� subjects�� Multiple� linear� regression� analysis� was� utilized�� Complete� the� following�summary�table�(α�=��05)�for�the�test�of�significance�of�the�overall�regression� model:
Source SS df MS F Critical Value and Decision
Regression — — 20 — Residual 400 — — Total — —
18.7� �Calculate�the�partial�correlation�r12�3�and�the�part�correlation�r1(2�3)�from�the�following� bivariate�correlations:�r12�=��5,�r13�=��8,�r23�=��9�
18.8� �Calculate�the�partial�correlation�r13�2�and�the�part�correlation�r1(3�2)�from�the�following� bivariate�correlations:�r12�=��21,�r13�=��40,�r23�=�−�38�
707Multiple Regression
18.9� �You�are�given�the�following�data,�where�X1�(verbal�aptitude)�and�X2�(prior�reading� achievement)�are�to�be�used�to�predict�Y�(reading�achievement):
Y X1 X2
2 2 5 1 2 4 1 1 5 1 1 3 5 3 6 4 4 4 7 5 6 6 5 4 7 7 3 8 6 3 3 4 3 3 3 6 6 6 9 6 6 8
10 8 9 9 9 6 6 10 4 6 9 5 9 4 8
10 4 9
Determine�the�following�values:�intercept,�b1,�b2,�SSres,�SSreg,�F,�s2res,�s(b1),�s(b2),�t1,�t2� 18.10� �You� are� given� the� following� data,� where� X1� (years� of� teaching� experience)� and� X2�
(salary�in�thousands)�are�to�be�used�to�predict�Y�(morale):
Y X1 X2
125 1 24 130 2 30 145 3 32 115 2 28 170 6 40 180 7 38 165 5 48 150 4 42 195 9 56 180 10 52 120 2 33 190 8 50 170 7 49 175 9 53 160 6 49
Determine�the�following�values:�intercept,�b1,�b2,�SSres,�SSreg,�F,�s2res,�s(b1),�s(b2),�t1,�t2�
708 An Introduction to Statistical Concepts
Interpretive problems
18.1� Use�SPSS�to�develop�a�multiple�regression�model�with�the�example�survey�1�dataset� on�the�website��Utilize�current�GPA�as�the�dependent�variable�and�find�at�least�two� strong�predictors�from�among�the�continuous�variables�in�the�dataset��Write�up�your� results,�including�interpretation�of�effect�size�and�testing�of�assumptions�
18.2� Use�SPSS�to�develop�a�multiple�regression�model�with�the�example�survey�1�dataset� on�the�website��Utilize�how�many�hours�of�television�watched�per�week�as�the�depen- dent� variable� and� find� at� least� two� strong� predictors� from� among� the� continuous� variables�in�the�dataset��Write�up�your�results,�including�interpretation�of�effect�size� and�testing�of�assumptions�
709
19 Logistic Regression
Chapter Outline
19�1� How�Logistic�Regression�Works 19�2� Logistic�Regression�Equation � 19�2�1� Probability � 19�2�2� Odds�and�Logit�(or�Log�Odds) 19�3� Estimation�and�Model�Fit 19�4� Significance�Tests � 19�4�1� Test�of�Significance�of�Overall�Regression�Model � 19�4�2� Test�of�Significance�of�Logistic�Regression�Coefficients 19�5� Assumptions�and�Conditions � 19�5�1� Assumptions � 19�5�2� Conditions 19�6� Effect�Size 19�7� Methods�of�Predictor�Entry � 19�7�1� Simultaneous�Logistic�Regression � 19�7�2� Stepwise�Logistic�Regression � 19�7�3� Hierarchical�Regression 19�8� SPSS 19�9� G*Power 19�10� Template�and�APA-Style�Write-Up 19�11� What�Is�Next?
Key Concepts
� 1�� Logit � 2�� Odds � 3�� Odds�ratio
In� the� past� two� chapters,� we� have� examined� ordinary� least� squares� (OLS)� regression— simple�and�multiple�regression�models—that�allow�us�to�examine�the�relationship�between� one�or�more�predictors�when�the�outcome�is�continuous��In�this�chapter,�we�are�introduced� to� logistic� regression,� which� can� be� used� when� the� outcome� is� categorical�� For� the� pur- poses�of�this�chapter,�we�will�concentrate�on�binary�logistic�regression�which�is�used�when�
710 An Introduction to Statistical Concepts
the� outcome� has� only� two� categories� (i�e�,� dichotomous,� binary,� or� sometimes� referred� to� as�a�Bernoulli�outcome)��The�logistic�regression�procedure�appropriate�for�more�than�two� categories�is�called�multinomial�(or�polytomous)�logistic�regression��Readers�interested�in� learning�more�about�multinomial�logistic�regression�will�be�provided�some�additional�ref- erences�later�in�this�chapter��Also�in�this�chapter,�we�discuss�methods�that�can�be�used�to� enter� predictors� in� logistic� regression� models�� Our� objectives� are� that� by� the� end� of� this� chapter,�you�will�be�able�to�(a)�understand�the�concepts�underlying�logistic�regression,�(b)� determine�and�interpret�the�results�of�logistic�regression,�(c)�understand�and�evaluate�the� assumptions�of�logistic�regression,�and�(d)�have�a�basic�understanding�of�methods�of�enter- ing�the�covariates�
19.1 How Logistic Regression Works
We�conclude�the�textbook�as�Marie�embarks�on�her�most�challenging�statistical�project� to�date�
With�excitement,�Marie�is�finishing�up�her�graduate�program�in�educational�research� and�has�been�assigned�by�her�faculty�advisor�to�one�additional�consultation��Malani�is� a�faculty�member�in�the�early�childhood�department�and�has�collected�data�on�20�chil- dren� who� will� be� entering� kindergarten� in� the� fall�� Interested� in� kindergarten� readi- ness�issues,�Malani�wants�to�know�if�a�teacher�observation�scale�for�social�development� and� family� structure� (single� family� vs�� two-family� home)� can� predict� whether� chil- dren�are�prepared�or�unprepared�to�enter�kindergarten��Marie�suggests�the�following� research�question�to�Malani:�Can kindergarten readiness (prepared vs. unprepared) be pre- dicted by social development and family structure (single family vs. two-family home)?�Given� that�the�outcome�is�dichotomous,�Marie�determines�that�binary�logistic�regression�is� the� appropriate� statistical� procedure� to� use� to� answer� Malani’s� question�� Marie� then� proceeds�with�assisting�Malani�in�analyzing�the�data�
If�the�dependent�variable�is�binary�(i�e�,�dichotomous�or�having�only�two�categories),�then� none� of� the� regression� methods� described� so� far� in� this� text� are� appropriate�� Although� simple� and� multiple� regression� can� easily� accommodate� dichotomous� independent� vari- ables�through�dummy�coding�(i�e�,�assignment�of�1�and�0�to�the�categories),�it�is�an�entirely� different� case� when� the� outcome� is� dichotomous�� Applying� OLS� regression� to� a� binary� outcome� creates� problems�� For� example,� a� dichotomous� outcome� violates� normality� and� homogeneity�assumptions�in�OLS�regression��In�addition,�OLS�estimates�are�based�on�lin- ear�relationships�between�the�independent�and�dependent�variables,�and�forcing�a�linear� relationship�(as�seen�in�Figure�19�1)�in�the�case�of�a�binary�outcome�is�erroneous�[although� we�found�at�least�one�author�(Hellevik,�2009)�who�argues�that�OLS�regression�can�be�used� with�dichotomous�outcomes]�
As�part�of�the�regression�family,�logistic�regression�still�allows�a�prediction�to�be�made;� however,�now�the�prediction�is�whether�or�not�the�unit�under�investigation�falls�into�one� of�the�two�categories�of�the�dependent�variable��Initially�used�mostly�in�the�hard�sciences,� this�method�has�become�more�broadly�popular�in�recent�years�as�there�are�many�situations� where� researchers� want� to� examine� outcomes� that� are� discrete,� rather� than� continuous,� in� nature�� Some� examples� of� dichotomous� dependent� variables� are� pass/fail,� surviving�
711Logistic Regression
surgery/not,� admit/reject,� vote� for/against,� employ/not,� win/lose,� or� purchase/not�� The� idea� of� using� a� dichotomous� variable� was� introduced� in� Chapter� 18� as� the� concept� of� a� dummy variable,�where�the�first�condition�is�indicated�by�a�value�of�1�(e�g�,�prepared�for� kindergarten),�whereas�a�value�of�0�indicates�the�opposite�condition�(e�g�,�unprepared� for�kindergarten)��For�the�purposes�of�this�text,�our�discussion�will�concentrate�on�dichoto- mous� outcomes� where� logistic� regression� is� appropriate� (i�e�,� binary� logistic� regression,� referred� to� throughout� this� chapter� simply� as� logistic� regression)�� Conditions� for� which� there�are�more�than�two�possible�categories�for�the�dependent�variable�(e�g�,�three�catego- ries,�such�as�remain�in�the�teaching�profession,�remain�in�teaching�but�change�schools,�or� leave�the�teaching�profession�entirely),�multinomial�logistic�regression�may�be�appropri- ate��An�example�of�the�data�structure�for�a�logistic�regression�model�with�a�binary�outcome� (prepared�vs��unprepared�for�kindergarten),�one�continuous�predictor�(social�development),� and� one� dichotomous� dummy� coded� predictor� (family� structure:� single-parent� vs�� two-parent�home)�is�presented�in�Table�19�1�
19.2 Logistic Regression Equation
As�we�learned�previously�with�OLS�regression,�knowledge�of�the�independent�variable(s)� provides�the�information�necessary�to�be�able�to�estimate�a�precise�numerical�value�of�the� dependent�variable,�a�predicted�value��The�following�formula�recaps�the�sample�multiple� regression� equation� where� Y� is� the� predicted� outcome� for� individual� i� based� on� (a)� the� Y intercept,�a,�the�value�of�Y�when�all�predictor�values�are�0;�(b)�the�product�of�the�value� of�the�independent�variables,�Xs,�and�the�regression�coefficients,�bk;�and�(c)�the�residual,�εi:
Y a b X b Xi m m i= + + + +1 1 ... ε
Age (months) at kindergarten entry
Re ad
in g
pr o�
ci en
t
50
.00
.20
.40
.60
.80
1.00
55 60 65 70 75 80
FIGuRe 19.1 Nonlinearity�of�binary�outcome�
712 An Introduction to Statistical Concepts
As�we�see,�the�logistic�regression�equation�is�similar�in�concept�to�simple�and�multiple�lin- ear�regression,�but�operates�much�differently��In�logistic�regression,�the�binary�dependent� variable� is� transformed� into� a� logit� variable� (which� is� the� natural� log� of� the� odds� of� the� dependent� variable� occurring� or� not� occurring),� and� the� parameters� are� then� estimated� using�maximum�likelihood��The�end�result�is�that�the�odds�of�an�event�occurring�are�esti- mated�through�the�logistic�regression�model�(whereas�OLS�estimates�a�precise�numerical� value�of�the�dependent�variable)�
To�understand�how�the�logistic�regression�equation�operates,�there�are�three�primary�com- putational�concepts�that�must�be�understood:�probability,�odds,�and�the�logit��These�express� the�same�thing,�only�in�different�ways�(Menard,�2000)��Let�us�first�consider�probability�
19.2.1 probability
The� overarching� difference� between� OLS� regression� (i�e�,� simple� and� multiple� linear� regres- sion)� and� logistic� regression� is� the� measurement� scale� of� the� outcome�� With� OLS� regres- sion,�our�outcome�is�continuous�in�scale�(i�e�,�interval�or�ratio�measurement�scale)��In�binary� logistic�regression,�our�outcome�is�dichotomous—one�of�two�categories��Let�us�use�kinder- garten� readiness� (“prepared� for� kindergarten”� coded� as� “1”� vs�� unprepared� coded� as� “0”)� as�an�example�of�our�logistic�regression�outcome��Therefore,�what�the�regression�equation� allows� us� to� predict� is� substantially� different� for� OLS� as� compared� to� logistic� regression�� In� comparison� to� OLS,� which� allows� us� to� compute� a� precise� numerical� value� (e�g�,� a� spe- cific�predicted�score�for�the�dependent�variable),�the�logistic�regression�equation�allows�us� to�compute�a�probability—more�specifically,�the�probability�that�the�dependent�variable�will�
Table 19.1
Kindergarten�Readiness�Example�Data
Child Social Development (X1) Family Structure (X2) Kindergarten Readiness (Y )
1 15 Single�family�(0) Unprepared�(0) 2 12 Single�family�(0) Unprepared�(0) 3 18 Single�family�(0) Prepared�(1) 4 20 Single�family�(0) Prepared�(1) 5 11 Single�family�(0) Unprepared�(0) 6 17 Single�family�(0) Prepared�(1) 7 14 Single�family�(0) Unprepared�(0) 8 18 Single�family�(0) Prepared�(1) 9 13 Single�family�(0) Unprepared�(0)
10 10 Single�family�(0) Unprepared�(0) 11 22 Two-parent�home�(1) Unprepared�(0) 12 25 Two-parent�home�(1) Prepared�(1) 13 23 Two-parent�home�(1) Prepared�(1) 14 21 Two-parent�home�(1) Prepared�(1) 15 30 Two-parent�home�(1) Prepared�(1) 16 27 Two-parent�home�(1) Prepared�(1) 17 26 Two-parent�home�(1) Prepared�(1) 18 28 Two-parent�home�(1) Prepared�(1) 19 24 Two-parent�home�(1) Unprepared�(0) 20 30 Two-parent�home�(1) Prepared�(1)
713Logistic Regression
occur��The logistic�regression�equation,�therefore,�generates�predicted�probabilities�that�fall� between� values� of� 0� and� 1�� The� probability� of� a� case� or� unit� being� classified� into� the� low- est� numerical� category� [i�e�,� P(Y� =� 0),� or� in� the� case� of� our� example,� the� probability� that� a� child�will�be�unprepared�for�kindergarten]�is�equal�to�1�minus�the�probability�that�it�falls� within�the�highest�numerical�category�[i�e�,�P(Y�=�1),�or�the�probability�that�a�child�will�be� prepared�for�kindergarten]��This�equates�to�P(Y�=�0)�=�1�−�P(Y�=�1)��Applied�to�our�example,� the�probability�that�a�child�will�be�unprepared�for�kindergarten�is�equal�to�1�minus�the�prob- ability�that�a�child�will�be�prepared�for�kindergarten��In�other�words,�the�knowledge�of�the� probability�of�one�category�occurring�(e�g�,�unprepared�for�kindergarten)�allows�us�to�easily� determine�the�probability�that�the�other�category�will�occur�(e�g�,�prepared)�as�the�total�prob- ability�must�equal�1�0��Remember,�however,�that�probabilities�have�to�fall�within�the�range� of�0�to�1��As�we�know�from�Chapter�5,�it�is�not�possible�to�have�a�negative�probability,�nor�is� it�possible�to�have�a�probability�greater�than�1�(i�e�,�greater�than�100%)��If�we�try�to�model�the� probability�as�the�dependent�variable�in�our�OLS�equation,�it�is�mathematically�possible�that� the�predicted�values�would�be�negative�or�greater�than�1—values�that�are�outside�the�range� of�what�is�feasible�when�considering�probabilities��Therefore�this�is�where�our�logistic�regres- sion�equation�takes�a�turn�from�what�we�learned�with�linear�regression�
19.2.2 Odds and logit (or log Odds)
So� far,� we� have� talked� about� the� outcome� of� our� logistic� regression� equation� as� being� a� probability,�and�we�also�know�that�predicted�probabilities�must�be�between�0�and�1��As�we� think�about�how�to�estimate�probabilities,�we�will�see�that�this�takes�a�few�steps�to�achieve�� Rather�than�the�dependent�variable�being�a�probability,�if�it�were�an�odds value,�then�values� greater�than�1�would�be�possible�and�appropriate��Odds�are�simply�the�ratio�of�the�prob- ability�of�the�dependent�variable’s�two�outcomes��The�odds�that�the�outcome�of�a�binary� variable�is�1�(i�e�,�public�school�attendance)�rather�than�0�(or�private�school�attendance)�is� simply�the�ratio�of�the�odds�that�Y�is�equal�1�to�the�odds�that�Y�does�not�equal�1��In�math- ematical�terms,�this�can�be�written�as�follows:
Odds Y P Y P Y
( ) ( )
( ) = =
= − =
1 1
1 1
As�we�see�in�Table�19�2,�when�the�probability�that�Y�=�1�(e�g�,�prepared�for�kindergarten)� equals��50�(column�1�in�Table�19�2),�then�1�−�P(Y = 1)�(or�unprepared�for�kindergarten)� is��50�(column�2)�and�the�odds�are�equal�to�1�00�(column�3)��When�the�probability�of�Y�=�1�
Table 19.2
Illustration�of�Logged�Odds
P(Y = 1) 1 − P(Y = 1) Odds Y P Y P Y
( ) ( 1)
1 ( 1) = =
= − =
1 ln[ ( 1)] ln ( 1)
1 ( 1) Odds Y
P Y P Y
= = =
− =
�001 �999 �001/�999�=��001 ln(�001)�=�−6�908 �100 �900 �100/�900�=��111 ln(�111)�=�−2�198 �300 �700 �300/�700�=��429 ln(�429)�=�−�846 �500 �500 �500/�500�=�1�000 ln(1�000)�=��000 �700 �300 �700/�300�=�2�333 ln(2�333)�=��847 �900 �100 �900/�100�=�9�000 ln(9�000)�=�2�197 �999 �001 �999/�001�=�999�000 ln(999)�=�6�907
714 An Introduction to Statistical Concepts
(e�g�,�prepared)�is�very�small�(say,��100�or�less),�then�the�odds�for�being�prepared�for�kindergar- ten�are�also�very�small�and�approach�0�the�smaller�the�probability�that�Y�=�1�(i�e�,�the�smaller� the�probability�that�a�child�is�prepared�for�kindergarten)��However,�as�the�probability�of�Y�=�1� (e�g�,�being�prepared�for�kindergarten)�increases,�the�odds�(column�3)�increase�tremendously�� Thus,�the�issue�that�we�are�faced�with�when�using�odds�is�that�while�odds�can�be�infinitely� large,�we�are�still�limited�in�that�the�minimum�value�is�0�and�we�still�do�not�have�data�that�can� be�modeled�linearly�
Changing�the�scale�of�the�odds�by�taking�the�natural�logarithm�of�the�odds�(also�called� logit Y�or�log odds)�provides�us�with�a�value�of�the�dependent�variable�that�can�theoretically� range�from�negative�infinity�to�positive�infinity��Thus,�taking�the�log�odds�of�Y�creates�a� linear�relationship�between�X�and�the�probability�of�Y�(Pampel,�2000)��The�natural�log�of� the�odds�is�calculated�as�follows�with�the�residual�being�the�difference�between�the�pre- dicted�probability�and�the�actual�value�of�the�dependent�variable�(0�or�1):
ln ( )
( ) ( )
P Y P Y
Logit Y =
− = =
1 1 1
In�column�4�of�Table�19�2,�we�see�what�happens�when�the�logit�transformation�is�made��As� the�odds�increase�from�1�to�positive�infinity,�the�logit�(or�log�odds)�of�Y�becomes�larger�and� larger�(and�remains�positive)��As�the�odds�decrease�from�1�to�0,�the�logit�(or�log�odds)�of� Y is�negative�and�grows�larger�and�larger�(in�absolute�value)�
The� logit� of� Y� equation� is� interpreted� very� similarly� to� that� of� OLS�� For� each� one-unit� change�in�the�independent�variable,�the�logistic�regression�coefficients�represent�the�change� in� the� predicted� log� odds� of� being� in� a� category�� In� comparison� to� OLS� regression,� the� regression�coefficients�have�the�exact�same�interpretation��The�difference�in�interpretation� with�logistic�regression�is�that�the�outcome�now�represents�a�log odds�rather�than�a�precise� numerical�value�as�we�saw�with�OLS�regression��Linking�the�logit�back�to�probabilities,�a� one-unit�change�in�the�logit�equals�a�bigger�change�in�probabilities�that�are�near�the�center� as� compared� to� the� extreme� values�� This� happens� because� of� the� linearization� once� we� take�the�natural�log��Taking�the�natural�log�stretches�the�S-shaped�curve�into�a�linear�form;� thus,�the�values�at�the�extreme�are�stretched�less,�so�to�speak,�as�compared�to�the�values� in�the�middle�(Pampel,�2000)��By�working�with�log�odds,�our�familiar�additive�regression� equation�is�applicable:
ln ( )
( ) ( ) ...
P Y P Y
Logit Y X X Xm m =
− = = = + + + +
1 1 1
1 1 2 2α β β β
It� is� important� to� note� that� although� we� were� accustomed� to� examining� standardized� regression�coefficients�in�OLS�regression,�it�is�not�the�norm�that�standardized�coefficients� are� computed� for� logistic� regression� models� by� statistical� software�� Standardization� is� ordinarily� accomplished� by� taking� the� product� of� the� unstandardized� regression� coef- ficient� and� the� ratio� of� the� standard� deviation� of� X� to� the� standard� deviation� of� Y�� The� interpretation�of�a�standard�deviation�change�in�a�continuous�variable�thus�makes�sense;� however,�this�is�not�the�case�for�a�dichotomous�variable,�nor�is�it�the�case�for�the�log�odds� (which�is�the�predicted�outcome�and�which�does�not�have�a�standard�deviation)�
While� interpretation� of� the� logistic� equation� is� relatively� straightforward� as� it� holds� many�similarities�to�OLS�regression,�log�odds�are�not�a�metric�that�we�use�often��Therefore� understanding�what�it�means�when�a�predictor,�X,�has�some�effect�on�the�log�odds,�Y,�can� be�difficult��This�is�where�odds�come�back�into�the�picture�
715Logistic Regression
If�we�exponentiate�the�logit�(Y)�(i�e�,�the�outcome�of�our�logistic�regression�equation),�then� it�converts�back�to�the�odds�(see�the�following�equation)��Now�we�can�interpret�the�inde- pendent�variables�as�affecting�the�odds�(rather�than�log�odds)�of�the�outcome:
Odds Y e e eY Odds Y X X Xm m( ) (log ( ) ln[ ( )] ...= = = = == + + + +1 1 1 1 2 2it α β β β ee e e eX X Xm mα β β β)( )( )...( )1 1 2 2
As� can� be� seen� here,� the� exponentiation� creates� an� equation� that� is� multiplicative� rather� than�additive,�and�this�then�changes�the�interpretation�of�the�exponentiated�coefficients��In� previous�regression�equations�we�have�studied,�when�the�product�of�the�regression�coef- ficient�and�its�predictor�is�0,�that�variable�adds�nothing�to�the�prediction�of�the�dependent� variable�� In� a� multiplicative� environment,� a� value� of� 0� corresponds� to� a� coefficient� of� 1�� In�other�words,�a�coefficient�of�1�will�not�change�the�value�of�the�odds�(i�e�,�the�outcome)�� Coefficients� greater� than� 1� increase� the� odds,� and� coefficients� less� than� 1� decrease� the� odds��In�addition,�the�odds�will�change�more�the�greater�the�distance�the�value�is�from�1�
Converting�the�odds�back�to�a�probability�can�be�done�through�the�following�formula:
P Y Odds Y Odds Y
e e
X X X
X
m m
( ) ( )
( )
...
= = =
+ = =
+
+ + + +
+1 1
1 1 1
1 1 2 2
1 1
α β β β
α β ++ + +β β2 2X Xm m...
Probability�values�close�to�1�indicate�increased�likelihood�of�occurrence��In�our�example,� since�“1”�indicates�public�school�attendance,�a�probability�close�to�1�would�indicate�a�child� was� more� likely� to�attend�public� school��Children� with� probabilities� close� to�0�suggest�a� decreased�probability�of�attending�public�school�(and�increased�probability�of�attending� private�school)�
19.3 Estimation and Model Fit
Now�that�we�understand�the�logistic�regression�process�and�resulting�equations�a�bit�better,� it�is�time�to�turn�our�attention�to�how�the�equation�is�estimated�and�how�we�can�determine� how� well� the� model� fits�� We� previously� learned� with� simple� and� multiple� regression� that� the�data�from�the�observed�values�of�the�independent�variables�in�the�sample�were�used�to� estimate�or�predict�the�values�of�the�dependent�variable��In�logistic�regression,�we�are�also� using�the�knowledge�of�the�values�of�our�predictor(s)�to�estimate�the�outcome�(i�e�,�log�odds)�� Now�we�are�using�a�method�called�maximum�likelihood�estimation�to�estimate�the�values� of�the�parameters�(i�e�,�the�logistic�coefficients)��As�we�just�learned,�the�dependent�variable�in� a�logistic�regression�model�is�transformed�into�a�logit�value,�which�is�the�natural�log�of�the� odds�of�the�dependent�variable�occurring�or�not�occurring��Maximum�likelihood�estimation� is�then�applied�to�the�model�and�estimates�the�odds�of�occurrence�after�transformation�into� the�logit��Very�simply,�maximum�likelihood�estimates�the�parameters�most�likely�to�occur� given�the�patterns�in�the�sample�data��Whereas�in�OLS�the�sum�of�squared�distance�of�the� observed�data�to�the�regression�line�was�minimized,�in�maximum�likelihood�the�log�likeli- hood�is�maximized�
The� log� of� the� likelihood� function� (sometimes� abbreviated� as� LL)� that� results� from� ML�estimation�then�reflects�the�likelihood�of�observing�the�sample�statistics�given�the�
716 An Introduction to Statistical Concepts
population� parameters�� The� log� likelihood� provides� an� index� of� how� much� has� not� been� explained� in� the� model� after� the� parameters� have� been� estimated,� and� as� such,� the�LL�can�be�used�as�an�indicator�of�model�fit��The�values�of�the�log�likelihood�func- tion� vary� from� 0� to� negative� infinity,� with� values� closer� to� 0� suggesting� better� model� fit�and�larger�values�(in�absolute�value�terms)�indicating�poorer�fit��The�log�likelihood� value�will�approach�0�the�closer�the�likelihood�value�is�to�1��When�this�happens,�this� suggests� the� observed� data� could� be� generated� from� these� population� parameters�� In� other�words,�the�smaller�the�log�likelihood,�the�better�the�model�fit��It�follows�therefore,� that� the� log� likelihood� value� will� grow� more� negative� the� closer� the� likelihood� func- tion� is� to� 0�� This� suggests� that� the� observed� data� are� less� likely� to� be� generated� from� these�population�parameters�
Maximum�likelihood�estimation�performed�by�statistical�software�usually�begins�the� estimation�process�with�all�regression�coefficients�equal�to�the�most�conservative�esti- mate�(i�e�,�the�least�squares�estimates)��Better�model�fit�is�accomplished�through�the�use� of�an�algorithm�which�generates�new�sets�of�regression�coefficients�that�produce�larger� log�likelihoods��This�is�an�iterative�process�that�stops�when�the�selection�of�new�param- eters�creates�very�little�change�in�the�regression�coefficients�and�very�small�increases�in� the�log�likelihood—so�small�that�there�is�little�value�in�any�further�estimation�
19.4 Significance Tests
As� with� multiple� regression,� there� are� two� tests� of� significance� in� logistic� regression�� Specifically,�these�involve�testing�the�significance�of�the�overall�logistic�regression�model� and�testing�the�significance�of�each�of�the�logistic�regression�coefficients�
19.4.1 Test of Significance of Overall Regression Model
The� first� test� is� the� test� of� statistical� significance� to� determine� overall� model� fit� and� provides� evidence� of� the� extent� to� which� the� predicted� values� accurately� represent� the� observed� values� (Xie,� Pendergast,� &� Clarke,� 2008)�� We� consider� several� overall� model� tests�including�(a)�change�in�log�likelihood,�(b)�Hosmer–Lemeshow�goodness-of-fit�test,� (c)� pseudovariance� explained,� and� (d)� predicted� group� membership�� Additional� work� (e�g�,� Xie� et� al�,� 2008)� has� recently� been� conducted� on� new� methods� to� assess� model� fit,� but�these�are�not�currently�available�in�statistical�software�nor�easily�computed��Also�in� this�section,�we�briefly�address�sensitivity,�specificity,�false�positive,�false�negative,�and� cross�validation�
19.4.1.1 Change in Log Likelihood
One�way�to�test�overall�model�fit�is�the�likelihood�ratio�test��This�test�is�based�on�the�change� in�the�log�likelihood�function�from�a�smaller�model�(often�the�baseline�or�intercept�only� model)�to�a�larger�model�that�includes�one�or�more�predictors�(sometimes�referred�to�as� the�fitted�model)��Although�we�indicate�that�the�smaller�model�is�often�the�intercept�only� model,�this�test�can�also�be�used�to�examine�changes�in�model�fit�from�one�fitted�model�to� another�fitted�model�and�we�will�discuss�this�in�a�bit��This�likelihood�ratio�test�is�similar�
717Logistic Regression
to�the�overall�F�test�in�OLS�regression�and�tests�the�null�hypothesis�that�all�the�regression� coefficients�are�equal�to�0��Using�statistical�notation,�we�can�denote�the�null�and�alternative� hypotheses�for�the�regression�coefficients�as�follows:
H
H H is false
m0 1 2
1 0
0: ...
:
β β β= = = =
For� explanation� purposes,� we� assume� the� smaller� model� is� the� baseline� or� intercept� only� model�� The� baseline� log� likelihood� is� estimated� from� a� logistic� regression� model� that� includes� only� the� constant� (i�e�,� intercept)� term�� The� model� log� likelihood� is� esti- mated� from� the� logistic� regression� model� that� includes� the� constant� and� the� relevant� predictor(s)��By�multiplying�the�difference�in�these�log�likelihood�functions�by�−2,�a�chi- square�test�is�produced�with�degrees�of�freedom�equal�to�the�difference�in�the�degrees�of� freedom�of�the�models�(df�=�dfmodel�−�df baseline)�(where�“model”�refers�to�the�fitted�model�that� includes�one�or�more�predictors)��In�the�case�of�the�constant�only�model,�there�is�only�one� parameter�estimated�(i�e�,�the�intercept),�so�there�is�only�one�degree�of�freedom��In�mod- els�that�include�independent�variables,�the�degrees�of�freedom�are�equal�to�the�number� of� independent� variables� in� the� model� plus� one� for� the� constant�� The� larger� the� differ- ence�between�the�baseline�and�model�LL�values,�the�better�the�model�fit��It�is�important� to�note�that�the�log�likelihood�difference�test�assumes�nested�models��In�other�words,�all� elements�that�are�included�in�the�baseline�or�smallest�model�must�also�be�included�in�the� fitted�model��As�alluded�to�previously,�the�change�in�log�likelihood�test�can�be�used�for� more�than�just�comparing�the�intercept�only�model�to�a�fitted�model��Researchers�often� use�this�test�in�the�model�building�process�to�determine�if�adding�predictors�(or�sets�of� predictors)�aids�in�model�fit�by�comparing�one�fitted�model�to�another�fitted�model��In� general,�the�change�in�log�likelihood�is�computed�as�follows:
χ2 2= − −( )LL LLbaselinemodel
19.4.1.2 Hosmer–Lemeshow Goodness-of-Fit Test
The�Hosmer–Lemeshow�goodness-of-fit�test�is�another�tool�that�can�be�used�to�examine� overall� model� fit�� The� Hosmer–Lemeshow� statistic� is� computed� by� dividing� cases� into� deciles� (i�e�,� 10� groups)� based� on� their� predicted� probabilities�� Then� a� chi-square� value� is�computed�based�on�the�observed�and�expected�frequencies��This�is�a�chi-square�test� for� which� the� researcher� does� not� want� to� find� statistical� significance�� Nonstatistically� significant�results�for�the�Hosmer–Lemeshow�test�indicate�the�model�has�acceptable�fit�� In� other� words,� the� predicted� or� estimated� model� is� not� statistically� significantly� dif- ferent� from� the� observed� values�� Although� the� Hosmer–Lemeshow� test� can� easily� be� requested� in� SPSS,� it� has� been� criticized� for� being� conservative� (i�e�,� lacking� sufficient� power�to�detect�lack�of�fit�in�instances�such�as�nonlinearity�of�an�independent�variable),� too� likely� to� indicate� model� fit� when� five� or� fewer� groups� (based� on� the� decile� groups� created�in�computing�the�statistic)�are�used�to�calculate�the�statistic,�and�offers�little�diag- nostics�to�assist�the�researcher�when�the�test�indicates�poor�model�fit�(Hosmer,�Hosmer,� LeCessie, &�Lemeshow,�1997)�
718 An Introduction to Statistical Concepts
19.4.1.3 Pseudovariance Explained
Another�overall�model�fit�index�for�logistic�regression�is�pseudovariance�explained��This� index�is�akin�to�multiple�R2�(or�the�coefficient�of�determination)�in�OLS�regression,�and�can� also�be�considered�an�effect�size�measure�for�the�model��The�reason�these�values�are�con- sidered�pseudovariance�explained�in�logistic�regression�is�that�the�variance�in�a�dichoto- mous�outcome,�as�evident�in�logistic�regression,�differs�as�compared�to�the�variance�of�a� continuous�outcome,�as�present�in�OLS�regression�
There�are�a�number�of�multiple�R2�pseudovariance�explained�values�that�can�be�computed� in�logistic�regression��We�discuss�the�following:�(a)�Cox�and�Snell�(1989),�(b)�Nagelkerke�(1991),� (c)�Hosmer�and�Lemeshow�(1989),�(d)�Aldrich�and�Nelson�(1984),�(e)�Harrell�(1986),�and�(f)�tra- ditional�R2��Of�these,�SPSS�automatically�computes�the�Cox�and�Snell�and�Nagelkerke�indi- ces��There�is,�however,�no�consensus�on�which�(if�any)�of�the�pseudovariance�explained� indices�are�best,�and�many�researchers�choose�not�to�report�any�of�them�in�their�published� results��If�you�do�choose�to�use�and/or�report�one�or�more�of�these�values,�they�should�be� used� only� as� a� guide� “without� attributing� great� importance� to� a� precise� figure”� (Pampel,� 2000,�p��50)�
The�Cox�and�Snell�R2�(1989)�is�computed�as�the�ratio�of�the�likelihood�values�raised�to� the�power�of�2/n�(where�n�is�sample�size)��A�problem�is�that�the�computation�is�such�that� the�theoretical�maximum�of�1�cannot�be�obtained,�even�when�there�is�perfect�prediction:
R LL LL
CS baseline
el
n 2
2
1= −
mod
/
Nagelkerke�(1991)�adjusts�the�Cox�and�Snell�value�so�that�the�maximum�value�of�1�can�be� achieved,�and�it�is�computed�as�follows:
R R
LL N
CS
baseline n
2 2
21 =
− ( ) /
Hosmer�and�Lemeshow’s�(1989)�R2�is�the�proportional�reduction�in�the�log�likelihood�(in� absolute� value� terms)�� Although� not� provided� by� SPSS,� it� can� easily� be� computed� by� the� ratio�of�the�model�to�baseline�−2LL��Ranging�from�0�to�1,�this�value�provides�an�indication� of�how�much�the�badness�of�fit�of�the�baseline�model�is�improved�by�the�inclusion�of�the� predictors�in�the�fitted�model��Hosmer�and�Lemeshow’s�(1989)�R2�is�computed�as
R LL LL
L model
baseline
2 2 2
= − −
Harrell� (1986)� proposed� that� Hosmer� and� Lemeshow’s� R2� be� adjusted� for� the� number� of� parameters�(i�e�,�independent�variables)�in�the�model��This�adjustment�(where�m�equals�the� number� of� independent� variables� in� the� model)� to� the� computation� makes� this� R2� value� akin�to�the�adjusted�R2�in�OLS�regression��It�is�computed�as
R LL m LL
LA model
baseline
2 2 2 2
= − −
− ( )
719Logistic Regression
Aldrich�and�Nelson�(1984)�provided�an�alternative�to�the�RL 2�that�is�equivalent�to�the�squared�
contingency�coefficient��This�measure�has�the�same�problem�as�the�Cox�and�Snell�R2;�the� theoretical�maximum�of�1�cannot�be�obtained�even�when�the�independent�variable(s)�per- fectly�predict�the�outcome��It�is�computed�as
pseudoR LL
LL n model
model
2 2 2
= −
− +
The�traditional�R2,�the�coefficient�of�determination�as�used�in�simple�and�multiple�regres- sion,� can� also� be� used� in� logistic� regression� (only� with� binary� logistic� regression,� as� the� mean�and�variance�of�a�dichotomous�variable�make�sense;�however�the�mean,�for�example,� in�a�dummy�coded�variable�situation,�is�equal�to�the�proportion�of�cases�in�the�category� labeled�as�1)��R2�can�be�computed�by�correlating�the�observed�values�of�the�binary�depen- dent� variable� with� the� predicted� values� (i�e�,� predicted� probabilities)� obtained� from� the� logistic� regression� model� and� then� squaring� the� correlated� value�� Predicted� probability� values�can�easily�be�saved�when�generating�logistic�regression�models�in�SPSS�
19.4.1.4 Predicted Group Membership
Another� test� of� model� fit� for� logistic� regression� can� be� accomplished� by� evaluating� predicted� to� observed� group� membership�� Assuming� a� cut� value� of� �50,� cases� with� predicted� probabilities� at� �5� or� above� are� predicted� as� 1� and� predicted� probabilities� below �5�are�predicted�as�0��A�crosstab�table�of�predicted�to�observed�predicted�prob- abilities� provides� the� frequency� and� percentage� of� cases� correctly� classified�� Correct� classification� would� be� seen� in� cases� that� have� the� same� value� for� both� the� predicted� and� observed� values�� A� perfect� model� produces� 100%� correctly� classified� cases�� A� model� that� classifies� no� better� than� chance� would� provide� 50%� correctly� classified� cases��Press’s�Q�is�a�chi-square�statistic�with�one�degree�of�freedom�and�can�be�used�as� a�formal�test�of�classification�accuracy��It�is�computed�as
Q N nK N K
= −[ ]
− ( )
( )
2
1
where N�is�the�total�sample�size n�represents�the�number�of�cases�that�were�correctly�classified K�equals�the�number�of�groups
As�with�other�chi-square�statistics�we�have�examined,�this�test�is�sensitive�to�sample�size�� Also,�it�is�important�to�note�that�focusing�solely�on�the�correct�classification�overall�(as�is� done�with�Press’s�Q)�may�result�in�overlooking�one�or�more�groups�that�have�unacceptable� classification��The�researcher�should�evaluate�the�classification�of�each�group�in�addition� to�the�overall�classification�
Sensitivity� is� the� probability� that� a� case� coded� as� 1� for� the� dependent� variable� (a�k�a�� “positive”)� is� classified� correctly�� In� other� words,� sensitivity� is� the� percentage� of� correct� predictions�of�the�cases�that�are�coded�as�1�for�the�dependent�variable��In�the�kindergarten� readiness�example�that�we�will�review�later,�of�those�12�children�who�were�prepared�for�
720 An Introduction to Statistical Concepts
kindergarten�(i�e�,�coded�as�1�for�the�dependent�variable),�11�were�correctly�classified��Thus,� the�sensitivity�is�11/12�or�about�92%�
Specificity� is� the� probability� that� a� case� coded� as� 0� for� the� dependent� variable� (a�k�a�� “negative”)� is� classified� correctly�� In� other� words,� specificity� is� the� percentage� of� correct� predictions�of�the�cases�that�are�coded�as�0�for�the�dependent�variable��In�the�kindergarten� readiness�example�that�we�will�review�later,�of�those�8�children�who�were�unprepared�for� kindergarten�(i�e�,�coded�as�0�for�the�dependent�variable),�7�were�correctly�classified��Thus,� the�specificity�is�7/8,�or�87�5%�
False positive rate�is�the�probability�that�a�case�coded�as�0�for�the�dependent�variable� (a�k�a�� “negative”)� is� classified� incorrectly�� In� other� words,� this� is� the� percentage� of� cases� in�error�where�the�dependent�variable�is�predicted�to�be�1�(i�e�,�prepared),�but�in�fact�the� observed�value�is�0�(i�e�,�unprepared)��In�the�kindergarten�readiness�example�that�we�will� review�later,�of�those�8�children�who�were�unprepared�for�kindergarten�(i�e�,�coded�as�0�for� the�dependent�variable),�1�was�incorrectly�classified��Thus,�the�false�positive�rate�is�1/8,�or� 12�5%��The�false�positive�rate�is�also�computed�as�1�minus�specificity�
False negative rate�is�the�probability�that�a�case�coded�as�1�for�the�dependent�variable� (a�k�a��“positive”)�is�classified�incorrectly��In�other�words,�this�is�the�percentage�of�cases�in� error�where�the�dependent�variable�is�predicted�to�be�0�(i�e�,�unprepared),�but�in�fact�the� observed� value� is� 1� (i�e�,� prepared)�� In� the� kindergarten� readiness� example� that� we� will� review�later,�of�those�12�children�who�were�prepared�for�kindergarten�(i�e�,�coded�as�1�for� the�dependent�variable),�1�was�incorrectly�classified��Thus,�the�false�negative�rate�is�1/12,�or� about�8%��The�false�negative�rate�is�also�computed�as�1�minus�sensitivity�
19.4.1.5 Cross Validation
A�recommended�best�practice�in�logistic�regression�is�to�cross�validate�the�results��If�the� sample� size� is� sufficient,� this� can� be� accomplished� by� using� 75%–80%� of� the� sample� to� derive�the�model�and�then�use�the�remaining�cases�(the�holdout�sample)�to�determine�its� accuracy��With�cross�validation,�you�are�in�essence�testing�the�model�on�two�samples— a� primary� sample� (which� represents� the� largest� percentage� of� the� sample� size)� and� a� holdout�sample�(that�which�remains)��If�classification�accuracy�of�the�holdout�sample�is� within� 10%� of� the� primary� sample,� this� provides� evidence� of� the� utility� of� the� logistic� regression�model�
19.4.2 Test of Significance of logistic Regression Coefficients
The�second�test�in�logistic�regression�is�the�test�of�the�statistical�significance�of�each�regres- sion�coefficient,�bk��This�test�allows�us�to�determine�if�the�individual�coefficients�are�statisti- cally�significantly�different�from�0��The�null�and�alternative�hypotheses�can�be�illustrated� in�the�same�mathematical�notation�as�we�used�with�OLS�regression:
H
H
k
k
0
1
0
0
:
:
β
β
=
≠
Interpreting�the�test�provides�evidence�of�the�probability�of�obtaining�the�observed�sample� coefficient�by�chance�if�the�null�hypothesis�was�true�(i�e�,�if�the�population�regression�coef- ficient� value� was� 0)�� The� Wald� statistic,� which� follows� a� chi-square� distribution,� is� used�
721Logistic Regression
as� the� test� statistic� for� regression� coefficients� in� SPSS�� For� continuous� predictors,� this� is� calculated�by�squaring�the�ratio�of�the�regression�coefficient�divided�by�its�standard�error:
W SE
k
k
= β
β
2
2
When�the�logistic�regression�coefficients�are�large�(in�absolute�value),�rounding�error�can� create�imprecision�in�estimation�of�the�standard�errors��This�can�result�in�inaccuracies�in� testing�the�null�hypothesis,�and�more�specifically,�increased�Type�II�errors�(i�e�,�failing�to� reject� the� null� hypothesis� when� the� null� hypothesis� is� false)�� An� alternative� to� the� Wald� test,�in�situations�such�as�this,�is�the�difference�in�log�likelihood�test�previously�described� to�compare�models�with�and�without�the�variable�of�interest�(Pampel,�2000)�
Raferty�(1995)�proposed�a�Bayesian�information�criterion�(BIC),�computed�as�the�differ- ence�between�the�chi-square�value�and�the�natural�log�of�the�sample�size,�that�could�also� be�applied�to�testing�logistic�regression�coefficients:
BIC n= −χ2 ln
To�reject�the�null�hypothesis,�the�BIC�should�be�positive�(i�e�,�greater�than�0)��That�is,�the�chi- square�value�must�be�greater�than�the�natural�log�of�the�sample�size��BIC�values�below�0�suggest� that�the�variable�contributes�little�to�the�model��BIC�values�between�0�and�+2�are�considered� weak;�between�2�and�6,�positive;�between�6�and�10,�strong;�and�more�than�10,�very�strong�
Beyond� determining� statistical� significance� of� the� individual� predictors,� you� may� also� want�to�assess�which�predictors�are�adding�the�most�to�the�model��In�OLS�regression,�we� examined�the�standardized�regression�coefficients��There�are�no�traditional�standardized� regression�coefficients�provided�in�SPSS�for�logistic�regression,�but�they�are�easy�to�calcu- late�� Simply� standardize� the� predictors� before� generating� the� logistic� regression� model,� and�then�run�the�model�as�desired��You�can�then�interpret�the�logistic�regression�coeffi- cients�as�standardized�regression�coefficients�(if�necessary,�review�Chapter�18)�
We�can�also�form�a�confidence�interval�(CI)�around�the�logistic�regression�coefficient,�bk�� The�CI�formula�is�the�same�as�in�OLS�regression:�the�logistic�regression�coefficient�plus�or� minus�the�product�of�the�tabled�critical�value�and�the�standard�error:
CI b b t sk k n m b( ) ( / ) ( )= ± − −α 2 1
The�null�hypothesis�that�we�tested�was�H0:�βk�=�0��It�follows�that�if�our�CI�contains�0,�then� the�logistic�regression�coefficient�(bk)�is�not�statistically�significantly�different�from�0�at�the� specified�significance�level��We�can�interpret�this�to�say�that�βk�will�be�included�in�(1�−�α)%� of�the�sample�CIs�formed�from�multiple�samples�
19.5 Assumptions and Conditions
Compared�to�OLS�regression,�the�assumptions�of�logistic�regression�are�somewhat�relaxed;� however�four�primary�assumptions�must�still�be�considered:�(a)�noncollinearity,�(b)�linear- ity,�(c)�independence�of�errors,�and�(d)�values�of�X�are�fixed��In�this�section,�we�also�discuss�
722 An Introduction to Statistical Concepts
conditions� that� are� needed� in� logistic� regression� as� well� as� diagnostics� that� can� be� per- formed�to�more�closely�examine�the�data�
19.5.1 assumptions
19.5.1.1 Noncollinearity
Noncollinearity�is�applicable�to�logistic�regression�models�with�multiple�predictors�just� as� it� was� in� multiple� regression� (but� is� not� applicable� when� there� is� only� one� predic- tor� in� any� regression� model)�� This� assumption� has� already� been� explained� in� detail� in� Chapter�18�and�thus�will�not�be�reiterated�other�than�to�explain�tools�that�can�be�used� to�detect�multicollinearity��Although�SPSS�does�not�provide�an�option�to�easily�generate� collinearity� statistics� in� logistic� regression,� you� can� generate� an� OLS� regression� model� (i�e�,�a�traditional�multiple�linear�regression)�with�the�same�variables�used�in�the�logistic� regression� model� and� request� collinearity� statistics� there�� Because� it� is� only� the� collin- earity�statistics�that�are�of�interest,�do�not�be�concerned�in�generating�an�OLS�regression� model�that�violates�some�of�OLS�basic�assumptions�(e�g�,�normality)��We�have�previously� discussed�tolerance�and�the�variance�inflation�factor�(VIF)�as�two�collinearity�diagnos- tics�(where�tolerance�is�computed�as�1 2− Rk ,�where�Rk
2 �is�the�variance�in�each�independent�
variable,�X,�explained�by�the�other�independent�variables,�and�VIF�is� 1
1 2− Rk ��In�reviewing�
these�statistics,�tolerance�values�less�than��20�suggest�multicollinearity�exists,�and�values� less�than��10�suggest�serious�multicollinearity��VIF�values�greater�than�10�indicate�a�viola- tion�of�noncollinearity�
The� effects� of� a� violation� of� noncollinearity� in� logistic� regression� are� the� same� as� that� in�Chapter�18��First,�it�will�lead�to�instability�of�the�regression�coefficients�across�samples,� where�the�estimates�will�bounce�around�quite�a�bit�in�terms�of�magnitude,�and�even�occa- sionally�result�in�changes�in�sign�(perhaps�opposite�of�expectation)��This�occurs�because� the�standard�errors�of�the�regression�coefficients�become�larger,�thus�making�it�more�dif- ficult�to�achieve�statistical�significance��Another�result�that�may�occur�involves�an�overall� regression�that�is�significant,�but�none�of�the�individual�predictors�are�significant��Violation� will�also�restrict�the�utility�and�generalizability�of�the�estimated�regression�model�
19.5.1.2 Linearity
In�OLS�regression,�the�dependent�variable�is�assumed�to�have�a�linear�relationship�with� the� continuous� independent� variable(s),� but� this� does� not� hold� in� logistic� regression�� Because�the�outcome�in�logistic�regression�is�a�logit,�the�assumption�of�linearity�in�logis- tic� regression� refers� to� linearity� between� logit of the dependent variable� and� the� continu- ous� independent� variable(s)�� Hosmer� and� Lemeshow� (1989)� suggest� several� strategies� for�detecting�nonlinearity,�the�easiest�of�which�to�apply�is�likely�the�Box–Tidwell�trans- formation��This�strategy�is�also�valuable�as�it�is�not�overly�sensitive�to�minor�violations� of�linearity��This�involves�generating�a�logistic�regression�model�that�includes�all�inde- pendent� variables� of� interest� along� with� an� interaction� term� for� each—the� interaction� term�being�the�product�of�the�continuous�independent�variable�and�its�natural�log�[i�e�,� X*ln(X)]��Statistically�significant�interaction�terms�suggest�nonlinearity��It�is�important�to� note�that�the�assumption�of�linearity�is�applicable�only�for�continuous�predictors��A�viola- tion�of�linearity�can�result�in�biased�parameter�estimates,�as�well�as�the�expected�change� in�the�logit�of�Y�not�being�constant�across�the�values�of�X�
723Logistic Regression
19.5.1.3 Independence of Errors
Independence�of�errors�is�applicable�to�logistic�regression�models�just�as�it�was�with�OLS� regression,�and�a�violation�of�this�assumption�can�result�in�underestimated�standard�errors� (and� thus� overestimated� test� statistic� values� and� perhaps� finding� statistical� significance� more�often�than�is�really�viable,�as�well�as�affecting�CIs)��This�assumption�has�already�been� explained�in�detail�during�the�discussion�of�assumptions�in�Chapters�17�and�18,�and,�thus,� additional�information�will�not�be�provided�here�
19.5.1.4 Fixed X
The�last�assumption�is�that�the�values�of�Xk�are�fixed,�where�the�independent�variables�Xk� are�fixed�variables�rather�than�random�variables��Because�this�assumption�was�discussed� in�detail�in�Chapters�17�and�18,�we�only�summarize�the�main�points��When�X�is�fixed,�the� regression�model�is�only�valid�for�those�particular�values�of�Xk�that�were�actually�observed� and� used� in� the� analysis�� Thus,� the� same� values� of� Xk� would� be� used� in� replications� or� repeated�samples��As�discussed�in�the�previous�two�chapters,�generally�we�may�not�want� to� make� predictions� about� individuals� having� combinations� of� Xk� scores� outside� of� the� range� of� values� used� in� developing� the� prediction� model;� this� is� defined� as� extrapolating� beyond�the�sample�predictor�data��On�the�other�hand,�we�may�not�be�quite�as�concerned�in� making�predictions�about�individuals�having�combinations�of�Xk�scores�within�the�range� of�values�used�in�developing�the�prediction�model;�this�is�defined�as�interpolating�within� the�range�of�the�sample�predictor�data��Table�19�3�summarizes�the�assumptions�of�logistic� regression�and�the�impact�of�their�violation�
19.5.2 Conditions
Although� not� assumptions,� the� following� conditions� should� be� met� with� logistic� regres- sion:�nonzero�cell�counts,�nonseparation�of�data,�lack�of�influential�points,�and�sufficient� sample�size�
19.5.2.1 Nonzero Cell Counts
The� first� condition� is� related� to� nonzero� cell� counts� in� the� case� of� nominal� independent� variables��A�zero�cell�count�occurs�when�the�outcome�is�constant�for�one�or�more�categories�
Table 19.3
Assumptions�and�Violation�of�Assumptions:�Logistic�Regression�Analysis
Assumption Effect of Assumption Violation
Noncollinearity�of�Xs •��Regression�coefficients�can�be�quite�unstable�across�samples�(as�standard� errors�are�larger)
•�Restricted�generalizability�of�the�model
Linearity •�Bias�in�slopes�and�intercept •�Expected�change�in�logit�of�Y�is�not�a�constant�and�depends�on�value�of�X
Independence •�Influences�standard�errors�of�the�model�and�thus�hypothesis�tests�and�CIs Values�of�Xs�are�fixed •��Extrapolating�beyond�the�range�of�X�combinations:�prediction�errors�
larger,�may�also�bias�slopes�and�intercept •��Interpolating�within�the�range�of�X�combinations:�smaller�effects�than�
when�extrapolating;�if�other�assumptions�met,�negligible�effect
724 An Introduction to Statistical Concepts
of�a�nominal�variable�(e�g�,�all�females�pass�the�course)��This�results�in�high�standard�errors� because� entire� groups� of� individuals� have� odds� of� 0� or� 1�� Strategies� to� remove� zero� cell� counts� include� recoding� the� categories� (e�g�,� collapsing� categories)� or� adding� a� constant� to� each� cell� of� the� crosstab� table�� If� the� overall� model� fit� is� what� is� of� primary� interest,� then�you�may�choose�not�to�do�anything�about�zero�cell�counts��The�overall�relationship� between� the� set� of� predictors� and� the� dependent� variable� is� not� generally� impacted� by� zero�cell�counts��However,�if�zero�cell�counts�are�retained�and�the�results�of�the�individual� predictors�are�what�is�of�interest,�it�would�be�wise�to�provide�a�limitation�to�your�results� recognizing� higher� standard� errors� that� are� produced� due� to� zero� cell� counts� as� well� as� caution�that�the�values�of�the�individual�regression�coefficients�may�be�affected��Careful� review�of�the�data�prior�to�computing�the�logistic�regression�model�can�help�thwart�poten- tial�problems�with�zero�cell�counts�
19.5.2.2 Nonseparation of Data
Another�condition�that�should�be�examined�is�that�of�complete�or�quasi-complete�separa- tion��Complete�separation�arises�when�the�dependent�variable�is�perfectly�predicted�and� results�in�an�inability�to�estimate�the�model��Quasi-complete�separation�occurs�when�there� is�less�than�complete�separation�and�results�in�extremely�large�coefficients�and�standard� errors��These�conditions�may�occur�when�the�number�of�variables�equals�(or�nearly�equals)� the�number�of�cases�in�the�dataset,�such�that�large�coefficients�and�standard�errors�result�
19.5.2.3 Lack of Influential Points
Outliers�and�influential�cases�are�problematic�in�logistic�regression�analysis�just�as�with�OLS� regression��Severe�outliers�can�cause�the�maximum�likelihood�estimator�to�reduce�to 0�(Croux,� Flandre,�&�Haesbroeck,�2002)��Residual�analysis�and�other�diagnostic�tests�are�equally�ben- eficial�for�detecting�miscoded�data�and�unusual�(and�potentially�influential)�cases�in�logistic� regression�as�it�is�in�OLS�regression��SPSS�provides�the�option�for�saving�a�number�of�values� including�predicted�values,�residuals,�and�influence�statistics��Both�probabilities�and�group� membership�predicted�values�can�be�saved��Residuals�that�can�be�saved�include�(a)�unstan- dardized,� (b)� logit,� (c)� studentized,� (d)� standardized,� and� (e)� deviance�� The� three� types� of� influence�values�that�can�be�saved�include�Cook’s,�leverage�values,�and�DfBeta�
The�wide�variety�of�values�that�can�be�saved�suggests�that�there�are�many�types�of�diag- nostics�that�can�be�performed��Review�should�be�conducted�when�standardized�or�studen- tized�residuals�are�greater�than�an�absolute�value�of�3�0�and�DfBeta�values�are�greater�than�1�� Leverage�values�greater�than�(m +�1)/N�(where�m�equals�the�number�of�independent�vari- ables)�indicate�an�influential�case�(values�closer�to�1�suggest�problems,�while�those�closer�to� 0�suggest�little�influence)��If�outliers�or�influential�cases�are�found,�it�is�up�to�you�to�decide� if�removal�of�the�case�is�warranted��It�may�be�that�they,�while�uncommon,�are�completely� plausible�so�that�they�are�retained�in�the�model��If�they�are�removed�from�the�model,�it�is� important�to�report�the�number�of�cases�that�were�removed�prior�to�analysis�(and�evidence� to� suggest� what� caused� you� to� remove� them)�� A� review� of� Chapters� 17� and� 18� provides� further�details�on�diagnostic�analysis�of�outliers�and�influential�cases�
19.5.2.4 Sample Size
Simulation� research� suggests� that� logistic� regression� is� best� used� with� large� samples�� Samples� of� size� 100� or� greater� are� needed� to� accurately� conduct� tests� of� significance� for�
725Logistic Regression
logistic�regression�coefficients�(Long,�1997)��Note�that�for�illustrative�purposes,�the�exam- ple�in�this�chapter�uses�a�sample�size�of�20��We�recognize�this�is�insufficient�in�practice,�but� have�used�it�for�greater�ease�in�presenting�the�data�
19.6 Effect Size
We� have� already� talked� about� multiple� R2� pseudovariance� explained� values� which� can� be� used�not�only�to�gauge�model�fit�but�also�as�measures�of�effect�size��Another�important�statistic� in�logistic�regression�is�the�odds ratio�(OR),�also�an�effect�size�index�that�is�similar�to�R2��The� odds�ratio�is�computed�by�exponentiating�the�logistic�regression�coefficient�ebk��Conceptually� this�is�the�odds�for�one�category�(e�g�,�prepared�for�kindergarten)�divided�by�the�odds�for�the� other�category�(e�g�,�unprepared�for�kindergarten)��The�null�hypothesis�to�be�tested�is�that�OR�=�1,� which�indicates�that�there�is�no�relationship�between�a�predictor�variable�and�the�dependent� variable��Thus,�we�want�to�find�OR�to�be�significantly�different�from�1�
When�the�independent�variable�is�continuous,�the�odds�ratio�represents�the�amount�by� which�the�odds�change�for�a�one-unit�increase�in�the�independent�variable��When�the�odds� ratio�is�greater�than�1,�the�independent�variable�increases�the�odds�of�occurrence��When� the�odds�ratio�is�less�than�1,�the�independent�variable�decreases�the�odds�of�occurrence�� The� odds� ratio� is� provided� in� SPSS� output� as� “Exp(B)”� in� the� table� labeled� “Variables� in� the�Equation�”�In�predicting�kindergarten�readiness,�social�development�is�a�continuous� covariate�with�a�resulting�odds�ratio�of�2�631��We�can�interpret�this�odds�ratio�to�be�that�for� every�one-unit�increase�in�social�development,�the�odds�of�being�ready�for�kindergarten� (i�e�,�prepared)�increase�by�263%,�controlling�for�the�other�variables�in�the�model�
In�the�case�of�categorical�variables,�including�dichotomous,�multinomial,�and�ordinal�vari- ables,�odds�ratios�are�often�interpreted�in�terms�of�their�relative�size�or�the�change�in�odds� ratios�in�comparing�models��Consider�first�the�case�of�a�dichotomous�variable��In�the�model� predicting�kindergarten�readiness,�type�of�household�is�one�independent�variable�included� in�the�model�where�a�two-parent�home�is�coded�as�“1”�and�a�single-parent�home�as�“0�”�An� odds�ratio�of��002�indicates�that�the�odds�of�being�prepared�for�kindergarten�(compared�to� unprepared�for�kindergarten)�are�decreased�by�a�factor�of��002�by�being�in�a�single-parent� home�(as�opposed�to�living�in�a�two-family�home)��We�could�also�state�that�the�odds�that�a� child�from�a�single-parent�home�will�be�prepared�for�kindergarten�are��998�(i�e�,�1�−��002)�
In� the� case� of� a� categorical� variable� with� more� than� two� categories,� the� odds� ratio� is� interpreted�relative�to�the�reference�(or�left�out)�category��For�example,�say�we�have�a�pre- dictor� in� our� model� that� is� mother’s� education� level� with� categories� that� include� (1)� less� than�high�school�diploma,�(2)�high�school�diploma�or�GED,�and�(3)�at�least�some�college�� Say� we� set� the� last� category� (“at� least� some� college”)� as� the� reference� category�� An� odds� ratio�of��86�for�the�category�of�“high�school�diploma�or�GED”�for�mother’s�education�level� suggests�that�the�odds�of�being�prepared�for�kindergarten�(as�compared�to�unprepared)� decrease� by� a� factor� of� �86� when� the� child’s� mother� has� a� high� school� diploma� or� GED,� relative�to�when�the�child’s�mother�has�at�least�some�college,�when�the�other�variables�in� the�model�are�controlled�
Odds�ratio�values�can�also�be�converted�to�Cohen’s�d�using�the�following�equation:
d OR
= ln( )
.1 81
726 An Introduction to Statistical Concepts
19.7 Methods of Predictor Entry
The� three� categories� of� model� building� that� will� be� discussed� include� (a)� simultaneous� logistic�regression,�(b)�stepwise�logistic�regression,�and�(c)�hierarchical�regression�
19.7.1 Simultaneous logistic Regression
With� simultaneous� logistic� regression,� all� the� independent� variables� of� interest� are� included� in� the� model� in� one� set�� This� method� of� model� building� is� usually� used� when� the�researcher�does�not�hypothesize�that�some�predictors�are�more�important�than�others�� This�method�of�entry�allows�you�to�evaluate�the�contribution�of�an�independent�variable� over�and�above�that�of�all�other�predictors�in�the�model�(i�e�,�each�independent�variable�is� evaluated�as�if�it�was�the�last�one�to�enter�the�equation)��One�problem�that�may�be�encoun- tered� with� this� method� of� entry� is� related� to� strong� correlations� between� the� predictor� and�the�outcome��An�independent�variable�that�has�a�strong�bivariate�correlation�with�the� dependent� variable� may� indicate� a� weak� correlation� when� entered� simultaneously� with� other�predictors��In�SPSS,�this�method�of�entry�is�referred�to�as�“Enter�”
19.7.2 Stepwise logistic Regression
Stepwise�logistic�regression�is�a�data-driven�model�building�technique�where�the�computer� algorithms� drive� variable� entry� rather� than� theory�� Issues� with� this� type� of� technique� have� previously� been� outlined� in� the� discussion� associated� with� this� method� in� multiple� regres- sion� and� thus� are� not� rehashed� here�� If� stepwise� logistic� regression� is� determined� to� be� the� most� appropriate� strategy� to� build� your� model,� Hosmer� and� Lemeshow� (2000)� suggest� set- ting�a�more�liberal�criterion�for�variable�inclusion�(e�g�,�α�=��15�to��20)��They�also�provide�spe- cific�recommendations�on�dealing�with�interaction�terms�and�scales�of�variables��Because�it�is� only�in�unusual�instances�that�this�method�of�model�building�is�appropriate�(e�g�,�exploratory� research),�additional�coverage�of�the�suggestions�by�Hosmer�and�Lemeshow�is�not�presented�
SPSS� offers� forward� and� backward� stepwise� methods�� For� both� forward� and� backward� methods,�options�include�conditional,�LR,�and�Wald��The�differences�between�these�options� are�mathematically�driven��The�LR�method�of�entry�uses�the�−2LL�for�estimating�entry�of� independent�variables��The�conditional�method�also�uses�the�likelihood�ratio�test,�but�one� that�is�considered�to�be�computationally�quicker��The�Wald�method�applies�the�Wald�test�to� determining�entry�of�the�independent�variables��With�forward�stepwise�methods,�the�model� begins�with�a�constant�only,�and�based�on�some�criterion,�independent�variables�are�added� one�at�a�time�until�a�specified�cutoff�is�achieved�(e�g�,�all�independent�variables�included�in� the�model�are�statistically�significant,�and�any�additional�variables�not�included�in�the�model� are� not� statistically� significant)�� Backward� stepwise� methods� work� in� the� reverse� fashion� where�initially�all�independent�variables�(and�the�constant)�are�included��Independent�vari- ables�are�then�removed�until�only�those�that�are�statistically�significant�remain�in�the�model,� and�including�an�omitted�independent�variable�would�not�improve�the�model�
19.7.3 hierarchical Regression
In� hierarchical� regression,� the� researcher� specifies� a� priori� a� sequence� for� the� individ- ual� predictor� variables� (not� to� be� confused� with� hierarchical� linear� models,� which� is� a� regression�approach�for�analyzing�nested�data�collected�at�multiple�levels,�such�as�child,�
727Logistic Regression
classroom,�and�school)��The�analysis�proceeds�in�a�forward�selection,�backward�elimina- tion,� or� stepwise� selection� mode� according� to� a� researcher-specified,� theoretically� based� sequence,� rather� than� an� unspecified,� statistically� based� sequence�� In� SPSS,� this� is� con- ducted� by� entering� predictors� in� blocks� and� selecting� their� desired� method� of� entering� variables� in� each� block� (e�g�,� simultaneously,� forward,� backward,� stepwise)�� Because� this� method� was� explained� in� detail� in� Chapter� 18� and� operation� of� this� method� of� variable� selection�is�the�same�in�logistic�regression,�additional�information�will�not�be�presented�
19.8 SPSS
Next�we�consider�SPSS�for�the�logistic�regression�model��Before�we�conduct�the�analysis,� let�us�review�the�data�(note�that�we�recognize�the�sample�size�of�20�does�not�meet�mini- mum�sample�size�criteria�previously�specified;�however�for�illustrative�purposes,�we�felt� it� important� that� we� be� able� to� show� the� entire� dataset,� and� this� would� have� been� more� difficult�with�the�recommended�sample�size�for�logistic�regression)��With�one�dependent� variable�and�two�independent�variables,�the�dataset�must�consist�of�three�variables�or�col- umns,� one� for� each� independent� variable� and� one� for� the� dependent� variable�� Each� row� still�represents�one�individual��As�seen�in�the�following�screenshot,�the�SPSS�data�are�in� the� form� of� three� columns� that� represent� the� two� independent� variables� (a� continuous� teacher-administered�social�development�scale�and�household—a�dichotomous�variable,� single-�vs��two-adult�household)�and�one�binary�dependent�variable�(kindergarten�readi- ness� screening� test—prepared� vs�� not� prepared)�� As� our� dependent� variable� is� dichoto- mous,�we�will�conduct�binary�logistic�regression��When�the�dependent�variable� consists� of�more�than�two�categories,�multinomial�logistic�regression�is�appropriate�(although�not� illustrated�here)�
�e independent variables are labeled “Social” and “Household” where each value represents the child’s
score on the teacher reported social development scale (interval
measurement) and whether the child lives with one or two parents (nominal measurement). A “1” for household
indicates two-parents and “0” represents a single-parent family.
�e dependent variable is “Readiness” and represents whether or not the child is prepared for kindergarten.
�is is a binary variable where “1” represents “prepared” and “0”
represents “unprepared.”
728 An Introduction to Statistical Concepts
Step 1:�To�conduct�a�binary�logistic�regression,�go�to�“Analyze”�in�the�top�pulldown� menu,�then�select�“Regression,”�and�then�select�“Binary Logistic.”�Following�the� screenshot�(step�1)�that�follows�produces�the�“Logistic Regression”�dialog�box�
Logistic regression: Step 1
A
B
C
Step 2:�Click�the�dependent�variable�(e�g�,�“Readiness”)�and�move�it�into�the�“Dependent” box�by�clicking�the�arrow�button��Click�the�independent�variables�and�move�them�into�the� “Covariate(s)”�box�by�clicking�the�arrow�button�(see�screenshot�step�2)�
Logistic regression: Step 2
Clicking on “Categorical”
will allow you to specify variables
that are categorical.
Clicking on “Save” will allow you to
save various predicted values,
residuals, and other statistics
useful for diagnostics.
Clicking on “Options” will
allow you to select various statistics
and plots.
Clicking on “Enter” will allow you to select different
types of methods of entering the
variables (e.g., forward,
backward). “Enter” is the default and all predictors are
entered as one set.
Had we been entering our variables hierarchically, we would have used the “Next” button to enter each set of variables in the order of progression.
Select the independent
variables from the list on the left and use the arrow to
move them to the “Covariates” box
on the right.
Select the dependent variable from the list on the
left and use the arrow to move it to
the “Dependent” box on the right.
Social development...
729Logistic Regression
Step 3:�From�the�“Logistics Regression”�dialog�box�(see�screenshot�step�2),�click- ing� on� “Categorical”� will� provide� the� option� to� define� as� categorical� those� variables� that� are� nominal� or� ordinal� in� scale� as� well� as� to� select� which� category� of� the� variable� is� the� reference� category� through� the� “Define Categorical Variables”� dialog� box� (see screenshot� step� 3a)�� From� the� list� of� covariates� on� the� left,� click� the� categorical� covariate(s)� (e�g�, “Household”)� and� move� it� into� the�“Categorical Covariates”� box� by�clicking�the�arrow�button��By�default,�“(Indicator)”�will�appear�next�to�the�variable� name��Indicator�refers�to�traditional�dummy�coding,�and�you�have�the�option�of�select- ing�which�value�is�the�reference�category��For�binary�variables�(only�two�categories),�using� the�“Last”�value�as�the�reference�category�means�that�the�category�coded�with�the�larg- est�value�will�be�the�category�“left�out”�of�the�model�(or�referent),�and�using�the�“First” value�as�the�reference�category�means�that�the�category�coded�with�the�smallest�value�will� be�the�category�“left�out”�of�the�model��Here�two-parent�households�were�coded�as�1�and� single-parent�households�as�0��We�use�single-parent�households�(coded�as�0)�as�the�refer- ence� category�� Thus,� we� select� the� radio� button� for� “First”� (see� screenshot� step� 3a)� to� define�single-parent�households�as�the�reference�category�
Logistic regression: Step 3a
Selecting “First” means that the category coded
with the smallest value is the reference category.
Selecting “Last” means that the category coded with the largest value is the reference category.
Next,�we�need�to�click�the�button�labeled�“Change”�(see�screenshot�step�3b)�to�define�the�first� value�(i�e�,�0�or�single-parent�household)�as�the�reference�(or�“left�out”)�category��By�doing� that,�the�name�of�our�categorical�covariate�will�now�read�Household(Indicator(first))�� Had�we�had�a�categorical�variable�with�more�than�two�categories,�we�could�just�define�the� variable�as�categorical�within�logistic�regression�and�select�either�the�first�or�last�value�as� the� reference� category�� If� neither� the� first� or� last� were� what� you� wanted� as� the� reference� category,�then�some�recoding�of�the�data�is�necessary�
730 An Introduction to Statistical Concepts
Logistic regression: Step 3b
Clicking “change” will define the smallest value
(0 in this illustration) as the reference
category that is “left out” of the model.
Before� we� move� on,� notice� that� the� button� for� “Contrast”� is� a� toggle� menu� with� “Indicator”�as�the�default�option��Selecting�the�toggle�menu�allows�you�to�select�other� types� of� contrasts� often� discussed� in� relation� to� analysis� of� variance� (ANOVA)� contrasts� (e�g�,�Simple,�Difference,�Helmert)��These�will�not�be�reviewed�here��Click�on�“Continue”� to�return�to�the�“Logistic Regression”�dialog�box�
Should a more complex contrast be desired,
additional options are available in SPSS.
Step 4:� From� the� “Logistic Regression” dialog� box� (see� screenshot� step� 2),� clicking�on�“Save”�will�provide�the�option�to�save�various�predicted�values,�residuals,� and� statistics� that� can� be� used� for� diagnostic� examination�� From� the�“Save”� dialog� box�under�the�heading�of�Predicted Values,�place�a�checkmark�in�the�box�next�to� the� following:� (1)�probabilities� and� (2)�group membership�� Under� the� heading� of�Residuals,� place� a� checkmark� in� the� box� next� to� the� following:�standardized�� Under� the� heading� of�Influences,� place� a� checkmark� in� the� box� next� to� the� follow- ing:�(1)�Cook’s,�(2)�Leverage values,�and�(3)�DfBeta(s).�Click�on “Continue”�to� return�to�the�original�dialog�box�
731Logistic Regression
Logistic regression: Step 4
Step 5:�From�the�“Logistic Regression”�dialog�box�(see�screenshot�step�2),�clicking� on�“Options”�will�allow�you�to�generate�various�statistics�and�plots��From�the�“Options” dialog�box�under�the�heading�of�Statistics and Plots,�place�a�checkmark�in�the�box� next� to� the� following:� (1)�Classification plots,� (2)�Hosmer–Lemeshow goodness- of-fit,�(3)�casewise listing of residuals,�(4)�outliers outside, and�(5)�CI for exp(B). For�Outliers outside, you�must�specify�a�numeric�value�of�standard� deviations� to� define� what� you� consider� to� be� an� outlier�� Common� values� may� be� 2� (in� a� normal�distribution,�95%� of�cases�will�be�within�±2�standard�deviations),�3�(in�a�normal� distribution,� about� 99%� of� cases� will� be� within� ±3� standard� deviations),� or� 3�29� (in� a� normal� distribution,� about� 99�9%� of� cases� will� be� within� ±3�29� standard� deviations)�� For� this�illustration,�we�will�use�a�value�of�2��For�CI for exp(B), you�must�specify�a�CI��This� should�be�the�complement�of�the�alpha�being�tested��If�you�are�using�an�alpha�of��05,�then�the� CI�will�be�1�−��05,�or��95��All�the�remaining�options�in�the�“Options”�dialog�box�will�be�left� as�the�default�settings��Click�on�“Continue”�to�return�to�the�original�dialog�box��From�the� “Logistic Regression”�dialog�box,�click�on “OK”�to�generate�the�output�
Logistic regression: Step 5
Interpreting the output:�Annotated�results�are�presented�in�Table�19�4�
732 An Introduction to Statistical Concepts
Table 19.4
SPSS�Results�for�the�Binary�Logistic�Regression�Kindergarten�Readiness�Example
Case Processing Summary
Unweighted Casesa N Percent
Included in analysis
Missing cases
Selected cases
Total Unselected cases Total
20
0
20 0
20
100.0
.0
100.0 .0
100.0 a If weight is in effect, see classification table for the total number of cases.
Dependent Variables Encodings
Original Value Internal Value Unprepared 0 Prepared 1
Categorical Variables Codings
Parameter Coding
Frequency (1)
Single parent household 10 .000Type of household Two-parent household 10 1.000
is table provides information on sample size
and missing data. e sample size is 20 and we have no missing data.
Information on how the values of the dependent
variable are coded is provided under “Internal Value.”
“Unprepared” is coded as 0 and “Prepared” is coded as 1.
Information on how the values of the categorical variable(s) are coded is
provided as “Parameter Coding.” “Single Parent Household” is coded as 0 and “Two-Parent Household”
is coded as 1. e sample size per group is presented in the “Frequency” column.
Block 0: Beginning Block
Classification Tablea,b
Predicted Kindergarten Readiness
Observed Unprepared Prepared Percentage
Correct
Unprepared .0Kindergarten readiness
Prepared
0
0
8
12 100.0
Step 0
Overall percentage 60.0 a Constant is included in the model. b �e cut value is .500.
Block 0 is a summary of the model with the constant only (i.e., none of the predictors are included). �e classification table
provides the percentage of cases correctly predicted given the constant only. Without including covariates, we can correctly
predict children who are prepared for kindergarten 100% of the time but fail to predict any children (0%) who are unprepared.
Here all children are predicted to be prepared.
Variables in the Equation
B SE Wald df Sig. Exp(B)
Step 0 Constant .405 .456 .789 1 .374 1.500
733Logistic Regression
Table 19.4 (continued)
SPSS�Results�for�the�Binary�Logistic�Regression�Kindergarten�Readiness�Example
Variables Not in the Equation
Score df Sig.
Social development .003Variables
Household(1) .068
Step 0
Overall statistics
8.860
3.333
11.168
1
1
2 .004
Variables not in the equation provides an indication of whether each covariate will statistically significantly contribute to predicting the outcome. Only social development ( p = .003) is of value in the logistic
model. �e value of 11.168 for overall statistics is a residual chi- square statistic. Since the p value for it indicates statistical significance (p = .004), this indicates that including the two covariates improves the
model as compared to the constant only model.
Block 1: Method = Enter
Omnibus Tests of Model Coefficients Chi-Square df Sig.
Step 2 Block 2
Step 1
Model
15.793 15.793 15.793 2
.000
.000
.000
Model Summary
Step –2 Log Likelihood Cox and Snell R
Square Nagelkerke
R Square
1 11.128a .546 .738
a Estimation terminated at iteration number 7 because parameter estimates changed by less than .001.
Method = Enter indicates that the method of entering the predictors was simultaneous
entry (recall this is the default method in SPSS and is called “Enter”).
Model summary statistics provide overall model fit. For good model fit, the value of
–2LL for the full model (11.128) should be less than –2LL for the constant only model
(26.921). �is is a chi-square value with degrees of freedom equal to the number of
parameters in the full model (i.e., two predictors plus one constant) minus the number of parameters in the baseline model (i.e., 1).
�us there are two df using the chi-square table, with an alpha of .05 and two df , the critical value is 5.99. Since 11.128 is larger than the
critical value, we reject the null hypothesis that the best prediction model is the constant only
model. In other words, the full model (with predictors) is better at predicting kindergarten
readiness than the constant only model.
�e –2LL for the constant only model is computed as the sum of chi-square for the
constant only model and –2LL for the full model:
�e two R2 values are pseudo R2 and are interpreted similarly to multiple R2. �ese can be used as effect
size indices for logistic regression and Cohen’s interpretations for correlation can be used to interpret.
Both values indicate a large effect.
2 Model + –2LL = 15.793 + 11.128 = 26.921
(continued)
734 An Introduction to Statistical Concepts
Table 19.4 (continued)
SPSS�Results�for�the�Binary�Logistic�Regression�Kindergarten�Readiness�Example
Contingency Table for Hosmer and Lemeshow Test
Kindergarten Readiness = Unprepared Kindergarten Readiness = Prepared
Observed Expected Observed Expected Total
1
2
3
4
5
6
7
8
Step 1
9
2
2
1
2
0
1
0
0
0
1.988
1.922
1.651
1.292
.607
.404
.100
.030
.005
0
0
1
0
2
2
2
2
3
.012
.078
.349
.708
1.393
2.596
1.900
1.970
2.995
2
2
2
2
2
3
2
2
3
�e classification table provides information on how well group membership was predicted. Cells on the diagonal indicate correct classification. For example, children who were prepared for kindergarten
were accurately classified 91.7% of the time as compared to unprepared children (87.5%). Overall, 90% of children were correctly classified. �is is computed as the number of correctly classified cases
divided by total sample size:
Using Press’s Q and given the chi-square critical value of 3.841 (df = 1), we find:
We reject the null hypothesis. �ere is evidence to suggest that the predictions are statistically significantly better than chance.
Q = = =12.8 [N –(nK)]2
N (K–1) 20 (2–1)
7+11 20
.90
[20 –(18)(2)]2
Classification Tablea
Predicted
Kindergarten Readiness Observed Unprepared Prepared
Percentage Correct
Unprepared 7 1 87.5Kindergarten readiness
Prepared 1 11 91.7
Step 1
Overall percentage 90.0 a �e cut value is .500.
Hosmer and Lemeshow Test
Step Chi-Square df Sig.
1 4.691 7 .698
As a measure of classification accuracy, non- statistical significance (p= .698) indicates good
model fit for the Hosmer and Lemeshow test. �is test is affected by small sample size, however;
caution should be used when interpreting the results of this test when sample size is less than 50.
735Logistic Regression
Table 19.4 (continued)
SPSS�Results�for�the�Binary�Logistic�Regression�Kindergarten�Readiness�Example
Variables in the Equation
95% CI for Exp(B) B SE Wald df Sig. Exp(B) Lower Upper
Social development Household(1)
1.097
.000
6.313
1.693
Step 1a
Constant
.967
–6.216 –15.404
.446
3.440 7.195
4.696
3.265 4.584
1
1 1
.030
.071
.032
2.631
.002
.000 a Variable(s) entered on step 1: Social development, household.
Since the odds of 1.00 (which indicates similar odds for falling into either category of the outcome) are not
contained within the interval for social development, this suggests the odds ratio is statistically significantly different from zero. Note that the odds ratio is only computed for the predictors and not for the intercept (i.e., constant).
�e p value for “Social” (p= .030) indicates that the
slope is statistically significantly different from zero. �is tells us that the independent variable is
contributing to predicting kindergarten preparedness. �e intercept (p = .032) is
also statistically significantly different from zero.
Exp(B) values are the odds ratios. �e odds ratio of 2.631 for social indicates that the odds for being
prepared for kindergarten are over 2–1/2 times greater (or 263%) for every one point increase in social
development. �e odds for household are nearly zero. �is indicates that the odds for being
prepared for kindergarten are about the same regardless of the child’s
household structure (single- versus two-parent home).
�e Wald statistic is used
to test the statistical
significance of each covariate.
�e B coefficient is interpreted as the change in the logit of the
dependent variable given a one- unit change in the independent
variable. Recall that the logit is the natural log of the dependent
variable occurring. With B equal to .967, this tells us that a one-unit
change in social development will result in nearly a one-unit change
in the logit of kindergarten preparedness. �e constant is the
expected value of the logit of kindergarten readiness for children
of single parents (recall this was coded as 0) and when social
development is zero. A positive B indicates that an increase in
value of that independent variable will result in an increase in the predicted
probability of the dependent variable.
A negative B indicates that an increase in value of that independent variable will result in an decrease in the predicted probability of the dependent variable.
NOTE! Interpretations of B coefficients are usually done via
odds ratios.
(continued)
736 An Introduction to Statistical Concepts
Table 19.4 (continued)
SPSS�Results�for�the�Binary�Logistic�Regression�Kindergarten�Readiness�Example
Casewise Lista
Observed Temporary Variable
Case Selected Statusb
Kindergarten Readiness Predicted Predicted Group Resid ZResid
8 S P 15 S
U** P**
.832
.214 U –.832
.786 –2.226
1.918
b S = Selected, U = Unselected cases, and ** = Misclassified cases.
a Cases with studentized residuals greater than 2.000 are listed.
Recall we told SPSS to identify residuals that were outside two standard deviations. Based on that decision,
cases 8 and 15 were identified as potential outliers. We review this output in the discussion on outliers.
“P” indicates “Prepared for Kindergarten” and “U”
indicates “Unprepared for Kindergarten.” P’s to the left of .50 indicate misclassified cases. U’s to the right of .50 indicate misclassified cases.
Although there are 4 P’s, this represents a frequency of one.
737Logistic Regression
Examining Data for Assumptions for Logistic Regression
Previously� we� described� a� number� of� assumptions� used� in� logistic� regression�� These� included� (a)� noncollinearity,� (b)� linearity� between� the� predictors� and� logit� of� the� depen- dent�variable,�and�(c)�independence�of�errors��We�also�review�the�data�to�ensure�there�are� no�outliers�
Before�we�begin�to�examine�assumptions,�let�us�review�the�values�that�we�requested�to� be�saved�to�our�data�file�(see�dataset�screenshot�that�follows):
� 1��PRE _ 1�represents�the�predicted�probabilities� � 2. PGR _ 1� is� the� predicted� group� membership� (here� group� membership� is� either�
prepared�or�unprepared�for�kindergarten)� � 3��COO _ 1�represents�Cook’s�influence�statistics��As�a�rule�of�thumb,�Cook’s�values�
greater�than�1�suggest�that�case�is�potentially�problematic� � 4��LEV _ 1�represents�leverage�values��As�a�general�guide,�leverage�values�less�than�
�20� suggest� there� are� no� problems� with� cases� exerting� undue� influence�� Values� greater�than��5�indicate�problems�
� 5��ZRE _ 1�pertains�to�standardized�residuals�computed�as�the�residual�divided�by� an�estimate�of�the�standard�deviation�of�the�residual��Standardized�residuals�have� a�mean�of�0�and�standard�deviation�of�1�
� 6��DFB0 _ 1, DFB1 _ 1, and DFB2 _ 1�are�DfBeta�values�and�indicate�the�differ- ence�in�a�beta�coefficient�if�that�particular�case�were�excluded�from�the�model�
1
As we look at the raw data, we see eight new variables have been
added to our dataset. ese are predicted values, residuals, and
other diagnostic statistics.
2 3 4 5 6 6 6
Noncollinearity
It� is� not� possible� to� request� multicollinearity� statistics,� such� as� tolerance� and� VIF,� using� logistic� regression� in� SPSS�� We� can,� however,� estimate� those� values� by� running� the� same� variables� in� a� multiple� regression� model� (see� Chapter� 18)� and� requesting� only� the� collin- earity� statistics�� We� are� not� interested� in� the� parameter� estimates� of� the� model—only� the� collinearity�statistics��Tolerance�values�less�than��10�and�VIF�values�greater�than�10�indicate� multicollinearity�(Menard,�1995)��Because�the�steps�for�generating�multiple�regression�were�
738 An Introduction to Statistical Concepts
presented�in�Chapter�18,�we�will�not�reiterate�them�here��Rather,�we�will�merely�present�the� applicable�portion�of�the�output�of�this�model��From�the�output�that�follows�with�a�tolerance� of��248�and�VIF�of�4�037,�we�have�evidence�that�we�do�not�have�multicollinearity��In�examin- ing�collinearity�diagnostics,�condition�index�values�that�are�substantially�larger�than�others� listed�indicate�potential�problems�with�multicollinearity�(although�“substantially�larger”�is� a�subjective�measure)��Here�the�condition�index�of�dimension�3(14�259)�is�about�five�times� larger�than�the�next�largest�condition�index��The�last�three�columns�refer�to�variance�propor- tions��Multiplying�these�values�by�100�provides�a�percentage�of�the�variance�of�the�regres- sion�coefficient�that�is�related�to�a�particular�eigenvalue��Multicollinearity�is�suggested�when� covariates� have� high� percentages� associated� with� a� small� eigenvalue�� Thus,� for� purposes� of� reviewing� for� multicollinearity,� concentrate� only� on� the� rows� with� small� eigenvalues�� In� this� example,� 100%� of� the� variance� of� the� regression� coefficient� for� social� development� and�73%�for�type�of�household�are�related�to�eigenvalue�3�(the�dimension�with�the�smallest� eigenvalue)��This�suggests�there�may�be�some�multicollinearity��In�summary,�we�have�met� the�assumption�of�noncollinearity�with�the�tolerance�and�VIF�values,�but�there�is�some�sug- gestion�of�multicollinearity�with�the�condition�index�and�variance�proportion�values�
Coefficientsa
Collinearity Diagnosticsa
Variance Proportions
Model
Model Dimension
1 2 3
2.683 .303 .013
1.000 2.974
14.259
.00
.05
.95
.00
.00 1.00
.01
.25
.73
1
Eigenvalue Condition
Index (Constant) Social
Development Type of
Household
Social development .248
Collinearity Statistics Tolerance VIF
4.037 .248 4.037
1 Type of household
a Dependent Variable: Kindergarten readiness.
a Dependent Variable: Kindergarten readiness.
Linearity
Recall�that�the�linearity�assumption�is�applicable�only�to�continuous�variables��Thus,�we� will�test�this�assumption�only�for�social�development��The�Box-Tidwell�transformation�test� can�be�used�to�test�that�the�assumption�of�linearity�has�been�met��To�generate�this�test,�for� each�continuous�independent�variable,�we�must�first�create�an�interaction�term�that�is�the� product�of�the�independent�variable�and�its�natural�log�(ln)��Here�we�have�only�one�con- tinuous�independent�variable—social�development��Thus,�only�one�interaction�term�will� be�created�
Step 1:�To�create�an�interaction�term�of�our�continuous�variable�and�the�natural�log�of�this� variable,�go�to�“Transform”�in�the�top�pulldown�menu,�then�select�“Compute Variable.”� Following�the�screenshot�(step�1)�that�follows�produces�the�“Compute Variable”�dialog�box�
739Logistic Regression
Creating an
interaction term:
Step 1
A
B
Step 2:�In�the�“Target Variable”�box�in�the�upper�left�corner,�enter�the�variable�name� that�you�want�to�appear�as�the�column�header��Since�this�is�the�column�header�name,�this� name� cannot� begin� with� special� characters� or� numbers� and� cannot� have� any� spaces�� If� you�wish�to�define�the�label�for�this�variable�(i�e�,�what�will�appear�on�the�output;�this�can� include�special�characters,�spaces,�and�numbers),�then�click�on�the�“Type & Label”�box� directly� underneath�“Target Variable”� where� additional� text� to� define� the� name� of� the�variable�can�be�included��Next,�click�on�the�continuous�covariate�(i�e�,�social�develop- ment)�and�move�it�into�the�“Numeric Expression”�box�by�clicking�on�the�arrow�in�the� middle�of�the�screen��Using�either�the�keyboard�on�screen�or�your�keyboard,�click�on�the� asterisks�key�(i�e�,�*)��This�will�be�used�as�the�multiplication�sign��Next,�under�“Function group,”�click�on�arithmetic�to�display�all�of�the�basic�mathematical�functions��From�this� alphabetized� list,� click� on�“Ln”� (natural� log)�� To� move� this� function� into� the�“Numeric Expression”�box,�click�on�the�arrow�key�in�the�right�central�part�of�the�dialog�box�
Select the continuous
covariate from the list on the left and use the arrow to
move it to the “Numeric
Expression” box on the right. �en use the
keyboard to insert an * directly after
our covariate.
Use the arrow key to move the “Ln” function into the
“Numeric Expression”
box.
Select “Arithmetic” to display basic mathematical
functions in the bottom right list.
Creating an interaction term:
Step 2
From the list of arithmetic
functions, select “Ln” (the natural log).
Use the arrow key to move it into the
“Numeric Expression”
box.
740 An Introduction to Statistical Concepts
Step 3: Once�the�natural�log�function�is�displayed�in�the�“Numeric Expression” box,� a�question�mark�enclosed�inside�parentheses�will�appear�(see�screenshot�step�3a)��This�is� SPSS’s�way�of�asking�which�variable�you�want�the�natural�log�computed�for��Here�it�is�the� continuous�covariate,�social�development�
Delete the question mark and replace it with the variable for which the natural
log should be computed.
Creating an interaction term:
Step 3a
Here�we�want�to�compute�the�natural�log�for�the�continuous�covariate,�social�develop- ment��To�move�this�variable�into�the�parentheses,�use�the�backspace�or�delete�key�to�remove� the�question�mark��Then,�click�on�the�continuous�covariate,�social�development,�and�move� it�into�the�parentheses�next�to�LN�in�the�“Numeric Expression”�box�by�clicking�on�the� arrow�in�the�middle�of�the�screen�(see�screenshot�step�3b)��The�numeric�expression�should� then�read�“Social*LN(Social).”�Click “OK”�to�compute�and�create�the�new�variable�in� the�dataset�
Creating an interaction term:
Step 3b
Step 4:�The�next�step�is�to�include�the�newly�created�variable�(i�e�,�the�interaction�of�the� continuous�variable�with�its�natural�log)�into�the�logistic�regression�model,�along�with�the� other�predictors��As�those�steps�have�been�presented�previously,�they�will�not�be�reiterated� here��The�output�indicates�that�the�interaction�term�is�not�statistically�significant�(p�=��300),� which�suggests�we�have�met�the�assumption�of�linearity�
741Logistic Regression
Variables in the Equation
Step 1a Social
Household(1)
Social ... Insocial
Constant
a Variable(s) entered on step 1: Social, household, social ... Insocial.
B SE Wald df Sig. Exp(B) Lower Upper
95% Cl for Exp(B)
12.953
–8.208
–2.948
–76.228
11.897
5.264
2.845
64.345
1.185
2.432
1.074
1.403
1
1
1
1
.276
.119
.300
.236
421981.259
.000
.052
.000
.000
.000
.000
5.647E15
8.236
13.845
Independence
We� plot� the� standardized� residuals� (which� were� requested� and� created� through� the� “Save” option)�against�the�values�of�X�to�examine�the�extent�to�which�independence�was� met��The�general�steps�for�generating�a�simple�scatterplot�through�“Scatter/dot”�have� been�presented�in�a�previous�chapter�(e�g�,�Chapter�10),�and�they�will�not�be�repeated�here�� From�the�“Simple Scatterplot”�dialog�screen,�click�the�standardized�residual�(called� “normalized�residual”�in�SPSS)�variable�and�move�it�into�the�“Y Axis”�box�by�clicking�on� the�arrow��Click�the�independent�variable�X�and�move�it�into�the�“X Axis”�box�by�clicking� on�the�arrow��Then�click�“OK.”
742 An Introduction to Statistical Concepts
Interpreting independence evidence:�If�the�assumption�of�independence�is�met,� the�points�should�fall�randomly�within�a�band�of�−2�0�to�+2�0��Here�we�have�pretty�good� evidence�of�independence,�especially�given�the�small�sample�size�relative�to�logistic�regres- sion,�as�all�but�one�point�(case�19)�are�within�an�absolute�value�of�2�0�
Social development
N or
m al
iz ed
re si
du al
19
10.00 15.00 20.00 25.00 30.00
–3.00000
–2.00000
–1.00000
.00000
1.00000
2.00000
Type of household
N or
m al
iz ed
re si
du al
.20 .40 .60 .80 1.00
19
.00
–3.00000
–2.00000
–1.00000
.00000
1.00000
2.00000
Absence of Outliers
Just�as�we�saw�in�multiple�regression,�there�are�a�number�of�diagnostics�that�can�be�used� to�examine�the�data�for�outliers�
Cook’s distance:� Cook’s� distance� provides� an� overall� measure� for� the� influence� of� individual�cases��Values�greater�than�one�suggest�that�a�case�may�be�problematic�in�terms� of�undue�influence�on�the�model��Examining�the�residual�statistics�provided�in�the�binary�
743Logistic Regression
logistic�regression�output�(see�following�table),�we�see�that�the�maximum�value�for�Cook’s� distance�is�1�58,�which�indicates�at�least�one�influential�point�
Leverage values:� These� values� range� from� 0� to� 1,� with� values� close� to� 1� indicating� greater�leverage��As�a�general�rule,�leverage�values�greater�than�(m +�1)/n�[where�m�equals� the�number�of�independent�variables;�here�(2�+�1)/20�=��15]�indicate�an�influential�case��With� a�maximum�of��307,�there�is�evidence�to�suggest�one�or�more�cases�are�exerting�leverage�
DfBeta:�We�saved�the�DfBeta�values�as�another�indication�of�the�influence�of�a�case��The� DfBeta�provide�information�on�the�change�in�the�predicted�value�when�the�case�is�deleted� from�the�model��For�logistic�regression,�the�DfBeta�values�should�be�smaller�than�1��Looking�at� the�minimum�and�maximum�DfBeta�values�for�the�intercept�(labeled�“constant”)�and�for� household,�we�have�at�least�one�case�that�is�suggestive�of�undue�influence�
Descriptive Statistics
Analog of Cook’s in�uence 20
N Minimum Maximum
20 20 20 20 20
.00000
.00691 –2.22568 –1.68367
–.41034 –1.36519
1.58721
.30726 1.91780 6.53464
.09948 4.10130
20
statistics
Leverage value Normalized residual DfBeta for constant
DfBeta for social DfBeta for household(1)
Valid N (listwise)
From� our� logistic� regression� output,� we� can� review� the� “Casewise List”� to� deter- mine�cases�with�studentized�residuals�larger�than�two�standard�deviations�(recall�from�the� “Options”�dialog�box�that�we�told�SPSS�to�identify�residuals�outside�two�standard�devia- tions)��Here�there�were�two�cases�(cases�8�and�15)�that�were�identified�as�outliers,�and�the� relevant�statistics�(e�g�,�observed�group,�predicted�value,�predicted�group,�residual,�and�stan- dardized�residual)�are�provided��We�examine�these�cases�to�make�sure�there�was�not�a�data� entry�error��If�the�data�are�correct,�then�we�determine�whether�to�keep�or�filter�out�the�case(s)�
Casewise Lista
Case
8 15 S
S U** P**
Selected Statusb
Observed
Kindergarten Readiness Predicted
.832
.214 P U
Predicted Group Resid
–.832 .786
ZResid
–2.226 1.918
Temporary Variable
b S = Selected, U = Unselected cases, and ** = Misclassified cases. a Cases with studentized residuals greater than 2.000 are listed.
Since�we�have�a�small�dataset,�we�can�easily�review�the�values�of�our�diagnostics�and�see� which�cases�are�problematic�in�terms�of�exerting�undue�influence�and/or�outliers��Those� that�are�circled�are�values�that�fall�outside�of�the�recommended�guidelines�and�thus�are� suggestive�of�outlying�or�influential�cases��Due�to�the�already�small�sample�size,�we�will�
744 An Introduction to Statistical Concepts
not� filter� out� any� of� these� potentially� problematic� cases�� However,� in� this� situation� (i�e�,� with� diagnostics� that� suggest� one� or� more� influential� cases),� you� may� want� to� consider� filtering�out�those�cases�or,�at�a�minimum,�reviewing�the�data�to�be�sure�that�there�was�not� a�data�entry�error�for�that�case�
Assessing Classification Accuracy
In�addition�to�examining�Press’s�Q�for�classification�accuracy,�we�can�generate�a�kappa�statis- tic��Kappa�is�the�proportion�of�agreement�above�that�expected�by�chance��A�kappa�statistic�of� 1�0�indicates�perfect�agreement,�whereas�a�kappa�of�0�indicates�chance�agreement��Negative� values�can�occur�and�indicate�weaker�than�chance�agreement��General�rules�of�interpreta- tion�for�kappa�are�as�follows:�small,�<�30;�moderate,��30�to��50;�large,�>�50�
Step 1:�Kappa�statistics�are�generated�through�the�“Crosstab”�procedure��Because�the� process�for�creating�a�crosstab�has�been�presented�previously�(see�Chapter�8),�it�will�not�be� reiterated�here��Once�the�“Crosstab”�dialog�box�is�open,�select�the�dependent�variable�from� the�list�on�the�left�and�use�the�arrow�key�to�move�it�to�“Row(s)�”�Select�the�predicted�group� (PGR_1)�from�the�list�on�the�left�and�use�the�arrow�key�to�move�it�to�“Column(s)”�(see�step�1)�
Kappa statistic: Step 1
Clicking on “Statistics”
will allow you to select the
Kappa statistic.
Clicking on “Cells” will allow you to display expected
counts and column/row/total
percentages.
Select the dependent variable from the list on the
left and use the arrow to move it to the “Row(s)” box
on the right.
Select the predicted group from the list on the left and use the arrow to move
it to the “Column(s)” box on
the right
745Logistic Regression
Step 2:�Click�on�the�“Statistics”�option�button��Place�a�checkmark�in�the�box�next�to� “Kappa”�(step�2)��Then�click�on�“Continue”�to�return�to�the�main�dialog�box�
Kappa statistic: Step 2
Step 3:�Click�on�the�“Cells”�option�button��In�the�“Cell Display”�dialog�box,�place� a�checkmark�in�the�box�next�to�observed, expected,�and�row�(step�3)��Then�click�on� “Continue”�to�return�to�the�main�dialog�box��Then�click�“OK”�to�generate�the�output�
Kappa statistic: Step 3
The�crosstab�table�is�interpreted�as�we�have�seen�in�the�past��The�columns�represent�the� predicted� group� membership,� and� the� rows� represent� the� observed� group� membership�� This�table�should�look�familiar�to�the�one�that�was�provided�to�us�with�the�logistic�regres- sion�results�
746 An Introduction to Statistical Concepts
Kindergarten Readiness * Predicted Group Crosstabulation
Kindergarten readiness
Unprepared
Unprepared
Prepared
Prepared
Count
Count
Count
Expected count
Expected count
Expected count
% Within Kindergarten readiness
% Within Kindergarten readiness
% Within Kindergarten readiness
Total
Total
Predicted Group
7 3.2
87.5%
1
4.8
8
8.0
8.3%
40.0%
1 4.8
12.5%
11 7.2
91.7% 12
12.0 60.0%
8
8.0
100.0%
12 12.0
100.0% 20
20.0 100.0%
What�is�of�most�interest�is�the�table�labeled�“Symmetric�Measures,”�as�this�table�contains� the�kappa�statistic��With�a�kappa�statistic�of��792,�and�using�our�rules�of�thumb�for�interpre- tation,�this�is�considered�to�be�a�large�value,�which�suggests�strong�agreement�
Symmetric Measures
Measure of agreement Value
20 .792 .140 3.540 .000
Errora Approx. Tb Approx. Sig Asymp. Std.
N of valid cases a Not assuming the null hypothesis. b Using the asymptotic standard error assuming the null hypothesis.
Kappa
19.9 G*Power
A� priori� and� post� hoc� power� can� again� be� determined� using� the� specialized� software� described�previously�in�this�text�(e�g�,�G*Power),�or�you�can�consult�a�priori�power�tables� (e�g�,�Cohen,�1988)��As�an�illustration,�we�use�G*Power�to�first�compute�post�hoc�power�of� our�example�
Post Hoc Power for Logistic Regression Using G*Power
The� first� thing� that� must� be� done� when� using� G*Power� for� computing� post� hoc� power� is�to�select�the�correct�test�family��For�logistic�regression,�we�select�“Tests”�in�the�top� pulldown� menu,� then� “Correlation and regression,”� and� finally� “Logistic regression.”�Once�that�selection�is�made,�the�“Test family” automatically�changes� to�“z tests.”
747Logistic Regression
A
B
C
Step 1
The�“Type of Power Analysis”�desired�then�needs�to�be�selected��To�compute�post�hoc� power,�select�“Post hoc:�Compute achieved power—given α, sample size, and effect size.”�For�this�illustration,�we�will�compute�power�for�the�continuous�covariate�
�e “Input Parameters” for computingpost hoc power
must be specified.
Following Step 1 will change the Test family to z
tests.
Following the procedures presented in Step 1 will automatically change the statistical test
to “Logistic regression.”
Here are the post-hoc power results.
Once the parameters are specifed, click on “Calculate.”
Step 2
748 An Introduction to Statistical Concepts
The�“Input Parameters”�must�then�be�specified��In�our�example,�we�conducted�a�two- tailed�test��The�odds�ratio�for�our�continuous�variable�social�development�was�2�631��The� probability�that�Y�=�1�given�that�X�=�1�under�the�null�hypothesis�is�set�to��50��The�alpha�level� we�used�was��05,�and�the�total�sample�size�was�20��“R2 other X”��refers�to�the�squared� correlation� between� social� development� and� our� other� covariate�� In� this� case,� the� simple� bivariate� correlation� between� these� variables� is� �867,� and� the� squared� correlation� is� �752�� Social�development�is�a�continuous�variable;�thus,�it�follows�a�normal�distribution��The�last� two�parameters�to�be�specified�are�for�the�mean�and�standard�deviation�of�our�covariate�� In�this�case,�the�mean�of�social�development�was�20�20,�and�the�standard�deviation�was�6�39�� Once�the�parameters�are�specified,�click�on�“Calculate”�to�find�the�power�statistics�
The�“Output Parameters”�provide�the�relevant�statistics�for�the�input�just�specified�� In�this�example,�we�were�interested�in�determining�post�hoc�power�for�a�logistic�regression� model��Based�on�the�criteria�specified,�the�post�hoc�power�was�substantially�less�than�1��In� other�words,�the�probability�of�rejecting�the�null�hypothesis�when�it�is�really�false�was�sig- nificantly�less�than�1%�(sufficient�power�is�often��80�or�above)��This�finding�is�not�surpris- ing�given�the�very�small�sample�size��Keep�in�mind�that�conducting�power�analysis�a�priori� is�recommended�so�that�you�avoid�a�situation�where,�post�hoc,�you�find�that�the�sample� size�was�not�sufficient�to�reach�the�desired�level�of�power�(given�the�observed�parameters)�
A Priori Power for Logistic Regression Using G*Power
For� a� priori� power,� we� can� determine� the� total� sample� size� needed� for� logistic� regression� given�the�same�parameters�just�discussed��In�this�example,�had�we�wanted�an�a�priori�power� of��80�given�the�same�parameters�just�defined,�we�would�need�a�total�sample�size�of�7094�
Here are the a priori power results.
A priori power
749Logistic Regression
19.10 Template and APA-Style Write-Up
Finally,� here� is� an� example� paragraph� for� the� results� of� the� logistic� regression� analysis�� Recall�that�our�graduate�research�assistant,�Marie,�was�assisting�Malani,�a�faculty�member� in�the�early�childhood�department��Malani�wanted�to�know�if�kindergarten�readiness�(pre- pared�vs��unprepared)�could�be�predicted�by�social�development�(a�continuous�variable)� and�type�of�household�(single-�vs��two-parent�home)��The�research�question�presented�to� Malani�from�Marie�included�the�following:�Can kindergarten readiness be predicted from social development and type of household?
Marie�then�assisted�Malani�in�generating�a�logistic�regression�as�the�test�of�infer- ence,�and�a�template�for�writing�the�research�question�for�this�design�is�presented�as� follows:
•� Can [dependent variable] be predicted from [list independent variables]?
It�may�be�helpful�to�preface�the�results�of�the�logistic�regression�with�information�on�an� examination�of�the�extent�to�which�the�assumptions�were�met��The�assumptions�include� (a)�independence,�(b)�linearity,�and�(c)�noncollinearity��We�will�also�examine�the�data�for� outliers�and�influential�points�
Logistic regression was conducted to determine whether social devel- opment and type of household (single- vs. two-parent home) could predict kindergarten readiness.
The assumptions of logistic regression were tested. Specifically, these include (a) noncollinearity, (b) linearity, and (c) indepen- dence of errors.
In terms of noncollinearity, a VIF value of 4.037 (below the value of 10.0 which indicates the point of concern) and tolerance of .248 (above the value of .10 which suggests multicollinearity) pro- vided evidence of noncollinearity. However, there was some indica- tion that multicollinearity existed. In examining the collinearity diagnostics, a condition index value of 14.259 was observed, about five times larger than the next largest condition index. Review of the variance proportions suggested that 100% of the variance of the regression coefficient for social development and 73% for type of household were related to the smallest eigenvalue. This also suggests multicollinearity.
Linearity was assessed by reestimating the model and including, along with the original predictors, an interaction term which was the prod- uct of the continuous independent variable (i.e., social development) and its natural logarithm. The interaction term was not statistically significant, thus providing evidence of linearity [social*ln(social), B = −2.948, SE = 2.845, Wald = 1.074, df = 1, p = .300].
750 An Introduction to Statistical Concepts
Independence was assessed by examining a plot of the standardized residuals against values of each independent variable. With the exception of one case which was slightly outside the band, all cases were within an absolute value of 2.0, thus indicating the assumption of independence has been met.
In reviewing for outliers and influential points, Cook’s distance values were generally within the recommended range of less than 1.0, although the maximum value was 1.587. Leverage values ranged from .007 to .307, well under the recommended .50, suggesting outliers were not problematic. DfBeta values beyond 1 also suggested cases that may be exerting influence on the model. Based on the evidence reviewed, there are some cases that are suggestive of outlying and influen- tial points. Due to the small sample size, however, these cases were retained. Readers are urged to interpret the results with caution given the possible influence of outliers.
Here�is�an�APA-style�example�paragraph�of�results�for�the�logistic�regression�(remember� that� this� will� be� prefaced� by� the� previous� paragraph� reporting� the� extent� to� which� the� assumptions�of�the�test�were�met)�
Logistic regression analysis was then conducted to determine whether kindergarten readiness (prepared vs. unprepared) could be predicted from social development and type of household (single- vs. two-parent home). Good model fit was evidenced by nonstatisti- cally significant results on the Hosmer–Lemeshow test, χ2 (n = 20) = 4.691, df = 7, p = .698, and large effect size indices when interpreted using Cohen (1988) (Cox and Snell R2 = .546; Nagelkerke R2 = .738). These results suggest that the predictors, as a set, reliably dis- tinguished between children who are ready for kindergarten (i.e., prepared) versus unprepared. Of the two predictors in the model, only social development was a statistically significant predic- tor of kindergarten readiness (Wald = 4.696, df = 1, p = .030). The odds ratio for social development suggests that for every one-point increase in social development, the odds are about two and two- thirds greater for being prepared for kindergarten as compared to unprepared. Type of household was not statistically significant, which suggests that the odds for being prepared for kindergarten (relative to unprepared) are similar regardless of being raised in a single-parent versus a two-parent household. The following table presents the results for the model including the regression coef- ficients, Wald statistics, odds ratios, and 95% CIs for the odds ratios. This is followed by a table which presents the group means and standard deviations of each predictor for both children who are prepared and unprepared for kindergarten.
751Logistic Regression
Logistic Regression Results
95% CI for Exp(B)
B SE Wald p Exp(B) Lower Upper
Intercept (constant)
−15.404 7.195 4.584 .032 NA
Social development .967 .446 4.696 .030 2.631 1.097 6.313 Type of household (two-parent home)
−6.216 3.440 3.265 .071 .002 .000 1.693
Group Means (and Standard Deviations) of Predictors
Predictor Prepared for Kindergarten
Unprepared for Kindergarten
Social development 23.58 (4.74) 15.13 (5.14) Type of household (two-parent home) .67 (.49) .25 (.46)
Overall, the logistic regression model accurately predicted 90% of the children in our sample, with children who are prepared for kindergar- ten slightly more likely to be classified correctly (91.7% of children prepared for kindergarten and 87.5% of children unprepared correctly classified). To account for chance agreement in classification, the kappa coefficient was computed and found to be .792, a large value. Additionally, Press’s Q was calculated to be 12.8, providing evidence that the predictions based on the logistic regression model are sta- tistically significantly better than chance.
19.11 What Is Next?
As�we�conclude�this�text,�the�natural�question�to�ask�is,�what�do�we�consider�next�in�sta- tistics?� There� are� two� likely� key� alternatives�� First,� you� could� consider� more� advanced� regression� models� such� as� multinomial� logistic� regression,� propensity� score� analysis,� or� regression�discontinuity��In�terms�of�more�advanced�regression�readings,�consider�Cohen� and� Cohen� (1983),� Grimm� and� Arnold� (1995),� Kleinbaum,� Kupper,� Muller,� and� Nizam,� (1998),�Meyers,�Gamst,�and�Guarino�(2006),�and�Pedhazur�(1997)��For�more�information�on� logistic� regression,� consider� Christensen� (1997),� Glass� and� Hopkins� (1996),� Hosmer� and� Lemeshow�(2000),�Huck�(2004),�Kleinbaum�et�al��(1998),�Meyers�et�al��(2006),�Pampel�(2000),� Pedhazur�(1997),�and�Wright�(1995)�
In� the� regression� framework,� one� of� the� hottest� topics� relates� to� multilevel� models� that� allow�for�the�examination�of�nested�cases�(e�g�,�children�within�classrooms,�employees�within� organizations,�residents�within�states)��There�are�a�number�of�excellent�resources�for�learn- ing�more�about�multilevel�modeling�including�Heck�and�Thomas�(2000),�Kreft�and�de�Leeuw� (1998),�O’Connell�and�McCoach�(2008),�Reise�and�Dunn�(2003),�and�Snijders�and�Bosker�(1999)�
752 An Introduction to Statistical Concepts
Alternatively� you� could� consider� multivariate� analysis� methods,� either� in� terms� of� readings� or� in� a� multivariate� course�� Briefly,� the� major� methods� of� multivariate� analysis� include� multivariate� analysis� of� variance� (MANOVA),� discriminant� analysis,� factor� and� principal� components� analysis,� canonical� correlation� analysis,� cluster� analysis,� multidi- mensional� scaling,� multivariate� regression,� and� structural� equation� modeling�� For� mul- tivariate� readings,� take� a� look� at� Grimm� and� Arnold� (1995,� 2000),� Johnson� and� Wichern� (1998),�Kleinbaum�et�al��(1998),�Manly�(2004),�Marcoulides�and�Hershberger�(1997),�Meyers� et�al��(2006),�Stevens�(2002),�and�Timm�(2002)�
19.12 Summary
In�this�chapter,�a�regression�method�appropriate�for�binary�categorical�outcomes�was�consid- ered��The�chapter�began�with�an�examination�of�how�logistic�regression�works�and�the�logis- tic�regression�equation��This�was�followed�by�estimation,�model�fit,�significance�tests,�and� assumptions�within�the�context�of�logistic�regression��Effect�size�indices�of�logistic�regression� models�were�also�discussed��In�addition,�several�new�concepts�were�introduced,�including� logit,�odds,�and�odds�ratio��Finally�we�examined�a�number�of�methods�of�variable�entry,�such� as�simultaneous,�stepwise�selection,�and�hierarchical�regression��At�this�point,�you�should� have�met�the�following�objectives:�(a)�be�able�to�understand�the�concepts�underlying�logistic� regression,�(b)�be�able�to�determine�and�interpret�the�results�of�logistic�regression,�(c)�be�able� to�understand�and�evaluate�the�assumptions�of�logistic�regression,�and�(d)�be�able�to�have� a�basic�understanding�of�methods�of�entering�the�covariates��This�concludes�our�statistical� concepts�text��We�wish�you�the�best�of�luck�in�your�future�statistical�adventures�
Problems
Conceptual problems
19.1� Which�one�of�the�following�represents�the�primary�difference�between�OLS�regres- sion�and�logistic�regression?
� a�� Computer�processing�time�to�estimate�the�model � b�� The�measurement�scales�of�the�independent�variables�that�can�be�included�in�the�
model � c�� The�measurement�scale�of�the�dependent�variable � d�� The�statistical�software�that�must�be�used�to�estimate�the�model 19.2� Which� one� of� the� following� is� NOT� an� appropriate� dependent� variable� for� binary�
logistic�regression? � a�� Bernoulli � b�� Dichotomous � c�� Multinomial � d�� One�variable�with�two�categories
753Logistic Regression
19.3� Which�of�the�following�would�NOT�be�appropriate�outcomes�to�examine�with�binary� logistic�regression?
� a�� Employment� status� (employed,� unemployed� not� looking� for� work,� unemployed� looking�for�work)
� b�� Enlisted�member�of�the�military�(member�vs��nonmember) � c�� Marital�status�(married�vs��not�married) � d�� Recreational�athlete�(athlete�vs��nonathlete)
19.4� Which� of� the� following� represents� what� is� being� predicted� in� binary� logistic� regression?
� a�� Mean�difference�between�two�groups � b�� Odds�that�the�unit�of�analysis�belongs�to�one�of�two�groups � c�� Precise�numerical�value � d�� Relationship�between�one�group�compared�to�the�other�group
19.5� While� probability,� odds,� and� log� odds� may� be� computationally� different,� they� all� relay�the�same�basic�information�
� a�� True � b�� False
19.6� A�researcher�is�studying�diet�soda�drinking�habits�and�has�coded�“diet�soda�drinker”� as�“1”�and�“non-diet�soda�drinker”�as�“0�”�Which�of�the�following�is�a�correct�inter- pretation�given�a�probability�value�of��52?
� a�� The�odds�of�being�a�diet�soda�drinker�are�about�equal�to�those�of�not�being�a�diet� soda�drinker�
� b�� The�odds�of�being�a�diet�soda�drinker�are�substantially�greater�than�not�being�a� diet�soda�drinker�
� c�� The�odds�of�being�a�diet�soda�drinker�are�substantially�less�than�not�being�a�diet� soda�drinker�
� d�� Cannot�be�determined�from�the�information�provided�
19.7� Which�of�the�following�is�a�correct�interpretation�of�the�logit? � a�� The�log�odds�become�larger�as�the�odds�increase�from�1�to�100� � b�� The�log�odds�become�smaller�as�the�odds�increase�from�1�to�100� � c�� The�log�odds�stay�relatively�stable�as�the�odds�decrease�from�1�to�0� � d�� The�change�in�log�odds�becomes�larger�when�the�independent�variables�are�cat-
egorical�rather�than�continuous� 19.8� Which�of�the�following�correctly�contrasts�the�estimation�of�OLS�regression�as�com-
pared�to�logistic�regression? � a�� �The� sum� of� the� squared� distance� of� the� observed� data� to� the� regression� line� is�
minimized� in� logistic� regression�� The� log� likelihood� function� is� maximized� in� OLS�regression�
� b�� �The� sum� of� the� squared� distance� of� the� observed� data� to� the� regression� line� is� maximized� in� logistic� regression�� The� log� likelihood� function� is� minimized� in� OLS�regression�
754 An Introduction to Statistical Concepts
� c�� �The�sum�of�the�squared�distance�of�the�observed�data�to�the�regression�line�is�maxi- mized� in� OLS� regression�� The� log� likelihood� function� is� minimized� in� logistic� regression�
� d�� �The� sum� of� the� squared� distance� of� the� observed� data� to� the� regression� line� is� minimized� in� OLS� regression�� The� log� likelihood� function� is� maximized� in� logistic�regression�
19.9� �Which�of�the�following�is�NOT�a�test�that�can�be�used�to�evaluate�overall�model�fit� for�logistic�regression�models?
� a�� Change�in�log�likelihood � b�� Hosmer–Lemeshow�goodness-of-fit � c�� Cox�and�Snell�R�squared � d�� Wald�test 19.10� �A�researcher�is�studying�diet�soda�drinking�habits�and�has�coded�“diet�soda�drinker”�
as�“1”�and�“non-diet�soda�drinker”�as�“0�”�She�has�predicted�drinking�habits�based� on�the�individual’s�weight�(measured�in�pounds)��Given�this�scenario,�which�of�the� following�is�a�correct�interpretation�of�an�odds�ratio�of�1�75?
� a�� �For�every�one-unit�increase�in�being�a�diet�soda�drinker,�the�odds�of�putting�on� an�additional�pound�increase�by�75%�
� b�� �For�every�one-unit�increase�in�being�a�diet�soda�drinker,�the�odds�of�putting�on� an�additional�pound�decrease�by�75%�
� c�� �For� every� 1-pound� increase� in� weight,� the� odds� of� being� a� diet� soda� drinker� decrease�by�75%�
� d�� �For� every� 1-pound� increase� in� weight,� the� odds� of� being� a� diet� soda� drinker� increase�by�75%�
Computational problems
19.1� �You�are�given�the�following�data,�where�X1�(high�school�cumulative�grade�point�aver- age)�and�X2�(participation�in�school-sponsored�athletics;�0�=�nonathlete�and�1�=�athlete;� use�0�as�the�reference�category)�are�used�to�predict�Y�(college�enrollment�immediately� after�high�school,�“1,”�vs��delayed�college�enrollment�or�no�enrollment,�“0”)�
X1 X2 Y
4�15 1 1 2�72 0 1 3�16 0 0 3�89 1 1 4�02 1 1 1�89 0 0 2�10 0 1 2�36 1 1 3�55 0 0 1�70 0 0
Determine�the�following�values�based�on�simultaneous�entry�of�independent�vari- ables:�intercept,�−2LL,�constant,�b1,�b2,�se(b1),�se(b2),�odds�ratios,�Wald1,�Wald2�
755Logistic Regression
19.2� You� are� given� the� following� data,� where� X1� (participation� in� high� school� honors� classes;�yes�=�1,�no�=�0;�use�0�as�the�reference�category)�and�X2�(participation�in�co-op� program�in�college;�yes�=�1,�no�=�0;�use�0�as�the�reference�category)�are�used�to�predict� Y�(baccalaureate�graduation�with�honors�=�1�vs��graduation�without�honors�=�0)�
X1 X2 Y
0 1 1 0 0 1 1 0 0 1 1 1 1 1 1 0 0 0 1 0 1 0 1 1 1 0 0 0 0 0
Determine� the� following� values� based� on� simultaneous� entry� of� independent� vari- ables:�intercept,�−2LL,�constant,�b1,�b2,�se(b1),�se(b2),�odds�ratios,�Wald1,�Wald2�
Interpretive problem
19.1� Use�SPSS�to�develop�a�logistic�regression�model�with�the�example�survey�1�dataset� on�the�website��Utilize�“do�you�smoke”�as�the�dependent�(binary)�variable�to�find�at� least�two�strong�predictors�from�among�the�continuous�and/or�categorical�variables� in�the�dataset��Write�up�the�results�in�APA�style,�including�testing�for�the�assump- tions��Determine�and�interpret�a�measure�of�effect�size�
757
Appendix: Tables
Table a.1
The�Standard�Unit�Normal�Distribution
z P(z) z P(z) z P(z) z P(z)
�00 �5000000 �50 �6914625 1�00 �8413447 1�50 �9331928 �01 �5039894 �51 �6949743 1�01 �8437524 1�51 �9344783 �02 �5079783 �52 �6984682 1�02 �8461358 1�52 �9357445 �03 �5119665 �53 �7019440 1�03 �8484950 1�53 �9369916 �04 �5159534 �54 �7054015 1�04 �8508300 1�54 �9382198 �05 �5199388 �55 �7088403 1�05 �8531409 1�55 �9394292 �06 �5239222 �56 �7122603 1�06 �8554277 1�56 �9406201 �07 �5279032 �57 �7156612 1�07 �8576903 1�57 �9417924 �08 �5318814 �58 �7190427 1�08 �8599289 1�58 �9429466 �09 �5358564 �59 �7224047 1�09 �8621434 1�59 �9440826 �10 �5398278 �60 �7257469 1�10 �8643339 1�60 �9452007 �11 �5437953 �61 �7290691 1�11 �8665005 1�61 �9463011 �12 �5477584 �62 �7323711 1�12 �8686431 1�62 �9473839 �13 �5517168 �63 �7356527 1�13 �8707619 1�63 �9484493 �14 �5556700 �64 �7389137 1�14 �8728568 1�64 �9494974 �15 �5596177 �65 �7421539 1�15 �8749281 1�65 �9505285 �16 �5635595 �66 �7453731 1�16 �8769756 1�66 �9515428 �17 �5674949 �67 �7485711 1�17 �8789995 1�67 �9525403 �18 �5714237 �68 �7517478 1�18 �8809999 1�68 �9535213 �19 �5753454 �69 �7549029 1�19 �8829768 1�69 �9544860 �20 �5792597 �70 �7580363 1�20 �8849303 1�70 �9554345 �21 �5831662 �71 �7611479 1�21 �8868606 1�71 �9563671 �22 �5870644 �72 �7642375 1�22 �8887676 1�72 �9572838 �23 �5909541 �73 �7673049 1�23 �8906514 1�73 �9581849 �24 �5948349 �74 �7703500 1�24 �8925123 1�74 �9590705 �25 �5987063 �75 �7733726 1�25 �8943502 1�75 �9599408 �26 �6025681 �76 �7763727 1�26 �8961653 1�76 �9607961 �27 �6064199 �77 �7793501 1�27 �8979577 1�77 �9616364 �28 �6102612 �78 �7823046 1�28 �8997274 1�78 �9624620 �29 �6140919 �79 �7852361 1�29 �9014747 1�79 �9632730 �30 �6179114 �80 �7881446 1�30 �9031995 1�80 �9640697 �31 �6217195 �81 �7910299 1�31 �9049021 1�81 �9648521 �32 �6255158 �82 �7938919 1�32 �9065825 1�82 �9656205 �33 �6293000 �83 �7967306 l�33 �9082409 1�83 �9663750 �34 �6330717 �84 �7995458 1�34 �9098773 1�84 �9671159 �35 �6368307 �85 �8023375 1�35 �9114920 1�85 �9678432 �36 �6405764 �86 �8051055 1�36 �9130850 1�86 �9685572
(continued)
758 Appendix: Tables
Table a.1 (continued)
The�Standard�Unit�Normal�Distribution
z P(z) z P(z) z P(z) z P(z)
�37 �6443088 �87 �8078498 1�37 �9146565 1�87 �9692581 �38 �6480273 �88 �8105703 1�38 �9162067 1�88 �9699460 �39 �6517317 �89 �8132671 1�39 �9177356 1�89 �9706210 �40 �6554217 �90 �8159399 1�40 �9192433 1�90 �9712834 �41 �6590970 �91 �8185887 1�41 �9207302 1�91 �9719334 �42 �6627573 �92 �8212136 1�42 �9221962 1�92 �9725711 �43 �6664022 �93 �8238145 1�43 �9236415 1�93 �9731966 �44 �6700314 �94 �8263912 1�44 �9250663 1�94 �9738102 �45 �6736448 �95 �8289439 1�45 �9264707 1�95 �9744119 �46 �6772419 �96 �8314724 1�46 �9278550 1�96 �9750021 �47 �6808225 �97 �8339768 1�47 �9292191 1�97 �9755808 �48 �6843863 �98 �8364569 1�48 �9305634 1�98 �9761482 �49 �6879331 �99 �8389129 1�49 �9318879 1�99 �9767045 �50 �6914625 1�00 �8413447 1�50 �9331928 2�00 �9772499
2�00 �9772499 2�50 �9937903 3�00 �9986501 3�50 �9997674 2�01 �9777844 2�51 �9939634 3�01 �9986938 3�51 �9997759 2�02 �9783083 2�52 �9941323 3�02 �9987361 3�52 �9997842 2�03 �9788217 2�53 �9942969 3�03 �9987772 3�53 �9997922 2�04 �9793248 2�54 �9944574 3�04 �9988171 3�54 �9997999 2�05 �9798178 2�55 �9946139 3�05 �9988558 3�55 �9998074 2�06 �9803007 2�56 �9947664 3�06 �9988933 3�56 �9998146 2�07 �9807738 2�57 �9949151 3�07 �9989297 3�57 �9998215 2�08 �9812372 2�58 �9950600 3�08 �9989650 3�58 �9998282 2�09 �9816911 2�59 �9952012 3�09 �9989992 3�59 �9998347 2�10 �9821356 2�60 �9953388 3�10 �9990324 3�60 �9998409 2�11 �9825708 2�61 �9954729 3�11 �9990646 3�61 �9998469 2�12 �9829970 2�62 �9956035 3�12 �9990957 3�62 �9998527 2�13 �9834142 2�63 �9957308 3�13 �9991260 3�63 �9998583 2�14 �9838226 2�64 �9958547 3�14 �9991553 3�64 �9998637 2�15 �9842224 2�65 �9959754 3�15 �9991836 3�65 �9998689 2�16 �9846137 2�66 �9960930 3�16 �9992112 3�66 �9998739 2�17 �9849966 2�67 �9962074 3�17 �9992378 3�67 �9998787 2�18 �9853713 2�68 �9963189 3�18 �9992636 3�68 �9998834 2�19 �9857379 2�69 �9964274 3�19 �9992886 3�69 �9998879 2�20 �9860966 2�70 �9965330 3�20 �9993129 3�70 �9998922 2�21 �9864474 2�71 �9966358 3�21 �9993363 3�71 �9998964 2�22 �9867906 2�72 �9967359 3�22 �9993590 3�72 �9999004 2�23 �9871263 2�73 �9968333 3�23 �9993810 3�73 �9999043 2�24 �9874545 2�74 �9969280 3�24 �9994024 3�74 �9999080 2�25 �9877755 2�75 �9970202 3�25 �9994230 3�75 �9999116 2�26 �9880894 2�76 �9971099 3�26 �9994429 3�76 �9999150 2�27 �9883962 2�77 �9971972 3�27 �9994623 3�77 �9999184 2�28 �9886962 2�78 �9972821 3�28 �9994810 3�78 �9999216 2�29 �9889893 2�79 �9973646 3�29 �9994991 3�79 �9999247 2�30 �9892759 2�80 �9974449 3�30 �9995166 3�80 �9999277
759Appendix: Tables
Table a.1 (continued)
The�Standard�Unit�Normal�Distribution
z P(z) z P(z) z P(z) z P(z)
2�31 �9895559 2�81 �9975229 3�31 �9995335 3�81 �9999305 2�32 �9898296 2�82 �9975988 3�32 �9995499 3�82 �9999333 2�33 �9900969 2�83 �9976726 3�33 �9995658 3�83 �9999359 2�34 �9903581 2�84 �9977443 3�34 �9995811 3�84 �9999385 2�35 �9906133 2�85 �9978140 3�35 �9995959 3�85 �9999409 2�36 �9908625 2�86 �9978818 3�36 �9996103 3�86 �9999433 2�37 �9911060 2�87 �9979476 3�37 �9996242 3�87 �9999456 2�38 �9913437 2�88 �9980116 3�88 �9996376 3�88 �9999478 2�39 �9915758 2�89 �9980738 3�39 �9996505 3�89 �9999499 2�40 �9918025 2�90 �9981342 3�40 �9996631 3�90 �9999519 2�41 �9920237 2�91 �9981929 3�41 �9996752 3�91 �9999539 2�42 �9922397 2�92 �9982498 3�42 �9996869 3�92 �9999557 2�43 �9924506 2�93 �9983052 3�43 �9996982 3�93 �9999575 2�44 �9926564 2�94 �9983589 3�44 �9997091 3�94 �9999593 2�45 �9928572 2�95 �9984111 3�45 �9997197 3�95 �9999609 2�46 �9930531 2�96 �9984618 3�46 �9997299 3�96 �9999625 2�47 �9932443 2�97 �9985110 3�47 �9997398 3�97 �9999641 2�48 �9934309 2�98 �9985588 3�48 �9997493 3�98 �9999655 2�49 �9936128 2�99 �9986051 3�49 �9997585 3�99 �9999670 2�50 �9937903 3�00 �9986501 3�50 �9997674 4�00 �9999683
Source:� Reprinted� from� Pearson,� E�S�� and� Hartley,� H�O�,� Biometrika Tables for Statisticians,� Cambridge� University� Press,� Cambridge,� U�K�,� 1966,� Table�1��With�permission�of�Biometrika�Trustees�
P(z)�represents�the�area�below�that�value�of�z.
760 Appendix: Tables
Table a.2
Percentage�Points�of�the�t�Distribution
v
α1 = .10 .05 .025 .01 .005 .0025 .001 .0005
α2 = .20 .10 .050 .02 .010 .0050 .002 .0010
1 3�078 6�314 12�706 31�821 63�657 127�32 318�31 636�62 2 1�886 2�920 4�303 6�965 9�925 14�089 22�327 31�598 3 1�638 2�353 3�182 4�541 5�841 7�453 10�214 12�924 4 1�533 2�132 2�776 3�747 4�604 5�598 7�173 8�610 5 1�476 2�015 2�571 3�365 4�032 4�773 5�893 6�869 6 1�440 1�943 2�447 3�143 3�707 4�317 5�208 5�959 7 1�415 1�895 2�305 2�998 3�499 4�029 4�785 5�408 8 1�397 1�860 2�306 2�896 3�355 3�833 4�501 5�041 9 1�383 1�833 2�262 2�821 3�250 3�690 4�297 4�781
10 1�372 1�812 2�228 2�764 3�169 3�581 4�144 4�587 11 1�363 1�796 2�201 2�718 3�106 3�497 4�025 4�437 12 1�356 1�782 2�179 2�681 3�055 3�428 3�930 4�318 13 1�350 1�771 2�160 2�650 3�012 3�372 3�852 4�221 14 1�345 1�761 2�145 2�624 2�977 3�326 3�787 4�140 15 1�341 1�753 2�131 2�602 2�947 3�286 3�733 4�073 16 1�337 1�746 2�120 2�583 2�921 3�252 3�686 4�015 17 1�333 1�740 2�110 2�567 2�898 3�222 3�646 3�965 18 1�330 1�734 2�101 2�552 2�878 3�197 3�610 3�922 19 1�328 1�729 2�093 2�539 2�861 3�174 3�579 3�883 20 1�325 1�725 2�086 2�528 2�845 3�153 3�552 3�850 21 1�323 1�721 2�080 2�518 2�831 3�135 3�527 3�819 22 1�321 1�717 2�074 2�508 2�819 3�119 3�505 3�792 23 1�319 1�714 2�069 2�500 2�807 3�104 3�485 3�767 24 1�318 1�711 2�064 2�492 2�797 3�091 3�467 3�745 25 1�316 1�708 2�060 2�485 2�787 3�078 3�450 3�725 26 1�315 1�706 2�056 2�479 2�779 3�067 3�435 3�707 27 1�314 1�703 2�052 2�473 2�771 3�057 3�421 3�690 28 1�313 1�701 2�048 2�467 2�763 3�047 3�408 3�674 29 1�311 1�699 2�045 2�462 2�756 3�038 3�396 3�659 30 1�310 1�697 2�042 2�457 2�750 3�030 3�385 3�646 40 1�303 1�684 2�021 2�423 2�704 2�971 3�307 3�551 60 1�296 1�671 2�000 2�390 2�660 2�915 3�232 3�460
120 1�289 1�658 1�980 2�358 2�617 2�860 3�160 3�373 ∞ 1�282 1�645 1�960 2�326 2�576 2�807 3�090 3�291
Source:� Reprinted� from� Pearson,� E�S�� and� Hartley,� H�O�,� Biometrika Tables for Statisticians,� Cambridge� University� Press,� Cambridge,� U�K�,� 1966,� Table� 12�� With�permission�of�Biometrika�Trustees�
αt�is�the�upper-tail�value�of�the�distribution�with�v�degrees�of�freedom;�appropriate�for� use�in�a�one-tailed�test�
Use�α2�for�a�two-tailed�test�
761Appendix: Tables
Table a.3
Percentage�Points�of�the�χ2�Distribution
α v 0.990 0.975 0.950 0.900 0.100 0.050 0.025 0.010
1 157088 ���10−9 982069���10−8 393214���10−8 0�0157908 2�70554 3�84146 5�02389 6�63490 2 0�0201007 0�0506356 0�102587 0�210721 4�60517 5�99146 7�37776 9�21034 3 0�114832 0�215795 0�351846 0�584374 6�25139 7�81473 9�34840 11�3449 4 0�297109 0�484419 0�710723 1�063623 7�77944 9�48773 11�1433 13�2767 5 0�554298 0�831212 1�145476 1�61031 9�23636 11�0705 12�8325 15�0863 6 0�872090 1�23734 1�63538 2�20413 10�6446 12�5916 14�4494 16�8119 7 1�239043 1�68987 2�16735 2�83311 12�0170 14�0671 16�0128 18�4753 8 1�64650 2�17973 2�73264 3�48954 13�3616 15�5073 17�5345 20�0902 9 2�08790 2�70039 3�32511 4�16816 14�6837 16�9190 19�0228 21�6660
10 2�55821 3�24697 3�94030 4�86518 15�9872 18�3070 20�4832 23�2093 11 3�05348 3�81575 4�57481 5�57778 17�2750 19�6751 21�9200 24�7250 12 3�57057 4�40379 5�22603 6�30380 18�5493 21�0261 23�3367 26�2170 13 4�10692 5�00875 5�89186 7�04150 19�8119 22�3620 24�7356 27�6882 14 4�66043 5�62873 6�57063 7�78953 21�0641 23�6848 26�1189 29�1412 15 5�22935 6�26214 7�26094 8�54676 22�3071 24�9958 27�4884 30�5779 16 5�81221 6�90766 7�96165 9�31224 23�5418 26�2962 28�8454 31�9999 17 6�40776 7�56419 8�67176 10�0852 24�7690 27�5871 30�1910 33�4087 18 7�01491 8�23075 9�39046 10�8649 25�9894 28�8693 31�5264 34�8053 19 7�63273 8�90652 10�1170 11�6509 27�2036 30�1435 32�8523 36�1909 20 8�26040 9�59078 10�8508 12�4426 28�4120 31�4104 34�1696 37�5662 21 8�89720 10�28293 11�5913 13�2396 29�6151 32�6706 35�4789 38�9322 22 9�54249 10�9823 12�3380 14�0415 30�8133 33�9244 36�7807 40�2894 23 10�19567 11�6886 13�0905 14�8480 32�0069 35�1725 38�0756 41�6384 24 10�8564 12�4012 13�8484 15�6587 33�1962 36�4150 39�3641 42�9798 25 11�5240 13�1197 14�6114 16�4734 34�3816 37�6525 40�6465 44�3141 26 12�1981 13�8439 15�3792 17�2919 35�5632 38�8851 41�9232 45�6417 27 12�8785 14�5734 16�1514 18�1139 36�7412 40�1133 43�1945 46�9629 28 13�5647 15�3079 16�9279 18�9392 37�9159 41�3371 44�4608 48�2782 29 14�2565 16�0471 17�7084 19�7677 39�0875 42�5570 45�7223 49�5879 30 14�9535 16�7908 18�4927 20�5992 40�2560 43�7730 46�9792 50�8922 40 22�1643 24�4330 26�5093 29�0505 51�8051 55�7585 59�3417 63�6907 50 29�7067 32�3574 34�7643 37�6886 63�1671 67�5048 71�4202 76�1539 60 37�4849 40�4817 43�1880 46�4589 74�3970 79�0819 83�2977 88�3794 70 45�4417 48�7576 51�7393 55�3289 85�5270 90�5312 95�0232 100�425 80 53�5401 57�1532 60�3915 64�2778 96�5782 101�879 106�629 112�329 90 61�7541 66�6466 69�1260 73�2911 107�565 113�145 118�136 124�116
100 70�0649 74�2219 77�9295 82�3581 118�498 124�342 129�561 135�807
Source:� Reprinted�from�Pearson,�E�S��and�Hartley,�H�O�,�Biometrika Tables for Statisticians,�Cambridge�University� Press,�Cambridge,�U�K�,�1966,�Table�8��With�permission�of�Biometrika�Trustees�
762 Appendix: Tables Ta
b le
a .4
P er
ce n
ta g
e� P
o i n
ts �o
f� th
e� F
�D i s
tr ib
u ti
o n
v 2
v 1
1 2
3 4
5 6
7 8
9 10
12 15
20 24
30 40
60 12
0 ∞
α �=
.1 0
1 39
�8 6
49 �5
0 53
�5 9
55 �8
3 57
�2 4
58 �2
0 58
�9 1
59 �4
4 59
�8 6
60 �1
9 60
�7 1
61 �2
2 61
�7 4
62 �0
0 62
�2 6
62 �5
3 62
�7 9
63 �0
6 63
�3 3
2 8 �
53 9�
00 9�
16 9�
24 9�
29 9�
33 9�
35 9�
37 9�
38 9�
39 9�
41 9�
42 9�
44 9�
45 9�
46 9�
47 9�
47 9�
48 9�
49 3
5 � 54
5� 46
5� 39
5� 34
5� 31
5� 28
5� 27
5� 25
5� 24
5� 23
5� 22
5� 20
5� 18
5� 18
5� 17
5� 16
5� 15
5� 14
5� 13
4 4 �
54 4�
32 4�
19 4�
11 4�
05 4�
01 3�
98 3�
95 3�
94 3�
92 3�
90 3�
87 3�
84 3�
83 3�
82 3�
80 3�
79 3�
78 3�
76 5
4 � 06
3� 78
3� 62
3� 52
3� 45
3� 40
3� 37
3� 34
3� 32
3� 30
3� 27
3� 24
3� 21
3� 19
3� 17
3� 16
3� 14
3� 12
3� 10
6 3 �
78 3�
46 3�
29 3�
18 3�
11 3�
05 3�
01 2�
98 2�
96 2�
94 2�
90 2�
87 2�
84 2�
82 2�
80 2�
78 2�
76 2�
74 2�
72 7
3 � 59
3� 26
3� 07
2� 96
2� 88
2� 83
2� 78
2� 75
2� 72
2� 70
2� 67
2� 63
2� 59
2� 58
2� 56
2� 54
2� 51
2� 49
2� 47
8 3 �
46 3�
11 2�
92 2�
81 2�
73 2�
67 2�
62 2�
59 2�
56 2�
54 2�
50 2�
46 2�
42 2�
40 2�
38 2�
36 2�
34 2�
32 2�
29 9
3 � 36
3� 01
2� 81
2� 69
2� 61
2� 55
2� 51
2� 47
2� 44
2� 42
2� 38
2� 34
2� 30
2� 28
2� 25
2� 23
2� 21
2� 18
2� 16
10 3 �
29 2�
92 2�
73 2�
61 2�
52 2�
46 2�
41 2�
38 2�
35 2�
32 2�
28 2�
24 2�
20 2�
18 2�
16 2�
13 2�
11 2�
08 2�
06 11
3� 23
2� 86
2� 66
2� 54
2� 45
2� 39
2� 34
2� 30
2� 27
2� 25
2� 21
2� 17
2� 12
2� 10
2� 08
2� 05
2� 03
2� 00
1� 97
12 3 �
18 2�
81 2�
61 2�
48 2�
39 2�
33 2�
28 2�
24 2�
21 2�
19 2�
15 2�
10 2�
06 2�
04 2�
01 1�
99 1�
96 1�
93 1�
90 13
3 � 14
2� 76
2� 56
2� 43
2� 35
2� 28
2� 23
2� 20
2� 16
2� 14
2� 10
2� 05
2� 01
1� 98
1� 96
1� 93
1� 90
1� 88
1� 85
14 3 �
10 2�
73 2�
52 2�
39 2�
31 2�
24 2�
19 2�
15 2�
12 2�
10 2�
05 2�
01 1�
96 1�
94 1�
91 1�
89 1�
86 1�
83 1�
80 15
3 � 07
2� 70
2� 49
2� 36
2� 27
2� 21
2� 16
2� 12
2� 09
2� 06
2� 02
1� 97
1� 92
1� 90
1� 87
1� 85
1� 82
1� 79
1� 76
16 3 �
05 2�
67 2�
46 2�
33 2�
24 2�
18 2�
13 2�
09 2�
06 2�
03 1�
99 1�
94 1�
89 1�
87 1�
84 1�
81 1�
78 1�
75 1�
72 17
3 � 03
2� 64
2� 44
2� 31
2� 22
2� 15
2� 10
2� 06
2� 03
2� 00
1� 96
1� 91
1� 86
1� 84
1� 81
1� 78
1� 75
1� 72
1� 69
18 3 �
01 2�
62 2�
42 2�
29 2�
20 2�
13 2�
08 2�
04 2�
00 1�
98 1�
93 1�
89 1�
84 1�
81 1�
78 1�
75 1�
72 1�
69 1�
66 19
2 � 99
2� 61
2� 40
2� 27
2� 18
2� 11
2� 06
2� 02
1� 98
1� 96
1� 91
1� 86
1� 81
1� 79
1� 76
1� 73
1� 70
1� 67
1� 63
20 2�
97 2�
59 2�
38 2�
25 2�
16 2�
09 2�
04 2�
00 1�
96 1�
94 1�
89 1�
84 1�
79 1�
77 1�
74 1�
71 1�
68 1�
64 1�
61 21
2 � 96
2� 57
2� 36
2� 23
2� 14
2� 08
2� 02
1� 98
1� 95
1� 92
1� 87
1� 83
1� 78
1� 75
1� 72
1� 69
1� 66
1� 62
1� 59
22 2 �
95 2�
56 2�
35 2�
22 2�
13 2�
06 2�
01 1�
97 1�
93 1�
90 1�
86 1�
81 1�
76 1�
73 1�
70 1�
67 1�
64 1�
60 1�
57 23
2 � 94
2� 55
2� 34
2� 21
2� 11
2� 05
1� 99
1� 95
1� 92
1� 89
1� 84
1� 80
1� 74
1� 72
1� 69
1� 66
1� 62
1� 59
1� 55
24 2 �
93 2�
54 2�
33 2�
19 2�
10 2�
04 1�
98 1�
94 1�
91 1�
88 1�
83 1�
78 1�
73 1�
70 1�
67 1�
64 1�
61 1�
57 1�
53 25
2 � 92
2� 53
2� 32
2� 18
2� 09
2� 02
1� 97
1� 93
1� 89
1� 87
1� 82
1� 77
1� 72
1� 69
1� 66
1� 63
1� 59
1� 56
1� 52
26 2 �
91 2�
52 2�
31 2�
17 2�
08 2�
01 1�
96 1�
92 1�
88 1�
86 1�
81 1�
76 1�
71 1�
68 1�
65 1�
61 1�
58 1�
54 1�
50 27
2 � 90
2� 51
2� 30
2� 17
2� 07
2� 00
1� 95
1� 91
1� 87
1� 85
1� 80
1� 75
1� 70
1� 67
1� 64
1� 60
1� 57
1� 53
1� 49
763Appendix: Tables 28
2� 89
2� 50
2� 29
2� 16
2� 06
2� 00
1� 94
1� 90
1� 87
1� 84
1� 79
1� 74
1� 69
1� 66
1� 63
1� 59
1� 56
1� 52
1� 48
29 2�
89 2�
50 2�
28 2�
15 2�
06 1�
99 1�
93 1�
89 1�
86 1�
83 1�
78 1�
73 1�
68 1�
65 1�
62 1�
58 1�
55 1�
51 1�
47 30
2� 88
2� 49
2� 28
2� 14
2� 05
1� 98
1� 93
1� 88
1� 85
1� 82
1� 77
1� 72
1� 67
1� 64
1� 61
1� 57
1� 54
1� 50
1� 46
40 2�
84 2�
44 2�
23 2�
09 2�
00 1�
93 1�
87 1�
83 1�
79 1�
76 1�
71 1�
66 1�
61 1�
57 1�
54 1�
51 1�
47 1�
42 1�
38 60
2� 79
2� 39
2� 18
2� 04
1� 95
1� 87
1� 82
1� 77
1� 74
1� 71
1� 66
1� 60
1� 54
1� 51
1� 48
1� 44
1� 40
1� 35
1� 29
12 0
2� 75
2� 35
2� 13
1� 99
1� 90
1� 82
1� 77
1� 72
1� 68
1� 65
1� 60
1� 55
1� 48
1� 45
1� 41
1� 37
1� 32
1� 26
1� 19
∞ 2�
71 2�
30 2�
08 1�
94 1�
85 1�
77 1�
72 1�
67 1�
63 1�
60 1�
55 1�
49 1�
42 1�
38 1�
34 1�
30 1�
24 1�
17 1�
00
α �=
.0 5
1 16
1� 4
19 9�
5 21
5� 7
22 4�
6 23
0� 2
23 4�
0 23
6� 8
23 8�
9 24
0� 5
24 1�
9 24
3� 9
24 5�
9 24
8� 0
24 9�
1 25
0� 1
25 1�
1 25
2� 2
25 3�
3 25
4� 3
2 18
�5 1
19 �0
0 19
�1 6
19 �2
5 19
�3 0
19 �3
3 19
�3 5
19 �3
7 19
�3 8
19 �1
0 19
�4 1
19 �4
3 19
�4 5
19 �4
5 19
�4 6
19 �4
7 19
�4 8
19 �4
9 19
�5 0
3 10
�1 3
9� 55
9� 28
9� 12
9� 01
8� 94
8� 89
8� 85
8� 81
8� 79
8� 74
8� 70
8� 66
8� 04
8� 62
8� 59
8� 57
8� 55
8� 53
4 7�
71 6�
94 6�
59 6�
39 6�
26 6�
16 6�
09 6�
04 6�
00 5�
96 5�
91 5�
86 5�
80 5�
77 5�
75 5�
72 5�
69 5�
66 5�
63 5
6� 61
5� 79
5� 41
5� 19
5� 05
4� 95
4� 88
4� 82
4� 77
4� 74
4� 68
4� 62
4� 50
4� 53
4� 50
4� 46
4� 43
4� 40
4� 36
6 5�
99 5�
14 4�
76 4�
53 4�
39 4�
28 4�
21 4�
15 4�
10 4�
06 4�
00 3�
94 3�
87 3�
84 3�
81 3�
77 3�
74 3�
70 3�
67 7
5� 59
4� 74
4� 35
4� 12
3� 97
3� 87
3� 79
3� 73
3� 68
3� 64
3� 57
3� 51
3� 44
3� 41
3� 38
3� 34
3� 30
3� 27
3� 23
8 5�
32 4�
46 4�
07 3�
84 3�
69 3�
58 3�
50 3�
44 3�
39 3�
35 3�
28 3�
22 3�
15 3�
12 3�
08 3�
04 3�
01 2�
97 2�
93 9
5� 12
4� 26
3� 86
3� 63
3� 48
3� 37
3� 29
3� 23
3� 18
3� 14
3� 07
3� 01
2� 94
2� 90
2� 80
2� 83
2� 79
2� 75
2� 71
10 4�
96 4�
10 3�
71 3�
48 3�
33 3�
22 3�
14 3�
07 3�
02 2�
98 2�
91 2�
85 2�
77 2�
74 2�
70 2�
66 2�
62 2�
58 2�
54 1 1
4� 84
3� 98
3� 59
3� 36
3� 20
3� 09
3� 01
2� 95
2� 90
2� 85
2� 79
2� 72
2� 65
2� 61
2� 57
2� 53
2� 49
2� 45
2� 40
12 4�
75 3�
89 3�
49 3�
26 3�
11 3�
00 2�
91 2�
85 2�
80 2�
75 2�
69 2�
62 2�
54 2�
51 2�
47 2�
43 2�
38 2�
34 2�
30 13
4� 67
3� 81
3� 41
3� 18
3� 03
2� 92
2� 83
2� 77
2� 71
2� 67
2� 60
2� 53
2� 46
2� 42
2� 38
2� 34
2� 30
2� 25
2� 21
14 4�
60 3�
74 3�
34 3�
1 1 2�
96 2�
85 2�
76 2�
70 2�
65 2�
60 2�
53 2�
46 2�
39 2�
35 2�
31 2�
27 2�
22 2�
18 2�
13 15
4� 54
3� 68
3� 29
3� 06
2� 90
2� 79
2� 71
2� 64
2� 59
2� 54
2� 48
2� 40
2� 33
2� 29
2� 25
2� 20
2� 16
2� 1 1
2� 07
16 4�
49 3�
63 3�
24 3�
01 2�
85 2�
74 2�
66 2�
59 2�
54 2�
49 2�
42 2�
35 2�
28 2�
24 2�
19 2�
15 2�
1 1 2�
06 2�
01 17
4� 45
3� 59
3� 20
2� 96
2� 81
2� 70
2� 61
2� 55
2� 49
2� 45
2� 38
2� 31
2� 23
2� 19
2� 15
2� 10
2� 06
2� 01
1� 96
18 4�
41 3�
55 3�
16 2�
93 2�
77 2�
66 2�
58 2�
51 2�
46 2�
41 2�
34 2�
27 2�
19 2�
15 2�
1 1 2�
06 2�
02 1�
97 1�
92 19
4� 38
3� 52
3� 13
2� 90
2� 74
2� 63
2� 54
2� 48
2� 42
2� 38
2� 31
2� 23
2� 16
2� 1 1
2� 07
2� 03
1� 98
1� 93
1� 88
20 4�
35 3�
49 3�
10 2�
87 2�
71 2�
60 2�
51 2�
45 2�
39 2�
35 2�
28 2�
20 2�
12 2�
08 2�
04 1�
99 1�
95 1�
90 1�
84 21
4� 32
3� 47
3� 07
2� 84
2� 68
2� 57
2� 49
2� 42
2� 37
2� 32
2� 25
2� 18
2� 10
2� 05
2� 01
1� 96
1� 92
1� 87
1� 81
22 4�
30 3�
44 3�
05 2�
82 2�
66 2�
55 2�
46 2�
40 2�
34 2�
30 2�
23 2�
15 2�
07 2�
03 1�
98 1�
94 1�
89 1�
84 1�
78 23
4� 28
3� 42
3� 03
2� 80
2� 64
2� 53
2� 44
2� 37
2� 32
2� 27
2� 20
2� 13
2� 05
2� 01
1� 96
1� 91
1� 86
1� 81
1� 76
24 4�
26 3�
40 3�
01 2�
78 2�
62 2�
51 2�
42 2�
36 2�
30 2�
25 2�
18 2�
1 1 2�
03 1�
98 1�
94 1�
89 1�
84 1�
79 1�
73 ( c
on ti
n u
ed )
764 Appendix: Tables
Ta b
le a
.4 (
co n
ti n
u ed
)
P er
ce n
ta g
e� P
o i n
ts �o
f� th
e� F
�D i s
tr ib
u ti
o n
v 2
v 1
1 2
3 4
5 6
7 8
9 10
12 15
20 24
30 40
60 12
0 ∞
α �=
.0 5
25 4 �
24 3�
39 2�
99 2�
76 2�
60 2�
49 2�
40 2�
34 2�
28 2�
24 2�
16 2�
09 2�
01 1�
96 1�
92 1�
87 1�
82 1�
77 1�
71 26
4 � 23
3� 37
2� 98
2� 74
2� 59
2� 47
2� 39
2� 32
2� 27
2� 22
2� 15
2� 07
1� 99
1� 95
1� 90
1� 85
1� 80
1� 75
1� 69
27 4 �
21 3�
35 2�
96 2�
73 2�
57 2�
46 2�
37 2�
31 2�
25 2�
20 2�
13 2�
06 1�
97 1�
93 1�
88 1�
84 1�
79 1�
73 1�
67 28
4 � 20
3� 34
2� 95
2� 71
2� 56
2� 45
2� 36
2� 29
2� 24
2� 19
2� 12
2� 04
1� 96
1� 91
1� 87
1� 82
1� 77
1� 71
1� 65
29 4 �
18 3�
33 2�
93 2�
70 2�
55 2�
43 2�
35 2�
28 2�
22 2�
18 2�
10 2�
03 1�
94 1�
90 1�
85 1�
81 1�
75 1�
70 1�
64 30
4 � 17
3� 32
2� 92
2� 69
2� 53
2� 42
2� 33
2� 27
2� 21
2� 16
2� 09
2� 01
1� 93
1� 89
1� 84
1� 79
1� 74
1� 68
1� 62
40 4 �
08 3�
23 2�
84 2�
61 2�
45 2�
34 2�
25 2�
18 2�
12 2�
08 2�
00 1�
92 1�
84 1�
79 1�
74 1�
69 1�
64 1�
58 1�
51 60
4 � 00
3� 15
2� 76
2� 53
2� 37
2� 25
2� 17
2� 10
2� 04
1� 99
1� 92
1� 84
1� 75
1� 70
1� 65
1� 59
1� 53
1� 47
1� 39
12 0
3 � 92
3� 07
2� 68
2� 45
2� 29
2� 17
2� 09
2� 02
1� 96
1� 91
1� 83
1� 75
1� 66
1� 61
1� 55
1� 50
1� 43
1� 35
1� 25
∞ 3 �
84 3�
00 2�
60 2�
37 2�
21 2�
10 2�
01 1�
94 1�
88 1�
83 1�
75 1�
67 1�
57 1�
52 1�
46 1�
39 1�
32 1�
22 1�
00
α �=
.0 1
1 40
52 49
99 �5
54 03
56 25
57 64
58 59
59 28
59 81
60 22
60 56
61 06
61 57
62 09
62 35
62 61
62 87
63 13
63 39
63 66
2 98
�5 0
99 �0
0 99
�1 7
99 �2
5 99
�3 0
99 �3
3 99
�3 6
99 �3
7 99
�3 9
99 �4
0 99
�4 2
99 �4
3 99
�4 5
99 �4
6 99
�4 7
99 �4
7 99
�4 8
99 �4
9 99
�5 0
3 34
�1 2
30 �8
2 29
�4 6
28 �7
1 28
�2 4
27 �9
1 27
�6 7
27 �4
9 27
�3 5
27 �2
3 27
�0 5
26 �8
7 26
�6 9
26 �6
0 26
�5 0
26 �4
1 26
�3 2
25 �2
2 26
�1 3
4 21
�2 0
18 �0
0 16
�6 9
15 �9
8 15
�5 2
15 �2
1 14
�9 8
14 �8
0 14
�6 6
14 �5
5 14
�3 7
14 �2
0 14
�0 2
13 �9
3 13
�8 4
13 �7
5 13
�5 5
13 �5
6 13
�4 6
5 16
�2 6
13 �2
7 12
�0 6
1 1 �3
9 10
�9 7
10 �6
7 10
�4 6
10 �2
9 10
�1 6
10 �0
5 9�
89 9�
72 9�
55 9�
47 9�
38 9�
29 9�
20 9�
11 9�
02 6
13 �7
5 10
�9 2
9� 78
9� 15
8� 75
8� 47
8� 26
8� 10
7� 98
7� 87
7� 72
7� 56
7� 40
7� 31
7� 23
7� 14
7� 06
6� 97
6� 88
7 12
�2 5
9� 55
8� 45
7� 85
7� 46
7� 19
6� 99
6� 84
6� 72
6� 62
6� 47
6� 31
6� 16
6� 07
5� 99
5� 91
5� 82
5� 74
5� 65
8 1 1
�2 6
8� 65
7� 59
7� 01
6� 63
6� 37
6� 18
6� 03
5� 91
5� 81
5� 67
5� 52
5� 36
5� 28
5� 20
5� 12
5� 03
4� 95
4� 86
9 10
�5 6
8� 02
6� 99
6� 42
6� 06
5� 80
5� 61
5� 47
5� 35
5� 26
5� 11
4� 96
4� 81
4� 73
4� 65
4� 57
4� 48
4� 40
4� 31
10 10
�0 4
7� 56
6� 55
5� 99
5� 64
5� 39
5� 20
5� 06
4� 94
4� 85
4� 71
4� 56
4� 41
4� 33
4� 25
4� 17
4� 08
4� 00
3� 91
11 9�
65 7�
21 6�
22 5�
67 5�
32 5�
07 4�
89 4�
74 4�
63 4�
54 4�
40 4�
25 4�
10 4�
02 3�
94 3�
86 3�
78 3�
69 3�
60 12
9 � 33
6� 93
5� 95
5� 41
5� 06
4� 82
4� 64
4� 50
4� 39
4� 30
4� 16
4� 01
3� 86
3� 78
3� 70
3� 62
3� 54
3� 45
3� 36
765Appendix: Tables 13
9� 07
6� 70
5� 74
5� 21
4� 86
4� 62
4� 44
4� 30
4� 19
4� 10
3� 96
3� 82
3� 66
3� 59
3� 51
3� 43
3� 34
3� 25
3� 17
14 8�
86 6�
51 5�
56 5�
04 4�
69 4�
46 4�
28 4�
14 4�
03 3�
94 3�
80 3�
66 3�
51 3�
43 3�
35 3�
27 3�
18 3�
09 3�
00 15
8� 68
6� 36
5� 42
4� 89
4� 56
4� 32
4� 14
4� 00
3� 89
3� 80
3� 67
3� 52
3� 37
3� 29
3� 21
3� 13
3� 05
2� 96
2� 87
16 8�
53 6�
23 5�
29 4�
77 4�
44 4�
20 4�
03 3�
89 3�
78 3�
69 3�
55 3�
41 3�
26 3�
18 3�
10 3�
02 2�
93 2�
84 2�
75 17
8� 40
6� 1 1
5� 18
4� 67
4� 34
4� 10
3� 93
3� 79
3� 68
3� 59
3� 46
3� 31
3� 16
3� 08
3� 00
2� 92
2� 83
2� 75
2� 65
18 8�
29 6�
01 5�
09 4�
58 4�
25 4�
01 3�
84 3�
71 3�
60 3�
51 3�
37 3�
23 3�
08 3�
00 2�
92 2�
84 2�
75 2�
66 2�
57 19
8� 18
5� 93
5� 01
4� 50
4� 17
3� 94
3� 77
3� 63
3� 52
3� 43
3� 30
3� 15
3� 00
2� 92
2� 84
2� 76
2� 67
2� 58
2� 49
20 8�
10 5�
85 4�
94 4�
43 4�
10 3�
87 3�
70 3�
56 3�
46 3�
37 3�
23 3�
09 2�
94 2�
86 2�
78 2�
69 2�
61 2�
52 2�
42 21
8� 02
5� 78
4� 87
4� 37
4� 04
3� 81
3� 64
3� 51
3� 40
3� 31
3� 17
3� 03
2� 88
2� 80
2� 72
2� 64
2� 55
2� 46
2� 36
22 7�
95 5�
72 4�
82 4�
31 3�
99 3�
76 3�
59 3�
45 3�
35 3�
26 3�
12 2�
98 2�
83 2�
75 2�
67 2�
58 2�
50 2�
40 2�
31 23
7� 88
5� 66
4� 76
4� 26
3� 94
3� 71
3� 54
3� 41
3� 30
3� 21
3� 07
2� 93
2� 78
2� 70
2� 62
2� 54
2� 45
2� 35
2� 26
24 7�
82 5�
61 4�
72 4�
22 3�
90 3�
67 3�
50 3�
36 3�
26 3�
17 3�
03 2�
89 2�
74 2�
66 2�
58 2�
49 2�
40 2�
31 2�
21 25
7� 77
5� 57
4� 68
4� 18
3� 85
3� 63
3� 46
3� 32
3� 22
3� 13
2� 99
2� 85
2� 70
2� 62
2� 54
2� 45
2� 36
2� 27
2� 17
26 7�
72 5�
53 4�
64 4�
14 3�
82 3�
59 3�
42 3�
29 3�
18 3�
09 2�
96 2�
81 2�
66 2�
58 2�
50 2�
42 2�
33 2�
23 2�
18 27
7� 68
5� 49
4� 60
4� 1 1
3� 78
3� 56
3� 39
3� 26
3� 15
3� 06
2� 93
2� 78
2� 63
2� 55
2� 47
2� 38
2� 29
2� 20
2� 10
28 7�
64 5�
45 4�
57 4�
07 3�
75 3�
53 3�
36 3�
23 3�
12 3�
03 2�
90 2�
75 2�
60 2�
52 2�
44 2�
35 2�
26 2�
17 2�
06 29
7� 60
5� 42
4� 54
4� 04
3� 73
3� 50
3� 33
3� 20
3� 09
3� 00
2� 87
2� 73
2� 57
2� 49
2� 41
2� 33
2� 23
2� 14
2� 03
30 7�
56 5�
39 4�
51 4�
02 3�
70 3�
47 3�
30 3�
17 3�
07 2�
98 2�
84 2�
70 2�
55 2�
47 2�
39 2�
30 2�
21 2�
1 1 2�
01 40
7� 31
5� 18
4� 31
3� 83
3� 51
3� 29
3� 12
2� 99
2� 89
2� 80
2� 66
2� 52
2� 37
2� 29
2� 20
2� 1 1
2� 02
1� 92
1� 80
60 7�
08 4�
98 4�
13 3�
65 3�
34 3�
12 2�
95 2�
82 2�
72 2�
63 2�
50 2�
35 2�
20 2�
12 2�
03 1�
94 1�
84 1�
73 1�
60 12
0 6�
85 4�
79 3�
95 3�
48 3�
17 2�
96 2�
79 2�
66 2�
56 2�
47 2�
34 2�
19 2�
03 1�
95 1�
86 1�
76 1�
66 1�
53 1�
38 ∞
6� 63
4� 61
3� 78
3� 32
3� 02
2� 80
2� 64
2� 51
2� 41
2� 32
2� 18
2� 04
1� 88
1� 79
1� 70
1� 59
1� 47
1� 32
1� 00
S ou
r c e:
� R
ep ri
n te
d � f
ro m
� P ea
rs o
n ,�
E �S
�� an
d � H
ar tl
ey ,�
H �O
�,� B
io m
et ri
ka T
ab le
s fo
r S
ta ti
st ic
ia n
s, � C
am b
ri d
g e�
U n
iv er
si ty
� P re
ss ,�
C am
b ri
d g
e, � U
�K �,�
19 66
,� Ta
b le
� 1 8�
� W it
h �
p er
m is
si o
n �o
f� B
io m
et ri
k a�
T r u
st ee
s� v 1
is �t
h e�
n u
m er
at o
r� d
eg re
es �o
f� fr
ee d
o m
,�a n
d �v
2� is
�t h
e� d
en o
m in
at o
r� d
eg re
es �o
f� fr
ee d
o m
�
766 Appendix: Tables
Table a.5
Fisher’s�Z�Transformed�Values
r Z r Z
�00 �0000 �50 �5493 1 �0100 1 �5627 2 �0200 2 �5763 3 �0300 3 �5901 4 �0400 4 �6042
�05 �0500 �55 �6184 6 �0601 6 �6328 7 �0701 7 �6475 8 �0802 8 �6625 9 �0902 9 �6777
�10 �1003 �60 �6931 1 �1104 1 �7089 2 �1206 2 �7250 3 �1307 3 �7414 4 �1409 4 �7582
�15 �1511 �65 �7753 6 �1614 6 �7928 7 �1717 7 �8107 8 �1820 8 �8291 9 �1923 9 �8480
�20 �2027 �70 �8673 1 �2132 1 �8872 2 �2237 2 �9076 3 �2342 3 �9287 4 �2448 4 �9505
�25 �2554 �75 0�973 6 �2661 6 0�996 7 �2769 7 1�020 8 �2877 8 1�045 9 �2986 9 1�071
�30 �3095 �80 1�099 1 �3205 1 1�127 2 �3316 2 1�157 3 �3428 3 1�188 4 �3541 4 1�221
�35 �3654 �85 1�256 6 �3769 6 1�293 7 �3884 7 1�333 8 �4001 8 1�376 9 �4118 9 1�422
767Appendix: Tables
Table a.5 (continued)
Fisher’s�Z�Transformed�Values
r Z r Z
�40 �4236 �90 1�472 1 �4356 1 1�528 2 �4477 2 1�589 3 �4599 3 1�658 4 �4722 4 1�738
�45 �4847 �95 1�832 6 �4973 6 1�946 7 �5101 7 2�092 8 �5230 8 2�298 9 �5361 9 2�647
Source:� Reprinted� from� Pearson,� E�S�� and� Hartley,� H�O�,� Biometrika Tables for Statisticians,� Cambridge� University� Press,� Cambridge,�U�K�,�1966,�Table�14�� With� permission� of� Biometrika� Trustees�
768 Appendix: Tables
Table a.6
Orthogonal�Polynomials
J Trend j = 1 2 3 4 5 6 7 8 9 10 Σcj2
J�=�3 Linear −1 0 1 2 Quadratic 1 −2 1 6
J�=�4 Linear −3 −1 1 3 20 Quadratic 1 −1 −1 1 4 Cubic −1 3 −3 1 20
J�=�5 Linear −2 −1 0 1 2 10 Quadratic 2 −1 −2 −1 2 14 Cubic −1 2 0 −2 1 10 Quartic 1 −4 6 −4 1 70
J�=�6 Linear −5 −3 −1 1 3 5 70 Quadratic 5 −1 −4 −4 −1 5 84 Cubic −5 7 4 −4 −7 5 180 Quartic 1 −3 2 2 −3 1 28 Quintic −1 5 −10 10 −5 1 252
J�=�7 Linear −3 −2 −1 0 1 2 3 28 Quadratic 5 0 −3 −4 −3 0 5 84 Cubic −1 1 1 0 −1 −1 1 6 Quartic 3 −7 1 6 1 −7 3 154 Quintic −1 4 −5 0 5 −4 1 84
J�=�8 Linear −7 −5 −3 −1 1 3 5 7 168 Quadratic 7 1 −3 −5 −5 −3 1 7 168 Cubic −7 5 7 3 −3 −7 −5 7 264 Quartic 7 −13 −3 9 9 −3 −13 7 616 Quintic −7 23 −17 −15 15 17 −23 7 2184
J�=�9 Linear −4 −3 −2 −1 0 1 2 3 4 60 Quadratic 28 7 −8 −17 −20 −17 −8 7 28 2772 Cubic −14 7 13 9 0 −9 −13 −7 14 990 Quartic 14 −21 −11 9 18 9 −11 −21 14 2002 Quintic −4 11 −4 −9 0 9 4 −11 4 468
J�=�10 Linear −9 −7 −5 −3 −1 1 3 5 7 9 330 Quadratic 6 2 −1 −3 −4 −4 −3 −1 2 6 132 Cubic −42 14 35 31 12 −12 −31 −35 −14 42 8580 Quartic 18 −22 −17 3 18 18 3 −17 −22 18 2860 Quintic −6 14 −1 −11 −6 6 11 1 −14 6 780
Source:� Reprinted� from� Pearson,� E�S�� and� Hartley,� H�O�,� Biometrika Tables for Statisticians,� Cambridge� University�Press,�Cambridge,�U�K�,�1966,�Table�47��With�permission�of�Biometrika�Trustees�
769Appendix: Tables
Table a.7
Critical�Values�for�Dunnett’s�Procedure
df 1 2 3 4 5 6 7 8 9
One tailed,�α�= .05 5 2�02 2�44 2�68 2�85 2�98 3�08 3�16 3�24 3�30 6 1�94 2�34 2�56 2�71 2�83 2�92 3�00 3�07 3�12 7 1�89 2�27 2�48 2�62 2�73 2�82 2�89 2�95 3�01 8 1�86 2�22 2�42 2�55 2�66 2�74 2�81 2�87 2�92 9 1�83 2�18 2�37 2�50 2�60 2�68 2�75 2�81 2�86
10 1�81 2�15 2�34 2�47 2�56 2�64 2�70 2�76 2�81 11 1�80 2�13 2�31 2�44 2�53 2�60 2�67 2�72 2�77 12 1�78 2�11 2�29 2�41 2�50 2�58 2�64 2�69 2�74 13 1�77 2�09 2�27 2�39 2�48 2�55 2�61 2�66 2�71 14 1�76 2�08 2�25 2�37 2�46 2�53 2�59 2�64 2�69 15 1�75 2�07 2�24 2�36 2�44 2�51 2�57 2�62 2�67 16 1�75 2�06 2�23 2�34 2�43 2�50 2�56 2�61 2�65 17 1�74 2�05 2�22 2�33 2�42 2�49 2�54 2�59 2�64 18 1�73 2�04 2�21 2�32 2�41 2�48 2�53 2�58 2�62 19 1�73 2�03 2�20 2�31 2�40 2�47 2�52 2�57 2�61 20 1�72 2�03 2�19 2�30 2�39 2�46 2�51 2�56 2�60 24 1�71 2�01 2�17 2�28 2�36 2�43 2�48 2�53 2�57 30 1�70 1�99 2�15 2�25 2�33 2�40 2�45 2�50 2�54 40 1�68 1�97 2�13 2�23 2�31 2�37 2�42 2�47 2�51 60 1�67 1�95 2�10 2�21 2�28 2�35 2�39 2�44 2�48
120 1�66 1�93 2�08 2�18 2�26 2�32 2�37 2�41 2�45 ∞ 1�64 1�92 2�06 2�16 2�23 2�29 2�34 2�38 2�42
One tailed,�α�= .01 5 3�37 3�90 4�21 4�43 4�60 4�73 4�85 4�94 5�03 6 3�14 3�61 3�88 4�07 4�21 4�33 4�43 4�51 4�59 7 3�00 3�42 3�66 3�83 3�96 4�07 4�15 4�23 4�30 8 2�90 3�29 3�51 3�67 3�79 3�88 3�96 4�03 4�09 9 2�82 3�19 3�40 3�55 3�66 3�75 3�82 3�89 3�94
10 2�76 3�11 3�31 3�45 3�56 3�64 3�71 3�78 3�83 11 2�72 3�06 3�25 3�38 3�48 3�56 3�63 3�69 3�74 12 2�68 3�01 3�19 3�32 3�42 3�50 3�56 3�62 3�67 13 2�65 2�97 3�15 3�27 3�37 3�44 3�51 3�56 3�61 14 2�62 2�94 3�11 3�23 3�32 3�40 3�46 3�51 3�56 15 2�60 2�91 3�08 3�20 3�29 3�36 3�42 3�47 3�52 16 2�58 2�88 3�05 3�17 3�26 3�33 3�39 3�44 3�48 17 2�57 2�86 3�03 3�14 3�23 3�30 3�36 3�41 3�45 18 2�55 2�84 3�01 3�12 3�21 3�27 3�33 3�38 3�42 19 2�54 2�83 2�99 3�10 3�18 3�25 3�31 3�36 3�40 20 2�53 2�81 2�97 3�08 3�17 3�23 3�29 3�34 3�38 24 2�49 2�77 2�92 3�03 3�11 3�17 3�22 3�27 3�31
(continued)
770 Appendix: Tables
Table a.7 (continued)
Critical�Values�for�Dunnett’s�Procedure
df 1 2 3 4 5 6 7 8 9
One tailed,�α�= .01
30 2�46 2�72 2�87 2�97 3�05 3�11 3�16 3�21 3�24 40 2�42 2�68 2�82 2�92 2�99 3�05 3�10 3�14 3�18 60 2�39 2�64 2�78 2�87 2�94 3�00 3�04 3�08 3�12
120 2�36 2�60 2�73 2�82 2�89 2�94 2�99 3�03 3�06 ∞ 2�33 2�56 2�68 2�77 2�84 2�89 2�93 2�97 3�00
Two tailed,�α�= .05 5 2�57 3�03 3�29 3�48 3�62 3�73 3�82 3�90 3�97 6 2�45 2�86 3�10 3�26 3�39 3�49 3�57 3�64 3�71 7 2�36 2�75 2�97 3�12 3�24 3�33 3�41 3�47 3�53 8 2�31 2�67 2�88 3�02 3�13 3�22 3�29 3�35 3�41 9 2�26 2�61 2�81 2�95 3�05 3�14 3�20 3�26 3�32
10 2�23 2�57 2�76 2�89 2�99 3�07 3�14 3�19 3�24 11 2�20 2�53 2�72 2�84 2�94 3�02 3�08 3�14 3�19 12 2�18 2�50 2�68 2�81 2�90 2�98 3�04 3�09 3�14 13 2�16 2�48 2�65 2�78 2�87 2�94 3�00 3�06 3�10 14 2�14 2�46 2�63 2�75 2�84 2�91 2�97 3�02 3�07 15 2�13 2�44 2�61 2�73 2�82 2�89 2�95 3�00 3�04 16 2�12 2�42 2�59 2�71 2�80 2�87 2�92 2�97 3�02 17 2�11 2�41 2�58 2�69 2�78 2�85 2�90 2�95 3�00 18 2�10 2�40 2�56 2�68 2�76 2�83 2�89 2�94 2�98 19 2�09 2�39 2�55 2�66 2�75 2�81 2�87 2�92 2�96 20 2�09 2�38 2�54 2�65 2�73 2�80 2�86 2�90 2�95 24 2�06 2�35 2�51 2�61 2�70 2�76 2�81 2�86 2�90 30 2�04 2�32 2�47 2�58 2�66 2�72 2�77 2�82 2�86 40 2�02 2�29 2�44 2�54 2�62 2�68 2�73 2�77 2�81 60 2�00 2�27 2�41 2�51 2�58 2�64 2�69 2�73 2�77
120 1�98 2�24 2�38 2�47 2�55 2�60 2�65 2�69 2�73 ∞ 1�96 2�21 2�35 2�44 2�51 2�57 2�61 2�65 2�69
Two tailed,�α�= .01 5 4�03 4�63 4�98 5�22 5�41 5�56 5�69 5�80 5�89 6 3�71 4�21 4�51 4�71 4�87 5�00 5�10 5�20 5�28 7 3�50 3�95 4�21 4�39 4�53 4�64 4�74 4�82 4�89 8 3�36 3�77 4�00 4�17 4�29 4�40 4�48 4�56 4�62 9 3�25 3�63 3�85 4�01 4�12 4�22 4�30 4�37 4�43
10 3�17 3�53 3�74 3�88 3�99 4�08 4�16 4�22 4�28 11 3�11 3�45 3�65 3�79 3�89 3�98 4�05 4�11 4�16 12 3�05 3�39 3�58 3�71 3�81 3�89 3�96 4�02 4�07 13 3�01 3�33 3�52 3�65 3�74 3�82 3�89 3�94 3�99 14 2�98 3�29 3�47 3�59 3�69 3�76 3�83 3�88 3�93 15 2�95 3�25 3�43 3�55 3�64 3�71 3�78 3�83 3�88 16 2�92 3�22 3�39 3�51 3�60 3�67 3�73 3�78 3�83
771Appendix: Tables
Table a.7 (continued)
Critical�Values�for�Dunnett’s�Procedure
df 1 2 3 4 5 6 7 8 9
Two tailed,�α�= .01
17 2�90 3�19 3�36 3�47 3�56 3�63 3�69 3�74 3�79 18 2�88 3�17 3�33 3�44 3�53 3�60 3�66 3�71 3�75 19 2�86 3�15 3�31 3�42 3�50 3�57 3�63 3�68 3�72 20 2�85 3�13 3�29 3�40 3�48 3�55 3�60 3�65 3�69 24 2�80 3�07 3�22 3�32 3�40 3�47 3�52 3�57 3�61 30 2�75 3�01 3�15 3�25 3�33 3�39 3�44 3�49 3�52 40 2�70 2�95 3�09 3�19 3�26 3�32 3�37 3�41 3�44 60 2�66 2�90 3�03 3�12 3�19 3�25 3�29 3�33 3�37
120 2�62 2�85 2�97 3�06 3�12 3�18 3�22 3�26 3�29 ∞ 2�58 2�79 2�92 3�00 3�06 3�11 3�15 3�19 3�22
Sources:� Reprinted� from� Dunnett,� C�W�,� J. Am. Stat. Assoc�,� 50,� 1096,� 1955,� Table� 1a� and� Table� 1b�� With� permission� of� the� American� Statistical� Association;� Dunnett,�C�W�,�Biometrics,�20,�482,�1964,�Table�II�and�Table�III��With�permis- sion�of�the�Biometric�Society�
The�columns�represent�J�=�number�of�treatment�means�(excluding�the�control)�
772 Appendix: Tables
Table a.8
Critical�Values�for�Dunn’s�(Bonferroni’s)�Procedure
Number of Contrasts
ν α 2 3 4 5 6 7 8 9 10 15 20
2 0�01 14�071 17�248 19�925 22�282 24�413 26�372 28�196 29�908 31�528 38�620 44�598 0�05 6�164 7�582 8�774 9�823 10�769 11�639 12�449 13�208 13�927 17�072 19�721 0�10 4�243 5�243 6�081 6�816 7�480 8�090 8�656 9�188 9�691 11�890 13�741 0�20 2�828 3�531 4�116 4�628 5�089 5�512 5�904 6�272 6�620 8�138 9�414
3 0�01 7�447 8�565 9�453 10�201 10�853 11�436 11�966 12�453 12�904 14�796 16�300 0�05 4�156 4�826 5�355 5�799 6�185 6�529 6�842 7�128 7�394 8�505 9�387 0�10 3�149 3�690 4�115 4�471 4�780 5�055 5�304 5�532 5�744 6�627 7�326 0�20 2�294 2�734 3�077 3�363 3�610 3�829 4�028 4�209 4�377 5�076 5�626
4 0�01 5�594 6�248 6�751 7�166 7�520 7�832 8�112 8�367 8�600 9�556 10�294 0�05 3�481 3�941 4�290 4�577 4�822 5�036 5�228 5�402 5�562 6�214 6�714 0�10 2�751 3�150 3�452 3�699 3�909 4�093 4�257 4�406 4�542 5�097 5�521 0�20 2�084 2�434 2�697 2�911 3�092 3�250 3�391 3�518 3�635 4�107 4�468
5 0�01 4�771 5�243 5�599 5�888 6�133 6�346 6�535 6�706 6�862 7�491 7�968 0�05 3�152 3�518 3�791 4�012 4�197 4�358 4�501 4�630 4�747 5�219 5�573 0�10 2�549 2�882 3�129 3�327 3�493 3�638 3�765 3�880 3�985 4�403 4�718 0�20 1�973 2�278 2�503 2�683 2�834 2�964 3�079 3�182 3�275 3�649 3�928
6 0�01 4�315 4�695 4�977 5�203 5�394 5�559 5�704 5�835 5�954 6�428 6�782 0�05 2�959 3�274 3�505 3�690 3�845 3�978 4�095 4�200 4�296 4�675 4�956 0�10 2�428 2�723 2�939 3�110 3�253 3�376 3�484 3�580 3�668 4�015 4�272 0�20 1�904 2�184 2�387 2�547 2�681 2�795 2�895 2�985 3�066 3�385 3�620
7 0�01 4�027 4�353 4�591 4�782 4�941 5�078 5�198 5�306 5�404 5�791 6�077 0�05 2�832 3�115 3�321 3�484 3�620 3�736 3�838 3�929 4�011 4�336 4�574 0�10 2�347 2�618 2�814 2�969 3�097 3�206 3�302 3�388 3�465 3�768 3�990 0�20 1�858 2�120 2�309 2�457 2�579 2�684 2�775 2�856 2�929 3�214 3�423
8 0�01 3�831 4�120 4�331 4�498 4�637 4�756 4�860 4�953 5�038 5�370 5�613 0�05 2�743 3�005 3�193 3�342 3�464 3�589 3�661 3�743 3�816 4�105 4�316 0�10 2�289 2�544 2�726 2�869 2�967 3�088 3�176 3�254 3�324 3�598 3�798 0�20 1�824 2�075 2�254 2�393 2�508 2�605 2�690 2�765 2�832 3�095 3�286
9 0�01 3�688 3�952 4�143 4�294 4�419 4�526 4�619 4�703 4�778 5�072 5�287 0�05 2�677 2�923 3�099 3�237 3�351 3�448 3�532 3�607 3�675 3�938 4�129 0�10 2�246 2�488 2�661 2�796 2�907 3�001 3�083 3�155 3�221 3�474 3�658 0�20 1�799 2�041 2�212 2�345 2�454 2�546 2�627 2�696 2�761 3�008 3�185
10 0�01 3�580 3�825 4�002 4�141 4�256 4�354 4�439 4�515 4�584 4�852 5�046 0�05 2�626 2�860 3�027 3�157 3�264 3�355 3�434 3�505 3�568 3�813 3�989 0�10 2�213 2�446 2�611 2�739 2�845 2�934 3�012 3�080 3�142 3�380 3�552 0�20 1�779 2�014 2�180 2�308 2�413 2�501 2�578 2�646 2�706 2�941 3�106
11 0�01 3�495 3�726 3�892 4�022 4�129 4�221 4�300 4�371 4�434 4�682 4�860 0�05 2�586 2�811 2�970 3�094 3�196 3�283 3�358 3�424 3�484 3�715 3�880 0�10 2�166 2�412 2�571 2�695 2�796 2�881 2�955 3�021 3�079 3�306 3�468 0�20 1�763 1�993 2�154 2�279 2�380 2�465 2�539 2�605 2�663 2�888 3�048
12 0�01 3�427 3�647 3�804 3�927 4�029 4�114 4�189 4�256 4�315 4�547 4�714 0�05 2�553 2�770 2�924 3�044 3�141 3�224 3�296 3�359 3�416 3�636 3�793 0�10 2�164 2�384 2�539 2�658 2�756 2�838 2�910 2�973 3�029 3�247 3�402 0�20 1�750 1�975 2�133 2�254 2�353 2�436 2�508 2�571 2�628 2�845 2�999
773Appendix: Tables
Table a.8 (continued)
Critical�Values�for�Dunn’s�(Bonferroni’s)�Procedure
Number of Contrasts
ν α 2 3 4 5 6 7 8 9 10 15 20
13 0�01 3�371 3�582 3�733 3�850 3�946 4�028 4�099 4�162 4�218 4�438 4�595 0�05 2�526 2�737 2�886 3�002 3�096 3�176 3�245 3�306 3�361 3�571 3�722 0�10 2�146 2�361 2�512 2�628 2�723 2�803 2�872 2�933 2�988 3�198 3�347 0�20 1�739 1�961 2�116 2�234 2�331 2�412 2�482 2�544 2�599 2�809 2�958
14 0�01 3�324 3�528 3�673 3�785 3�878 3�956 4�024 4�084 4�138 4�347 4�497 0�05 2�503 2�709 2�854 2�967 3�058 3�135 3�202 3�261 3�314 3�518 3�662 0�10 2�131 2�342 2�489 2�603 2�696 2�774 2�841 2�900 2�953 3�157 3�301 0�20 1�730 1�949 2�101 2�217 2�312 2�392 2�460 2�520 2�574 2�779 2�924
15 0�01 3�285 3�482 3�622 3�731 3�820 3�895 3�961 4�019 4�071 4�271 4�414 0�05 2�483 2�685 2�827 2�937 3�026 3�101 3�166 3�224 3�275 3�472 3�612 0�10 2�118 2�325 2�470 2�582 2�672 2�748 2�814 2�872 2�924 3�122 3�262 0�20 1�722 1�938 2�088 2�203 2�296 2�374 2�441 2�500 2�553 2�754 2�896
16 0�01 3�251 3�443 3�579 3�684 3�771 3�844 3�907 3�963 4�013 4�206 4�344 0�05 2�467 2�665 2�804 2�911 2�998 3�072 3�135 3�191 3�241 3�433 3�569 0�10 2�106 2�311 2�453 2�563 2�652 2�726 2�791 2�848 2�898 3�092 3�228 0�20 1�715 1�929 2�077 2�190 2�282 2�359 2�425 2�483 2�535 2�732 2�871
17 0�01 3�221 3�409 3�541 3�644 3�728 3�799 3�860 3�914 3�963 4�150 4�284 0�05 2�452 2�647 2�783 2�889 2�974 3�046 3�108 3�163 3�212 3�399 3�532 0�10 2�096 2�296 2�439 2�547 2�634 2�706 2�771 2�826 2�876 3�066 3�199 0�20 1�709 1�921 2�068 2�179 2�270 2�346 2�411 2�488 2�519 2�713 2�849
18 0�01 3�195 3�379 3�508 3�609 3�691 3�760 3�820 3�872 3�920 4�102 4�231 0�05 2�439 2�631 2�766 2�869 2�953 3�024 3�085 3�138 3�186 3�370 3�499 0�10 2�088 2�287 2�426 2�532 2�619 2�691 2�753 2�806 2�857 3�043 3�174 0�20 1�704 1�914 2�059 2�170 2�259 2�334 2�399 2�455 2�505 2�696 2�830
19 0�01 3�173 3�353 3�479 3�578 3�658 3�725 3�784 3�835 3�881 4�059 4�185 0�05 2�427 2�617 2�750 2�852 2�934 3�004 3�064 3�116 3�163 3�343 3�470 0�10 2�080 2�277 2�415 2�520 2�605 2�676 2�738 2�791 2�839 3�023 3�152 0�20 1�699 1�908 2�052 2�161 2�250 2�324 2�388 2�443 2�493 2�682 2�813
20 0�01 3�152 3�329 3�454 3�550 3�629 3�695 3�752 3�802 3�848 4�021 4�144 0�05 2�417 2�605 2�736 2�836 2�918 2�986 3�045 3�097 3�143 3�320 3�445 0�10 2�073 2�269 2�405 2�508 2�593 2�663 2�724 2�777 2�824 3�005 3�132 0�20 1�695 1�902 2�045 2�154 2�241 2�315 2�378 2�433 2�482 2�668 2�798
21 0�01 3�134 3�308 3�431 3�525 3�602 3�667 3�724 3�773 3�817 3�987 4�108 0�05 2�408 2�594 2�723 2�822 2�903 2�970 3�028 3�080 3�125 3�300 3�422 0�10 2�067 2�261 2�396 2�498 2�581 2�651 2�711 2�764 2�810 2�989 3�114 0�20 1�691 1�897 2�039 2�147 2�234 2�306 2�369 2�424 2�472 2�656 2�785
22 0�01 3�118 3�289 3�410 3�503 3�579 3�643 3�698 3�747 3�790 3�957 4�075 0�05 2�400 2�584 2�712 2�810 2�889 2�956 3�014 3�064 3�109 3�281 3�402 0�10 2�061 2�254 2�387 2�489 2�572 2�641 2�700 2�752 2�798 2�974 3�096 0�20 1�688 1�892 2�033 2�141 2�227 2�299 2�361 2�415 2�463 2�646 2�773
(continued)
774 Appendix: Tables
Table a.8 (continued)
Critical�Values�for�Dunn’s�(Bonferroni’s)�Procedure
Number of Contrasts
ν α 2 3 4 5 6 7 8 9 10 15 20
23 0�01 3�103 3�272 3�392 3�483 3�558 3�621 3�675 3�723 3�766 3�930 4�046 0�05 2�392 2�574 2�701 2�798 2�877 2�943 3�000 3�050 3�094 3�264 3�383 0�10 2�056 2�247 2�380 2�481 2�563 2�631 2�690 2�741 2�787 2�961 3�083 0�20 1�685 1�888 2�028 2�135 2�221 2�292 2�354 2�407 2�455 2�636 2�762
24 0�01 3�089 3�257 3�375 3�465 3�539 3�601 3�654 3�702 3�744 3�905 4�019 0�05 2�385 2�566 2�692 2�788 2�866 2�931 2�988 3�037 3�081 3�249 3�366 0�10 2�051 2�241 2�373 2�473 2�554 2�622 2�680 2�731 2�777 2�949 3�070 0�20 1�682 1�884 2�024 2�130 2�215 2�286 2�347 2�400 2�448 2�627 2�752
25 0�01 3�077 3�243 3�359 3�449 3�521 3�583 3�635 3�682 3�723 3�882 3�995 0�05 2�379 2�558 2�683 2�779 2�856 2�921 2�976 3�025 3�069 3�235 3�351 0�10 2�047 2�236 2�367 2�466 2�547 2�614 2�672 2�722 2�767 2�938 3�058 0�20 1�679 1�881 2�020 2�125 2�210 2�280 2�341 2�394 2�441 2�619 2�743
26 0�01 3�066 3�230 3�345 3�433 3�505 3�566 3�618 3�664 3�705 3�862 3�972 0�05 2�373 2�551 2�675 2�770 2�847 2�911 2�966 3�014 3�058 3�222 3�337 0�10 2�043 2�231 2�361 2�460 2�540 2�607 2�664 2�714 2�759 2�928 3�047 0�20 1�677 1�878 2�016 2�121 2�205 2�275 2�335 2�388 2�435 2�612 2�735
27 0�01 3�056 3�218 3�332 3�419 3�491 3�550 3�602 3�647 3�688 3�843 3�952 0�05 2�368 2�545 2�668 2�762 2�838 2�902 2�956 3�004 3�047 3�210 3�324 0�10 2�039 2�227 2�356 2�454 2�534 2�600 2�657 2�707 2�751 2�919 3�036 0�20 1�675 1�875 2�012 2�117 2�201 2�270 2�330 2�383 2�429 2�605 2�727
28 0�01 3�046 3�207 3�320 3�407 3�477 3�536 3�587 3�632 3�672 3�825 3�933 0�05 2�383 2�539 2�661 2�755 2�830 2�893 2�948 2�995 3�038 3�199 3�312 0�10 2�036 2�222 2�351 2�449 2�528 2�594 2�650 2�700 2�744 2�911 3�027 0�20 1�672 1�872 2�009 2�113 2�196 2�266 2�326 2�378 2�424 2�599 2�720
29 0�01 3�037 3�197 3�309 3�395 3�464 3�523 3�574 3�618 3�658 3�809 3�916 0�05 2�358 2�534 2�655 2�748 2�823 2�886 2�940 2�967 3�029 3�189 3�301 0�10 2�033 2�218 2�346 2�444 2�522 2�588 2�644 2�693 2�737 2�903 3�018 0�20 1�671 1�869 2�006 2�110 2�193 2�262 2�321 2�373 2�419 2�593 2�713
30 0�01 3�029 3�188 3�298 3�384 3�453 3�511 3�561 3�605 3�644 3�794 3�900 0�05 2�354 2�528 2�649 2�742 2�816 2�878 2�932 2�979 3�021 3�180 3�291 0�10 2�030 2�215 2�342 2�439 2�517 2�582 2�638 2�687 2�731 2�895 3�010 0�20 1�669 1�867 2�003 2�106 2�189 2�258 2�317 2�369 2�414 2�587 2�707
40 0�01 2�970 3�121 3�225 3�305 3�370 3�425 3�472 3�513 3�549 3�689 3�787 0�05 2�323 2�492 2�606 2�696 2�768 2�827 2�878 2�923 2�963 3�113 3�218 0�10 2�009 2�189 2�312 2�406 2�481 2�544 2�597 2�644 2�686 2�843 2�952 0�20 1�656 1�850 1�983 2�083 2�164 2�231 2�288 2�338 2�382 2�548 2�663
60 0�01 2�914 3�056 3�155 3�230 3�291 3�342 3�386 3�425 3�459 3�589 3�679 0�05 2�294 2�456 2�568 2�653 2�721 2�777 2�826 2�869 2�906 3�049 3�146 0�10 1�989 2�163 2�283 2�373 2�446 2�506 2�558 2�603 2�643 2�793 2�897 0�20 1�643 1�834 1�963 2�061 2�139 2�204 2�259 2�308 2�350 2�511 2�621
775Appendix: Tables
Table a.8 (continued)
Critical�Values�for�Dunn’s�(Bonferroni’s)�Procedure
Number of Contrasts
ν α 2 3 4 5 6 7 8 9 10 15 20
120 0�01 2�859 2�994 3�067 3�158 3�215 3�263 3�304 3�340 3�372 3�493 3�577 0�05 2�265 2�422 2�529 2�610 2�675 2�729 2�776 2�816 2�852 2�967 3�081 0�10 1�968 2�138 2�254 2�342 2�411 2�469 2�519 2�562 2�600 2�744 2�843 0�20 1�631 1�817 1�944 2�039 2�115 2�178 2�231 2�278 2�319 2�474 2�580
∞ 0�01 2�806 2�934 3�022 3�089 3�143 3�188 3�226 3�260 3�289 3�402 3�480 0�05 2�237 2�388 2�491 2�569 2�631 2�683 2�727 2�766 2�800 2�928 3�016 0�10 1�949 2�114 2�226 2�311 2�378 2�434 2�482 2�523 2�560 2�697 2�791 0�20 1�618 1�801 1�925 2�018 2�091 2�152 2�204 2�249 2�289 2�438 2�540
Source:� Reprinted�from�Games,�P�A�,�J. Am. Stat. Asso�,�72,�531,�1977,�Table�1��With�permission�of�the�American� Statistical�Association�
776 Appendix: Tables
Table a.9
Critical�Values�for�the�Studentized�Range�Statistic
J or r
v 2 3 4 5 6 7 8 9 10
α = .10 1 8�929 13�44 16�36 18�49 20�15 21�51 22�64 23�62 24�48 2 4�130 5�733 6�773 7�538 8�139 8�633 9�049 9�409 9�725 3 3�328 4�467 5�199 5�738 6�162 6�511 6�806 7�062 7�287 4 3�015 3�976 4�586 5�035 5�388 5�679 5�926 6�139 6�327 5 2�850 3�717 4�264 4�664 4�979 5�238 5�458 5�648 5�816 6 2�748 3�559 4�065 4�435 4�726 4�966 5�168 5�344 5�499 7 2�680 3�451 3�931 4�280 4�555 4�780 4�972 5�137 5�283 8 2�630 3�374 3�834 4�169 4�431 4�646 4�829 4�987 5�126 9 2�592 3�316 3�761 4�084 4�337 4�545 4�721 4�873 5�007
10 2�563 3�270 3�704 4�018 4�264 4�465 4�636 4�783 4�913 11 2�540 3�234 3�658 3�965 4�205 4�401 4�568 4�711 4�838 12 2�521 3�204 3�621 3�922 4�156 4�349 4�511 4�652 4�776 13 2�505 3�179 3�589 3�885 4�116 4�305 4�464 4�602 4�724 14 2�491 3�158 3�563 3�854 4�081 4�267 4�424 4�560 4�680 15 2�479 3�140 3�540 3�828 4�052 4�235 4�390 4�524 4�641 16 2�469 3�124 3�520 3�804 4�026 4�207 4�360 4�492 4�608 17 2�460 3�110 3�503 3�784 4�004 4�183 4�334 4�464 4�579 18 2�452 3�098 3�488 3�767 3�984 4�161 4�311 4�440 4�554 19 2�445 3�087 3�474 3�751 3�966 4�142 4�290 4�418 4�531 20 2�439 3�078 3�462 3�736 3�950 4�124 4�271 4�398 4�510 24 2�420 3�047 3�423 3�692 3�900 4�070 4�213 4�336 4�445 30 2�400 3�017 3�386 3�648 3�851 4�016 4�155 4�275 4�381 40 2�381 2�988 3�349 3�605 3�803 3�963 4�099 4�215 4�317 60 2�363 2�959 3�312 3�562 3�755 3�911 4�042 4�155 4�254
120 2�344 2�930 3�276 3�520 3�707 3�859 3�987 4�096 4�191 ∞ 2�326 2�902 3�240 3�478 3�661 3�808 3�931 4�037 4�129
v
J or r
11 12 13 14 15 16 17 18 19
α�= .10
1 25�24 25�92 26�54 27�10 27�62 28�10 28�54 28�96 29�35 2 10�01 10�26 10�49 10�70 10�89 11�07 11�24 11�39 11�54 3 7�487 7�667 7�832 7�982 8�120 8�249 8�368 8�479 8�584 4 6�495 6�645 6�783 6�909 7�025 7�133 7�233 7�327 7�414 5 5�966 6�101 6�223 6�336 6�440 6�536 6�626 6�710 6�789 6 5�637 5�762 5�875 5�979 6�075 6�164 6�247 6�325 6�398 7 5�413 5�530 5�637 5�735 5�826 5�910 5�838 6�061 6�130 8 5�250 5�362 5�464 5�558 5�644 5�724 5�799 5�869 5�935 9 5�127 5�234 5�333 5�423 5�506 5�583 5�655 5�723 5�786
10 5�029 5�134 5�229 5�317 5�397 5�472 5�542 5�607 5�668 11 4�951 5�053 5�146 5�231 5�309 5�382 5�450 5�514 5�573 12 4�886 4�986 5�077 5�160 5�236 5�308 5�374 5�436 5�495 13 4�832 4�930 5�019 5�100 5�176 5�245 5�311 5�372 5�429
777Appendix: Tables
Table a.9 (continued)
Critical�Values�for�the�Studentized�Range�Statistic
J or r
v 11 12 13 14 15 16 17 18 19
α�= .10 14 4�786 4�882 4�970 5�050 5�124 5�192 5�256 5�316 5�373
15 4�746 4�841 4�927 5�006 5�079 5�147 5�209 5�269 5�324 16 4�712 4�805 4�890 4�968 5�040 5�107 5�169 5�227 5�282 17 4�682 4�774 4�858 4�935 5�005 5�071 5�133 5�190 5�244 18 4�655 4�746 4�829 4�905 4�975 5�040 5�101 5�158 5�211 19 4�631 4�721 4�803 4�879 4�948 5�012 5�073 5�129 5�182 20 4�609 4�699 4�780 4�855 4�924 4�987 5�047 5�103 5�155 24 4�541 4�628 4�708 4�780 4�847 4�909 4�966 5�021 5�071 30 4�474 4�559 4�635 4�706 4�770 4�830 4�886 4�939 4�988 40 4�408 4�490 4�564 4�632 4�695 4�752 4�807 4�857 4�905 60 4�342 4�421 4�493 4�558 4�619 4�675 4�727 4�775 4�821
120 4�276 4�353 4�422 4�485 4�543 4�597 4�647 4�694 4�738 ∞ 4�211 4�285 4�351 4�412 4�468 4�519 4�568 4�612 4�654
v
J or r
2 3 4 5 6 7 8 9 10
α = .05 1 17�97 26�98 32�82 37�08 40�41 43�12 45�40 47�36 49�07 2 6�085 8�331 9�798 10�88 11�74 12�44 13�03 13�54 13�99 3 4�501 5�910 6�825 7�502 8�037 8�478 8�853 9�177 9�462 4 3�927 5�040 5�757 6�287 6�707 7�053 7�347 7�602 7�826 5 3�635 4�602 5�218 5�673 6�033 6�330 6�582 6�802 6�995 6 3�461 4�339 4�896 5�305 5�628 5�895 6�122 6�319 6�493 7 3�344 4�165 4�681 5�060 5�359 5�606 5�815 5�998 6�158 8 3�261 4�041 4�529 4�886 5�167 5�399 5�597 5�767 5�918 9 3�199 3�949 4�415 4�756 5�024 5�244 5�432 5�595 5�739
10 3�151 3�877 4�327 4�654 4�912 5�124 5�305 5�461 5�599 11 3�113 3�820 4�256 4�574 4�823 5�028 5�202 5�353 5�487 12 3�082 3�773 4�199 4�508 4�751 4�950 5�119 5�265 5�395 13 3�055 3�735 4�151 4�453 4�690 4�885 5�049 5�192 5�318 14 3�033 3�702 4�111 4�407 4�639 4�829 4�990 5�131 5�254 15 3�014 3�674 4�076 4�367 4�595 4�782 4�940 5�077 5�198 16 2�998 3�649 4�046 4�333 4�557 4�741 4�897 5�031 5�150 17 2�984 3�628 4�020 4�303 4�524 4�705 4�858 4�991 5�108 18 2�971 3�609 3�997 4�277 4�495 4�673 4�824 4�956 5�071 19 2�960 3�593 3�977 4�253 4�469 4�645 4�794 4�924 5�038 20 2�950 3�578 3�958 4�232 4�445 4�620 4�768 4�896 5�008 24 2�919 3�532 3�901 4�166 4�373 4�541 4�684 4�807 4�915 30 2�888 3�486 3�845 4�102 4�302 4�464 4�602 4�720 4�824 40 2�858 3�442 3�791 4�039 4�232 4�389 4�521 4�635 4�735 60 2�829 3�399 3�737 3�977 4�163 4�314 4�441 4�550 4�646
120 2�800 3�356 3�685 3�917 4�096 4�241 4�363 4�468 4�560 ∞ 2�772 3�314 3�633 3�858 4�030 4�170 4�286 4�387 4�474
(continued)
778 Appendix: Tables
Table a.9 (continued)
Critical�Values�for�the�Studentized�Range�Statistic
J or r
v 11 12 13 14 15 16 17 18 19
α�= .05
1 50�59 51�96 53�20 54�33 55�36 56�32 57�22 58�04 58�83 2 14�39 14�75 15�08 15�38 15�65 15�91 16�14 16�37 16�57 3 9�717 9�946 10�15 10�35 10�53 10�69 10�84 10�98 11�11 4 8�027 8�208 8�373 8�525 8�664 8�794 8�914 9�028 9�134 5 7�168 7�324 7�466 7�596 7�717 7�828 7�932 8�030 8�122 6 6�649 6�789 6�917 7�034 7�143 7�244 7�338 7�426 7�508 7 6�302 6�431 6�550 6�658 6�759 6�852 6�939 7�020 7�097 8 6�054 6�175 6�287 6�389 6�483 6�571 6�653 6�729 6�802 9 5�867 5�983 6�089 6�186 6�276 6�359 6�437 6�510 6�579
10 5�722 5�833 5�935 6�028 6�114 6�194 6�269 6�339 6�405 11 5�605 5�713 5�811 5�901 5�984 6�062 6�134 6�202 6�265 12 5�511 5�615 5�710 5�798 5�878 5�953 6�023 6�089 6�151 13 5�431 5�533 5�625 5�711 5�789 5�862 5�931 5�995 6�055 14 5�364 5�463 5�554 5�637 5�714 5�786 5�852 5�915 5�974 15 5�306 5�404 5�493 5�574 5�649 5�720 5�785 5�846 5�904 16 5�256 5�352 5�439 5�520 5�593 5�662 5�720 5�786 5�843 17 5�212 5�307 5�392 5�471 5�544 5�612 5�675 5�734 5�790 18 5�174 5�267 5�352 5�429 5�501 5�568 5�630 5�688 5�743 19 5�140 5�231 5�315 5�391 5�462 5�528 5�589 5�647 5�701 20 5�108 5�199 5�282 5�357 5�427 5�493 5�553 5�610 5�663 24 5�012 5�099 5�179 5�251 5�319 5�381 5�439 5�494 5�545 30 4�917 5�001 5�077 5�147 5�211 5�271 5�327 5�379 5�429 40 4�824 4�904 4�977 5�044 5�106 5�163 5�216 5�266 5�313 60 4�732 4�808 4�878 4�942 5�001 5�056 5�107 5�154 5�199
120 4�641 4�714 4�781 4�842 4�898 4�950 4�998 5�044 5�086 ∞ 4�552 4�622 4�685 4�743 4�796 4�845 4�891 4�934 4�974
J or r
v 2 3 4 5 6 7 8 9 10
α�= .01 1 90�03 135�0 164�3 185�6 202�2 215�8 227�2 237�0 245�6 2 14�04 19�02 22�29 24�72 26�63 28�20 29�53 30�68 31�69 3 8�261 10�62 12�17 13�33 14�24 15�00 15�64 16�20 16�69 4 6�512 8�120 9�173 9�958 10�58 11�10 11�55 11�93 12�27 5 5�702 6�976 7�804 8�421 8�913 9�321 9�669 9�972 10�24 6 5�243 6�331 7�033 7�556 7�973 8�318 8�613 8�869 9�097 7 4�949 5�919 6�543 7�005 7�373 7�679 7�939 8�166 8�368 8 4�746 5�635 6�204 6�625 6�960 7�237 7�474 7�681 7�863 9 4�596 5�428 5�957 6�348 6�658 6�915 7�134 7�325 7�495
10 4�482 5�270 5�769 6�136 6�428 6�669 6�875 7�055 7�213 11 4�392 5�146 5�621 5�970 6�247 6�476 6�672 6�842 6�992 12 4�320 5�046 5�502 5�836 6�101 6�321 6�507 6�670 6�814 13 4�260 4�964 5�404 5�727 5�981 6�192 6�372 6�528 6�667
779Appendix: Tables
Table a.9 (continued)
Critical�Values�for�the�Studentized�Range�Statistic
J or r
v 2 3 4 5 6 7 8 9 10
α�= .01
14 4�210 4�895 5�322 5�634 5�881 6�085 6�258 6�409 6�543 15 4�168 4�836 5�252 5�556 5�796 5�994 6�162 6�309 6�439 16 4�131 4�786 5�192 5�489 5�722 5�915 6�079 6�222 6�349 17 4�099 4�742 5�140 5�430 5�659 5�847 6�007 6�147 6�270 18 4�071 4�703 5�094 5�379 5�603 5�788 5�944 6�081 6�201 19 4�046 4�670 5�054 5�334 5�554 5�735 5�889 6�022 6�141 20 4�024 4�639 5�018 5�294 5�510 5�688 5�839 5�970 6�087 24 3�956 4�546 4�907 5�168 5�374 5�542 5�685 5�809 5�919 30 3�889 4�455 4�799 5�048 5�242 5�401 5�536 5�653 5�756 40 3�825 4�367 4�696 4�931 5�114 5�265 5�392 5�502 5�599 60 3�762 4�282 4�595 4�818 4�991 5�133 5�253 5�356 5�447
120 3�702 4�200 4�497 4�709 4�872 5�005 5�118 5�214 5�299 ∞ 3�643 4�120 4�403 4�603 4�757 4�882 4�987 5�078 5�157
J or r
v 11 12 13 14 15 16 17 18 19
α�= .01 1 253�2 260�0 266�2 271�8 277�0 281�8 286�3 290�4 294�3 2 32�59 33�40 34�13 34�81 35�43 36�00 36�53 37�03 37�50 3 17�13 17�53 17�89 18�22 18�52 18�81 19�07 19�32 19�55 4 12�57 12�84 13�09 13�32 13�53 13�73 13�91 14�08 14�24 5 10�48 10�70 10�89 11�08 11�24 11�40 11�55 11�68 11�81 6 9�301 9�485 9�653 9�808 9�951 10�08 10�21 10�32 10�43 7 8�548 8�711 8�860 8�997 9�124 9�242 9�353 9�456 9�554 6 8�027 8�176 8�312 8�436 8�552 8�659 8�760 8�854 8�943 9 7�647 7�784 7�910 8�025 8�132 8�232 8�325 3�412 8�495
10 7�356 7�485 7�603 7�712 7�812 7�906 7�993 8�076 8�153 11 7�128 7�250 7�362 7�465 7�560 7�649 7�732 7�809 7�883 12 6�943 7�060 7�167 7�265 7�356 7�441 7�520 7�594 7�665 13 6�791 6�903 7�006 7�101 7�188 7�269 7�345 7�417 7�485 14 6�664 6�772 6�871 6�962 7�047 7�126 7�199 7�268 7�333 15 6�555 6�660 6�757 6�845 6�927 7�003 7�074 7�142 7�204 16 6�462 6�564 6�658 6�744 6�823 6�898 6�967 7�032 7�093 17 6�381 6�480 6�572 6�656 6�734 6�806 6�873 6�937 6�997 18 6�310 6�407 6�497 6�579 6�655 6�725 6�792 6�854 6�912 19 6�247 6�342 6�430 6�510 6�585 6�654 6�719 6�780 6�837 20 6�191 6�285 6�371 6�450 6�523 6�591 6�654 6�714 6�771 24 6�017 6�106 6�186 6�261 6�330 6�394 6�453 6�510 6�563 30 5�849 5�932 6�008 6�078 6�143 6�203 6�259 6�311 6�361 40 5�686 5�764 5�835 5�900 5�961 6�017 6�069 6�119 6�165 60 5�528 5�601 5�667 5�728 5�785 5�837 5�886 5�931 5�974
120 5�375 5�443 5�505 5�562 5�614 5�662 5�708 5�750 5�790 ∞ 5�227 5�290 5�348 5�400 5�448 5�493 5�535 5�574 5�611
Source:� Reprinted�from�Harter,�H�L�,�Ann. Math. Statist�,�31,�1122,�1960,�Table�3��With�permission�of�the� Institute�of�Mathematical�Statistics�
J for�Tukey��r�for�Newman–Keuls�
780 Appendix: Tables
Table a.10
Critical�Values�for�the�Bryant–Paulson�Procedure
α = .05
v J = 2 J = 3 J = 4 J = 5 J = 6 J = 7 J = 8 J = 10 J = 12 J = 16 J = 20
X = 1
2 7�96 11�00 12�99 14�46 15�61 16�56 17�36 18�65 19�68 21�23 22�40 3 5�42 7�18 8�32 9�17 9�84 10�39 10�86 11�62 12�22 13�14 13�83 4 4�51 5�84 6�69 7�32 7�82 8�23 8�58 9�15 9�61 10�30 10�82 5 4�06 5�17 5�88 6�40 6�82 7�16 7�45 7�93 8�30 8�88 9�32 6 3�79 4�78 5�40 5�86 6�23 6�53 6�78 7�20 7�53 8�04 8�43 7 3�62 4�52 5�09 5�51 5�84 6�11 6�34 6�72 7�03 7�49 7�84 8 3�49 4�34 4�87 5�26 5�57 5�82 6�03 6�39 6�67 7�10 7�43
10 3�32 4�10 4�58 4�93 5�21 5�43 5�63 5�94 6�19 6�58 6�87 12 3�22 3�95 4�40 4�73 4�98 5�19 5�37 5�67 5�90 6�26 6�53 14 3�15 3�85 4�28 4�59 4�83 5�03 5�20 5�48 5�70 6�03 6�29 16 3�10 3�77 4�19 4�49 4�72 4�91 5�07 5�34 5�55 5�87 6�12 18 3�06 3�72 4�12 4�41 4�63 4�82 4�98 5�23 5�44 5�75 5�98 20 3�03 3�67 4�07 4�35 4�57 4�75 4�90 5�15 5�35 5�65 5�88 24 2�98 3�61 3�99 4�26 4�47 4�65 4�79 5�03 5�22 5�51 5�73 30 2�94 3�55 3�91 4�18 4�38 4�54 4�69 4�91 5�09 5�37 5�58 40 2�89 3�49 3�84 4�09 4�29 4�45 4�58 4�80 4�97 5�23 5�43 60 2�85 3�43 3�77 4�01 4�20 4�35 4�48 4�69 4�85 5�10 5�29
120 2�81 3�37 3�70 3�93 4�11 4�26 4�38 4�58 4�73 4�97 5�15
X = 2 2 9�50 13�18 15�59 17�36 18�75 19�89 20�86 22�42 23�66 25�54 26�94 3 6�21 8�27 9�60 10�59 11�37 12�01 12�56 13�44 14�15 15�22 16�02 4 5�04 6�54 7�51 8�23 8�80 9�26 9�66 10�31 10�83 11�61 12�21 5 4�45 5�68 6�48 7�06 7�52 7�90 8�23 8�76 9�18 9�83 10�31 6 4�10 5�18 5�87 6�37 6�77 7�10 7�38 7�84 8�21 8�77 9�20 7 3�87 4�85 5�47 5�92 6�28 6�58 6�83 7�24 7�57 8�08 8�46 8 3�70 4�61 5�19 5�61 5�94 6�21 6�44 6�82 7�12 7�59 7�94
10 3�49 4�31 4�82 5�19 5�49 5�73 5�93 6�27 6�54 6�95 7�26 12 3�35 4�12 4�59 4�93 5�20 5�43 5�62 5�92 6�17 6�55 6�83 14 3�26 3�99 4�44 4�76 5�01 5�22 5�40 5�69 5�92 6�27 6�54 16 3�19 3�90 4�32 4�63 4�88 5�07 5�24 5�52 5�74 6�07 6�33 18 3�14 3�82 4�24 4�54 4�77 4�96 5�13 5�39 5�60 5�92 6�17 20 3�10 3�77 4�17 4�46 4�69 4�88 5�03 5�29 5�49 5�81 6�04 24 3�04 3�69 4�08 4�35 4�57 4�75 4�90 5�14 5�34 5�63 5�86 30 2�99 3�61 3�98 4�25 4�46 4�62 4�77 5�00 5�18 5�46 5�68 40 2�93 3�53 3�89 4�15 4�34 4�50 4�64 4�86 5�04 5�30 5�50 60 2�88 3�46 3�80 4�05 4�24 4�39 4�52 4�73 4�89 5�14 5�33
120 2�82 3�38 3�72 3�95 4�13 4�28 4�40 4�60 4�75 4�99 5�17
781Appendix: Tables
Table a.10 (continued)
Critical�Values�for�the�Bryant–Paulson�Procedure
α = .05
v J = 2 J = 3 J = 4 J = 5 J = 6 J = 7 J = 8 J = 10 J = 12 J = 16 J = 20
X = 3
2 10�83 15�06 17�82 19�85 21�45 22�76 23�86 25�66 27�08 29�23 30�83 3 6�92 9�23 10�73 11�84 12�72 13�44 14�06 15�05 15�84 17�05 17�95 4 5�51 7�18 8�25 9�05 9�67 10�19 10�63 11�35 11�92 12�79 13�45 5 4�81 6�16 7�02 7�66 8�17 8�58 8�94 9�52 9�98 10�69 11�22 6 4�38 5�55 6�30 6�84 7�28 7�64 7�94 8�44 8�83 9�44 9�90
7 4�11 5�16 5�82 6�31 6�70 7�01 7�29 7�73 8�08 8�63 9�03 8 3�91 4�88 5�49 5�93 6�29 6�58 6�83 7�23 7�55 8�05 8�42
10 3�65 4�51 5�05 5�44 5�75 6�01 6�22 6�58 6�86 7�29 7�62 12 3�48 4�28 4�78 5�14 5�42 5�65 5�85 6�17 6�43 6�82 7�12 14 3�37 4�13 4�59 4�93 5�19 5�41 5�59 5�89 6�13 6�50 6�78 16 3�29 4�01 4�46 4�78 5�03 5�23 5�41 5�69 5�92 6�27 6�53 18 3�23 3�93 4�35 4�66 4�90 5�10 5�27 5�54 5�76 6�09 6�34 20 3�18 3�86 4�28 4�57 4�81 5�00 5�16 5�42 5�63 5�96 6�20 24 3�11 3�76 4�16 4�44 4�67 4�85 5�00 5�25 5�45 5�75 5�98 30 3�04 3�67 4�05 4�32 4�53 4�70 4�85 5�08 5�27 5�56 5�78 40 2�97 3�57 3�94 4�20 4�40 4�56 4�70 4�92 5�10 5�37 5�57 60 2�90 3�49 3�83 4�08 4�27 4�43 4�56 4�77 4�93 5�19 5�38
120 2�84 3�40 3�73 3�97 4�15 4�30 4�42 4�62 4�77 5�01 5�19
α = .01
v J = 2 J = 3 J = 4 J = 5 J = 6 J = 7 J = 8 J = 10 J = 12 J = 16 J = 20
X = 1
2 19�09 26�02 30�57 33�93 36�58 38�76 40�60 43�59 45�95 49�55 52�24 3 10�28 13�32 15�32 16�80 17�98 18�95 19�77 21�12 22�19 23�82 25�05 4 7�68 9�64 10�93 11�89 12�65 13�28 13�82 14�70 15�40 16�48 17�29 5 6�49 7�99 8�97 9�70 10�28 10�76 11�17 11�84 12�38 13�20 13�83 6 5�83 7�08 7�88 8�48 8�96 9�36 9�70 10�25 10�70 11�38 11�90 7 5�41 6�50 7�20 7�72 8�14 8�48 8�77 9�26 9�64 10�24 10�69 8 5�12 6�11 6�74 7�20 7�58 7�88 8�15 8�58 8�92 9�46 9�87
10 4�76 5�61 6�15 6�55 6�86 7�13 7�35 7�72 8�01 8�47 8�82 12 4�54 5�31 5�79 6�15 6�48 6�67 6�87 7�20 7�46 7�87 8�18 14 4�39 5�11 5�56 5�89 6�15 6�36 6�55 6�85 7�09 7�47 7�75 16 4�28 4�96 5�39 5�70 5�95 6�15 6�32 6�60 6�83 7�18 7�45 18 4�20 4�86 5�26 5�56 5�79 5�99 6�15 6�42 6�63 6�96 7�22 20 4�14 4�77 5�17 5�45 5�68 5�86 6�02 6�27 6�48 6�80 7�04 24 4�05 4�65 5�02 5�29 5�50 5�68 5�83 6�07 6�26 6�56 6�78 30 3�96 4�54 4�89 5�14 5�34 5�50 5�64 5�87 6�05 6�32 6�53 40 3�88 4�43 4�76 5�00 5�19 5�34 5�47 5�68 5�85 6�10 6�30 60 3�79 4�32 4�64 4�86 5�04 5�18 5�30 5�50 5�65 5�89 6�07
120 3�72 4�22 4�52 4�73 4�89 5�03 5�14 5�32 5�47 5�69 5�85 (continued)
782 Appendix: Tables
Table a.10 (continued)
Critical�Values�for�the�Bryant–Paulson�Procedure
α = .01
v J = 2 J = 3 J = 4 J = 5 J = 6 J = 7 J = 8 J = 10 J = 12 J = 16 J = 20
X = 2
2 23�11 31�55 37�09 41�19 44�41 47�06 49�31 52�94 55�82 60�20 63�47 3 11�97 15�56 17�91 19�66 21�05 22�19 23�16 24�75 26�01 27�93 29�38 4 8�69 10�95 12�43 13�54 14�41 15�14 15�76 16�77 17�58 18�81 19�74 5 7�20 8�89 9�99 10�81 11�47 12�01 12�47 13�23 13�84 14�77 15�47 6 6�36 7�75 8�64 9�31 9�85 10�29 10�66 11�28 11�77 12�54 13�11 7 5�84 7�03 7�80 8�37 8�83 9�21 9�53 10�06 10�49 11�14 11�64 8 5�48 6�54 7�23 7�74 8�14 8�48 8�76 9�23 9�61 10�19 10�63
10 5�02 5�93 6�51 6�93 7�27 7�55 7�79 8�19 8�50 8�99 9�36 12 4�74 5�56 6�07 6�45 6�75 7�00 7�21 7�56 7�84 8�27 8�60 14 4�56 5�31 5�78 6�13 6�40 6�63 6�82 7�14 7�40 7�79 8�09 16 4�42 5�14 5�58 5�90 6�16 6�37 6�55 6�85 7�08 7�45 7�73 18 4�32 5�00 5�43 5�73 5�98 6�18 6�35 6�63 6�85 7�19 7�46 20 4�25 4�90 5�31 5�60 5�84 6�03 6�19 6�46 6�67 7�00 7�25 24 4�14 4�76 5�14 5�42 5�63 5�81 5�96 6�21 6�41 6�71 6�95 30 4�03 4�62 4�98 5�24 5�44 5�61 5�75 5�98 6�16 6�44 6�66 40 3�93 4�48 4�82 5�07 5�26 5�41 5�54 5�76 5�93 6�19 6�38 60 3�83 4�36 4�68 4�90 5�08 5�22 5�35 5�54 5�70 5�94 6�12
120 3�73 4�24 4�54 4�75 4�91 5�05 5�16 5�35 5�49 5�71 5�88
X = 3 2 26�54 36�26 42�64 47�36 51�07 54�13 56�71 60�90 64�21 69�25 73�01 3 13�45 17�51 20�17 22�15 23�72 25�01 26�11 27�90 29�32 31�50 33�13 4 9�59 12�11 13�77 15�00 15�98 16�79 17�47 18�60 19�50 20�87 21�91 5 7�83 9�70 10�92 11�82 12�54 13�14 13�65 14�48 15�15 10�17 16�95 6 6�85 8�36 9�34 10�07 10�65 11�13 11�54 12�22 12�75 13�59 14�21 7 6�23 7�52 8�36 8�98 9�47 9�88 10�23 10�80 11�26 11�97 12�51 8 5�81 6�95 7�69 8�23 8�67 9�03 9�33 9�84 10�24 10�87 11�34
10 5�27 6�23 6�84 7�30 7�66 7�96 8�21 8�63 8�96 9�48 9�88 12 4�94 5�80 6�34 6�74 7�05 7�31 7�54 7�90 8�20 8�65 9�00 14 4�72 5�51 6�00 6�36 6�65 6�89 7�09 7�42 7�69 8�10 8�41 16 4�56 5�30 5�76 6�10 6�37 6�59 6�77 7�08 7�33 7�71 8�00 18 4�44 5�15 5�59 5�90 6�16 6�36 6�54 6�83 7�06 7�42 7�69 20 4�35 5�03 5�45 5�75 5�99 6�19 6�36 6�63 6�85 7�19 7�45 24 4�22 4�86 5�25 5�54 5�76 5�94 6�10 6�35 6�55 6�87 7�11 30 4�10 4�70 5�06 5�33 5�54 5�71 5�85 6�08 6�27 6�56 6�78 40 3�98 4�54 4�88 5�13 5�32 5�48 5�61 5�83 6�00 6�27 6�47 60 3�86 4�39 4�72 4�95 5�12 5�27 5�39 5�59 5�75 6�00 6�18
120 3�75 4�25 4�55 4�77 4�94 5�07 5�18 5�37 5�51 5�74 5�90
Source:� Reprinted�from�Bryant,�J�L��and�Paulson,�A�S�,�Biometrika,�63,�631,�1976,�Table�1(a)�and�Table�1(b)��With� permission�of�Biometrika�Trustees�
X is�the�number�of�covariates�
783
References
Agresti,�A�,�&�Finlay,�B��(1986)��Statistical methods for the social sciences�(2nd�ed�)��San�Francisco:�Dellen� Agresti,� A�,� &� Pendergast,� J�� (1986)�� Comparing� mean� ranks� for� repeated� measures� data��
Communications in Statistics—Theory and Methods,�15,�1417–1433� Aldrich,�J��H�,�&�Nelson,�F��D��(1984)��Linear probability, logit, and probit models��Beverly�Hills,�CA:�Sage� Algina,� J�,� Blair,� R�� C�,� &� Coombs,� W�� T�� (1995)�� A� maximum� test� for� scale:� Type� I� error� rates� and�
power��Journal of Educational and Behavioral Statistics,�20,�27–39� Algina,�J�,�&�Keselman,�H��J��(2003)��Approximate�confidence�intervals�for�effect�sizes��Educational and
Psychological Measurement,�63(4),�537–553� Algina,� J�,� Keselman,� H�� J�,� &� Penfield,� R�� D�� (2005)�� Effect� sizes� and� their� intervals:� The� two-� level�
repeated�measures�case��Educational and Psychological Measurement,�65(2),�241–258� American� Psychological� Association�� (2010)�� Publication manual of the American Psychological
Association��Washington,�DC:�Author� Andrews,�D��F��(1971)��Significance�tests�based�on�residuals��Biometrika,�58,�139–148� Andrews,�D��F�,�&�Pregibon,�D��(1978)��Finding�the�outliers�that�matter��Journal of the Royal Statistical
Society, Series B,�40,�85–93� Applebaum,�M��I�,�&�Cramer,�E��M��(1974)��Some�problems�in�the�nonorthogonal�analysis�of�variance��
Psychological Bulletin,�81,�335–343� Atiqullah,�M��(1964)��The�robustness�of�the�covariance�analysis�of�a�one-way�classification��Biometrika,�
51,�365–373� Atkinson,�A��C��(1985)��Plots, transformations, and regression��Oxford,�U�K�:�Oxford�University�Press� Barnett,�V�,�&�Lewis,�T��(1978)��Outliers in statistical data��New�York:�Wiley� Barnett,�V�,�&�Lewis,�T��(1994)��Outliers in statistical data (3rd�ed�)��New�York:�Wiley� Basu,�S�,�&�DasGupta,�A��(1995)��Robustness�of�standard�confidence�intervals�for�location�parameters�
under�departure�from�normality��Annals of Statistics,�23,�1433–1442� Bates,�D��M�,�&�Watts,�D��G��(1988)��Nonlinear regression analysis and its applications��New�York:�Wiley� Beal,�S��L��(1987)��Asymptotic�confidence�intervals�for�the�difference�between�two�binomial�param-
eters�for�use�with�small�samples��Biometrics,�43,�941–950� Beckman,�R�,�&�Cook,�R��D��(1983)��Outliers…�s��Technometrics,�25,�119–149� Belsley,�D��A�,�Kuh,�E�,�&�Welsch,�R��E��(1980)��Regression diagnostics��New�York:�Wiley� Benjamini,�Y�,�&�Hochberg,�Y��(1995)��Controlling�the�false�discovery�rate:�A�practical�and�powerful�
approach�to�multiple�testing��Journal of the Royal Statistical Society, B,�57,�289–300� Bernstein,�I��H��(1988)��Applied multivariate analysis��New�York:�Springer-Verlag� Berry,�W��D�,�&�Feldman,�S��(1985)��Multiple regression in practice��Beverly�Hills,�CA:�Sage� Boik,�R��J��(1979)��Interactions,�partial�interactions,�and�interaction�contrasts�in�the�analysis�of�vari-
ance��Psychological Bulletin,�86,�1084–1089� Boik,�R��J��(1981)��A�priori�tests�in�repeated�measures�designs:�Effects�of�nonsphericity��Psychometrika,�
46,�241–255� Box,� G�� E�� P�� (1954a)�� Some� theorems� on� quadratic� forms� applied� in� the� study� of� analysis� of� vari-
ance�problems,�I:�Effects�of�inequality�of�variance�in�the�one-way�model��Annals of Mathematical Statistics,�25,�290–302�
Box,�G��E��P��(1954b)��Some�theorems�on�quadratic�forms�applied�in�the�study�of�analysis�of�variance� problems,�II:�Effects�of�inequality�of�variance�and�of�correlation�between�errors�in�the�two-way� classification��Annals of Mathematical Statistics,�25,�484–498�
Box,�G��E��P�,�&�Anderson,�S��L��(1962)��Robust tests for variances and effect of non-normality and variance heterogeneity on standard tests��Tech��Rep��No��7,�Ordinance�Project�No��TB�2-0001�(832),�Dept��of� Army�Project�No��599-01-004�
784 References
Box,�G�E�P��&�Cox,�D�R���(1964)���An�analysis�of�transformations�(with�discussion)���Journal of the Royal Statistical Society,�Series�B,�26,�211–246�
Bradley,�J��V��(1978)��Robustness?�British Journal of Mathematical and Statistical Psychology, 31,�144–152�� Bradley,�J�V���(1982)���The�insidious�L-shaped�distribution���Bulletin of the Psychonomic Society,�20(2),�
85–88� Brown,�M��B�,�&�Forsythe,�A��(1974)��The�ANOVA�and�multiple�comparisons�for�data�with�heteroge-
neous�variances��Biometrics,�30,�719–724� Brunner,� E�,� Detta,� H�,� &� Munk,� A�� (1997)�� Box-type� approximations� in� nonparametric� factorial�
designs��Journal of the American Statistical Association,�92,�1494–1502� Bryant,� J�� L�,� &� Paulson,�A�� S�� (1976)��An� extension� of� Tukey’s� method� of� multiple� comparisons� to�
experimental�designs�with�random�concomitant�variables��Biometrika,�63,�631–638� Campbell,�D��T�,�&�Stanley,�J��C��(1966)��Experimental and quasi-experimental designs for research��Chicago:�
Rand�McNally� Carlson,�J��E�,�&�Timm,�N��H��(1974)��Analysis�of�nonorthogonal�fixed-effects�designs��Psychological
Bulletin,�81,�563–570� Carroll,� R�� J�,� &� Ruppert,� D�� (1982)�� Robust� estimation� in� heteroscedastic� linear� models�� Annals of
Statistics,�10,�429–441� Chakravart,� I�� M�,� Laha,� R�� G�,� &� Roy,� J�� (1967)�� Handbook of methods of applied statistics� (Vol�� 1)��
New York:�Wiley� Chambers,�J��M�,�Cleveland,�W��S�,�Kleiner,�B�,�&�Tukey,�P��A��(1983)��Graphical methods for data analysis��
Belmont,�CA:�Wadsworth� Chatterjee,�S�,�&�Price,�B��(1977)��Regression analysis by example��New�York:�Wiley� Christensen,�R��(1997)��Log-linear models and logistic regression�(2nd�ed�)��New�York:�Springer-Verlag� Cleveland,�W��S��(1993)��Elements of graphing data��New�York:�Chapman�&�Hall� Clinch,�J��J�,�&�Keselman,�H��J��(1982)��Parametric�alternatives�to�the�analysis�of�variance��Journal of
Educational Statistics,�7,�207–214� Coe,�P��R�,�&�Tamhane,�A��C��(1993)��Small�sample�confidence�intervals�for�the�difference,�ratio�and�
odds�ratio�of�two�success�probabilities��Communications in Statistics- Simulation and Computation,� 22,�925–938�
Cohen,�J��(1988)��Statistical power analysis for the behavioral sciences�(2nd�ed�)��Hillsdale,�NJ:�Erlbaum� Cohen,� J�,� &� Cohen,� P�� (1983)�� Applied multiple regression/correlation analysis for the behavioral sciences�
(2nd�ed�)��Hillsdale,�NJ:�Erlbaum� Conover,�W�,�&�Iman,�R��(1981)��Rank�transformations�as�a�bridge�between�parametric�and�nonpara-
metric�statistics��The American Statistician,�35,�124–129� Conover,� W�,� &� Iman,� R�� (1982)��Analysis� of� covariance� using� the� rank� transformation�� Biometrics,�
38, 715–724� Cook,�R��D��(1977)��Detection�of�influential�observations�in�linear�regression��Technometrics,�19,�15–18� Cook,�T��D�,�&�Campbell,�D��T��(1979)��Quasi-experimentation: Design and analysis issues for field settings��
Chicago:�Rand�McNally� Cook,�R��D�,�&�Weisberg,�S��(1982)��Residuals and influence in regression��London:�Chapman�&�Hall� Coombs,�W��T�,�Algina,�J�,�&�Ottman,�D��O��(1996)��Univariate�and�multivariate�omnibus�hypothesis�
tests�selected�to�control�Type�I�error�rates�when�population�variances�are�not�necessarily�equal�� Review of Educational Research,�66,�137–179�
Cotton,� J�� W�� (1998)�� Analyzing within-subjects experiments�� Mahwah,� NJ:� Lawrence� Erlbaum� Associates�
Cox,�D��R�,�&�Snell,�E��J��(1989)��Analysis of binary data�(2nd�ed�)��London:�Chapman�&�Hall� Cramer,� E�� M�,� &� Applebaum,� M�� I�� (1980)�� Nonorthogonal� analysis� of� variance—Once� again��
Psychological Bulletin,�87,�51–57� Croux,�C�,�Flandre,�C�,�&�Haesbroeck,�G��(2002)��The�breakdown�behavior�of�the�maximum�likelihood�
estimator�in�the�logistic�regression�model��Statistics and Probability Letters,�60,�377–386� Cumming,�G�,�&�Finch,�S��(2001)��A�primer�on�the�understanding,�use,�and�calculation�of�confidence�
intervals� that� are� based� on� central� and� noncentral� distributions�� Educational and Psychological Measurement,�61(4),�532–574�
785References
D’Agostino,� R�� B�� (1971)�� An� omnibus� test� of� normality� for� moderate� and� large� size� samples�� Biometrika,�58,�341–348�
Derksen,�S�,�&�Keselman,�H��J��(1992)��Backward,�forward�and�stepwise�automated�subset�selection� algorithms:�Frequency�of�obtaining�authentic�and�noise�variables��British Journal of Mathematical and Statistical Psychology,�45,�265–282�
Duncan,�G��T�,�&�Layard,�M��W��J��(1973)��A�Monte-Carlo�study�of�asymptotically�robust�tests�for�cor- relation�coefficients��Biometrika,�60,�551–558�
Dunn,�O��J��(1961)��Multiple�comparisons�among�means��Journal of the American Statistical Association,� 56,�52–64�
Dunn,�O��J��(1974)��On�multiple�tests�and�confidence�intervals��Communications in Statistics,�3,�101–103� Dunn,� O�� J�,� &� Clark,� V�� A�� (1987)�� Applied statistics: Analysis of variance and regression� (2nd� ed�)��
New York:�Wiley� Dunnett,� C�� W�� (1955)��A� multiple� comparison� procedure� for� comparing� several� treatments� with� a�
control��Journal of the American Statistical Association,�50,�1096–1121� Dunnett,�C��W��(1964)��New�tables�for�multiple�comparisons�with�a�control��Biometrics,�20,�482–491� Dunnett,� C�� W�� (1980)�� Pairwise� multiple� comparisons� in� the� unequal� variance� case�� Journal of the
American Statistical Association,�75,�796–800� Durbin,�J�,�&�Watson,�G��S��(1950)��Testing�for�serial�correlation�in�least�squares�regression,�I��Biometrika,�
37,�409–428� Durbin,� J�,� &� Watson,� G�� S�� (1951)�� Testing� for� serial� correlation� in� least� squares� regression,� II��
Biometrika,�38,�159–178� Durbin,� J�,� &� Watson,� G�� S�� (1971)�� Testing� for� serial� correlation� in� least� squares� regression,� III��
Biometrika,�58,�1–19� Educational�and�Psychological�Measurement��(2000,�October)��Special�section:�Statistical�significance�
with�comments�by�editors�of�marketing�journals��Educational and Psychological Measurement,�60,� 661–696�
Educational� and� Psychological� Measurement�� (2001a,�April)�� Special� section:� Colloquium� on� effect� sizes:� The� roles� of� editors,� textbook� authors,� and� the� publication� manual�� Educational and Psychological Measurement,�61,�181–228�
Educational�and�Psychological�Measurement��(2001b,�August)��Special�section:�Confidence�intervals� for�effect�sizes��Educational and Psychological Measurement,�61,�517–674�
Elashoff,� J�� D�� (1969)��Analysis� of� covariance:�A� delicate� instrument�� American Educational Research Journal,�6,�383–401�
Feldt,�L��S��(1958)��A�comparison�of�the�precision�of�three�experimental�designs�employing�a�concomi- tant�variable��Psychometrika,�23,�335–354�
Ferguson,�G��A�,�&�Takane,�Y��(1989)��Statistical analysis in psychology and education�(6th�ed�)��New�York:� McGraw-Hill�
Fidler,� F�,� &� Thompson,� B�� (2001)�� Computing� correct� confidence� intervals� for� ANOVA� fixed-� and� random-effects�effect�sizes��Educational and Psychological Measurement,�61,�575–604�
Finch,� S�,� &� Cumming,� G�� (2009)�� Putting� research� in� context:� Understanding� confidence� intervals� from�one�or�more�studies��Journal of Pediatric Psychology,�34(9),�903–916�
Fink,�A��(1995)��How to sample in surveys��Thousand�Oaks,�CA:�Sage� Fisher,�R��A��(1949)��The design of experiments��Edinburgh,�U�K�:�Oliver�&�Boyd,�Ltd� Friedman,�M��(1937)��The�use�of�ranks�to�avoid�the�assumption�of�normality�implicit�in�the�analysis�
of�variance��Journal of the American Statistical Association,�32,�675–701� Games,�P��A�,�&�Howell,�J��F��(1976)��Pairwise�multiple�comparison�procedures�with�unequal�n’s�and/
or�variances:�A�Monte�Carlo�study��Journal of Educational Statistics,�1,�113–125� Geisser,� S�,� &� Greenhouse,� S�� (1958)�� Extension� of� Box’s� results� on� the� use� of� the� F� distribution� in�
multivariate�analysis��Annals of Mathematical Statistics,�29,�855–891� Ghosh,�B��K��(1979)��A�comparison�of�some�approximate�confidence�intervals�for�the�binomial�param-
eter��Journal of the American Statistical Association,�74,�894–900� Glass,�G��V�,�&�Hopkins,�K��D��(1996)��Statistical methods in education and psychology�(3rd�ed�)��Boston:�
Allyn�&�Bacon�
786 References
Glass,� G�� V�,� Peckham,� P�� D�,� &� Sanders,� J�� R�� (1972)�� Consequences� of� failure� to� meet� assumptions� underlying�the�fixed�effects�analyses�of�variance�and�covariance��Review of Educational Research,� 42,�237–288�
Grimm,� L�� G�,� &� Arnold,� P�� R�� (Eds�)�� (1995)�� Reading and understanding multivariate statistics�� Washington,�DC:�American�Psychological�Association�
Grimm,� L�� G�,� &� Arnold,� P�� R�� (Eds�)�� (2002)�� Reading and understanding more multivariate statistics�� Washington,�DC:�American�Psychological�Association�
Grissom,� R�� J�,� &� Kim,� J�� J�� (2005)�� Effect sizes for research: A broad practical approach�� Mahwah,� NJ:� Lawrence�Erlbaum�Associates�
Hair,�J��F�,�Black,�W��C�,�Babin,�B��J�,�Anderson,�R��E�,�&�Tatham,�R��L��(2006)��Multivariate data analysis� (6th�ed�)��Upper�Saddle�River,�NJ:�Pearson�Prentice�Hall�
Harlow,�L�,�Mulaik,�S�,�&�Steiger,�J��(Eds�)��(1997)��What if there were no significance tests?�Mahwah,�NJ:� Lawrence�Erlbaum�Associates�
Harrell,� F�� E�� J�� (1986)�� The� LOGIST� procedure�� In� I�� SAS� Institute� (Ed�),� SUGI supplemental library user’s guide�(5th�ed�,�pp��269–293)��Cary,�NC:�SAS�Institute,�Inc�
Harwell,�M��(2003)��Summarizing�Monte�Carlo�results�in�methodological�research:�The�single-�factor,� fixed-effects�ANCOVA�case��Journal of Educational and Behavioral Statistics,�28,�45–70�
Hawkins,�D��M��(1980)��Identification of outliers��London:�Chapman�&�Hall� Hays,�W��L��(1988)��Statistics�(4th�ed�)��New�York:�Holt,�Rinehart�and�Winston� Hayter,�A��J��(1986)��The�maximum�familywise�error�rate�of�Fisher’s�least�significant�difference�test��
Journal of the American Statistical Association,�81,�1000–1004� Heck,� R�� H�,� &� Thomas,� S�� L�� (2000)�� An introduction to multilevel modeling techniques�� Mahwah,� NJ:�
Lawrence�Erlbaum� Heck,�R��H�,�Thomas,�S��L�,�&�Tabata,�L��N��(2010)��Multilevel and longitudinal modeling with IBM SPSS��
New�York:�Routledge� Hellevik,� O�� (2009)�� Linear� versus� logistic� regression� when� the� dependent� variable� is� a� dichotomy��
Quality & Quantity,�43(1),�59–74� Heyde,�C��C�,�Seneta,�E�,�Crepel,�P�,�Feinberg,�S��E�,�&�Gani,�J��(Eds�)��(2001)��Statisticians of the centuries��
New�York:�Springer� Hochberg,� Y�� (1988)�� A� sharper� Bonferroni� procedure� for� multiple� tests� of� significance�� Biometrika,�
75, 800–802� Hochberg,�Y�,�&�Tamhane,�A��C��(1987)��Multiple comparison procedures��New�York:�Wiley� Hochberg,� Y�,� &� Varon-Salomon,� Y�� (1984)�� On� simultaneous� pairwise� comparisons� in� analysis� of�
covariance��Journal of the American Statistical Association,�79,�863–866� Hocking,�R��R��(1976)��The�analysis�and�selection�of�variables�in�linear�regression��Biometrics,�32, 1–49� Hoenig,�J��M�,�&�Heisey,�D��M��(2001)��The�abuse�of�power:�The�pervasive�fallacy�of�power�calcula-
tions�for�data�analysis��The American Statistician,�55,�19–24� Hoerl,� A�� E�,� &� Kennard,� R�� W�� (1970a)�� Ridge� regression:� Biased� estimation� for� non-orthogonal�
models��Technometrics,�12,�55–67� Hoerl,� A�� E�,� &� Kennard,� R�� W�� (1970b)�� Ridge� regression:� Application� to� non-orthogonal� models��
Technometrics,�12,�591–612� Hogg,�R��V�,�&�Craig,�A��T��(1970)��Introduction to mathematical statistics��New�York:�Macmillan� Hosmer,�D��W�,�Hosmer,�T�,�LeCessie,�S�,�&�Lemeshow,�S��(1997)��A�comparison�of�goodness-of-fit�tests�
for�the�logistic�regression�model��Statistics in Medicine,�16,�965–980� Hosmer,�D��W�,�&�Lemeshow,�S��(1989)��Applied logistic regression,�New�York:�Wiley� Hosmer,�D��W�,�&�Lemeshow,�S��(2000)��Applied logistic regression�(2nd�ed�)��New�York:�Wiley� Howell,�D���(1997)���Statistical methods for psychology�(4th�ed�)���Belmont,�CA:�Wadsworth� Huberty,�C��J��(1989)��Problems�with�stepwise�methods—Better�alternatives��In�B��Thompson�(Ed�),�
Advances in social science methodology�(Vol��1,�pp��43–70)��Greenwich,�CT:�JAI�Press� Huck,�S��W��(2004)��Reading statistics and research�(4th�ed�)��Boston:�Allyn�&�Bacon� Huck,� S�� W�,� &� McLean,� R��A�� (1975)�� Using� a� repeated� measures�ANOVA� to� analyze� data� from� a�
pretest-posttest�design:�A�potentially�confusing�task��Psychological Bulletin,�82,�511–518�
787References
Huitema,�B��E��(1980)��The analysis of covariance and alternatives��New�York:�Wiley� Huberty,�C��J��(2002)��A�history�of�effect�size�indices��Educational and Psychological Measurement,�62(2),�
227–240� Huynh,� H�,� &� Feldt,� L�� S�� (1970)�� Conditions� under� which� mean� square� ratios� in� repeated� mea-
surement� designs� have� exact� F-distributions�� Journal of the American Statistical Association,� 65, 1582–1589�
Jaeger,�R��M��(1984)��Sampling in education and the social sciences��New�York:�Longman� James,�G��S��(1951)��The�comparison�of�several�groups�of�observations�when�the�ratios�of�the�popula-
tion�variances�are�unknown��Biometrika,�38,�324–329� Jennings,�E��(1988)��Models�for�pretest-posttest�data:�Repeated�measures�ANOVA�revisited��Journal of
Educational Statistics,�13,�273–280� Johansen,�S��(1980)��The�Welch-James�approximation�to�the�distribution�of�the�residual�sum�of�squares�
in�a�weighted�linear�regression��Biometrika,�67,�85–93� Johnson,�P��O�,�&�Neyman,�J��(1936)��Tests�of�certain�linear�hypotheses�and�their�application�to�some�
educational�problems��Statistical Research Memoirs,�1,�57–93� Johnson,�R��A�,�&�Wichern,�D��W��(1998)��Applied multivariate statistical analysis�(4th�ed�)��Upper�Saddle�
River,�NJ:�Prentice�Hall� Kaiser,�L�,�&�Bowden,�D��(1983)��Simultaneous�confidence�intervals�for�all�linear�contrasts�of�means�
with�heterogeneous�variances��Communications in Statistics—Theory and Methods,�12,�73–88� Kalton,�G��(1983)��Introduction to survey sampling��Thousand�Oaks,�CA:�Sage� Keppel,�G��(1982)��Design and analysis: A researcher’s handbook�(2nd�ed�)��Englewood�Cliffs,�NJ:�Prentice-Hall� Keppel,�G�,�&�Wickens,�T��D��(2004)��Design and analysis: A researcher’s handbook�(3rd�ed�)��Upper�Saddle�
River,�NJ:�Pearson� Kirk,� R�� E�� (1982)�� Experimental design: Procedures for the behavioral sciences� (2nd� ed�)�� Monterey,� CA:�
Brooks/Cole� Kleinbaum,�D��G�,�Kupper,�L��L�,�Muller,�K��E�,�&�Nizam,�A��(1998)��Applied regression analysis and other
multivariable methods�(3rd�ed�)��Pacific�Grove,�CA:�Duxbury� Kramer,�C��Y��(1956)��Extension�of�multiple�range�test�to�group�means�with�unequal�numbers�of�rep-
lications��Biometrics,�12,�307–310� Kreft,�I�,�&�de�Leeuw,�J��(1998)��Introducing multilevel modeling��Thousand�Oaks,�CA:�Sage� Kruskal,�W��H�,�&�Wallis,�W��A��(1952)��Use�of�ranks�on�one-criterion�variance�analysis��Journal of the
American Statistical Association,�47,�583–621�(with�corrections�in�48,�907–911)� Lamb,�G��S��(1984)��What�you�always�wanted�to�know�about�six�but�were�afraid�to�ask��The Journal of
Irreproducible Results,�29,�18–20� Larsen,� W�� A�,� &� McCleary,� S�� J�� (1972)�� The� use� of� partial� residual� plots� in� regression� analysis��
Technometrics,�14,�781–790� Levy,� P�� S�,� &� Lemeshow,� S�� (1999)�� Sampling of populations: Methods and applications� (3rd� ed�)�� New�
York:�Wiley� Li,�J�,�&�Lomax,�R��G��(2011)��Analysis�of�variance:�What�is�your�statistical�software�actually�doing?�
Journal of Experimental Education,�73,�279–294� Lilliefors,� H�� (1967)�� On� the� Kolmogorov-Smirnov� test� for� normality� with� mean� and� variance�
unknown��Journal of the American Statistical Association,�62,�399–402� Lomax,�R��G�,�&�Surman,�S��H��(2007)��Factorial�ANOVA�in�SPSS:�Fixed-,�random-,�and�mixed-effects�
models��In�S��S��Sawilowsky�(Ed�),�Real data analysis��Greenwich,�CT:�Information�Age� Long,� J�� S�� (1997)�� Regression� models� for� categorical� and� limited� dependent� variables�� Thousand�
Oaks,�CA:�Sage� Lord,�F��M��(1960)��Large-sample�covariance�analysis�when�the�control�variable�is�fallible��Journal of
the American Statistical Association,�55,�307–321� Lord,�F��M��(1967)��A�paradox�in�the�interpretation�of�group�comparisons��Psychological Bulletin,�68,�
304–305� Lord,�F��M��(1969)��Statistical�adjustments�when�comparing�preexisting�groups��Psychological Bulletin,�
72,�336–337�
788 References
Manly,�B��F��J��(2004)��Multivariate statistical methods: A primer�(3rd�ed�)��London:�Chapman�&�Hall� Mansfield,� E�� R�,� &� Conerly,� M�� D�� (1987)�� Diagnostic� value� of� residual� and� partial� residual� plots��
The American Statistician,�41,�107–116� Marascuilo,� L�� A�,� &� Levin,� J�� R�� (1970)�� Appropriate� post� hoc� comparisons� for� interactions� and�
nested�hypotheses�in�analysis�of�variance�designs:�The�elimination�of�type�IV�errors��American Educational Research Journal,�7,�397–421�
Marascuilo,� L�� A�,� &� Levin,� J�� R�� (1976)�� The� simultaneous� investigation� of� interaction� and� nested� hypotheses� in� two-factor� analysis� of� variance� designs�� American Educational Research Journal,� 13,�61–65�
Marascuilo,� L��A�,� &� McSweeney,� M�� (1977)�� Nonparametric and distribution-free methods for the social sciences��Monterey,�CA:�Brooks/Cole�
Marascuilo,�L��A�,�&�Serlin,�R��C��(1988)��Statistical methods for the social and behavioral sciences��New� York:�Freeman�
Marcoulides,�G��A�,�&�Hershberger,�S��L��(1997)��Multivariate statistical methods: A first course��Mahwah,� NJ:�Lawrence�Erlbaum�Associates�
Marquardt,� D�� W�,� &� Snee,� R�� D�� (1975)�� Ridge� regression� in� practice�� The American Statistician,� 29, 3–19�
Maxwell,� S�� E�� (1980)�� Pairwise� multiple� comparisons� in� repeated� measures� designs�� Journal of Educational Statistics,�5,�269–287�
Maxwell,�S��E�,�&�Delaney,�H��D��(1990)��Designing experiments and analyzing data: A model comparison perspective��Belmont,�CA:�Wadsworth�
Maxwell,� S�� E�,� Delaney,� H�� D�,� &� Dill,� C�� A�� (1984)�� Another� look� at� ANOVA� versus� blocking�� Psychological Bulletin,�95,�136–147�
McCulloch,�C��E��(2005)��Repeated�measures�ANOVA,�RIP?�Chance,�18,�29–33� Menard,�S��(1995)��Applied logistic regression analysis��Thousand�Oaks,�CA:�Sage� Menard,�S��(2000)��Applied logistic regression analysis�(2nd�ed�)��Thousand�Oaks,�CA:�Sage� Mendoza,� J�� L�,� &� Stafford,� K�� L�� (2001)�� Confidence� intervals,� power� calculation,� and� sample� size�
estimation�for�the�squared�multiple�correlation�coefficient�under�the�fixed�and�random�regres- sion� models:� A� computer� program� and� useful� standard� tables�� Educational and Psychological Measurement,�61,�650–667�
Meyers,�L��S�,�Gamst,�G�,�&�Guarino,�A��J��(2006)��Applied multivariate research: Design and interpretation�� Thousand�Oaks,�CA:�Sage�
Mickey,�R��M�,�Dunn,�O��J�,�&�Clark,�V��A��(2004)��Applied statistics: Analysis of variance and regression� (3rd�ed�)��Hoboken,�NJ:�Wiley�
Miller,�A��J��(1984)��Selection�of�subsets�of�regression�variables�(with�discussion)��Journal of the Royal Statistical Society, A,�147,�389–425�
Miller,�A��J��(1990)��Subset selection in regression��New�York:�Chapman�&�Hall� Miller,�R��G��(1997)��Beyond ANOVA: Basics of applied statistics��Boca�Raton,�FL:�CRC�Press� Morgan,�G��A�,�Leech,�N��L�,�Gloeckner,�&�Barrett,�K��C���(2011)���IBM SPSS for introductory statistics:�
Use and interpretation�(4th�edition)���New�York:�Routledge� Morgan,� G�� A�,� &� Griego,� O�� V�� (1998)�� Easy use and interpretation of SPSS for Windows: Answering
research questions with statistics��Mahwah,�NJ:�Lawrence�Erlbaum�Associates� Morgan,�G��A�,�Leech,�N��L�,�Gloeckner,�G��W�,�&�Barrett,�K��C��(2005)��IBM SPSS for introductory statis-
tics: Use and interpretation�(4th�ed�)��New�York:�Routledge� Mosteller,�F�,�&�Tukey,�J�W���(1977)���Data analysis and regression���Reading,�MA:�Addision-Wesley� Murphy,�K��R�,�&�Myors,�B��(2004)��Statistical power analysis: A simple and general model for traditional
and modern hypothesis tests�(2nd�ed�)��Mahwah,�NJ:�Lawrence�Erlbaum�Associates� Murphy,�K��R�,�Myors,�B�,�&�Wolach,�A��(2008)��Statistical power analysis:�A simple and general model for
traditional and modern hypothesis tests�(3rd�ed�)��New�York:�Routledge Myers,�R��H��(1979)��Fundamentals of experimental design�(3rd�ed�)��Boston:�Allyn�and�Bacon� Myers,�R��H��(1986)��Classical and modern regression with applications��Boston:�Duxbury� Myers,�R��H��(1990)��Classical and modern regression with applications�(2nd�ed�)��Boston:�Duxbury�
789References
Myers,� J�� L�,� &� Well,� A�� D�� (1995)�� Research design and statistical analysis�� Mahwah,� NJ:� Lawrence� Erlbaum�Associates�
Nagelkerke,� N�� J�� D�� (1991)�� A� note� on� a� general� definition� of� the� coefficient� of� determination�� Biometrika,�78,�691–692�
Noreen,�E��W��(1989)��Computer intensive methods for testing hypotheses��New�York:�Wiley� O’Connell,�A��A�,�&�McCoach,�D��B��(Eds�)��(2008)��Multilevel modeling of educational data��Charlotte,�
NC:�Information�Age�Publishing� O’Grady,� K�� E�� (1982)�� Measures� of� explained� variance:� Cautions� and� limitations�� Psychological
Bulletin,�92,�766–777� Olejnik,�S��F�,�&�Algina,�J��(1987)��Type�I�error�rates�and�power�estimates�of�selected�parametric�and�
nonparametric�tests�of�scale��Journal of Educational Statistics,�21,�45–61� Overall,�J��E�,�Lee,�D��M�,�&�Hornick,�C��W��(1981)��Comparison�of�two�strategies�for�analysis�of�vari-
ance�in�nonorthogonal�designs��Psychological Bulletin,�90,�367–375� Overall,� J�� E�,� &� Spiegel,� D�� K�� (1969)�� Concerning� least� squares� analysis� of� experimental� data��
Psychological Bulletin,�72,�311–322� Page,� M�� C�,� Braver,� S�� L�,� &� MacKinnon,� D�� P�� (2003)�� Levine’s guide to SPSS for analysis of variance��
Mahwah,�NJ:�Lawrence�Erlbaum�Associates� Pampel,�F��C��(2000)��Logistic regression: A primer��Thousand�Oaks,�CA:�Sage� Pavur,� R�� (1988)�� Type� I� error� rates� for� multiple� comparison� procedures� with� dependent� data�� The
American Statistician,�42,�171–173� Pearson,�E��S��(Ed�)��(1978)��The history of statistics in the 17th and 18th centuries��New�York:�Macmillan� Peckham,� P�� D�� (1968)�� An investigation of the effects of non-homogeneity of regression slopes upon the
F-test of analysis of covariance��Laboratory� of� Educational� Research,� Rep�� No�� 16,� University� of� Colorado,�Boulder,�CO�
Pedhazur,� E�� J�� (1997)�� Multiple regression in behavioral research� (3rd� ed�)�� Fort� Worth,� TX:� Harcourt� Brace�
Pingel,� L�� A�� (1969)�� A comparison of the effects of two methods of block formation on design precision�� Paper�presented�at�the�annual�meeting�of�the�American�Educational�Research�Association,�Los� Angeles�
Porter,�A��C��(1967)��The effects of using fallible variables in the analysis of covariance��Unpublished�doc- toral�dissertation,�University�of�Wisconsin,�Madison,�WI�
Porter,�A��C�,�&�Raudenbush,�S��W��(1987)��Analysis�of�covariance:�Its�model�and�use�in�psychological� research��Journal of Counseling Psychology,�34,�383–392�
Puri,� M�� L�,� &� Sen,� P�� K�� (1969)�� Analysis� of� covariance� based� on� general� rank� scores�� Annals of Mathematical Statistic,�40,�610–618�
Quade,� D�� (1967)�� Rank� analysis� of� covariance�� Journal of the American Statistical Association,� 62,� 1187–1200�
Raferty,�A��E��(1995)��Bayesian�model�selection�in�social�research��In�P��V��Marsden�(Ed�),�Sociological methodology 1995�(pp��111–163)��London:�Tavistock�
Ramsey,� P�� H�� (1989)�� Critical� values� of� Spearman’s� rank� order� correlation�� Journal of Educational Statistics,�14,�245–253�
Ramsey,� P�� H�� (1994)�� Testing� variances� in� psychological� and� educational� research�� Journal of Educational Statistics,�19,�23–42�
Reichardt,�C��S��(1979)��The�statistical�analysis�of�data�from�nonequivalent�control�group�designs��In� T��D��Cook�&�D��T��Campbell�(Eds�),�Quasi-experimentation: Design and analysis issues for field set- tings��Chicago:�Rand�McNally�
Reise,�S��P�,�&�Duan,�N��(Eds�)��(2003)��Multilevel modeling: Methodological advances, issues, and applica- tions��Mahwah,�NJ:�Lawrence�Erlbaum�
Robbins,�N��B��(2004)��Creating more effective graphs��San�Francisco:�Jossey-Bass� Rogosa,�D��R��(1980)��Comparing�non-parallel�regression�lines��Psychological Bulletin,�88,�307–321� Rosenthal,�R�,�&�Rosnow,�R��L��(1985)��Contrast analysis: Focused comparisons in the analysis of variance��
Cambridge,�U�K�:�Cambridge�University�Press�
790 References
Rousseeuw,�P��J�,�&�Leroy,�A��M��(1987)��Robust regression and outlier detection��New�York:�Wiley� Rudas,�T��(2004)��Probability theory: A primer��Thousand�Oaks,�CA:�Sage� Ruppert,�D�,�&�Carroll,�R��J��(1980)��Trimmed�least�squares�estimation�in�the�linear�model��Journal of
the American Statistical Association,�75,�828–838� Rutherford,�A��(1992)��Alternatives�to�traditional�analysis�of�covariance��British Journal of Mathematical
and Statistical Psychology,�45,�197–223� Sawilowsky,�S��S�,�&�Blair,�R��C��(1992)��A�more�realistic�look�at�the�robustness�and�type�II�error�prop-
erties�of�the�t-test�to�departures�from�population�normality��Psychological Bulletin,�111,�352–360� Scariano,�S��M�,�&�Davenport,�J��M��(1987)��The�effects�of�violations�of�independence�assumptions�in�
the�one-way�ANOVA��The American Statistician,�41,�123–129� Schafer,� W�� D�� (1991)�� Reporting� hierarchical� regression� results�� Measurement and Evaluation in
Counseling and Development,�24,�98–100� Scheffe’,� H�� (1953)�� A� method� for� judging� all� contrasts� in� the� analysis� of� variance�� Biometrika,�
40, 87–104� Schmid,�C��F��(1983)��Statistical graphics: Design principles and practices��New�York:�Wiley� Seber,�G��A��F�,�&�Wild,�C��J��(1989)��Nonlinear regression��New�York:�Wiley� Shapiro,�S��S�,�&�Wilk,�M��B��(1965)��An�analysis�of�variance�test�for�normality�(complete�samples)��
Biometrika,�52,�591–611� Shadish,�W��R�,�Cook,�T��D�,�&�Campbell,�D��T��(2002)��Experimental and quasi-experimental designs for
generalized causal inference��Boston:�Houston�Mifflin� Shavelson,�R��J��(1988)��Statistical reasoning for the behavioral sciences�(2nd�ed�)��Boston:�Allyn�&�Bacon� Sidak,�Z��(1967)��Rectangular�confidence�regions�for�the�means�of�multivariate�normal�distributions��
Journal of the American Statistical Association,�62,�626–633� Smithson,�M��(2001)��Correct�confidence�intervals�for�various�regression�effect�sizes�and�parameters:�
The�importance�of�noncentral�distributions�in�computing�intervals��Educational and Psychological Measurement,�61,�605–632�
Snijders,�T��A��B�,�&�Bosker,�R��J��(1999)��Multilevel analysis: An introduction to basic and advanced multi- level modeling��Thousand�Oaks,�CA:�Sage�
Steiger,�J��H�,�&�Fouladi,�R��T��(1992)��R2:�A�computer�program�in�interval�estimation,�power�calcula- tion,� and� hypothesis� testing� for� the� squared� multiple� correlation�� Behavior Research Methods, Instruments, and Computers,�4,�581–582�
Stevens,�J��P��(1984)��Outliers�and�influential�data�points�in�regression�analysis��Psychological Bulletin,� 95(2),�334–344�
Stevens,�J��P��(2002)��Applied multivariate statistics for the social sciences�(4th�ed�)��Mahwah,�NJ:�Lawrence� Erlbaum�Associates�
Stevens,� J�� P�� (2009)�� Applied multivariate statistics for the social sciences� (5th� ed�)�� New� York:� Routledge�
Stigler,�S��M��(1986)��The history of statistics: The measurement of uncertainty before 1900��Cambridge,�MA:� Harvard�
Storer,�B��E�,�&�Kim,�C��(1990)��Exact�properties�of�some�exact�test�statistics�for�comparing�two�bino- mial�proportions��Journal of the American Statistical Association,�85,�146–155�
Sudman,�S��(1976)��Applied sampling��New�York:�Academic� Tabachnick,�B��G�,�&�Fidell,�L��S��(2007)��Using multivariate statistics�(5th�ed�)��Boston:�Pearson� Tabatabai,�M�,�&�Tan,�W��(1985)��Some�comparative�studies�on�testing�parallelism�of�several�straight�
lines�under�heteroscedastic�variances��Communications in Statistics—Simulation and Computation,� 14,�837–844�
Thompson,�M��L��(1978)��Selection�of�variables�in�multiple�regression��Part�I:�A�review�and�evalua- tion�� Part� II:� Chosen� procedures,� computations� and� examples�� International Statistical Review,� 46,�1–19�and�129–146�
Tijms,� H�� (2004)�� Understanding probability: Chance rules in everyday life�� New� York:� Cambridge� University�Press�
Tiku,� M�� L�,� &� Singh,� M�� (1981)�� Robust� test� for� means� when� population� variances� are� unequal�� Communications in Statistics—Theory and Methods,�10,�2057–2071�
791References
Timm,�N��H��(2002)��Applied multivariate analysis��New�York:�Springer-Verlag� Timm,� N�� H�,� &� Carlson,� J�� E�� (1975)�� Analysis� of� variance� through� full� rank� models�� Multivariate
Behavioral Research Monographs,�No��75-1� Tomarken,�A�,�&�Serlin,�R��(1986)��Comparison�of�ANOVA�alternatives�under�variance�heterogeneity�
and�specific�noncentrality�structures��Psychological Bulletin,�99,�90–99� Tufte,�E��R��(1992)��The visual display of quantitative information��Cheshire,�CT:�Graphics�Press� Tukey,�J��W��(1949)��One�degree�of�freedom�for�nonadditivity��Biometrics,�5,�232–242� Tukey,�J��W��(1953)��The problem of multiple comparisons�(396pp)��Ditto:�Princeton�University� Tukey,�J��W��(1977)��Exploratory data analysis��Reading,�MA:�Addison-Wesley� Wainer,�H��(1984)��How�to�display�data�badly��The American Statistician,�38,�137–147� Wainer,�H��(1992)��Understanding�graphs�and�tables��Educational Researcher,�21,�14–23� Wainer,�H��(2000)��Visual revelations��Mahwah,�NJ:�Lawrence�Erlbaum�Associates� Wallgren,�A�,�Wallgren,�B�,�Persson,�R�,�Jorner,�U�,�&�Haaland,�J�-A��(1996)��Graphing statistics & data��
Thousand�Oaks,�CA:�Sage� Weinberg,� S�� L�,� &� Abramowitz,� S�� K�� (2002)�� Data analysis for the behavioral sciences using SPSS��
Cambridge,�U�K�:�Cambridge�University�Press� Weisberg,�H��I��(1979)��Statistical�adjustments�and�uncontrolled�studies��Psychological Bulletin,�86,�1149–1164� Weisberg,�S��(1985)��Applied linear regression�(2nd�ed�)��New�York:�Wiley� Welch,�B��L��(1951)��On�the�comparison�of�several�mean�values:�An�alternative�approach��Biometrika,�
38,�330–336� Wetherill,�G��B��(1986)��Regression analysis with applications��London:�Chapman�&�Hall� Wilcox,� R�� R�� (1986)�� Controlling� power� in� a� heteroscedastic� ANOVA� procedure�� British Journal of
Mathematical and Statistical Psychology,�39,�65–68� Wilcox,�R��R��(1987)��New statistical procedures for the social sciences: Modern solutions to basic problems��
Hillsdale,�NJ:�Lawrence�Erlbaum�Associates� Wilcox,� R�� R�� (1988)��A� new� alternative� to� the�ANOVA� F� and� new� results� on� James’� second-� order�
method��British Journal of Mathematical and Statistical Psychology,�41,�109–117 Wilcox,�R��R��(1989)��Adjusting�for�unequal�variances�when�comparing�means�in�one-way�and�two-
way�fixed�effects�ANOVA�models��Journal of Educational Statistics,�14,�269–278� Wilcox,� R�� R�� (1993)�� Comparing� one-step� M-estimators� of� location� when� there� are� more� than� two�
groups��Psychometrika,�58,�71–78� Wilcox,�R��R��(1996)��Statistics for the social sciences��San�Diego,�CA:�Academic� Wilcox,�R��R��(1997)��Introduction to robust estimation and hypothesis testing��San�Diego,�CA:�Academic� Wilcox,� R�� R�� (2002)�� Comparing� the� variances� of� two� independent� groups�� British Journal of
Mathematical and Statistical Psychology,�55,�169–175� Wilcox,�R��R��(2003)��Applying contemporary statistical procedures��San�Diego,�CA:�Academic� Wilkinson,�L��(2005)��The grammar of statistics�(2nd�ed�)��New�York:�Springer� Wonnacott,�T��H�,�&�Wonnacott,�R��J��(1981)��Regression: A second course in statistics��New�York:�Wiley� Wright,�R��E��(1995)��Logistic�regression��In�L��G��Grimm�&�P��R��Arnold�(Eds�)��Reading and understand-
ing multivariate statistics�(pp��217–244)��Washington,�DC:�American�Psychological�Association� Wu,� L�� L�� (1985)�� Robust� M-estimation� of� location� and� regression�� In� N�� B�� Tuma� (Ed�),� Sociological
methodology, 1985��San�Francisco:�Jossey-Bass� Xie,�X�-J�,�Pendergast,�J�,�&�Clarke,�W��(2008)��Increasing�the�power:�A�practical�approach�to�goodness-
of-fit�test�for�logistic�regression�models�with�continuous�predictors��Computational Statistics & Data Analysis,�52(5),�2703–2713�
Yu,�M��C�,�&�Dunn,�O��J��(1982)��Robust�tests�for�the�equality�of�two�correlation�coefficients:�A�Monte� Carlo�study��Educational and Psychological Measurement,�42,�987–1004�
Yuan,� K�-H�,� &� Maxwell�� S�� (2005)�� On� the� post� hoc� power� in� testing� mean� differences�� Journal of Educational and Behavioral Statistics,�30,�141–167�
Zimmerman,�D��W��(1997)��A�note�of�interpretation�of�the�paired-samples�t-test��Journal of Educational and Behavioral Statistics,�22,�349–360�
Zimmerman,� D�� W�� (2003)�� A� warning� about� the� large-sample� Wilcoxon-Mann-Whitney� test�� Understanding Statistics,�2,�267–280�
793
Odd-Numbered Answers to Problems
Chapter 1
Conceptual Problems
1.1� �Constant� (all� individuals� in� the� study� are� married;� thus,� the� marital� status� will� be� “married”�for�everyone�participating;�in�other�words,�there�is�no�variation�in�“marital� status”�for�this�particular�scenario)�
1.3� c�(true�ratios�cannot�be�formed�with�interval�variables)� 1.5� d�(true�ratios�can�only�be�formed�with�ratio�variables)� 1.7� �d�(an�absolute�value�of�zero�would�indicate�an�absence�of�what�was�measured—i�e�,�the�
number�of�years�playing�in�a�band—and�thus�ratio�is�the�scale�of�measure;�although� an� answer� of� zero� is� not� likely� given� that� the� students� in� the� band� are� those� being� measured,�if�someone�were�to�respond�with�an�answer�of�zero,�that�value�would�truly� indicate�“no�years�playing�an�instrument”)�
1.9� �True�(there�are�only�population�parameters�and�sample�statistics;�no�other�combina- tions�exist)�
1.11� �True�(categorical�variables�can�have�any�number�of�qualitative�values;�dichotomous� variables�are�limited�to�only�two�values)�
1.13� c�(equal�intervals�is�not�a�characteristic�of�an�ordinal�variable)� 1.15� No�(equal�intervals�is�not�a�characteristic�of�an�ordinal�variable)�
Computational Problems
1.1
Value Rank
10 7 15 5 12 6 8 8
20 2 17 4 5 9
21 1 3 10
19 3
794 Odd-Numbered Answers to Problems
1.3
Value Rank
8 6 6 8 3 10
12 4 19 3 7 7
10 5 25 2 4 9
42 1
Chapter 2
Conceptual Problems
2.1� c�(percentile�and�percentile�rank�are�two�sides�of�the�same�coin;�if�the�50th�percen- tile =�100,�then�PR(100)�=�50)�
2.3� a�(for�96,�crf�=��09�for�both�X�and�Y�and�crf�=��10�for�Z)� 2.5� d�(ethnicity�is�not�continuous,�so�only�a�bar�graph�is�appropriate)� 2.7� c�(see�Section�2�2�3)� 2.9� False�(the�proportion�is��25�by�definition)� 2.11� a�(eye�color�is�nominal�and�not�continuous)� 2.13� True�(with�the�same�interval�width,�each�is�based�on�exactly�the�same�information)� 2.15� No�(it�is�most�likely�that�Q1�will�be�smaller�for�the�negatively�skewed�variable)� 2.17� c�(if�the�relative�frequency�for�the�value�55�is�20%�and�for�70�is�30%,�the�cumulative�
relative�frequency�for�the�value�70�is�50%)�
Computational Problems
2.1� (a–d)�Frequency�Distributions:
X f cf rf crf
41 2 2 f/n�=�2/50�=��04 �04 42 2 4 �04 �08 43 4 8 �08 �16 44 5 13 �10 �26 45 6 19 �12 �38 46 8 27 �16 �54 47 11 38 �22 �76 48 4 42 �08 �84 49 5 47 �10 �94 50 3 50 �06 1�00
n�=�50 1�00
795Odd-Numbered Answers to Problems
x 50494847464544434241
Fr eq
ue nc
y
12
10
8
6
4
2
0
� (e)� Frequency�polygon � (g)� Q1�=�44�4,�Q2�=�46�25,�Q3�=�47�4545�(using�“values�are�group�midpoints”�option) � (h)� P10�=�42�75,�P90�=�49�1 � (i)� PR(41)�=�2%,�PR(49�5)�=�94% � (j)� Box-and-Whisker�plot
52504846444240
� (k)� Stem-and-leaf�display
Frequency Stem & Leaf
2.00 41 . 00 2.00 42 . 00 4.00 43 . 0000 5.00 44 . 00000 6.00 45 . 000000 8.00 46 . 00000000 11.00 47 . 00000000000 4.00 48 . 0000 5.00 49 . 00000 3.00 50 . 000
796 Odd-Numbered Answers to Problems
2.3 (a–c)�Q1�=�4�4,�Q2�=�5�375,�Q3�=�7�3333�(using�“values�are�group�midpoints”�option) � (d)� P44�5�=�5�169 � (e)� PR(7)�=�71�6667% � (f)� Box-and-Whisker�plot
.0 2.0 4.0 6.0 x
8.0 10.0
� (g)� Histogram
8.0
6.0
2.0
.0 .0 2.0 4.0 6.0
x 8.0 10.0 12.0
Mean = 5.8 Std. dev = 2.041
N = 30
4.0C ou
nt
797Odd-Numbered Answers to Problems
Chapter 3
Conceptual Problems
3.1� b�(will�affect�variance�the�most)� 3.3� d�(variance�cannot�be�negative)� 3.5� False�(that�proportion�is�always��25)� 3.7� No�(class�rank�is�ordinal,�so�mean�inappropriate)� 3.9� Yes�(middle�score�still�the�same)� 3.11� No�(will�be�different�for�small�samples)� 3.13� True�(they�are�based�on�the�same�measurement�scales)� 3.15� �No� (impossible� as,� by� nature� of� the� median� being� the� second� quartile,� the� median�
must�be�larger�than�the�first�quartile;�fire�the�statistician)� 3.17� �d�(range�as�it�is�computed�as�the�difference�between�the�two�extreme�values�in�the�
data)� 3.19 No�(interval�or�ratio�data�must�be�used�to�compute�the�variance)�
Computational Problems
3.1� �Mode� =� 47,� median� =� 46�25,� mean� =� 46,� exclusive� range� =� 9,� inclusive� range� =� 10,� H = 3�0546,�variance�=�5�28,�standard�deviation�=�2�2978�
3.3 Mode� =� 5,� median� =� 5�375,� mean� =� 5�80,� exclusive� range� =� 8,� inclusive� range� =� 9,� H = 2�9334,�variance�=�4�1655,�standard�deviation�=�2�041�
3.5� �Mode�=�12,�median�=�11�5,�mean�=�12,�exclusive�range�=�12,�inclusive�range�=�13,�H�= 2,� variance�=�8�0690,�standard�deviation�=�2�8406�
3.7� Distribution�Z�(it�has�more�extreme�scores�than�the�other�distributions)�
Chapter 4
Conceptual Problems
4.1� d�(skewness�is�zero�for�normal)� 4.3� b�(±2�standard�deviations)� 4.5� b�(only�median�is�a�value�of�X)� 4.7� c�(positive�value�=�leptokurtic)� 4.9� True�(see�z�score�equation)� 4.11� False�(mean�can�be�any�value)� 4.13� �a�(a�long�left�tail�due�to�the�substantial�negative�skewness,�and�a�very�flat�distribution,�
platykurtic,�due�to�the�large�negative�kurtosis�value)� 4.15� c�(where�there�is�the�highest�concentration�of�scores�in�the�middle)� 4.17� �False�(the�variance�of�z�is�always�1�while�the�variance�of�the�raw�scores�can�be�any�
non-negative�value)� 4.19� �a� (a� is� 90th� percentile,� b� is� 84th� percentile,� c� is� 75th� percentile,� and� d� is� 84th�
percentile)� 4.21� �a�(once�standardized�into�a�unit�normal�distribution,�the�mean�is�always�zero,�regard-
less�of�the�values�of�the�original�distribution)�
798 Odd-Numbered Answers to Problems
Computational Problems
4.1� a�=��0485,�b�=��6970,�c�=�10�16,�d�=�46�31,�e�=�approximately�79�67%,�f�=�approximately� 21�48%,�g�=�76�12%�
4.3� a�=��9332,�b�=��7611,�c�=�8�97,�d�=�104,200,�e�=�approximately�97�72%,�f�=�approximately� 62�93%,�g�=�78�87%�
Chapter 5
Conceptual Problems
5.1� c�(see�definition�in�Section�5�2�2)� 5.3� a�(2�out�of�9)� 5.5� a�(see�Section�5�2�2)� 5.7� True�(less�sampling�error�as�n�increases)� 5.9� False�(90%�CI�has�a�wider�range�than�68%�CI)� 5.11� Yes�(extreme�mean�more�likely�with�smaller�n)� 5.13� �b�(probability�of�winning�the�lottery�is�the�same�for�each�attempt,�regardless�of�how�
long�it�has�been�since�a�winner�was�announced)� 5.15� �c� (for� all� teachers� to� have� an� equal� and� independent� probability� of� being� selected,�
the� sampling� procedure� must� be� a� type� of� simple� random� sampling;� the� nature� of� Malani’s�research�is�such�that�this�should�be�done�without�replacement�as�she�would� not�want�to�survey�the�same�teacher�twice)�
5.17� c�(due�to�the�central�limit�theorem�with�large�size�samples)�
Computational Problems
5.1� �(a)� population� mean� =� 5;� population� variance� =� 6;� (b)� construct� table� of� possible� sample�means�like�Table�5�1;�(c)�mean�of�the�sampling�distribution�of�the�mean�=�5;� variance�of�the�sampling�distribution�of�the�mean�=�3�
5.3� 256� 5.5� Standard�error�of�the�mean�=��6325;�90%�CI�=�1�9595–4�0405�
Chapter 6
Conceptual Problems
6.1� c�(see�definition)� 6.3� b�(willing�to�reject�only�if�sample�mean�is�below�100)� 6.5� a�(cannot�make�Type�II�error�there)� 6.7� e�(most�extreme�value�regardless�of�sign)� 6.9� False�(cannot�make�a�Type�I�error�there)� 6.11� Yes�(the�p�value�is�less�than�the�alpha�level,�so�there�is�a�statistical�significance)� 6.13� No�(cannot�tell�just�from�mean�difference,�need�more�information)� 6.15� No�(the�range�will�be�wider�for�the�99%�CI)� 6.17� False�(the�mean�is�zero�for�any�t�distribution)� 6.19� True�(the�width�of�the�CI�only�depends�on�the�critical�value�and�the�standard�error)�
799Odd-Numbered Answers to Problems
Computational Problems
6.1� �(a)�B�may�or�may�not�reject;�(b)�A�also�rejects;�(c)�B�also�fails�to�reject;�(d)�A�may�or�may� not�fail�to�reject�
6.3� (a)�95th,�(b)�90th,�(c)�97�5th,�(d)�0,�(e)�0,�(f)�1�25,�(g)�1�761� 6.5� (a)�t�=�−�884,�critical�values�=�−2�093�and�+2�093,�fail�to�reject�H0; �� (b)�(2�3265,�3�2735),�includes�hypothesized�value�of�3�0�and�thus�fail�to�reject�H0�
Chapter 7
Conceptual Problems
7.1� �e� (if� null� hypothesis� is� true� and� you� reject,� then� you� have� definitely� made� a� Type� I� error)�
7.3� c�(see�definition)� 7.5� False�(sampling�error�is�less�for�larger�samples)� 7.7� Yes�(smaller�value�when�all�of�critical�region�is�in�one�tail;�see�t�table)� 7.9� d�(there�is�no�such�test;�the�tests�mentioned�all�deal�with�means)� 7.11� �a�(the�independent�t�test�is�appropriate�to�use�for�testing�mean�differences�between�
groups—as�is�the�case�here)� 7.13� No�(it�will�decrease,�as�shown�in�Table�A�2)� 7.15� �d�(homogeneity�of�variances,�via�Levene’s�test,�is�provided�by�default�in�SPSS�when�
conducting�the�independent�t�test)�
Computational Problems
7.1� �(a)�t�=�−2�1097,�critical�values�are�approximately�−2�041�and�+2�041,�reject�H0��(b)�(−9�2469,� −�1531),�does�not�include�hypothesized�value�of�0�and�thus�reject�H0�
7.3� �(a)�t�=�−3�185,�critical�values�are�−2�074�and�+2�074,�reject�H0��(b)�(−6�742,�−1�4248),�does� not�include�hypothesized�value�of�0�and�thus�reject�H0�
7.5� �(a)�t�=�4�117,�critical�values�are�−2�145�and�+2�145,�reject�H0��(b)�(9�7396,�30�9271),�does�not� include�hypothesized�value�of�0�and�thus�reject�H0�
7.7� t�=�2�4444,�critical�value�is�1�658,�reject�H0�
Chapter 8
Conceptual Problems
8.1� b�(4��6�=�24)� 8.3� True�(see�definition)� 8.5� No�(cannot�have�a�negative�proportion)� 8.7� No�(reject�when�test�statistic�exceeds�critical�value)� 8.9� �d� (as� the� difference� between� the� observed� and� expected� proportions� increases,� the�
chi-square�test�statistic�increases,�and,�thus,�we�are�more�likely�to�reject)� 8.11� �a� (chi-square� goodness-of-fit� test� given� there� is� only� one� variable� and� the� goal�
is� to� determine� if� the� proportions� within� the� categories� of� that� variable� are� the� same)�
800 Odd-Numbered Answers to Problems
Computational Problems
8.1� p�=��75,�z�=�2�1898,�critical�values�=�−1�96�and�+1�96,�thus�reject�H0� 8.3� z�=�−�1644,�critical�values�=�−1�96�and�+1�96,�thus�fail�to�reject�H0� 8.5� �Critical�value�=�9�48773,�fail�to�reject�H0�as�the�test�statistic�does�not�exceed�the�criti-
cal�value� 8.7� χ2�=��404,�critical�value�=�2�70554,�thus�fail�to�reject�H0�
Chapter 9
Conceptual Problems
9.1� c�(see�Section�9�4)� 9.3� Yes�(cannot�reject�if�sample�variances�are�equal)� 9.5� �Yes�(this�is�a�right-tailed�test�and�the�sample�variance�is�in�the�direction�of�the�right�
tail)� 9.7� No,�not�enough�information�(do�not�know�hypothesized�variance)� 9.9� b�(involves�naturally�occurring�couples�or�pairs)�
Computational Problems
9.1� �(a)�sample�variance�=�27�9292,�χ2�=�5�5858,�critical�values�=�7�2609�and�24�9958,�thus� reject�H0��(b)�(16�7603,�57�6978),�thus�reject�H0�as�the�interval�does�not�contain�75�
9.3� t�=�2�3474,�critical�values�=�−2�042�and�+2�042,�thus�reject�H0� 9.5� χ2�=�8�0,�critical�values�=�9�59078�and�34�1696,�thus�reject�H0� 9.7� t�=�−2�6178,�critical�values�=�−2�756�and�+2�756,�thus�fail�to�reject�H0�
Chapter 10
Conceptual Problems
10.1� d�[2/(3)(2)�=��3333]� 10.3� c�(weakest�means�correlation�nearest�to�0)� 10.5� �a� (a� linear� relationship� will� fall� into� a� reasonably� linear� scatterplot,� although� not�
necessarily�a�perfectly�straight�line)� 10.7� �False� (the� correlation� will� become� smaller;� see� the� correlation� equation� involving�
covariance)� 10.9� Yes�(a�perfect�relationship�implies�a�perfect�correlation,�assuming�linearity)� 10.11� �False� (in� negative� relationships,� the� higher� the� score� on� one� variable,� the� lower� the�
score�on�the�other�variable)� 10.13� �False�(a�correlation�simply�means�that�two�variables�are�related,�not�why�they�are�
related�and�not�because�there�is�definite�causation)� 10.15� �False� (the� Pearson� is� most� appropriate� for� interval/ratio� variables,� while� the�
Spearman’s�rho�or�Kendall’s�τ�are�most�appropriate�for�ordinal�variables)�
801Odd-Numbered Answers to Problems
Computational Problems
10.1� (a)�scatterplot�shown�in�the�following;�(b)�covariance�=�3�250;�(c)�r�=��631;�(d)�r�=��400�
7
6
5
C ar
ds _b
al an
ce
4
3
2
1
2 3 4 5 6 Cards_owned
7 8
10.3� t�=�3�9686,�critical�values�are�approximately�−2�048�and�+2�048,�fail�to�reject�H0� 10.5��(�a)�scatterplot�shown�in�the�following;�(b)�nonlinear�relationship;�(c)�r�=�approximately�
zero�
5
4
3
Bi lls
2
1
2 3 4 5 Coins
6 7
802 Odd-Numbered Answers to Problems
10.7� (a)�r�=��78;�(b)�strong�effect�
40
30
20
W or
ds re
ad
10
0
9 12 15 Letters written
18 21
Chapter 11
Conceptual Problems
11.1� a�(if�the�sample�means�are�all�equal,�then�MSbetw�is�0)� 11.3� c�(lose�1�df�from�each�group;�63�−�3�=�60)� 11.5� d�(equals�the�dfbetw�+�dfwith�=�dftotal;�60�+�2�=�62)� 11.7� d�(null�hypothesis�does�not�consider�SS�values)� 11.9� a�(for�between�source�=�5�−�1�=�4�and�for�within�source�=�250�−�5�=�245)� 11.11� c�(an�F�ratio�of�1�0�implies�that�between-�and�within-groups�variations�are�the�same)� 11.13� True�(mean�square�is�a�variance�estimate)� 11.15� True�(F�ratio�must�be�greater�than�or�equal�to�0)� 11.17� �No�(rejecting�the�null�hypothesis�in�ANOVA�only�indicates�that�there�is�some�differ-
ence�among�the�means,�not�that�all�of�the�means�are�different)� 11.19� c�(the�more�t�tests�conducted,�the�more�likely�a�Type�I�error�for�the�set�of�tests)� 11.21� True�(basically�the�definition�of�independence)� 11.23� No�(find�a�new�statistician�as�a�negative�F�value�is�not�possible�in�this�context)�
Computational Problems
11.1� �dfbetw�=�3,�dfwith�=�60,�dftotal�=�63,�SSwith�=�9�00,�MSbetw�=�3�25,�MSwith�=�0�15,�F�=�21�6666,� critical�value�=�2�76�(reject�H0)�
11.3� �SSbetw�=�150,�SStotal�=�1110,�dfbetw�=�3,�dfwith�=�96,�dftotal�=�99,�MSbetw�=�50,�MSwith�=�10,� critical�value�approximately�2�7�(reject�H0)�
11.5� �SSbetw� =� 25�333,� SSwith� =� 27�625,� SStotal� =� 52�958,� df betw� =� 2,� dfwith� =� 21,� dftotal� =� 23,� MSbetw�=�12�667,�MSwith�=�1�315,�F�=�9�629,�critical�value�=�3�47�(reject�H0)�
803Odd-Numbered Answers to Problems
Chapter 12
Conceptual Problems
12.1� False�(requires�equal�n’s�and�equal�variances;�we�hope�the�means�are�different)� 12.3� c�(c�is�not�legitimate�as�the�contrast�coefficients�do�not�sum�to�0)� 12.5� a�(see�flowchart�of�MCPs�in�Figure�12�2)� 12.7� False�(use�Dunnett�procedure)� 12.9� e�(Scheffe’�is�most�flexible�of�all�MCPs;�can�test�simple�and�complex�contrasts)� 12.11� False�(conducted�to�determine�why�null�has�been�rejected)� 12.13� True�(see�characteristics�of�Tukey�HSD)� 12.15� a�(see�Figure�12�2)� 12.17� �Yes� (each� contrast� is� orthogonal� to� the� others� as� they� rely� on� independent�
information)� 12.19� d�(see�Figure�12�2)� 12.21� No�(do�not�know�the�values�of�the�standard�error,�t,�critical�value,�etc�)�
Computational Problems
12.1� �Contrast� =� −5,� standard� error� =� 1;� t� =� −5,� critical� values� are� 5�10� and� −5�10,� fail� to� reject�
12.3� Standard�error�=� 60 20 3 1 7321= = . :
•� q1�=�(85�−�50)/1�7321�=�20�2073� •� q2�=�(85�−�70)/1�7321�=�8�6603� •� q3�=�(70�−�50)/1�7321�=�11�5470� •� Critical� values� approximately� 3�39� and� −3�39;� all� contrasts� are� statistically�
significant� 12.5� (a)� μ�1�−�μ�2,�μ�3�−�μ�4,�(μ�1�+�μ�2)/2�−�(μ�3�+�μ�4)/2;�all�of�the�Σcj�are�equal�to�0� � (b)� No,�as�Σcj�is�not�equal�to�0� � (c)� H0:�μ�1�−�[(μ�2�+�μ�3�+�μ�4)/3]�
Chapter 13
Conceptual Problems
13.1� c�(a�plot�of�the�cell�means�reveals�an�interaction)� 13.3� �b�(product�of�number�of�degrees�of�freedom�for�each�main�effect;�(J�−�1)(K�−�1)�=�(2)
(2)�=�4)� 13.5� d�(p�less�than�alpha�only�for�the�interaction�term)� 13.7� c�(c�is�one�definition�of�an�interaction)� 13.9� b�(interaction�df�=�product�of�main�effects�df )� 13.11� �d�(the�effect�of�one�factor�depends�on�the�second�factor;�see�definition�of�interaction�
as�well�as�example�profile�plots�in�Figure�13�1)� 13.13� �False� (when� the� interaction� is� significant,� this� implies� nothing� about� the� main�
effects)�
804 Odd-Numbered Answers to Problems
13.15� No�(the�numerator�degrees�of�freedom�for�factor�B�can�be�anything)� 13.17� e�(3�levels�of�A,�2�levels�of�B,�thus�6�cells)� 13.19� a�(check�F�table�for�critical�values;�only�reject�main�effect�for�factor�A)� 13.21� b�(as�dftotal�=�14,�then�total�sample�size�=�15)�
Computational Problems
13.1� �SSwith�=�225;�dfA�=�1;�dfB�=�2;�dfAB�=�2;�dfwith�=�150;�dftotal�=�155;�MSA�=�6�15;�MSB�=�5�30;� MSAB�=�4�55;�MSwith�=�1�50;�FA�=�4�10;�FB�=�3�5333;�FAB�=�3�0333;�critical�value�for�A�is� approximately�3�91,�thus�reject�H0�for�A;�critical�value�for�B�and�AB�approximately� 3�06,�thus�reject�H0�for�B�and�fail�to�reject�H0�for�AB�
13.3� See�the�following�completed�table:
Source SS df MS F Critical Value Decision
A 14�06 1 14�06 �25 4�75 Fail�to�reject�H0 B 39�06 1 39�06 �70 4�75 Fail�to�reject�H0 AB 1�56 1 1�56 �03 4�75 Fail�to�reject�H0 Within 668�75 12 55�73 Total 723�43 15
13.5� �FA�=�4�0541,�FB�=�210�1622,�FC�=�31�7838,�FAB�=�7�9459,�FAC�=�13�1351,�FBC�=�10�3784,�FABC�=� 4�0541,�all�but�ABC�and�A�are�significant�
Chapter 14
Conceptual Problems
14.1� No�(there�is�no�covariate�mentioned�for�which�to�control)� 14.3� �c�(evidence�of�meeting�the�assumption�of�independence�can�be�examined�by�a�scat-
terplot� of� residuals� by� group� or� category� of� the� independent� variable;� a� random� display�of�points�suggests�the�assumption�is�met)�
14.5� b�(see�discussion�on�homogeneity�of�regression�slopes)� 14.7� b�(14�df�per�group,�3�groups,�42�df�−�2�df�for�covariates�=�40)� 14.9� c�(want�covariate�having�a�high�correlation�with�the�dependent�variable)� 14.11� �c� (the� covariate� and� dependent� variable� need� not� be� the� same� measure;� could� be�
pretest�and�posttest,�but�does�not�have�to�be)� 14.13� �b� (an� interaction� indicates� that� the� regression� lines� are� not� parallel� across� the�
groups)� 14.15� �c�(a�post�hoc�covariate�typically�results�in�an�underestimate�of�the�treatment�effect,�
due�to�confounding�or�interference�of�the�covariate)� 14.17� �No�(if�the�correlation�is�substantial,�then�error�variance�will�be�reduced�in�ANCOVA�
regardless�of�its�sign)� 14.19� b�(11�df�per�group,�6�groups,�66�df�−�1�df�for�covariate�=�65)� 14.21� �No�(there�will�be�no�adjustment�due�to�the�covariate�and�one�df�will�be�lost�from�the�
error�term)�
805Odd-Numbered Answers to Problems
Computational Problems
14.1� �The�adjusted�group�means�are�all�equal�to�150;�this�resulted�because�the�adjust- ment�moved�the�mean�for�Group�1�up�to�150�and�the�mean�for�Group�3�down�to� 150�
14.3� �ANOVA� results:� SSbetw� =� 4,763�275,� SSwith� =� 9,636�7,� dfbetw� =� 3,� dfwith� =� 36,� MSbetw� =� 1,587�758,� MSwith� =� 267�686,� F� =� 5�931,� critical� value� approximately� 2�88� (reject� H0)�� Unadjusted�means�in�order:�32�5,�60�4,�53�1,�39�9�
ANCOVA�results:�SSbetw�=�5402�046,�SSwith�=�3880�115,�dfbetw�=�3,�dfwith�=�35,�MSbetw�=� 1800�682,�MSwith�=�110�8604,�F�=�16�24,�critical�value�approximately�2�88�(reject�H0),� SScov�=�5117�815,�Fcov�=�46�164,�critical�value�approximately�4�12�(reject�H0)�
Adjusted�means�in�order:�30�7617,�61�2544,�53�1295,�40�7544�
Chapter 15
Conceptual Problems
15.1� b�(when�there�are�both�random�and�fixed�factors,�then�the�design�is�mixed)� 15.3� c�(gender�is�fixed,�order�is�random,�thus�a�mixed-effects�model)� 15.5� �a�(clinics�were�randomly�selected�from�the�population;�thus,�the�one-factor�random-
effects�model�is�appropriate)� 15.7� �False�(the�F�ratio�will�be�the�same�for�both�the�one-factor�random-�and�fixed-effects�
models)� 15.9� �Yes�(the�test�of�the�interaction�is�exactly�the�same�for�both�models�yielding�the�same�
F�ratio)� 15.11� �Yes� (SStotal� is� the� same� for� both� models;� the� total� amount� of� variation� is� the�
same;�it�is�just�divided�up�in�different�ways;�review�the�example�dataset�in�this� chapter)�
15.13� �c�(see�definition�of�design)� 15.15� �True� (rarely� is� one� interested� in� particular� students;� thus,� students� are� usually�
random)� 15.17� �False� (the� F� test� is� not� very� robust� in� this� situation� and� we� should� be� concerned�
about�it)�
Computational Problems
15.1� �SSwith�=�1�9,�dfA�=�2,�dfB�=�1,�dfAB�=�2,�dfwith�=�18,�dftotal�=�23,�MSA�=�1�82,�MSB�=��57,�MSAB =� 1�035,� MSwith� =� �1056,� FA� =� 1�7585,� FB� =� 5�3977,� FAB� =� 9�8011,� critical� value� for� AB� =� 6�01� (reject�H0�for�AB),�critical�value�for�B�=�8�29�(fail�to�reject�H0�for�B),�critical�value�for�A�=� 99�(fail�to�reject�H0�for�A)�
15.3� �SStime�=�126�094,�SStime��program�=�2�594,�SSprogram�=�3�781,�MStime�=�42�031,�MStime��program =� 0�865,�MSprogram�=�3�781,�Ftime�=�43�078�(p�<��001),�Ftime��program�=�0�886�(p�>��05),�Fprogram�=� 0�978�(p�>��05)�
15.5� �SStime� =� 691�467,� SStime� � mentor� =� 550�400,� SSmentor� =� 1968�300,� MStime� =� 345�733,� MStime� � mentor�=�275�200,�MSmentor�=�1968�300,�Ftime�=�2�719�(p�=��096),�Ftime��mentor�=�2�164�(p�=��147),� Fmentor�=�7�073�(p�<��001)�
806 Odd-Numbered Answers to Problems
Chapter 16
Conceptual Problems
16.1� �d�(teachers� are�ranked�according�to�a�ratio�blocking�variable;� a�random�sample�of� blocks�are�drawn;�then�teachers�within�the�blocks�are�assigned�to�treatment)�
16.3� a�(children�are�randomly�assigned�to�treatment�based�on�ordinal�SES�value)� 16.5� d�(interactions�only�occur�among�factors�that�are�crossed)� 16.7� a�(this�is�the�notation�for�teachers�nested�within�methods;�see�also�Problem�16�2)� 16.9� False�(cannot�be�a�nested�design;�must�be�a�crossed�design)� 16.11� Yes�(see�the�discussion�on�the�types�of�blocking)� 16.13� c�(physician�is�nested�within�method)� 16.15� Yes�(age�is�an�appropriate�blocking�factor�here)� 16.17� a�(use�of�a�covariate�is�best�for�large�correlations)� 16.19� a�(see�the�summary�of�the�blocking�methods)�
Computational Problems
16.1� �(a)�Yes�(b)�at�age�4�type�1�is�most�effective,�at�age�6�type�2�is�most�effective,�and�at�age� 8�type�2�is�most�effective�
16.3� �SStotal� =� 560,� dfA� =� 2,� dfB� =� 1,� dfAB� =� 2,� dfwith� =� 24,� dftotal� =� 29,� MSA� =� 100,� MSB� =� 100,� MSAB =�10,�MSwith�=�10,�FA�=�10,�FB�=�10,�FAB�=�1,�critical�value�for�B�=�4�26�(reject�H0� for B),�critical�value�for�A�and�AB�=�3�40�(reject�H0�for�A�and�fail�to�reject�H0�for�AB)�
16.5� �Fsection� =� 44�385,� p� =� �002;� FGRE-Q� =� 61�000,� p� =� �001;� thus� reject� H0� for� both� effects;� Bonferroni�results:�all�but�sections�1�and�2�are�different,�and�all�but�blocks�1�and�2� are�statistically�different�
Chapter 17
Conceptual Problems
17.1� c�(see�definition�of�intercept;�a�and�b�refer�to�the�slope�and�d�to�the�correlation)� 17.3� �c�(the�intercept�is�37,000�which�represents�average�salary�when�cumulative�GPA�
is�zero)� 17.5� �a�(the�predicted�value�is�a�constant�mean�value�of�14�regardless�of�X;�thus,�the�vari-
ance�of�the�predicted�values�is�0)� 17.7� �d� (linear� relationships� are� best� represented� by� a� straight� line,� although� all� of� the�
points�need�not�fall�on�the�line)� 17.9� a�(if�the�slope�=�0,�then�the�correlation�=�0)� 17.11� �b�(with�the�same�predictor�score,�they�will�have�the�same�residual�score;�whether�the�
residuals�are�the�same�will�only�depend�on�the�observed�Y)� 17.13� d�(see�definition�of�homogeneity)� 17.15� �d�(various�pieces�of�evidence�for�normality�can�be�assessed,�including�formal�tests�
such�as�the�Shapiro–Wilk�test)� 17.17 True�(the�value�of�Y�is�irrelevant�when�the�correlation�=�0,�so�the�mean�of�Y�is�the�
best�prediction)�
807Odd-Numbered Answers to Problems
17.19� �False�(if�the�variables�are�positively�correlated,�then�the�slope�would�be�positive�and� a�low�score�on�the�pretest�would�predict�a�low�score�on�the�posttest)�
17.21� No�(the�regression�equation�may�generate�any�number�of�points�on�the�regression�line)�
Computational Problems
17.1� a�—�b�(slope)�=��8571,�a�(intercept)�=�1�9716;�b�—Y�(outcome)�=�7�1142� 17.3� 118�
Chapter 18
Conceptual Problems
18.1� b�(partial�correlations�correlate�two�variables�while�holding�constant�a�third)� 18.3� c�(perfect�prediction�when�the�standard�error�=�0)� 18.5 False�(adding�an�additional�predictor�can�result�in�the�same�R2)� 18.7� No�(R2�is�higher�when�the�predictors�are�uncorrelated)� 18.9� �c� (given� there� is� theoretical� support,� the� best� method� of� selection� is� hierarchical�
regression)� 18.11� �No�(the�purpose�of�the�adjustment�is�to�take�the�number�of�predictors�into�account;�
thus�Radj 2 �may�actually�be�smaller�for�the�most�predictors)�
Computational Problems
18.1� �Intercept�=�28�0952,�b1�=��0381,�b2�=��8333,�SSres�=�21�4294,�SSreg�=�1128�5706,�F�=�105�3292� (reject�at��01),�s2res�=�5�3574,�s(b1)�=��0058,�s(b2)�=��1545,�t1�=�6�5343�(reject�at��01),�t2�= 5�3923� (reject�at��01)�
18.3� In�order,�the�t�values�are�0�8�(not�significant),�0�77�(not�significant),�−8�33�(significant)� 18.5� r1(2�3)�=�−�0140� 18.7� r12�3�=�−�8412,�r1(2�3)�=�−�5047� 18.9� �Intercept�=�−1�2360,�b1�=��6737,�b2�=��6184,�SSres�=�58�3275,�SSreg�=�106�6725,�F�=�15�5453�
(reject�at��05),�s2res�=�3�4310,�s(b1)�=��1611,�s(b2)�=��2030,�t1�=�4�1819�(reject�at��05),�t2�=�3�0463� (reject�at��05)�
Chapter 19
Conceptual Problems
19.1� c—The�measurement�scale�of�the�dependent�variable� 19.3� �a—Employment�status�(employed;�unemployed,�not�looking�for�work;�unemployed,�
looking�for�work)�as�there�are�more�than�two�groups�or�categories� 19.5� a—True� 19.7� a—The�log�odds�become�larger�as�the�odds�increase�from�1�to�100� 19.9� d—Wald�test�(assesses�significance�of�individual�predictors)�
Computational Problems
19.1� �−2LL�=�7�558,�bHSGPA�=�−�366,�bathlete�=�22�327,�bconstant�=��219,�se(bHSGPA)�=�1�309,�se(bathlete)�=� 20006�861,�odds�ratioHSGPA�=��693,�odds�ratioathlete�<��001,�WaldHSGPA�=��078,�Waldathlete�=��000�
809
Author Index
A
Abramowitz,�S�K�,�311,�680,�703 Agresti,�A�,�440,�498,�499 Aldrich,�J�H�,�718,�719 Algina,�J�,�139,�250,�310,�311,�313 Anderson,�R�E�,�495,�640,�691 Anderson,�S�L�,�437 Andrews,�D�F�,�630,�672 Applebaum,�M�I�,�394 Arnold,�P�R�,�751,�752 Atiqullah,�M�,�437 Atkinson,�A�C�,�628
B
Babin,�B�J�,�495,�640,�691 Barnett,�V�,�630,�640,�691 Barrett,�K�C�,�331 Basu,�S�,�143 Bates,�D�M�,�680 Beal,�S�L�,�213 Beckman,�R�,�630 Belsley,�D�A�,�630,�672 Benjamini,�Y�,�361 Bernstein,�I�H�,�679 Berry,�W�D�,�680 Black,�W�C�,�495,�640,�691 Blair,�R�C�,�171,�187,�195,�250 Boik,�R�J�,�383,�571 Bosker,�R�J�,�751 Bowden,�D�,�358 Box,�G�E�P�,�310,�311,�381,�437,�497,�505,�571 Bradley,�J�V�,�310,�311 Braver,�S�L�,�331,�399,�444,�508,�562 Brown,�M�B�,�310,�312,�313,�358 Brunner,�E�,�381 Bryant,�J�L�,�436,�782
C
Campbell,�D�T�,�270,�295,�430,�443 Carlson,�J�E�,�394 Carroll,�R�J�,�630,�672 Chakravart,�I�M�,�148 Chambers,�J�M�,�29 Chatterjee,�S�,�675
Christensen,�R�,�751 Clarke,�W�,�716 Clark,�V�A�,�394,�431,�498,�561,�562,�565,�628,�
630, 679 Cleveland,�W�S�,�29 Clinch,�J�J�,�313 Coe,�P�R�,�213 Cohen,�J�,�137,�139,�155,�156,�168,�169,�177,�196–198,�
211,�224,�231,�234,�235,�265,�267,�269,�272,� 274,�275,�278,�283,�286,�303,�304,�307,�331,� 384,�416,�436,�439,�468,�478,�548,�622,�647,� 649,�652,�667,�668,�679,�680,�698,�701,�703,� 746,�750,�751
Cohen,�P�,�439,�679,�680,�751 Conerly,�M�D�,�674 Conover,�W�,�175,�180,�444 Cook,�R�D�,�628,�630 Cook,�T�D�,�270,�295,�430 Coombs,�W�T�,�250,�310,�311,�313 Cotton,�J�W�,�499,�508 Cox,�D�R�,�311,�718 Craig,�A�T�,�267 Cramer,�E�M�,�394 Crepel,�P�,�4,�5 Croux,�C�,�724 Cumming,�G�,�139
D
D’Agostino,�R�B�,�672 DasGupta,�A�,�143 Davenport,�J�M�,�309,�437 Delaney,�H�D�,�439,�440,�575 de�Leeuw,�J�,�751 Derksen,�S�,�679 Detta,�H�,�381 Dill,�C�A�,�575 Duan,�N�,�751 Duncan,�G�T�,�269 Dunnett,�C�W�,�354,�355,�361 Dunn,�O�J�,�269,�355,�357,�394,�431,�498,�561,�562,�
565,�628,�630,�679 Durbin,�J�,�309,�437,�628
E
Elashoff,�J�D�,�431
810 Author Index
F
Feinberg,�S�E�,�4,�5 Feldman,�S�,�680 Feldt,�L�S�,�495,�497,�505,�569,�571,�572,�575 Ferguson,�G�A�,�444 Fidell,�L�S�,�495,�679 Fidler,�F�,�304,�384,�478 Finch,�S�,�139 Fink,�A�,�111 Finlay,�B�,�440 Fisher,�R�A�,�361 Flandre,�C�,�724 Forsythe,�A�,�310,�312,�313,�358 Fouladi,�R�T�,�622,�667 Friedman,�M�,�498,�574
G
Games,�P�A�,�361 Gamst,�G�,�637,�680,�703,�751,�752 Gani,�J�,�4,�5 Geisser,�S�,�497,�505,�571 Ghosh,�B�K�,�211 Glass,�G�V�,�171,�187,�195,�221,�309–311,�394,�440,�
508,�560,�562,�567,�633,�660,�675,�751 Gloeckner,�G�W�,�331 Greenhouse,�S�,�497,�505,�571 Griego,�O�V�,�637,�703 Grimm,�L�G�,�751,�752 Grissom,�R�J�,�139 Guarino,�A�J�,�637,�680,�703,�751,�752
H
Haaland,�J�-A�,�29 Haesbroeck,�G�,�724 Hair,�J�F�,�495,�640,�691 Harlow,�L�,�139 Harrell,�F�E�J�,�718 Harwell,�M�,�437,�438,�440,�444,�469 Hawkins,�D�M�,�630 Hays,�W�L�,�570,�660 Hayter,�A�J�,�361 Heck,�R�H�,�576,�751 Heisey,�D�M�,�138 Hellevik,�O�,�710 Hershberger,�S�L�,�752 Heyde,�C�C�,�4,�5 Hochberg,�Y�,�361,�436,�498 Hocking,�R�R�,�677 Hoenig,�J�M�,�138 Hoerl,�A�E�,�675 Hogg,�R�V�,�267
Hopkins,�K�D�,�221,�311,�394,�508,�560,�562,�567,� 633,�660,�675,�751
Hornick,�C�W�,�394 Hosmer,�D�W�,�717,�718,�722,�726,�751 Hosmer,�T�,�717 Howell,�D�,�274 Howell,�J�F�,�361 Huberty,�C�J�,�168,�169,�679 Huck,�S�W�,�576,�751 Huitema,�B�E�,�431,�436,�438–440,�444,�576 Huynh,�H�,�495,�497,�505,�569,�571
I
Iman,�R�,�175,�180,�444
J
Jaeger,�R�M�,�111 James,�G�S�,�312,�313 Jennings,�E�,�576 Johansen,�S�,�381 Johnson,�P�O�,�440 Johnson,�R�A�,�752 Jorner,�U�,�29
K
Kaiser,�L�,�358 Kalton,�G�,�111 Kennard,�R�W�,�675 Keppel,�G�,�304,�309,�311,�345,�355,�357,�361,�383,�
384,�390,�393,�394,�431,�438,�439,�444,�478,� 493,�495,�498,�499,�505,�508,�562,�565,�567,� 569,�572,�576
Keselman,�H�J�,�139,�313,�679 Kim,�C�,�213 Kim,�J�J�,�139 Kirk,�R�E�,�308,�352,�354,�357,�358,�361,�394,�444,�495,�
498,�505,�508,�561,�565,�569–571,�574,�576 Kleinbaum,�D�G�,�629,�630,�672,�675,�679,�680,�
751, 752 Kleiner,�B�,�29 Kramer,�C�Y�,�360 Kreft,�I�,�751 Kruskal,�W�H�,�312 Kuh,�E�,�630,�672 Kupper,�L�L�,�629,�630,�672,�675,�679,�680,�751,�752
L
Laha,�R�G�,�148 Lamb,�G�S�,�272 Larsen,�W�A�,�674
811Author Index
Layard,�M�W�J�,�269 LeCessie,�S�,�717 Leech,�N�L�,�331 Lee,�D�M�,�394 Lemeshow,�S�,�111,�717,�718,�722,�726,�751 Leroy,�A�M�,�630,�672 Levin,�J�R�,�383 Levy,�P�S�,�111 Lewis,�T�,�630,�640,�691 Li,�J�,�448,�514,�580 Lilliefors,�H�,�148 Lomax,�R�G�,�448,�514,�580 Long,�J�S�,�725 Lord,�F�M�,�439,�444
M
MacKinnon,�D�P�,�331,�399,�444,�508,�562 Manly,�B�F�J�,�752 Mansfield,�E�R�,�674 Marascuilo,�L�A�,�269,�383,�390,�499,�574 Marcoulides,�G�A�,�752 Marquardt,�D�W�,�675 Maxwell,�S�E�,�138,�439,�440,�498,�571,�575 McCleary,�S�J�,�674 McCoach,�D�B�,�751 McCulloch,�C�E�,�508 McLean,�R�A�,�576 McSweeney,�M�,�499,�574 Menard,�S�,�712,�737 Mendoza,�J�L�,�622,�667 Meyers,�L�S�,�637,�680,�703,�751,�752 Mickey,�R�M�,�394,�431,�498,�562,�630,�679 Miller,�A�J�,�679 Miller,�R�G�,�381,�436,�672 Morgan,�G�A�,�331,�637,�703 Mosteller,�F�,�311 Mulaik,�S�,�139 Muller,�K�E�,�629,�630,�672,�675,�679,�680,�751,�752 Munk,�A�,�381 Murphy,�K�R�,�137,�304,�384,�478,�493,�668 Myers,�J�L�,�311,�313,�390,�431,�444,�495,�498,�499,�
508,�565,�572,�576,�633,�675 Myers,�R�H�,�498,�508,�565,�572,�672,�675 Myors,�B�,�137,�304,�384,�478,�493,�668
N
Nagelkerke,�N�J�D�,�718 Nelson,�F�D�,�718,�719 Neyman,�J�,�440 Nizam,�A�,�629,�630,�672,�675,�679,�680,�751,�752 Noreen,�E�W�,�142
O
O’Connell,�A�A�,�751 O’Grady,�K�E�,�304,�384 Olejnik,�S�F�,�250 Ottman,�D�O�,�310,�311,�313 Overall,�J�E�,�394
P
Page,�M�C�,�331,�399,�444,�508,�562 Pampel,�F�C�,�714,�718,�721,�751 Paulson,�A�S�,�436,�782 Pavur,�R�,�349 Pearson,�E�S�,�4,�5 Peckham,�P�D�,�171,�187,�195,�309,�310,�440 Pedhazur,�E�J�,�348,�439,�618,�630,�633,�660,�665,�
675,�679,�680,�751 Pendergast,�J�,�498,�499,�716 Penfield,�R�D�,�139 Persson,�R�,�29 Pingel,�L�A�,�571,�572 Porter,�A�C�,�439,�444 Pregibon,�D�,�630 Price,�B�,�675 Puri,�M�L�,�444
Q
Quade,�D�,�444
R
Raferty,�A�E�,�721 Ramsey,�P�H�,�250,�273 Raudenbush,�S�W�,�444 Reichardt,�C�S�,�439,�576 Reise,�S�P�,�751 Robbins,�N�B�,�29 Rogosa,�D�R�,�440 Rosenthal,�R�,�355 Rosnow,�R�L�,�355 Rousseeuw,�P�J�,�630,�672 Roy,�J�,�148 Rudas,�T�,�106 Ruppert,�D�,�630,�672 Rutherford,�A�,�444
S
Sanders,�J�R�,�171,�187,�195,�309,�310,�440 Sawilowsky,�S�S�,�171,�187,�195 Scariano,�S�M�,�309,�437
812 Author Index
Schafer,�W�D�,�679 Scheffé,�H�,�357 Schmid,�C�F�,�29 Seber,�G�A�F�,�680 Seneta,�E�,�4,�5 Sen,�P�K�,�444 Serlin,�R�C�,�269,�313,�390 Shadish,�W�R�,�270,�295,�430 Shapiro,�S�S�,�148,�171,�186,�191,�323,�407,�437,�457,�
541,�587,�600,�644,�672,�694 Shavelson,�R�J�,�506 Sidak,�Z�,�357 Singh,�M�,�171 Smithson,�M�,�304,�384,�622,�667 Snee,�R�D�,�675 Snell,�E�J�,�718 Snijders,�T�A�B,�751 Spiegel,�D�K�,�394 Stafford,�K�L�,�622,�667 Stanley,�J�C�,�295,�430,�443 Steiger,�J�H�,�139,�622,�667 Stevens,�J�P�,�640,�675,�690,�752 Stigler,�S�M�,�4,�5 Storer,�B�E�,�213 Sudman,�S�,�111 Surman,�S�H�,�514
T
Tabachnick,�B�G�,�495,�679 Tabatabai,�M�,�440 Tabata,�L�N�,�576 Takane,�Y�,�444 Tamhane,�A�C�,�213,�436,�498 Tan,�W�,�440 Tatham,�R�L�,�495,�640,�691 Thomas,�S�L�,�576,�751 Thompson,�B�,�304,�384,�478 Thompson,�M�L�,�677 Tijms,�H�,�106 Tiku,�M�L�,�171 Timm,�N�H�,�394,�570,�752 Tomarken,�A�,�313 Tufte,�E�R�,�24,�29 Tukey,�J�W�,�28,�33,�58,�311,�358,�360,�570,�573 Tukey,�P�A�,�29
V
Varon-Salomon,�Y�,�436
W
Wainer,�H�,�29 Wallgren,�A�,�29 Wallgren,�B�,�29 Wallis,�W�A�,�312 Watson,�G�S�,�309,�437,�628 Watts,�D�G�,�680 Weinberg,�S�L�,�311,�680,�703 Weisberg,�H�I�,�439 Weisberg,�S�,�628,�629,�672,�674,�675,�679,�680 Welch,�B�L�,�313,�381 Well,�A�D�,�311,�313,�390,�431,�444,�495,�498,�499,�
508,�565,�572,�576,�633,�675 Welsch,�R�E�,�630,�672 Wetherill,�G�B�,�675 Wichern,�D�W�,�752 Wickens,�T�D�,�304,�309,�311,�355,�357,�361,�
383,�384,�390,�393,�394,�431,�438,� 439,�444,�478,�493,�498,�499,�505,� 508,�562,�565,�567,�572,�576
Wilcox,�R�R�,�142,�143,�171,�175,�180,�211,�245,� 247,�249,�268,�269,�271,�272,�304,�310,� 311,�313,�355,�357,�358,�361,�381,�384,� 440,�444,�478,�482,�498,�499,�625,� 629,�630,�632,�672,�679
Wild,�C�J�,�680 Wilkinson,�L�,�29 Wilk,�M�B�,�148,�171,�186,�191,�323,�407,�437,�457,�
541,�587,�600,�644,�672,�694 Wolach,�A�,�137,�304,�384,�478,�493 Wonnacott,�R�J�,�675 Wonnacott,�T�H�,�675 Wright,�R�E�,�751 Wu,�L�L�,�630,�672
X
Xie,�X�-J�,�716
Y
Yuan,�K�-H�,�138 Yu,�M�C�,�269
Z
Zimmerman,�D�W�,�172,�175,�309
813
Subject Index
A
Additive�effects,�ANOVA,�380 Additive�model,�569 All�possible�subsets�regression,�678 Analysis�of�covariance�(ANCOVA)
adjusted�means�and�related�procedures,� 434–436
assumptions�and�violation�of�assumptions,� 436–441
characteristics,�428–431 example,�441–443 G*Power,�445–469 layout�of�data,�431 more�complex�models,�444 nonparametric�procedures,�444 one-factor�fixed-effects�model,�431–432 partitioning�the�sums�of�squares,�433 population�parameters,�431–432 SPSS,�445–469 summary�table,�432–433 template�and�APA-style�paragraph,�
469–471 without�randomization,�443–444
Analysis�of�variance�(ANOVA) alternative�procedures
Brown–Forsythe�procedures,�313 James�procedures,�313 Kruskal–Wallis�test,�312–313 Welch�procedures,�313
vs��ANCOVA,�575–576 assumptions�and�violation�of�assumptions,�
309–312,�380–381 characteristics�of�one-factor�model,�292–296 effect�size�measures,�confidence�intervals,�
and�power,�303–304,�383–384 examples,�304–307,�384–389 factorial
SPSS,�395–417 template�and�APA-style�write-up,�
417–419 three-factor�and�higher-order,�390–393 two-factor�model,�372–390 with�unequal�n’s,�393–394
Friedman�test,�574 layout�of�data,�296 model,�302–309
multiple�comparison�procedures,�382–383 one-factor�fixed-effects�model,�291–336 one-factor�random-effects�model
assumptions�and�violation� of assumptions,�482–483
characteristics,�479–480 hypotheses,�480 multiple�comparison�procedures,�483 population�parameters,�480 residual�error,�480 SPSS�and�G*Power,�508–513 summary�table�and�mean�squares,�
481–482 one-factor�repeated�measures�design
assumptions�and�violation� of assumptions,�495–496
characteristics,�493–494 example,�499–500 Friedman�test,�498–499 hypotheses,�495 layout�of�data,�494 multiple�comparison�procedures,�498 population�parameters,�494 residual�error,�494 SPSS�and�G*Power,�515–524 summary�table�and�mean�squares,�
496–498 parameters�of�the�model,�302–303 partitioning�the�sums�of�squares,�299,�381 summary�table,�300–301,�381–382,�
391–392 template�and�APA-style�write-up,�548–551,�
603–605 theory,�296–302 three�factor�and�higher-order,�390–393 triple�interaction,�393 two-factor�hierarchical�model
characteristics,�559–561 example,�565–566 hypotheses,�562–563 layout�of�data,�561 multiple�comparison�procedures,�565 nested�factor,�562 population�parameters,�562 SPSS,�576–581 summary�table�and�mean�squares,�
563–565
814 Subject Index
two-factor�mixed-effects�model assumptions�and�violation�
of assumptions,�492 characteristics,�488 hypotheses,�489–490 multiple�comparison�procedures,�492–493 population�parameters,�488 residual�error,�488 SPSS�and�G*Power,�514–515 summary�table�and�mean�squares,�
490–492 two-factor�model
assumptions�and�violations� of assumptions,�380–381
characteristics,�373–374 effect�size�measures,�confidence�intervals,�
and�power,�383–384 examples,�384–389 expected�mean�squares,�389–390 layout�of�data,�374 main�effects�and�interaction�effects,�
377–380 multiple�comparison�procedures,�382–383 partitioning�the�sums�of�squares,�381 summary�tables,�381–382
two-factor�random-effects�model assumptions�and�violation�
of assumptions,�487 characteristics,�483–484 hypotheses,�484–485 multiple�comparison�procedures,�487 population�parameters,�484 residual�error,�484 SPSS�and�G*Power,�513–514 summary�table�and�mean�squares,�485–487
two-factor�randomized�block�design� for�n�>�1,�574
SPSS,�603 two-factor�randomized�block�design�
for�n�=�1 assumptions�and�violation�
of assumptions,�569–570 block�formation�methods,�571–572 characteristics,�567–568 example,�572–573 G*Power,�603 hypotheses,�569 layout�of�data,�568 multiple�comparison�procedures,�571 population�parameters,�568 SPSS,�589–563 summary�table�and�mean�squares,�
570–571
two-factor�split-plot/mixed�design assumptions�and�violation�
of assumptions,�503 characteristics,�500 example,�506–508 hypotheses,�495 layout�of�data,�500–501 multiple�comparison�procedures,�505–506 population�parameters,�501–502 residual�error,�502 SPSS�and�G*Power,�526–548 summary�table�and�mean�squares,�503–505
unequal�n’s,�312 APA-style�paragraph
data�representation,�41–42 univariate�population�parameters,�69–70
A�priori�power,�137 Assumption�of�linearity,�269–270 Asymptotic�curve,�83–84
B
Backward�elimination,�676–677 Balanced�case,�296 Bar�graph,�23–24 Between-groups�variability,�298 Binomial�distribution,�proportion,�209 Bivariate�measures�of�association,�see�Measures�
of�association Blockwise�regression,�678 Box,�33 Box-and-whisker�plot,�33 Brown–Forsythe�procedures,�249–251,�313 Bryant–Paulson�test,�780–782
C
Categorical�variable,�definition,�7 Causation,�correlation�coefficients,�270 Cell,�222 Central�limit�theorem,�116–117 Chi-square�distribution
goodness-of-fit�test,�218–221 percentage�points,�761 SPSS,�225–231 test�of�association,�221–224
Chunkwise�regression,�678 Coefficient�of�determination,�620–622,�
665–668 College�Entrance�Examination�Board�(CEEB)�
score,�86 Column�marginal,�222 Comparisons,�342
815Subject Index
Complete�factorial�design,�559 Completely�randomized�design,�296 Completely�randomized�factorial�design,�
ANOVA,�374 Complex�post�hoc�contrasts,�Scheffé�and�
Kaiser–Bowden�methods,�357–358 Compound�symmetry,�495,�569 Computational�formula,�299 Conditional�distribution,�628–629 Confidence�interval�(CI),�115–116,�133–134 Constant,�definition,�7 Contingency�table
chi-square�test�of�association,�221–222 proportion,�215–216
Continuous�variable,�definition,�8 Contrast-based�multiple�comparison�
procedures,�346 Contrasts,�343–345 Correlation�coefficients
assumption�of�linearity,�269–270 correlation�and�causality,�270 different�types,�275–276 Pearson�product-moment,�265–269 restriction�of�range,�271
Covariance�analysis,�relationship�among� variables,�263–265
Covariate definition,�429 independence�of,�438–439 measured�without�error,�439
Cramer’s�phi�type�correlation,�275 Crossed�design,�559 Cross�validation,�720 Cumulative�frequency�distribution,�22 Cumulative�frequency�polygon,�26–27 Cumulative�relative�frequency�
distribution,�23 Cumulative�relative�frequency�polygon,�27
D
Data�representation APA-style�paragraph,�41–42 appropriate�techniques,�measurement�scale�
types,�42–43 graphical�display
bar�graph,�23–24 cumulative�frequency�polygon,�26–27 frequency�distribution�shapes,�27–28 frequency�polygon,�25–26 histogram,�25 relative�frequency�polygon,�26 stem-and-leaf�display,�28–29
percentiles box-and-whisker�plot,�33 computing�formula,�29–31 definition,�29 quartiles,�31 ranks,�31–32
SPSS�procedures,�33–41 tabular�display
cumulative�frequency�distribution,�22 cumulative�relative�frequency�
distribution,�23 frequency�distribution,�19–22 relative�frequency�distribution,�
22–23 Decision�errors,�124–126 Decision-making
example�situation,�124–125 full�context,�134–136 overview�of�steps,�129–130 table,�125–126
Definitional�(conceptual)�formula,�299 Degrees�of�freedom�concept,�140 Dependent�proportions,�215–217 Dependent�samples,�164–165 Dependent�t�test
assumptions,�180 confidence�interval,�177 effect�size,�177 example,�177–179 recommendations,�180 standard�error,�176
Dependent�variable,�criterion,�292 Dependent�variance,�246–248 Descriptive�statistics,�definition,�6 Deviational�measures
deviation�score,�58–59 population�variance
characteristics,�61 computational�formula,�60–61 definitional�formula,�60
sample�variance,�62–64 standard�deviation
characteristics,�61 description,�61 and�population�variance,�61–62 and�sample�variance,�62–64
Deviation�score,�58–59 Dichotomous�variable,�definition,�8 Directional�alternative�hypothesis,�128 Discrete�variable,�definition,�8 Dummy�variable,�681,�711 Dunnett�test,�769–771 Dunn�(or�Bonferroni)�method,�772–775
816 Subject Index
E
Effect�size,�139,�267,�725 in�chi-square�test�of�association,�224 in�G*Power,�151 in�inferences�about�2�dependent�means,�177 in�inferences�about�2�independent�
means, 168–169 measures�of,�139,�303–304,�383–384 in�proportions�involving�chi-square�
distribution,�220–221 Equal�n’s,�296 Errors�of�estimate,�619 Exact�probability,�132 Expected�proportion,�218–219 Experiment-wise�type�I�error�rate,�293 Extrapolation,�value�of�X,�632
F
Factorial�analysis�of�variance SPSS,�395–417 template�and�APA-style�write-up,�417–419 three-factor�and�higher-order
characteristics,�390–391 summary�table,�391–392 triple�interaction,�393
two-factor�model assumptions�and�violations�
of assumptions,�380–381 characteristics,�373–374 effect�size�measures,�confidence�intervals,�
and�power,�383–384 examples,�384–389 expected�mean�squares,�389–390 layout�of�data,�374 main�effects�and�interaction�
effects, 377–380 multiple�comparison�procedures,�382–383 partitioning�the�sums�of�squares,�381 summary�tables,�381–382
with�unequal�n’s,�393–394 Factorial�design,�ANOVA,�373 Fail�to�reject,�125 False�negative�rate,�720 False�positive�rate,�720 Family�of�curves,�80 Family-wise�multiple�comparison�
procedures, 346 F�distribution,�243,�762–765 Fisher’s�Z�transformation,�268,�766–767 Fixed�independent�variable,�assumption�
in ANCOVA,�438
Fixed�X assumptions�in�linear�regression,�632–633 assumptions�in�logistic�regression,�723 assumptions�in�multiple�regression,�674–675
Forced�stepwise�regression,�678 Forward�selection,�677 Frequency�distributions
shapes,�27–28 tabular�display,�19–22
Frequency�polygon,�25–26 Friedman�test
hierarchical�and�randomized�block� ANOVA, 574
nonparametric�one-factor�repeated�measures� ANOVA,�524–526
one-factor�repeated�measures�ANOVA,� 498–499
Fully�crossed�design,�ANOVA,�373
G
G*Power ANCOVA�model,�445–469 chi-square�distribution,�233 dependent�t�test,�193–194 independent�t�test,�192–193 linear�regression,�647–650 logistic�regression,�746–748 measures�of�association,�283–285 multiple�regression,�698–701 one-factor�ANOVA,�313–334 one-factor�random-effects�model,�508–513 one-factor�repeated�measures�design,�515–524 testing�hypothesis,�149–154 two-factor�mixed-effects�model,�514–515 two-factor�random-effects�model,�513–514 two-factor�split-plot/mixed�design,�526–548
Grouped�frequency�distributions,�21
H
Hierarchical�design,�559 Hierarchical�regression,�678
logistic�regression,�726–727 Hinge,�33 Histogram,�25 Homogeneity�of�regression�slopes,�ANCOVA�
model,�440–441 Homogeneity�of�variance,�310–311
assumption�in�ANCOVA,�437 assumption�in�ANOVA,�310–311 assumptions�in�linear�regression,�628–629 assumptions�in�multiple�regression,�672
817Subject Index
Homogeneity�of�variances,�248 assumption�in�ANOVA,�249,�251–252
Homoscedasticity,�310 Hosmer–Lemeshow�goodness-of-fit�test,�717 H�spread,�58 Hypotheses
differences�between�two�means,�165–166 types,�122–124
Hypothesis�testing confidence�intervals,�133–134 decision�errors,�124–126 decision-making
example�situation,�124–125 full�context,�134–136 overview�of�steps,�129–130 table,�125–126
G*Power,�149–154 level�of�significance,�127–129 power
determinants,�136–138 type�II�error�and,�134–136
SPSS,�145–149 statistical�vs��practical�significance,�138–139 template�and�APA-style�write-up,�155–156 type�II�error�(β),�134–138 types�of,�122–124 z�test,�130–133
I
Incomplete�factorial�design,�559 Independence
assumption�in�ANCOVA,�436–437 assumption�in�ANOVA,�309–310 assumptions�in�linear�regression,�628 assumptions�in�multiple�regression,�
671–674 random-and�mixed-effect�ANOVA�
assumptions,�542–544 two-factor�hierarchical�ANOVA�
assumptions,�589 two-factor�randomized�block�ANOVA�
assumptions,�601–602 Independence�of�errors,�723 Independent�proportion,�212–215 Independent�samples,�164–165 Independent�t�test
assumptions,�171–172 confidence�interval,�168 effect�size,�168–169 example,�169–171 measurement�scales,�167 recommendations,�174–175
standard�error,�167 Welch�t’�test,�172–174
Independent�variable ANCOVA�model,�438–439 predictor,�612
Independent�variances,�248–252 Inferential�statistics,�definition,�6–7,�109 Intact�groups,�429,�443 Interaction�effect
ANOVA�model,�377–380 and�main�effects,�377–380 two-factor�ANOVA�model,�373
Interpolation,�value�of�X,�632 Interval�measurement�scale,�11–12 Intervals
in�data�sets,�20 midpoint,�19–21,�26 width,�21
Intuition�vs��probability,�108–109
K
Kendall’s�tau,�measures�of�association,�273–274 Kruskal–Wallis,�follow-up�tests�to,�361–362 Kurtosis,�89–91
nonzero,�630
L
Least�squares�criterion,�620 Leptokurtic�distribution,�89–90 Level�of�significance,�127–129 Likelihood�ratio�test,�716–717 Linearity
assumption�in�ANCOVA,�438 assumptions�in�linear�regression,�631–632 assumptions�in�logistic�regression,�722 assumptions�in�multiple�regression,�672,�674
Linear�regression concepts,�612–614 G*Power,�647–650 population,�614–615 sample�model
assumptions�and�violation� of assumptions,�627–633
coefficient�of�determination,�620–622 least�squares�criterion,�620 prediction�errors,�619–620 significance�tests�and�confidence�
intervals,�622–627 standardized�regression�model,�618 unstandardized�regression�model,�
615–617
818 Subject Index
SPSS,�634–647 template�and�APA-style�write-up,�650–652
Linear�relationship,�269 Logistic�regression
assumptions,�722–723 conditions
lack�of�influential�points,�724 nonseparation�of�data,�724 nonzero�cell�counts,�723–724 sample�size,�724–725
description,�710–712 effect�size,�725 equation
odds�and�logit,�713–715 probability,�712–713
estimation�and�model�fit,�715–716 G*Power,�746–748 predictor�entry�methods
hierarchical�regression,�726–727 simultaneous,�726 stepwise,�726
significance�tests logistic�regression�coefficients,�720–721 overall�regression�model,�716–720
SPSS,�727–746 template�and�APA-style�write-up,�749–751
Logistic�regression�coefficients,�720–721
M
Main�effect,�ANOVA�model,�377–380 Mean,�54–55
differences�between�two,�163–198 independent�vs��dependent�samples,�
164–165 inferences�about�two�dependent,�175–180 inferences�about�two�independent,�166–175 sampling�distribution�of�the�differences,�166 standard�error�of�the�difference�between�
two,�167 Mean�squares�term,�301 Measurement,�definition,�8 Measures�of�association
correlations,�269–271 covariance,�263–265 Cramer’s�phi,�275 G*Power,�283–285 Kendall’s�tau,�273–274 Pearson�product-moment�correlation�
coefficient,�265–269 phi�coefficient,�274–275 scatterplot,�260–263 Spearman’s�rho,�272–273
SPSS,�276–282 template�and�APA-style�write-up,�286
Measures�of�central�tendency advantages,�55–56 disadvantages,�55–56 mean,�54–55 median,�53–54 mode,�51–53
Measures�of�dispersion advantages,�64 deviational�measures,�58–64 disadvantages,�64 H�spread,�58 range,�56–58
Median,�53–54 Mesokurtic�distribution,�89–90 Midpoint,�intervals,�19–21,�26 Mixed�design,�500 Mode,�51–53 Moments�around�the�mean,�89 Multilevel�model,�559 Multiple�comparison�procedure�(MCP),�382–383
concepts�of,�342–348 Dunn�(or�Bonferroni)�and�Dunn–Sidak�
methods,�355–357 Dunnett�method,�354–355 flowchart,�366–367 follow-up�tests�to�Kruskal–Wallis,�361–362 Games–Howell,�Dunnett�T3�and�C�tests,�361 selected,�348–362 SPSS,�362–365 template�and�APA-style�write-up,�366 Tukey�HSD,�Tukey–Kramer,�Fisher�LSD,�
and Hayter�tests,�358–361 Multiple�linear�regression
assumptions,�671–676 coefficient�of�multiple�determination�
and correlation,�665–668 significance�tests,�668–671 standardized�regression�model,�664–665 unstandardized�regression�model,�661–664
Multiple�regression categorical�predictors,�680–681 G*Power,�698–701 interactions,�680 linear�regression,�661–676 multiple�predictor�model
all�possible�subsets�regression,�678 backward�elimination,�676–677 forward�selection,�677 hierarchical�regression,�678 sequential�regression,�676,�678–679 simultaneous�regression,�676
819Subject Index
stepwise�selection,�677–678 variable�selection�procedures,�676
nonlinear�relationships,�679–680 part�correlation,�660–661 partial�correlation,�659–660 semipartial�correlation,�660–661 SPSS,�682–698 template�and�APA-style�write-up,�701–703
N
Negatively�skewed�distribution,�28,�88–89 Nested�design,�559 Nominal�measurement�scale,�9,�12 Noncollinearity
assumptions�in�logistic�regression,�722 assumptions�in�multiple�regression,�675
Nondirectional�alternative�hypothesis,�128 Nonlinear�models,�632 Nonlinear�relationship,�270,�679–680 Nonparametric�tests,�171 Normal�distribution,�27,�28
characteristics area,�80–81 family�of�curves,�80 standard�curve,�79 unit�normal�distribution,�80
history,�78–79 proportions�involving,�206–217 standard�scores�and,�77–99
Normality assumption�in�ANCOVA,�437–438 assumption�in�ANOVA,�311–312 assumptions�in�linear�regression,�629–631 assumptions�in�multiple�regression,�672 two-factor�hierarchical�ANOVA�
assumptions,�585–589 two-factor�randomized�block�ANOVA�
assumptions,�598–601 two-factor�split-plot�ANOVA�assumptions,�
538–542 Null�hypothesis,�122–123 Numerical�variable,�definition,�8
O
O’Brien�procedure,�251–252 Observed�proportions,�218 Odds�ratio�(OR),�725 Omnibus�test,�294 One-tailed�test�of�significance,�128 Ordinal�measurement�scale,�10–12 Orthogonal�contrasts,�347–348
planned,�352–354
Orthogonal�polynomials,�768 Outliers,�33,�629 Overall�regression�model
cross�validation,�720 Hosmer–Lemeshow�goodness-of-fit�test,�717 likelihood�ratio�test,�716–717 predicted�group�membership,�719–720 pseudovariance�explained,�718–719
P
Parameter,�definition,�5–6 Parametric�tests,�171 Part�correlation,�660–661 Partial�correlation,�659–660 Partially�sequential�approach,�factorial�ANOVA�
with�unequal�n’s,�393 Pearson�product-moment�correlation�coefficient
inference�for�a�single�sample,�267–268 inference�for�two�independent�samples,�
268–269 Percentile�rank,�31–32 Percentiles
box-and-whisker�plot,�33 computing�formula,�29–31 definition,�29 quartiles,�31 rank,�31–32
Phi�type�of�correlation,�274–275 Planned�analysis�of�trend,�MCP,�349–352 Planned�contrasts,�345–346
Dunn�(or�Bonferroni)�and�Dunn–Sidak� methods,�355–357
orthogonal,�352–354 with�reference�group,�Dunnett�method,�
354–355 SPSS,�364
Platykurtic�distribution,�89–90 Points�of�inflection,�83–84 Population,�definition,�5 Population�parameters
definition,�5–6 estimation�of
central�limit�theorem,�116–117 confidence�interval,�115–116 sampling�distribution�of�the�mean,�112–113 standard�error�of�the�mean,�114–115 variance�error�of�the�mean,�113–114
univariate,�49–71 Population�prediction�model,�614 Population�proportion,�207 Population�regression�model,�614 Population�variance,�proportions�of,�208
820 Subject Index
Positively�skewed�distribution,�27–28,�88–89 Post�hoc�blocking�method,�572 Post�hoc�contrasts,�346
SPSS,�363 Post�hoc�power,�137 Power
definition,�134,�575 determinants,�136–138 type�II�error�and,�134–136
Practical�significance,�vs��statistical�significance,� 138–139
Precision,�definition,�575 Predicted�group�membership,�719–720 Prediction�errors,�619–620 Probability
definition�of,�106–108 importance�of,�106 intuition�vs.,�108–109 logistic�regression�equation,�712–713 sampling�and�estimation,�109–117
Profile�plot,�377 Proof�(prove),�126 Proportions
binomial�distribution,�209 chi-square�distribution,�217–224 definition,�205 dependent,�215–217 independent,�212–215 inferences,�205–235 normal�distribution,�206–217 sampling�distribution,�208 single,�210–212 standard�error,�209 standard�error�of�difference�between�two,�213 tests�of,�206–207 variance�error,�209
Pseudovariance�explained,�718–719
Q
Quartiles,�31 Quasi-experimental�designs,�429,�443
R
Randomization,�definition,�443 Randomized�block�designs,�567 Range,�56–58
exclusive,�57 inclusive,�57
Ratio�measurement�scale,�12 Raw�residuals,�628 Raw�scores,�20 Real�limits,�in�data�sets,�20–21
Regression�approach,�factorial�ANOVA� with unequal�n’s,�393
Relative�frequency�distribution,�22–23 Relative�frequency�polygon,�26 Repeated�factor,�478 Repeated-measures�models,�295 Replacement,�simple�random�sampling�
with and�without,�110–111 Research�hypothesis,�122–123 Restriction�of�range,�271 Row�marginal,�222
S
Sample,�definition,�6 Sampled�range�blocking�method,�572 Sampled�value�blocking�method,�572 Sample�prediction�model,�616 Sample�proportion,�208 Sample�regression�model,�615 Sample�size,�19–20,�724–725 Sample�statistics
probability�and,�77–91 univariate�population�parameters
APA-style�paragraph,�69–70 appropriate�descriptive�statistics,�70–71 measures�of�central�tendency,�51–56 measures�of�dispersion,�56–64 SPSS,�65–69 summation�notation,�50–51
Sample�variance,�62–64 Sampling�distribution
difference�between�two�means,�166 full�decision-making�context,�134–136 intelligence�test�case,�135–136 of�the�mean,�112–113 proportion,�208 variance,�242
Sampling�error,�112–113 Scales�of�measurement,�8–12 Scatterplot
measures�of�association,�260–263 two-factor�randomized�block�ANOVA�
assumptions,�601–602 two-factor�split-plot�ANOVA�
assumptions, 543 Scientific�hypothesis,�122–123 Semipartial�correlation,�660–661 Sensitivity,�719–720 Sequential�approach,�factorial�ANOVA�
with unequal�n’s,�393 Sequential�regression�model,�676
commentary�on,�678–679
821Subject Index
Setwise�regression,�678 Significance�tests�and�confidence�intervals,�
622–627 Simple�post�hoc�contrasts
Tukey�HSD,�Tukey–Kramer,�Fisher�LSD,� and Hayter�tests,�358–361
for�unequal�variances,�Games–Howell,� Dunnett�T3�and�C�tests,�361
Simple�random�sampling with�replacement,�110 without�replacement,�111
Simultaneous�logistic�regression,�726 Simultaneous�regression�model,�676 Single�variances,�244–246 Skewed�distribution,�88–89 Skewness
definition,�88 nonzero,�630
Spearman’s�rank�correlation,�272–273 Specificity,�720 Sphericity,�495,�569 Split-plot�design,�500 SPSS
ANCOVA�model,�445–469 chi-square�distribution,�225–231 data�representation,�33–41 dependent�t�test,�188–192 factorial�analysis�of�variance,�395–417 independent�t�test,�180–187 logistic�regression,�727–746 measures�of�association,�276–282 multiple�regression,�682–698 normal�distribution�and�standard�scores,�
91–97 one-factor�ANOVA,�313–334 one-factor�random-effects�model,�508–513 one-factor�repeated�measures�design,�
515–524 simple�linear�regression,�634–647 testing�hypothesis,�145–149 two-factor�mixed-effects�model,�514–515 two-factor�random-effects�model,�513–514 two-factor�randomized�block�design�
for n > 1,�603 two-factor�randomized�block�design�
for n = 1,�589–563 two-factor�split-plot/mixed�design,�526–548 univariate�population�parameters,�65–69 variances,�252
Standard�curve,�79 Standard�deviation
constant�relationship�with,�82–83 sample�variance,�62–64
Standard�error difference�between�two�means,�167 difference�between�two�proportions,�213 of�the�mean,�114–115 proportion,�209
Standard�error�of�estimate,�624 Standardized�regression�model,�
618, 664–665 Standardized�residuals,�628 Standard�scores
College�Entrance�Examination�Board�(CEEB)� score,�86
IQ�score,�86 normal�distribution�and,�77–99 T�score,�86 z�scores,�84–86
Standard�unit�normal�distribution,� 80, 757–759
Statistical�hypothesis,�122–123 Statistical�significance,�vs��practical�significance,�
138–139 Statistic,�definition,�6 Statistics
definitions,�5–7 history�of,�4–5 scales�of�measurement,�8–12 value�of,�3–4 variables,�7–8
Stem-and-leaf�display,�28–29 Stepwise�logistic�regression,�726 Stepwise�selection,�677–678 Studentized�range�test,�358,�776–779 Studentized�residuals,�628 Summation�notation,�50–51 Symmetric�around�the�mean,�87 Symmetric�distributions,�27,�88
T
t�distribution,�140–142,�760 Template�and�APA-style�write-up
ANCOVA�model,�469–471 chi-square�distribution,�234–235 dependent�t�test,�196–198 factorial�analysis�of�variance,�417–419 independent�t�test,�195–196 linear�regression,�650–652 logistic�regression,�749–751 measures�of�association,�286 multiple�regression,�701–703 normal�distribution�and�standard�scores,�
98–99 one-factor�ANOVA,�334–336
822 Subject Index
testing�hypothesis,�155–156 variances,�253
Tetrad�difference,�ANOVA,�383 Tied�ranks,�10 Transformations,�632 Trend�analysis,�349 True�experimental�designs,�429 True�experiments,�443 t�test,�140,�142–145
correlated�samples,�165 dependent,�176–180,�188–192,�196–198 dependent�samples,�164–165 independent,�167–174,�180–187,�
195–196 independent�samples,�164–165 paired�samples,�165 Welch,�313
Two-tailed�test�of�significance,�128 Type�II�error�(β),�134–138
U
Unbalanced�case,�296,�312 Unequal�n’s,�296,�312 Ungrouped�frequency�distribution,�
19, 21–22 Unit�normal�distribution
description,�80 transformation�of,�82
Univariate�analysis,�260;�see also�Univariate� population�parameters
Univariate�population�parameters APA-style�paragraph,�69–70 appropriate�descriptive�statistics,�70–71 measures�of�central�tendency
advantages,�55–56 disadvantages,�55–56 mean,�54–55 median,�53–54 mode,�51–53
measures�of�dispersion advantages,�64 deviational�measures,�58–64 disadvantages,�64
H�spread,�58 range,�56–58
SPSS,�65–69 summation�notation,�50–51
Unstandardized�regression�model,�615–617,� 661–664
Untied�ranks,�10
V
Variables definition,�7 types,�7–8
Variable�selection�procedures,�676 Variance�error
of�the�mean,�113–114 proportion,�209
Variance�error�of�estimate,�624 Variance�of�the�residuals,�624 Variances
Brown–Forsythe�procedure,�249–251,�313 F�distribution,�243 homogeneity,�248,�310–311 independent,�248–252 O’Brien�procedure,�251–252 sampling�distribution,�242 single,�244–246 SPSS,�252 template�and�APA-style�write-up,�253 traditional�test,�248–249 two�dependent,�246–248
Variance�stabilizing�transformations,�629
W
Welch�t’�test,�172–174,�293 Whiskers,�33 Within-groups�variability,�298 Within-subjects�design,�493
Z
z�scores,�84–86 z�test,�130–133
- An Introduction to Statistical Concepts
- Copyright
- Contents
- Preface
- Acknowledgments
- 1. Introduction
- 1.1 What Is the Value of Statistics?
- 1.2 Brief Introduction to History of Statistics
- 1.3 General Statistical Definitions
- 1.4 Types of Variables
- 1.5 Scales of Measurement
- 1.6 Summary
- Problems
- 2. Data Representation
- 2.1 Tabular Display of Distributions
- 2.2 Graphical Display of Distributions
- 2.3 Percentiles
- 2.4 SPSS
- 2.5 Templates for Research Questions and APA-Style Paragraph
- 2.6 Summary
- Problems
- 3. Univariate Population Parameters and Sample Statistics
- 3.1 Summation Notation
- 3.2 Measures of Central Tendency
- 3.3 Measures of Dispersion
- 3.4 SPSS
- 3.5 Templates for Research Questions and APA-Style Paragraph
- 3.6 Summary
- Problems
- 4. Normal Distribution and Standard Scores
- 4.1 Normal Distribution
- 4.2 Standard Scores
- 4.3 Skewness and Kurtosis Statistics
- 4.4 SPSS
- 4.5 Templates for Research Questions and APA-Style Paragraph
- 4.6 Summary
- Problems
- 5. Introduction to Probability and Sample Statistics
- 5.1 Brief Introduction to Probability
- 5.2 Sampling and Estimation
- 5.3 Summary
- Appendix: Probability That at Least Two Individuals Have the Same Birthday
- Problems
- 6. Introduction to Hypothesis Testing: Inferences About a Single Mean
- 6.1 Types of Hypotheses
- 6.2 Types of Decision Errors
- 6.3 Level of Significance (α)
- 6.4 Overview of Steps in Decision-Making Process
- 6.5 Inferences About μ When σ Is Known
- 6.6 Type II Error (β) and Power (1 − β)
- 6.7 Statistical Versus Practical Significance
- 6.8 Inferences About μ When σ Is Unknown
- 6.9 SPSS
- 6.10 G*Power
- 6.11 Template and APA-Style Write-Up
- 6.12 Summary
- Problems
- 7. Inferences About the Difference Between Two Means
- 7.1 New Concepts
- 7.2 Inferences About Two Independent Means
- 7.3 Inferences About Two Dependent Means
- 7.4 SPSS
- 7.5 G*Power
- 7.6 Template and APA-Style Write-Up
- 7.7 Summary
- Problems
- 8. Inferences About Proportions
- 8.1 Inferences About Proportions Involving Normal Distribution
- 8.2 Inferences About Proportions Involving Chi-Square Distribution
- 8.3 SPSS
- 8.4 G*Power
- 8.5 Template and APA-Style Write-Up
- 8.6 Summary
- Problems
- 9. Inferences About Variances
- 9.1 New Concepts
- 9.2 Inferences About Single Variance
- 9.3 Inferences About Two Dependent Variances
- 9.4 Inferences About Two or More Independent Variances (Homogeneity
- 9.5 SPSS
- 9.6 Template and APA-Style Write-Up
- 9.7 Summary
- Problems
- 10. Bivariate Measures of Association
- 10.1 Scatterplot
- 10.2 Covariance
- 10.3 Pearson Product–Moment Correlation Coefficient
- 10.4 Inferences About Pearson Product–Moment Correlation Coefficient
- 10.5 Assumptions and Issues Regarding Correlations
- 10.6 Other Measures of Association
- 10.7 SPSS
- 10.8 G*Power
- 10.9 Template and APA-Style Write-Up
- 10.10 Summary
- Problems
- 11. One-Factor Analysis of Variance: Fixed-Effects Model
- 11.1 Characteristics of One-Factor ANOVA Model
- 11.2 Layout of Data
- 11.3 ANOVA Theory
- 11.4 ANOVA Model
- 11.5 Assumptions and Violation of Assumptions
- 11.6 Unequal n’s or Unbalanced Procedure
- 11.7 Alternative ANOVA Procedures
- 11.8 SPSS and G*Power
- 11.9 Template and APA-Style Write-Up
- 11.10 Summary
- Problems
- 12. Multiple Comparison Procedures
- 12.1 Concepts of Multiple Comparison Procedures
- 12.2 Selected Multiple Comparison Procedures
- 12.3 SPSS
- 12.4 Template and APA-Style Write-Up
- 12.5 Summary
- Problems
- 13. Factorial Analysis of Variance: Fixed-Effects Model
- 13.1 Two-Factor ANOVA Model
- 13.2 Three-Factor and Higher-Order ANOVA
- 13.3 Factorial ANOVA With Unequal n’s
- 13.4 SPSS and G*Power
- 13.5 Template and APA-Style Write-Up
- 13.6 Summary
- Problems
- 14. Introduction to Analysis of Covariance: One- Factor Fixed-Effects Model
- 14.1 Characteristics of the Model
- 14.2 Layout of Data
- 14.3 ANCOVA Model
- 14.4 ANCOVA Summary Table
- 14.5 Partitioning the Sums of Squares
- 14.6 Adjusted Means and Related Procedures
- 14.7 Assumptions and Violation of Assumptions
- 14.8 Example
- 14.9 ANCOVA Without Randomization
- 14.10 More Complex ANCOVA Models
- 14.11 Nonparametric ANCOVA Procedures
- 14.12 SPSS and G*Power
- 14.13 Template and APA-Style Paragraph
- 14.14 Summary
- Problems
- 15. Random- and Mixed-Effects Analysis of Variance Models
- 15.1 One-Factor Random-Effects Model
- 15.2 Two-Factor Random-Effects Model
- 15.3 Two-Factor Mixed-Effects Model
- 15.4 One-Factor Repeated Measures Design
- 15.5 Two-Factor Split-Plot or Mixed Design
- 15.6 SPSS and G*Power
- 15.7 Template and APA-Style Write-Up
- 15.8 Summary
- Problems
- 16. Hierarchical and Randomized Block Analysis of Variance Models
- 16.1 Two-Factor Hierarchical Model
- 16.2 Two-Factor Randomized Block Design for n = 1
- 16.3 Two-Factor Randomized Block Design for n > 1
- 16.4 Friedman Test
- 16.5 Comparison of Various ANOVA Models
- 16.6 SPSS
- 16.7 Template and APA-Style Write-Up
- 16.8 Summary
- Problems
- 17. Simple Linear Regression
- 17.1 Concepts of Simple Linear Regression
- 17.2 Population Simple Linear Regression Model
- 17.3 Sample Simple Linear Regression Model
- 17.4 SPSS
- 17.5 G*Power
- 17.6 Template and APA-Style Write-Up
- 17.7 Summary
- Problems
- 18. Multiple Regression
- 18.1 Partial and Semipartial Correlations
- 18.2 Multiple Linear Regression
- 18.3 Methods of Entering Predictors
- 18.4 Nonlinear Relationships
- 18.5 Interactions
- 18.6 Categorical Predictors
- 18.7 SPSS
- 18.8 G*Power
- 18.9 Template and APA-Style Write-Up
- 18.10 Summary
- Problems
- 19. Logistic Regression
- 19.1 How Logistic Regression Works
- 19.2 Logistic Regression Equation
- 19.3 Estimation and Model Fit
- 19.4 Significance Tests
- 19.5 Assumptions and Conditions
- 19.6 Effect Size
- 19.7 Methods of Predictor Entry
- 19.8 SPSS
- 19.9 G*Power
- 19.10 Template and APA-Style Write-Up
- 19.11 What Is Next?
- 19.12 Summary
- Problems
- Appendix: Tables
- References
- Odd-Numbered Answers to Problems
- Author Index
- Subject Index