Is college education in line with the job market?

Deciding on Which Statistical Test to Use

1) What type of quantitative research strategy was used?

A) Experimental or Quasi-Experimental

B) Nonexperimental, Ex post Facto, Correlational

2) How many independent variables, if experimental, are in your investigation?

3) If experimental, is your design a between groups, within groups, or mixed/match group design?

4) How many predictor variables, if nonexperimental etc., do you have in your investigation?

5) What is the measurement scale (Nominal, Ordinal, Interval, or Ratio) of your operational definitions?

· In SPSS interval and ratio data are combined and noted as continuous

· Likert Scales are interval level data

Experiments or Quasi-Experimental

One Independent Variable where there is simply one experimental condition and one control condition or where there are two levels of experimental conditions. And, The scale of measurement for the dependent variable is either interval or ratio. Then, you could use the T-Test for independent groups

Example : You hypothesize that participants exposed to Intervention A (Experimental Condition) will report significantly less anxiety than those exposed to a placebo control condition.

a) There are two groups where the participants were randomly assigned to Intervention A or Placebo and the measure you use for the dependent variable is either interval or ratio

The T-Test for independent groups could be used.

· T-Test for Dependent Groups is used if matching is used for assigning participants to groups

· Wilcoxon Paired-Sample Test is used when the DV is ordinal (rank data)

b) Instead of simply having a placebo control group, your are interested in finding out which of two experimental conditions result in significant change in the dependent variable but also want to use a placebo to rule out psychological accommodations as a potentially confounding factor.

Example : You hypothesize that Intervention A will result in a statistically significant decrease in anxiety in comparison to Intervention B and Placebo (or no intervention group). This is a Single Factor-Multiple Conditions design where there are actually three conditions (levels of the IV).

a) Three conditions eliminates the T-Test as the statistic of choice. T-Tests are only appropriate for comparing two groups. Trying to use multiple T-Tests would actually raise the alpha level from .05 to.15 because you would conduct three separate analyses of the data and doing so is unacceptable.

b) Instead, you would use One-Way ANOVA, where one-way indicates that there is one IV If the One-Way ANOVA indicates there is a statistically significant difference in the DV between the three groups, you must then find out where that difference is. To do so, you would run a post-hoc statistical test. You could select one of from the list of post-hoc tests found on this website. (https://www.statisticshowto.com/probability-and-statistics/statistics-definitions/post-hoc/ ) The decision then becomes which one to use. Scheffe’s Method, Tukey’s Test and Dunnett’s Correction have been used the most by our students in the past. The post-hoc test is found in SPSS and you would simply indicate by checking the appropriate box that you would want that test to be run along with the ANOVA

Note: Both the T-Test and ANOVA require that the IV be nominal/categorical. Keep this in mind because sometimes interval or ratio level data are divided at the median (the median split) when the research strategy was nonexperimental and scores on some measure (For instance a depression scale) are split in half or thirds in order to run a T-Test or ANOVA. The DV is some other measured factor (for example, the number of cigarettes smoked in a month or the amount of money spent online)

The problem with using the median split or dividing the distribution of IV scores into thirds is that doing so does not truly differentiate between high, low, and medium levels of depression. Statistically, if you desired to differentiate between high and low levels of depression, you would calculate the standard deviation and select participants who scored two standard deviations above and below. Only those persons who exceeded two standard deviations above and those who fell below two standard deviations from the mean would comprise the two groups needed for comparison using the T-Test. This would require a large pool of potential participants having completed the screening measure of the IV in order for there to be adequate numbers of persons falling below and above two standards deviations from the mean.

Factorial Designs – When an experiment investigates two or more independent variables, then one is using a factorial design. Such designs are identified like so:

· 2 X 2 Factorial design (Indicates that there are two independent variables and that there are two levels(conditions) that have been established(manipulated) by the experimenter for each of the variables.

· 3 X 3 X 4 Factorial design ( indicates three IVs, the first and second IV have three levels each and the third IV has 4 levels)

· The more IVs in an experiment that uses a between subjects design will require an increasing number of participants. For example, if each cell of the between subjects design needs to have 25 participants, then a 2 x 2 factorial design would require 100 participants to have been randomly assigned to the four conditions. If, you use a 3 x 3 x 4 factorial - between subjects design, then there are 36 conditions that would require 900 participants.

· Using a mixed/match factorial design where the first IV is a subject variable such as religion (Protestant, Catholic, Muslim) that is used to select and match participants relative to the remaining conditions, then fewer than 900 participants would be required. Mixed designs require fewer participants, as do Within Subjects Designs, in comparison to Between Subjects Designs.

Statistical Analysis of Factorial Designs

ANOVA (F-Test) or Multifactor Analysis of Variance (MANOVA) is used. ANOVA is used when there is one DV and MANOVA can be used when there are multiple DVs

· One-Way ANOVA there are two or more levels of one IV, but when we have a factorial design there are more than one IV and the number of IVs are reflected in which ANOVA is indicated as having been used for analyzing the data. Two-Way ANOVA is sues when there are two IVs, Three-Way ANOVA is used when there are four IVs and so forth. MANOVA is specified similarly.

· If the research design is a within subjects research design, then either T-Test for repeated measures (sometimes referred to as T-Test for gain scores) or the ANOVA for repeated measures is what can be used.

Statistical Analysis of Matched/Mixed Factorial Designs (Designs that incorporate both between and within comparisons)

· Data that is interval or ratio and there are two factors would require a two-way, mixed design ANOVA (same as a two-factor, mixed design analysis of variance)

Nominal and Ordinal Level data most often require the use of nonparametric statistics that are similar to the parametric statistics. (If there are just two categories (i.e., male and female, or African American and European American then one can still use parametric statistics. If there are more than two categories, then the categories can be converted to dummy variables in order to use parametric statistics (i.e., African American, Hispanic American, European American could be converted into three groups of African American compared to Hispanic American combined with European American, Hispanic American compared to African American combined with European American, and European American compared to African American combined with Hispanic American)

Chi Square can be used for two group comparisons when the DV is categorical. This statistic indicates the likelihood of a statistically significant larger number (or lessor number) of participants falling into one of the two categories/conditions/levels.

Two-Variable Chi-Square Test of Independence is used when there are two IVs with each IV having two or more levels and the DV is a frequency count

Wilcoxon Signed-Rank Test is another nonparametric statistic that can be used when two levels of the IV are measured by an instrument that is ordinal (rank ordered). (example: a study where two treatments are being compared and interest is in whether participants show improvement or decline, the sign test can determine whether the changes are consistently in one direction or the other to a statistically significant degree.

Statistical Analysis of Correlational Investigations

Studies where only two factors being compared :

Pearson Product Moment Correlation Coefficient (r) is used when the variables are normally distributed and are measured on an interval or ratio scale.

Point Biserial Correlation Coefficient (rpb) is used when one variable is continuous (interval or ratio) and the other variable is binary (two categories).

Spearman Rho Correlation Coefficient (p) is used when data is ordinal. Is interpreted the same way as Pearson r.

The Phi Coefficient is used when both variables are genuine dichotomies but the Pearson Correlation Coefficient can also be used

Eta Correlation is used when the relationship between two variables is curvilinear

Multivariate Correlational Studies :

Multiple Regression is used when you have two or more predictor variables and a single criterion variable (In SPSS the terms independent and dependent variables are used when setting up your database. However, in writing your findings out in a report, thesis, or dissertation, you should identify factors that are the predictor factors and indicate what the criterion variable is.)

Additional Multivariate Analyses taught in Graduate school

· Factor Analysis

· Canonical Correlation

· Discriminant Analysis

· Path Analysis

· Structural Equation Modeling

Analysis of Qualitative Data

Interviews and Focus Groups can be audio or video recorded. Written notes taken during the recording can be compared with the verbatim transcripts one must make. The written notes can capture pertinent information that may be missed on the recordings but aid in substantiating noticeable characteristics and behaviors associated with certain respondents that come to the attention of the facilitator. These notes aid in describing the context of the data gathering session. The verbatim transcripts are subjected to thematic analysis. Thematic Analysis is the identification of themes or major ideas expressed. This type of analysis can also be used for analyzing written documents, one’s own and others’ field notes, technical papers, and newspaper articles. Sometimes it is useful to train assistants as to extrapolating themes and gauging the reliability of themes identified by the assistants.