Lecture: Analysis and Interpretation of Data

ANALYSIS AND INTERPRETATION OF DATA Analysis and Interpretation of Data

Slide 1 Transcript

In a qualitative design, the information gathered and studied often is nominal or narrative in form. Finding trends, patterns, and relationships is discovered inductively and upon reflection. Some describe this as an intuitive process. In Module 4, qualitative research designs were explained along with the process of how information gained shape the inquiry as it progresses. For the most part, qualitative designs do not use numerical data, unless a mixed approach is adopted. So, in this module the focus is on how numerical data collected in either a qualitative mixed design or a quantitative research design are evaluated. In quantitative studies, typically there is a hypothesis or particular research question. Measures used to assess the value of the hypothesis involve numerical data, usually organized in sets and analyzed using various statistical approaches. Which statistical applications are appropriate for the data of interest will be the focus for this module.

Data and Statistics

Match the data with an appropriate statistic

Approaches based on data characteristics

Collected for single or multiple groups Involve continuous or discrete variables Data are nominal, ordinal, interval, or ratio Normal or non-normal distribution

Statistics serve two functions

Descriptive: Describe what data look like Inferential: Use samples to estimate population characteristics

Slide 3 Transcript There are, of course, far too many statistical concepts to consider than time allows for us here. So, we will limit ourselves to just a few basic ones and a brief overview of the more common applications in use. It is vitally important to select the proper statistical tool for analysis, otherwise, interpretation of the data is incomplete or inaccurate. Since different statistics are suitable for different kinds of data, we can begin sorting out which approach to use by considering four characteristics:

1. Have data been collected for a single group or multiple groups 2. Do the data involve continuous or discrete variables 3. Are the data nominal, ordinal, interval, or ratio, and 4. Do the data represent a normal or non-normal distribution.

We will address each of these approaches in the slides that follow. Statistics can serve two main functions – one is to describe what the data look like, which is called descriptive statistics. The other is known as inferential statistics which typically uses a small sample to estimate characteristics of the larger population. Let’s begin with descriptive statistics and the measures of central tendency.

Descriptive Statistics and Central Measures

Descriptive statistics organize and present data

Mode The number occurring most frequently; nominal data

Quickest or rough estimate Most typical value

Measures of central tendency

Median The number in the very middle

Don’t have time to compute mean Distributions markedly skewed See if values in upper or lower halves If distribution incomplete

Mean The arithmetic average of values

Greatest reliability is desired Other computations likely to follow Symmetrical distribution Locates “center of gravity” of a sample

Slide 5 Transcript Descriptive statistics are used to organize a set of measures and typically are presented in tables, graphs, or summaries. Presenting all the raw data without summarizing or grouping them is not very helpful for others to understand what the research reveals. The term measures of central tendency refer to finding a point around which the data revolve, kind of like a point or number that centers the data. Three measures, very familiar to you by now, are the mean, median and mode. Beginning with the mode, which is the number or score that occurs most frequently, it is the measure of limited value because it is not very stable, and, doesn’t always appear near the middle of the distribution. One thing about the mode, though, is that it is the only appropriate measure of central tendency for nominal data. You would consider computing the mode when the quickest estimate, or a rough estimate, of central value will do, or, when you want to know what the most typical value is. On the other hand, the median is the number in the very middle of the values with exactly the same number of values above it as below it. You might compute the median when you don’t have sufficient time to compute the mean, distributions are markedly skewed, when you are just interested in whether the values fall within the upper or lower halves, or when an incomplete distribution is given. Finally, there is the mean, which is the balance point in the data. The mean is the arithmetic average of the data. You would use the mean when the greatest reliability is desired, when other computations like variability are to follow, where the distribution is symmetrical about the center, and when you wish to know the center of gravity of a sample. So, to wrap up this slide, just keep in mind that the nature of the data determines which statistical technique is suitable for the research design.

Variability

Scattered data (variability) show less average tendencies Indications of variability

Range: high to low scores Interquartile Range: equal to Q3 – Q1 Deviation: difference between score and mean Average Deviation: all deviations/number of scores Standard Deviation: square root of all deviations squared/number of scores Variance: standard deviation squared, used with ANOVA

Slide 7 Transcript The more data cluster around a point of central tendency, the probability of estimating other particular data points improves. It is a different story when the data points are scattered. This is because, as data points move farther from the mean, they are less and less average. So, in addition to calculating the mean, it also is important to determine how spread out the data are in order to derive any meaning. The most basic measure of variability is range of the data, which is the spread from highest to lowest. The range of values is not too useful as a measure of variability and can be misleading where the extreme limits are atypical of the rest of the values. To reduce the problem with extremes, the interquartile range is useful. This is where the range is divided into four segments, with Quartile 1 where 25% of the numbers lie below it, Quartile 2 is at the median, and Quartile 3 is where 75% of the values are below it. The interquartile range is equal to Quartile 3 minus Quartile 1. When researchers use the median for central tendency, the quartile deviation would be appropriate as a statistical measure of variability. Now, when using the mean for central tendency, researchers can calculate the deviation, which is the difference between the score and the mean. However, this is seldom used in reporting results. Likewise, by adding the deviations from the mean and dividing by the number of scores, the average deviation can be found. Again, this is not too useful for researchers. Instead, the standard deviation is the measure of variability used more commonly as a statistical procedure. To calculate the standard deviation, all the deviations are squared (which removes any negative numbers), then the total is divided by the number of scores, and the final step is to find the square root. You might see the term variance which is sometimes used by researchers and is the standard deviation squared. Variance is frequently found in ANOVA procedures.

Means and Correlation

Finds relation to average score (normal distributions)

Mean used with interval and ratio data Standard score: difference expressed in SD units Simplest is z-score (raw score – mean/SD)

Pearson r most commonly used

Empirical Rule 68% = 1 SD 95% = 2 SD 99.7% = 3 SD

R² is coefficient of determination

Variance accounted for by the correlation

Correlational coefficient used with multiple variables

Continuum of -1.0 to +1.0

Slide 9 Transcript A percentile value is used with ordinal data and shows only how many others scored lower, sometimes referred to as a norm-referenced score. To find where individuals are in regard to the average, researchers use the mean value with interval and ratio data. A standard score shows how far a score is from the mean in terms of standard deviation units, the simplest standard score is the z-score which is the raw score minus the mean divided by the standard deviation. Other versions of standard scores use a pre-specified mean and standard deviation. The standard deviation is found mostly with normal distributions, and the empirical rule is commonly applied. The empirical rule is the one where we say 68% of the scores will be within one standard deviation of the mean, 95% within 2, and 99.7% within 3 standard deviations. Earlier, we covered the correlational research design. There also is a correlational statistic. As we learned previously, measures of central tendency and variability are used when there is one variable. To understand the relationships with two or more variables, we can develop a correlational coefficient. You may recall this tells us the direction of a relationship, positive or negative, and also tells us the strength of the relationship on a continuum of -1 to +1. The most used measure is the product moment correlation called the Pearson r. There are many others that typically are used with different types of variables and data. A particular one to know about is the coefficient of determination, or R2, which is the Pearson squared and expresses how much of the variance is accounted for by the correlation.

Inferential Statistics

Population parameters

Inferential statistics assess likelihood, not proof

Determine probability sample results

Test of null hypothesis Usual criterion is 5% due to chance, reject null hypothesis Expressed as significance level (alpha)

Educated guess

Generalization from sample is probability, not a guarantee

Inferential statistics tell how close to parameters

Research hypothesis Statistical hypothesis

May reject null hypothesis and accept research hypothesis

Slide 11 Transcript Inferential statistics use techniques for determining the probability that results obtained from samples are the same as what would have been found in the whole population. You might recall that evidence about samples is expressed as statistics, while evidence about a population is expressed as parameters. It is important to keep in mind that inferential statistics using data from samples assess likelihood, not proof. The extent to which results from a sample can be generalized to a population is expressed as a probability, not a guarantee. Previously, we covered some of the threats to validity and such, for example the issue of sampling error and standard error of the mean. Even under ideal conditions, samples will never exactly equal what is in the population. So, inferential statistics tell us how close we come to the presumed actual parameters. Another function of inferential statistics is to test hypotheses. As we noted before, a research hypothesis differs from a statistical hypothesis. A research hypothesis is a prediction, an educated guess. A statistical hypothesis, however, is a test. Usually it means a test of the null hypothesis, that there is no real effect beyond chance. So, if we were to compare two means and find the difference between means to be something that would occur, say, once time in a thousand, the conclusion might be that something other than chance was making the difference, maybe what we had introduced as the independent variable. The most common criterion used for this is 5%, or the probability that chance was the factor 1 in 20 times. This is known as the significance level, expressed as alpha, and we would call this a statistically significant result. When results are due to something other than chance, we reject the null hypothesis It would be correct to say we reject the null or statistical hypothesis and confirm the research hypothesis, and it is important in your reporting that you specify to which of these you are referring.

Types of Error and Power

Type I error (alpha) Reject null hypothesis when true

Claim that results not due to chance, but were false positive To decrease likelihood, increase significance level

Tradeoff Reducing risk for one error increases risk for another

Type II error (beta) Fail to reject null hypothesis when false More serious of error types Missed a significant finding To decrease likelihood, increase significance level

Type II error (beta) Allows rejection of false null hypothesis Increased power reduces risk of Type II error

Slide 13 Transcript It is possible to make errors in hypothesis testing. When we reject the null hypothesis, but it is true, this is called Type I or alpha error. This means we said the results were not due to chance, but in fact they were. Sometimes you may hear this referred to as a false positive. Type I error might occur when the sample was atypical, for instance. Now, to decrease the possibility of Type I error, the researcher could increase the significance level, say from .05 to .01 to further reduce the possibility of error, however, that increases the likelihood for Type II error. Type II error, also called beta error, occurs when we fail to reject the null hypothesis when it is actually false. Typically regarded as the more serious of the two types of error, Type II means we have missed a significant finding. To reduce the likelihood of Type II error, the research would increase the significance level, but, this increases the possibility of Type I error. So, there is a tradeoff when making adjustments, because when you reduce the risk of making one type of error, you increase the risk of making another. To reduce the risk for Type II error, a researcher can use the largest sample size possible, maximize validity and reliability measures, and use parametric rather than nonparametric statistics. It is worth noting that when testing more than one statistical hypothesis in the same study, the probability of Type I error increases. If using the .05 level of significance, testing two hypotheses changes the likelihood from 1 in 20 to 2 in 20 risks of error, and so on with each added hypothesis. The ability of a significance test to identify a true research finding is known as power and allows the researcher to reject a null hypothesis that is false. Increasing power reduces risk of Type II error.

Parametric Tests

Most preferred and most powerful tests for significance

Tests depend on scale of measurement used

Assumptions to be met include: Data are interval or ratio Normal distribution Selection of participants is independent Sample means and variances nearly equal

Typical tests are: t-tests ANOVA or ANCOVA Pearson r Factor analysis Structural equation modeling

Slide 15 Transcript The most preferred tests of significance are parametric tests because they are usually more powerful. These tests can be used with nonexperimental, quasi, and experimental research designs. Remembering that parameters refer to populations, there are several assumptions to be met before applying a parametric measure. These include the requirement that data are interval or ratio, that the population is normally distributed with few at the extremes, that selection of participants is independent and selection of one will not affect selection of another, which is satisfied when random selection is used, and finally that when several samples are involved, the means and variances are nearly equal, In actual practice, except for independence, a small violation of these assumptions does not greatly affect the results of tests for significance.. For parametric tests, there are many kinds of measures that can be applied, and they depend in part on the scale of measurement. Remember that the scale is whether data are nominal, ordinal, interval, or ratio. However, typical measures considered include t-tests, ANOVAs, and the Pearson r for correlation. In selecting the test, researchers must consider how many factors are being tested, the number of groups, and whether between or within groups observation is used. Some of the other parametric tests used are factor analysis and structural equation modeling.

Nonparametric Tests

Used with nominal and ordinal data, skewed distribution

More difficult to reject null hypothesis (need large sample)

Typical tests are: Chi-square goodness of fit Mann-Whitney U Kruskal-Wallis Wilcoxon signed rank Fisher’s exact

Appropriate when parametric assumption violated

Limitations: Multiple hypotheses not tested Regressions not conducted (no variances)

Slide 17 Transcript These tests are used with nominal and ordinal data and when the distributions are skewed. Nonparametric measures are appropriate when a parametric assumption has been greatly violated or the nature of distribution is not known. Because nonparametric tests are less powerful than parametric measures, it is more difficult to reject a null hypothesis at a particular level of significance. To offset this, a very large sample is needed. Then too, not all hypotheses lend themselves to nonparametric tests. A limitation with nonparametric measures is that multiple hypotheses cannot be tested. Also, you would not be calculating variance using a nonparametric test, so regressions are not conducted. The decision on whether nonparametric tests are appropriate to test significance requires knowing how many groups are involved and whether participants are observed between or within groups, remembering that usually only one variable is being examined. One of the tests used only with nonparametric measures is the chi-square test for independence that analyzes nominal data. Also called goodness of fit test since it determines whether two categorical variables are similar or different from the frequencies expected. Some of the other nonparametric tests are the Mann-Whitney U, Kruskal-Wallis, Wilcoxon signed rank, and Fisher’s exact test. So, nonparametric techniques exist for relatively simple statistical analyses, like comparing measures of central tendency or testing statistical significance of correlations.

Effect Size and Confidence Intervals

Rejecting the null hypothesis means an effect is present

Difference between observed mean and expected mean Magnitude of effect not known

Measures of effect size Cohen’s d Proportion of variance Cramers’s V (Chi-square)

Effect size identifies magnitude of effect in population

Difference determined in standard deviation units

Confidence interval Estimated range in which true population mean lies Adds error mean to confidence level limits

Slide 19 Transcript When testing the null hypothesis and it is rejected, it means the researcher has found an effect from the variable examined. Usually this is determined when the mean difference between what was observed and what was expected is significant. However, it does not indicate the magnitude of the effect in a population. To determine this, an estimate known as effect size is computed. This can be done with parametric or nonparametric tests for significance, and can be used with most research designs, whether causal or not. Often, effect size is an important issue with meta-analyses comparing several studies. Calculating effect size was covered in a previous module and determines how much of a difference each intervention has made in terms of standard deviation units. This is done using Cohen’s d. Another measure is proportion of variance that reports the percent of variability in a dependent variable explained by the intervention. A third measure is Cramer’s V that is used with the Chi-square test. When a researcher wants to estimate a population mean, they also want to be confident they are getting it right. To do this they calculate a range within which the population mean probably lies. We start with a confidence level, like 95% (or the inverse of the .05 level of significance). That represents a particular point. To include the range, or interval, of confidence we adjust by adding to each end of the range the standard error of the mean. This creates a bracket within which the researcher can be confident the true population mean lies. In more complicated calculations issues like degrees of freedom and t-multipliers are involved. Well, that wraps up this session. Thanks, and have a healthy and productive week ahead.