Identifying Variables and Choosing Samples

IDENTIFYING VARIABLES & CHOOSING SAMPLES Variables and Samples

Slide 1 Transcript

This module will focus on variable and samples to determine a methodology to use, in relation to what you are seeking to know, a narrowing of the potential influences needs to occur. These influences are known as variables and play a crucial role in your research intentions. For the next few minutes we will explore some of these. Once the variables are considered, the next step in obtaining data is to identify a source. Often this will be one or more samples taken to better understand and resolve the research question. Being able to assess the validity of findings taken from samples, and the extent of reliability if repeated, are objectives to focus on here.

Clarify the Problem or Purpose

Single variable research

Remaining open to outcomes

Multiple variable research

Crafting a hypothesis or research question

Slide 3 Transcript If you are studying a single phenomenon you will be addressing a single variable primarily. Otherwise, you can study two or more variables, but there will be tradeoffs. Depending on which type of research approach you choose, you may develop relationships or compare ideas which may not be that clear at the beginning of your research. So, it is important not to imply you already know the outcome. With a hypothesis, you believe there will be confirmation of a difference you suspect. The research question will be working from a tentative definition to launch the effort.

Relevance of Hypothesis or Research Statement

Hypothesis for quantitative research designs

Both apply to data collection, analysis, and interpretation

Research statement for qualitative research designs

Both express the purpose, scope, and direction of study

States assumptions

Provides delimitations

Slide 5 Transcript We can expect to see a hypothesis used with experimental research, while research questions (also called statements) are more common in qualitative designs. Both provide guidance for the type of data to collect and suggest how analysis and interpretation may be accomplished. As mentioned previously, a research statement announces the purpose or reason for the inquiry, scope of how much of the problem is being taken on, and direction or trajectory of the outcomes expected. Another factor is in delimiting the research which is to say what is not going to be studied and why. This can be thought of as establishing parameters for your inquiry, and necessarily will limit how far you can generalize or broaden the applicability. Part of setting the limitations is in stating assumptions, which must be carefully considered since they invite critical assessment and the need to justify including them while excluding others.

This is where variables and sampling play a critical role, since the extent of influences that bear upon your focus for study, and the characteristics of samples to be used as evidence, can become entangled and may dilute the strength of your findings.

Various Variables

Quantitative Approach Qualitative Approach Seeks to measure outcomes Discovers variables for study at a

later opportunity

Other Types of Variables Intervening Organismic Dummy Confounding Extraneous Dichotomous Control Criterion Latent Manifest Predictor

Independent variables controlled by researcher

Mediating variables explain effects

Dependent variables measure the effect

Slide 7 Transcript In quantitative research, we typically refer to independent and dependent variables. This can appear simplistic. We often hear that the independent variable is what is introduced or manipulated by the researcher to see what effect it has on something, while the dependent variable is a measure of the extent of that effect. Also playing key roles are any mediating variables that help explain why the independent variable produced this effect – so, the independent variable influences the mediating variable which in turn influences the dependent variable. We can begin to see the complications where multiple independent, mediating, and dependent variables are interacting. There are many other names for variables, including intervening, confounding, or control variables. For instance, variables might include on-time airline performance, fatal accident rates, age of an airline fleet, customer satisfaction, weather delays, or maintenance costs per mile. In qualitative designs, variables emerge during the inquiry and are of less emphasis because numerical measures are not used. While quantitative measures seek to confirm something, qualitative concepts emerge and subsequently can generate hypotheses and associated variables to measure.

When to Use Which Variables

Important for design and statistical analysis

Qualitative Not usually involved with qualitative designs Non-numerical

Dependent variable considerations

Quantitative To obtain numerical properties Ordinal Ranked Continuous Discrete Variable Interval

Slide 9 Transcript Identifying which variables will be considered is important in helping to choose which research design to use, and, what statistical analysis is appropriate. Quantitative variables are numeric and represent a measurable quantity. For example, measures of height, temperature, or speed can be expressed in centimeters, Celsius, or kilometers per hour. Measures can be ordinal, ranked, continuous, discrete, variable, or in intervals. Depending on how you intend to measure the dependent variable will determine how you can express relationships among other variables. In quantitative designs, variables will vary by amount, whereas in qualitative designs properties will be non-numeric and identified with labels or categories. So, quantitative variables are measures on a continuous scale (or in identifiable discrete categories), while what might be considered qualitative variables are generally not on a continuous scale and are mostly in discrete categories.

Establishing Validity

The Issue is Credibility Many Types of Validity Face Construct Content Criterion Ecological (Multiple types associated with statistical results)

Strong Evidence That Assertion is Genuine

Slide 11 Transcript Validity refers to how credible a research finding might be – that is, are the findings genuine and believable. We have heard the definition of validity as “the extent to which something measures what it is supposed to measure”, but this have never been a satisfactory explanation for me. Validity derives from a Latin term for strong. So, if something is valid, there is strong evidence that is it is genuine. Several types of validity have been identified and include face validity (or it seems apparent), construct validity (is measuring the variable of interest), content validity (is measuring what is contained in the construct), criterion-related validity (the measure can predict an expected outcome), and ecological validity (measures what actually exists in real conditions).

Demonstrating Reliability

If Valid, Then Reliable Two Basic Types of Reliability

Internal Split-half Cronbach’s alpha

Repetition Means Reliable

External Test-retest Inter-rater Cohen’s kappa

Slide 13 Transcript If data are valid, they also must be reliable. Reliability indicates how likely the same result would be found if the study were repeated. There are two basic types of reliability – internal and external. Internal reliability indicates how consistent the measure is within itself. A common method for this is the split half procedure. For example, you might split an examination or set of measures into two equal parts (selecting items at random, for instance), then comparing if the means are significantly different. External reliability measures how different results are compared with each other over several trials or episodes. A couple of examples would be test-retest where the same examination administered to different participants remains stable over time with similar score distributions. Another measure for external reliability would be the inter-rater assessment which would evaluate the degree to which different raters give consistent estimates for what is being rated. Statistical measures for reliability might include using several measures for internal consistency with the same variable, a measure known as Cronbach’s alpha. For inter-rater reliability you might use a measure known as Cohen’s kappa.

Methods for Sampling

Sample Convenience Quota Stratified Systematic Cluster

Nonprobability Sampling

Probability Sampling

Taking participants from the entire population True random sampling (or, without replacement)

Taking a portion of the population to study But, is that portion representative

Taking participants from the entire population True random sampling (or, without replacement)

Other Names for Sampling

Slide 15 Transcript Seldom can a researcher capture everything or everyone who would be included in interpreting results from a study. So, they take a portion of the total population and, hopefully, infer that what applies for the sample is also true among the population. Clearly, issues of representativeness arise and whether the sample accurately resembles what would be found respectively in the population under consideration. When humans are involved, were refer to them as participants, not as subjects. Two categories are often used to describe sampling – probability and nonprobability. These terms refer to whether you expect to be able to predict something accurately about the population. Probability sampling involves selecting directly from an entire population, while nonprobability sampling selects participants from the population who are accessible, thereby excluding some. This was a problem in the days of designing drug testing samples for the workplace. A truly random sample, for instance, meant that everyone in the target group was accessible for selection. However, when some members may have been unavailable because of sickness, travel, or other reasons, only those remaining were available for testing. Various names for sampling include convenience sampling (ease of access to participants, e.g., whoever walks up at the time), quota sampling (an equal number per category are chosen), stratified random sampling (which differs because there are subgroups identified), systematic sampling (e.g., every third person), cluster sampling (readily identifiable or accessible persons in a subgroup), and so on. A word about random sampling – truly random sampling means each participant has an equal chance of being selected each time a selection is made. To do this, for example, you might have a bowl with the names of everyone on it. One is selected, and then returned to the bowl. This means they might be selected again. The more common technique used, though, is random sampling without replacement, meaning their name does not get returned to the bowl. This is what is used in the workplace drug testing scenario, for instance.

Considerations for Predicting Outcomes

Participant Reactivity

Changes (limitations) affect variables

Expectancy Effects

Experimenter Bias

Aspects relating to qualitative approaches

Confirmation bias

Sensitivity and Range Effects

We find what we look for,

and we look for what we know.

"

"

Slide 17 Transcript

There are some considerations we want to look at when we are thinking of a methodology to predict some type of outcome. Participant reactivity, which is like the Hawthorne Effect from your Introductory Psychology course, refers to influences on participants when they know they are being measured. Experimenter bias relates to when the behavior of a researcher influences results, perhaps through tone of voice, body language, and so on. Often, the researcher is not aware of these influences. It is worth noting that, in qualitative research designs, this often has been a criticism of how the results are distorted. However, it actually is at the very heart of the difference in that variety can produce some very different results and lead to unforeseen discoveries. Expectancy effects are like the self-fulfilling prophesy that David Hamburg described. I am reminded of something a neurologist who was supervising my residency said, “We find what we look for, and we look for what we know.” This is a bias studied extensively and can shape the way results are produced, collected, and interpreted. So, when setting up an inquiry, if we already think we know the results, we are biased for look for confirmation. Sensitivity and range effects involve situations where the actual measure can change or be different based on how the independent variable is applied. The range issue is one where the researcher may limit which scores or values are considered in the extreme, and therefore are not included in the calculations.

Types of Sampling Error

Sampling Error

Favoritism

RememberStandard Error of the Mean

Suggests samples are not representative

Parameter = Population

Bias

How much mean values differ NonresponseStatistic = Sample

Slide 19 Transcript

When we do samples, we introduce error. Sampling error refers to the extent to which the mean values from samples are not really comparable because they do not accurately represent identical portions of the population. Standard error of the mean refers to how far a sample mean might be deviating from the actual population mean. The error can be reduced by increasing the size of the sample. Remember, a parameter is a characteristic of a population, where a statistic is a characteristic of a sample. Inferential statistics allows a researcher to make a calculated estimate about a population parameter based on a statistic from a sample randomly drawn. Error can be introduced through bias, which means the researcher used a sampling procedure that favored certain participants for instance. For example, an on-line survey might exclude those not computer savvy or with less sophisticated experience using the software. Another type of sampling bias is nonresponse bias, where some targeted participants choose not to respond, for a variety of reasons, but which skews the results and is not representative of the population studied. These are flaws built in to the design. That’s it for variables and samples so far. Thanks for listening and have a great week.