PAFF 510 MATH ASSIGNMENT
Woodward Johnson
PAFF510WeekIISlides.pptx
Chapter 2: Theories & Models
Week II – Slides
1
Chapter 2: Theories & Models
Week II – Slides
2
What we are Covering:
What is a theory?
Where do theories come from?
What is a model?
Unit of analysis
Logic models
Usefulness of a logic model
Additional issues in theory building
Finding and focusing a research question
3
3
What is a theory?
Theories are nets cast to catch what we call the world, to rationalize, to master, and to explain it. We endeavor to make the mesh ever finer & finer.“
Karl Popper, The Logic of Scientific Discovery
4
4
But, before theory . . .
Ask a Question or Observe/Identify a Puzzle
Goal: General Explanation
What is the general phenomenon you are seeking to explain?
Think in terms of concepts, not specific examples
Primary interest:
Explain change (a.k.a. variation) in the phenomenon of interest (a.k.a. dependent variable)
5
5
What is a theory?
Theories identify key variables
So we know what concepts to measure and observe
Theories tell causal stories
Often focusing on just one cause at a time
Example: broken windows theory, which looks at the variable disorder as a possible factor in crime
Theories explain variation
6
6
Dimensions & Variation
Does the variation we are interested in occur over time, across units, or both?
Spatial Variation
Multiple units are measured at one moment in time
Cross Sectional (CS) (e.g., # of giving campaigns by each non-profit in Broome County, in 2020)
Temporal Variation
Repeated measurement of one unit at different moments in time
Time-Series (TS) (e.g., # Broome county residents diagnosed each day with Covid19 from March 15th – September 1st)
7
7
Cross-sectional variation example
8
8
Longitudinal variation example
9
9
Dimensions & Variation
Can look at both space and time variation
Time-series cross-sectional (TSCS)
(e.g. Binghamton University mean GPAs across majors and semesters)
10
10
TSCS example
11
11
What is a theory?
Theories generate testable hypotheses
Hypotheses are predictions of what will happen if a theory is correct
Hypotheses can be compared with the facts, and can potentially falsify a theory
12
12
What is theory?
Theories focus on modifiable variables
(Note: this is a more PA/PP specific concern)
Social and policy research tends to focus upon modifiable variables as a way to offer guidance in policy and practice
Modifiable and nonmodifiable variables
Applied theories focus on modifiable variables—causes of an outcome that we can influence
Nonmodifiable variables cannot be changed by policy or practice
(example: policymaking in the US will be done under democratic process & norms)
13
13
Where do theories come from?
Grand social theories
Sometimes referred to as theoretical paradigms, which shape a researcher’s view of the variables and mechanisms involved in explaining human behavior
Example: Rational-choice theory
Individuals know all potential action that they can take
Will select decision which maximizes their benefits (utility)
14
14
Where do theories come from?
Academic disciplines
Such as political science, psychology, economics, etc.
Induction
Building up theory from empirical evidence and observation
Important caution about induction: You cannot test an inductive theory with the same set of facts used to create the theory
Deduction
Starting from initial ideas or logical principles
Often theory comes from both thought processes
15
15
Where do theories come from?
Exploratory and qualitative research
Linking threads of empirical evidence from exploratory studies in a field
Qualitative research often used to generate theory
This is very hard to do! But, when done well, is usually the most valuable research in the discipline.
16
16
Where do theories come from?
Theories, norms, and values
Scientific theories are positive—about how things are
Not normative—about how things should be
Still theories reflect values, beliefs, and interests
(Example: we study human rights violation because we normatively care about curtailing them in the future)
17
17
What is a model?
A graphical or mathematical representation of two items
Variables
That can take on different values or assume different attributes—they vary
Relationships
That show how change in one variable produces change in another variable
18
18
Why do we do this?
It forces you to make your assumptions explicit
Establishes that implications follow logically from assumptions
19
19
More on assumptions
Explicit statement of our assumptions leads us to think precisely about our concepts
What are the precise definitions?
Thinking about the assumptions could lead to promising lines of research
Are the assumptions in a well-known theory flawed?
Assumptions do not always hold in all cases.
What are the implications if the assumptions do not hold?
20
20
Why do we do this?
Empirical tests of hypotheses are not the only way in which we evaluate theories: we also evaluate them on logical and other grounds.
It is worth our time to “kick the tires” before we invest a lot of time collecting data
21
21
What question should we ask during this stage?
Is your theory causal?
It should explain how and why change in the values of the independent variable change the values of the dependent variable.
Does your theory generate testable hypotheses?
For a theory to be testable, it must be falsifiable
You should justify how your measurements match your concepts
22
22
Hallmarks of a good model
Keep it simple
Connect x and y via the shortest explanatory route
Parsimonious models are better models
Occam's Razor
Is your model novel and interesting?
Your model should make new predictions
Your model should not propose explanations that are obvious to all
23
23
What is a model? A quick refresher
Independent and dependent variables
X Y
“Cause” “Effect”
Independent Dependent
Causal mechanisms
The process by which change in X is presumed to cause change Y
24
24
Note: we use a ton of different terms here
Left-hand right-hand
They all mean the same thing
25
25
Path diagram – a basic bivariate design
26
26
Path diagram – a basic bivariate design
Q: Indentify the independent and dependent variables?
Q: What is the presumed causal mechanism?
27
27
What is a model?
Direction of a relationship
Positive (+) relationship
High values of X tend to occur with high values of Y
X and Y vary in the same direction
Negative (−) relationship
High values of X tend to occur with low values of Y
X and Y vary in the opposite direction
28
28
Positive relationship
29
29
Negative relationship
30
30
Positive or negative?
Are these examples of relationships positive, or negative?
Age Health
Education Earnings
Class size Test scores
Air pollution Asthma
31
31
Unit of analysis
Unit of analysis
The objects or things described by the variables in a model
Same theory may use different unit of analysis
A good theory should – more often than not – explain patterns across many different units
In longitudinal research, the unit of analysis includes the time period
Days, months, quarters, years
32
32
Unit of analysis (income)
33
33
Unit of analysis
Broken Windows Theory
Disorder Crime
34
34
Logic models
Also referred to as
Program theories
Outcome-sequence charts
Theories of change
Graphical models showing how a program produces desired outcomes
35
35
Logic models
The simple bivariate model
A direct effect from the IV on the DV
36
36
Logic models
The simple causal model
A direct effect from the IV on the DV
Usually we will have variable names and give an expected direction on the arrow
-
37
37
Logic models
The simple causal model
A direct effect from the IV on the DV
Usually we will have variable names and give an expected direction on the arrow
Models are often bivariate, but reality is multivariate
38
38
Adding intervening variables
Intervening variables Variables that intervene between the independent variables and dependent variables
Also known as mediators in some disciplines, or intermediate outcomes in program evaluation
They help articulate the causal process(es) – sometimes termed causal chains – through which X produces Y
39
39
Adding intervening variables
40
40
Adding intervening variables – An example
41
41
Adding intervening variables – An example
42
42
Small Class Sizes
+
One-on-one Attention
+
Higher Test Scores
Problems with poorly thought out path models
Stop and think considerations while developing
Is there reverse causation?
Is there spuriousness?
More on this in week 4
43
43
You can have more than one intervening variable
44
44
Why we use logic models
Helps identify previously unrecognized variables to track as performance indicators
Helps in planning the design of a program evaluation
Suggests logical weak links a program
45
45
A DIY guide to logic models
Start with a single outcome or Y variable
Add a single X variable representing the program
Put the program (X) on the left and the outcome (Y) on the right
Add intervening variables between X and Y
Distinguish causal “chains” from separate “pathways”
Look for links that need explanation—consider additional intervening variables
Give nondirectional names to variables and add “+” or “–” signs to the relationships (arrows)
Make sure there is not too much, or too little, detail for your audience
46
46
Let’s practice
47
47
Logic models in program implementation
Often logic models are used to represent implementation of a program and include
Inputs
Financial, human, and material resources
Activities
Training, counseling, marketing, and other tasks
Outputs
The immediate products of activities (people trained, vaccinations given, etc.)
Outcomes
The results, including short-term, intermediate, and long-term outcomes
48
48
49
Additional issues in theory building
Moderator
A variable that influence (strengthens or weakens) the relationship between two other variables
50
50
Additional issues in theory building
Aggregation problem and ecological fallacy
Relationships that hold at one unit of analysis may not hold at more aggregated levels
Key takeaway – often our unit of analysis matters
Think carefully
51
51
Additional issues in theory building
Hierarchical (multilevel) models and contextual variables
It’s weird the book places this here. Revisit this after we have covered regression for this to make more sense.
Key takeaways:
Many times we will have questions about society where observations exist within groups:
Students within classrooms
Patients within wards
Citizens within counties
52
52
Additional issues in theory building
Variables that influence DVs that care about may be influenced by either individual-level or group-level characteristics
Example: A student's success in the classroom may be a function of both their socio-economic background (individual-level variable) and the experience of the teacher (group-level variable)
The group-level variables affect all students in the classroom
We have to use special modeling strategies to accurately capture these affects in the real world
53
53
Additional issues in theory building
Theoretical research
Theoretical research uses existing facts to gain insight, make valuable predictions and recommendations.
54
54
How to find and focus research questions
Research question
The question that motivated the researcher to do the study
Applied research questions
Arise from the practical concerns of policymakers and practitioners
A good research question . . .
should be answerable
may be descriptive or causal
should be positive, not normative
55
55
