need someone who know statistics
byorv20n7
lecture_8.pptx1
EDRE 9400 Measurement Theory and Test Construction
Putting it all together…..an example
Entering data into ConstructMap software
2
Example…..
Today we will go over an example analysis of the technical adequacy of a test focusing on the concepts we have covered thus far.
Your next homework will involve the steps that we discuss today…..
3
Example…..
Steps to analyzing your instrument
Step 1: create your construct map and your instrument and administer to respondents.
Step 2: enter data into SPSS to obtain classical measurement theory statistical procedures
Step 3: enter data into files for IRT analysis to obtain estimates of theta, delta, SE’s, test information, and separation reliability.
BTW, separation refers to fact that we are separating the true variance from the total variance.
4
Adding a New Person…..
To add new cases go to Edit and Add Student
5
Adding a New Person…..
Now there are 10 new cases which you can add data to…..not as versatile as Excel or SPSS!
6
Adding a New Item…..
New variable is created and you can manually type in the data.
7
Adding a New Item…..
Just type the abbreviated (short) and the long name
8
Adding a New Item…..
Just type the abbreviated (short) and the long name
Notice the active item box. That is how you indicate to the software that the new item will be included in calibration of estimates….
9
Creating Data Files Outside of Construct Map
Pages 8 and 9 of the Quick Start Guide
http:// bearcenter.berkeley.edu/sites/default/files/cmap/cm44_qsg.pdf
Let’s look at the students and items files for the PF10 dichotomously scored example…
10
Creating Data Files Outside of Construct Map: Items File
Let’s open an existing Items file in Excel:
First line is:
System model information
11
Creating Data Files Outside of Construct Map: Items File
We can open the Items file in Excel:
s for system
Min value on graphs
Min logit value for estimation
Max value on graphs
Max logit value for estimation
Max quadrature points for estimation, keep at 20
12
Creating Data Files Outside of Construct Map: Items File
We can open the Items file in Excel:
Model type: r = rating scale; d = dichotomous; p = partial credit; m = mixed dichotomous and partial credit. Note: you can change this in the calibration window later if you want.
13
Creating Data Files Outside of Construct Map: Items File
We can open the Items file in Excel:
Next line is:
Variable entries
14
Creating Data Files Outside of Construct Map: Items File
We can open the Items file in Excel:
Put a v, stands for variable
Short name of your variable
Max score
Long name of your variable
Mean of population; optional
Variance of population; optional
15
Creating Data Files Outside of Construct Map: Items File
We can open the Items file in Excel:
Enter step difficulties is you want; optional.
Covariances; optional
16
Creating Data Files Outside of Construct Map: Items File
We can open the Items file in Excel:
Item entries
17
Creating Data Files Outside of Construct Map: Items File
We can open the Items file in Excel:
Put an i, for item
Name of item set
Short name of variable
Generic name of the variables
Long name of variable
Item comments; optional
18
Creating Data Files Outside of Construct Map: Items File
We can open the Items file in Excel:
Put in the admin date or insert word “true”
Step interval difficulties; optional
Max score for item
19
Now Let’s Create the Students File
20
Creating Data Files Outside of Construct Map: Students File
We can open the Items file in Excel:
Date items entered
21
Creating Data Files Outside of Construct Map: Students File
We can open the Items file in Excel:
I expanded the column so you can see date format
22
Creating Data Files Outside of Construct Map: Students File
We can open the Items file in Excel:
Indicate Name; but note that research numbers can go there too!
Put item set name then “:” and then short name of variable. Make sure names match the item file!!
23
Be Sure to Save Your Data Files in
Tab Delimited Format (you can use
Excel or SPSS or whatever)
24
Classical Test Theory
Let’s examine the PF10 dichotomously scored data using classical test theory
Just item difficulties and reliability....
25
Classical Test Theory
Open the items data in SPSS
Use the open with text option.
Note: you need to clean up the file so that it only has row of column names and the scores…otherwise you won’t get column names entered into SPSS
26
Classical Test Theory
Test item difficulties are often called p-values
Do NOT get this mixed up with type I error rate p-values!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!!
In this case, the p-value is just the mean of each item. It represents proportion of sample that scored a 1 on the item.
27
Classical Test Theory
Test item difficulties are often called p-values
28
Classical Test Theory
Name of item
Mean for items after removing that item
Variance for items after removing that item
See next slide
Reliability when item removed
29
Classical Test Theory
How does each item correlate with totals (after removing that item). Used as index of Item Discrimination. Does item help to determine which respondents score higher or lower on the instrument?
