Seeking help with week 4 UOP PSY/525 individual assignment.

1

week 4 Individual Assignment:

scale creation

DUE DATE

This assignment is due on Monday, Day 7 of Week 4.

ASSIGNMENT

For this assignment you will create a 5-question test to measure one of the psychological

characteristics from a list provided. Write a 1,050- to 1,400-word analysis of the process of

developing your scale, containing the following information:

1) Choose a construct you would like to measure from the following list:

a) Introversion-Extraversion

b) Test Anxiety

c) Social Phobia

d) Self-Esteem

e) Masculinity-Femininity (also called Androgyny)

f) Emotional Intelligence

g) Conscientiousness

h) Sociability

i) Flexibility/Adaptability

2) Create an operational definition of your construct using at least three peer-reviewed

journal articles as references. Be sure to cite these references in the text when you

present your operational definition.

3) Select and list topics for five items used to sample the domain. (They can be your own

ideas or topics found in your references. The five test items will measure these five

topics. If you did not write them, give a citation indicating where you found them.)

4) Select a format for your items (multiple choice, numerical rating scale, Likert scale,

true/false, etc.) that is appropriate for the construct you are measuring. Justify your

choice. Explain how you plan to score the individual items and compute each person’s

total score on the assessment.

5) Format the items into an instrument with which you would query respondents. Justify

whether this is an interview or self-report instrument. Put your instrument on a page by

itself at the end of your assignment.

~ ~ ~

-- For the remainder of the assignment, describe your plans for an initial tryout of your little

test. Assume that you plan to give your five-item scale to a sample of people, and to collect

their scores (in other words, you will give a pilot test). You plan to use the information

from the pilot test scores to determine the norms for the scale, to determine its reliability

and validity, and to determine whether the items are measuring effectively. Note that you

do not have to actually perform this pilot test. Simply describe what you would plan to do.

Because these are plans, the remainder of the paper should be written in the

future tense. --

2

6) Describe how you plan to norm your instrument.

a) To whom would you want to generalize the results of your pilot test? b) Describe the type of individuals you would use as test takers for the pilot test. c) How many people would you give the pilot test to?

7) Describe the steps you plan to follow to estimate the reliability of your scale.

8) Describe the steps you plan to follow to measure the validity of your scale.

9) You will want to perform an item analysis after you receive the results of the pilot test.

Describe the steps you plan to follow to determine whether any of your five test items

had poor discrimination.

FORMAT SPECIFICATIONS

DOCUMENT TYPE REQUIRED: MS Word, .RTF, or .PDF document

LENGTH: 1,050 to 1,400 words

INSTRUCTOR’S FORMAT: Use the template provided by the instructor. See details below.

APA FORMAT REQUIREMENTS: double-spaced, 12-point Times New Roman font, title page,

page headers, citations in text, reference list (abstract is optional)

REFERENCE REQUIREMENTS: at least 3 references from peer-reviewed journals plus any

others you think appropriate. The additional references are not required to be from

peer-reviewed journals.

PAGES AND HEADINGS FOR THE WEEK 4 PAPER

The headings in black font below are the headings to use in your paper for Week 4. If you

don’t use these headings, I’ll deduct 10 points from your grade.

Center the References heading; the others are at the left margin.

PAGE 1: TITLE PAGE PAGES 2 TO (?): BODY OF THE PAPER

Operational Definition of Construct

[In this section define the construct using an operational definition, referencing the three peer- reviewed journal articles.]

Topics for Items

[In this section list the five topics you’ve chosen for your test to measure. Don't list the items, just the topics they will measure.]

3

Scaling

[In this section explain how you plan to score the 5-item test, and why you would use this approach.]

Norming

[In this section describe how you would collect data to be the basis for the norms for the test.]

Estimation of Reliability

[What steps would you follow to estimate the reliability of your test?]

Estimation of Validity

[What steps would you follow to estimate the validity of your test?]

Item Analysis

[Based on the item analysis from pilot testing, you would want to determine which items discriminate well. How would you investigate item discrimination?]

Conclusion

[Write a concluding paragraph.] ON A NEW PAGE AFTER THE CONCLUDING PARAGRAPH:

References ON THE LAST PAGE:

[The last page should contain your 5-item instrument, formatted for presentation to test-takers.]

WATCH OUT FOR . . .

Students often go wrong by . . .

* not including three references from peer-reviewed journals

* not citing 3 peer-reviewed references when they present their operational definition of the

construct they are measuring

* simply naming the methods of reliability and validity they would use, instead of actually

listing the steps they would use.

* failing to answer question 9 about item analysis (see textbook pp. 254-260)

4

WEEK 4 INDIVIDUAL ASSIGNMENT FREQUENTLY ASKED QUESTIONS (FAQ)

Q1: Is it OK if I don't use the headings you provided, if I do a good job?

A1: No. If you don't use them I'll reduce your grade by 10% (you can revise it and resubmit it; I'll treat it as a late paper).

Q2: Is it OK if I create a test on a construct that wasn't on your list, if I do a good job?

A2: No. If you don't use one of them I'll reduce your grade by 10% (you can revise it and resubmit it; I'll treat it as a late paper).

Q3: Is it OK if I use items from a test that is used where I work?

A3: Yes, if (a) the items measure one of the constructs on the list, and (b) you include only 5 of the items.

Q4: If I use items from an existing test (one that I find online or at my job), should I describe how the original designers handled norming, reliability, validity, item analysis, etc.?

A4: No. Describe what you would do if it was a test you were designing. Write it in the future tense. If your paper sounds like a review of someone’s published article I will count

off.

Q5: Am I supposed to design an experiment for this assignment?

A5: No. This assignment does not involve designing an experiment or answering a research

question. The only thing you need to design is a 5-item test.

Q6: What is a peer-reviewed journal?

A6: A peer-reviewed journal is a journal related to an academic discipline or professional practice. The “peer reviewed” description refers to their practice of having several

professionals review and approve each article before it is published. Often peer-reviewed

journals will have the word “Journal” in their title, such as:

The Journal of Applied Psychology

The Canadian Journal of Behavioural Science

American Journal of Community Psychology

5

The Journal of Black Psychology

although some will not, such as:

Human Factors

Educational Psychology

American Psychologist

You can almost always tell by simply looking at the article. If it looks like it’s from a

professional journal, it’s OK. If it looks like it’s from a magazine, it’s not OK. If it is from an

encyclopedia, website, or a book of any kind, it’s not OK.

Note that the EBSCO side menu usually displays a check box for “Peer Reviewed Journals.”

Check that box, and you’re good to go.

Websites, dictionaries, encyclopedias, magazines, and textbooks are not peer-

reviewed journals and do not count toward your three journal articles. The DSM can contain useful information for defining your construct, but the DSM does not count as a

peer-reviewed journal. Use the University library to find articles from professional

journals.

Q7: What is an operational definition?

A7: An operational definition is simply the process of defining something by describing how you’d measure it. The operational definition of a psychological variable is usually a pretty

specific description of what you’d ask the test takers to do – “solve math problems from the

3 rd Grade curriculum,” “answer questions about happy memories from their childhood,” etc.

Q8: What do you mean by “scaling”?

A8: Scaling refers to how you will compute the scores of your test, and what kind of scale

(nominal, ordinal, etc.) the scores will be on.

Q9: What do you mean by “norming”?

A9: Norming refers to giving the test for the first time in order to gather data about its mean, standard deviation, percentiles, etc. The important issue is whom you give it to at

this point. You should give it to a sample of people who are representative of the population

you would normally give it to or generalize to. For example, if it is a test intended to be

taken by typical American high school students, it should be normed on a group of

teenagers with the proper proportions of males & females, racial/ethnic groups, rural/urban

schools, etc. You would not want to norm such a test on college students or only on home-

schooled kids, for example.

Q10: What is an “item”? A10: Item is the professional term for a test question. It’s used because many test

“questions” are not actually questions: often they are instructions (“Compute the standard

score”), self-descriptive statements (“I am usually happy”), true/false statements

(“Abraham Lincoln was President when the Civil War ended”), etc.

6

Q11: What do you mean by “item discrimination”?

A11: Discrimination is a bad thing when we’re talking about human behavior, but a good thing when we’re talking about test items. Discrimination refers to how well a test item

distinguishes between people who do have the thing we’re measuring and those who don’t

have it. For example, an item on a test to measure depression should be answered one

way by individuals who are depressed and a different way by individuals who aren’t

depressed. If so, the item discriminates well. The two main ways an item may show poor

discrimination are (a) all test takers give the same response to the item and (b) test takers

give a variety of responses to the item, but their responses are not related to whether they

“have” the thing being measured. Check what your textbook has to say about Test Tryouts

Item Analysis, and Item Discrimination.

Q12: For the “item discrimination” discussion, can I just say that I’ll assume all

the items are “keepers” because they were carefully chosen and I think they look good.

A12: Nice try, but no.

Q13: You ask us to describe the steps we'd follow to determine reliability and

validity, but I don't understand those methods very well. How do I know what steps I would need?

A13: Try Dr. Tom's Reliability and Validity Cookbook.

7

RUBRIC

TASK CRITERIA WEIGHT

I. Development (34%)

1) Choose a construct you would like to

measure from the list provided.

Measures a single psychological construct,

chosen from the list posted with the

assignment.

4%

2) Create an operational definition of your

construct using at least three peer-

reviewed journal articles as references. Be

sure to cite these references in the text

when you present your operational

definition.

Gives a clear definition, using the phrase

"defined as" or the equivalent.

5%

Reference list includes three peer-reviewed

journal articles - NOT from magazines, books,

or websites.

9%

Operational definition is accompanied by

citations in the text for 3 or more peer-

reviewed articles.

6%

3) Select and list the five topics for items

you used to sample the domain.

Lists five topics that are key aspects of the

construct.

5%

4) Select a format for your items (multiple

choice, numerical rating scale, Likert scale,

true/false, etc.) that is appropriate for the

construct you are measuring. Justify your

choice. Explain how you plan to score the

individual items and compute each

person’s total score on the assessment.

Describes the format of the 5-item test. 3%

Describes how the 5-item test would be

scored.

2%

II. Analysis of Pilot Test Development (35%)

Write a 1,050- to 1,400-word analysis of

the process of developing your instrument.

Entire paper is 1,050 to 1,400 words (score =

0 to 10)

3%

The remainder of the paper should be

written in future tense.

Discussions of norming, reliability, validity,

and item selection are written in future tense

3%

6) Describe how you plan to norm your

instrument. To whom would you want to

generalize the results of your pilot test?

Describe the type of individuals you would

use as test takers for the pilot test. How

many people would you give the pilot test

to?

States to whom the instrument would be

generalized

2%

Procedure is appropriate and described

thoroughly

4%

Characteristics of the norming sample match

the intended test takers

3%

7) Describe the steps you plan to follow to

estimate the reliability of your scale.

Lists the steps involved in obtaining

reliability.

8%

8) Describe the steps you plan to follow to

measure the validity of your scale.

Lists the steps involved in obtaining validity. 8%

8

9) You will want to perform an item

analysis after you receive the results of the

pilot test. Describe the steps you plan to

follow to determine whether any of your

five test items had poor discrimination.

Describes the process for investigating

whether any items had poor discrimination.

4%

Completed 5-Item Instrument (11%)

5) Format the items into an instrument

with which you would query respondents.

Justify whether this is an interview or self-

report instrument. Put your instrument on

a page by itself at the end of your

assignment.

5-item instrument is on a page by itself at the

end of the document

3%

Items match the 5 topics listed above in (3) 3%

Instrument is appropriate for use with

respondents.

5%

Format, Readability, and Style (20%)

Clarity, readability, organization Writing is clear, readable, and well-organized. 5%

Grammar, spelling, punctuation Uses correct grammar, spelling, and

punctuation.

5%

APA Style: Double-spaced, 12-point Times

New Roman font, title page, page headers,

citations in text, reference list

In-text citations and References list are in

correct APA format (from form below). (0-10)

5%

Pages are formatted correctly with double-

spaced, 12-point Times New Roman font and

APA-style page headers.

5%

See the Purdue University OWL website for excellent guidance on APA style: http://owl.english.purdue.edu/owl/section/2/10/