Methods of Measuring

Tips when Evaluating Instruments

Whether you are evaluating question items that you yourself created or existing question items in instruments, it is a good idea to review instruments with a critical eye. You may want to review instruments with these questions in mind:

• What is the instrument measuring? Does it measure what you want to actually measure? • Are there any peer-reviewed articles published that establish the reliability and validity of the

instrument? Is the instrument judged to be reliable and valid? • Is the instrument reliable? In other words, does it produce consistent data? • Who is the instrument targeting (i.e., children, adolescents, adults, patients, etc.)? Which

cultural group(s) was the instrument normed on? If you are using an instrument designed for one population but you want to use it for another population, be sure to ask if this makes sense.

• What cognitive or developmental level is the instrument designed for? • What is the context or setting is the instrument designed for? For example, is it meant to be

administered in a group setting? Or individually face-to-face? • How practical is the instrument to administer?

o How long does it take for it to be completed? o How expensive is it to purchase it? o Does it involve someone with a particular expertise or license to administer the

instrument? • Is the instrument free of any gender or cultural biases? • Are the question items worded clearly? • Are there any double barrel question items (two thoughts embedded in one question)? For

example, if you asked a client: “How depressed and anxious are you?” These are two separate concepts. If the client responds: “Very.” Then how will you know if they are responding to being depressed or being anxious?

• Are the question items worded in a way that leads to a specific response? (e.g., how satisfied are you with the consumer-friendly services we have here?)

• Are the questions worded succinctly? • Do the question items contain any unfamiliar terminology or jargon? • Do the question items contain double negatives?