1
Running head: EXPLORING RELIABILITY AND VALIDITY
Exploring Reliability and Validity
Fallon Graham
Liberty University
2
EXPLORING RELIABILITY AND VALIDITY
Type of Reliability and Validity Used
According to the Values and Motives Questionnaire: The Technical Manual (n.d.), the
Values and Motives Instrument, or the VMI, uses internal consistencies reliability by measuring
the Cronbach’s Alpha of each item on the scale. Reliability is the degree to which the consistency
of a test characteristic is measured. It seeks to determine the consistency, reproducibility, and
dependability of a test (Drummond, Sheperis, & Jones, 2016; What makes a good test?, n.d.).
The alpha coefficient () indicates the levels of internal consistency of the questionnaire. The
higher the inter-correlations between items, the higher the alpha coefficient. These measurements
evaluate if the various items on a test measure the same content domain, construct, or trait
(Drummond et al., 2016; Values and motives questionnaire, n.d.).
The Values and Motives Questionnaire (n.d.) measures construct validity to indicate its
internal consistency, or inter-correlations, validity. Validity refers to the characteristic being
measured and how well it’s measured (What makes a good test?, n.d.). Construct validity
measures scientific developed concepts, or ideas, that describes behavior (Drummond et al.,
2016). Evaluating the internal structure, or construct validity, of an assessment reveals if the
components measure an underlying single construct (Drummond et al., 2016). A validity
coefficient indicates the degree of the relationship between a test and its measure of performance.
The higher the coefficient, the more beneficial the test’s use can be (What makes a good test?,
n.d.). The VMI evaluates construct validity by demonstrating the consistencies of scores between
the VMI and other tests. These other tests measure similar and dissimilar constructs (Values and
motives questionnaire, n.d.).
Reliability and Cronbach Alpha Coefficients
3
EXPLORING RELIABILITY AND VALIDITY
Using the Evaluation of Reliability Coefficients table by Drummond, Sheperis, and Jones
(2016), the VMI alpha coefficients reveal the following: finance and aesthetics were high
reliability; affiliative, altruistic, affection, security, traditional, and ethical were acceptable
reliability; moral, independence, and social desirability were moderate/acceptable reliability;
and, achievement and infrequency were low/unacceptable reliability (Values and motives
questionnaire, n.d.). According to the article “What makes a good test?” (n.d.), the reliability
coefficients reveal the following: finance and aesthetics were considered good reliability ( =
0.83); affiliative, altruistic, affection, security, traditional, and ethical were adequate reliability (
= 0.70 - = 0.79); and, moral, independence, social desirability, achievement, and infrequency
were limited capability ( < 0.70). The VMI scales reveal a moderate percentage of the scales to
be in the good (two scales) and adequate (six scales) categories, while a slightly smaller
percentage were in the limited capability (five scales) category. This reveals that a slightly larger
percentage of the scales have a good to adequate possibility of inter-correlating, while the other
scales may lack homogeneity and interrelated consistencies (What makes a good test?, n.d.).
Sample Size and Nature of the Population
Validity
According to Values and Motives Questionnaire (n.d.), the VMI was given to 159
Psychology and MBA students as a personal assessment into an introductory class. Other tests
were also used to validate the VMQ by evaluating correlations between the two tests. These
comparisons include the VMI and MAPP (Motivational Appraisal of Personal Potential), VMI
and 16 PF (Personality Factor), and the VMI and OPP (Occupational Personality Profile). The
VMI and MAPP population included a sample of 59 volunteering Psychology undergraduates.
The VMI and 16 PF used a sample of 100 MBA students from a prestigious business school as
4
EXPLORING RELIABILITY AND VALIDITY
an introduction to personal assessment methods. The VMI and OPP used a sample of 59
Psychology undergraduate volunteers, which seems to be a typo error as it reads “VMI and
MAPP” (Values and motives questionnaire, n.d., p. 24). This could cause a bit of confusion as it
seems to provide false information and leaves the audience guessing at the details of the sample.
Nevertheless, the populations of the other tests seem to be comparable to the VMI. The issue that
could arise and affect the validity is the error in population information of the VMI and OPP, the
variance in sample sizes of the tests, and the differences in nature of the populations.
VMQ Norming Population
The VMI population were males (68) and females (87), psychology students, and MBA
students (Values and motives questionnaire, n.d.). Due to the nonuse of a variety of populations,
the VMI may not be representative of the population the reader may want to use. The VMI does
not provide age, ethnicity, race, SES, or geographic location of the sample population. Due to the
lack of demographic information, it is not sufficient to draw conclusions or generalize the results
to other populations. The sample sizes of 159, 100, and 59 are not sufficient to capture various
traits of other populations. The population size difference between the females and males could
also interfere with generalizability. The nature of the populations, psychology or MBA students,
are consistent strengths between the VMI, MAPP, 16PF, and OPP.
Opinion
Overall, the VMI has moderate reliability, moderate validity, and a poor representation of
generalizability based on the sample populations evaluated. Based on the aforementioned
findings, the authors did not sufficiently establish the reliability and validity of the VMI. The
concerns consist of the following: the percentage of good and adequate reliability versus the
percentage of limited capability reliability; various scales revealing poor reliability coefficients,
5
EXPLORING RELIABILITY AND VALIDITY
which reflect a lack of homogeneity; the variance in sample sizes of the VMI, MAPP, 16PF, and
OPP; the number of females to males in the population sample of the VMI; and, the lack of
demographic information and its effect on generalizability.
References
Drummond, R. J., Sheperis, C., & Jones, K. D. (2016). Assessment procedures for counselors
and helping professionals (8th ed.). Retrieved from
https://etext.pearson.com/eplayer/pdfbook?
bookid=84494&userid=23999876&smsuserid=93882690&languageid=1&roletypeid=2&
scenario=1&sessionid=f9bc041dd0d66ec2e4fa571ed04f581e&invoketype=et1&bookserv
er=1&platform=1027&uid=20180123101440&ubd=20190115110728&ubsd=201901151
10728&pagenumber=91&hsid=c6114f35bd98048c380cda0a31cfcc21
Values and motives questionnaire: The technical manual (n.d.). Retrieved from
http://www.psytech.co.za/images/PsytechSA/VMI/VMIMan.pdf
What makes a good test?. (n.d.). Retrieved from
https://learn.liberty.edu/webapps/blackboard/content/listContent.jsp?
course_id=_490037_1&content_id=_28224414_1
Powered by TCPDF (www.tcpdf.org)
