RELIABILITY AND VALIDITY 2
Correspondence concerning this article should be addressed to Melissa G. Longley
Email: mglongley@liberty.edu
Exploring Reliability and Validity
The Value and Motives Questionnaire (VMQ) the technical manual (2020), measures the
personality values and interest of a sampling of students. Value scales are broken down into three
categories including interpersonal, intrinsic, and extrinsic. This paper will serve to provide an in-
depth analysis, review, and opinion of the psychometrics provided in the VMQ relating to
reliability and validity.
Types of Reliability and Validity
Reliability and validity are qualities used to measure and analyze the caliber of a test.
Reliability is an aspect used to determine the consistency and dependability within an assessment
(Sheperis et al., 2020). Consideration of reliability factors are observed across time, different
forms of the same assessment, and within various items within the assessment (Sheperis et al.,
2020). The VMQ discusses stability and internal consistency as types of reliability and ultimately
utilizes the most common measure of internal consistency, Cronbach’s Alpha (Psytech, n.d.). A
high coefficient alpha corresponds to high inter-correlation and shows that items on a scale are
measuring the same information accurately (Psytech, n.d.).
Validity considers the specific characteristic a test measures and how well the test
evaluates those characteristics (Liberty University, n.d). Validity gives meaning to the
assessment scores and is the most important consideration when selecting a test (Liberty
University, n.d). Construct and criterion validity are the two types discussed withing the VMQ
RELIABILITY AND VALIDITY 3
manual. Construct validity is employed for assessment as it evaluates if the characteristic being
measured are meaningful and dependable with the testing purpose (Liberty University, n.d).
Reliability: Cronbach Alpha Coefficients
The choice assessment to measure reliability in the VMQ, is the Cronbach Alpha
Coefficients. This scale considers inter-correlations, both high and low, to determine reliability.
If the items assessed have high inter-correlations both with one another and with total scores,
coefficient alpha will be high and demonstrate that items on the scale are measuring the same
information accurately (Psytech, n.d.).
Guidelines for understanding reliability coefficient values and interpreting the data
effectively have been established. A reliability coefficient value of .90 or higher is
excellent, .80-.89 is considered good, .70-.79 is adequate, and levels below .70 have limited
applicability (Liberty University, n.d). When considering reliability in the Values & Motives
Inventory (VMI) used for assessment in the VMQ, eight of the 11 scales utilized reflected
acceptable or higher internal consistency (Psytech, n.d.). The ranges below demonstrate that the
VMQs exceeded acceptable levels of reliability through internal consistency and showed low
error.
VMI Scale Alpha Interpretation
Affiliative 0.74 acceptable
Altruistic 0.74 acceptable
Affection 0.79 acceptable
Achievement 0.53 problematic
Finance 0.83 high
Security 0.79 acceptable
Aesthetics 0.83 high
Moral 0.68 low
Traditional 0.70 acceptable
Independence 0.66 low
Ethical 0.70 acceptable
S.D. 0.64 low
Infrequency 0.52 problematic
RELIABILITY AND VALIDITY 4
Sample Size and the Nature of the Population
Validity
When considering validity within the VMQ, it is necessary to consider population data.
The population utilized for the assessment included 159 psychology under-graduate and MBA
students. The total population utilized three varied assessments, with 59 under-graduate students
completing both the VMI and MAPP and 100 MBS students completing the VMI and the 16PF
version five (Psytech, n.d.). The data provided in the VMQ shows consistent validity among both
comparable reference groups. The similarities of the groups provide enough consistency to be
used with both groups while still maintaining some unique aspects to provide variability (Liberty
University, n.d). While a higher validity result is typical of groups with similar characteristics
(both students), enough variation was shown in one alternative test (Sheperis et al., 2020).
VMQ Norming Population
When considering norming of the population in the VMQ, a first critique is a small
sample size. Additional members of the sample groups would have helped to reiterate conclusive
findings in both reliability and validity. While normative consistency among certain
characteristics is important with reference groups, in this case college students, the VMQ did not
include enough data regarding other characteristics to present a comprehensive picture. For
example, although some gender information was provided with numbers of each there was not an
in-depth analysis within gender to conclude potential gender related variables.
Opinion of the VMQ
The VMQ did provide reasonable internal consistency reliability with eight of the 11
scales demonstrating high or acceptable interpretation (Psytech, n.d.). However, validity is the
RELIABILITY AND VALIDITY 5
most important factor when selecting and utilizing a test (Liberty University, n.d). Due to a small
sample size, absent measurable characteristics, and variables within interests of students
measured, definitive validity was not achieved.
References
Liberty University. (n.d.). COUC 521: Assessment techniques in counseling. Week three: What
makes a good test? https://canvas.liberty.edu/courses/321147/pages/read-what-makes-a-
good test?module_item_id=38576559.
Psytech. (n.d.). Values and motives questionnaire: The technical manual.
https://psytech.com/Content/TechnicalManuals/EN/VMIMan.pdf.
Sheperis, C., Drummond, R. J., & Jones, K. D. (2020).JAssessment procedures for counselors
and helping professionals. Pearson.J
Powered by TCPDF (www.tcpdf.org)
