2
RELIABILITY AND VALIDITY
Types of Reliability and Validity
Within this manual there is an acknowledgment of two types of reliability and two types
of validity, however one of each will remain the focus of the manual. Reliability is explained
through terms of stability or internal consistency, while validity is through construct or criterion.
However, the main focus is placed on internal consistency reliability and construct validity.
Internal consistence reliability seeks to ensure that the assessment items share an underlying
construct and share a correlation across different items (Chiang et al., 2015). When internal
consistency is present, various items of an assessment that seek to know the same result may be
asked or displayed in different ways however, they will produce the same correlation. If there is
inconsistency between the items, then there is a lack of internal consistence reliability. This
reliability can then be divided into two subcategories of internal consistency reliability and that is
split half and Coefficient Alpha. Construct validity is when an assessment measures the construct
that it is supposed to, proving to produce similar results as other assessments measuring the same
construct and different results from those that are measuring a different construct (Clark &
Watson, 2019).
Reliability: Cronbach Alpha Coefficients
The Values and Motives Questionnaire uses 11 scales of measurement. When assessing
the internal consistency reliability within these 11 scales through the use of Cronbach’s alpha,
the majority of the scales fall within an adequate level or above of reliability, which is .70. The
independence (.66), moral (.66), and achievement (.53) scales fall below the acceptable level of
reliability. However, the majority such as the affiliative (.74), altruistic (.74), affection (.79),
security (.79), traditional (.70), and ethical (.70) scales produced an adequate alpha coefficient.
Lastly, the financial (.83) and aesthetic (.83) produced good reliability. When viewing the
validity of the Values and Motives Questionnaire (2020) one can see a range in correlation from
3
RELIABILITY AND VALIDITY
0 to .57, with a median of .10. However, high correlations can be seen between the following
scales: affiliation and affection (.57), morality and tradition (.49), achievement and financial
(.40), altruism and affection (.39), as well as morality and ethics (.32). All of which are relative
to the guidelines provided by the article “What Makes a Good Test?”. While both reliability and
validity are important, one must remember that an assessment can be reliable but not valid
however, it can never be valid and not reliable (Sheperis et al., 2020).
Sample Size and Nature of the Population
Validity
Knowing the population that is being targeted is critical when establishing an assessment.
The population will influence the structure and layout of the assessment as well as the normative
results. The form, purpose, and proposed population of each assessment are all components that
makeup validity (Hajjar, 2018). The population that was used for the VMI consisted of 155
students seeking varying degrees but required to complete the assessment for a course. To
determine the validity of the VMI, the findings were compared to the MAPP and 16PF. The
MAPP consisted of 59 undergraduate students who volunteered, with return compensation in the
form of feedback of results. 16PF consisted of 100 MBA students that took the assessment as a
requirement for a course. Overall, it appears comparable with 155 students taking the VMI and
159 students taking one of the two alternative assessments. However, there is a concern. The
number of students who participated in the MAPP is significantly lower than the number of
students who participated in the VMI or the PF16. Furthermore, the VMI consisted of a diverse
group of educated students completing the assessment while the other two components were
completed by a narrower audience. That of which were either MBA students from a prestige
school or voluntary undergraduate students. The lack of diversity and inclusion found within
each additional assessment as it is within the VMI possess a concern.
4
RELIABILITY AND VALIDITY
VMQ Norming Population
This assessment has been targeted specifically at students, with slight variations in
educational background. The details were given around education were limited and only
classified as prestigious MBA students or psychology under-graduate students. While both
genders are included in the assessment, the distribution does not near a balanced ratio. The
information previously discussed is the extent of information provided, which is lacking.
Cultural considerations have not been addressed or considered. Due to the lack of information
provided and the sparsity of what was discussed, I do not believe this assessment could
adequately be applied to a larger or more general population.
Opinion of the VMQ
Overall, the VMI produced on average an adequate level of reliability between the 11
scales measured. It is important to note that adequate is classified as the minimum required to not
be dismissed for lack of reliability. Not only did the majority of scales sit at an adequate level but
only two received good reliability and none received excellent. The intercorrelations between
scales are described as modest, in terms of the VQIs validity. Although 5 scales produced fairly
strong correlations, the majority of the scales proved to be independent in nature. For this reason,
the assessment appears to produce validity however, I believe it lacks reliability and the ability to
be generalized. An assessment can provide consistent measures, but not measure what it is
supposed to (Values and Motives Questionnaire, 2020). In the case of this assessment, I have
found that it may loosely measure what it’s supposed to, but it is not consistent and therefore
should not be utilized as an assessment can be reliable and not valid but cannot be valid and
unreliable (Sheperis et al., 2020).
5
RELIABILITY AND VALIDITY
References
Chiang, I., Jhangiani, R., & Price, P. (2015). Research methods in psychology – 2nd Canadian
edition. BCcampus. Retrieved from https://opentextbc.ca/researchmethods/
Clark, L. A., & Watson, D. (2019). Constructing validity: New developments in creating
objective measuring instruments.Psychological Assessment,31(12), 1412-
1427.Khttps://doi.org/10.1037/pas0000626
Hajjar, S. (2018). Statistical analysis: Internal- consistency reliability and construct
validity.KInternational Journal of Quantitative and Qualitative Research Methods,K6(1),
46–57
Sheperis, C. J., Drummond, R. J., & Jones, K. D. (2020)KAssessment procedures for counselors
and helping professionalsK(9th ed.). Pearson.
Values and motives questionnaire, The technical manual. (2020). Retrieved from
https://psytech.com/Content/TechnicalManuals/EN/VMIMan.pdf
Powered by TCPDF (www.tcpdf.org)
