2
CASE STUDY: MEASUREMENT ASSIGNMENT
Case Study: The Phonemin Company
Improving Customer Service Representatives (CSR) performance is crucial for
companies like The Phonemin Company, where the telephone ordering system is a key
component of their business model. To enhance CSR performance, it's essential to provide
comprehensive training and support, ensuring that all representatives are equipped with the
necessary skills and knowledge to handle customer inquiries efficiently. Additionally,
implementing a robust feedback system can help identify areas for improvement and recognize
top-performing CSRs. By focusing on these areas, the company can aim to reduce error rates,
increase the speed of order taking, and minimize customer complaints, thereby improving the
overall customer experience and driving sales growth (Judge & Kammeyer-Mueller, 2021).
The staffing process for CSRs is designed to be efficient and ongoing, reflecting the high
turnover rate in the role. Recruitment leverages both employee referrals and traditional
newspaper advertising, ensuring a diverse applicant pool. The initial screening is minimal,
focusing on eliminating only those clearly unsuitable for the role, which allows for a high
percentage of applicants to proceed to the interview stage. The interview, though brief, is a
critical step where the staffing specialist assesses the candidate's fitness for the position. Given
the competitive labor market, most interviewees are offered positions, reflecting the
organization's need to maintain adequate staffing levels (Judge & Kammeyer-Mueller, 2021).
To enhance the effectiveness of CSR, the organization is exploring improved staffing
practices. Recognizing the limitations of current applicant assessment methods, it has partnered
with a consulting firm to develop a more comprehensive evaluation of candidate KSAOs. This
collaboration aims to refine the selection process, ensuring that new hires possess the essential
qualities for optimal performance in CSR roles. The joint study conducted by the consulting firm
3
CASE STUDY: MEASUREMENT ASSIGNMENT
and the HR staffing specialist is a strategic move towards achieving this goal (Judge &
Kammeyer-Mueller, 2021).
The job analysis for a Customer Service Representative (CSR) highlights the importance
of specific KSAOs for optimal job performance, with a focus on clerical speed, clerical accuracy,
and interpersonal skills underscores their critical impact on the role. The effectiveness of an
employee's performance is highly dependent on the job enrichment strategies implemented by
their organization, as noted by Alabri, et.al, (2022). The innovative assessment methods
introduced, including a timed clerical test and a practical work sample during the interview, aim
to evaluate these KSAOs effectively. The work sample simulates real-world scenarios, allowing
for a dynamic assessment of the applicant's ability to handle various customer interactions with
tactfulness and concern, which are essential qualities for a CSR.
The study's methodology appears robust, utilizing a random sample of CSRs and a two-
time-point data collection approach to assess the correlation between clerical tests, work
samples, and performance metrics. The inclusion of a second rater from the consulting firm adds
an extra layer of reliability to the performance evaluations. If the hypothesis holds true, the
clerical test and work sample could be predictive of a CSR's speed and accuracy, potentially
serving as valuable tools for employee selection and training processes (Judge & Kammeyer-
Mueller, 2021).
Question 1: How do you interpret the reliability results for the clerical test and work
sample? Are they favorable enough for Phonemin to consider using them “for keeps” in
selecting new job applicants?
4
CASE STUDY: MEASUREMENT ASSIGNMENT
In terms of reliability, Phenomin can confidently adopt the clerical examination and work
sample as part of its recruitment selection procedures. This approach is justified by the study that
disclosed high coefficient alpha scores of 0.85 and 0.86 for the test-retest reliability, also known
as T, and the work sample C, respectively. Moreover, the inter-rater reliability derived from the
work sample C indicated a consistency of 80% at T1 and 82% at T2, surpassing the 75%
benchmark for reliability (Judge & Kammeyer-Mueller, 2021). This evidence suggests that both
methods exceed the reliability threshold, thereby affirming their effectiveness in evaluating the
suitability of assessment techniques (Judge & Kammeyer-Mueller, 2021).
Test-retest and interrater agreement are essential for assessing the reliability across two
different time intervals. Given the objective nature of the clerical tests, employing test-retests to
determine reliability during these intervals is recommended. With the intervals being a week
apart, it's reasonable to anticipate test-retest results exceeding .9. Consequently, the clerical test
demonstrates its capacity to reach the 0.92 reliability threshold. It's significant to acknowledge
that the correlation coefficient in clerical tests is quite crucial, indicating a low likelihood of
falsely identifying a correlation where none exists. Regarding the work sample's reliability in the
first and second assessments, the brief interval supports the expectation of at least an 80%
probability of consistent measurement. Thus, it can be observed that the work sample outcomes
are consistently consistent when evaluated through interrater measures (Judge & Kammeyer-
Mueller, 2021).
Question 2: How do you interpret the validity results for the clerical test and work
sample? Are they favorable enough for Phonemin to consider using them “for keeps” in
selecting new job applicants?
5
CASE STUDY: MEASUREMENT ASSIGNMENT
Validity pertains to the precision of a measurement and its predictive accuracy, as
indicated by the scores derived from an assessment. Studies related to criterion and content
validation are performed to understand a measure's validity. Criterion-related validation involves
correlating scores from a predictor measure KSAO with those from a criterion measure HR
outcome (Judge & Kammeyer-Mueller, 2021). Content validation involves evaluating the
relevance of a predictor's content to the intended attribute and the job's content. Validation study
results were once considered specific to the situation; however, recent research indicates that
predictor validity may extend across different contexts, potentially reducing the need for
extensive, situation-specific validation studies (Judge & Kammeyer-Mueller, 2021).
The process of validating clerical assessments and practical task simulations starts with
an analysis of the job to pinpoint its fundamental duties, as noted by Judge & Kammeyer-
Mueller, (2021). This step guides the development of both the standards and predictive elements
in the evaluations. The study indicates a variance in how well practical task simulations and
clerical assessments correlate. However, this variation is eclipsed by the substantial and
consistent correlation observed among the practical task simulations themselves, which tend to
produce uniform results. For example, the frequency of errors in data exchange during
communication, as well as less significant aspects such as speed and the number of complaints,
appear to exert little influence. In a similar manner, practical task simulations provide consistent
results, highlighting their applicability to client-related issues. It is advised that the Phenomenon
integrate practical task simulations for their alignment and incorporate clerical assessments in the
selection process to identify the most fitting applicant. This method is intended to facilitate the
selection of the most suitable candidate from the talent pool, thus enhancing the collective
performance desired.
6
CASE STUDY: MEASUREMENT ASSIGNMENT
Question 3: What limitations in the above study should be kept in mind when interpreting
the results and deciding whether to use the clerical test and work sample?
In interpreting the results of the study, it is essential to acknowledge that the evaluations
were largely comprised of experienced and current employees. Moreover, the research
encompassed serving Customer Service Representatives, integrating their existing knowledge of
the role. Such familiarity with the organization implies that their results in tests are not directly
comparable with those of new applicants who do not have this advantage. For example, the
ability to manage customer grievances is impacted by the length of service at the company,
which results in different skills. Recognizing the importance of familiarity with the company's
processes over the course of time is crucial when assessing outcomes and devising customer
service strategies. It's also important for hiring managers to understand that self-assurance and
situational management are skills developed through practical work experience.
Conclusion
The Phonemin Company's commitment to elevating CSR performance through extensive
training, support, and a dynamic feedback system is a strategic approach to refining their
business operations. The innovative assessment methods, particularly the timed clerical test and
practical work sample, provide a promising opportunity for predicting candidate success in these
roles. The coefficient alpha scores and inter-rater reliability percentages presented in the study
provide an empirical basis for the incorporation of the clerical examination and work sample into
Phenomin's selection procedures. These methods exceed the established standards for reliability,
ensuring a high level of consistency and accuracy in assessing candidates' qualifications.
7
CASE STUDY: MEASUREMENT ASSIGNMENT
Personal Applications
I’m currently employed as a Substance Abuse Counselor. In my experience, hiring
Substance Abuse Counselors requires a careful approach that prioritizes both credentials and
personal qualities. My organization looks for candidates with a strong educational background in
psychology or social work, and a certification in substance abuse counseling. My organization
utilizes a recruitment company for prescreening candidates, to rule out individuals who are not
qualified for the position. Once the candidate passes the prescreening, they then meet with
management at the facility for an in-person interview. It's essential to assess the candidate’s
empathy, communication skills, and commitment to ethical standards. The candidate’s ability to
work collaboratively within a multidisciplinary team is also considered. Ultimately, the goal is to
hire professionals who can effectively support individuals on their journey to recovery.
8
CASE STUDY: MEASUREMENT ASSIGNMENT
References
Alabri, I., Siron, R. B., & Alam, M. N. (2022). Human resource management (HRM) practices and
employees' adaptive performance: The moderating role of transformational leadership.
Webology, 19(2), 6215-6245.
https://go.openathens.net/redirector/liberty.edu?url=https://www.proquest.com/scholarly-
journals/human-resource-management-hrm-practices-employees/docview/2695093540/se-2
Judge, T., & Kammeyer-Mueller, J. (2021). Staffing Organizations (10th ed.). McGraw-Hill Higher
Education (US). https://mbsdirect.vitalsource.com/books/9781264072590
Powered by TCPDF (www.tcpdf.org)
