Reflection, paper
Elibet2022
ASystematicReviewofResearchEvaluatingtheUseofPreferenceAssessmentMethodologyintheWorkplace.pdf
Full Terms & Conditions of access and use can be found at https://www.tandfonline.com/action/journalInformation?journalCode=worg20
Journal of Organizational Behavior Management
ISSN: (Print) (Online) Journal homepage: https://www.tandfonline.com/loi/worg20
A Systematic Review of Research Evaluating the Use of Preference Assessment Methodology in the Workplace
Michael J. Simonian , Denys Brand , Makenna A. Mason , Megan R. Heinicke & Shannon M. Luoma
To cite this article: Michael J. Simonian , Denys Brand , Makenna A. Mason , Megan R. Heinicke & Shannon M. Luoma (2020) A Systematic Review of Research Evaluating the Use of Preference Assessment Methodology in the Workplace, Journal of Organizational Behavior Management, 40:3-4, 284-302, DOI: 10.1080/01608061.2020.1819933
To link to this article: https://doi.org/10.1080/01608061.2020.1819933
Published online: 27 Sep 2020.
Submit your article to this journal
Article views: 446
View related articles
View Crossmark data
Citing articles: 1 View citing articles
A Systematic Review of Research Evaluating the Use of Preference Assessment Methodology in the Workplace Michael J. Simonian, Denys Brand, Makenna A. Mason, Megan R. Heinicke, and Shannon M. Luoma
California State University, Sacramento, Department of Psychology, Sacramento, California, USA
ABSTRACT Preference assessment methodology has largely been utilized to inform behavior-analytic treatment for clinical populations. However, the use of preference assessments has been extended to organizational settings when developing performance man- agement interventions and identifying preferred stimuli and activities that may serve as potential reinforcers. Thus, the pur- pose of this review was to synthesize the existing research evaluating the use of preference assessment methodology in workplace settings. Twelve articles consisting of 13 studies were included in this review. We coded and summarized a number of key study features, including participant characteristics, stimuli used in preference assessments, cost of stimuli, method and frequency of preference assessments, the use of reinforcer assessments, and social validity measures. We also provide sev- eral suggestions for future research.
KEYWORDS Employee preference; organizational behavior management; preference assessment; reinforcer identification; workplace
Behavior analysts regularly conduct preference assessments to identify stimuli that serve as potential reinforcers to facilitate behavior change for individuals with developmental disabilities (Graff & Karsten, 2012). The literature docu- menting the efficacy of preference assessments with clinical populations is extensive, and we refer the reader to published reviews of these methods (e.g., Hagopian, Long, & Rush, 2004; Heinicke, Carr, & Copsey, 2019; Tullis et al., 2011). More recently, researchers in the field of organizational behavior manage- ment have investigated the use of preference assessment procedures (typically used as part of clinical treatment) within organizational settings (e.g., Wilder, Therrien, & Wine, 2006; Wine, Reis, & Hantula, 2014b). The aim of employee preference assessments is to identify preferred stimuli and activities that can serve as potential reinforcers when developing performance management inter- ventions (Wilder et al., 2006). Making assumptions regarding employee prefer- ences may not lead to effective outcomes. That is, employees may not be motivated to increase their levels of productivity if offered non-preferred items or activities as incentives (Wilder et al., 2006). Thus, it is important to use empirically validated methods when conducting employee preference
CONTACT Denys Brand [email protected] Department of Psychology, California Street University, Sacramento, Sacramento, CA 95819-6007
JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT 2020, VOL. 40, NOS. 3–4, 284–302 https://doi.org/10.1080/01608061.2020.1819933
© 2020 Taylor & Francis
assessments to avoid the possibility of selecting incentives that hold little to no value for employees (Waldvogel & Dixon, 2008).
A variety of methods for assessing employee preference currently exist (Wine, Kelley, & Wilder, 2014a). These include direct observation methods, such as forced-choice (Fisher et al., 1992) and multiple stimulus without replacement (MSWO; DeLeon & Iwata, 1996). The forced-choice method involves presenting the employee with a pair of stimuli and requiring them to make a choice (e.g., a 10 USD gift card vs. finishing work 30 min early one day). Other example stimuli might include extra lunch time, movie tickets, or a preferred parking space. A preference hierarchy is formed by pairing all possible combinations of the assessed stimuli across multiple trials and rank- ing the stimuli that were chosen more frequently as more preferred than those chosen fewer times (e.g., Wilder et al., 2006). In the MSWO method, the employee is presented with an array of multiple stimuli and required to make a selection from the array. On the next trial, the employee makes another selection from the array with all previously selected stimuli no longer available. This process repeats until the employee has selected all stimuli, resulting in a preference hierarchy. Stimuli chosen earlier in the process are ranked as more preferred relative to those selected later (e.g., Waldvogel & Dixon, 2008).
Indirect methods of assessing employee preference include surveys (Daniels, 1989) and rank order methods (e.g., Wine, Gilroy, & Hantula, 2012). The survey method requires the employee to rate how much work they are prepared to complete to earn a particular item. For example, Wilder et al. (2006) required participants to rate the amount of work they would perform to access one of six items using a Likert-type scale. Wilder et al. (2006) identified highly preferred items as those rated with a score of 3 or 4 on the rating scale. In a rank order employee preference assessment, employees receive a list of stimuli and/or activities and rank them in order of most preferred to least preferred (e.g., Wine et al., 2012).
Continuing the development and refinement of preference assessment methods used within organizational settings represents an important area of research. Conducting employee preference assessments comes at a cost to organizations – that is, organizations must invest both time and money to conduct the assessments and purchase the items that serve as potential rein- forcers. Therefore, if preference assessments do not correctly identify reinfor- cers for improved performance, it appears unlikely that organizations would adopt such methods as part of their regular business practices (Waldvogel & Dixon, 2008). Moreover, research shows that employee preference does not remain stable over time (e.g., Wine et al., 2012). Thus, developing employee preference assessment methods that supervisors can easily and repeatedly administer is important. Finally, there are many other factors that contribute to optimal employee performance, such as performance feedback (Alvero, Bucklin, & Austin, 2001), shift scheduling (Strouse, Carroll-Hernandez,
JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT 285
Sherman, & Sheldon, 2004), and the type of tasks employees have to perform as part of their regular duties (Reed, DiGennaro Reed, Campisano, Lacourse, & Azulay, 2012). Employees may have individual preferences with respect to how these factors are applied within the workplace. For example, an employee may prefer written over verbal feedback, and this preference may greatly affect their level of performance on a particular task. Thus, preference assessments may also offer a method for identifying preference for some of these perfor- mance-related variables in addition to reinforcers for improved performance (Green, Reid, Passante, & Canipe, 2008; Reed et al., 2012).
The purpose of this review is to synthesize the existing research evaluating the use of preference assessment methodology (typically applied in clinical settings) in the workplace. Specifically, we aim to provide professionals in the field of organizational behavior management information regarding: (a) how preference assessment methodology is utilized within organizational settings; (b) which methods most accurately identify preferred stimuli that can serve as incentives for improved employee performance; (c) the associated costs of conducting assessments and purchasing items intended to serve as incentives; (d) social validity; and (e) areas for future research.
Methods
Literature search procedures
We conducted a search of the published literature through May 2020 using the PsycINFO and ERIC databases with the keyword combinations preference assessment, reinforcer identification, or reinforcer assessment AND employee, organizations, workplace, staff, performance, management, supervisor, employer, or incentive. We restricted our searches to articles published both in English and in peer-reviewed journals but did not restrict them by date. Moreover, we conducted manual table of contents searches for the following journals: Journal of Organizational Behavior Management, Journal of Applied Behavior Analysis, Behavior Analysis in Practice, Behavioral Interventions, Behavior Analysis: Research and Practice, and Behavior Modification. All searches took place in May 2020.
Following the initial searches, we excluded studies if a) they did not include adult employees as participants; b) they included participants diagnosed with a developmental disability; c) they were not conducted in applied workplace settings; and d) the main experimental question did not involve administering formal preference assessments with employees to identify preferred items/ activities or other performance-related variables (e.g., preference for certain types of tasks) that can serve as potential reinforcers. We also conducted ancestral searches of all included articles and applied the same exclusionary
286 M. J. SIMONIAN ET AL.
criteria. This systematic review is consistent with all applicable PRISMA standards (Moher, Liberati, Tetzlaff, Altman, & Group, 2009).
Data coding
If an article consisted of multiple experiments, each experiment was coded separately. We coded the following eight study variables for analysis: (a) topic of study (i.e., a short description of the main purpose of the investigation); (b) participant characteristics (i.e., number, employee type); (c) stimuli used in preference assessments (e.g., items, activities, performance-related variables); (d) cost of stimuli used during preference assessments; (e) preference assessment method (e.g., survey, MSWO); (f) frequency of and time between preference assessments; (g) the use of reinforcer assessments; and (h) reinforcer effectiveness.
Intercoder agreement
We calculated intercoder agreement across two independent raters (i.e., the first and third authors) for all study variables. Intercoder agreement was assessed using the point-by-point method for 100% of included studies. An agreement was defined as both coders recording the same study feature. Our mean intercoder agreement score was 100%.
Results
Twelve articles consisting of 13 experiments were included in this review (see Figure 1). These studies were published between 1995 and 2014. Of the 12 articles, 11 were published in the Journal of Organizational Behavior Management and one in the Journal of Applied Behavior Analysis. Table 1 provides a summary of the coded variables per article.
Topic of study
We identified six topics of study: (a) temporal stability of employee preference; (b) managers’ accuracy in predicting employee preference; (c) reliability between preference assessment methods; (d) manipulation of low-preferred work tasks into high-preferred ones; (e) preference for reinforcers other than money; and (f) mixed-preference reinforcer delivery. Two articles assessed the temporal stability of employee stimulus preference. Wine et al. (2012) con- ducted preference assessments using the same stimuli once per month for six months, and in a follow-up study (Wine et al., 2014a) assessments were conducted once per week for four weeks. Preference for stimuli was only stable across a one-week interval and all other time intervals had some degree of fluctuation, the extent of which varied among participants.
JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT 287
Sc he
m at
ic o
ve rv
ie w
o f t
he s
el ec
tio n
an d
in cl
us io
n/ ex
cl us
io n
of s
tu di
es
In iti
al S
ea rc
h Pr
oc ed
ur es
: Ps
yc IN
FO (n
= 1
26 s
tu di
es )
E R
IC =
(n =
3 3
st ud
ie s)
T
ot al
(n =
1 59
s tu
di es
)
Fu ll-
te xt
s tu
di es
re vi
ew ed
fo r e
lig ib
ili ty
(n
= 1
22 )
11 3
St ud
ie s
E xc
lu de
d 9
St ud
ie s
In cl
ud ed
D id
n ot
m ee
t p ar
tic ip
an t
in cl
us io
n cr
ite ri
a:
N ot
a du
lt em
pl oy
ee s
(n =
9 1)
Pa
rt ic
ip an
ts w
ith d
ev el
op m
en ta
l di
sa bi
lit ie
s (n
= 5
8 )
St ud
ie s
no t c
on du
ct ed
in
ap pl
ie d
w or
k se
tti ng
s (n
= 7
6)
4 St
ud ie
s in
cl ud
ed v
ia T
ab le
o f
C on
te nt
s Se
ar ch
es
0 St
ud ie
s In
cl ud
ed v
ia
A nc
es tr
al S
ea rc
he s
T he
p ri
m ar
y ex
pe ri
m en
ta l
qu es
tio n
di d
no t i
nv ol
ve
co nd
uc tin
g pr
ef er
en ce
as
se ss
m en
ts w
ith e
m pl
oy ee
s to
id
en tif
y pr
ef er
re d
ite m
s/ ac
tiv iti
es to
u se
a s
in ce
nt iv
es (n
= 1
02 )
13 S
tu di
es In
cl ud
ed in
P re
se nt
R
ev ie
w
Pr el
im in
ar y
Sc re
en in
g:
D up
lic at
es a
cr os
s da
ta ba
se s
(n =
2 2)
R
ev ie
w s/
m et
a- an
al ys
es e
tc . (
n =
15 )
Fi gu
re 1
. S ch
em at
ic o
ve rv
ie w
o f t
he s
el ec
tio n
an d
in cl
us io
n/ ex
cl us
io n
of s
tu di
es . S
om e
st ud
ie s
m et
m or
e th
an o
ne e
xc lu
si on
ar y
cr ite
rio n
288 M. J. SIMONIAN ET AL.
Ta bl
e 1.
C ha
ra ct
er is
tic s
of p
re fe
re nc
e as
se ss
m en
t st
ud ie
s.
St ud
y To
pi c
Pa rt
ic ip
an ts
St im
ul i
Co st
PA T
yp e
Fr eq
ue nc
y Re
in fo
rc er
As
se ss
m en
t
W in
e et
a l.
(2 01
2) Te
m po
ra l
st ab
ili ty
n =
1 0.
E m
pl oy
ed 6
+ m
on th
s. 9
di
pl om
a; 1
B A.
T ec
hn ic
al
sp ec
ia lis
ts ; a
dm in
is tr
at iv
e as
si st
an ts
G ift
c ar
ds ; c
off ee
m ug
; e ve
nt ti
ck et
; c om
pu te
r m
em or
y ca
rd $5
Ra nk
o rd
er 7
(o nc
e pe
r m
on th
) N
o
W in
e et
a l.
(2 01
4a )
Te m
po ra
l st
ab ili
ty n
= 1
0. 5
fe m
al e;
5 m
al e.
1 0
Ca uc
as ia
n. 4
d ip
lo m
a; 2
B A;
2
M A.
D ire
ct c
ar e
st aff
G ift
c ar
ds ; c
an dy
; s el
ec tio
n of
w or
k ta
sk s;
se
le ct
io n
of w
or k
lo ca
tio n;
c om
m un
ity
ou tin
g; b
re ak
; s ki
p m
ee tin
g
$0 -5
Su rv
ey ; r
an k
or de
r 5
pe r P
A ty
pe
(1 –4
w ee
k in
te rv
al s)
N o
W ild
er e
t al
. ( 20
07 )
Pr ed
ic tin
g em
pl oy
ee
pr ef
er en
ce
n =
3 2
(2 7
em pl
oy ee
s an
d 5
m an
ag er
s) . 1
8– 62
y ea
rs o
ld
Sa te
lli te
c us
to m
er s
er vi
ce ;
su pp
ly s
to re
c us
to m
er s
er vi
ce ;
ad m
in is
tr at
iv e
as si
st an
ts ; f
oo d
se rv
ic e
G ift
c ar
ds ; e
ve nt
t ic
ke ts
; c al
lig ra
ph y
su pp
lie s;
co
m pu
te r
su pp
lie s;
t im
e off
w or
k;
be ve
ra ge
s/ fo
od ; c
os m
et ic
s up
pl ie
s;
m an
ag er
r ec
og ni
tio n;
a tt
en di
ng in
du st
ry
ev en
t; tu
iti on
a w
ar d
fo r
ch ild
; l ot
te ry
tic
ke ts
$0 -1
0 Ra
nk o
rd er
O nc
e pe
r em
pl oy
ee ; o
nc e
pe r
m an
ag er
N o
W ild
er e
t al
. ( 20
11 )
Pr ed
ic tin
g em
pl oy
ee
pr ef
er en
ce
n =
1 15
(1 00
e m
pl oy
ee s,
1 5
m an
ag er
s) . 1
8– 65
y ea
rs o
ld .
Em pl
oy ed
3 +
m on
th s.
G ym
st
aff ; h
os pi
ta l s
ta ff;
r es
id en
tia l
ho m
e st
aff ; i
nt ra
m ur
al s
po rt
s st
aff ; s
po rt
s sa
le s
as so
ci at
es ,
el ec
tr on
ic s
al es
a ss
oc ia
te s,
cr
af t
sa le
s re
p. , f
oo d
se rv
ic e,
m
ov ie
s al
es r
ep .,
te ch
ni ci
an s,
de
pa rt
m en
t st
or e
re p.
G ift
c ar
ds ; e
ve nt
t ic
ke ts
; m ea
l p as
s;
m an
ag er
r ec
og ni
tio n;
lu nc
he on
; em
pl oy
ee o
f t he
m on
th ; g
am es
a t
w or
k; s
na ck
s/ de
ss er
ts ; b
re ak
; l ea
ve
w or
k ea
rly ; p
ay b
on us
; a ss
is ta
nc e
in
w or
k ta
sk ; d
ay o
ff; h
ot el
s ta
y;
re du
ce d
w or
k lo
ad ; e
m pl
oy ee
di
sc ou
nt ; l
et te
r of
re
co m
m en
da tio
n; p
re fe
rr ed
pa
rk in
g
N ot
s ta
te d
Ra nk
o rd
er O
nc e
pe r
em pl
oy ee
; o nc
e pe
r m
an ag
er
N o
Re id
a nd
P ar
so ns
(1 99
5) PA
r el
ia bi
lit y
n =
1 8.
N ew
ly e
m pl
oy ed
. D ire
ct -
se rv
ic e
st aff
in r
es id
en tia
l fa
ci lit
y fo
r in
di vi
du al
s w
ith
di sa
bi lit
ie s
Fa m
ili ar
o bs
er va
tio n
m et
ho d,
un
fa m
ili ar
o bs
er va
tio n
m et
ho d
N /A
Li ke
rt , f
or ce
d ch
oi ce
(3
op tio
ns )
O nc
e pe
r PA
t yp
e N
o
W ild
er e
t al
. ( 20
06 )
PA r
el ia
bi lit
y n
= 4
. 4 fe
m al
e. A
dm in
is tr
at iv
e as
si st
an ts
G ift
c ar
ds ; l
ea ve
w or
k ea
rly ; a
ss is
ta nc
e in
w or
k ta
sk ; s
na ck
s; c
as ua
l cl
ot he
s da
y
$0 -5
Su rv
ey (v
oc al
); fo
rc ed
ch
oi ce
(v oc
al )
O nc
e pe
r PA
t yp
e Ye
s
(C on
tin ue
d)
JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT 289
Ta bl
e 1.
(C on
tin ue
d) .
St ud
y To
pi c
Pa rt
ic ip
an ts
St im
ul i
Co st
PA T
yp e
Fr eq
ue nc
y Re
in fo
rc er
As
se ss
m en
t
W in
e et
a l.
(2 01
4b )
PA r
el ia
bi lit
y Ex
p 1:
n =
3 . 2
fe m
al e;
1 m
al e.
2 3–
30 y
ea rs
ol
d. E
m pl
oy ed
6 +
m on
th s.
2 d
ip lo
m a;
1
BA . C
ar e
st aff
a t
re si
de nt
ia l h
om e
Ex p
2: n
= 2
. 2 fe
m al
e. 2
9– 34
y ea
rs o
ld .
Em pl
oy ed
6 +
m on
th s.
C ar
e st
aff a
t re
si de
nt ia
l h om
e
G ift
c ar
ds ; l
ot te
ry t
ic ke
ts ;
ca nd
y; o
ffi ce
s up
pl ie
s $5
Su rv
ey ; r
an k
or de
r; M
SW O
(E
XP 1
o nl
y) O
nc e
pe r
PA t
yp e
Ye s
W al
dv og
el a
nd D
ix on
(2 00
8) PA
r el
ia bi
lit y
n =
4 . 4
fe m
al e.
2 2–
30 y
ea rs
ol
d. E
m pl
oy ed
6 <
m on
th s.
2
di pl
om a;
1 c
ol le
ge e
nr ol
le d;
1
M A.
P ay
ed $
8. 75
p /h
. D
ire ct
c ar
e st
aff in
b ra
in
in ju
ry r
eh ab
ili ta
tio n
pr og
ra m
G ift
c ar
ds ; b
re ak
; c an
dy ;
t- sh
irt ; p
re fe
rr ed
p ar
ki ng
; pi
ct ur
e on
w al
l
$0 -5
Ra nk
o rd
er ; M
SW O
O nc
e pe
r PA
t yp
e N
o
G re
en e
t al
. ( 20
08 )
M an
ip ul
at in
g ta
sk
pr ef
er en
ce n
= 4
. 3 –9
y ea
rs e
xp er
ie nc
e. 4
di
pl om
a. R
es id
en tia
l h om
e su
pe rv
is or
s
W or
k ta
sk s
N /A
Li ke
rt ; r
an k
or de
r 7–
9 (a
cr os
s 3–
6 w
ee ks
) N
o
G re
en e
t al
. ( 20
08 )
M an
ip ul
at in
g ta
sk p
re fe
re nc
e n
= 4
. 3 –9
y ea
rs e
xp er
ie nc
ea . 4
di pl
om a.
R es
id en
tia l h
om e
su pe
rv is
or s
W or
k ta
sk s
N /A
Li ke
rt ; r
an k
or de
r 7–
9 (a
cr os
s 3–
6 w
ee ks
) N
o
Re ed
e t
al . (
20 12
) M
an ip
ul at
in g
ta sk
p re
fe re
nc e
n =
8 . 7
fe m
al e;
1 m
al e.
M a
ge =
24
.7 1
ye ar
s ol
d. M
e m
pl oy
ed =
17
.5 7
m on
th s.
7 B
A. D
ire ct
s up
po rt
st
aff in
h um
an s
er vi
ce o
rg an
iz at
io n
W or
k ta
sk s
N /A
Fo rc
ed c
ho ic
e (4
t as
ks )
9– 25
(o nc
e pe
r da
y) N
o
W in
e et
a l.
(2 01
3) Re
in fo
rc er
s ot
he r
th an
m on
ey n
= 2
4. 1
2 fe
m al
eb . 2
0– 37
y ea
rs o
ld .
“V ar
io us
” et
hn ic
iti es
. Em
pl oy
ed 6
+ m
on th
s. G
ro up
h om
e ca
re s
ta ff
$1 0;
g ift
c ar
ds ; l
ot te
ry
tic ke
ts ; c
an dy
; l ea
vi ng
ea
rly ; b
re ak
; s ki
p m
ee tin
g; s
el ec
t w
or k
ta sk
s; s
el ec
t w
or k
lo ca
tio n;
o ut
in gs
; pr
ef er
re d
pa rk
in g
$0 -1
0 Su
rv ey
; r an
k or
de r
O nc
e pe
r PA
t yp
e N
o
W in
e an
d Ax
el ro
d (2
01 4)
M ix
ed -p
re fe
re nc
e re
in fo
rc er
de
liv er
y
n =
5 . “
Re pr
es en
te d
bo th
g en
de rs
.” 27
–4 3
ye ar
s ol
d.
Em pl
oy ed
6 m
on th
s –
7 ye
ar s
D ire
ct
ca re
s ta
ff in
a du
lt di
sa bi
lit y
re si
de nt
ia l
fa ci
lit y
G ift
c ar
ds , c
an dy
, o ffi
ce s
up pl
ie s,
ca
rb on
at ed
b ev
er ag
e, lo
tt er
y tic
ke ts
$5 Su
rv ey
O nc
e Ye
s
PA =
p re
fe re
nc e
as se
ss m
en t;
M SW
O =
m ul
tip le
s tim
ul us
w ith
ou t
re pl
ac em
en t;
EX P
= e
xp er
im en
t. BA
= b
ac he
lo r’s
d eg
re e;
M A
= m
as te
r’s d
eg re
e a St
ud y
re po
rt ed
y ea
rs o
f m an
ag er
ia l e
xp er
ie nc
e ra
th er
t ha
n le
ng th
o f e
m pl
oy m
en t
at t
he c
om pa
ny .
b St
ud y
re po
rt ed
h ow
m an
y pa
rt ic
ip an
ts w
er e
fe m
al e
bu t
di d
no t
re po
rt o
n ot
he r
ge nd
er s.
290 M. J. SIMONIAN ET AL.
Two studies assessed the ability of managers to predict employee prefer- ence. The results showed that managers were not very accurate when predict- ing employee preference. Wilder, Rost, and McMahon (2007) found that managers were able to successfully predict the most preferred stimulus from a list of six items for 15 out of 27 employees but were not able to correctly rank the remaining items. Comparisons of manager and employee preference ranking across items were quantified using Kendall Rank-Order correlations. The mean correlation coefficient across all participants was 0.11 (range = −1.0 and 1.0), indicating a weak positive association (Cohen, 1988). Wilder, Harris, Casella, Wine, and Postma (2011) conducted a follow-up study with a larger, more diverse pool of employees and found that managers correctly identified the most preferred stimulus for 36 out of 100 employees with a mean Kendall rank-order correlation coefficient of 0.25 (range = −0.6 to 1.0) indicating a weak positive association.
Four articles (Reid & Parsons, 1995; Waldvogel & Dixon, 2008; Wilder et al., 2006; Wine et al., 2014b), consisting of five studies, compared the results of different preference assessment methods to assess reliability when identifying preferred and non-preferred stimuli. Four studies compared indirect to direct preference assessment methods and have produced mixed results. Of those four studies, two (Wilder et al., 2006; Wine et al., 2014b) used reinforcer assessments to compare the results across different preference assessment methods. Wilder et al. (2006) found the survey method to be more accurate at predicting reinforcers compared to a verbal-forced choice method, and Wine et al. (2014b; Experiment 1) found all items identified as high preference functioned as reinforcers when comparing surveys, rank order, and MSWO methods. The other two studies that compared the results from indirect and direct methods (Reid & Parsons, 1995; Waldvogel & Dixon, 2008) did not use reinforcer assessments. Waldvogel and Dixon (2008) measured correlations between preference assessment hierarchies identified by MSWO and reinfor- cer surveys using Spearman’s rank order correlation coefficients. The coeffi- cients ranged between .6 and 1.0, indicating strong positive correlations between methods (these coefficients are all above the critical r value found by Hanley, Iwata, & Roscoe, 2006). Reid and Parsons (1995) found that choice measures (i.e., employees were asked to state their preference for a specific type of observation procedure used by supervisors to monitor staff perfor- mance during a training program) provided a more sensitive measure of employee preference compared with questionnaires.
One study (Wine et al., 2014b: Experiment 2) compared the results across two indirect methods (survey vs. rank order) and found that although both methods identified reinforcers (using reinforcer assessments) the rank order method failed to identify high preference items that were identified by the survey. However, the authors noted that the aforementioned finding may simply be an artifact of the scoring system (i.e., participants can rate all
JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT 291
items on a survey as preferred, whereas the rank order method produces a preference hierarchy).
Two studies (Green et al., 2008; Reed et al., 2012) involved changing low- preferred work tasks into more-preferred ones by altering aversive aspects of these tasks. Both studies used the Task Enjoyment Motivation Protocol (TEMP), which is a collaborative method for employees and managers to identify undesirable aspects of the task and how to remove them or make them more tolerable. In each study, the tasks employees identified as low preferred were successfully changed to be more preferable by removing aver- sive components when possible and/or adding extra incentives for completing the task. For example, Green et al. (2008) identified frequent interruptions from staff as an aversive component when supervisors were required to review timesheets. Based on the results from the TEMP, staff interruptions were reduced by having supervisors work in a private office during pre-scheduled times (i.e., staff did not have access to supervisors when they were completing the task). Thus, the aversive component of the task was removed.
The Wine and Axelrod (2014) study involved an investigation of mixed- preference reinforcer delivery. Preference assessments were conducted via surveys and included snacks, gift cards, office supplies, and lottery tickets. Initially, the probability of earning a high preference item following successful task completion was 100%. Across successive opportunities to earn reinfor- cers, the probability of earning high-preference items decreased, while at the same time the likelihood of earning low-preference items increased. The results showed that when the probability of earning high-preference reinfor- cers decreased to less than 100%, participants did not reliably complete the task. The study did not provide support for the use of reinforcers varying in quality in applied settings.
Participants
The number of participants included in the studies ranged from two (Wine et al., 2014b) to 115 (Wilder et al., 2011). Twenty-four supervisors and 214 employees completed the preference assessments. Seven studies reported participant gender; 36 participants were identified as females and 7 as males. One study (Wine, Gugliemella, & Axelrod, 2013) reported that 12 of their 24 participants were identified as female but did not explicitly state the gender of the remaining 12 participants.
The only participant characteristic reported consistently across all studies was job position. Eight studies exclusively recruited participants from human service providers (e.g., staff employed at group homes, residential settings, and a brain injury rehabilitation facility). One study’s (Wilder et al., 2006) parti- cipants were university administrative assistants and two studies (Wilder et al., 2007; Wine et al., 2012) recruited university administrative assistants in
292 M. J. SIMONIAN ET AL.
conjunction with other types of employees. Two studies (Wilder et al., 2011, 2007) involved a varied pool of employees. Participants in the Wilder et al. (2007) study included satellite company customer service workers, supply store customer service workers, food service workers, and university admin- istrative assistants for a total of 32 participants. In Wilder et al. (2011), the range of employees included gym staff, hospital staff, residential home staff, intramural sports staff, electronics sales associates, craft store sales represen- tatives, movie sales representatives, department store representatives, food service workers, and technicians for a total of 115 participants. Waldvogel and Dixon (2008) was the only study to report employee wages ($8.75 - per hour). Eight studies included participant age, ranging between 18 and 65 years old. Seven studies reported education level, which ranged from high school diplomas to master’s degrees.
Eight studies had inclusionary criteria involving length of employment. Reid and Parsons (1995) recruited newly hired employees, the Waldvogel and Dixon (2008) study involved participants employed for less than six months, and the Wilder et al. (2011) investigation consisted of employees that the manager had known for at least three months. The remaining five studies (Wine et al., 2012, 2013, 2014a, 2014b) involved participants employed for at least six months. Of these five studies, Wine and Axelrod (2014) was the only one to report a range (six months to seven years employed) in addition to their inclusionary criteria. Reed et al. (2012) did not have a specific inclu- sionary criterion but the average length of participant employment was reported to be 17.57 months. Green et al. (2008) reported the number of years of participants’ field experience (3–9 years) rather than length of employ- ment at their current company.
Stimuli and cost
The cost of stimuli used in assessments ranged from 0 USD-$10. Ten studies used some form of gift card or coupon as one of their stimuli, making this the most commonly used stimulus class. Other common stimuli included snacks/ food, breaks/leaving work early, selection of work tasks, tangible items (e.g., office supplies, computer supplies), and praise/recognition. One study focused on identifying potential reinforcers other than money or gift cards (e.g., preferred parking, leaving early, and choosing work tasks) and found that employees often rank such stimuli as preferred over a choice of 10 USD (Wine et al., 2013). Other than Wine et al. (2013), authors did not specify whether stimulus classes were assessed separately or mixed, meaning we are unable to draw conclusions regarding the potential for stimulus displacement (e.g., Bojak & Carr, 1999). Lastly, we did not identify any studies that compared the costs and effort associated with conducting preference assessments relative to gains in productivity.
JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT 293
Preference assessment type
Indirect assessment methods – that is, rank order (n = 7), surveys (n = 5), and Likert scales (n = 2) – were the most commonly used assessment methods. Three studies (Reed et al., 2012; Reid & Parsons, 1995; Wilder et al., 2006) involved forced-choice methods, and two used an MSWO (Waldvogel & Dixon, 2008; Wine et al., 2014b). It is worth noting that no studies evaluating direct observation methods reported that employees directly interacted with the actual stimuli assessed. Rather, participants were presented with verbal or written descriptions of the stimuli when preference assessments were conducted.
Frequency
Four studies assessed preference more than once. Two studies (Wine et al., 2012, 2014a) involved assessing temporal stability and two (Green et al., 2008; Reed et al., 2012) investigated manipulating task preference. In the temporal stability studies (Wine et al., 2012, 2014a), the frequency of assessments was a manipulated variable with intervals ranging between one and four weeks. In the task preference manipulation studies, assessments were more frequent. Reed et al. (2012) assessed preference once per day for a range of 9 to 25 days. Green et al. (2008) assessed preference 7 to 9 times across a range of 3 to 6 weeks; when reassessed eight weeks following the completion of the inter- vention, preference for the modified task remained unchanged for one super- visor and increased slightly for another.
Reinforcer assessment & effectiveness
Three studies consisting of four experiments included reinforcer assessments to evaluate the predictive validity of preference assessment results (Wilder et al., 2006; Wine & Axelrod, 2014; Wine et al., 2014b). Eight studies did not include any employee performance data (Reed et al., 2012; Reid & Parsons, 1995; Wilder et al., 2011, 2007; Wine et al., 2012, 2013, 2014a). The focus of one study was manipulating task preference rather than directly assessing employee performance (Green et al., 2008). Generally, the results showed that items identified as highly preferred function as reinforcers.
Wilder et al. (2006) conducted reinforcer assessments across all stimuli used during the initial preference assessments regardless of whether they were identified as more or less preferred using a multielement design. Participants completed a paper-filing task and the outcome of interest was whether parti- cipants increased task productivity during conditions in which one specific item was made available if responding increased above initial baseline levels (i.e., a single-operant arrangement). Sessions were not timed, and participants
294 M. J. SIMONIAN ET AL.
could continue with the task until they either exceeded baseline levels of responding or chose to stop working. The reinforcer assessment found that the survey method identified high-preference items with greater accuracy compared to the forced-choice method. Performance decreased to below baseline levels for some participants when completing the task for low- preference items.
Wine et al. (2014b) conducted two reinforcer assessments using a multielement design. The first involved only stimuli identified as highly preferred, and the second consisted of the addition of a low-preference item meant to serve as a control. The task required participants to complete a number of data sheets that matched or exceeded baseline responding during 10-min sessions. Meeting or exceeding baseline levels of responding resulted in access to one stimulus at a time. They found that all highly preferred stimuli functioned as reinforcers during both reinforcer assessments. As was the case for Wilder et al. (2006), performance decreased to below baseline levels for a majority of participants when completing the task resulted in the least- preferred item as identified via preference assessment.
Wine and Axelrod (2014) found that providing highly preferred items with 100% certainty resulted in performance increases across all participants. However, once the probability of receiving less-preferred stimuli progressively increased, performance quickly decreased across a majority of participants. Thus, items tended to function as reinforcers if they were both highly pre- ferred and provided with 100% certainty.
Social validity
Three studies (Green et al., 2008; Reed et al., 2012; Wine et al., 2014b) reported social validity data. Wine et al. (2014b) used a 4-point Likert-like scale (administered with three direct care staff) to measure the social validity of three preference assessment methods: rank order, surveys, and MSWO. The MSWO and survey methods were rated as least and most preferred, respectively. Participants also rated the MSWO as more complex relative to the survey and rank order methods and reported that they were less likely to use it.
Green et al. (2008) required participants (i.e., four supervisors) to complete a survey utilizing a 7-point Likert scale to rate the social acceptability of the intervention they received to improve their work tasks. All participants gave the highest possible rating and indicated they would like the intervention to continue. Similarly, Reed et al. (2012) administered a modified version of the Intervention Rating Profile-15 (Martens, Witt, Elliott, & Darveaux, 1985), which is a 15-item inventory using a 6-point Likert scale to measure the acceptability of an intervention. Eight direct support staff members completed the social validity measure and rated the TEMP intervention as acceptable.
JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT 295
Discussion
The purpose of this review was to provide a synthesis of the existing research evaluating the use of preference assessment methodology in work- place settings and to summarize key study features. Our review included 13 studies, with most studies published in the Journal of Organizational Behavior Management. We found no articles published on this topic between 2015 and 2019. Given the potential benefits of conducting prefer- ence assessments in organizational settings, we want to encourage research- ers in the field of organizational behavior management to continue this line of research. Moreover, the studies included in this review were conducted by a small number of researchers. Thus, the literature is limited in that respect. The employee preference assessment literature could benefit from direct and/or systematic replications across a more diverse group of researchers.
A majority of the employee preference assessment research cited in this review focused on identifying preferred stimuli and activities that can serve as potential reinforcers. Although such assessments are important (Waldvogel & Dixon, 2008), the current state of the literature is limited given the narrow focus of the research. Assessing preference regarding other features of the workplace that may also contribute to improved employee performance (e.g., preference for different types of feedback, immediacy of feedback, types of tasks employees are assigned) represents an important extension of the research. The results from such assessments have the potential to provide supervisors with important information they can use to address potential barriers impeding optimal employee performance by making certain aspects of the job less aversive (e.g., Green et al., 2008; Reed et al., 2012).
It is also worth noting that the studies included as part of this review conducted preference assessments with the aim of improving performance for existing employees at an organization. Future research could investigate the use of preference assessments as an antecedent strategy when hiring employees to assess whether their preferences align with the established practices of the organization. Such a strategy may prove to be a more time and cost-effective approach to ensuring high levels of employee performance compared to how preference assessments are currently utilized.
The rank order and survey methods were the most commonly used meth- ods for assessing employee preference. These assessment methods appear to be more practical for use within organizational settings compared with direct observation methods. That is, the self-report nature of the rank order and survey methods may be more time and resource-efficient relative to methods that require participants to access stimuli and make repeated selections across multiple trials (e.g., MSWO, forced choice). Moreover, these methods allow for the inclusion of a greater number of stimuli during a single assessment,
296 M. J. SIMONIAN ET AL.
especially compared to forced-choice methods where adding more stimuli increases the number of combinations that need to be assessed.
The aforementioned benefits of the rank order and survey methods are especially important given that employee preference changes over time (e.g., Wine & Axelrod, 2014; Wine et al., 2012). To maintain improved employee performance, assessments need to be conducted at regular intervals to avoid possible satiation of the items and activities used as incentives. Satiation may result in employees losing the motivation to obtain the arranged incentives, which may reduce the effectiveness of the incentive program (Waldvogel & Dixon, 2008). It may also be worth investigating whether the ability of managers to accurately predict employee preference increases when assess- ments are conducted more frequently (e.g., Wilder et al., 2011, 2007).
We also found that it was common practice for researchers to conduct direct observation preference assessments by presenting employees with ver- bal or written descriptions of items to be used as potential reinforcers rather than presenting the actual items in a tangible format. Heinicke et al. (2019) reported that these alternative modalities have low predictive validity for individuals with developmental disabilities; they seem to be more accurate for individuals with strong matching skills and discrimination abilities. However, alternative modalities such as those used in this review appear to accurately identify reinforcers for individuals who do not demonstrate deficits in intellectual functioning. For example, Wilder, Wilson, Ellsworth, and Heering (2003) conducted preference assessments with four adults diagnosed with schizophrenia and found that verbal statements and the presentation of tangible items produced very similar preference hierarchies. This finding may aid the adoption of alternative modality preference assessments in the work- place, for example they are convenient for displaying a large variety of stimuli (e.g., manager recognition, preferred parking, computer supplies, tuition rewards) and decrease administration time. Overall, more research involving validation benchmarks are needed before recommendations for practice can be made regarding the use of indirect versus alternative-modality methods for conducting preference assessments in the workplace.
We found inconsistencies within the literature regarding the reporting of participant demographic information. The only variable that was consistently reported across all studies was the occupational role of study participants. Other potentially important employee characteristics, such as wages, experi- ence, and length of employment were less frequently reported. Consistently reporting such information (to the extent possible) may be directly relevant to this line of research. For example, a 10 USD gift card may be more preferred for an employee earning minimum wage compared to a preferred parking spot, whereas a preferred parking spot may be more preferred by an employee in a managerial role who earns a much higher wage. Research investigating the relation between demographic variables of interest and employee preference is
JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT 297
needed (e.g., Wilder et al., 2007). Articles by Brodhead, Durán, and Bloom (2014), Jones, St. Peter, and Ruckle (2020), and Li, Wallace, Ehrhardt, and Poling (2017) have discussed the importance of reporting demographic data more frequently as part of our dissemination efforts. We want to encourage researchers to carefully consider the demographic variables that need to be collected and reported when conducting preference assessment studies within organizational settings (Li et al., 2017).
At present, the employee preference assessment literature lacks social valid- ity data. Moreover, social validity data have been measured across two distinct areas: the methods used to assess preference (Wine et al., 2014b) and the interventions used to modify preference for certain tasks (Green et al., 2008; Reed et al., 2012). However, given the scarcity of social validity data in the literature, all conclusions regarding social validity should be considered pre- liminary. Future research should focus on collecting social validity data across both employees and supervisors. Collecting such data can help establish whether organizations value the use of preference assessment methods and are likely to retain them as part of their regular business practices. Furthermore, the literature may also benefit from investigations into how social validity can be improved when using preference assessment procedures in the workplace. For example, one area that can be examined is how different dimensions of diversity (e.g., culture, values) can be considered and incorpo- rated when using preference assessments to develop effective employee per- formance improvement plans. Incorporating such factors as part of the intervention may result in greater buy-in from key stakeholders (e.g., man- agers, supervisors) and represents a potentially important line of research.
The results from this review suggest several areas for future research. Although research has begun to explore the use of preference assessments in the workplace, cost-benefit analyses have yet to be conducted. Future research should aim to replicate the findings of previous studies and assess the various costs associated with conducting preference assessments, such as the number of staff required to implement the assessment and the time needed to conduct them. Small or modest improvements in employee performance may not result in enough additional revenue for companies to justify the allocation of resources associated with conducting assessments and purchasing reinforcers. We also suggest that future research investigate both the short and long-term benefits of conducting these assessments and providing additional incentives. In the short-term, providing additional reinforcers may be costly (Wilder et al., 2007), but could potentially lead to better long-term job satisfaction and decreased levels of employee turnover. Such a finding may be important, especially when considering the resources associated with hiring and training new staff (e.g., Strouse et al., 2004). Thus, in order to save time and money, it would be beneficial for organizations to implement programs that have been accurately assessed for employee preferences and reinforcer effectiveness prior
298 M. J. SIMONIAN ET AL.
to implementation, rather than through trial-and-error (Waldvogel & Dixon, 2008). It is important to remember that the main goal of an organization is to produce financial gains, thus research should aim to relate findings to cost- efficiency in order to promote organizational buy-in of empirically validated performance improvement methods.
Future studies can also focus on extending the research to employees outside of the human services sector, which only reflects one segment of the total workforce. The generality of the findings could potentially increase if the same results can be obtained across a wider variety of organizations and employees. Additionally, we suggest the inclusion of reinforcer assess- ments in preference assessment research. Conducting reinforcer assess- ments may be impractical in many cases given the nature of a job and the logistics of the workplace, hence its lack of application in the literature. However, if putative reinforcers identified via preference assessments do not produce increases in employee performance, organizations may be wasting resources when purchasing items identified as highly preferred (Wilder et al., 2006). While we found that the handful of existing studies that included reinforcer assessments generally reported high predictive validity (Wilder et al., 2006; Wine & Axelrod, 2014; Wine et al., 2014b), more research is needed to assess how reinforcer assessments can be conducted in the workplace once preferred items and activities have been identified via formal assessment.
It should be noted that some additional studies evaluating or comparing different workplace interventions (e.g., Bowman et al., 2019; Buckley et al., 2020) have included preference assessments as ancillary procedures. For example, Bowman et al. (2019) conducted a study to increase employee hand washing. The intervention involved using a lottery system in which each time an employee was observed washing their hands, they earned an entry into the lottery. To determine which reinforcer to use in the lottery system, a rank order preference assessment was conducted. We did not include such evaluations in our review. Instead, we synthesized studies in which the primary empirical question involved the efficacy of preference assessment methods, because they contain sufficient methodological details (e.g., a full list of all putative reinforcers assessed, cost of reinforcers) for comparison across evaluations. In addition, authors of studies in which pre- ference assessments are included as ancillary analyses do not always list preference-related search terms, making it difficult to capture them system- atically via relational database search engines. Future reviews of the literature involving employee preference assessments might consider expanding the scope to include studies such as those mentioned above (e.g., Bowman et al., 2019) as well as studies involving computerized human operant procedures (e.g., Wine & Wilder, 2009).
JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT 299
Disclosure statement
The authors declare that they have no conflict of interest.
Ethical approval
This article does not contain any studies with human participants performed by any of the authors.
Informed consent
No informed consent was required to conduct this research.
References
References marked with an asterisk indicate studies included in this review. Alvero, A. M., Bucklin, B. R., & Austin, J. (2001). An objective review of the effectiveness
and essential characteristics of performance feedback in organizational settings (1985-1998). Journal of Organizational Behavior Management, 21(1), 3–29. doi:10.1300/ J075v21n01_02
Bojak, S. L., & Carr, J. E. (1999). On the displacement of leisure items by food during multiple-stimulus preference assessments. Journal of Applied Behavior Analysis, 32(4), 515–518. doi:10.1901/jaba.1999.32-515
Bowman, L. G., Hardesty, S. L., Sigurdsson, S. O., McIvor, M., Orchowitz, P. M., Wagner, L. L., & Hagopian, L. P. (2019). Utilizing group-based contingencies to increase hand washing in a large human service setting. Behavior Analysis in Practice, 12(3), 600–611. doi:10.1007/ s40617-018-00328-z
Brodhead, M. T., Durán, L., & Bloom, S. E. (2014). Cultural and linguistic diversity in recent verbal behavior research on individuals with disabilities: A review and implications for research and practice. The Analysis of Verbal Behavior, 30(1), 75–86. doi:10.1007/s40616- 014-0009-8
Buckley, J., Fountain, J., Meuse, S., Whelan, C., Maguire, H., Harper, J. M., & Luiselli, J. K. (2020). Performance improvement of care providers in a child services setting: Effects of an incentive-based negative reinforcement intervention on data recording. Child & Family Behavior Therapy, 42(2), 125–133. doi:10.1080/07317107.2020.1738733
Cohen, J. (1988). Statistical power analysis for the behavioral sciences (2nd ed.). New York: Academic Press.
Daniels, A. C. (1989). Performance management: Improving quality productivity through positive reinforcement (3rd ed.). Tucker: GA, Performance Management Publications.
DeLeon, I. G., & Iwata, B. A. (1996). Evaluation of a multiple-stimulus presentation format for assessing reinforcer preferences. Journal of Applied Behavior Analysis, 29(4), 519–533. doi:10.1901/jaba.1996.29-519
Fisher, W., Piazza, C. C., Bowman, L. G., Hagopian, L. P., Owens, J. C., & Slevin, I. (1992). A comparison of two approaches for identifying reinforcers for persons with severe and profound disabilities. Journal of Applied Behavior Analysis, 25(2), 491–498. doi:10.1901/ jaba.1992.25-491
300 M. J. SIMONIAN ET AL.
Graff, R. B., & Karsten, A. M. (2012). Assessing preferences of individuals with developmental disabilities: A survey of current practices. Behavior Analysis in Practice, 5(2), 37–48. doi:10.1007/BF03391822
Green, C. W., Reid, D. H., Passante, S., & Canipe, V. (2008). *Changing less-preferred duties to more-preferred: A potential strategy for improving supervisor work enjoyment. Journal of Organizational Behavior Management, 28(2), 90–109. doi:10.1080/ 01608060802100899
Hagopian, L. P., Long, E. S., & Rush, K. S. (2004). Preference assessment procedures for individuals with developmental disabilities. Behavior Modification, 28(5), 668–677. doi:10.1177/0145445503459836
Hanley, P. H., Iwata, B. A., & Roscoe, E. M. (2006). Some determinants of changes in preference over time. Journal of Applied Behavior Analysis, 39(2), 189–202. doi:10.1901/ jaba.2006.163-04
Heinicke, M. R., Carr, J. E., & Copsey, C. J. (2019). Assessing preferences of individuals with developmental disabilities using alternative stimulus modalities: A systematic review. Journal of Applied Behavior Analysis, 52(3), 847–869. doi:10.1002/jaba.565
Jones, S. H., St. Peter, C. C., & Ruckle, M. M. (2020). Reporting of demographic variables in the Journal of Applied Behavior Analysis. Journal of Applied Behavior Analysis, 53(3), 1304–1315. doi:10.1002/jaba.722
Li, A., Wallace, L., Ehrhardt, K. E., & Poling, A. (2017). Reporting participant characteristics in intervention articles published in five behavior-analytic journals, 2013–2015. Behavior Analysis: Research and Practice, 17(1), 84–91. doi:10.1037/bar0000071
Martens, B. K., Witt, J. C., Elliott, S. N., & Darveaux, D. X. (1985). Teacher judgments concerning the acceptability of school-based interventions. Professional Psychology: Research and Practice, 16(2), 191–198. doi:10.1037/0735-7028.16.2.191
Moher, D., Liberati, A., Tetzlaff, J., Altman, D. G., & Group, T. P. R. I. S. M. A. (2009). Preferred reporting items for systematic reviews and meta-analyses: The PRISMA statement. PLoS Medicine, 6(7), e1000095. doi:10.1371/journal.pmed.1000097
Reed, D. D., DiGennaro Reed, F. D., Campisano, N., Lacourse, K., & Azulay, R. L. (2012). *Assessing and increasing staff preference for job tasks using concurrent-chains schedules and probabilistic outcomes. Journal of Organizational Behavior Management, 32(3), 253–262. doi:10.1080/01608061.2012.698121
Reid, D. H., & Parsons, M. B. (1995). *Comparing choice and questionnaire measures of the acceptability of a staff training procedure. Journal of Applied Behavior Analysis, 28(1), 95–96. doi:org10.1901/jaba.1995.28-95
Strouse, M. C., Carroll-Hernandez, T. A., Sherman, J. A., & Sheldon, J. B. (2004). Turning over turnover: The evaluation of a staff scheduling system in a community-based program for adults with developmental disabilities. Journal of Organizational Behavior Management, 23 (2–3), 45–63. doi:10.1300/J075v23n02_04
Tullis, C. A., Cannella-Malone, H. I., Basbigill, A. R., Yeager, A., Fleming, C. V., Payne, D., & Wu, P. (2011). Review of the choice and preference assessment literature for individuals with severe to profound disabilities. Education and Training in Autism and Developmental Disabilities, 46(4), 576–595. https://www.jstor.org/stable/24232368
Waldvogel, J. M., & Dixon, M. R. (2008). *Exploring the utility of preference assessments in organizational behavior management. Journal of Organizational Behavior Management, 28 (1), 76–87. doi:10.1080/01608060802006831
Wilder, D. A., Harris, C., Casella, S., Wine, B., & Postma, N. (2011). *Further evaluation of the accuracy of managerial prediction of employee preference. Journal of Organizational Behavior Management, 31(2), 130–139. doi:10.1080/01608061.2011.569202
JOURNAL OF ORGANIZATIONAL BEHAVIOR MANAGEMENT 301
Wilder, D. A., Rost, K., & McMahon, M. (2007). *The accuracy of managerial prediction of employee preference. Journal of Organizational Behavior Management, 27(2), 1–14. doi:10.1300/J075v27n02_01
Wilder, D. A., Therrien, K., & Wine, B. (2006). *A comparison between survey and verbal choice methods of identifying potential reinforcers among employees. Journal of Organizational Behavior Management, 25(4), 1–13. doi:10.1300/J075v25n04_01
Wilder, D. A., Wilson, P., Ellsworth, C., & Heering, P. W. (2003). A comparison of verbal and tangible stimulus preference assessment methods in adults with schizophrenia. Behavioral Interventions, 18(3), 191–198. doi:10.1002/bin.136
Wine, B., & Axelrod, S. (2014). *The effects of progressively thinning high-preference item delivery on responding in employees. Journal of Organizational Behavior Management, 34 (4), 291–299. doi:10.1080/01608061.2014.973629
Wine, B., Gilroy, S., & Hantula, D. A. (2012). *Temporal (in)stability of employee preferences for rewards. Journal of Organizational Behavior Management, 32(1), 58–64. doi:10.1080/ 01608061.2012.646854
Wine, B., Gugliemella, C., & Axelrod, S. (2013). *An examination of generalized-conditioned reinforcers in stimulus preference assessments. Journal of Organizational Behavior Management, 33(4), 244–251. doi:10.1080/01608061.2013.843433
Wine, B., Kelley, D. P., III, & Wilder, D. A. (2014a). *An initial assessment of effective preference assessment intervals among employees. Journal of Organizational Behavior Management, 34(3), 188–195. doi:10.1080/01608061.2014.944747
Wine, B., Reis, M., & Hantula, D. (2014b). *An evaluation of stimulus preference assessment methodology in organizational behavior management. Journal of Organizational Behavior Management, 34(1), 7–15. doi:10.1080/01608061.2013.873379
Wine, B., & Wilder, D. A. (2009). The effects of varied versus constant high-, medium-, and low-preference stimuli on performance. Journal of Applied Behavior Analysis, 42(2), 321–326. doi:10.1901/jaba.2009.42-321
302 M. J. SIMONIAN ET AL.
- Abstract
- Methods
- Literature search procedures
- Data coding
- Intercoder agreement
- Results
- Topic of study
- Participants
- Stimuli and cost
- Preference assessment type
- Frequency
- Reinforcer assessment & effectiveness
- Social validity
- Discussion
- Disclosure statement
- Ethical approval
- Informed consent
- References
