I need assistance with a research paper and critique an article

cat-_motor_and_process_skill.pdf

Home >Applied Sciences homework help >I need assistance with a research paper and critique an article

Validity of Using the Assessment of Motor and Process Skills to Determine the Need for Assistance

Brenda K. Merritt

KEY WORDS

� activities of daily living � disability evaluation � motor skills � sensitivity and specificity � task performance and analysis

Brenda K. Merritt, PhD, OT Reg(NS), OTR, is

Assistant Professor, School of Occupational Therapy,

Dalhousie University, Room 215, 5869 University Avenue,

PO Box 15000, Halifax, Nova Scotia B3H 4R2 Canada;

[email protected]

OBJECTIVE. This study evaluated the validity of using Assessment of Motor and Process Skills (AMPS) measures as evidence of the need for assistance in the community.

METHOD. In a retrospective analysis of existing data (N 5 64,466), receiver operating characteristic (ROC) curves were generated, and a split-sample method was used to validate the predictions.

RESULTS. When identifying people who need assistance versus those who do not need assistance in the community, activity of daily living (ADL) motor and ADL process measures have fair and good discriminating

value, respectively (areas under the ROC curves were .78 and .84). Evidence supports placing ADL motor and

ADL process independence cutoff measures at 1.50 logits (sensitivity 5 .67, specificity 5 .72) and 1.00

logit (sensitivity 5 .81, specificity 5 .70), respectively. Accuracy was highest when matched motor and

process decisions occurred (sensitivity 5 .85, specificity 5 .83).

CONCLUSION. Evidence supports using ADL ability measures from the AMPS to provide evidence of a client’s need for assistance in the community.

Merritt, B. K. (2011). Validity of using the Assessment of Motor and Process Skills to determine the need for assistance.

American Journal of Occupational Therapy, 65, 643–650. doi: 10.5014/ajot.2011.000547

E very day, health care professionals conduct evaluations and make recom-

mendations regarding the most appropriate discharge plans and living

environments for their clients. The overarching goal in this process is to ensure

that clients can safely and efficiently manage, either independently or with the

assistance of another, all the daily life tasks that are required for independent

living (e.g., self-care, home maintenance, money management, medication

adherence, leisure, social interaction, community transportation; e.g., Wehman,

2001).

In their exploration of this complex evaluation process, Meinow, Kåreholt,

and Lagergren (2005) found that the strongest predictors of the amount of

assistance received in the home were dependency in personal activities of daily

living (PADLs; R 2 5 .34, p < .001), dependency in instrumental activities

of daily living (IADLs; R 2 5 .39, p < .001), and cognitive function (R 2 5 .13,

p < .001). Because PADL ability and IADL ability (hereinafter jointly referred

to as activity of daily living [ADL] ability) are two of the key predictors of

the need for assistance in the community, occupational therapists play a vital

role in this evaluation process. If occupational therapists are to contribute to the

decision-making process from an evidence-informed perspective, additional

research must be conducted to document the validity and accuracy of using

ADL measures to predict the need for assistance in the community.

The Assessment of Motor and Process Skills (AMPS; Fisher, 2006a, 2006b)

is an internationally recognized, occupational therapy–specific assessment of the

quality of a client’s ADL performance. The linear ADL process ability scale of

the AMPS has shown promise when used to identify clients who need assistance

The American Journal of Occupational Therapy 643

to live in the community (Bernspång & Fisher, 1995;

Fisher, 2006a; Hartman, Fisher, & Duran, 1999; Kottorp,

2008; Merritt, 2010); however, additional research is

needed to determine the sensitivity and specificity of the

AMPS ADL cutoff measures. Sensitivity is an estimate of the true positive rate and within this context represents

the proportion of the independent sample correctly

identified as independent. In contrast, specificity is an estimate of the true negative rate, or the proportion of the

sample in need of assistance that is correctly identified as

needing assistance. Identifying the sensitivity and speci-

ficity estimates that arise when using AMPS ADL ability

measures to predict the need for assistance could provide

occupational therapists with additional evidence to sup-

port their clinical judgments. For example, this evidence

could potentially be used to acquire support from fund-

ing agencies to pay for a client’s care, establish the need

for additional health care services to maximize a client’s

independence and lessen caregiver burden, and document

the efficacy of occupational therapy services and the im-

pact they have had on the client’s need for assistance and

burden of care.

To address this gap in the literature, this study

addressed three issues: (1) whether significant association

exists between ADL ability (motor ability, process ability,

or both) and global functional level, (2) the sensitivity and

specificity estimates that arise when using the current

AMPS ADL motor and ADL process cutoff measures to

differentiate clients who are independent in the com-

munity from those who are in need of assistance, and

(3) the need to create new cutoff points that reflect more

acceptable estimates of sensitivity and specificity (i.e.,

lower rates of error).

Method

I conducted a retrospective, criterion-based validation

study of the AMPS ADL motor and ADL process ability

cutoff measures. Per standard protocol, before beginning

this research project, I obtained approval from the Col-

orado State University regulatory compliance office.

Participants and Site

I used existing data from the AMPS Project International

database in Fort Collins, Colorado. The data were from

people who were assessed by occupational therapy prac-

titioners within typical occupational therapy settings (e.g.,

hospitals, client homes, rehabilitation clinics). The sample

potentially included all available data records in the AMPS

database that were evaluated between January 1, 1999, and

January 1, 2005, and met the following criteria: (1) were

age 16 yr or older; (2) were not associated with rater

scoring error, as evidenced by artificially high ADL motor

or ADL process ability measures; (3) were not scored by

multiple raters as part of rater calibration; and (4) had

known sex, age, and global functional level ratings. To

clarify, only data from 1999 to 2005 were selected, because

new criteria for rating each global functional level were

created in 1999 and remained consistent until the data were

extracted in 2006 (Fisher, 1999). Before 1999, specific

criteria for rating functional level had not been developed.

For this study, 64,466 data records (58% of the

potential sample in the AMPS database) met the criteria

for inclusion. Data records eliminated from consideration

were those of people younger than age 16 (23% of the

potential sample), were associated with rater scoring

error (10% of the potential sample), or had incomplete

demographic data (9% of the potential sample). Data

records included those for men (42.9%) and women

(57.1%) from North America (14.8%), the United

Kingdom (22.3%), the Nordic countries (34.2%), other

European countries (11.6%), New Zealand and Australia

(8.7%), Asia (8.0%), and unknown regions (0.4%). Data

records were grouped into global diagnostic categories

(e.g., the psychiatric diagnostic category included people

with bipolar disorder, depression, schizophrenia, or per-

sonality disorder; see Table 1). Although specific di-

agnostic information could be lost by coding the data

in this manner, it ensured that sample sizes in each di-

agnostic group were sufficient for statistical analyses.

Instrumentation

AMPS. The AMPS was administered by occupational

therapists who attended a 5-day AMPS training workshop

and subsequently were calibrated as AMPS raters (e.g.,

demonstrated valid and reliable administration and

scoring of the AMPS). After observing the client perform

at least two AMPS tasks, each rater used the criteria in the

AMPS manual (Fisher, 2006b) to score the 16 ADL

motor skill and 20 ADL process skill items. Globally,

ADL motor skills are defined as the observable actions the person enacts when moving the self or task objects

(Fisher, 2006a). ADL process skills are defined as “the observable actions of performance the person enacts to

logically sequence the actions of the ADL task perfor-

mance over time, select and use appropriate tools and

materials, and adapt his or her performance when prob-

lems are encountered” (Fisher, 2006a, p. 4). According to

the criteria in the AMPS manual, each ADL skill item

was scored using a 4-point ordinal rating scale based

on the global criterion of competence (1 5 deficient

644 November/December 2011, Volume 65, Number 6

performance and/or task breakdown, 2 5 ineffective perfor- mance, 3 5 questionable performance, 4 5 competent per- formance). Each participant had at least two sets of ADL motor scores and two sets of ADL process scores: one set

of scores for each task performed (i.e., two sets of scores

were used to generate one ADL motor ability measure and

one ADL process ability measure for each person).

Previous studies have supported using the AMPS cutoff

measures as indicators of the need for assistance (Bernspång

& Fisher, 1995; Hartman et al., 1999). Bernspång and

Fisher (1995) developed a “risk” zone, delineated by ADL

measures that fall within ±0.3 logit of the ADL motor

cutoff measure of 2.0 logits or ADL process cutoff measure

of 1.0 logit. Their results indicate that approximately two-

thirds of the clients who were frail or at risk of functional

decline had ADL ability measures within the risk zones.

With a sample of older adults (with and without de-

mentia), Hartman et al. (1999) documented a 94% overall

correct classification rate when using the AMPS ADL

process cutoff measure as an indicator of the need for as-

sistance. Previous studies have also supported the reliability

and validity of the AMPS ability measures across age

groups (Hayase et al., 2004), between men and women

(Merritt & Fisher, 2003), and with a variety of diagnoses

(e.g., Doble, Fisk, Fisher, Ritvo, & Murray, 1994;

Hartman et al., 1999; Rexroth, Fisher, Merritt, & Gliner,

2005). Last, the AMPS has also been shown to be a sensitive

outcome measure (e.g., Chard, Liu, & Mulholland, 2009;

Goverover, Johnston, Toglia, & Deluca, 2007; Kinnman,

Andersson, Wetterquist, Kinnman, & Andersson, 2000;

Oakley, Khin, Parks, Bauer, & Sunderland, 2002). Research

has demonstrated that the characteristics of the test

function independently of the person being tested (e.g.,

the item and task challenge calibrations do not vary

between samples; Fisher, 2006b); thus, the linear ADL

ability measures of men and women from different

world regions and with different diagnoses can be val-

idly generated and compared.

Global Functional Level. Considering the evidence that

clinical judgment appears to be one of the most accurate

means of determining global functioning within the

community (Pinholt et al., 1987; Rogers et al., 2003),

that direct observation of ADL performance tends to lead

to more accurate estimates of global functioning within

the community (Rogers et al., 2003), and that AMPS

ADL ability measures and AMPS global functional level

ratings are concurrently documented, I concluded that

a practical and valid external criterion available for eval-

uating the accuracy of the AMPS cutoff measures was the

AMPS global functional-level rating. Global functional

level was rated by trained and calibrated AMPS raters

according to the specific criteria outlined in the AMPS

manual (Fisher, 2006b; see Table 2). More specifically,

each occupational therapist used his or her clinical judg-

ment to determine whether the client was able to live

independently, in need of minimal assistance or super-

vision, or in need of moderate to maximal assistance to

live in the community.

The AMPS global functional level was not based solely

on the AMPS task observations; rather, the rating was

based on all information the therapist gathered about the

client. As a result, the therapist used multiple sources of

information to determine the most accurate rating (e.g.,

interview, caregiver report, reports from other health care

providers, observation of ADL performance, assessment

of body functions). In support, other researchers have

Table 1. Number of Participants by Diagnostic Group and Global Functional Level

Diagnostic Category

Global Functional Level

TotalIndependent Minimal Assistance Moderate Assistance

Well 1,226 0 0 1,226

At risk 424 21 0 445

Frail older adult 0 61 6 67

Mild learning disability 42 52 32 126

Neurological developmental 57 315 636 1,008

Mental retardation 44 621 1,015 1,680

Other neurological 1,055 3,258 4,454 8,767

Hemispheric stroke 777 3,145 4,879 8,801

Musculoskeletal 2,598 3,003 1,780 7,381

Medical 705 1,009 851 2,565

Psychiatric 1,580 3,998 2,978 8,556

Dementia 99 681 1,708 2,488

Other memory 27 73 90 190

Other 1,824 7,442 11,900 21,166

Total 10,458 23,679 30,329 64,466

The American Journal of Occupational Therapy 645

reported that using multiple sources of information, in-

cluding direct observation of ADL ability, is the most

accurate means of determining global functioning in the

community (Jette, Grover, & Keck, 2003; Rogers et al.,

2003). To minimize the risk of biased ratings, at the time

of rating the client’s AMPS global functional level the

raters were blind to the person’s final AMPS ability

measures and unaware that the functional level ratings

would be used to investigate the validity of the AMPS

ability measures.

Procedure and Data Analysis

A specialized, many-facet Rasch analysis program, Facets

(Linacre, 2006), was used to convert the raw ordinal ADL

skill item scores into linear ADL motor and ADL process

ability measures. Such analyses adjust the final ADL

ability estimates to account for task challenge, skill item

difficulty, and severity of the rater. Demographic in-

formation and ADL motor and ADL process ability

measures were imported into SPSS Version 17.0 for

Windows (SPSS, Inc., Chicago).

To evaluate the association between ADL ability

measures and global functional level, two one-way analyses

of variance (ANOVAs) were conducted (one for ADL

motor ability and one for ADL process ability), and the

size of the effect for each analysis was determined. Effect

size (h 2 ) was interpreted on the basis of Cohen’s measure

of association (Cohen, 1988), in which .01, .06, and .14

indicate small, medium, or large effects, respectively.

To prepare the data for receiver operating charac-

teristic (ROC) curve analysis, AMPS global functional

level ratings were recoded into a dichotomous variable

denoting independent (n 5 10,458) or in need of assistance to live in the community (n 5 54,008). Next, the data were prepared to conduct a split-sample validation of the

predictive model (Kohavi, 1995). Therefore, approxi-

mately 60% of the sample (n 5 38,540) was randomly selected using the SPSS random sampling function; this

subsample became the test sample and was used to gen-

erate ROC curves and subsequently develop the pre-

dictive model. The remaining data records in the sample

(n 5 25,926) became the validation sample, which was used to validate the predictive model. The test and vali-

dation samples had the same proportion of data records

in each functional level group as did the entire sample;

16% of data records in each sample were independent,

36% were in need of minimal assistance, and 47% were

in need of moderate to maximal assistance.

Using the test sample (n 5 38,540), the dichotomous state variable (need for assistance) was used to generate two ROC curves, one with ADL motor ability as the test

variable and one with ADL process ability as the test

variable. Normal distribution and homogeneity of vari-

ance were not assumed, and thus a nonparametric model

was used to generate the ROC curves (Brown & Davis,

2006).

The area under the curve (AUC) was examined to

globally determine whether the ADL motor and ADL

process ability measures had merit in correctly categorizing

clients who were independent from those who needed

assistance to live in the community. Rough guidelines for

determining the discriminating value of a test by exam-

ining the AUC values are as follows: fail 5 .50–.60, poor 5 .60–.70, fair 5 .70–.80, good 5 .80–.90, and excellent 5 .90–1.00 (Perneczky et al., 2006).

After verification that the ADL motor and ADL process

scales had merit in predicting independence in the com-

munity (i.e., AUC ³ .70), the data were examined more closely. First, the ROC curves were examined to determine

the sensitivity and specificity estimates of the current ADL

motor measure of 2.00 logits and ADL process ability

cutoff measure of 1.00 logit. Next, the ROC curves were

Table 2. Age and ADL Motor and ADL Process Ability Results by Global Functional Level

Characteristic

Global Functional Level

Independent (n 5 10,458) Minimal Assistance (n 5 23,679) Maximal Assistance (n 5 30,329)

Age, yr

M 55.1 57.6 61.7

SD 17.9 20.2 20.6

ADL motor ability, logits

Range 20.03–3.91 20.76–3.47 23.05–3.46

M 1.83 1.30 0.64

SD 0.69 0.76 0.98

ADL process ability, logits

Range 0.07–2.91 20.36–2.35 22.75–2.05

M 1.45 0.99 0.35

SD 0.51 0.48 0.96

Note. ADL 5 activity of daily living; M 5 mean; SD 5 standard deviation.

646 November/December 2011, Volume 65, Number 6

examined to determine whether a need existed to create

new cutoff measures to reflect more accurate estimates of

independence or need for assistance to live in the community.

With regard to identifying community independence,

I found no specific recommendations for desired levels of

sensitivity and specificity in the literature. Potential cutoff

measures were thus determined by examining the ROC

curve at the clinically relevant area of the curve, defined as the point on the curve where false positive rates (i.e., test

incorrectly indicates that the person is able to live in-

dependently) are minimized without resulting in large

false negative rates (i.e., test result incorrectly indicates

that the person needs assistance in the community). In

support of the decision to minimize false positive errors,

Dijkstra, Tiesinga, Plantinga, Veltman, and Dassen (2005)

also chose cutoff values that minimized errors that could

lead to clients not receiving the care, treatment, or support

that they actually needed.

Once the optimal cutoff measures for ADL motor and

ADL process ability were determined, the validation sample

(n 5 25,926) was used to create a predicted need for as- sistance variable that was based on each person’s ADL motor ability; people under the cutoff were coded as

needing assistance, whereas those above the cutoff were

coded as being independent in the community. In a simi-

lar manner, a second predicted need for assistance variable was created on the basis of the ADL process cutoff mea-

sure. Congruence between predicted need for assistance and the clinician’s judgment of the need for assistance (i.e.,

AMPS global functional level) was then analyzed to vali-

date the predictive model. To determine whether cate-

gorical accuracy could be improved on, the accuracy

achieved when decision points matched (i.e., when both

ADL motor and ADL process abilities were either above or

below the cutoff measures) was investigated.

Finally, from among all of the data records (n 5 64,466), separate ROC curves for samples within each

diagnostic category were generated using the occupa-

tional therapist’s original rating of need for assistance as the state variable and ADL motor ability or ADL process ability as the test variable. For this analysis, the AUC was examined to globally determine the relative predictive

validity of the AMPS ability measures across the dif-

ferent diagnostic categories. Data were drawn from the

entire data set to maximize the sample size of each di-

agnostic grouping.

Results

Global functional level was found to be significantly as-

sociated with ADL motor ability, F(2, 64463) 5 8,728,

p < .01, and ADL process ability, F(2, 64463) 5 15,916, p < .01. According to Cohen’s (1988) criteria, ADL motor and ADL process ability measures have large

effects with regard to global functional level; h 2 values

were .213 and .331, respectively. Although age was

also found to be significantly associated with functional

level, F(2, 64463) 5 526, p < .01, the effect was small (h

2 5 .016). The ROC curve for examining the utility of using

ADL motor ability measures to determine community

independence revealed an AUC estimate of .78, indicating

fair discriminating value. Examination of the ROC curve

revealed that the sensitivity (i.e., rate of those correctly

identified as being independent within the community) of

the current ADL motor cutoff measure of 2.00 logits was

.40, and the specificity (i.e., rate of those correctly

identified as needing assistance to live in the community)

was .87. On further examination of the clinically relevant

area on the ROC curve, an ideal cutoff measure for ADL

motor ability was determined to be 1.50 logits, for which

sensitivity and specificity estimates were .67 and .72, re-

spectively. Using the validation sample, the accuracy

estimates were validated; when the predicted need for

assistance was compared with the clinician’s judgment of

the need for assistance, the new ADL motor cutoff of 1.5

logits resulted in a sensitivity of .68 and a specificity of

.72, thus validating the ROC curve estimates.

When ADL process ability measures were used to

determine community independence, the AUC was .84,

indicating good discriminating value. Examination of the

ROC curve revealed that the sensitivity and specificity

estimates of the current ADL process cutoff measure of

1.00 logits were .80 and .70, revealing an acceptable and

ideal cutoff measure. Within the validation sample, when

the predicted need for assistance was compared with the

clinician’s judgment of the need for assistance, the ADL

process cutoff of 1.0 logit resulted in a sensitivity rating of

.81 and a specificity rating of .70, thus validating the

ROC curve estimates.

To prepare the data to evaluate the accuracy of using

matched ADL motor and ADL process decisions, only

those participants in the validation sample (n 5 25,926) with matched predicted ADL motor and ADL process

decision points were selected (i.e., ADL motor and ADL

process ability measures were either both above or both

below the respective cutoff measures). Approximately

65% (n 5 16,807) of the validation sample had matched predicted decisions, and the sensitivity and specificity

estimates of this sample were .85 and .83, respectively.

Thus, accuracy estimates were highest when matched

ADL motor and ADL process decisions occurred.

The American Journal of Occupational Therapy 647

Finally, separate ROC curves were generated for each

diagnostic group. Because of invariant ratings of the need

for assistance, the data coded with a diagnosis of either well or frail older adult were not analyzed. AUC estimates ranged from .68 to .85 (poor to good ratings) for ADL

motor ability and from .72 to .92 (fair to excellent rat-

ings) for ADL process ability (Table 3).

Discussion

The analyses indicate that both ADL motor ability and

ADL process ability have utility as indicators of the need

for assistance to live in the community. At best, when the

data records in the validation sample had matched ADL

motor and ADL process decision points (n 5 16,807), 85% of the independent sample and 83% of the sample

in need of assistance were correctly categorized. Until

now no study has investigated the use of matched ADL

motor and ADL process decisions, and thus the findings

have the potential to provide the clinical and research

communities with a new means of examining the con-

stellation of ability measures for the purpose of estimating

the need for assistance to live in the community.

When decision points did not match, the most ac-

curate predictions were obtained when using the ADL

process ability cutoff measure of 1.0 logit; 80% of the

independent sample and 70% of the sample in need of

assistance were correctly classified. The ADL process scale

of the AMPS continues to be more closely associated with

global functioning in the community than the ADL motor

scale, in line with previous studies (Fisher, 2006a; Hartman

et al., 1999; Kottorp, 2008; Merritt, 2010).

Although the current ADL motor cutoff measure of

2.00 logits has not been used as an indicator of in-

dependence in the community, the data revealed that the

current cutoff measure is too high for use in categorizing

independence versus the need for assistance (i.e., 60% of

the independent sample was incorrectly coded as needing

assistance). As a result, a new ADL motor cutoff measure

of 1.50 logits is proposed, resulting in more acceptable

estimates of sensitivity and specificity. Although additional

research is warranted, this study documents the first line of

evidence that ADL motor ability has some promise for use

as an indicator of community independence.

Global estimates of the accuracy of using AMPS ability

measures to predict the need for assistance were evaluated

across the various diagnostic groups (Table 3). The di-

agnosis-specific AUC estimates ranged from poor to good

for ADL motor ability and from fair to excellent for ADL

process ability. Although this global statistic does not

provide conclusive evidence of an assessment’s utility and

accuracy (Zweig & Campbell, 1993), the AUC statistics in

Table 3 suggest that the ADL motor and ADL process

ability scales are not equally accurate across all diagnostic

categories. More specifically, this is the first line of evi-

dence that suggests that the ADL motor ability scale may

be more accurate than the ADL process ability scale for

clients within specific diagnostic categories (e.g., muscu-

loskeletal, medical), whereas the ADL process ability scale

may be more accurate for clients with dementia, neuro-

logical developmental disabilities, and so forth.

Although the diagnosis-specific AUC statistics do not

provide conclusive evidence, they do provide a foundation

for comparing the accuracy estimates of other assessments.

For example, Mausbach and colleagues (2008) evaluated

the accuracy of using either the University of California,

San Diego, Performance-Based Skills Assessment (UPSA;

Patterson, Goldman, McKibbin, Hughs, & Jeste, 2001)

or the Dementia Rating Scale (DRS; Mattis, 1973) to

predict residential independence of people with schizo-

phrenia (n 5 434). They determined that the AUC es- timates were .74 for the UPSA (an evaluation of ADL

ability) and .65 for the DRS (an evaluation of cognition).

In contrast, the current findings indicate that with

a sample of clients with psychiatric illness, the AUC

estimate for the AMPS ADL process scale was .77 (Table

3). Additional studies are warranted to fully evaluate and

compare the accuracy estimates of the AMPS cutoff mea-

sures across different diagnoses.

The results of this study are promising and indicate

that the AMPS ability measures have merit when used to

document the need for assistance to live in the community;

however, the predictions are not 100% accurate. One

Table 3. Area Under the Curve (AUC) Estimates by Diagnostic Category

Diagnostic Categorya n

AUC

ADL Motor ADL Process

At risk 445 .75 .72

Mild learning disability 126 .67 .75

Neurological developmental 1,008 .73 .85

Mental retardation 1,680 .75 .83

Other neurological 8,767 .79 .83

Hemispheric stroke 8,801 .82 .82

Musculoskeletal 7,381 .82 .77

Medical 2,565 .85 .81

Psychiatric 8,556 .68 .77

Dementia 2,488 .78 .92

Other memory 190 .85 .91

Other 21,166 .79 .83

Note. ADL 5 activity of daily living. aWell and frail older people were not analyzed because of invariant ratings of the need for assistance.

648 November/December 2011, Volume 65, Number 6

explanation for this is the fact that the AMPS was not

designed to measure global functioning within the com-

munity. The ability to live independently in the com-

munity requires the performance of tasks that extend

beyond those included in the AMPS (e.g., participation in

social tasks, community transportation, money manage-

ment, phone use, computer use); thus, the AMPS ADL

ability measures can be expected to explain some, but

not all, of the variation in the construct of community

independence.

Another area to explore is that of rater scoring error.

For example, occupational therapists working with clients

who have physical disabilities may overemphasize obser-

vation and scoring of ADL motor skill deficits relative to

ADL process skill deficits, resulting in inflated ADL

process ability measures and higher false positive errors.

Likewise, rater scoring error can occur when occupational

therapists working with clients who have cognitive or

psychiatric deficits focus their observations and scoring on

ADL process skills while failing to observe and accurately

score ADL motor skills, resulting in inflated ADL motor

ability measures and higher false positive errors. In such

instances, rater error is introduced; rater severity should

not change on the basis of the characteristics of the clients

being assessed.

Another potential source of error within the current

study is the accuracy of the occupational therapists’ ratings

of global functional level. The possibility exists that

clinicians are more accurate when determining the global

functional levels of those who are on the extreme ends of

the scale (e.g., clients who are clearly independent in the

community and those who require substantial assistance

to live in the community). When clients require some

assistance to live in the community or have inconsistent

needs for assistance, the possibility exists that clinicians’

judgments of global functional level are inaccurate. Ad-

ditional research is warranted to verify this assumption

and to further investigate the potential causes of error in

determining global functional level.

Although the lack of evidence supporting the validity

and reliability of the AMPS global functional ratings

constitutes a limitation of this study, use of this rating scale

created an economical opportunity to use a large existing

database to generate initial evidence of the merit of using

AMPS ADL ability measures to determine a person’s need

for assistance in the community. Clearly, follow-up

studies are warranted that prospectively gather ADL

measures or AMPS global functional ratings and measure

them against additional criteria (e.g., narrative reports

from caregivers, overall time that assistance was provided

over the course of a week, daily assistance logs). Such

studies could seek to validate use of the AMPS cutoff

measures as evidence of the need for assistance and could

document the validity and reliability of the AMPS global

functional level ratings.

Conclusion

Occupational therapists play a critical role in determining

clients’ discharge needs and need for assistance in the

community. This study provides evidence that occupa-

tional therapists can use the AMPS ability measures to

support decisions regarding a client’s potential need for

support or assistance in the community. The findings

support using AMPS ADL motor and ADL process

ability measures as evidence of a client’s need for assis-

tance to live in the community, but it is important to

stress that such decisions are rarely made on the basis

of the results from one assessment. Although many occu-

pational therapists use functional assessments to determine

discharge needs, they also consider constructs beyond

personal and domestic ADL performance, including the

client’s wants and needs, the client’s life context, the se-

verity and prognosis of the client’s impairment, and other

professionals’ opinions (Dijkstra et al., 2005). Thus, AMPS

ability measures should contribute to and support the

clinician’s judgments and recommendations and should

not serve as the sole piece of evidence when determining

the need for assistance. s

Acknowledgments

I thank Anne G. Fisher for her support and extensive

expertise. Previous versions of this article were presented at

the 2008 International Assessment of Motor and Process

Skills Symposium: Measuring, Planning, and Implementing

Occupation-Based Programs, Halifax, Nova Scotia, and the

2009 American Congress of Rehabilitation Medicine–

American Society of Neurorehabilitation Joint Conference,

Denver, Colorado.

References Bernspång, B., & Fisher, A. G. (1995). Differences between

persons with right or left cerebral vascular accident on the Assessment of Motor and Process Skills. Archives of Phys- ical Medicine and Rehabilitation, 76, 1144–1151. doi: 10.1016/S0003-9993(95)80124-3

Brown, C. D., & Davis, H. T. (2006). Receiver operating characteristics curves and related decision measures: A tu- torial. Chemometrics and Intelligent Laboratory Systems, 80, 24–38. doi: 10.1016/j.chemolab.2005.05.004

Chard, G., Liu, L., & Mulholland, S. (2009). Verbal cueing and environmental modifications: Strategies to improve engagement in occupations in persons with Alzheimer

The American Journal of Occupational Therapy 649

disease. Physical and Occupational Therapy in Geriatrics, 27, 197–211. doi: 10.1080/02703180802206280

Cohen, J. (1988). Statistical power and analysis for the behav- ioral sciences (2nd ed.). Hillsdale, NJ: Erlbaum.

Dijkstra, A., Tiesinga, L. J., Plantinga, L., Veltman, G., & Dassen, T. W. (2005). Diagnostic accuracy of the Care Dependency Scale. Journal of Advanced Nursing, 50, 410–416. doi: 10.1111/j.1365-2648.2005.03406.x

Doble, S. E., Fisk, J. D., Fisher, A. G., Ritvo, P. G., & Murray, T. J. (1994). Functional competence of community- dwelling persons with multiple sclerosis using the Assess- ment of Motor and Process Skills. Archives of Physical Medicine and Rehabilitation, 75, 843–851. doi: 10.1016/ 0003-9993(94)90107-4

Fisher, A. G. (1999). Assessment of Motor and Process Skills (3rd ed.). Fort Collins, CO: Three Star Press.

Fisher, A. G. (2006a). Assessment of Motor and Process Skills: Vol. 1. Development, standardization, and administration manual (6th ed.). Fort Collins, CO: Three Star Press.

Fisher, A. G. (2006b). Assessment of Motor and Process Skills: Vol. 2. User manual (6th ed.). Fort Collins, CO: Three Star Press.

Goverover, Y., Johnston, M. V., Toglia, J., & Deluca, J. (2007). Treatment to improve self-awareness in persons with acquired brain injury. Brain Injury, 21, 913–923. doi: 10.1080/02699050701553205

Hartman, M. L., Fisher, A. G., & Duran, L. (1999). Assess- ment of functional ability of people with Alzheimer’s dis- ease. Scandinavian Journal of Occupational Therapy, 6, 111–118. doi: 10.1080/110381299443690

Hayase, D., Mosenteen, D. A., Thimmaiah, D., Zemke, S., Atler, K., & Fisher, A. G. (2004). Age-related changes in activities of daily living (ADL) ability. Australian Occupa- tional Therapy Journal, 51, 192–198. doi: 10.1111/j.1440- 1630.2004.00425.x

Jette, D. U., Grover, L., & Keck, C. P. (2003). A qualitative study of clinical decision making in recommending dis- charge placement from the acute care setting. Physical Therapy, 83, 224–236.

Kinnman, J., Andersson, U., Wetterquist, L., Kinnman, Y., & Andersson, U. (2000). Cooling suit for multiple sclerosis: Functional improvement in daily living. Scandinavian Journal of Rehabilitation Medicine, 32, 20–24. doi: 10.1080/003655000750045695

Kohavi, R. (1995). A study of cross validation and bootstrap for accuracy estimation and model selection. In Proceedings of the 14th International Joint Conference on Artificial Intelligence (pp. 338–345). San Francisco: Morgan Kaufmann.

Kottorp, A. (2008). The use of the Assessment of Motor and Process Skills (AMPS) in predicting need of assistance for adults with mental retardation. OTJR: Occupation, Partic- ipation and Health, 28, 72–80. doi: 10.3928/15394492- 20080301-04

Linacre, J. M. (2006). Facets Rasch measurement computer pro- gram. Chicago: Winsteps.

Mattis, S. (1973). Dementia Rating Scale. Odessa, FL: Psycho- logical Assessment Resources.

Mausbach, B. T., Bowie, C. R., Harvey, P. D., Twamley E. W., Goldman, S. R., Jeste, D. V., et al. (2008). Use- fulness of the UCSD Performance-Based Skills Assessment (UPSA) for predicting residential independence in patients with chronic schizophrenia. Journal of Psychiatric Research, 42, 320–327. doi: 10.1016/j.jpsychires.2006.12.008

Meinow, B., Kåreholt, I., & Lagergren, M. (2005). According to need? Predicting the amount of municipal home help allocated to elderly recipients in an urban area of Sweden. Health and Social Care in the Community, 13, 366–377. doi: 10.1111/j.1365-2524.2005.00570.x

Merritt, B. K. (2010). Utilizing AMPS ability measures to predict level of community dependence. Scandinavian Journal of Occupational Therapy, 17, 70–76.

Merritt, B. K., & Fisher, A. G. (2003). Gender differences in the performance of activities of daily living. Archives of Physical Medicine and Rehabilitation, 84, 1872–1877. doi: 10.1016/S0003-9993(03)00483-0

Oakley, F., Khin, N. A., Parks, R., Bauer, L., & Sunderland, T. (2002). Improvement in activities of daily living in elderly following treatment for post-bereavement depres- sion. Acta Psychiatrica Scandinavica, 105, 231–234. doi: 10.1034/j.1600-0447.2002.1sc021.x

Patterson, T. L., Goldman, S., McKibbin, C. L., Hughs, T., & Jeste, D. V. (2001). UCSD Performance-Based Skills As- sessment: Development of a new measure of everyday functioning for severely mentally ill adults. Schizophrenia Bulletin, 27, 235–245.

Perneczky, R., Pohl, C., Sorg, C., Hartmann, J., Komossa, K., Alexopoulos, P., et al. (2006). Complex activities of daily living in mild cognitive impairment: Conceptual and di- agnostic issues. Age and Ageing, 35, 240–245. doi: 10. 1093/ageing/afj054

Pinholt, E. M., Kroenke, K., Hanley, J. F., Kussman, M. J., Twyman, P. L., & Carpenter, J. L. (1987). Functional assessment of the elderly: A comparison of standard instru- ments with clinical judgment. Archives of Internal Medi- cine, 147, 484–488. doi: 10.1001/archinte.147.3.484

Rexroth, P., Fisher, A. G., Merritt, B. K., & Gliner, J. (2005). Ability differences in persons with unilateral hemispheric stroke. Canadian Journal of Occupational Therapy, 72, 212–221.

Rogers, J. C., Holm, M. B., Beach, S., Schulz, R., Cipriani, J., Fox, A., et al. (2003). Concordance of four methods of disability assessment using performance in the home as the criterion method. Arthritis Care and Research, 49, 640–647. doi: 10.1002/art.11379

Wehman, P. (2001). Life beyond the classroom: Transition strat- egies for young people with disabilities. Baltimore: Paul H. Brooks.

Zweig, M. H., & Campbell, G. (1993). Receiver-operating characteristic (ROC) plots: A fundamental evaluation tool in clinical medicine. Clinical Chemistry, 39, 561–577.

650 November/December 2011, Volume 65, Number 6

Association and its content may not be copied or emailed to multiple sites or posted to a listserv without the

individual use.