advanced health assessment

Comparative accuracies of two common screening instruments for the classification of Alzheimer’s disease, mild cognitive impairment and healthy aging

David R. Roalf1, Paul J. Moberg1,2,4, Sharon X. Xie3,4, David A. Wolk2,4, Stephen T. Moelter5, and Steven E. Arnold1,2,4

1Department of Psychiatry, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA

2Department of Neurology, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA

3Department of Biostatistics and Epidemiology, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA

4Alzheimer’s Disease Center, University of Pennsylvania Perelman School of Medicine, Philadelphia, PA

5Department of Behavioral and Social Sciences, University of the Sciences, Philadelphia, PA, 19104

Abstract

Objective—To compare the utility and diagnostic accuracy of the MoCA and MMSE in the diagnosis of Alzheimer’s disease (AD) and Mild Cognitive Impairment (MCI) in a clinical cohort.

Method—321 AD, 126 MCI and 140 older adults with healthy cognition (HC) were evaluated using the the MMSE, MoCA, a standardized neuropsychological battery according to the

Consortium to Establish a Registry of Alzheimer’s Disease (CERAD-NB) and an informant based

measure of functional impairment, the Dementia Severity Rating Scale (DSRS). Diagnostic

accuracy and optimal cut-off scores were calculated for each measure, and a method for

converting MoCA to MMSE scores is presented also.

Results—The MMSE and MoCA offer reasonably good diagnostic and classification accuracy as compared to the more detailed CERAD-NB; however, as a brief cognitive screening measure the

MoCA was more sensitive and had higher classification accuracy for differentiating MCI from

HC. Complementing the MMSE or the MoCA with the DSRS significantly improved diagnostic

accuracy.

Corresponding Author: David Roalf, University of Pennsylvania Department of Psychiatry, Neuropsychiatry, 3400 Spruce Street, Gates Building 10th Floor, Philadelphia, PA 19104; Phone:215-662-7559; [email protected].

Conflicts: The authors report no conflicts of interest

Publisher's Disclaimer: This is a PDF file of an unedited manuscript that has been accepted for publication. As a service to our customers we are providing this early version of the manuscript. The manuscript will undergo copyediting, typesetting, and review of the resulting proof before it is published in its final citable form. Please note that during the production process errorsmaybe discovered which could affect the content, and all legal disclaimers that apply to the journal pertain.

HHS Public Access Author manuscript Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

Published in final edited form as: Alzheimers Dement. 2013 September ; 9(5): 529–537. doi:10.1016/j.jalz.2012.10.001.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

Conclusion—The current results support recent data indicating that the MoCA is superior to the MMSE as a global assessment tool, particularly in discerning earlier stages of cognitive decline. In

addition, we found that overall diagnostic accuracy improves when the MMSE or MoCA is

combined with an informant-based functional measure. Finally, we provide a reliable and easy

conversion of MoCA to MMSE scores. However, the need for MCI-specific measures is still

needed to increase the diagnostic specificity between AD and MCI.

Keywords

Alzheimer’s disease; Mild Cognitive Impairment; MMSE; MoCA; Diagnostic accuracy

1. Introduction

Alzheimer’s disease (AD) is a chronic, debilitating condition causing significant disease

burden and mortality in adults over the age of 65 (1–4). The health and economic impact of

AD has lead to a pressing need to prevent or slow disease onset and progression, and recent

research efforts have focused on the transitional period from normal cognitive aging to

dementia (5). This transition period, typically associated with Mild Cognitive Impairment

(MCI), is signified by a measurable deterioration in cognitive function that is greater than

expected based upon an individual’s age and education but which has not meaningfully

affected a person’s daily functioning (6). Despite its being a major research focus in recent

years (7), establishing the diagnosis of MCI with standard neuropsychological assessment

instruments has remained challenging (8).

In the absence of MCI screening measures with established cut-off scores and confidence

intervals, more extensive neuropsychological testing is often advised to reliably differentiate

MCI from healthy cognitive aging (HC) or AD, although professional availability, time and

cost do not often allow this. An efficient battery used at many specialized memory clinics

and Alzheimer’s Disease Research Centers is the Consortium to Establish a Registry for

Alzheimer’s neuropsychological battery (CERAD-NB) (9–13). While this battery was

developed specifically for AD, it also has been used to discriminate between MCI and HC

with some success (12–14). While the latter battery is briefer than standard

neuropsychological batteries, the CERAD-NB is still lengthy and requires specialized

training for proper administration making it less practical for use in the typical neurology or

geriatric practices. Thus, brief, validated, practical measures that can differentiate MCI and

dementia would be very useful given the increase in the proportion of elderly individuals

seeking medical help for memory-related changes.

Brief screening measures, such as the Mini–Mental State Examination (MMSE) (15) and

Montreal Cognitive Assessment (MoCA) (16), are easily administered with little training

and have demonstrated diagnostic utility (17, 18), particularly in differentiating dementia

from normal cognitive aging (19, 20). Recently, the diagnostic accuracy of these two

screening measures has received increased attention as the importance of differentiating AD

and MCI has grown (17, 21). Both the MMSE and MoCA accurately differentiate cognitive

impairment (MCI or AD) from normal cognitive aging (20), but the MoCA has shown more

utility in other disorders (e.g. Parkinson’s disease(22). To our knowledge, only one study

has directly compared the MoCA and the MMSE in differentiating MCI from AD and

Roalf et al. Page 2

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

healthy aging in a large clinical cohort (19) and one study examined MCI subtypes (23). The

latter study, which included AD patients, found improved accuracy for the MoCA, but the

study was limited to the amnestic subtype of MCI in a Portuegese sample. There is a dearth

of research comparing the diagnostic accuracy of the MoCA to other standardized measures

in a well-characterized clinical dementia cohort. Furthermore, if the MoCA is a more

accurate screening measure for MCI, as shown in other neurological disorders (24, 25), then

a straightforward conversion of MoCA to MMSE scores would be useful given the vast

amount of clinical and research that has been collected using the MMSE.

It is doubtful that brief cognitive screening instruments alone would be sufficient to

accurately distinguish mild dementia from MCI. Such a distinction is made when cognitive

decline is by definition associated with meaningful decline in daily personal, social, or

occupational functioning (26–28). Such functional decline may not be reported by the

patient and requires report from a family member or other informant. There are many

observer-based functional rating scales that together with a brief cognitive measure may

significantly improve diagnostic accuracy (29–35).

The aims of this study were: 1) to compare the diagnostic accuracy of the MMSE and

MoCA to a more detailed assessment of neuropsychological performance (CERAD-NB); 2)

to measure the improvement in diagnostic accuracy when the MMSE or MoCA were used in

combination with a measure of functional impairment (DSRS; 35) and 3) to determine a

simple and reliable algorithm for conversion of MoCA to MMSE scores in diagnosis of AD

and MCI in a large, well-characterized community-dwelling cohort evaluated and diagnosed

by clinicians at a specialized memory center.

2. Methods

2.1 Study population

All participants were recruited from the Penn Memory Center and Clinical Core of the

University of Pennsylvania’s Alzheimer’s Disease Center. Participants included 321

individuals with clinical consensus diagnoses of AD, 126 individuals with MCI and 140 HC

adults (Table 1). Diagnostic assessments included history, physical, and neurologic

examinations conducted by experienced clinicians, including the review of neuroimaging,

psychometric and laboratory data. On the basis of all this data, a consensus diagnosis was

established using standardized clinical criteria for AD, MCI, or other neurological or

psychiatric conditions presenting with cognitive impairment (5, 6, 26, 36).

Screening and cognitive assessments included the MMSE, MoCA and the CERAD-NB. A

composite score for the CERAD battery was derived using a method previously described

by Chandler et al. (12). We used the method from this paper that excluded the MMSE from

the CERAD composite total score, and also provided age-, education-, and gender corrected

scores (12). The MoCA was administered on the same day as the clinical and

neuropsychological evaluation. In addition, the Dementia Severity Rating Scale (DSRS) was

administered to assess functional capacity. The DSRS is a reliable and simple 12-item

multiple-choice questionnaire that collects information from a knowledgeable informant on

impairment severity in twelve cognitive and functional domains (37). The MMSE, CERAD-

Roalf et al. Page 3

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

NB and DSRS were available during consensus diagnosis, but the MoCA was not. MCI

subtypes and dementias other than AD were not analyzed in this study. HC subjects were

also recruited and assessed identically to the patients with AD and MCI. Informed consent

for the use of all data was obtained from all persons, in accord with university institutional

review board–approved protocols. All participants completed the MMSE and MoCA. Most,

but not all, participants completed the CERAD-NB and DSRS. Most participants also

received the Geriatric Depression Scale (GDS) as a screening measure for mood

disturbances (38) (See Table 1).

2.2 Statistical analysis

Demographic characteristics were compared across diagnostic groups using Pearson χ2 or

one-way ANOVAs with post-hoc t-tests. The diagnosis accuracy for each measure (or

combination of measures) was calculated as the area under the receiver operating

characteristic (ROC) curve (AUC). The AUC measure represents the mean sensitivity value

for all possible values of specificity. A cut-off score for each measure that best differentiated

diagnostic groups was determined using the Youden Index (39), which maximizes the

tradeoffs between sensitivity and specificity. The classification accuracy (probability of

correct classification of subject with or without impairment at a given cut-off score) was

calculated based upon these cut-off scores (Table 2). Diagnostic accuracy of the MMSE,

MoCA and DSRS (or combination of measures) was compared to the CERAD-NB using

Chi-Square analysis. Classification accuracy of each measure was compared using the

Wilcoxon Signed Rank test. These statistical analyses were performed using SAS Program

software (v 9.1, SAS Institute, Cary, NC).

MoCA scores were equated to MMSE scores using the equipercentile equating method (40).

Equipercentile equating has been used to equate numerous standardized tests including

equating of the Telephone Interview for Cognitive Status (TICS) scores to MMSE scores in

older adults (41). This statistical method allows for the determination of comparable test

scores from two different measures on the basis of their corresponding percentile ranks. The

advantage of equipercentile equating method is that the equated scores always fall within the

range of possible scores, which is not always true when using traditional mean and linear

equating methods. However, this method can lead to an irregular distribution of scores, thus

a log-linear transformation (42) was used to smooth the raw scores of MoCA and MMSE

into a regular distribution. Equipercentile equating and log-linear smoothing was performed

using the ‘equate’ library in the R statistical package (43–46).

3. Results

Demographic characteristics along with significance values are presented in Table 1. The

diagnostic groups differed in age, education, and sex distribution, but not race or ethnicity.

The groups differed in GDS score [F(2, 536)=30.14, p<.01], with the AD group reporting

the highest score on this measure followed by the MCI group and the NC group. GDS mean

scores were, however, below five for all three groups and for the majority of individual

participants, suggesting that no subject suffered from clinically significant depression (Table

1). The AD group performed significantly poorer than the MCI and HC group on each task;

the MCI group performed significantly poorer on each cognitive task than the HC group

Roalf et al. Page 4

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

(Table 1). Across all participants age was weakly, but significantly, correlated with each

screening and cognitive measure: age and MMSE [r=−0.19, p<.001, n=587]; age and MoCA

[r=−0.20, p<.001, n=587]; age and CERAD-NB [r=−0.25, p<.001, n=534]; age and DSRS

[r=0.23, p<.001, n=579].

3.1 ROC analysis of standard screening measures: a comparison of CERAD-NB, MMSE and MoCA

The ROC curve analysis was used to evaluate the diagnostic accuracy of each measure

(CERAD-NB, MMSE and MoCA) to discriminate AD and MCI from each other and from

healthy cognitive aging. Graphic representations of the ROC curves are provided in Figure

1. Table 2A shows clinically relevant cut-offs for the CERAD-NB, MMSE and MoCA. The

diagnostic accuracies of the CERAD-NB, MMSE and MoCA were all excellent for HC vs.

AD, with AUCs > 0.99. Diagnostic accuracies were lower, but still very good in the

CERAD-NB (0.93), MMSE (0.85) and MoCA (0.89) for HC vs. MCI. The CERAD-NB

resulted in a significantly higher AUC than either the MoCA [χ2(1)=6.18, p<.05] or MMSE

[χ2(1)=11.40, p<.05], and the MoCA was nominally better at differentiating HC from MCI

than the MMSE [χ2(1)=3.14, p=.07]. Diagnostic accuracies of all three cognitive measures

were the lowest when differentiating AD from MCI, yet the AUCs were still good and

equally high for the three measures: CERAD-NB (0.85), MMSE (0.86) and MoCA (0.85).

3.2 Cut-off scores: CERAD, MMSE and MoCA

3.3 HC vs. AD—A cut-off score of 83 on the adjusted CERAD-NB, 28 on the MMSE, and 23 on the MoCA yielded the highest Youden Index for discriminating between HC and AD

(Table 2A). The classification accuracy of a score of 83 on CERAD-NB was nominally

better at classifying healthy and AD individuals than the optimal MoCA score (Z=1.71, p=.

08) and significantly better than the optimal MMSE score (Z=2.53, p=.01). The

classification accuracy of the MMSE and MoCA were comparable (Z=1.33, p=.18).

3.4 HC vs. MCI—A cut-off score of 89 on the CERAD-NB, 29 on the MMSE, and 25 on the MoCA yielded the highest Youden Index for the discriminating between HC and MCI

(Table 2A). The classification accuracy of the optimal CERAD-NB score was better at

classifying healthy and AD individuals than either the optimal MMSE (Z=5.74, p<.001) or

MoCA (Z=2.94, p<.01) score. Classification accuracy of the MoCA was superior to the

MMSE (Z=3.56, p<.001).

3.5 AD vs. MCI—A cut-off score of 69 on the CERAD-NB, 25 on the MMSE, and 19 on the MoCA yielded the highest Youden index for discriminating between AD and MCI

(Table 2A). The classification accuracy of the cut-off score was equally good for the

CERAD-NB, the MMSE, and the MoCA.

3.6 ROC analysis of informant based measure: the DSRS

Area under the ROC curve, the optimal cut-off score, sensitivity, specificity, and

classification accuracy for the DSRS is presented in Table 2B. The DSRS provided

comparably good discrimination between each group as the cognitive and screening

measures. The diagnostic accuracy of the DSRS for HC and AD, was equal to the CERAD-

Roalf et al. Page 5

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

NB, but better than the MMSE [χ2(1)=6.26, p<.05] or MoCA, [χ2 (1)=8.59, p<.01], albeit

these increases in AUC were very small (<0.5%). The diagnostic accuracy of the DSRS for

HC and MCI was superior to the MMSE [χ2 (1)=21.67, p<.0001] or MoCA [χ2 (1)=13.52,

p<.001]. The DSRS was equally discriminant as the CERAD-NB, MMSE and MoCA when

differentiating AD and MCI.

3.7 Cut-off scores: DSRS

A cut-off score of below 3 on the DSRS, yielded the highest Youden index for

discriminating between HC and AD, a score of 2 was best for discriminating HC and MCI,

and a score of 10 best differentiated AD from MCI. When discriminating between HC and

AD or HC and MCI the DSRS yielded higher classification accuracy than the MMSE [HC

vs AD: Z=3.89, p<.001; HC vs MCI: Z=5.90, p<.001] or MoCA [HC vs AD: Z=3.02, p<.01;

HC vs MCI: Z=3.56, p<.001]. In contrast, the DSRS was poorer than the MMSE [z=2.25,

p=.03], nominally poorer than the MoCA [Z=1.84, p=.07], but equally as accurate as the

CERAD-NB when discriminating MCI and AD (Table 2B).

3.8 ROC analysis of combining an informant-based measure with cognitive screening tasks

Combining the DSRS with the MMSE or MoCA significantly improved diagnostic accuracy

as compared to using either screening measure alone (Table 3). Diagnostic accuracy of the

MMSE improved by 13% when differentiating HC from MCI [χ2 (1)=32.53, p<.0001], and

5% when differentiating AD from MCI [χ2 (1)=7.90, p<.0005]. Using the combination of

optimal cut-off scores (a score below either cut-off score) of the MMSE and/or DSRS to

classify individuals resulted in an increase of 27% for HC and MCI and 12% for MCI and

AD. Diagnostic accuracy (AUC) of the MoCA was improved by 9% when differentiating

HC from MCI [χ2 (1)=25.12, p<.001], and 5% when differentiating AD from MCI [χ2

(1)=9.84, p<.001]. Using the combination of optimal cut-off scores of the MoCA and/or

DSRS to classify individuals resulted in an increase of 17% for HC and MCI and 13% for

MCI and AD. Changes in diagnostic accuracy were not assessed for HC vs. AD, as

diagnostic accuracy of each test alone was already excellent (>0.99).

3.9 Equating MMSE and MoCA scores

The plot of equipercentile equivalent scores on the MMSE and MoCA is presented in Figure

2. As an example, Figure 2 indicates that a score of 18 on the MoCA is equivalent to a score

of 24 on the MMSE and, both of these scores fall at approximately the 50th percentile within

a clinical sample with a wide range of cognitive impairment. Table 4 shows scores on the

MoCA and their respective equivalents on the MMSE. In general, lower MoCA scores are

equal to higher MMSE scores. For example, scores of 28–30 on the MoCA are equivalent to

the highest score of 30 on the MMSE.

4. Discussion

Overall, our findings in a relatively large, community-dwelling cohort evaluated at a

specialty memory clinic suggest that the individual screening measures (MMSE and MoCA)

typically used in clinical practice to aid in the diagnosis of AD or MCI offer reasonably high

Roalf et al. Page 6

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

classification and diagnostic accuracy when compared to a more detailed

neuropsychological evaluation (CERAD-NB). However, as a brief, stand-alone cognitive

screening measure, the MoCA appears to be more sensitive than the older, but more widely

used MMSE. Furthermore, using the optimal cut-off score, the classification accuracy of the

MoCA exceeded that of the MMSE when differentiating MCI from HC. This finding is

consistent with a recent study that indicates the MoCA is more sensitive than the MMSE at

detecting patients with mild cognitive impairment in Parkinson’s disease (PD) from those

patients with a diagnosis of PD alone. Additionally, using an informant-based questionnaire

of functional impairment (DSRS) resulted in comparable, or higher, overall diagnostic

accuracy for most comparisons. However, accurately differentiating MCI from AD proved

the most difficult regardless of the measure used. Finally, we provide a simple, yet reliable,

method for equating MoCA scores to traditional MMSE scores.

Our results confirm and extend prior findings on the diagnostic utility of cognitive tests in

AD and MCI. First, we find that an efficient but multi-dimensional neuropsychological

inventory, the CERAD-NB, is more accurate at distinguishing MCI or AD from healthy

individuals than brief screening measures, but not more accurate than the MMSE or MoCA

at differentiating AD from MCI. The diagnostic accuracy of the CERAD-NB (93%) when

differentiating MCI from the HC group in our study is very similar to a recent, multinational

study of the diagnostic accuracy of the total CERAD score (13). Better diagnostic accuracy

using the CERAD-NB is expected as it is more comprehensive, encompasses multiple

cognitive domains and is highly correlated with the MMSE and MoCA (12). Thus it can

serve as a reliable standard to which the MMSE and MoCA can be compared.

We extend upon previous findings (19, 20, 47, 48) that the MoCA is a superior screen than

the MMSE for detecting AD and MCI. Both measures easily identified AD compared to HC

and were comparable to the CERAD-NB. More striking was the accuracy of the MoCA at

differentiating MCI from HC as compared to the MMSE. These data suggest that the MoCA

may be more sensitive to early changes in cognitive ability as it includes more robust

measures of visuospatial and executive function (16). Diagnostic accuracy declined when

using these instruments to distinguish between AD and MCI. Although we found that the

MMSE and MoCA were comparably accurate in differentiating AD from MCI, and the

accuracy of both screens was similar to the CERAD-NB, this transition in clinical status

remains a challenge for psychometric instruments. In particular, difficulty in differentiating

AD from MCI is likely due to several factors including, but not limited to: 1) the

heterogeneity of the MCI diagnosis; 2) progression rates of 10–15% from MCI to AD per

year (5); 3) limited research that has focused on cross-sectional differentiation of MCI from

AD and healthy older adults (see (8)),and 4) the relative dearth of MCI-specific screening

measures. The importance of this last point cannot be understated as the real clinical value

of any test will be in its prediction of further decline, and its relationship with disease-

specific biomarkers that lead to significantly improved diagnostic accuracy.

We extend upon previous findings by assessing the diagnostic accuracy of the DSRS, an

informant-based measure of functional impairment. Notably, the DSRS provided better

classification accuracy than the MMSE or MOCA, and comparable accuracy to the CERAD-

NB, for AD and MCI as compared to HC. Yet, the diagnostic and classification accuracy of

Roalf et al. Page 7

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

the DSRS fell, like the other measures, when differentiating AD from MCI. The DSRS may

provide additional insight into the functional levels of disease severity or subtypes of MCI

especially in the absence of overt dementia. Given the capability of the DSRS to correctly

discriminate dementia from healthy cognitive aging, we found that combining this measure

with either the MMSE or MoCA increased diagnostic accuracy. This increase is of note as

the DSRS is a reliable tool for indicating the severity of AD throughout the course of the

disease (37). Furthermore, the DSRS, unlike many cognitive measures, changes at a steady

predictable rate (35) making it an ideal measure to monitor changes in the disease course

and may provide a useful metric to measure response to treatment. Previous studies have

found that combining cognitive and informant-based screens (e.g. Informant Questionnaire

on Cognitive Decline in the Elderly) have improved diagnostic accuracy (30, 31) and

suggest that measures of instrumental activities of daily living be incorporated into the

diagnostic criteria for MCI (49). To our knowledge, this is the first study to combine the

DSRS with other screening measures to differentiate AD from MCI from HC. Given these

results, further studies should consider including the DSRS during diagnostic assessment.

Finally, we used equipercentile equating to develop a conversion score between the MoCA

and MMSE. Here, we provide a table of conversion scores that will enable the widely

recognized cut-off scores on the MMSE to be reliably compared with scores on the MoCA.

While the MMSE is the most commonly used clinical screening for monitoring acute or

declining cognitive impairment, it has several limitations such as marked ceiling effects in

younger, well-educated individuals and inconsistent performance in differentiating MCI

from healthy older adults as many individuals with MCI score above the recommended

MMSE cut-off (score of 26) for impairment (for review see (8)). Thus the MMSE has

limited utility in detecting subtle changes in cognition that may signal pending impairment

(50, 51). The MoCA does not overcome all of the shortcomings of the MMSE, but recent

data, including ours, indicate that the MoCA is a superior screening measure for classifying

MCI. Equating scores on these two measures provides a straightforward way of comparing

the MoCA to the MMSE, thus allowing for continuity of cognitive tracking in the clinic and

comparability of data in longitudinal studies of MCI or dementing illness.

Our use of a large, well-characterized community-dwelling cohort provides a useful

perspective for determining the diagnostic accuracy of standardized global screening

measures in AD and MCI. However, this study is not without limitations. The differences in

gender distribution across the groups present challenges in interpretation of the current

results. As expected, there were more women diagnosed with AD than men (52), however,

the gender distribution within the MCI group was essentially a 50/50 split. It is possible the

men are overrepresented in the group due to other factors. A comparison of gender-specific

diagnostic accuracy was beyond the scope of this study, but should be examined in future

investigations. The impact of co-morbid clinical factors, such as depression, were not

considered in this study but could impact the utility of some screening measures, however

the MoCA appears robust to depression symptoms within healthy cohorts (53) and in the

current sample self-reported rates of depression were low. The sample was well-educated,

particularly the healthy cognition group, which makes generalization to less well educated

cohorts more challenging as education does impact scores MMSE (8) and MoCA (53)

performance. Overall, the sample size of this study was adequate, but more precise cut-off

Roalf et al. Page 8

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

values for each diagnostic test could be achieved with an increase in the number of MCI

patients. An increase in number and subtypes of MCI, which was not considered in this

study, would provide more specificity and may lead to improved diagnostic accuracy and

utility. Since the MoCA shows a wider range of performances in AD and MCI than the

MMSE, it seems likely that the MoCA can be used to determine differing levels of severity

or subtypes of MCI. This is particularly important as the diagnosis of MCI or AD increases

with age, thus performance on the MMSE or MoCA may systematically differ in an 80-year-

old MCI patient as compared to a 60-year-old MCI patient. Thus, future studies should

consider age- and domain-specific performance cut-offs. Finally, the aim of the current

study was to compare individuals with a known diagnosis of AD or MCI to a healthy

comparison group in specialty memory and aging clinic. As a result, generalizing to other

clinical settings should be done with caution. Given that the clinicians were not blinded to

the MMSE, CERAD-NB and DSRS during consensus diagnosis it is possible that diagnostic

decisions were influenced by the availability of these measures. However, there was no

systematic rubric employed at consensus and the data do not suggest that one or more of

these measures were used preferentially to achieve diagnosis. Given the paucity of screening

tools that are specific to differentiating MCI from AD and normal cognitive aging, we

believe that the results of this study provide practical results about well-utilized standardized

metrics and their diagnostic effectiveness.

In conclusion, the current results support recent data that the MoCA is superior to the

MMSE as a global assessment tool and we provide a reliable way to compare the two

screening measures. Furthermore, we find that when these traditional screening measures are

used in the diagnosis of AD and MCI with complementary informant-based ratings of

functional impairment, overall diagnostic accuracy improves.

Acknowledgments

The authors express appreciation to the research participants and staff of the Penn Memory Center/Clinical Core of the University of Pennsylvania Alzheimer’s Disease Center.

Funding Sources: This work was supported by NIA AG10124 and the Marian S. Ware Alzheimer’s Program/ National Philanthropic Trust, and NIH T32 MH019112.

References

1. Lobo A, Launer LJ, Fratiglioni L, Andersen K, Di Carlo A, Breteler MM, et al. Prevalence of dementia and major subtypes in Europe: A collaborative study of population-based cohorts. Neurologic Diseases in the Elderly Research Group. Neurology. 2000; 54(11 Suppl 5):S4–9. [PubMed: 10854354]

2. Comas-Herrera A, Northey S, Wittenberg R, Knapp M, Bhattacharyya S, Burns A. Future costs of dementia-related long-term care: exploring future scenarios. Int Psychogeriatr. 2011; 23(1):20–30. [PubMed: 20374684]

3. Ferri CP, Prince M, Brayne C, Brodaty H, Fratiglioni L, Ganguli M, et al. Global prevalence of dementia: a Delphi consensus study. Lancet. 2005; 366(9503):2112–7. [PubMed: 16360788]

4. Wimo A, Winblad B, Aguero-Torres H, von Strauss E. The magnitude of dementia occurrence in the world. Alzheimer Dis Assoc Disord. 2003; 17(2):63–7. [PubMed: 12794381]

5. Petersen RC, Smith GE, Waring SC, Ivnik RJ, Tangalos EG, Kokmen E. Mild cognitive impairment: clinical characterization and outcome. Arch Neurol. 1999; 56(3):303–8. [PubMed: 10190820]

Roalf et al. Page 9

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

6. Petersen RC. Mild cognitive impairment as a diagnostic entity. J Intern Med. 2004; 256(3):183–94. [PubMed: 15324362]

7. Winblad B, Palmer K, Kivipelto M, Jelic V, Fratiglioni L, Wahlund LO, et al. Mild cognitive impairment--beyond controversies, towards a consensus: report of the International Working Group on Mild Cognitive Impairment. J Intern Med. 2004; 256(3):240–6. [PubMed: 15324367]

8. Lonie JA, Tierney KM, Ebmeier KP. Screening for mild cognitive impairment: a systematic review. Int J Geriatr Psychiatry. 2009; 24(9):902–15. [PubMed: 19226524]

9. Morris JC, Heyman A, Mohs RC, Hughes JP, van Belle G, Fillenbaum G, et al. The Consortium to Establish a Registry for Alzheimer’s Disease (CERAD). Part I. Clinical and neuropsychological assessment of Alzheimer’s disease. Neurology. 1989; 39(9):1159–65. [PubMed: 2771064]

10. Welsh-Bohmer KA, Mohs RC. Neuropsychological assessment of Alzheimer’s disease. Neurology. 1997; 49(3 Suppl 3):S11–3. [PubMed: 9310507]

11. Collie A, Shafiq-Antonacci R, Maruff P, Tyler P, Currie J. Norms and the effects of demographic variables on a neuropsychological battery for use in healthy ageing Australian populations. Aust N Z J Psychiatry. 1999; 33(4):568–75. [PubMed: 10483853]

12. Chandler MJ, Lacritz LH, Hynan LS, Barnard HD, Allen G, Deschner M, et al. A total score for the CERAD neuropsychological battery. Neurology. 2005; 65(1):102–6. [PubMed: 16009893]

13. Paajanen T, Hanninen T, Tunnard C, Mecocci P, Sobow T, Tsolaki M, et al. CERAD neuropsychological battery total score in multinational mild cognitive impairment and control populations: the AddNeuroMed study. J Alzheimers Dis. 2010; 22(4):1089–97. [PubMed: 20930314]

14. Seo EH, Lee DY, Lee JH, Choo IH, Kim JW, Kim SG, et al. Total scores of the CERAD neuropsychological assessment battery: validation for mild cognitive impairment and dementia patients with diverse etiologies. Am J Geriatr Psychiatry. 2010; 18(9):801–9. [PubMed: 20220577]

15. Folstein MF, Folstein SE, McHugh PR. “Mini-mental state”. A practical method for grading the cognitive state of patients for the clinician. J Psychiatr Res. 1975; 12(3):189–98. [PubMed: 1202204]

16. Nasreddine ZS, Phillips NA, Bedirian V, Charbonneau S, Whitehead V, Collin I, et al. The Montreal Cognitive Assessment, MoCA: a brief screening tool for mild cognitive impairment. J Am Geriatr Soc. 2005; 53(4):695–9. [PubMed: 15817019]

17. Smith T, Gildeh N, Holmes C. The Montreal Cognitive Assessment: validity and utility in a memory clinic setting. Can J Psychiatry. 2007; 52(5):329–32. [PubMed: 17542384]

18. Damian AM, Jacobson SA, Hentz JG, Belden CM, Shill HA, Sabbagh MN, et al. The Montreal Cognitive Assessment and the mini-mental state examination as screening instruments for cognitive impairment: item analyses and threshold scores. Dement Geriatr Cogn Disord. 2011; 31(2):126–31. [PubMed: 21282950]

19. Freitas S, Simoes MR, Alves L, Santana I. Montreal Cognitive Assessment: Validation Study for Mild Cognitive Impairment and Alzheimer Disease. Alzheimer Dis Assoc Disord. 2011

20. Larner AJ. Screening utility of the Montreal Cognitive Assessment (MoCA): in place of - or as well as - the MMSE? Int Psychogeriatr. 2012; 24(3):391–6. [PubMed: 22014176]

21. Lee JY, Dong Woo L, Cho SJ, Na DL, Hong Jin J, Kim SK, et al. Brief screening for mild cognitive impairment in elderly outpatient clinic: validation of the Korean version of the Montreal Cognitive Assessment. J Geriatr Psychiatry Neurol. 2008; 21(2):104–10. [PubMed: 18474719]

22. Dalrymple-Alford JC, MacAskill MR, Nakas CT, Livingston L, Graham C, Crucian GP, et al. The MoCA: well-suited screen for cognitive impairment in Parkinson disease. Neurology. 2010; 75(19):1717–25. [PubMed: 21060094]

23. Dong Y, Lee WY, Basri NA, Collinson SL, Merchant RA, Venketasubramanian N, et al. The Montreal Cognitive Assessment is superior to the Mini-Mental State Examination in detecting patients at higher risk of dementia. Int Psychogeriatr. 2012:1–7.

24. Ihara M, Okamoto Y, Takahashi R. Suitability of the Montreal Cognitive Assessment versus the Mini-Mental State Examination in Detecting Vascular Cognitive Impairment. J Stroke Cerebrovasc Dis. 2012

Roalf et al. Page 10

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

25. MacKenzie G, Gould L, Ireland S, LeBlanc K, Sahlas D. Detecting cognitive impairment in clients with mild stroke or transient ischemic attack attending a stroke prevention clinic. Can J Neurosci Nurs. 2011; 33(1):47–50. [PubMed: 21560886]

26. Association AP. Diagnostic and Statistical Manual of Mental Disorders. 4. Washington, DC: American Psychiatric Association; 2000. text revision (DSM-IV-TR) ed

27. McKhann GM, Knopman DS, Chertkow H, Hyman BT, Jack CR Jr, Kawas CH, et al. The diagnosis of dementia due to Alzheimer’s disease: recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011; 7(3):263–9. [PubMed: 21514250]

28. Jack CR Jr, Albert MS, Knopman DS, McKhann GM, Sperling RA, Carrillo MC, et al. Introduction to the recommendations from the National Institute on Aging-Alzheimer’s Association workgroups on diagnostic guidelines for Alzheimer’s disease. Alzheimers Dement. 2011; 7(3):257–62. [PubMed: 21514247]

29. Jorm AF. The Informant Questionnaire on cognitive decline in the elderly (IQCODE): a review. Int Psychogeriatr. 2004; 16(3):275–93. [PubMed: 15559753]

30. Flicker L, Logiudice D, Carlin JB, Ames D. The predictive value of dementia screening instruments in clinical populations. Int J Geriatr Psychiatry. 1997; 12(2):203–9. [PubMed: 9097213]

31. Hancock P, Larner AJ. Diagnostic utility of the Informant Questionnaire on Cognitive Decline in the Elderly (IQCODE) and its combination with the Addenbrooke’s Cognitive Examination- Revised (ACE-R) in a memory clinic-based population. Int Psychogeriatr. 2009; 21(3):526–30. [PubMed: 19327203]

32. Carr DB, Gray S, Baty J, Morris JC. The value of informant versus individual’s complaints of memory impairment in early dementia. Neurology. 2000; 55(11):1724–6. [PubMed: 11113230]

33. Berg L. Clinical Dementia Rating (CDR). Psychopharmacol Bull. 1988; 24(4):637–9. [PubMed: 3249765]

34. Galasko D, Bennett D, Sano M, Ernesto C, Thomas R, Grundman M, et al. An inventory to assess activities of daily living for clinical trials in Alzheimer’s disease. The Alzheimer’s Disease Cooperative Study. Alzheimer Dis Assoc Disord. 1997; 11 (Suppl 2):S33–9. [PubMed: 9236950]

35. Xie SX, Ewbank DC, Chittams J, Karlawish JH, Arnold SE, Clark CM. Rate of decline in Alzheimer disease measured by a Dementia Severity Rating Scale. Alzheimer Dis Assoc Disord. 2009; 23(3):268–74. [PubMed: 19812470]

36. Morris JC, Mohs RC, Rogers H, Fillenbaum G, Heyman A. Consortium to establish a registry for Alzheimer’s disease (CERAD) clinical and neuropsychological assessment of Alzheimer’s disease. Psychopharmacol Bull. 1988; 24(4):641–52. [PubMed: 3249766]

37. Clark CM, Ewbank DC. Performance of the dementia severity rating scale: a caregiver questionnaire for rating severity in Alzheimer disease. Alzheimer Dis Assoc Disord. 1996; 10(1): 31–9. [PubMed: 8919494]

38. Sheikh, J.; Yesavage, J. Clinical Gerontology: A Guide to Assessment and Intervention. NY: The Haworth Press; 1986. Geriatric Depression Scale (GDS): Recent evidence and development of a shorter version; p. 165-73.

39. Youden WJ. Index for rating diagnostic tests. Cancer. 1950; 3(1):32–5. [PubMed: 15405679]

40. Kolen, MJ.; Brennan, RL. Test equating methods and practice. New York: Springer; 2005.

41. Fong TG, Fearing MA, Jones RN, Shi P, Marcantonio ER, Rudolph JL, et al. Telephone interview for cognitive status: Creating a crosswalk with the Mini-Mental State Examination. Alzheimers Dement. 2009; 5(6):492–7. [PubMed: 19647495]

42. Moses, T.; vonDavier, AA. An SAS macro for loglinear smoothing: applications and implications. Montreal: American Educational Research Association; 2005.

43. Braun, HI.; Holland, PW. Observed-score test equating: A mathematical analysis of some ETS equating procedures. In: Holland, PW.; Rubin, DB., editors. Test Equating. New York: Academic; 1982. p. 9-49.

44. Livingston SA, Kim S. The circle-arc method for equating in small samples. Journal of Educational Measurement. 2009; 46:330–43.

45. Kolen, MJ.; Brennan, RL. Test Equating, Scaling, and Linking. 2. New York: Springer; 2004.

Roalf et al. Page 11

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

46. Lord F. The standard error of equipercentile equating. Journal of Educational Statistics. 1982; 7:165–74.

47. Hancock P, Larner AJ. Test Your Memory test: diagnostic utility in a memory clinic population. Int J Geriatr Psychiatry. 2011; 26(9):976–80. [PubMed: 21845600]

48. Luis CA, Keegan AP, Mullan M. Cross validation of the Montreal Cognitive Assessment in community dwelling older adults residing in the Southeastern US. Int J Geriatr Psychiatry. 2009; 24(2):197–201. [PubMed: 18850670]

49. Brown PJ, Devanand DP, Liu X, Caccappolo E. Functional impairment in elderly patients with mild cognitive impairment and mild Alzheimer disease. Arch Gen Psychiatry. 2011; 68(6):617–26. [PubMed: 21646578]

50. Ravaglia G, Forti P, Maioli F, Servadei L, Martelli M, Brunetti N, et al. Screening for mild cognitive impairment in elderly ambulatory patients with cognitive complaints. Aging Clin Exp Res. 2005; 17(5):374–9. [PubMed: 16392411]

51. Slavin MJ, Sandstrom CK, Tran TT, Doraiswamy PM, Petrella JR. Hippocampal volume and the Mini-Mental State Examination in the diagnosis of amnestic mild cognitive impairment. AJR Am J Roentgenol. 2007; 188(5):1404–10. [PubMed: 17449789]

52. Vina J, Lloret A. Why women have more Alzheimer’s disease than men: gender and mitochondrial toxicity of amyloid-beta peptide. J Alzheimers Dis. 2010; 20 (Suppl 2):S527–33. [PubMed: 20442496]

53. Freitas S, Simoes MR, Alves L, Santana I. Montreal Cognitive Assessment: influence of sociodemographic and health variables. Arch Clin Neuropsychol. 2012; 27(2):165–75. [PubMed: 22277128]

Roalf et al. Page 12

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

Figure 1. ROC curve analysis of the MMSE, MoCA, CERAD-NB, and DSRS. HC=Healthy Controls;

AD=Alzheimer’s disease; MCI= Mild Cognitive Impairment

Roalf et al. Page 13

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

Figure 2. Corresponding raw scores and raw percentile ranks for the MMSE and MoCA. As an

example, a score of 18 on the MoCA is equivalent to a score of 24 on the MMSE.

Roalf et al. Page 14

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

Roalf et al. Page 15

Table 1

Demographic characteristics and performance for Alzheimer’s disease (AD), Mild Cognitive Impairment

(MCI) and healthy comparison (HC) participants.

Characteristic AD (n=321) MCI (n=126) HC (n=140)

Age, years 75.69 (8.21)d,e 72.29 (8.12)e 71.19 (9.20)

Age Range, years 53–93 52–88 50–88

Sex, Men/Women 122/199 64/62f 46/94

Race, (AA/As/C/M/O) 45/3/232/37/4 12/5/93/15/0 22/2/107/9/0

Education, years 13.33 (4.15)d,e 14.86 (4.20)e 15.91 (3.03)

GDSa 3.07 (2.92)d,e 2.69 (2.68)e 0.99 (1.64)

Ethnicity (% Latino) 14% 14% 11%

MMSE 19.04 (5.88)d,e 26.00 (3.46)e 29.30 (0.87)

MoCA 13.04 (6.05)d,e 20.94 (4.50)e 26.83 (2.64)

Neuropsychological Score (CERAD)b 44.89 (13.71)d,e 64.21 (11.66)e 86.87 (9.02)

Adjusted CERADc 58.29 (13.72) 76.08 (10.93) 97.28 (8.43)

DSRS 16.01 (8.50)d,e 6.20 (4.22)e 0.28 (0.69)

AA= African American, As= Asian, C= Caucasian, M=Multiple Races reported, O=Other (e.g. Native American); GDS= Geriatric Depression Scale; MMSE= Mini Mental Status Examination; MoCA= Montreal Cognitive Assessment; CERAD= Consortium to Establish a Registry for Alzheimer’s Disease; DSRS= Dementia Severity Rating Scale.

a AD: n=283; MCI: n=119, HC: n=137;

b AD: n=275; MCI: n=122, HC: n=138;

c adjusted for age, education and gender based on Chandler et al., 2005; dAD: n=318; MCI: n=124, HC: n=137

d p<.01 as compared to MCI;

e p<.01 as compared to HC;

f p<.01, larger proportion of men as compared to AD and HC

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

Roalf et al. Page 16

Table 2

Diagnostic parameters for MMSE, MoCA, CERAD-NB and DSRS across AD, MCI and healthy older groups.

A.

Parameter HC and AD HC and MCI MCI and AD

CERAD total scorea

AUC (+/− 95%CI) 0.99 (0.98–1.00) 0.93c,e (0.90–0.96) 0.84 (0.82–0.89)

Cut-offb 83 89 69

Sensitivity/Specificity 96/97 87/90 74/73

Youden Index 0.94 0.77 0.47

Classification Accuracy 0.97f 0.90f,g 0.76

MMSE total score

AUC(+/− 95%CI) 0.99 (0.98–0.99) 0.85 (0.80–0.89) 0.86 (0.82–0.90)

Cut-off 28 29 25

Sensitivity/Specificity 96/97 82/73 77/83

Youden Index 0.93 0.55 0.60

Classification Accuracy 0.93 0.70 0.81h

MoCA total score

AUC(+/− 95%CI) 0.99 (0.97–0.99) 0.89d (0.85–0.92) 0.85 (0.81–0.89)

Cut-off 23 25 19

Sensitivity/Specificity 94/96 84/79 77/80

Youden Index 0.91 0.63 0.57

Classification Accuracy 0.95 0.80f 0.79i

B.

DSRS total score

AUC(+/− 95%CI) 0.99c,e (0.99–1.00) 0.96c,e (0.94–0.98) 0.86 (0.82–0.90)

Cut-off 3 2 10

Sensitivity/Specificity 99/98 89/93 74/78

Youden Index 0.97 0.82 0.52

Classification Accuracy 0.98f,g 0.91f,g 0.75

AUC = area under the curve; MMSE = Mini-Mental State Examination; MoCA = Montreal Cognitive Assessment; HC= healthy older controls; MCI = mild cognitive impairment; AD= Alzheimer’s disease

a age-, education-, gender-adjusted score (Chandler et al., 2005)

b Cut-off scores are greater than or equal to

c AUC was better than MMSE AUC (p<.001)

d AUC was marginally better than MMSE (p=.07)

e AUC was better than MoCA AUC (p<.005)

f Classification accuracy better than MMSE

g Classification accuracy better than MoCA

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

Roalf et al. Page 17

h Classification accuracy better than DSRS

i Classification accuracy marginally better than DSRS (p=.07).

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

Roalf et al. Page 18

Table 3

Comparison of AUCs for MMSE or MoCA alone and when combined with DSRS for differentiating Healthy

from MCI and MCI from AD.

Parameter HC and MCI MCI and AD

MMSE total score

AUC 0.84 (0.80–0.89) 0.86 (0.83–0.90)

Classification Accuracy 0.70 0.81

MMSE + DSRS

AUC 0.97a (0.95–0.99) 0.91a (0.88–0.94)

Δ AUC +0.12 +0.05

Classification Accuracy 0.97c 0.93c

Δ Classification Accuracy +0.27 +0.12

MoCA

AUC 0.88 (0.84–0.92) 0.85 (0.82–0.90)

Classification Accuracy 0.80 0.79

MoCA + DSRS

AUC 0.97b (0.95–0.99) 0.90b (0.87–0.93)

Δ AUC +0.09 +0.05

Classification Accuracy 0.97c 0.92c

Δ Classification Accuracy +0.17 +0.13

AUC = area under the curve; Δ AUC = change in AUC; MMSE = Mini-Mental State Examination; MoCA = Montreal Cognitive Assessment; HC= healthy older controls; MCI = mild cognitive impairment; AD= Alzheimer’s disease

a Score was better than MMSE AUC (p<.0001);

b Score was better than MoCA AUC (P<.001);

c Classification accuracy higher than single measure

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.

A u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t A

u th

o r M

a n u scrip

t

Roalf et al. Page 19

Table 4

Conversion table for MMSE and MoCA screening measures based upon equipercentile equating in 321 AD,

126 MCI and 140 HC. Equivalent scores were derived from equipercentile equating with log-linear

smoothing.

Raw MoCA Score Equivalent MMSE Score

0 3

1 6

2 8

3 9

4 10

5 11

6 12

7 13

8 14

9 15

10 16

11 17

12 19

13 20

14 21

15 22

16 22

17 23

18 24

19 25

20 26

21 26

22 27

23 28

24 28

25 29

26 29

27 29

28 30

29 30

30 30

Alzheimers Dement. Author manuscript; available in PMC 2014 September 01.