TEST CRITIQUE ASSIGNMENT 1
Test Critique Assignment
Comprehensive Executive Function Inventory
Cara Soto
School of Behavioral Sciences in Mental Health Counseling, Liberty University
Author Note
Cara Soto
I have no known conflict of interest to disclose.
Correspondence concerning this article should be addressed to Cara Soto
Email: clsoto@liberty.edu
TEST CRITIQUE ASSIGNMENT 2
Abstract
This paper is to critique and explore the overall performance and psychometric properties of the
Comprehensive Executive Function Inventory (CEFI) produced by Jack A. Naglieri, Ph.D. and
Sam Goldstein, Ph.D. This test is a standardized behavior rating scale and has sound
psychometric properties and is able to measure the emotional functioning of children ages 5-18.
Emotional functioning is important part of daily functioning as it involves reasoning and
problem solving. This test is a multiple rater test involving teacher, parent and child (ages 12-18
can self-report). It has 100 items that take approximately 15 minutes to complete and can be
administered individually or as a group. It is also computer friendly and easy to use, it is scored
by software or by the MHS Online Assessment Center. The option of paper and pencil test can be
administered which is then hand scored.
Keywords: Comprehensive Executive Function Inventory, assessment, Executive
Function
TEST CRITIQUE ASSIGNMENT 3
Comprehensive Executive Function Inventory
The name of the test is the Comprehensive Executive Function Inventory (CEFI). It was
developed by two authors Jack A. Naglieri, Ph.D. and Sam Goldstein, Ph.D. it was published in
2013. On the web site MHS Beyond Assessments (https://storefront.mhs.com) it is available for
purchase. The CEFI kit costs $385.00, which includes manual and software scoring, along with
25 paper Self-Report/Parent/Teacher forms. Forms can also be purchased in individual packs or
in multiples. An online version of the test is assessable with an access link to the client who can
then fill out the inventory, submit and the results are sent to the counselor. The CEFI manual can
be bought separately for $109.00. Continuing education credits can be earned by studying the
manual (3 CE) and taking the quiz, a $45 cost, or watching an online class and taking a short
quiz and earning 1.5 CE for the price of $25.
The Comprehensive Executive Function Inventory (CEFI) was operationalized in 2011
(Goldstein et. al 2014). It was developed for children and adolescents ages 5-18. This test is a
standardized behavior rating scale. Executive functioning skills are important for reasoning,
adaptive behavior and problem solving. Parents and teachers can take the test for the child or
adolescent. This can also be used as an objective test in which a 12-year-old to 18-year-old can
take the test for themselves. Furthermore, this test can be compared three ways with the parent,
teacher and self-administration being scored to each other. Children under the age of 12 are not
allowed to self-report.
This test can be used for a variety of reasons in one study it was used to measure the
agreement between teachers, student, and parents whose student/child has ADHD (Krieger et. al
2019). In another study conducted by Ali et. al (2019), they used the test to assess children with
TEST CRITIQUE ASSIGNMENT 4
different degrees of executive functioning (EF) which include autism spectrum disorder along
with Fetal Alcohol Spectrum Disorder.
There are several other EF assessments available to use with children. The Behavior
Rating Inventory of Executive Function (BRIEF) also measures EF based on parent and teacher
rating of child. Another assessment is the Delis–Kaplan Executive Function System (D-KEFS)
which can only be administered by a person with a Ph.D. The Minnesota Executive Function
Scale (MEFS) can be given to children as young as 24 months old and to those in adulthood
(Reflection Sciences, 2016).
In the 1840's an unfortunate railroad accident left Phineas Gage with a damaged
prefrontal cortex. His behaviors and EF which is processed in the prefrontal cortex, became a
curious case to study. By the 1950's psychologists and neuroscientists became more interested in
the study of the functions of the prefrontal cortex. From the 1970's-2000's EF was hard to
conceptualize with over 30 constructs. In 2013 Naglieri and Goldstein narrowed it down to 9
constructs and hypothesized that EF involved decision making and planning actions (Goldstein
et. al 2014). Goldstein et. al (2014) also suggest that EF involves how individuals go about
acquiring knowledge and how well they can solve problems. Determining the level of executive
functioning is necessary because it contributes to the success of academics, positive behaviors,
health, relationships, and success at work (Center on the Developing Child, 2012).
In the textbook by Sheperis et. al (2020) they point out that these types of formal
assessments (i.e., CEFI, BRIEF, D-KEFS and others) are in line with diagnosing in the DSM-5.
These formal assessments help in diagnosing and forming a treatment plan. With the passing of
the Individuals with Disabilities Education Improvement Act (IDEA) of 2004 it became more
important for children to be diagnosed early in life so treatment plans and adjustments in the
TEST CRITIQUE ASSIGNMENT 5
school system could accommodate the child with learning disabilities. Some of the assumptions
of an assessment as mentioned by Sheperis et. al (2020) include thoughts that tests are biased;
they are unfair and discriminate against diverse groups. To counter this argument the CEFI was
normed on the U.S. censes demographics according to race, SES, male and female, religion, and
parents’ educational background (Naglieri & Goldstein, 2017). However, would this test be valid
in another country besides the U.S.? That is a consideration for further research.
TEST DESCRIPTION
The CEFI was designed for children ages 5-18 years old. It provides a rating of
behavioral and executive functioning (EF) skills. Deficiency in EF interfere with problem
solving and reasoning. This test takes 15 minutes to administer and can be reported by a parent,
teacher and/or self-report (ages 12-18). Naglieri & Goldstein have condensed the 30 EF abilities
down to 9 EF abilities.
The test has 9 scales, the first EF skill is attention and measures how well a child can
avoid distractions, concentrate, and retain attention (12 items). Scores from the section on
emotional control allows the parent or teacher to see how well the child can manage emotions
(9items). Flexibility sees how well a child can adjust to new situations and includes problem
solving abilities (7 items). To gauge how well a child can control impulses and behavior,
inhibitory control is measured (10 items). Initiation score will measure if a child can initiate a
project or task without being asked (10 items). Organization indicates how well a child manages
multiple tasks, personal effects, and school/work (10 items). The seventh EF is planning, how
well a child can develop and implement strategies to accomplish tasks (11 items), also self-
monitoring is an EF that describes how a young person measures their own behavior or
performance (10 items). And finally working memory reflects how well a child can remember
TEST CRITIQUE ASSIGNMENT 6
information of steps and how to do things (11 items) (Naglieri & Goldstein, 2017, Fenwick &
McCrimmon, 2015).
Criterion related validity was conducted from the general population, a sample of
children from clinical populations as well as correlation between the CEFI and other related
measures. Results found that children with Autism Spectrum disorder, ADHD, mood disorders
and learning disorders had significantly lower scores then the general population. Comparison
between BREIF and CEFI indicated a strong correlation (.64 to .85). However, comparison with
the other assessments such as Woodcock–Johnson Tests of Achievement (WJ-III) and Cognitive
Assessment System (CAS) indicated significant but small correlation which can be assumed that
the CEFI can score EF impairment but yield low detection scores for cognitive ability and
achievement, which indicates that the CEFI measures unique aspects of functioning not related to
the other tests measuring cognitive abilities and achievements (Fenwick & McCrimmon, 2015).
CEFI is a unidimensional construct of 100 questions based on the 6-point Likert scale.
Theoretical and empirical evidence are centered around one test conducted in 2011 on a sample
of 3,500 English speaking respondents. There were 1,400 parents, 1,400 teachers and 700
children chosen to represent the population of the United States. The test sample represented the
population based on the U.S. censes which included, race/ethnicity, religion, parents educational
background and gender (Naglieri & Goldstein, 2017).
The purpose of the CEFI is to measure many of the EF abilities in children ages 5-18. It
is able to rate the strengths and weaknesses of an individuals EF. It can be used to diagnose,
evaluate, and used for the intervention or treatment process of the child (Primus et. al 2014).
Once again, the CEFI is 100 question 6-point Likert scale test and can be taken in approximately
15 minutes. The CEFI contains nine subscales which measure attention, emotional regulation,
TEST CRITIQUE ASSIGNMENT 7
flexibility, inhibitory control, initiation, organization, planning self-motivation, and working
memory (Naglieri & Goldstein, 2017).
The administration procedures for the CEFI are done in a formal but untimed manner. It
can be administered individually or in a group setting. The teacher and parent will fill out the
assessment based on the observations of the child. If the child is 12-18 years of age, they can
self-test and it will be compared to the results of the teacher and parents (Naglieri & Goldstein,
2017). This test is considered a level B test which must be administered by a professional with an
educational degree of a Masters or higher. No special conditions are required. It may be taken
with pencil and paper test or through a link on the computer that the counselor can send to the
client. The paper test is scored by the assessor or if taken through the computer the software will
score electronically.
TECHNICAL EVALUATION
In 2011, the standardization of the CEFI began in January and ended in December
(Fenwick & McCrimmon, 2015). The sampling process consisted of 3,500 English speaking
respondents. There were 1,400 parents, 1,400 teachers and 700 children chosen to represent the
population of the United States. The test sample represented the population based on the U.S.
censes which included, race/ethnicity, religion, parents educational background and gender. At
each year of age (Parent and Teacher 5–18 years, Self-Report 12–18 years), 50 males and 50
females were collected for each form, resulting in 1,400 rated youth (age M=11.5, SD=4.0) for
the parent and teacher normative samples and 700 youth for the self-report (age M=15.0,
SD=2.0) (Naglieri & Goldstein, 2017). According to the Standards of Educational and
Psychological Testing (2014) there is no set standard on how many people or what the
TEST CRITIQUE ASSIGNMENT 8
composition of the sample may be for a test. However, the sample must be thoroughly detailed
and explained in the methods section of the paper.
The CEFI provides standard scores ( = 100; = 15), and high scores are associated M SD
with good executive functioning. The clinical cut-off point is a standard score below 90. Internal
consistency reliability was demonstrated through strong Cronbach's alpha values for the Full
Scale and the nine CEFI Scales.
The reliability (Cronbach's alpha) of the full-scale score in parent, teacher and self-report
is very good: α = .98 parent (CEFI Scales median=.89) and α = .99 teacher (CEFI Scales
median=.93) and α = .97 self-report (CEFI Scales median=.80). Test-retest reliability studies also
yielded excellent results. Parent Full Scale r= .91 (CEFI Scales median=.87) Teacher Full Scale
r= .90 (CEFI Scales median=.90), Self-Report Full Scale r= .77 (CEFI Scales median=.79).
For the content validity of the CEFI a comprehensive review of current EF literature,
research and theory was examined. Also, the authors research experience as well as their clinical
work was used to enhance this tests validity. Fenwick and McCrimmon (2015) convey that in the
Technical Manual for the CEFI it indicates exploratory factor analysis was utilized to determine
the dimensionality of the CEFI. For criterion related validity a clinical population was included
who had an EF deficit. specifically, Attention-Deficit/Hyperactivity Disorder (ADHD), Autism
Spectrum Disorder (ASD), or Mood Disorders were compared. Moderate to large size effect was
found Cohen’s d = −.62 to −1.59, all p <.001 (Climie et. al 2014).
PRACTICAL EVALUATION
There were no customer reviews available for the CEFI on Amazon.com, eBay.com or
any of the purchasing websites that were examined. According to Climie et. al (2014), the
instructions for the CEFI are clearly stated and written at about a seventh-grade level. The 100-
TEST CRITIQUE ASSIGNMENT 9
items which are consistent across the three response forms are written about a fifth-grade level
reading.
A 15-minute time frame is needed to complete the CEFI. The instruction manual is easy
to navigate, and the instructions are easy to follow, there is a table of contents specifically for the
tables and charts that are included in the manual. Each of these tables give instructions on how to
hand score the different forms (Climie et. al 2014). There is also a software to score the tests and
the MHS Online Assessment Center which provides a more accurate and more time efficient way
of scoring (Fenwick & McCrimmon, 2015).
SUMMARY EVALUATION & CRITIQUE
One of the strengths that the CEFI has is the ease of using it. Form instructions are easy
to read and concise. The computer software is user friendly. One of the wonderful advantages of
the assessment it that after the scoring is complete the computer will generate relevant
interventions for the child. Another great strength is the reliability and validity of the test indicate
that it is psychometrically sound. Also, the test has been administered and tested with the
Canadian population and has been found to be as effective in measuring EF in Canadian children
as it is here in the U.S. One other benefit of the test is that it is available in Spanish (Climie et. al
2014)
There are a few weaknesses to consider with the CEFI. It was found that the teacher’s
consistency may be poor due to the role of the teacher. One other concern was the cost to
administer the test online to multiple students could be costly (Fenwick & McCrimmon, 2015).
One area of use that this test could be considered for is in assessing older adults who are
low functioning in EF to determine if they qualify for special educational help from the school
TEST CRITIQUE ASSIGNMENT 10
system or maybe an adult assisted program. It might also be used to determine disability benefits
depending on the score of the adult taking the EF.
When reading though the test one area of weakness that needs more research is knowing
if the test is culturally sensitive. Even though the sample included different ethnic groups it was
still in the U.S. that the test was administered. However, if the test is administered in a school
that has a predominate presence of minority children, for example Middle Eastern, how would
this effect the test? It is suggested that more testing needs to be done to assess the applicability to
different racial and cultural backgrounds.
TEST CRITIQUE ASSIGNMENT 11
References
Ali, S., Macoun, S. J., Bedir, B., & MacDonald, S. W. S. (2019). Intraindividual variability in
children is related to informant ratings of attention and executive function. Journal of
Clinical and Experimental Neuropsychology, 41(7), 740-
748. https://doi.org/10.1080/13803395.2019.1617249
Center on the Developing Child (2012). (InBrief). Retrieved Executive Function
from www.developingchild.harvard.edu.
Climie, EA, Cadogan S, Goukon, R. (2014) Test Review: Comprehensive Executive Function
Inventory. Journal of Psychoeducational Assessment. 2014;32(2):173-177.
doi:10.1177/0734282913494169
Fenwick, M., & McCrimmon, A. W. (2015). Test review: Comprehensive executive function
inventory by J. A. naglieri and S. goldstein. SAGE Publications.
https://doi.org/10.1177/0829573514546318
Goldstein S., Naglieri J.A., Princiotta D., Otero T.M. (2014) Introduction: A History of Executive
Functioning as a Theoretical and Clinical Construct. In: Goldstein S., Naglieri J. (eds)
Handbook of Executive Functioning. Springer, New York, NY. https://doi-
org.ezproxy.liberty.edu/10.1007/978-1-4614-8106-5_1
Krieger, V., Amador-Campos, J. A., & Peró-Cebollero, M. (2019). Interrater agreement on
behavioral executive function measures in adolescents with attention deficit hyperactivity
disorder. International Journal of Clinical and Health Psychology, 19(2), 141-
149. https://doi.org/10.1016/j.ijchp.2019.02.007
Naglieri, J., & Goldstein, S. (2017) Comprehensive executive function
inventory™. https://storefront.mhs.com/collections/cefi
TEST CRITIQUE ASSIGNMENT 12
Primus, M., Warnick, B., Svenkerud, N., & Greene, E. (2014). B-05 test review of the
comprehensive executive function inventory (CEFI). Archives of Clinical
Neuropsychology, 29(6), 538-538. https://doi.org/10.1093/arclin/acu038.93
Reflection Sciences. (2016). Comparing and selecting an executive function
assessment. https://reflectionsciences.com/wp-content/uploads/2016/02/Reflection-
Science-Comparative-Article-1.pdf
Sheperis, C. J., Drummond, R. J., & Jones, K. D. (2020). Assessment procedures for counselors
and helping professionals [eBook edition]. Pearson Education. https://etext-
ise.pearson.com/products/2P0DORR3OVY/pages/ab8cb38c28060a054372d7c4641e603f
f3f8169fb?locale=null&isTpi=Y&key=3243331312219339626152021
Standards for Educational and Psychological Testing, (2014) American Educational Research
Association (AERA), the American Psychological Association (APA) and the National
Council on Measurement in Education (NCME).
(2014). https://www.testingstandards.net/uploads/7/6/6/4/76643089/standards_2014editio
n.pdf
