evaluate assessment of intelligence and achievements

Acronym:

AAB Screening Form

Authors:

Messer, Melissa, A.

Publication Date:

2014.

Publisher Information:

Psychological Assessment Resources, Inc., 16204 N. Florida Avenue, Lutz, FL, 33549-8119, [email protected], www.parinc.com

Source:

J. F. Carlson, K. F. Geisinger, & J. L. Jonson (Eds.), The twentieth mental measurements yearbook. 2017.

Reviewers:

Acosta, Sandra; Patelis, Thanos

Purpose:

Designed to assess 'basic academic skills including reading, spelling, and mathematical calculation' with an optional writing subtest.

Test Category:

Achievement.

Population:

Ages 4 to 85.

Scores:

5: Letter/Word Reading, Spelling, Written Comprehension (optional), Mathematical Calculation, Screening Academic Achievement Battery Composite.

Administration:

Individual (certain subtests may be administered to groups).

Time:

(20-30) minutes for Letter/Word Reading, Spelling, and Mathematical Calculation subtests; (15) minutes for optional Written Composition subtest

Price Data:

2020: $154 per print kit including professional manual (200 pages) with Fast Guide, 25 item booklets, 25 response booklets, stimulus card, and 5 online score reports; $88 per manual (print or digital) including Fast Guide; $28 per 25 item booklets; $28 per 25 response booklets; $13 per stimulus card; $2 per score report use (minimum 5).

Comments:

Computer scoring available. Written Composition subtest administered only to examinees in Grade 3 and above; subtest is optional, but it is required to calculate Screening Academic Achievement Battery Composite score. Content is a subset of content in the Academic Achievement Battery Comprehensive Form

Cross References:

For reviews by Sandra T. Acosta and Thanos Patelis, see 20:3.

Published Test Description:

Academic Achievement Battery Screening Form. Purpose: Designed to assess "basic academic skills including reading, spelling, and mathematical calculation" with an optional writing subtest. Population: Ages 4 to 85. Publication Date: 2014. Acronym: AAB Screening Form. Scores, 5: Letter/Word Reading, Spelling, Written Comprehension (optional), Mathematical Calculation, Screening Academic Achievement Battery Composite. Administration: Individual (certain subtests may be administered to groups). Price Data, 2016: $149 per print kit including professional manual with fast guide, 25 item booklets, 25 response booklets, stimulus card, and 5 free score reports on PARiConnect; $80 per manual (200 pages) including fast guide; $30 per 25 item booklets; $30 per 25 response booklets; $10 per stimulus card. Time: (20-30) minutes for Letter/Word Reading, Spelling, and Mathematical Calculation subtests; (15) minutes for optional Written Composition subtest. Comments: Computer scoring available. Written Composition subtest administered only to examinees in Grade 3 and above; subtest is optional, but it is required to calculate Screening Academic Achievement Battery Composite score. Content is a subset of content in the Academic Achievement Battery Comprehensive Form. Author: Melissa A. Messer. Publisher: Psychological Assessment Resources, Inc.

Language:

English

Accession Number:

test.6872

Translate Full Text:

Choose Language

Academic Achievement Battery Screening Form

This content may contain URLs/links that would redirect you to a non-EBSCO site. EBSCO does not endorse the accuracy or accessibility of these sites, nor of the content therein.

✖

Full Text

Listen

Review of the Academic Achievement Battery Screening Form by SANDRA T. ACOSTA, Assistant Professor, Educational Psychology Department, Texas A&M University, College Station, TX:

DESCRIPTION. The Academic Achievement Battery (AAB) Screening Form is an achievement screening test for evaluating basic skills in reading, writing, and mathematics. The battery was developed for English speakers between the ages of 4 and 85 and may be administered in less than 45 minutes. Four subtests comprise the AAB Screening Form: Letter/Word Reading, Spelling, Written Composition, and Mathematical Computation. Subtests vary according to the number of sections, whether they can be group administered, and age/grade level of examinees. Letter/Word Reading includes Letter Reading (15 items, individual administration, prekindergarten [PK] through Grade 3) and Word Reading (60 items, individual or group administration, all ages). Spelling includes Letter Writing (8 items, individual administration, PK through Grade 3) and Word Writing (60 items, individual or group administration, all ages). Written Composition consists of one timed expository essay that can be administered individually or in groups to examinees in Grade 3 and above. Writing quality is assessed using a seven-criteria rubric comprising theme/main idea, organization, voice, word choice, sentence fluency, presentation (handwriting), and conventions. The Mathematical Calculation subtest has two parts: Part 1 (10 items, individual administration, PK through Grade 3) and Part 2, which is timed (50 items, individual or group administration, all ages).

The AAB Screening Form print kit consists of the professional manual, fast guide, letter/word reading card, item booklets, and response booklets. The item booklet contains test administration scripts as well as instructions for grading and calculating scores. Scoring is facilitated by the score summary table on the booklet’s cover and the scoring conversion tables in the test manual. Score reports include subtest and composite score profiles, standard scores with confidence intervals at 90% and 95%, age and grade equivalent scores, percentile ranks, stanines, and normal curve equivalents. Score discrepancies between subtests are also reported as well as the statistical significance level and percentage of the normative sample within each significance level.

DEVELOPMENT. The AAB has two versions that were developed simultaneously, the AAB Comprehensive Form and the AAB Screening Form. The AAB Comprehensive Form has a 15-subtest and eight-composite structure whereas the AAB Screening Form has a four-subtest and one-composite structure. Test item development comprised two components—item selection and difficulty analysis. The theoretical underpinnings for this process were classical test theory and Rasch model/analysis, a special case of item response theory (see McDonald, 1999).

All test items were selected based on standards published by professional associations such as the National Council of Teachers of Mathematics (NCTM) and the Common Core State Standards established by the National Governors Association Center for Best Practices and the Council of Chief State School Officers. For the reading and spelling subtests, word selection was based on the Educator’s Word Frequency Guide (words drawn from texts commonly used in U.S. schools and universities) and the Common Core three-tiered vocabulary taxonomy—basic, general academic, and domain-specific (content or subject area). Further, for spelling item selection, criteria also included word attributes such as silent vowels and consonants as well as homophones and irregular words. Mathematics calculation problem selection was based on NCTM standards for mathematics in schools and Common Core. The writing prompt was based on the 1996 National Council of Teachers of English (NCTE) standards. Two panels evaluated the test items and provided feedback: an expert panel (for determining age/grade appropriate items and validity) and a bias panel (for identifying possible cultural bias in item content).

TECHNICAL.

Standardization. AAB test developers conducted three phases of data collection: pilot testing, refinement, and standardization. The sample of the pilot study, conducted during winter 2011, was composed of 133 participants, ages 4–83 years. The sample of the refinement study, conducted during spring 2011, was composed of 280 participants, ages 4–70 years. Standardization or normative sample data were collected from 30 states between January 2013 and March 2014. The standardization sample consisted of age- and grade-based samples. The age-based sample comprised 1,274 participants, ages 4–83 years. The grade-based sample comprised 1,447 participants ages 4–19 years. The demographic characteristics—age, gender, ethnicity, and educational levels—were representative of the 2012 U.S. Census. Test developers applied weighting procedures to correct for discrepancies between the age- and grade-samples and the U.S. Census proportions. A continuous norming approach was used to correct for sampling irregularities and maximize the normative score accuracy.

Reliability. To evaluate the accuracy and consistency of the normative sample (age- and grade-based) scores, the following indicators of reliability were used: standard error of measurement (SEM), internal coefficient alpha statistics, test-retest coefficients, and interrater reliability coefficients. SEM statistics were calculated for the test scores of the age-and grade-level normative samples. Average SEM statistics ranged from 2.57–6.10. Internal coefficient alpha statistics derived from subtest scores ranged from .89–.97. To produce test-retest coefficients, a subset of the normative sample (147 persons) was retested with a median time between testing of 18 days. Coefficients ranged from .83 to .95. Interrater reliability was calculated for the Written Composition subtest scores. Scores were drawn from 20 randomly selected protocols independently scored by two trained research assistants. Intraclass correlation coefficients derived from the writing rubric scores ranged from .78 to .95 for the seven individual criteria, with a coefficient of .97 for the total score.

Validity. The validation process for the AAB had four components: intercorrelations, content-related evidence of validity, criterion-related evidence of validity, and construct-related evidence of validity. Intercorrelations were calculated between the subtest scores and the composite scores for the age- and grade-based normative samples. The evaluation of content evidence of validity consisted of expert panel review and examination of test items in relation to professional standards (e.g., Common Core). Criterion evidence of validity (to predict performance for clinical evaluations) was evaluated comparing the scores of students with disabilities drawn from the following samples: specific learning disorder, intellectual developmental disorder, attention deficit disorder with hyperactivity, and speech/language impairment. To evaluate construct-related evidence of validity, AAB scores were compared to the subtest and composite scores of four achievement tests and two versions of the Wechsler Intelligence Scale (children and adults).

Score correlations between the AAB Screening Form and the Wide Range Achievement Test 4 (WRAT4) included: reading subtests, r = .86, p < .01; spelling subtests, r = .85, p < .01; mathematical computation subtests, r = .75, p < .01. Score correlations between the AAB Screening Form and the Wechsler Individual Achievement Test, Third Edition (WIAT-III) included: word reading subtests, r = .77, p < .01; spelling subtests, r = .88, p < .01; AAB Written Composition subtest and WIAT-III writing composite, r = .30, not statistically significant; AAB Mathematical Calculation subtest and WIAT-III mathematics fluency composite, r = .23, not statistically significant. For the Kaufman Test of Educational Achievement, Second Edition (KTEA-II) and the AAB Screening Form, score correlations included: word reading subtests, r = .49, p < .01; spelling subtests, r = .77, p < .01; AAB Written Composition subtest and KTEA-II written language composite, r = .65, p < .01; mathematical calculation subtests, r = .85, p < .01. The Woodcock-Johnson Test of Achievement, Third Edition (WJ-III) and AAB score correlations included: word reading subtests, r = .67, p < .01; spelling subtests, r = .70, p < .01; AAB written composition subtest and WJ-III writing samples subtest, r = .10, not statistically significant; mathematical calculation subtests, r = .65, p < .01.

COMMENTARY. Overall strengths of the AAB Screening Form are documentation of the development and standardization process, test booklets with clear instructions for administering and scoring the test, and reliability and validity evidence. Nonetheless, three issues should be mentioned. First, although the test author named the methodological theories for analyzing the data, there is no information about how well the IRT (Rasch) model fit the data. Bachman (2004) states that IRT models must be evaluated empirically by describing model fit. Second, albeit a minor proofing error, some of the citations in the Fast Guide do not appear in the references list. Finally, the manual mentions that the AAB was developed and normed on English-speaking individuals. Given the increasing numbers of English language learners (ELLs) in public schools and the suggested usage of the AAB Screening Form for clinical purposes, consumers should be cautioned about validity issues when testing ELLs. English language proficiency can introduce construct-irrelevant variance, which is a threat to validity, by confounding the interpretation of the data, affecting measurement accuracy, and producing questionable results (Acosta et al., 2015).

SUMMARY. The AAB Screening Form is a test of basic academic skills for individuals whose primary language is English. The manual provides detailed descriptions of the test development and standardization process. Reliability and validity evidence appears to yield robust results for supporting the potential usefulness of the AAB as a screening test. Nonetheless, it is important to remember that a “test is useful only to the extent that the consumer’s intended use overlaps with the evidence supporting the purported use of the test” (Sawilowsky, 2003, p. 113). Therefore, this reviewer recommends the AAB Screening Form with the following caution: psychometric properties of a test are not static. They develop over time and may change with different test administrations and samples (Thompson, 2003).

REVIEWER’S REFERENCES

Acosta, S., Garza, T., Goodson, P., Padrón, Y., Goltz, H. H., & Johnston, A. (2015). A systematic review of high-stakes accountability testing and English language learners during the first decade of the No Child Left Behind act. Manuscript submitted for publication.

Bachman, L. F. (2004). Statistical analyses for language assessment. Cambridge, UK: Cambridge University Press.

McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Erlbaum.

Sawilowsky, S. S. (2003). Reliability as psychometrics versus datametrics. In B. Thompson (Ed.), Score reliability (pp. 103–121). Thousand Oaks, CA: Sage.

Thompson, B. (Ed.). (2003). Score reliability. Thousand Oaks, CA: Sage.

Review of the Academic Achievement Battery Screening Form by THANOS PATELIS, Senior Associate, National Center for the Improvement of Educational Assessment, Dover, NH, and Research Scholar, Graduate School of Education, Fordham University, New York, NY:

DESCRIPTION. The Academic Achievement Battery (AAB) Screening Form is a group and/or individually administered assessment taking less than 45 minutes to administer for school-aged individuals (ages 4-19 years), college-aged students (no age range indicated), and adults (ages 18 to 85 years). The test is designed to identify academic strengths and weaknesses, and for use in eligibility decisions and planning for specific interventions in some of the academic areas defined by the Individuals with Disabilities Education Act (IDEA; 2004). The test manual indicates that the AAB Screening Form can be used in educational, clinical, vocational, and research settings. A composite and four subtest scores in Letter/Word Reading, Spelling, Written Composition (optional), and Mathematical Calculation are provided. The manual indicates that the examiner is required to have “adequate training and skill in psychological assessment procedures” (p. 3) including familiarity with measurement theory and psychometric concepts. Administration directions call for standardized procedures and instructions with a recognition that modifications may be made for individuals with specific needs. In such cases, the examiner should know the individual’s needs prior to testing, note the modifications, use them in the interpretation, and evaluate the modification consistent with recommendations on the administration and accommodations of individuals with special needs (e.g., U.S. Department of Education, 2007). For several subtests, opportunities are provided to the examiner for indicating notes about the examinee’s performance. These notes are not used in scoring, but can be incorporated in the interpretations offered. Directions for starting and stopping rules are provided. Scoring rules are provided including rubrics and exemplars for the Written Composition subtest. Age-based (4 years and older) and grade-based (prekindergarten to Grade 12) norms are provided to obtain subtest scaled scores that range from 50 or below to 150 and above, as well as confidence intervals. The composite standard score is calculated by adding the subtest standard scores and using age or grade norms to obtain a standard score. Composite scores can be calculated only if all subtests are administered, except for with examinees in Grade 2 and below with whom the Written Composition subtest is not used. Percentile ranks, stanines, and normal curve equivalents may be obtained from standard scores. Additionally, score discrepancies are calculated by subtracting each subtest score from selected other subtest scores. Statistical significance levels of the differences can be obtained. An examiner can obtain an ability-achievement discrepancy by using scores from a specific intelligence test and either calculating a difference or using tables of predicted achievement scores from scores on the ability measure. Norm-referenced interpretations are made for subtest and composite scores. Performance level descriptors are provided using norm-referenced terms.

DEVELOPMENT. The test manual indicates that the content of the AAB Screening Form is a subset of the AAB Comprehensive Form. The basis for the content of the AAB Comprehensive Form was the areas of achievement specified in IDEA. Of the 20 subtests created to reflect those content areas, 15 were selected for inclusion in the Comprehensive Form, and four of those comprise the Screening Form. Five of the original 20 subtests were eliminated during test development. Selection of the content involved using national standards (i.e., Common Core State Standards tier levels [National Governors Association Center for Best Practices, 2010], National Council of Teachers of English Standards, 1996, and National Council of Teachers of Mathematics Standards, 2000). A process of expert content and bias reviews was implemented. Pilot testing was conducted involving item analyses, estimates of reliability, and, where appropriate, distractor analyses. Sample sizes for the pilot studies were relatively small at 133 and 280 participants, selected based on demographic characteristics. These analyses resulted in the standardization version administered to the norming samples.

TECHNICAL.

Standardization. Age- and grade-based norms were developed using 1,274 participants for the age-based norms, 737 participants for the fall grade-based norms, and 710 participants for the spring grade-based norms. The samples involved individuals between the ages of 4 and 83 (ages 4 to 19 for the grade-based norms) and were selected to be representative of the 2012 U.S. Census in terms of age, gender, ethnicity, and education level. Individual scores were weighted to reflect the U.S. Census proportions, and continuous norming was used to develop the age- and grade-based normative tables.

Reliability. Several indicators of reliability were provided including internal consistency, test-retest reliability involving time intervals that ranged from 7 to 49 days (median of 18 days), and interscorer reliability for the Written Composition subtest. Alpha coefficients were reported for each subtest and the composite based on age- and two grade-based samples with values ranging from .89 to .98 representing a high degree of internal consistency. The test manual provided tables with alpha coefficients for each subtest and composite at each age or age range and grade. Standard errors of measurement were reported overall for each subtest and composite, as well as by age range and grade. The standard error of measurement was smallest for the composite scores for the age- and grade-based samples at 3.0 when rounded to the nearest whole number, whereas the standard error of measurement for the subtests varied between 3.62 and 6.10 depending on the sample and subtest. Test-retest reliability using correlation coefficients corrected for attenuation for a sample of 127 examinees for the Written Composition subtest and 147 for the other subtests and composite ranged from .83 to .95. Interscorer consistency from a sample of 20 examinees for the Written Composition subtest using intraclass correlation coefficients based on a two-way random effects model ranged from .78 to .97 for each of the specific criteria and the total score. Intercorrelations between subtests and composite scores using age- and grade-based samples were reported with all correlation coefficients being positive and statistically significant. Magnitudes ranged from small to moderate to large values with values for the Grade 6-12 sample consistently smaller than the values for the Prekindergarten to Grade 5 and age-based samples.

Validity. The test manual provided evidence in support of the validity of the scores from the AAB Screening Form based on content alignment, relationship to other achievement measures, relationship to measures of intelligence, and differential performance of examinees using external criteria. Although results of formal alignment studies were not presented, during item review when the AAB Screening Form was being developed, expert panels reviewed all items to ensure that the content represented the intended areas. Four studies examining the relationship of AAB Screening Form scores and scores from other achievement tests showed positive, strong correlations involving sample sizes from 24 to 100. Two studies examining the relationship of AAB Screening Form scores and scores from intelligence tests showed weak to moderate correlations. Two studies examining differential performance on the AAB Screening Form showed large differences for students with various disabilities in comparison to the normative samples. Two other studies involving students diagnosed with attention-deficit/hyperactivity disorder (ADHD) and speech/language impairment mostly showed small to moderate differences based on a standardized effect size, but there was variability in these results by subtest. Students diagnosed with ADHD showed a moderate difference from the normative sample on the Mathematical Calculation subtest with the rest of the subtests showing small standardized mean differences. For students with speech/language impairments, the standardized mean differences were large for the Written Composition subtest, moderate for the Spelling subtest, and small for the other two subtests.

COMMENTARY. The AAB Screening Form, designed to assess academic skills in reading, spelling, writing, and mathematical computation to inform eligibility and planning for interventions in educational, clinical, vocational, and research settings, showed strong evidence in the reliability of the scores and evidence to support the measurement of academic skills in reading, spelling, and mathematical computation, as well as its ability to show differential performance of students with disabilities in comparison to normative samples. The directions for administration and scoring are clear. Although the normative samples seemed to be representative of the U.S. population overall, in some age ranges and possibly some performance levels, the samples may be relative sparse. The weighting and continuous norming procedures help maximize the accuracy of the normative scores, but efforts to increase the normative sample across all levels of performance should be made in future test development efforts. As a screening tool, based on the results of the differential performance of students with disabilities in comparison to the normative samples, it seems that examinees likely would be correctly identified for additional assessment, and are less likely to be missed. A nice feature is the ability of the examiner to capture qualitative information about the performance of the individual for use in the interpretation. The test manual does indicate, as part of a case example provided, that additional information should be gathered and used in designing an appropriate education plan. Although some evidence was provided to support the content of the assessment in representing skills in reading, spelling, and mathematical computation, more formal alignment studies are needed to help understand the extent of the skills represented. Additionally, evidence showing the efficacy of the AAB Screening Form as a screening instrument is needed.

SUMMARY. The AAB Screening Form is an individually administered assessment with some subtests able to be group-administered. The battery takes less than 45 minutes to administer for a variety of ages and is designed to assess specific academic skills to inform eligibility and planning for educational interventions. The directions for administration and scoring are clearly provided. Age- and grade-based norms are provided for each of the four subtests and a composite score. The reliability evidence is strong, and there is evidence to support the use of the scores to differentiate performance of individuals with disabilities. However, as noted in the test manual, additional information is needed from other sources in order to develop an appropriate education plan. Future efforts to enhance the assessment should focus on expanding the size of the normative sample, implementing an alignment study, and examining the efficacy of the AAB Screening Form as a screening tool.

REVIEWER'S REFERENCES

Individuals with Disabilities Education Improvement Act of 2004 (IDEA). Public Law No. 108-446, 118 Stat. 2657.

National Council of Teachers of English. (1996). Standards for the English language arts. Urbana, IL: Author.

National Council of Teachers of Mathematics. (2000). Principles and standards for school mathematics. Reston, VA: Author.

National Governors Association Center for Best Practices, Council of Chief State School Officers (2010). Common Core State Standards. Washington, DC: Author.

U.S. Department of Education (2007). No Child Left Behind modified academic achievement standards: Non-regulatory guidance. Washington, DC: Author.

*** Copyright © 2024. The Board of Regents of the University of Nebraska and the Buros Center for Testing. All rights reserved. Any unauthorized use is strictly prohibited. Buros Center for Testing, Buros Institute, Mental Measurements Yearbook, and Tests in Print are all trademarks of the Board of Regents of the University of Nebraska and may not be used without express written consent.