Week 7 Discussion

Review of the Personal Experience Inventory for Adults by MARK D. SHRIVER, Assistant Professor, University of Nebraska Medical Center, Omaha, NE:

The stated purpose of the Personal Experience Inventory for Adults (PEI-A) is to function as "a comprehensive, standardized self-report inventory to assist in problem identification, treatment referral, and individualized planning associated with addressing the abuse of alcohol and other drugs by adults" (p. 3). It is designed to yield "comprehensive information about an individual's substance abuse patterns and problems [and] . . .also helps to identify the psycho-social difficulties of the referred individual" (p. 3). The PEI-A is developed from, and an extension of, the PEI (1989) which is a self-report measure of drug and alcohol use for adolescents ages 12 to 18 (see Mental Measurements Yearbook (1992) 11:284 for reviews by Tony Toneatto and Jalie A. Tucker).

The manual states that the PEI-A "was designed primarily as a clinical descriptive tool for use by addiction professionals" and that it "is intended to supplement a comprehensive assessment process" (p. 5). Specifically, the PEI-A was developed to measure the following characteristics:

1. The presence of the psychological, physiological, and behavioral signs of alcohol and other drug abuse and dependence.

2. The nature and style of drug use (e.g., consequences, personal effects, and setting).

3. The onset, duration, and frequency of use for each of the major drug categories.

4. The characteristics of psychosocial functioning, especially factors identified as precipitating or maintaining drug involvement and expected to be relevant to treatment goals.

5. The existence of behavioral or mental problems that may accompany drug use (e.g., sexual abuse or co-addiction).

6. The sources of invalid response tendencies (e.g., "faking bad," "faking good," inattention, or random responding).

The appropriate use of any measure for its intended purposes is dependent on its sample representation, reliability, and validity. This information is reviewed respective to the PEI-A's stated purposes described above.

TEST ADMINISTRATION. The manual provides easy-to-read instructions on test administration, which will assist with increasing standardization of the administration. The manual states that the reading level of the measure is approximately sixth grade (p. 7), and the test may be read to clients with lower reading abilities. No discussion is presented, however, regarding whether this type of administration occurred during norming of the test, or how this type of administration may affect the reliability or validity of the self-report measure, as the examiner is directly involved with administration, which runs counter to one of the intended goals for test development (p. 29).

The test is scored by computer, either through a mail-in service, a FAX service, or by computer disk. The mail-in service typically requires approximately 3-5 working days to return scores (p. 75). This may be too long in some clinical settings. Interpretations of each scale are provided in the computer-generated report.

TEST DEVELOPMENT. The content of the PEI-A is largely derived from the PEI. A panel of experts is reported to have examined the items on the PEI and made changes where necessary to adapt the items to an adult population. The panel of experts was composed of "Groups of researchers and drug treatment service providers" (p. 30), but no further indication is provided in the manual about who these individuals are, where they are from, and what respective experience/expertise they have in item/test development. Item selection and scale development proceeded on a "rational basis" (p. 30). In addition to the PEI, a drug use frequency checklist adapted from adolescent and adult national survey instruments was incorporated into the drug consumption section of the PEI-A.

The initial items and scales of the PEI-A were examined with a sample of 300 drug clinic subjects (150 males, 150 females) for internal scale consistency (alpha coefficients) and interscale correlations. Correlations between Problem Severity Scales "were somewhat higher than desired for scales intended to contribute substantial unique and reliable information about the respondent, ranging from .55 to .92" (p. 33). Alpha correlations of individual scales were good, typically within the .75 to .93 range (pp. 32-33). Following examination of the correlations on this initial sample of subjects "only minor adjustments in item assignment" were made in scales (p. 33).

The content of the Problem Severity Scales is described as "multidimensional, oriented around signs and symptoms of drug abuse and dependence, and not anchored in any single theoretical model" (p. 30). Review of the items on the measure suggests that the content appears appropriate for the purposes listed above. Review of the empirical evidence for sample comparisons, reliability, and validity will help determine if items originally developed for adolescents and refined for adults based on unknown expert opinion are truly valid for adults.

The test items were not analyzed statistically for possible bias. The test was examined for differences in internal consistency across gender and race, and significant differences were not found; however, predictive validity and differential decision making across gender and race have not yet been examined. Differences in primary language of the subjects was not discussed and it is difficult to determine how language (i.e., not primarily English) might affect responses (written or oral) on the PEI-A.

SAMPLES FOR TEST VALIDATION AND NORMING. Three samples were chosen for test norming: 895 drug clinic clients from Minnesota, Illinois, Washington, California, Missouri, and Ontario, although specific numbers from each state are not provided; 410 criminal offenders, most from Minnesota; and 690 nonclinical participants, all from Minnesota. All sample subjects were volunteers. No discussion is provided in the manual regarding the possible impact on client self-report due to the sample selection process, and whether valid interpretation of the results can be made with individuals who may be tested under some type of coercion such as for court-ordered treatment. Sample demographic information is provided in the manual regarding mean age, age range, gender, minority, percent in prior treatment, marital status, employment status, and education (p. 36).

Scores from the measure are compared with the drug clinic sample in the form of T scores; however, score comparisons (T scores) are also provided at the end of the computerized report for the nonclinic sample. Given the restricted geographic sampling of the nonclinic group, it is difficult to determine if this comparison provides useful information for individuals who are not from Minnesota. It is also unclear if the nonclinic sample participated by mail as described on page 35 of the manual or through group testing as described on page 36 of the manual. Both contexts for test taking are somewhat different from the typical administration (e.g., individualized, in drug clinic) described in the manual and may limit score interpretations even further.

The drug clinic sample is described in terms of two groups: outpatient and residential treatment. Only 37.5% of the outpatient drug clinic sample is female (approximately 188). Only 36.3% of the residential drug clinic sample is female (approximately 143). Separate T scores are provided for male and female samples (p. 36). Only approximately 113 members of the outpatient drug clinic sample are of an ethnic minority status, and approximately 68 members of the residential drug clinic sample are of an ethnic minority status. Minority is not defined further (e.g., African American, Hispanic, Native American), although it is conceivable that minority status may differentially impact drug use. In addition, although the gender representation may be an accurate reflection of general population drug use, the small sample size for females limits normative comparisons. Reported drug use patterns may also differ by gender.

Information is not provided on whether the norm groups come from rural or urban settings. A rural or urban context may impact drug use (i.e., availability of drugs). Also, specific numbers are not provided relative to the geographic regions from which the drug clinic samples originate, and as indicated by the authors (p. 36), geographic region may impact reported drug use (i.e., higher cocaine use in California and Washington reported relative to Midwest states and Ontario).

In summary, caution is advised in using the PEI-A with females, minorities, and individuals from geographic regions other than those sampled. In addition, the comparison with nonclinic population may not be useful for individuals outside of Minnesota. The test norms appear to be useful for comparing Caucasian males with possible drug use history with the drug clinic sample.

RELIABILITY. Internal consistency reliabilities are provided (coefficient alpha) for the entire sample and provided for male, female, white, and minority samples (pp. 37-41). In addition, test-retest reliabilities are presented for one week and for one month using the drug clinic sample, although there was some intervening treatment between pre- and posttest scores (pp. 42-43). Only reliabilities for the drug clinic and nonclinic samples will be discussed as these represent the primary comparative groups for examinees.

Coefficient alphas are generally good for the Problem Severity Scales (median .89 range .72 to .94) and the Psychosocial Scales (median .81 range .67 to .91). The coefficient alphas are low for the Validity Scales (median .63 range .58 to .77) (p. 37). The authors claim the validity reliability estimates compare favorably with other instruments, and this may be true; however, these values are just acceptable given the use to be made of these scores.

Median test-retest reliabilities at one week (70 individual) were as follows: Problem Severity scales .71 (.60 to .88), Psychosocial scales .66 (.55 to .87), and Validity Indices .52 (.40 to .57). One-month test-retest reliabilities were lower as expected given intervening treatment (pp. 42-43). Given that some subjects were provided intervening treatment in the one-week test-retest group also, it can reasonably be said that test-retest reliability has not been adequately examined and no conclusions can be drawn regarding temporal stability of the test. This makes it less useful pre- and post-treatment as it is difficult to determine if changes in scores are due to treatment or lack of score stability. This is in conflict with the authors' conclusions, however, that scores can be compared pre- and posttreatment (p. 43).

In summary, the internal consistency estimates of the Problem Severity Scales and the Psychosocial Scales range from good to acceptable. More research is definitely needed on the stability of the test scores (test-retest) before conclusions can be drawn regarding the test's usefulness pre- and posttreatment.

VALIDITY. One potential use of this instrument is to determine appropriate treatment options for individuals. Drug clinic subjects (N = 251) were classified into three referral categories: no treatment, outpatient treatment, and residential treatment based on clinical staff ratings. Mean scores on the PEI-A Problem Severity Scales were examined and expected differences in scores were found for the three groups (p. 46). Future researchers, however, may want to look at the contribution the PEI-A provides above and beyond other information used in making referral decisions. In other words, are these mean score differences useful? Also, significant differences in mean scores according to sample group membership (nonclinical, drug clinic, and criminal offender) were also found (p. 46). Again, an empirical examination as to how this information contributes as part of a comprehensive assessment would be useful.

Seven of the Problem Screens were compared with staff ratings to determine sensitivity and specificity of the screens, essentially the degree of agreement regarding the existence of problems (p. 48). For the total sample, there were significant correlations (p<.05) for agreement between the PEI-A and staff ratings for negative ratings (i.e., individual not identified with having problem), but not for positive ratings (i.e., individual identified as having problem) (p. 49).

The Validity Indices were found to correlated as expected with Minnesota Multiphasic Personality Inventory (MMPI) Validity scales (p. 48).

To assess the construct validity of the scale, correlations with tests purported to measure similar constructs were examined. Moderate correlations were found between the Problem Severity Basic Scales scores and the Alcohol Dependence Scale (.41-.66; p. 44; ADS; Horn, Skinner, Wanberg, & Foster, 1982). In addition, correlations are also provided for Problem Severity Scale scores and the Drug Use Frequency Checklist; however, the Drug Use Frequency Checklist is actually part of the PEI-A so the usefulness of this information for construct validity is weakened. The Psychosocial Scales of the PEI-A were found to correlate significantly with MMPI scales, suggesting the psychosocial Scales are measuring psychopathology to some extent (p. 45), but there does not appear to be much differentiation between the PEI-A scales as all but Rejecting Convention and Spiritual Isolation correlate highly with each of the MMPI scales. Finally, information is provided that "select" PEI-A scales (p. 45) correlate significantly with a Significant Other Questionnaire. However, the Significant Other Questionnaire is also developed from PEI-A items, which again attenuates the meaningfulness of this relationship.

In summary, the validity evidence presented in the manual does not appear to address specifically the intended purposes/applications of the test noted above. The content looks good, but much more empirical research is needed on the validity of this instrument specifically related to the applications for which it is intended. Future research should address whether this instrument contributes significantly (above and beyond other information in a comprehensive assessment) to decision making involved in assessing and treating individuals with alcohol and drug use problems.

SUMMARY. The PEI-A may be most useful for examining alcohol and drug use in white males who are compared with a drug clinic sample. Results of this test are intended to tell the clinician whether an individual is similar to individuals in the drug clinic sample and to provide some information on the impact of drugs on the individual's life. Caution is urged in using the PEI-A with females and minorities given the small sample sizes. Geographic region and urban-rural differences may also impact reports of drug use and should be considered by the test user. In addition, this test may not be useful for individuals whose primary language is not English. The use of the nonclinic scores for comparisons is questionable for individuals outside Minnesota. Estimates of the internal consistency reliability of the scales and content appear good. Additional research on test-retest reliability is needed. More research on the validity of the PEI-A as part of a comprehensive assessment is needed. The PEI-A looks promising, but users are encouraged to heed the test author's statement that this test should only be used as part of comprehensive assessment.

REVIEWER'S REFERENCES

Horn, J. L., Skinner, H. A., Wanberg, K., & Foster, F. M. (1982). Alcohol Dependence Scale (ADS). Toronto: Addiction Research Foundation.

Toneatto, T. (1992). [Review of the Personal Experience Inventory.] In J. J. Kramer & J. C. Conoley (Eds.), The eleventh mental measurements yearbook (pp. 660-661). Lincoln, NE: Buros Institute of Mental Measurements.

Tucker, J. A. (1992). [Review of the Personal Experience Inventory.] In J. J. Kramer & J. C. Conoley (Eds.), The eleventh mental measurements yearbook (pp. 661-663). Lincoln, NE: Buros Institute of Mental Measurements.

Review of the Personal Experience Inventory for Adults by CLAUDIA R. WRIGHT, Professor of Educational Psychology, California State University, Long Beach, CA:

The Personal Experience Inventory for Adults (PEI-A) is a standardized self-report instrument for use by service providers in the substance abuse treatment field to assess patterns of abuse and related problems in adult clients (age 19 or older). The two-part, 270-item PEI-A is made up of 10 problem severity scales and 11 psychosocial scales, 5 validity indicators, and 10 problem screens; it parallels in content and form the two-part, 300-item Personal Experience Inventory (PEI; 11:284) developed for use with adolescents (age 18 or younger). A broad theoretical framework, influenced by Alcoholics Anonymous, social learning, and psychiatric models, underlies the development of both inventories. The manual presents a thorough treatment of test development, standardization, and validation procedures along with clear test administration and computer-scoring guidelines and useful strategies for score interpretation. The inventory is written at a sixth-grade reading level. No provisions are made for non-English-speaking test takers.

NORMING PROCEDURES. Norm tables were constructed separately for males and females in two standardization samples (clinical and nonclinical). Normative data were obtained primarily from Midwestern Whites, raising concerns about the generalizability of score interpretations to clients classified as nonwhite. Demographic information presented in the PEI-A manual indicates that 20% of the clinical sample (n = 895) was classified as minority. Clinic respondents attended outpatient and residential Alcoholics Anonymous-based programs at 12 sites (located in 3 midwestern and 2 western states and 1 Canadian province). No rationale was provided for site selection. A total of 690 Minnesota residents comprised the nonclinical sample; 11% were classified as minority. A sample of 410 criminal offenders (77% were male; 68% of the sample was nonwhite) was used to provide data for some validation analyses.

Caution is warranted in applying the PEI-A norms to members of nonwhite groups in either clinical or nonclinical settings. The test developer is to be commended for briefly acknowledging this limitation. Sampling that includes more regions, broader ethnic representation, and types of treatment program sites is essential.

RELIABILITY. For 1,995 respondents, median Cronbach alphas were (a) Problem Severity Scales = .89 (range: .81-.93); (b) Psychosocial Scales = .80 (range: .75-.88); and (c) three of the five Validity Indicators = .70 (range: .65-.73). When subsamples were broken out by gender, ethnicity (white or minority), and setting (nonclinical, drug clinic, or criminal offender), patterns of reliability estimates were comparable to those obtained with the total sample. One-week (n = 58; .42-.78, mdn = .69) and one-month (n = 49; .39-.72, mdn = .52) stability indexes for problem screens were lower than desired due to respondents' exposure to treatment programs during the test-retest intervals.

CONTENT VALIDATION. Common content validation procedures were followed. Researchers and treatment providers rated PEI items intended for inclusion in the PEI-A with respect to clinical relevance and importance to adult substance abuse. Based upon rater feedback, minor item modifications were made.

CRITERION-RELATED VALIDITY. Concurrent validity evidence for the PEI-A was provided by data comparisons examining the effects on scale scores of (a) treatment history for substance abuse among drug clinic clients (no sample size reported); (b) referral recommendation (no treatment, outpatient, or residential) (N = 251); (c) setting (nonclinical, drug clinic, or criminal offender) (N = 1,978); and (d) DSM-III-R (American Psychiatric Association, 1987) diagnosis of abuse or dependence upon alcohol or drugs (N = 244). The observed group differences obtained from scores on the 10 Problem Severity Scales supported the view that individuals referred to treatment settings (outpatient or residential) had greater problems with higher substance use, dependence, and related consequences of usage compared to those for whom no drug treatment was recommended. The 11 Psychosocial Scales fared less well in distinguishing among the three groups with only three scales (Negative Self-Image, Deviant Behavior, and Peer Drug Use) yielding statistically significant differences. In a separate analysis, scores obtained from a nonclinical subsample (n = 687) were significantly lower on each of the 21 scales (all p < .01) when compared with those from drug clinic (n = 887) and offender (n = 404) groups. For the DSM-III-R Diagnosis comparison, clients identified as dependent on alcohol or drugs had significantly higher scores on the 5 Basic Scales when compared to those classified as abusing these substances.

Although the measure is purportedly used to assist in treatment referral, no predictive validity information was presented linking referral decisions based upon standing on the PEI-A scales and outcome success.

CONSTRUCT VALIDITY. Only modest to moderate levels of construct validity evidence were presented based on correlations between PEI-A Problem Severity Basic scale scores and performance on the Alcohol Dependence Scale (ADS; Horn, Skinner, Wanberg, & Foster, 1982) and the PEI-A Drug Use Frequency Checklist. Moderate coefficients were obtained for a sample of 89 clients indicating that the 5 Basic Scale scores were somewhat related to ADS scores (.52-.63, mdn = .59) and Checklist scores (.41-.66; mdn = .55). For a sample of 213 clinic respondents, correlations among the 11 PEI-A Psychosocial Scales and 9 Minnesota Multiphasic Personality Inventory (MMPI) Scales yielded 62 out of 99 possible coefficients ranging from .20-.69, mdn = .38 (all p < .001) indicating, for the most part, only modest levels of shared variance (4% to 48% explained, mdn = 14%). Moderate coefficients (above the median) were associated with PEI-A scales that deal with personal adjustment issues (e.g., Negative Self-Image, Psychological Disturbance, Social Isolation, and Absence of Goals). PEI-A scale scores dealing with personal values and environmental influences (e.g., Rejecting Convention and Spiritual Isolation) yielded negligible correlations with the MMPI. PEI-A and MMPI validity indicators also were moderately correlated.

Inspection of intercorrelations among the 10 Problem Severity Scales revealed moderate to strong coefficients posing a multicollinearity problem. It is evident from data reported in the manual that the statistical contribution of unique variance to score interpretation associated with each of the 5 Clinical Scales adds little or no unique information (rxys ranged from .04 to .09, mdn = .05). This outcome was consistent with that reported for the same 10 scales of the PEI. The 5 Clinical Scales were retained "because users have found these scales helpful" (manual, p. 33). The retention of redundant scales requires more detailed explanation than that provided in the manual. For future research and test development purposes, targeting items from scales that contribute unique information for provider applications and removing redundant items would strengthen this section of the inventory.

Intercorrelations among the Psychosocial Scales revealed patterns of coefficients more distinctive of a multidimensional scale (as intended) with proportions of unique variance ranging from .18 to .57 (mdn = .29). However, lower reliability estimates and the inability of these scales to distinguish between referral groups is of concern.

SUMMARY. The Personal Experience Inventory for Adults (PEI-A) offers a beginning point to the service provider for assessment. Most PEI-A scale scores demonstrate adequate levels of reliability and distinguish between clinical and nonclinical groups. Current norms may be too restrictive for some settings. Based upon validity evidence provided, caution is warranted in all testing with use of scores from the Clinical Scales, which are redundant with the Basic Scales and with scores from the Psychosocial Scales, which have shown only low to moderate relationships with related constructs. PEI-A computer-generated recommendations for individual clients should be considered in light of these limitations and decisions made in conjunction with other measures.

REVIEWER'S REFERENCES

Horn, J. L., Skinner, H. A., Wanberg, K., & Foster, F. M. (1982). Alcohol Dependence Scale (ADS). Toronto: Addiction Research Foundation.

American Psychiatric Association. (1987). Diagnostic and statistical manual of mental disorders (3rd ed.). Washington, DC: Author.