Week 7 Discussion

Review of The Psychiatric Diagnostic Screening Questionnaire by MICHAEL G. KAVAN, Associate Dean for Student Affairs and Associate Professor of Family Medicine, Creighton University School of Medicine, Omaha, NE:

DESCRIPTION. The Psychiatric Diagnostic Screening Questionnaire (PDSQ) is a self-report instrument designed to screen for Diagnostic and Statistical Manual of Mental Disorders, Fourth Edition (DSM-IV; American Psychiatric Association, 1994) Axis I disorders that are most commonly seen in medical and outpatient mental health settings. It is designed to be completed by individuals 18 years of age and older prior to their initial diagnostic interview. The PDSQ covers 13 Axis I areas including Major Depressive Disorder, Posttraumatic Stress Disorder, Bulimia/Binge-Eating Disorder, Obsessive-Compulsive Disorder, Panic Disorder, Psychosis, Agoraphobia, Social Phobia, Alcohol Abuse/Dependence, Drug Abuse/Dependence, Generalized Anxiety Disorder, Somatization Disorder, and Hypochondriasis. It also provides a PDSQ Total score, which acts as a global measure of psychopathology.

According to the manual, the PDSQ is designed to be used in "any clinical or research setting where screening for psychiatric disorders is of interest" (manual, p. 2). It may be administered and scored by any appropriately trained and supervised technician; however, clinical interpretation should only be undertaken by a professional with appropriate psychometric and clinical training.

The PDSQ consists of 125 items in which respondents are requested to answer yes or no to each test booklet question according to "how you have been acting, feeling, or thinking" during the past 2 weeks or 6 months, depending on the symptom cluster. Typical administration time is between 15 and 20 minutes. Scoring is completed by hand and entails counting the number of yes responses on each PDSQ subscale and entering that number in the space provided on the accompanying summary sheet. Subscale scores are then compared to cutoff scores to determine whether follow-up interviewing is indicated. In addition, the scorer is to circle critical items to which the respondent answered "yes." All subscale scores are then summed in order to obtain a PDSQ Total raw score. Finally, the PDSQ Total raw score is transferred to a PDSQ Score Conversion table that converts the total score into a T-score. On the back side of the summary sheet is a table that includes diagnosis percentages of persons who endorsed each item and either qualified or failed to qualify for a subscale diagnosis. An accompanying CD provides follow-up interview guides for all 13 disorders. These may be printed and then used to gather additional diagnostic information regarding these syndromes.

As noted previously, scores from the PDSQ are then used to facilitate the initial diagnostic evaluation. The author notes that "results should be verified whenever possible against all available information, including the results of patient interviews, clinical history, professional consultations, service agency records, and the results of additional psychological tests" (manual, p. 11).

DEVELOPMENT. The PDSQ was developed to be a relatively brief, self-administered questionnaire for the assessment of various DSM-IV Axis I disorders in psychiatric patients. Development of the measure began over 10 years ago with an instrument entitled the SCREENER, which was originally designed to screen for psychiatric disorders in primary care settings and later in outpatient mental health settings. Following subscale revisions, the SCREENER became a 102-item version of the PDSQ. Through additional modifications the PDSQ took its present form as a scale of 125 items.

TECHNICAL. The author stresses the importance of patients being able to understand any self-administered instrument. As such, readability studies of the initial version of the PDSQ were conducted and ranged from a 5.8 grade level (Flesch-Kincaid method) to a 9.2 grade level (Bermuth formula). Additional understandability studies using psychiatric outpatients demonstrated that PDSQ items were "written at a level that most individuals ... would understand" (manual, p. 27). The author acknowledges that one-third of the sample patients were college graduates and only 5% of the sample patients had less than a high school diploma.

Initial and replication studies were conducted to estimate internal consistency and test-retest reliability on 112- and 139-item versions of the PDSQ. Samples were large, but dominated by white, married or single, and educated females. Internal consistency values (Cronbach alpha) for the initial study on 732 psychiatric outpatients ranged from .73 (Somatization Disorder) to .95 (Drug Abuse/Dependence), whereas a replication study involving 994 psychiatric outpatients found internal consistency estimates to range from .66 (Psychosis and Somatization Disorder) to .94 (Posttraumatic Stress Disorder). Test-retest reliability coefficients on a subsample of these patients ranged from .66 (Bulimia/Binge-Eating Disorder) to .98 (Drug Abuse/Dependence) for the initial study (mean interval of 4.8 days) and from .61 (Mania/Hypomania) to .93 (Drug Abuse/Dependence) in the replication study (mean interval of 1.6 days).

The author reports that 27 of the 112 items did not achieve a minimum endorsement base rate of 5% during the initial study and were not used to determine test-retest reliability. Eighty-three of the 85 remaining items had a Cohen's kappa coefficient, which corrects for chance levels of agreement, between .67 and .92. In the replication study, only two items were excluded in the test-retest reliability study. Cohen's kappa for the remaining items ranged from .50 to .83. Although there is some disagreement regarding the interpretation of kappa, Spitzer, Fleiss, and Endicott (1978) suggest that values greater than .75 demonstrate good reliability, values between .50 and .75 suggest fair reliability, and values below .50 connote poor reliability. In the initial study, 7 subscales (Major Depressive Disorder, Dysthymic Disorder, Bulimia/Binge-Eating Disorder, Mania/Hypomania, Agoraphobia, Generalized Anxiety Disorder, and Hypochondriasis) would be considered to have fair reliability and 7 (PTSD, Obsessive-Compulsive Disorder, Panic Disorder, Psychosis, Social Phobia, Alcohol Abuse/Dependence, and Somatization Disorder) would be considered to have good reliability (1 subscale did not meet the base rate standard). In the replication study, 14 subscales would be considered to have fair reliability and 1 (Drug Abuse/Dependence) would be considered to have good reliability.

To document discriminant and convergent validity, corrected item/subscale total correlation coefficients were calculated between each item and subscale. The mean of the correlations between each subscale item and that subscale's total score were compared to the mean of correlations between each subscale item and the other 14 subscale scores. The author points out that in 90.2% of the calculations the item/parent-subscale correlation was higher than each of the item/other-subscale correlations. A similar pattern emerged from the replication study with 97.1% of items having a higher correlation with their parent subscale. Data are not provided on correlations between each subscale mean and other individual subscales within the PDSQ.

The PDSQ subscales were also compared to "other measures of the same construct versus measures of different constructs" (manual, pp. 31-32). In all instances, the PDSQ subscale scores were significantly correlated with measures of similar syndromes. In addition, correlations were higher between scales assessing the same symptom domain than scales assessing other symptom domains. Interpretation is somewhat clouded by the manual's lack of clarity regarding the nature of these measures.

Finally, criterion validity was documented by comparing the scores of respondents with and without a particular DSM-IV diagnosis. In both the initial and replication studies, the average PDSQ score was significantly higher for those with versus those without the disorder (the only exception was Mania-Hypomania, which was subsequently dropped from the PDSQ).

Cutoff scores are provided based on a study of 630 psychiatric outpatients who were interviewed with the Structured Clinical Interview for DSM-IV Axis I Disorders (SCID; First, Spitzer, Gibbon, & Williams, 1997). Based on results from this study and the fact that the PDSQ is intended to be used as an aid for conducting an initial diagnostic evaluation, the author has recommended a cutoff score resulting in diagnostic sensitivity of 90%. These cutoff scores are provided on the PDSQ Summary Sheet. In addition, a table within the manual includes cutoff scores, sensitivity, negative and positive predictive values, and separate columns estimating the rates of occurrence among psychiatric patients and the general population-the latter being based on information obtained from the DSM-IV.

Limited data are provided within the manual on the PDSQ Total Score. The author states that it is the only norm-referenced score in the instrument. The Total Score is expressed as a standard T-score and is a means for "comparing the patient's level of symptom endorsement with that of the average patient seen for intake in a clinical psychiatric outpatient setting" (manual, p. 11). Apparently, it provides a "rough measure of the overall level of psychopathology and consequent dysfunction that a patient reports" (manual, p. 11). However, the author states that it is only loosely related to the distress a patient may be experiencing and it should not be used as an index of severity.

COMMENTARY. The purpose of the PDSQ is to screen for DSM-IV Axis I disorders that are most commonly seen in outpatient mental health settings. With any measure such as this, the real question is: Is it accurate and does it improve efficiency? In regard to accuracy, the PDSQ has respectable internal consistency and test-retest reliability. In addition, convergent and discriminate validity studies demonstrate that PDSQ items are correlated more strongly with their parent subscale than with other subscales within the PDSQ. Also, the PDSQ items were more strongly correlated with other measures of the same construct versus measures of different constructs; although the manual is somewhat unclear as to the nature of these "measures." Finally, it appears as though the PDSQ has decent sensitivity and specificity and does well at identifying both principal and comorbid disorders. A problem, however, is that the PDSQ has no validity indices, thereby allowing patients to misrepresent themselves on the instrument. Any interpretation should, therefore, be done cautiously and with corroborating information.

In regard to the question of efficiency, the author admits that this, as well as the issue of accuracy, remain empirical questions. Despite the lack of supportive data within the manual, the PDSQ does appear to readily guide the interview toward symptom areas requiring more detailed assessment. In and of itself, this should streamline the diagnostic interview.

Potential PDSQ users are cautioned about several other areas. The first relates to the samples used in studying the PDSQ. Although numbers are typically adequate, the generalizability of findings are somewhat limited by rather homogeneous (i.e., mostly white, female, married/single, and well-educated patients) samples used within the various studies. Finally, users of the PDSQ are reminded of the fairly high reading level necessary for self-administration and the lack of validity indices within the instrument.

SUMMARY. The author should be commended for developing a self-report screening measure that is relatively easy to administer and score and has acceptable evidence of reliability and validity. As noted by the authors, the PDSQ "is not a substitute for a diagnostic interview .... There are no special questions on the PDSQ that allow it to detect psychopathology that otherwise would go undetected during a clinical evaluation" (Zimmerman, 2003, p. 284). Nonetheless, the PDSQ will likely guide clinicians toward those areas of clinical concern that need additional assessment. In doing this, the PDSQ should serve its intended purpose of increased clinical diagnostic accuracy and efficiency. Additional studies will need to be completed to determine the overall impact of the PDSQ on these issues and whether it leads to improved treatment outcome.

REVIEWER'S REFERENCES

American Psychiatric Association. (1994). Diagnostic and statistical manual of mental disorders (4th ed.). Washington, DC: Author.

First, M. B., Spitzer, R. L., Gibbon, M., & Williams, J. B. W. (1997). Structured clinical interview for DSM-IV Axis I Disorders (SCID). Washington, DC: American Psychiatric Association.

Spitzer, R. L., Fleiss, J. L., & Endicott, J. (1978). Problems of classification: Reliability and validity. In M. A. Lipton, A. DiMarco, & K. Killam (Eds.), Psychopharmacology: A generation of progress (pp. 857-869). New York: Raven.

Zimmerman, M. (2003). What should the standard of care for psychiatric diagnostic evaluations be? Journal of Nervous and Mental Disease, 191, 281-286.

Review of The Psychiatric Diagnostic Screening Questionnaire by SEAN P. REILLEY, Assistant Professor of Psychology, Morehead State University, Morehead, KY:

DESCRIPTION. The Psychiatric Diagnostic Screening Questionnaire (PDSQ) consists of 125 items (111 numbered items, 2 with multiple parts) that tap symptoms of several DSM-IV Axis I disorders commonly seen in outpatient settings. The PDSQ can be completed on-site in as little as 20 minutes or at home in advance of an appointment. Respondents use one of three time frames (past 2 weeks, past 6 months, lifetime recollection) to specify the presence ("Yes") or absence ("No") of symptoms. Responses can be rapidly handsummed into raw subscale scores and converted to t-scores by clinicians and appropriately trained staff. The inventory yields a total score and 13 subscale scores, denoted in brackets, which tap mood [Major Depressive Disorder], anxiety [Posttraumatic Stress Disorder, Obsessive Compulsive Disorder, Panic Disorder, Agoraphobia, Social Phobia, Generalized Anxiety Disorder], eating [Bulimia/Binge-Eating Disorder], somatoform [Somatization Disorder, Hypochondriasis], substance abuse/dependence problems [Alcohol Abuse/Dependence, Drug Abuse/Dependence], and psychotic [Psychosis] symptoms. Summary sheets assist with identification of 45 possible critical items and comparison of subscale scores with recommended clinical cutting scores. A compact disc containing follow-up interview guides with prompts related to DSM-IV criteria is available from the test publisher for each subscale.

DEVELOPMENT. The PDSQ is an atheoretical inventory. Items were written to reflect symptom criteria for DSM-IV Axis I disorders most common in epidemiological surveys and in published research articles. Most items adequately represent the DSM-IV nosology except those comprising the Alcohol and Substance Abuse/Dependence and the Psychosis subscales. The former reflect abuse/dependence symptoms broader than those required by the DSM-IV, whereas the latter assess critical symptoms of several nonspecific psychotic disorders. During the item revision process, 89% of items successfully passed four criteria established by the developer. The addition of new and revised items was not successful in meeting five additional subscale retention criteria for: Anorexia Nervosa, Body Dysmorphic Disorder, Mania/Hypomania, Dysthymic Disorder, Generalized Anxiety Disorder, Psychosis, Somatization, and Hypochondriasis subscales. The latter four subscales were retained, however, partially based on adequate diagnostic performance with outpatient clinical groups, but could benefit from further modification. The 13 subscales comprising the current version of the PDSQ contain uneven item distributions ranging from 5 to 22 items. This, in addition to a lack of items to assess response bias, raises concern. Namely, endorsement of a single transparent item for some subscales is sufficient to exceed the clinical screening criterion, which could potentially lower their positive predictive power. The developers do offer practical suggestions for detecting response bias using the PDSQ Total score. However, this indicant is norm referenced and not criterion referenced like subscale scores. Thus, prior to acceptance, bias detection procedures using the PDSQ Total score need empirical validation with clinical groups.

TECHNICAL. Multiple, adult, medical, and psychiatric outpatient samples, of at least four hundred individuals (over 3,000 combined) per sample, were used to standardize the PDSQ. Although certainly commendable, these data are predominantly from Caucasian (range 85 to 94%) high school graduates (89 to 94%) living in Providence, RI. The impact of gender on PDSQ norms is not reported, despite women outnumbering men by a 2:1 ratio in all standardization samples. Because several DSM-IV syndromes tapped by the inventory show marked gender differences (e.g., Major Depressive Disorder), gender impact studies are needed. Perhaps more salient is the need for a broader and more representative normative sample to improve the generalizability of the PDSQ for diverse populations including rural, multi-ethnic, and geriatric adults, as well as those with lower education and/or socioeconomic status.

Readability analyses using the Flesch-Kincaid and Bermuth methods indicate PDSQ items range from fifth to ninth grade reading levels. Studies employing simpler forced-choice procedures (Understand/Don't Understand) suggest greater than 95% of adults with a high school degree or equivalency understood all PDSQ items. Despite these initial data, no minimum level of reading skill is recommended in the manual.

Reliability estimates are reported for previous PDSQ versions that include items and subscales not found on the present version. Extrapolating from these data, the Cronbach alpha coefficients of subscales common to the current PDSQ are adequate (.66) to excellent (.94). To date, internal consistency estimates have been used to estimate the latent variables comprising the PDSQ. No attempts are reported to validate its primary factor structure. This could be accomplished using advanced modeling procedures such as confirmatory factor analysis or structural equation modeling. Test-retest estimates for approximately a week are borderline adequate (kappa = .56) to excellent (kappa = .98) at the item level, and slightly higher at the subscale level (rs ranging .72 to .93). Studies involving longer test-retest durations are needed to bolster initial data, given the longer temporal requirements of several DSM-IV syndromes (e.g., Major Depression; 2 weeks, PTSD; 4 weeks) tapped by the PDSQ.

Data concerning convergent and discriminant validity of the PDSQ are based on initial outpatient (n = 732) and replication samples (n = 994) using multiple methods. Across studies, the mean corrected item-parent PDSQ subscale correlations (rs ranging .42 to .85) are significantly higher than 90% of those afforded by item-other PDSQ subscale relations (rs ranging .15 to .35). Subscale-specific correlations with externally recognized instruments are modest (r = .25) to very good (r = .77), and higher than those afforded by nonspecific PDSQ subscales (rs ranging .15 to .35). Thus, initial internal and external comparisons of PDSQ subscales suggest appropriate convergence and discriminant validity. Adequate criterion validity is initially examined by showing significantly higher diagnosis-specific PDSQ subscale scores (e.g., Major Depressive Disorder) among outpatient groups with the corresponding DSM-IV disorder than for those without the disorder. Absent from the manual, however, are comparisons of non-diagnosis-specific PDSQ scores between outpatient groups. Inclusion of these data could further bolster the criterion validity evidence of the PDSQ.

The initial sensitivity, specificity, and positive and negative predictive power of PDSQ cutting scores for primary DSM-IV diagnoses are based on a single sample of psychiatric outpatients (n = 630). Subscale sensitivity is generally adequate (75%) to very good (100%) in this sample with less variability noted for rates of specificity (range 83 to 100%). The developer recommends a sensitivity level of 90% for establishing cutting scores for clinical practice. However, four subscales, Obsessive Compulsive Disorder (89%), Psychosis (75%), and both Alcohol (85%) and Drug (85%) Abuse/Dependence, fail to reach this sensitivity level. Using the most liberal cutoff scores, positive predictive values range considerably (18 to 100%), whereas negative predictive values are high and fairly consistent (97 to 100%). Seven subscales yield positive predictive values below 60%, which, in part, may be due to low base rates of disorders tapped by Bulimia/Binge Eating Disorder, Somatization Disorder, Hypochondriasis, and Psychosis subscales. However, as noted, Drug Abuse/Dependence and Psychosis subscales provide less than an adequate mapping of the DSM-IV nosology, which may negatively impact their predictive ability. Finally, the differential validity evidence of all PSDQ cutting scores needs to be clarified for gender and diversity considerations.

COMMENTARY. The PDSQ appears to be a potentially valuable screening instrument for common DSM-IV Axis I disorders in outpatient settings. Several issues need to be addressed in order to firmly anchor the psychometrics and generalizability of the PDSQ. First, a more representative standardization sample needs to be collected using the current version of the PDSQ. In that sample, gender and diversity contributions to PDSQ scores need clarifying and a minimum reading level should be established. Second, response bias needs to be addressed either by inclusion of new items or additional studies designed to empirically validate bias detection techniques using the PDSQ Total Score. Third, factor analysis or structural equation modeling is needed to adequately assess the overall PDSQ factor structure and to address less than adequate homogeneity in several subscales. Fourth, longer test-retest studies are needed to bridge the existing stability of several subscales with specific temporal requirements of several selected DSM-IV disorders. Finally, the positive predictive power of Psychosis, Bulimia/Binge Eating Disorder, Generalized Anxiety Disorder, Somatization Disorder, Hypochondriasis, and Alcohol and Drug Abuse/Dependence subscales needs to be improved.

SUMMARY. The developer, to his credit, has produced a potentially valuable screening instrument, and one of the first that directly incorporates the DSM-IV nosology for common Axis I disorders. Significant care was taken in initial studies to evaluate PDSQ items and subscales using multiple reliability and validity indices. In order for this instrument to become a gold standard, a more representative standardization sample is needed. Careful, continued validation work will also be required to solidify the PDSQ factor structure, to enhance the homogeneity and test-retest reliability of specific subscales, and to improve their positive predictive power. As a whole, this inventory is recommended for screening purposes with an eye to its current limitations.