Anxiety Disorders

A p p lic a t io n s o f A s s e s s m e n t

P a t h A n a l y s i s o f t h e S C L - 9 0 - R :

E x p l o r i n g U s e i n O u t p a t i e n t

A s s e s s m e n t

Todd L. G rande1, M ark D. N ew m eyer2, Lee A. Underwood2, and Cyrus R. W illiam s III 2

A b s t r a c t

The Symptom Checklist-90-Revised (SCL-90-R) is a widely used assessment of mental health pathology; however, its factor structure has been called into question by numerous studies. This study assessed a community mental health outpatient sample (N = 336) with the SCL-90-R and analyzed the factor structure. The results indicated that the SCL-90-R measures one large factor, but the test items held together reasonably well when a nine-factor extraction was executed. A shorter 67-item variant, which was a by-product of this study, is hypothesized as having some key advantages over the original 90-item version. Implications fo r the assessment of the outpatient population with the SCL-90-R and its variants are discussed.

K e y w o r d s

factor analysis, SCL-90-R, outpatient, community mental health

M e a s u r e m e n t a nd E v a lu a tio n in

C o u n s e lin g a n d D e v e lo p m e n t

2 0 1 4 , V o l. 4 7 ( 4 ) 2 7 1 - 2 9 0

© T h e A u t h o r ( s ) 2 0 1 4

R e p r in ts a n d p e rm is s io n s :

s a g e p u b .c o m /jo u rn a ls P e rm is s io n s .n a v

D O I : 10 . 117 7 / 0 7 4 8 17 5 6 14 5 3 8 0 6 1

m e c .s a g e p u b .c o m

®SAGE

The Symptom Checklist-90-Revised (SCL- 90-R; Derogatis, 1994) has been a popular assessment for psychopathology in a variety o f clinical settings, including psychiatric out­ patient and inpatient settings. The 90-item self-report SCL-90-R was designed to mea­ sure nine distinct dimensions o f psychopa­ thology (Derogatis & Cleary, 1977a, 1977b). However, this purported nine-factor structure, the instrument’s general utility, and its useful­ ness as a diagnostic aid have been repeatedly challenged (Bonynge, 1993; Brophy, Norvel, & Kiluk, 1988; Cyr, McKenna-Foley, & Pea­ cock, 1985; Pederson & Karterud, 2004; Rauter, Leonard, & Swett, 1996). The instru­ ment’s predecessor, the SCL-90 (Derogatis, Lipman, & Covi, 1973), which like the SCL- 90-R was designed to measure nine factors, has experienced similar questions, scrutiny, and evaluation to determine what mental health characteristics it actually captures (Hoffman & Overall, 1978).

The SCL-90-R is a 90-item self-report instrument designed to assess mental health symptoms across nine subscales generally associated with mental health pathology and three global scales (Derogatis, 1992). The nine subscales o f the SCL-90-R include (a) Somati­ zation, (b) Obsessive Compulsive, (c) Interper­ sonal Sensitivity, (d) Depression, (e) Anxiety, (f) Hostility, (g) Phobic Anxiety, (h) Paranoid Ideation, and (i) Psychoticism (Derogatis, 1992). The three global scales are the Global Severity Index (GSI), the Positive Symptom Distress Index (PSDI), and the Positive Symp­ tom Total (PST; Derogatis, 1992). Respon­ dents are asked to rate the severity of their

'W ilm in g to n U niversity, N e w a rk, DE, USA 2Regent U niversity, Virginia Beach, VA , USA

C o r r e s p o n d i n g A u t h o r :

T odd L. Grande, W ilm in g to n U niversity, 320 N D uP ont Highway, N e w Castle, DE 19720, USA. Email: toddgrande@ gm ail.com

272 M easurem ent and Evaluation in Counseling and Developm ent 4 7 (4 )

symptoms on a scale o f 0 to 4 (0 = not at all, 1 = a little bit, 2 = moderate, 3 = quite a bit, or 4 = extreme', Derogatis, 1992). The instrument has been found to have high construct validity as well as high concurrent validity with simi­ lar instruments (Derogatis & Cleary, 1977a). The SCL-90-R can be administered in a paper-and-pencil format or through a com­ puter-based method (Schmitz, Hartkamp, Brinschwitz, Michalek, & Tress, 2000). After assessing a sample o f psychosomatic outpa­ tients (n = 282), Schmitz, Hartkamp, Brin­ schwitz, et al. (2000) found noticeable, but small, differences between these two SCL- 90-R administration delivery methods.

S ta te m e n t o f th e P roblem

Contemplating the number o f studies chal­ lenging the factor structure and general utility of the SCL-90-R and this instrument’s wide­ spread use and popularity in a variety o f clini­ cal settings, there appears to be a clear need to determine the true factor structure and utility of the SCL-90-R and to develop implications for this instrument’s use. A variety o f studies have demonstrated that the factor structure of the SCL-90-R is not consistent with the origi­ nal nine-dimension design and how a great deal o f variance is explained by the first extracted factor. Although a number o f these studies have suggested areas in which and cir­ cumstances under which this instrument can be successfully deployed (e.g., brief screening instrument), few provide detailed information on how individual items and groups of items can be interpreted to maximize utility o f the SCL-90-R. Additionally, there is a paucity o f SCL-90-R research regarding assessment with community mental health outpatient samples and with those diagnosed with co­ occurring disorders.

Numerous researchers have called for further experiments and analyses o f the SCL-90-R characteristics, including the evaluation o f specific populations, disorders, and research targeted toward the discrimina­ tory properties o f the subscales (Buckelew, Burk, Brownlee-Duffeck, Frank, & DeGood, 1988; Eich et al., 2003; Elliott et al., 2006;

Gilliss, Moore, & Martinson, 1997; Kaplan et al., 1998; Recklitis, Licht, Ford, Oeffinger, & Diller, 2007). The research questions explored by the current study correspond to the calls for further research in other studies both broadly and in many specific facets. Recklitis et al. (2007) noted that using only one criterion measure in their study was a major limitation, whereas the current study uses several. The importance o f identifying new subscales in the SCL-90-R is a key component o f this study, which addresses the call for further research issued by Gilliss et al. (1997).

Buckelew et al. (1988) and Kaplan et al. (1998) stressed the importance of analyzing and interpreting the individual item responses o f the SCL-90-R to aid clinicians in making accurate diagnoses and correctly interpreting mental health assessments, which is congruent with the design and objectives of the study. The study’s sample includes participants who suffer from severe pathology, including substance abuse, psychosis, personality disorders, and severe depression and anxiety. Elliott et al. (2006) noted the importance that future research use participants who suffer from severe mental health conditions. Eich et al. (2003) cautioned that the SCL-90-R carries the risk of underesti­ mating the prevalence of severe psychopathol­ ogy, which complicates research studies such as those suggested by Elliott et al. (2006). The set­ ting o f this study allowed for the evaluation of participants with co-occurring disorders, which is a population that is only represented in a pau­ city of studies using SCL-90 or SCL-90-R and who could benefit from further research (e.g., Carpenter & Hittner, 1995; Zack, Toneatto, & Streiner, 1998).

Purpose o f this Study

The purpose of this study comprises the answering o f several research questions related to the factor structure and utility of the SCL-90-R. One purpose is to determine, eval­ uate, and explore the factor structure of SCL- 90-R using a community mental health outpatient sample (n = 336), which includes participants who suffer from severe psycho­ pathology and co-occurring disorders. This

G ra n d e e t at. 273

purpose is consistent with the call for addi­ tional research made by numerous researchers who have conducted a variety o f studies with the SCL-90-R (Cyr et al., 1985; Pederson & Karterud, 2004; Rauter et al., 1996). The size o f this sample (n = 336) is sufficient to draw inferences regarding the factor structure o f the SCL-90-R using exploratory factor analyses, according to standards established regarding the total sample size. Comrey and Lee (1992) noted that a sample size of 300 was “good” on their scale, which ranged from “poor” (100 cases) to “excellent” (1,000 or more cases). The research design includes the collection of data using the SCL-90-R as well as other pop­ ular psychometric measurements, so that the concurrent validity o f this instrument can be explored. The results o f the exploratory factor analysis are used to explain and understand the factor structure o f the SCL-90-R and to postulate a shorter and more clearly defined variant o f the original SCL-90-R.

A second purpose is to determine the nature and usefulness o f the SCL-90-R variant that is uncovered by repeated exploratory factor analyses. This variant is analyzed with path analysis, and the underlying constructs are identified. The type and strength o f the rela­ tionships among the variables, namely the new factors revealed in the variant and the underlying constructs, is determined and quantified using path analysis. An improved model is developed in light o f the model fit indices, and this model is compared to the hypothesized model.

The description and relationship o f the underlying constructs is used to offer practical advice on how to use and interpret the indi­ vidual items and item groupings from this instrument and the proposed variant. The util­ ity and factor structure o f both the original instrument and the proposed variant are dis­ cussed in terms o f the implications for com­ munity mental health agency settings, including suggestions on how to interpret individual items and groups o f items o f these instruments. The challenges o f assessing and treating community mental health outpatient clients using the SCL-90-R and the proposed variant are discussed.

Usage o f th e SCL-90-R

If the true factor structure of the SCL-90-R does, in fact, vary greatly from its intended design as suggested by numerous studies, the potential effect on the clinical community may be significant (Brophy et al., 1988; Cyr et al., 1985; Gilliss et al., 1997; Pederson & Kar­ terud, 2004; Rauter et al., 1996; Schmitz, Hartkamp, Kruse, et al., 2000; Steer, Clark, & Ranieri, 1994). Pearson Education (2009), which distributes the SCL-90-R, published a bibliography for the SCL-90-R that reflected works published from the instrument’s incep­ tion through 2007. According to this bibliog­ raphy, more than 400 articles had been written that referenced the SCL-90-R and the SCL-90 (Pearson Education, 2009). Myriad studies have been conducted using the SCL-90-R as the only dependent variable or have used it in conjunction with other instruments. Further­ more, the SCL-90-R has been deployed in an effort to determine the presence and severity o f numerous mental health symptoms and dis­ orders and has been used to assess a large number o f participants from a wide variety of cultural, ethnic, geographic, medical health, and socioeconomic categories.

The popularity o f the SCL-90-R has been evident through the numerous and varied types o f studies in which it has been featured. The instrument has been deployed to gain knowledge on topics that included effective outpatient screening (Holi, Marttunen, & Aal- berg, 2003), mental health symptoms in sub­ stance abusers (Jansson, Hesse, & Fridell, 2008; Maremmani et al., 2010; Numan, 2004; Parrott, Milani, Pannar, & Turner, 2001), patients with psychopathological symptoms due to medical conditions (Kaplan et al., 1998; Siri et al., 2010; Torres et al., 2010), the instrument’s ability to distinguish between depressive and anxiety disorders (Kennedy, Morris, Pedley, & Schwab, 2001; Tomassini et al., 2009; Yong Woo et al., 2009), and the SCL-90-R’s reliability and validity in a multi­ cultural context (Barker-Collo, 2003; Marti­ nez, Stillerman, & Waldo, 2005). Other studies have examined the SCL-90-R with reference to assessing patients with panic disorder

274 M easurem ent and Evaluation in Counseling and Developm ent 4 7 (4 )

(Masdrakis, Papakostas, Vaidakis, Papageor- giou, & Pehlivanidis, 2008), measuring the effectiveness o f pharmacotherapy and psy­ chotherapy combined on depression (Mole- naar et al., 2007), evaluating the validity o f other instruments (Recklitis et al., 2007), and measuring the effects o f physical activity on depression (Ryan, 2008). Less researched areas o f interest have also been studied with this instrument, including childhood abuse (Barker-Collo & Read, 2011), excessive Inter­ net use (Chang-Kook, Byeong-Moo, Baity, Jeong-Hyeong, & Jin-Seok, 2005), participa­ tion patterns in research studies (Eich et al., 2003), adolescent inpatients (Hart, Bryer, & Martines, 1991), college students external locus o f control (Holder & Levi, 1988), and inpatients who committed serious acts o f vio­ lence (Bjorkly, 2002).

The SCL-90-R has been studied with regard to its use with other instruments, selec­ tive use o f subscales, evaluative accuracy, and assessment o f disorders. The SCL-90-R has been used in conjunction with other instru­ ments— in some cases to evaluate the validity o f other measurements (Chang-Kook et al., 2005; Recklitis et al., 2007). On a few occa­ sions, researchers have assessed participants using one or more o f the subscales o f the SCL-90-R, but avoided using the entire instru­ ment for their assessment. This utility o f the selective use o f the subscales was often sup­ ported by the results (Holder & Levi, 1988; Ruwaard et al., 2009; Ryan, 2008). Underre­ porting or underestimating o f severe pathol­ ogy, the possible low discrimination value, and the possibility o f flawed norms of the SCL-90-R has been reported (Bjorkly, 2002; Eich et al., 2003; Hart et al., 1991). Various mental health disorders have been assessed with the SCL-90-R including but not limited to (a) depression, (b) anxiety and panic, (c) personality disorders, (d) ADHD, (e) obses­ sive-compulsive disorders, and (f) dissocia­ tive disorders (Barkley, Murphy, & Kwasnik, 1996; Jansson et al., 2008; Kennedy et al., 2001; Kirkcaldy, Fumham, & Siefen, 2010; Masdrakis et al., 2008; Molenaar et al., 2007; Steinberg, Barry, Sholomskas, & Hall, 2005; Tomassini et al., 2009; Yong Woo et al., 2009).

The SCL-90 has been used to study partici­ pants who are diagnosed with substance depen­ dence or abuse or who are at risk for such a diagnosis (Johnson, Bomstein, & Sherman, 1996; Maremmani et a l, 2010; Numan, 2004; Parrott et al., 2001), with most studies indicat­ ing the SCL-90 was able to detect relationships between substance abuse and other mental health symptoms. The SCL-90 has been used to assess participants who are diagnosed with co-occurring disorders (simultaneous sub­ stance abuse and nonsubstance abuse diagno­ ses), and analysis of these assessments have produced similar findings when compared with studies of nonsubstance abusing participants (Carpenter & Hittner, 1995; Zack et al., 1998).

The SCL-90-R’s validity and usefulness have been tested using participants from a variety o f cultures and was generally found to be appropriate for use in a wide variety of cul­ tural contexts (Barker-Collo, 2003; Barker- Collo & Read, 2011; Martinez et al., 2005, Plante, Manuel, Menendez, & Marcotte, 1995; Qouta, Punamaki, & El Sarraj, 2005). The findings in studies that examine various multi­ cultural applications are consistent with results reported from research that examined samples similar to the norm group of the SCL-90-R.

Several abbreviated versions o f the SCL- 90-R have been developed and used in various studies and clinical settings (Hardt, Dragan, & Kappis, 2011; Kuhl et al., 2010). Some o f the short versions o f the SCL-90-R include the SCL-27, SCL-25, SCL-21, SCL-10, SCL-6, SCL-5, BSI-18, and BSI, all o f which have demonstrated value for specific mental health assessment purposes (Asner-Self, Schreiber, & Marotta, 2006; Cepeda-Benito & Gleaves, 2000; Derogatis, 1993, 2000; Hardt & Ger- bershagen, 2001; Piersma, Boes, & Reaume, 1994; Strand, Dalgard, Tambs, & Rognerud, 2003; Tate, Kewman, & Maynard, 1990).

Factor Structure a n d U tility

Both the factor structure and the usefulness of the SCL-90 and SCL-90-R have been called into question by some studies (Pederson & Karterud, 2004; Rauter et al., 1996) yet sup­ ported by others (Evenson, Holland, Mehta, &

Grande et al. 275

Yasin, 1980). Although a substantial number o f studies have supported a unidimensional factor structure for these instruments (Cyr et al., 1985; Zack et al., 1998), many studies have supported these instruments’ ability to discriminate between various mental health symptoms despite the presence o f a factor structure that is incongruent with the original design specifications (e.g., Paap et al., 2011; Schmitz, Hartkamp, Kruse, et al., 2000).

The results o f several factor analyses dem­ onstrated variations in the factor structure and loading o f the SCL-90-R, but most studies provided results that are indicative o f the pres­ ence of a primary unidimensional factor that accounts for significantly more item variance than any o f the subsequent factors (Brophy et al., 1988; Cyr et al., 1985; Gilliss et al., 1997; Schmitz et al., 2000; Steer et al., 1994). Although these studies have the identification o f a unidimensional factor in common, there is some disagreement among about how this affects the overall utility o f the SCL-90-R and how alternate versions of this instrument may address these concerns. A series of studies have attempted to use the analysis o f the SCL- 90-R to support the introduction o f differently structured subscales, items, and item group­ ings. The findings have not pointed to one consistent and accepted refinement, rather several different subscales and item groupings have been identified across many studies (Buckelew et al., 1988; Elliott et al., 2006; Paap et al., 2011).

The SCL-90-R M e a s u re m e n t U tility

Some research has suggested that the best use o f the SCL-90-R is to assess a general level of mental health symptomology, stress, or dis­ comfort (Clark & Friedman, 1983; Elliott et al., 2006; Schwarzwald, Weisenberg, & Solo­ mon, 1991), while many studies have sug­ gested that the SCL-90-R should not be deployed as a primary instrument to form diagnoses or to accurately assess or predict mental symptoms or disorders (Cyr et al., 1985; McGough & Curry, 1992; O ’Donnell, DeSoto, & Reynolds, 1984; Pederson & Kar- terud, 2004; Wegner, Rabiner, & Kane, 1985).

Several studies concluded that the SCL-90 or SCL-90-R contained significant limitations, however, could prove to be a useful screening assessment for psychiatric disorders (Dinning & Evans, 1977; Schmitz, Kruse, Heckrath, Alberti, & Tress 1999; Holi et al., 2003). Not all the studies o f the SCL-90 or SCL-90-R placed an emphasis on the existence o f a unidi­ mensional primary factor, rather various other factors have been identified, but no consistent theme has been established (Arrindell, Barelds, Janssen, Buwalda, & Van Der Ende, 2006; Evenson etal., 1980; Hafkenscheid, 1993).

Research Q uestions

The popularity and extensive use o f the SCL- 90-R necessitates that the instrument be well understood by the mental health clinicians and supervisors who use it to inform their decisions regarding client treatment. The cur­ rent study is designed to answer the following research questions:

Research Question 1: What is the factor structure o f the SCL-90-R? Research Question 2: Is there a useful variant that can be generated from this study’s sample? Research Question 3: If there is a variant, what are the implications for its use and interpretation for mental health clinicians?

Only with a firm understanding o f the fac­ tor structure of this instrument will mental health professionals be able to make consis­ tent, empirically based, and accurate deci­ sions regarding client assessment with the SCL-90-R. This study not only addresses this pressing need in the mental health counseling community but also identifies a shorter and more precise variant o f this instrument. The data used to identify this variant were gath­ ered from a population that is underrepre­ sented in research regarding the SCL-90-R. A detailed description of the factors extant in this variant as well as an explanation about how the factors relate to one another is pro­ vided for the benefit o f mental health clini­ cians and clients they assess.

276 M easurem ent and Evaluation in Counseling and Developm ent 4 7 (4 )

Method

The method of the study includes components commonly observed in previous research stud­ ies involving the factor structure of the SCL- 90-R. A significant quantity o f participants were sampled (n = 336), assessed using the SCL-90 R, and the data were analyzed using a factor and path analysis. The examination of data yielded by the SCL-90-R using a factor analysis has been a method identified in several research studies (e.g., Brophy et al., 1988; Clark & Friedman, 1983; Gilliss et al., 1997; Rauter et al., 1996; Steer et al., 1994). Simi­ larly, several studies have used the confirma­ tory factor analysis to analyze data from the SCL-90-R; however, the use of a path analysis has not been as common. The proposal o f a shorter and more accurate variant including the precise breakdown o f items, as demonstrated in this study, has not been commonly observed in prior research studies with the community mental health outpatient population. The ana­ lytical methodology of this study is more robust than the methodologies typically featured in studies, and the sample observed in this study has not been widely researched as it relates to the use of the SCL-90-R. Both the factor and path analyses are statistical processes that are well-suited to contribute to the fulfillment of this study’s purpose.

Participants

The participants for this study were outpatient mental health clients being treated at a com­ munity mental health agency in the Mid- Atlantic region of the United States. A group of psychometric assessments, collectively referred to as the Assessment Battery, was administered to every client who came in for specified mental health services during the month o f July 2011 and again during the month o f October 2011, except for those cli­ ents who refused to take the Assessment Bat­ tery or those who were incapable. The number o f unique participants assessed in July 2011 (n = 243) was greater than the number assessed in October 2011 (n = 93), resulting in a total sample size o f 336 unique participants for this

study (59 participants were assessed in both July and October, so only their July scores were considered for the purposes o f this study). The average age o f the participants was 42 years, and females ( n = 210; 62.5%) outnumbered males (n = 126, 37.5%). Partici­ pants receiving ongoing services from the agency numbered 236 (70%), while 100 par­ ticipants were assessed at intake (30%).

Given the total number of participants being treated at the agency at the end of Octo­ ber 2011 (n = 994), this project assessed approximately 34% o f agency’s clients. O f the total number of clients asked to complete the Assessment Battery, 95 refused and approxi­ mately 20 failed to report to the administra­ tion area, effectively indicating a passive refusal. Additionally, some Assessment Bat­ teries were not admitted due to clients being deemed incapable o f completing the battery (n = 18) due to cognitive deficits, apparent alcohol or substance-induced intoxication, or severe psychoticism. Several participants’ scores had to be discarded because they did not complete any or part o f the Assessment Battery properly ( n = 42).

The community mental health agency, from which the sample is drawn, treats a wide variety o f clients who present with diverse pathology. Common client profiles included pathological elements such as Bipolar Disor­ der, Depression, Anxiety, Panic, Substance Dependence, and Schizophrenia. The agency placed an emphasis on the treatment of co­ occurring mental health and substance-abuse disorders, and these disorders were overrepre­ sented in the client sample (slightly greater than 50%). Serious and chronic diagnoses on Axis III were common, but no one diagnosis was overrepresented in this sample. Axis IV elements that were often observed in this set­ ting were the following: (a) low or no income, (b) no employment, (c) housing problems, (d) poor social skills, (e) legal trouble, (f) limited support from family or friends, and (g) low educational level. The average Global Assess­ ment o f Functioning score (Axis V) was 50.29; however, there were too many missing values in this category to assert the accuracy o f a score on this dimension.

Grande et al. 277

Consistent with agency policy and best practices, clients were typically seen individ­ ually by their counselors once every 2 or 3 weeks and by their psychiatrists once a month. Many clients attended one group-session every week. Groups offered include cogni­ tive-behavioral therapy, addiction treatment, an intensive outpatient program, and creative expression. The therapeutic treatment pro­ vided by this facility at the time o f the study included (a) counseling provided by one o f six therapists whose modalities are characterized as integrative, (b) psychotropic medication provided by one o f two psychiatrists (98% of the sample received prescriptions for medica­ tion), (c) group therapy (for those clients who are amenable or who are court ordered), and (d) case management. This treatment approach, which includes the treatment o f co­ occurring disorders as needed, offered fre­ quent client and mental health service provider contact, and addressed a wide range o f mental illnesses and psychosocial problems using evidence-based treatments.

P r o c e d u r e

As part o f a broader study, an Assessment Bat­ tery, including the SCL-90-R, and an informed consent were assembled. Human subjects review board approval was obtained from a university associated with the study, and insti­ tutional review board approval was provided by the community mental health clinic from which the sample o f participants was drawn. The Assessment Battery was administered to those participants who consented and were capable o f taking the Assessment Battery. The administration was typically executed prior to the clients’ attendance o f an individual ther­ apy, group therapy, psychiatric, or case man­ agement appointment, but the administration of the Assessment Battery in this regard was not consistent and was sometimes performed after the client received services or on a day when the client did not receive any services. Each participant who was eligible and willing to take the Assessment Battery was adminis­ tered the Assessment Battery only one time during the month when the testing took place.

The Assessment Battery comprised three different assessments in addition to the SCL- 90-R: (a) Depression Anxiety Stress Scales (DASS-42; Lovibond & Lovibond, 1995), (b) Quick Inventory o f Depressive Symp­ toms (QIDS-SR)6; Rush et al., 2003), and (c) Outcome Rating Scale (ORS; Miller, Dun­ can, Brown, Sparks, & Claud, 2003). One remaining instrument, the Session Rating Scale (SRS; Duncan et al., 2003), was admin­ istered to randomly assigned clients during the last 5 minutes o f an individual counseling session as recommended by the designers of the instrument (Duncan, Miller, & Sparks, 2004). All the instruments used in this study were printed. The individual measures o f the Assessment Battery were always presented to the participants in the same order, with the SCL-90-R being the first measure in the Assessment Battery that was given to the participants after the informed consent was signed. The remaining instruments in the Assessment Battery were given the follow­ ing order: (a) DASS-42, (b) Q I D S - S R . and (c) ORS; however, the battery was provided to the participants in one paper-clipped packet, so participants were able to complete the instruments o f the Assessment Battery out of order if they chose to do so. This phe­ nomenon was only observed a few times dur­ ing the duration o f the study. The average amount o f time the clients spent to complete the Assessment Battery was about 35 min­ utes, with the range being between approxi­ mately 20 minutes and about 1 hour. The participants were not compensated in any manner for their participation in the study, and were made aware o f this policy from the onset o f the study.

M e a s u re m e n ts

Due to the nature and purpose o f the study, the description o f the SCL-90-R is provided in the introduction. The remaining measurements deployed in this study (DASS-42, QIDS-SR , ORS, and SRS) were administered consis­ tently so that any testing effect would be equal for all participants. The results from the other instruments were not included in this study.

278 M easurem ent and Evaluation in Counseling and Developm ent 4 7 (4 )

A n a l y s i s

The raw scores o f the SCL-90-R were con­ verted into t scores using outpatient male and female matrices provided by Derogatis (1992). After the scores were entered into a Microsoft Excel 2007 worksheet and config­ ured properly, they were imported into SPSS and AMOS for analysis. Descriptive statis­ tics— including standard deviations, means, and medians—were calculated for the nine subscales o f the instrument and the three global scales. Cronbach’s a was calculated for each o f the nine predefined factors of the SCL-90-R in an effort to determine the inter­ nal consistency o f the instrument.

Exploratory factor analyses were per­ formed on the scores o f the SCL-90-R items. The additional items o f the SCL-90-R were excluded from all the analyses as they are not linked to any of the nine subscales (i.e., Items 19,44, 59, 60, 64, 66, and 89). An exploratory factor analysis was performed on the remain­ ing 83 items. Nine exploratory factor analyses were performed on each o f the nine item sets representing the nine constructs o f the SCL- 90-R as defined by Derogatis (1992). Every exploratory factor analysis was performed using an eigenvalue o f one. A separate factor analysis was performed using nine extracted factors, which matches the number o f factors the instrument is intended to measure. A vari- max rotation was performed on the compo­ nent matrices o f the factor analyses.

The nine-factor rotated component matrix, derived from analyzing the 83 items, was fur­ ther analyzed. Items with a factor loading of greater than .45 on any factor were retained and a subsequent factor analysis was executed using only these items. This factor loading cut­ off o f .45 was slightly higher than the factor loading cut-off o f .40 used by similar studies (Asner-Self et al., 2006; Brophy et ah, 1988). This procedure was repeated once more with the result being a list of items that met an acceptable factor loading on at least one factor. At the end of this process, the resulting factor structure was studied using path analysis.

By interpreting the results o f these factor analyses, a hypothetical path analysis was

devised. This path analysis hypothesized that there were two underlying constructs that exhibited a causal relationship to the newly identified factors. Various path diagrams were drawn and tested to produce a model that was the best possible fit for the data. This process included identifying significant covariance that existed between various factors in the path diagram. Through a process o f repeat­ edly examining values indicating how well the model fit the data, establishing new rela­ tionships between factors, and reexamining the model fit, an optimal model was gener­ ated.

R e s u lts

The results o f the SCL-90-R analysis indi­ cated that 170 participants (50%) returned a t score o f 60 or greater (one standard deviation above the mean or greater) and 281 (84%) returned a t score o f 50 or greater (the mean or greater) on at least one o f the subscales o f the instrument including the GSI. The average number o f positive symptoms endorsed (PST) was 48 of the 90 measured by the instrument. These results indicate that this sample self- reported a high frequency and severity o f mental health symptoms.

Internal consistency as measured by Cron­ bach’s a was commensurate with other previ­ ous findings reported by Derogatis (1994), suggesting that the sample that participated in this study approached the SCL-90-R in a simi­ lar manner as the nonnative group (see Table 1). To determine whether any individual items detracted from overall homogeneity o f each scale, an item analytic technique was employed. For each scale, each item was removed from the scale in turn, and the coefficient alpha was computed on the remaining items. No item on any scale, when removed, had any appreciable impact on the scale’s overall homogeneity.

The primary exploratory factor analysis (eigenvalue > 1) that was performed using all 83 o f the items indicated that one factor explained 34% o f the variance, which sug­ gests that the SCL-90-R measures one large factor o f general mental health symptomatol­ ogy. Fifteen other components were identified

G ra n d e e t at. 279

T a b l e I . S C L -9 0 -R Scale In te rn a l C o n s is te n c y .

In te rn a l C o n s is te n c y ( C o e ff ic ie n t a)

2b

S o m a tiz a tio n .86 .88

O b s e s s iv e -C o m p u ls iv e .86 .90

In te rp e rs o n a l S e n s itiv ity .86 .87

D e p re s s io n .90 .92

A n x ie t y .85 .89

H o s t ilit y .84 .86 P h o b ic A n x ie t y .82 .85

P a ra n o id Id e a tio n .80 .80

P s y c h o tic is m .77 .81

a. N = 2 0 9 “ s y m p to m a tic v o lu n te e r s ” (D e ro g a tis , 1994). b. N = 33 6 c u r r e n t sam ple f o r th is stu d y: a c o m m u n ity m e n ta l h e a lth o u tp a tie n t clin ic se ttin g.

that had an eigenvalue greater than one, with the next highest percentage o f variance explained being small (6%) in comparison with the primary factor. The nine exploratory factor analyses that were performed on each o f the item sets (representing the original nine constructs o f the instrument) indicated that the item sets all loaded on one factor each and held together well for each factor. These fac­ tor analyses, when considered together, sug­ gest that the instrument’s items possess shared communality, which explains how they can measure one large factor o f general mental health symptomatology, but simultaneously appear to measure nine independent con­ structs that represent narrower mental health symptom ranges.

The repeated factor analyses, which used a factor loading o f .45 as a cut-off to retain items, resulted in the retention of 65 items plus two items that were just below the .45 cutoff (.444 and .446). The resulting 67-item matrix demonstrated the presence o f nine distinct fac­ tors. Each o f the 67 items was reviewed for content and a conceptualization for what the items measured for each factor was proposed. The nine proposed conceptualizations o f the 67-item variant corresponded to the original instrum ent’s nine intended subscales to varying degrees. The nine proposed subscales based on the 67 items are (a) Depression and

Related Subthemes, (b) Pain/Discomfort, (c) Anxiety and Physically Associated Symptoms, (d) Other-Focused Negativity, (e) Phobic Anx­ iety, (f) Immobilization/Feeling Stuck, (g) Reality Testing/Danger Potential, (h) Compul- sivity, and (i) External Locus o f Control (see Table 2).

Based on these proposed subscales o f the 67-item variant, a path analysis was con­ ducted to identify casual relationships among the variables and to uncover possible underly­ ing constructs. On examining the factor load­ ings and the content o f each retained item, it was hypothesized that two underlying con­ structs were related to the new variables. These two constructs were Internal Perceived Mental Health Concerns (InPeMHC) and External Perceived Mental Health Concerns (ExPeMHC). A strong relationship was detected between InPeMHC and six o f the variables: (a) Depression and Related Sub­ themes, (b) Pain/Discomfort, (c) Anxiety and Physically Associated Symptoms, (d) Phobic Anxiety, (e) Immobilization/Feeling Stuck, and (f) Compulsivity. Conversely, ExPeMHC was found to be strongly associated with the remaining three variables: (a) Other-Focused Negativity, (b) Reality Testing/Danger Poten­ tial, and (c) External Locus o f Control. This hypothesized model was then compared with an improved model, which was developed by a process o f repeatedly analyzing the modifi­ cation indices, denoting new relationships in the path analysis diagram, and recalculating the estimates.

Two second-order factor analyses were conducted using the selected 67 items and dividing by InPeMHC items and ExPeMHC items. The results were consistent with what was hypothesized, with the InPeMHC items demonstrating high factor loadings for the corresponding six proposed constructs and the ExPeMHC items producing high factor load­ ings for the associated three proposed factors. The InPeMHC items factor loading ranges were acceptable move forward with the path analysis and had the following factor loading ranges: (a) Component 1 = .44 to .84, (b) Component 2 = .44 to .74, (c) Component 3 = .62 to .73, (d) Component 4 = .44 to .72,

280 M easurem ent and Evaluation in Counseling and D evelopm ent 4 7 (4 )

T a b l e 2 . 6 7 - Ite m S y m p to m C h e c k lis t.

R e vised S y m p to m Subscales C o r r e s p o n d in g S C L -9 0 -R ite m s E x a m p le

D e p re s s io n an d R e la te d 79, 5 4 , 30, 29, 77, 2 6 , 6 9 , 7 1 , 4 1 , F eeling h o p e le s s a b o u t th e S u b th e m e s 9 0 , 8 0 , 3 1 , 3 7 , 88, 4 6 , 36, 3, 32,

14, 34, 2 0 , 3 3 , 1 5 , 6 1 , 2 8 f u t u r e

P a in /D is c o m fo r t 4 2 , 5 2 , 2 7 , 5 6 , 5 8 , 4 0 , 1 ,4 9 Pain in head, c h e s t, o r b a c k

A n x ie t y an d P hy s ic a lly A s s o c ia te d S y m p to m s

4, 12, 17, 3 9 , 2 , 4 8 , 72 N e rv o u s n e s s , fa in tn e s s , o r tr e m b lin g

O th e r - F o c u s e d N e g a tiv ity 8 1 , 6 7 , 7 4 , 2 4 , 63, 8, 76 W a n t in g t o h a rm s o m e o n e

P h o b ic A n x ie t y 13, 4 7 , 70, 50, 25, 73 F eeling a fra id o f c e rta in places

Im m o b iliz a tio n /F e e lin g S tu c k 9 , 5 1 , 5 , 5 5 T r o u b le c o n c e n tr a tin g

R e a lity T e s tin g /D a n g e r P o te n tia l 8 6 , 8 4 , 85, 16 H e a rin g v o ic e s

C o m p u ls iv ity 38, 4 5 , 65 D o u b le -c h e c k in g w o r k

E x te rn a l L o c u s o f C o n t r o l 7, 35, 6 2 H a v in g t h o u g h ts f r o m an o u ts id e s o u rc e

T a b l e 3. M o d e l F it S u m m a ry : R o o t m ea n S q u are E r r o r o f A p p r o x im a t io n ; C o m p a ris o n o f M o d e ls .

M o d e l R M S E A L O 9 0 H I 9 0 P C L O S E

H y p o th e s iz e d

D e fa u lt m o d e l .09 7 .07 9 .11 6 .00 0

In d e p e n d e n c e m o d e l .32 8 .31 4 .343 .00 0

Im p ro v e d

D e fa u lt m o d e l .03 6 .00 0 .061 .79 8

In d e p e n d e n c e m o d e l .32 8 .31 4 .343 .000

T a b l e 4 . M o d e l F it S u m m a ry : B ase line C o m p a ris o n s ; C o m p a ris o n o f M o d e ls .

M o d e l N F I D e lt a 1 RFI r h o l IFI D e lta 2 T L I rh o 2 C FI

H y p o th e s iz e d

D e fa u lt m o d e l .92 0 .889 .9 3 7 .91 2 .93 7

S a tu ra te d m o d e l 1.000 1.000 1.000 In d e p e n d e n c e m o d e l .00 0 .00 0 .00 0 .00 0 .00 0

Im p ro v e d

D e fa u lt m o d e l .97 7 .963 .993 .98 8 .993

S a tu ra te d m o d e l 1.000 1.000 1.000 In d e p e n d e n c e m o d e l .00 0 .00 0 .00 0 .000 .00 0

(e) Component 5 = .47 to .65, and (g) Compo­ nent 6 = .47 to .54. Similar high factor loading ranges were observed for ExPeMHC: (a) Component 1 = .48 to .84, (b) Component 2 = .65 to .77, and (c) Component 3 = .59 to .78.

The hypothesized model demonstrated a poor to moderate fit to the data (see Tables 3 and 4). The root mean square error o f approxi­ mation (RMSEA= .97) and the comparative fit

index (CFI = .94) were suggestive of an unac­ ceptable risk of an incorrectly specified model (Hooper, Coughlan, & Mullen, 2008). The improved model, however, was found to be a remarkably good fit to the data under even the more stringent guidelines for model fit indices (RMSEA= .36; CFI = .99; Hooper et al., 2008).

The difference between the hypothesized model (see Figure 1) and the improved

G ra n d e e t al. 281

F ig u re I . Hypothesized model; path analysis o f the 67-item Symptom Checklist. Note. InPeMHC = Internal Perceived Mental Health C oncerns; ExPeMHC = External Perceived Mental Health Concerns.

model, w hich becam e the final model because it best represents the data (see F igure 2), consisted o f additional identi­ fied relationships between symptom factors. Although the hypothesized model did not specify any o f these relationships, the final model had four: (a) Pain/Discomfort and Anxiety and Physically Associated Symp­ toms, (b) Pain/Discomfort and Immobiliza- tion/Feeling Stuck, (c) Phobic Anxiety and External Locus o f Control, and (d) Pain/Dis­ comfort and Phobic Anxiety.

Discussion

The results of this study offer both support for and challenges to the factor structure o f the SCL-90-R as defined by Derogatis (1992). Since only 83 items on the instrument are intended to be part o f a scale, this study’s analyses focused mainly on these items. An exploratory factor analysis resulted in one large factor when performed using 90 and 83 items, which answered the first research ques­ tion for this study—What is the factor structure

282 M easurem ent and Evaluation in Counseling and Developm ent 4 7 (4 )

84

F i g u r e 2. Final m odel; path analysis o f th e 6 7 -ite m S ym ptom C h ecklist. Note. InPeMHC = Internal Perceived Mental Health Concerns; ExPeMHC = External Perceived Mental Health Concerns.

o f the SCL-90-R? This one factor appears to measure general mental health symptom severity, which supports the use o f this instru­ ment to assess clients for mental health symp- tomology, but challenges its usefulness to distinguish between the nine defined sub­ scales. When the analysis was configured to extract exactly nine factors, a much different impression o f the instrument was visible.

Using the results o f factor analyses that extracted nine factors, 67 o f these 83 items appeared to load highly on factors and mea­ sured distinct constructs, which verified the existence of a useful variant and answered the second research question— Is there a useful variant that can be generated from this study’s

sample? These 67 items were retained, and the remaining 16 items were eliminated based on low factor loadings. The constructs identified from the 67-item analysis only partially cor­ responded with the instrument’s scale defini­ tions, but these new constructs were generally similar to those specified in the SCL-90-R. To effectively use the SCL-90-R for the assess­ ment of distinct mental health constructs, it may be beneficial to examine the items endorsed, rather than solely relying on the specified item sets o f the instrument.

The results of the data analysis may simulta­ neously reflect characteristics of the SCL-90-R and potential incongruence with the DSM-IV- TR (American Psychiatric Association, 2000).

G r a n d e e t al. 283

The instrument appears to be able to distin­ guish between the nine constructs o f symptom­ atology as it was designed to do and also measures one general mental health distress factor. Many o f the items appear to be measur­ ing symptomatology on more than one con­ struct, yet qualitatively the content o f most of the items appears to be clearly focused toward one construct. The results suggest that the con­ structs o f mental health disorder classification may not be distinct and separated cleanly; rather, they contain broad overlapping catego­ ries o f mental illness.

It was hypothesized that the path analysis o f the 67-item variant was straightforward, with internal and external perceived mental health concerns contributing directly to one distinct subset o f symptoms factors each. These underlying constructs were thought to make this contribution and to possess utility without necessarily having to consider the relationship between the symptom factors themselves. This model proved to be inade­ quate in explaining the complicated relation­ ships among the variables. These findings did, however, provide guidance toward the devel­ opment of an improved model, which repre­ sented a much better fit to the data.

The results indicated by the final model revealed significant relationships between these specified factors, which have important implications that go beyond the scope o f the hypothesized model. Since a path analysis simultaneously accounts for all the variability in a model, covariance levels that may at a glance seem insignificant, may represent sig­ nificant relationships. In the final model, the covariance between Pain/Discomfort and Anxiety and Physically Associated Symp­ toms was .45 (see Figure 2). In a path analy­ sis, this is a strong relationship and indicates a great deal o f variance. Meaningful correla­ tions between Pain/Discomfort and Phobic Anxiety (.13) as well as Pain Discomfort and Immobilization/Feeling Stuck (.17) were also observed. Although these relationships are less prominent than the relationship between Pain/Discomfort and Anxiety and Physically Associated Symptoms, they still represent moderate relationships between these factors.

Similarly, the covariance level (.17) between Phobic Anxiety and External Locus o f Con­ trol is noteworthy in path analysis.

Implications for the Treatment Community

The results o f the analyses included in this study indicate that the SCL-90-R and a short­ ened 67-item variant measure specific types o f mental health symptomology with a fair degree o f accuracy. The original nine factors o f the SCL-90-R hold together reasonably well for the community mental health popula­ tion sample, although high multicollinearity restricts distinctness among the factors. The 67-item variant’s factors held together more tightly, although the factors themselves were not identical to those in the SCL-90-R.

The third research question is answered by detailing how clinicians may use and interpret responses from this 67-item variant— What are the implications for the variant’s use and interpretation for mental health clinicians? For either version o f this instrument, and for the other SCL-90-R versions available, men­ tal health professionals are encouraged to interpret not only the total and the subscales scores but also additionally view the key item responses individually. Valuable information can be gathered from an item analysis that otherwise may be overlooked by only exam­ ining final scores. It is reasonable to deduce that this method reduces the risk o f poor or inaccurate interpretations and amplifies the usefulness of this instrument and its variants.

Using the factors o f the 67-item variant as outlined in Table 2, mental health clinicians are able to analyze the results o f SCL-90-R at an additional level. Another nine dimensions have been extracted in this study and are made available to clinicians performing assessment; however, it should be noted that three o f the factors were already captured in the original SCL-90-R (Depression, Phobic Anxiety, and Compulsivity). Considering the sample for this study was drawn from a community men­ tal health agency, clinicians perfonning assessments in this setting may realize the most benefit from this added analytical utility.

284 M easurem ent and Evaluation in Counseling and Developm ent 4 7 (4 )

Also, it is important clinicians recognize that the number of items associated with each fac­ tor o f the shortened version varies, which lends greater strength to detecting the pres­ ence and severity o f certain factors while pro­ viding less support for others.

The final path analysis revealed two main underlying constructs that were associated with the new nine factors o f the variant (see Figure 1). Six o f the nine factors indicated a strong association with an Internal Perception o f the Mental Health Concerns (InPeMHC), while the remaining three were more closely associated with External Perception o f the Mental Health Concerns (ExPeMHC). The two underlying constructs were also strongly associated with each other, which indicated that the 67-item variant’s underlying con­ structs highly influence each other. As clini­ cians set out to use the results o f this study to aid in assessment, it is crucial that they recog­ nize the difference in the two underlying con­ structs, the related factors, and how they should be carefully interpreted.

This results o f the study demonstrated that there are more items and factors associated with InPeMHC. Subsequently, this underly­ ing construct can be evaluated by clinicians with more precision. Furthermore, due to the disproportionate number o f items asso­ ciated with InPeMHC, the likelihood o f it becoming germane to any given assessment is higher than that o f ExPeMHC. O f the four factors with the lowest number o f associated items (Immobilization/Feeling Stuck [four items], Reality Testing/Danger Potential [four items], Compulsivity [three items], and External Locus o f Control [three items]), two are associated with ExPeMHC (Reality Test­ ing/Danger Potential and External Locus o f Control). The remaining factor associated with ExPeMHC is Other Focused Negativity, which is associated with seven items. While ExPeMHC has maintained an association with just half the factors o f InPeMHC and only 21 % o f the total number o f items included in the variant, clinicians are encouraged to weigh the severity and potentially serious consequences o f ExPeMHC when interpret­ ing assessment results. Treatment modalities,

counseling techniques, level of care, safety planning, and goal setting may vary greatly between clients suffering from primarily ExPeMHC as compared with InPeMHC.

The final model demonstrates the impor­ tance o f recognizing the symptom factors’ relationship to one another, specifically how the Pain/Discomfort factor is moderately to strongly correlated with three other factors (Anxiety and Physically Associated Symp­ toms, Immobilization/Feeling Stuck, and Phobic Anxiety) and the strong correlation between InPeMHC and ExPeMHC. Mental health clinicians may want to consider this connection and what it may mean for the cli­ ent, if they observe high scores on the Pain/ Discomfort factor. All three o f these correlates possess an element of anxiety, which may not have been observed or interpreted in the absence o f this improved model. The most notable relationship is the strong correlation between Pain/Discomfort and Anxiety and Physically Associated Symptoms. Similarly, clinicians may want to integrate the moderate relationship between Phobic Anxiety and External Locus o f Control into client assess­ ment and treatment planning. For example, asking questions designed to explore clients’ rating o f their anxiety levels and how they view external stressors as potentially contrib­ uting to anxiety, may aid clinicians with case conceptualization.

O f the various relationships indicated in the final model, the role o f the Pain/Discom­ fort factor is the most significant, as the mod­ erate to high covariance levels were observed between this factor and three other factors (Anxiety and Physically Associated Symp­ toms, Phobic Anxiety, and Immobilization/ Feeling Stuck). This finding is of particular importance for mental health clinicians con­ sidering the ramifications o f high scores on the Pain/Discomfort scale and leads to a few practical applications for clinicians.

One practical application these relation­ ships could offer is influencing how mental health clinicians consider the importance of clients’ suffering from physical pain or dis­ comfort. As physical pain appears to be a good predictor o f high physical anxiety and a

Grande et al. 285

fair predictor o f phobic anxiety and feeling stuck, clinicians may want to consider refer­ ring clients to medical providers concur­ rently as they treat the clients for mental health concerns. The resolution or ameliora­ tion o f physical pain may tend to decrease anxiety and the feeling o f being stuck and, therefore, allow for more significant clinical improvement.

Another practical application identified from these relationships focuses on how men­ tal health clinicians collaborate with clients to determine treatment goals. While clients may present with anxiety and feelings o f being stuck, the underlying cause of these symp­ toms, or perhaps a significant contributor, may be physical pain or discomfort. Mental health clinicians may want to consider explor­ ing with clients the role the pain and discom­ fort may have in relation to their mental health symptoms. More accurate treatment planning and the selection o f more appropriate treat­ ment modalities (e.g., Acceptance and Com­ mitment Therapy for pain) may result from this exploration of the impact o f pain.

In examining the final model from a broad view, the high correlation (.84) between InPeMHC and ExPeMHC provides powerful insight to mental health clinicians, which can be interpreted in a variety o f ways. For exam­ ple, clients who score highly on an ExPeMHC factor are likely to score highly on the factors associated with InPeMHC and vice versa. Mental health clinicians may want to explore possible external concerns even with clients who only appear to be presenting with internal concerns, given how strongly these constructs are linked to each other.

The value o f this model can be tested by applying it to other important clinical issues faced by community mental health outpa­ tient clients. By examining and assessing the model’s utility related to outcomes measure­ ment, treatment compliance, identifying chief complaints, and connecting item responses with diagnoses from the DSM-IV- TR (American Psychiatric Association, 2000), mental health clinicians can derive further and significant benefits from the use o f this SCL-90-R variant.

Lim itations

While the factor loadings for the 67-item vari­ ant were relatively robust (range o f .444 to .857), the last two factors (Compulsivity and External Locus o f Control) only had three items each that loaded onto them. These two 3-item factors may indicate that the 67-item variant does not measure the corresponding constructs accurately. Further research is war­ ranted to determine the nature and usefulness o f the constructs o f the variant, with special attention to the last two factors. Alternatively, research can be focused on the creation and testing o f additional items, including items that would load on validity scales, which may be used to strengthen the usefulness o f all the factors.

Even with high Cronbach’s a values and a relatively substantial sample size (n = 336), a high noncompliance rate (25%) among poten­ tial participants denied the researchers valuable data, especially related to those participants with severe psychopathology. This community mental health population proved to be difficult to assess, which should serve as an important point o f caution to researchers who are inter­ ested in the appraisal of this population. Appro­ priate planning for potential lack o f cooperation may increase participation rates in similar stud­ ies in the future.

While community mental health outpa­ tients used in this study are underrepresented in the research and more studies are needed with this population, the research community is encouraged to test the final model with dif­ ferent samples to evaluate the generalizability. Suggested populations that are also underrep­ resented in the research are clients in inpatient care facilities, clients with co-occurring disor­ ders, clients suffering from trauma, and clients with personality disorders. When formulating a research design to include some o f the sug­ gested populations, researchers are encour­ aged to take note o f some of the compliance difficulties experienced in this study and take measures to overcome similar challenges.

This study used a sufficient sample size for its intended purpose. However, future studies with increased sample size will allow the

286 M easurem ent and Evaluation in Counseling and D evelopm ent 4 7 (4 )

inclusion o f other potentially useful variables such as gender, intensity o f symptoms, age, educational level, socioeconomic status, eth­ nicity, substance use, diagnosis, and presence of legal history. The inclusion o f such vari­ ables would increase the robustness and util­ ity o f the final model and potentially lead to other areas o f research related to the SCL- 90-R as well as variants of the instrument.

The deployment o f statistical processes such as path analysis to analyze assessment instruments brings some challenges in terms o f research design, but it also generates a potential for significant and important find­ ings related to assessment, treatment plan­ ning, and positive outcomes for clients. As demonstrated in this study, instruments such as the SCL-90-R, and likely many other popu­ lar and common instruments, are used by mental health clinicians frequently despite potentially serious limitations in the function­ ality, accuracy, and validity of these instru­ ments. Continued exploration, analysis, and refinement o f instruments such as the SCL- 90-R are encouraged to promote more precise assessment and to provide mental health clini­ cians with new and more accurate interpreta­ tion methods. The primary purpose o f these endeavors is to enhance the quality o f services delivered to clients and make the best use o f the mental health clinical resources that are available to the community.

D e c la r a t io n o f C o n f lic t in g In t e r e s t s

The author(s) declared no potential conflicts o f interest with respect to the research, authorship, and/or publication o f this article.

F u n d in g

The author(s) received no financial support for the research, authorship, and/or publication o f this article.

R e fe r e n c e s

American Psychiatric Association. (2000). Diagnostic and statistical manual o f mental

■ disorders (4th ed., text revision). Washington, DC: Author.

Arrindell, W. A., Barelds, D. H., Janssen, I. M., Buwalda, F. M., & Van Der Ende, J. (2006).

Invariance o f SCL-90-R dimensions o f symptom distress in patients with peri par- tum pelvic pain (PPPP) syndrome. British Journal o f Clinical Psychology, 45, 377-391. doi:10.1348/014466505X68924

Asner-Self, K., Schreiber, J., & Marotta, S. (2006). A cross-cultural analysis o f the Brief Symptom Inventory-18. Cultural Diversity & Ethnic Minority Psychology, 12, 367-375. doi: 10.1037/1099-9809.12.2.367

Barker-Collo, S. (2003). Culture and validity o f the Symptom Checklist-90-Revised and Profile o f Mood States in a New Zealand student sample. Cultural Diversity & Ethnic Minority Psychology, 9, 185-196. doi: 10/1037/1099- 9809.9.2.185

Barker-Collo, S., & Read, J. (2011). The roles o f gen­ der and coping styles in the relationship between child abuse and the SCL-90-R Subscales “Psychoticism” and “Paranoid Ideation.” New Zealand Journal o f Psychology, 40(2), 30-40.

Barkley, R., Murphy, K., & Kwasnik, D. (1996). Psychological adjustment and adaptive impair­ ments in young adults with ADHD. Journal o f Attention Disorders, 1, 41-54.

Bjorkly, S. (2002). SCL-90-R profiles in a sam­ ple o f severely violent psychiatric inpatients. Aggressive Behavior, 28, 446^157.

Bonynge, E. (1993). Unidimensionality o f SCL- 90-R scales in adult and adolescent crisis samples. Journal o f Clinical Psychology, 49, 212-215.

Brophy, C., Norvel, N., & Kiluk, D. (1988). An examination o f the factor structure and conver­ gent and discriminant validity o f the SCL-90R in an outpatient clinic population. Journal o f Personality Assessment, 52, 334-340.

Buckelew, S., Burk, J., Brownlee-Duffeck, M., Frank, R., & DeGood, D. (1988). Cognitive and somatic aspects o f depression among a reha­ bilitation sample: Reliability and validity o f SCL-90-R research subscales. Rehabilitation Psychology, 33(2), 67-75. doi: 10.1037/ h0091679

Carpenter, K. M., & Hittner, J. B. (1995). Dimensional characteristics o f the SCL-90-R: Evaluation o f gender differences in dually diagnosed inpatients. Journal o f Clinical Psychology, 51, 383-390.

Cepeda-Benito, A., & Gleaves, D. (2000). Symptom Checklist-21 in European American, African American, and Latino college stu­ dents. Cultural Diversity & Ethnic Minority Psychology, 6, 297-308.

Grande et al. 287

Chang-Kook, Y., Byeong-Moo, C., Baity, M., Jeong-Hyeong, L., & Jin-Seok, C. (2005). SCL-90-R and 16PF profiles of senior high school students with excessive Internet use. Canadian Journal o f Psychiatry, 50, 407-414.

Clark, A., & Friedman, M. (1983). Factor struc­ ture and discriminant validity o f the SCL-90 in a veteran psychiatric population. Journal o f Personality Assessment, 47, 396-404.

Comrey, A. L., & Lee, H. B. (1992). A first course in factor analysis. Hillsdale, NJ: Erlbaum.

Cyr, J. E., McKenna-Foley, J. M., & Peacock, E. (1985). Factor structure of the SCL-90-R: Is there one? Journal o f Personality Assessment, 49, 571-578.

Derogatis, L. R. (1992). SCL-90-R, administra­ tion, scoring & procedures manual-II fo r the R(evised) version and other instruments o f the psychopathology rating scale series. Townson, MD: Clinical Psychometric Research.

Derogatis, L. R. (1993). The Brief Symptom Inventory (BSI): Administration, scoring, and procedures manual (3rd ed.). Minneapolis, MN: National Computer Systems.

Derogatis, L. R. (1994). Symptom Checklist-90-R: Administration, scoring, and procedures man­ ual. San Antonio, TX: Pearson.

Derogatis, L. R. (2000). The Brief Symptom Inventory-18 (BSI-18): Administration, scor­ ing, and procedures manual. Minneapolis, MN: National Computer Systems.

Derogatis, L. R., & Cleary, P. A. (1977a). Confirmation of the dimensional structure of the SCL-90: A study in construct validation. Journal o f Clinical Psychology, 33, 981-989.

Derogatis, L. R., & Cleary, P. A. (1977b). Factorial invariance across gender for the primary symp­ tom dimensions of the SCL-90. British Journal o f Social <£ Clinical Psychology, 16, 347-356.

Derogatis, L. R., Lipman, R. S., & Covi, L. (1973). SCL-90: An outpatient psychiatric rating scale- Preliminary report. Psychopharmacology Bulletin, 9, 13-27.

Dinning, W., & Evans, R. (1977). Discriminant and convergent validity of the SCL-90 in psychiatric inpatients. Journal o f Personality Assessment, 41, 304-310.

Duncan, B., Miller, S., & Sparks, J. (2004). The heroic client: A revolutionary way to improve effectiveness through client-directed, out­ come-informed therapy. San Francisco, CA: Jossey-Bass.

Duncan, B., Miller, S., Sparks, J., Claud, D., Reynolds, L., Brown, J., & Johnson, L. (2003).

The Session Rating Scale: Preliminary psy­ chometric properties of a “working” alliance measure. Journal o f Brief Therapy, 3(1), 2—12.

Eich, D., Ajdacic-Gross, V., Condrua, M., Huber, H., Gamma, A., Angst, J., & Rossler, W. (2003). The Zurich Study: Participation pat­ terns and Symptom Checklist 90-R scores in six interviews, 1979-99. Acta Psychiatrica Scandinavica, 108, 11-14.

Elliott, R., Fox, C., Beltyukova, S., Stone, G., Gunderson, J., & Zhang, X. (2006). Deconstructing therapy outcome measurement with Rasch analysis of a measure of general clinical distress: The Symptom Checklist-90- Revised. Psychological Assessment, 18, 359- 372. doi: 10.1037/1040-3590.18.4.359

Evenson, R., Holland, R., Mehta, S., & Yasin, F. (1980). Factor analysis of the Symptom Checklist-90. Psychological Reports, 47, 695-699.

Gilliss, C., Moore, I., & Martinson, I. (1997). Measuring parental grief after childhood cancer: Potential use of the SCL-90R. Death Studies, 21, l l l - l V l .

Hafkenscheid, A. (1993). Psychometric evaluation of the Symptom Checklist (SCL-90) in psy­ chiatric inpatients. Personality and Individual Differences, 14, 751-756.

Hardt, J., Dragan, M., & Kappis, B. (2011). A short screening instrument for mental health problems: The Symptom Checklist-27 (SCL- 27) in Poland and Germany. International Journal o f Psychiatry in Clinical Practice, 75(1), 42-49.

Hardt, J., & Gerbershagen, H. (2001). Cross- validation of the SCL-27: A short psychometric screening instrument for chronic pain patients. European Journal o f Pain, 5, 187-197.

Hart, K., Bryer, J., & Martines, K. (1991). SCL- 90-R symptom patterns for adolescent psy­ chiatric inpatients. Journal o f Personality Assessment, 56(1), 168-178.

Hoffman, N. G., & Overall, P. B. (1978). Factor structure of the SCL-90 in a psychiatric pop­ ulation. Journal o f Consulting and Clinical Psychology, 45, 1187-1191.

Holder, E., & Levi, D. (1988). Mental health and locus o f control: SCL-90-R and Levenson’s IPC Scales. Journal o f Clinical Psychology, 44, 753-755.

Holi, M. M., Marttunen, M., & Aalberg, V. (2003). Comparison of the GHQ-36, the GHQ- 12 and the SCL-90 as psychiatric screen­ ing instruments in the Finnish population.

288 M easurem ent and Evaluation in Counseling and D evelopm ent 4 7 (4 )

Nordic Journal o f Psychiatry, 57, 233-238. doi: 10:1080/08039480310001418

Hooper, D., Coughlan, J., & Mullen, M. (2008). Structural equation modelling: Guidelines for determining model fit. Electronic Journal o f Business Research Methods, 6, 53-60.

Jansson, I., Hesse, M., & Fridell, M. (2008). Personality disorder features as predictors of symptoms five years post-treatment. American Journal on Addictions, 17, 172-175.

Johnson, J., Bomstein, R., & Sherman, M. (1996). A modified scoring algorithm for the PDQ-R: Psychiatric symptomatology and substance use in adolescents with personality disorders. Educational and Psychological Measurement, 56, 76-89.

Kaplan, C., Miner, M., Mervis, L., Newton, H., McGregor, J., & Goodman, J. (1998). Interpretive risks: The use of the Hopkins Symptom Checklist 90 Revised SCL 90 R with brain tumour patients. Brain Injury, 12, 199-205.

Kennedy, B., Morris, R., Pedley, L., & Schwab, J. (2001). The ability of the Symptom Checklist SCL-90 to differentiate various anxiety and depressive disorders. Psychiatric Quarterly, 72, 277-288.

Kirkcaldy, B., Fumham, A., & Siefen, R. (2010). Social, family, and psychological predictors of obsessive-compulsive behaviour among children and adolescents. School Psychology International, 31, 42-59.

Kuhl, H., Hartwig, I., Petitjean, S., Muller-Spahn, F., Margraf, J., & Bader, K. (2010). Validation of the Symptom Checklist SCL-27 in psy­ chiatric patients: Psychometric testing of a multidimensional short form. International Journal o f Psychiatry in Clinical Practice, 14, 145-149.

Lovibond, S., & Lovibond, P. (1995). Manual fo r the Depression Anxiety Stress Scales. Sydney, Australia: Psychology Foundation.

Maremmani, I., Pani, P., Pacini, M., Bizzarri, J. V., Trogu, E., Maremmani, A. I . , .. .Dell’Osso, L. (2010). Subtyping patients with heroin addic­ tion at treatment entry: Factor derived from the Self-Report Symptom Inventory (SCL- 90). Annals o f General Psychiatry, 9, 15-27. doi: 10.1186/1744-859X-9-15

Martinez, S., Stillerman, L., & Waldo, M. (2005). Reliability and validity of the SCL-90-R with Hispanic college students. Hispanic Journal o f Behavioral Sciences, 27, 254-264. doi: 10:1177/0739986305274911

Masdrakis, V., Papakostas, Y., Vaidakis, N., Papageorgiou, C., & Pehlivanidis, A. (2008). Caffeine challenge in patients with panic dis­ order: Baseline differences between those who panic and those who do not. Depression and Anxiety, 25(9), E72-E79.

McGough, J., & Curry, J. (1992). Utility of the SCL-90-R with depressed and conduct- disordered adolescent inpatients. Journal o f Personality Assessment, 59, 552-563.

Miller, S., Duncan, B., Brown, J., Sparks, J., & Claud, D. (2003). The Outcome Rating Scale: A preliminary study of the reliability, validity, and feasibility of a brief visual analog mea­ sure. Journal o f Brief Therapy, 2, 92-100.

Molenaar, P., Dekker, J., Van, R., Hendriksen, M., Vink, A., & Shoevers, R. (2007). Does adding psychotherapy to pharmacotherapy improve social functioning in the treatment of outpa­ tient depression? Depression and Anxiety, 24, 553-562.

Numan, N. (2004). Exploration of adverse psycho­ logical symptoms in Yemeni khat users by the Symptoms Checklist-90 (SCL-90). Addiction, 99( 1), 61-65.

O’Donnell, W., DeSoto, C., & Reynolds, D. (1984). A cognitive deficits subscale of the SCL-90-R. Journal o f Clinical Psychology, 40, 241-246.

Paap, M., Meijer, R., Van Bebber, J., Pedersen, G., Karterud, S., Hellem, F., & Haraldsen, I. (2011). A study of the dimensionality and measurement precision of the SCL-90-R using item response theory. International Journal o f Methods in Psychiatric Research, 20, e39- e55. doi:10.1002/mpr.347

Parrott, A., Milani, R., Parmar, R., & Turner, J. (2001). Recreational ecstasy/MDMA and other drug users from the UK and Italy: Psychiatric symptoms and psychobiological problems. Psychopharmacology, 159, 77-82.

Pearson Education, Inc. (2009). Symptom Checklist 90-Revised (SCL-90-R) bibliography. Retrieved from http://psych-corp.pearsonassessments. com/NR/rdonlyres/77DBA537-A85A-4F6C- BA78-1 D36C06-67230/0/SCL90R_bib.pdf

Pederson, G., & Karterud, S. (2004). Is SCL-90R helpful for the clinician in assessing DSM-IV symptom disorders? Acta Psychiatrica Scandinavica, 110, 215-224.

Piersma, H., Boes, J., & Reaume, W. (1994). The Brief Symptom Inventory as an outcome measure for adolescent psychiatric inpatients. Assessment, 1, 151-157.

Grande et al. 289

Plante, T., Manuel, G., Menendez, A., & Marcotte, D. (1995). Coping with stress among Salvadoran immigrants. Hispanic Journal o f Behavioral Sciences, 17, 471^-79.

Qouta, S., Punamaki, R., & El Sarraj, E. (2005). M other-child expression o f psychological dis­ tress in war trauma. Clinical Child Psychology and Psychiatry, 10, 135-156.

Rauter, U., Leonard, C., & Swett, C. (1996). SCL- 90-R factor structure in an acute, involuntary, adult psychiatric inpatient sample. Journal o f Clinical Psychology, 52, 625-629.

Recklitis, C., Licht, I., Ford, J., Oeffinger, K., & Diller, L. (2007). Screening adult survivors o f childhood cancer with the distress ther­ mometer: A comparison with the SCL-90-R. Psycho-Oncology, 16, 1046-1049. doi:10.1002/ pon.1212

Rush, A., Trivedi, M., Ibrahim, H., Carmody, T., Amow, B., Klein, D., . . .Keller, M. (2003). The 16-item Quick Inventory o f Depressive Symptomatology (QIDS) Clinician Rating (QIDS-C) and Self-Report (QIDS-SR): A psy­ chometric evaluation in patients with chronic major depression. Biological Psychiatry, 54, 573-583.

Ruwaard, J., Schrieken, B., Schrijver, M., Broeksteeg, J., Dekker, J., Vermeulen, H., & Lange, A. (2009). Standardized web-based cognitive behavioural therapy o f mild to mod­ erate depression: A randomized controlled trial with a long-term follow-up. Cognitive Behaviour Therapy, 38, 206-221.

Ryan, M. (2008). The antidepressant effects o f physical activity: Mediating self-esteem and self-efficacy mechanisms. Psychology & Health, 23, 279-307.

Schmitz, N., Hartkamp, N., Brinschwitz, C., Michalek, S., & Tress, W. (2000). Comparison o f the standard and the computerized ver­ sions o f the Symptom Check List (SCL- 90-R): A randomized trial. Acta Psychiatrica Scandinavica, 102, 147-152.

Schmitz, N., Hartkamp, N., Kruse, J., Franke, G., Reister, G., & Tress, W. (2000). The Symptom Check-List-90-R (SCL-90-R): A German valida­ tion study. Quality o f Life Research, 9, 185—193.

Schmitz, N., Kruse, J., Heckrath, C., Alberti, L., & Tress, W. (1999). Diagnosing mental dis­ orders in primary care: The General Health Questionnaire (GHQ) and the Symptom Check List (SCL-90-R) as screening instru­ ments. Social Psychiatry and Psychiatric Epidemiology, 34, 360-366.

Schwarzwald, J., Weisenberg, M., & Solomon, Z. (1991). Factor invariance o f SCL-90-R: The case o f combat stress reaction. Psychological Assessment, 3, 385-390.

Siri, C., Cilia, R., De Gaspari, D., Villa, F., Goldwurm, S., Marco, C., . . .Antonini, A. (2010). Psychiatric symptoms in Parkinson’s disease assessed with the SCL-90R self- reported questionnaire. Neurological Sciences, 31, 35-40. doi: 10.1007/s 10072-009-0165-0

Steinberg, M., Barry, D. T., Sholomskas, D., & Hall, P. (2005). SCL-90 symptom patterns: Indicators o f dissociative disorders. Bulletin o f the Menninger Clinic, 69, 237-249.

Steer, R., Clark, D., & Ranieri, W. (1994). Symptom dimensions o f the SCL-90-R: A test o f the tripartite model o f anxiety and depres­ sion. Journal o f Personality Assessment, 62, 525-536.

Strand, B., Dalgard, O., Tambs, K., & Rognerud, M. (2003). Measuring the mental health status o f the Norwegian population: A comparison o f the instruments SCL-25, SCL-10, SCL-5 and MHI-5 (SF-36). Nordic Journal o f Psychiatry, 5 7 ,113-118. doi: 10.1080/08039480310000932

Tate, D., Kewman, D., & Maynard, F. (1990). The B rief Symptom Inventory: Measuring psychological distress in spinal cord injury. Rehabilitation Psychology, 35, 211—216. doi: 10.1037/h0079062

Tomassini, A., Struglia, F., Stratta, P., Riccardi, I., Tempesta, D., Pacifico, R., & Rossi, A. (2009). Affective temperaments and residual symptoms in patients with mood and anxiety disorders. International Journal o f Psychiatry in Clinical Practice, 73(1), 31-35.

Torres, X., Bailies, E., Collado, A., Tabemer, J., Gutierrez, F., Peri, J., . . .Valdes, M. (2010). The Symptom Checklist-Revised (SCL-90-R) is able to discriminate between simulation and fibromyalgia. Journal o f Clinical Psychology, 66, 774-790. doi:10.1002/jclp.20698

Wegner, J., Rabiner, C., & Kane, J. (1985). The cognitive deficit subscale o f the SCL-90-R and prediction o f outcome in the first epi­ sode o f schizophrenics. Journal o f Clinical Psychology, 41, 750-752.

Yong Woo, K., Sang-Hyuk, L., Tae Kyou, C., Shin Young, S., Borah, K., Chan Mo, K., & Ki-Hwan, Y. (2009). Effectiveness o f mind- fulness-based cognitive therapy as an adju­ vant to pharmacotherapy in patients with panic disorder or generalized anxiety disorder. Depression and Anxiety, 26, 601-606.

290 M easurem ent and Evaluation in Counseling and D evelopm ent 4 7 (4 )

Zack, M., Toneatto, T., & Streiner, D. (1998). The SCL-90 factor structure in comorbid substance abusers. Journal o f Substance Abuse, 10, 85-101.

A u th o r Biographies

Todd L. Grande, PhD is an assistant professor in the Clinical Mental Health Counseling Program at Wilmington University. He conducts research and consults in several counseling-related areas includ­ ing assessment, etiology o f psychopathology, gen­ der differences, measurement o f treatment effec­ tiveness, and pedagogy. He is a licensed professional counselor and licensed chemical dependency professional in Delaware.

Mark D. Newmeyer, EdD is an assistant professor in the School o f Psychology and Counseling at Regent University, where he is the program coordinator for the PhD in Counselor Education &

Supervision. He is a licensed counselor in both Ohio and Virginia, with over 15 years o f clinical experience.

Lee A. Underwood, PsyD is a Professor in the School o f Psychology & Counseling at Regent University. He conducts research and consults in areas including screening, assessment, and pro­ gram development for juvenile sex offenders, sur­ vivors o f human trafficking, and mentally ill offenders. Dr. Underwood is licensed as a psychol­ ogist in several states and is a certified sex offender treatment provider.

Cyrus R. Williams III is an assistant professor in the Counseling Department at Regent University. He holds a PhD in Counselor Education from the University o f Florida his primary research interests include: multiculturalism, advocacy counseling, addictive behaviors and career development. He is a licensed counselor in Virginia and Florida.

Copyright of Measurement & Evaluation in Counseling & Development (Sage Publications Inc. ) is the property of Sage Publications Inc. and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.

Copyright of Measurement & Evaluation in Counseling & Development is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.