W2-DQ

Research Article

Validation of Measures of Cyberbullying Perpetration and Victimization in Emerging Adulthood

Jungup Lee1, Neil Abell1, and Jennifer L. Holmes2

Abstract Objective: Cyber bullying represents a new and alarming form of bullying that potentially leads to serious and long-lasting consequences for young people; yet, there is a dearth of research on the assessment of cyberbullying behaviors among emer- ging adults. Thus, this study aims to close this gap by assessing the development and validation of the cyberbullying behavior scales for application in social work research and practice settings. Methods: Two scales, cyberbullying perpetration (CBP) and cyber- bullying victimization (CBV), were validated using a purposive sample of 286 undergraduate students aged 18 to 25. Results: Both CBP and CBV scales showed excellent reliability (a ¼ .93 for CBP and a ¼ .95 for CBV), good fit, and strong convergent validity. Conclusions: The cyberbullying behavior scales provide valid and reliable measures of emerging adults’ bullying behaviors. Implications for further social work research and practice are discussed.

Keywords scale development, scale validation, cyberbullying perpetration, cyberbullying victimization, emerging adulthood

Bullying is recognized as a pervasive social problem that

potentially results in severe and long-lasting consequences for

young people. In recent years, a new form of bullying, known

as cyberbullying, has emerged. Cyberbullying is an aggressive

or harmful behavior that occurs through electronic technologies

such as the Internet or mobile phones and can direct toward

and/or be carried out by an individual or a group (Belsey,

2006; Patchin & Hinduja, 2006). Due to the technical medium

involved, the unique characteristics of cyberbullying include

anonymity, free access to a time or place, and rapid dissemina-

tion. Cyberbullying behaviors have important consequences,

such as emotional distress, substance use, suicide and delin-

quent behavior, depression, and anxiety (Hemphill et al., 2012).

Youth who perpetrate cyberbullying behavior are more likely

to engage in substance use, delinquency, and aggression;

whereas, those who are cyber bullied by others are more likely

to feel sad, anxious, and fearful as well as to drop out of school

or to engage in suicidal ideation or self-injury (Ybarra, Diener-

West, & Leaf, 2007).

As cyberbullying has received extensive attention in the

popular media, research on cyberbullying behaviors is also

growing, at both the national and the international levels. The

majority of existing studies have focused on the prevalence

of the phenomenon (Kowalski, Limber, & Agatston, 2008;

Merch, 2009; Olweus, 2012), school-based bullying (Hazler,

Miller, Carney, & Green, 2001; O’Connell, Pepler, & Craig,

1999; Olweus, 1993; Patchin & Hinduja, 2006; Smith et al.,

2008; Wang, Ronald, & Tonja, 2009), and the comparative

analysis of traditional bullying and cyberbullying (Hay,

Meldrum, & Mann, 2010; Hemphill et al., 2012; Patchin &

Hinduja, 2011; Raskauskas & Stoltz, 2007). However, at

present, studies focusing on the measurement of cyberbully-

ing are still scarce.

Existing cyberbullying instruments are in the relatively

early stages of use and provide only preliminary investigations

of measurement characteristics. Modeled on measures of tradi-

tional bullying, two different methods are generally used for

instruments assessing cyberbullying, namely, the evaluation

of the presence of the phenomenon on the basis of a global

definition (Erdur-Baker & Kavsut, 2007; Olweus, 1993) and

the description of more specific bullying behaviors (e.g., perpe-

tration or victimization; Menesini, Nocentini, & Calussi, 2011;

Smith et al., 2008). They have been constructed as unidimen-

sional structures, often composed of binary items, generally

addressing cyberbullying perpetration (CBP) or cyberbullying

victimization (CBV).

1 College of Social Work, Florida State University, Tallahassee, FL, USA 2 College of Criminology and Criminal Justice, Florida State University, Talla-

hassee, FL, USA

Corresponding Author:

Jungup Lee, College of Social Work, Florida State University, 296 Champions

Way, Tallahassee, FL 32306, USA.

Email: [email protected]

Research on Social Work Practice 2017, Vol. 27(4) 456-467 ª The Author(s) 2015 Reprints and permission: sagepub.com/journalsPermissions.nav DOI: 10.1177/1049731515578535 journals.sagepub.com/home/rsw

More specifically, Hinduja and Patchin (2007, 2008)

employed two general cyberbullying measures, cyberbullying

victimization and cyberbullying offending. Adolescents were

asked ‘‘Have you been bullied online?’’ and ‘‘Have you bullied

others using online?’’ For this measure, cyberbullying included

the following items: bothering someone online, teasing in a

mean way, calling someone hurtful names, intentionally leav-

ing persons out of things, threatening someone, and saying

unwanted sexually related things to someone. Items in these

measures were dichotomously coded where 0¼ no and 1¼ yes

and showed acceptable reliability. Cronbach’s as were .76 for

cyberbullying victimization and .66 for cyberbullying offend-

ing. In addition, Menisini et al. (2011) created two separate

scales, one for cyberbullying and the other for cybervictimiza-

tion. Each scale contained 10 items, asking how often in the last

2 months youth had been bullied by or had bullied other students.

Although each item was originally designed with a 5-point

Likert-type scale (1 ¼ never, 2 ¼ only once or twice, 3 ¼ 2 or

3 times a month, 4 ¼ about once a week, and 5 ¼ several times

a week or always), it was binary recoded (e.g., 0 ¼ never; 1 ¼ from only once or twice to always) for the final. These scales

showed acceptable Cronbach’s as (cyberbullying a ¼ .77;

cybervictimization a¼ .80), and related factor analyses showed

that each performed best as a unidimensional model. There is no

current measurement standard in the field. We found no scales

specifically designed to measure multidimensional structures.

As such, existing measures may not reflect the underlying com-

plexity of cyberbullying phenomena or optimize psychometric

properties of the resulting cyberbullying instruments.

Over the past decade, a number of studies have also addressed

cyberbullying behavior for children or adolescents (Patchin &

Hinduja, 2006; Smith et al., 2008). A significant proportion of

children and youth are engaged in bullying behaviors across their

school years. Children and youth who are cyber bullied report a

series of problems, including emotional concerns such as anxi-

ety and depression (Bond, Carlin, Thomas, Rubin, & Patton,

2001; Nansel et al., 2001; Patchin & Hinduja, 2006), low self-

esteem (Patchin & Hinduja, 2010), lowered academic perfor-

mance (Juvonen, Nishina, & Graham, 2000), and self-harm or

suicidal behavior (Hay & Meldrum, 2010; Klomek, Marrocco,

Kleinman, Schonfeld, & Gould, 2007). It has been found that

cyberbullying experiences in childhood are highly associated

with cyberbullying behaviors among emerging adults aged

18–25 (Bellis, 2002; Stewart, Livingston, & Dennison, 2008).

In modern society, emerging adults have unique characteris-

tics. Emerging adulthood is the period between the ages of

18 and 25, when youths become more independent and explore

diverse life possibilities (Arnett, 2000). This period includes

late adolescence and young adulthood. Some believe that those

who are in emerging adulthood should constantly struggle with

‘‘identity exploration, instability, self-focus, feeling in-between,

and possibility’’ (Arnett, 2007, p. 69). Arnett (2007) argues that

the transition to adulthood is the longest process of growing up,

including a transition as well as a distinct life course stage.

These people are typically expected to get jobs, engage in postse-

condary education, or train to improve their skills. As such, from a

life course perspective, emerging adulthood is neither adoles-

cence nor young adulthood but overlaps the two (Arnett, 2007).

According to the Chapell et al’s. (2006) study with 119 under-

graduate students, bullying behaviors including perpetration, vic-

timization, or perpetration/victimization continue from childhood

into emerging adulthood. Over 70% of students who are bullied in

their childhood or adolescence bully others in their emerging

adulthood. Approximately 50% of students who experience per-

petration/victimization or perpetration behaviors in elementary

and high school repeat the pattern in university (Chapell et al.,

2006). Despite the significance of cyberbullying behaviors in

emerging adulthood, previous studies have mainly aimed to

develop methods of measuring cyberbullying for children or

adolescents. Given this, the development and initial validation

of comprehensive measures are needed for the insightful test-

ing of cyberbullying behavior among emerging adults.

In response to these issues, this study focuses on the initial

validation of the cyberbullying behavior scales in emerging

adulthood. The cyberbullying behavior scales were developed

as a pair of complementary self-report measures of CBP and

CBV that could be administered to emerging adults aged 18–25.

Each of the two scales contains three subscales describing

specific behaviors that encompass the range of cyberbullying

behavior. The development of the cyberbullying behavior

scales was aimed at providing multidimensional assessments

of CBP and CBV for a single individual. It is expected to pro-

vide more valid and comprehensive measures for examining

cyberbullying behaviors among emerging adults.

Method

Development of the CBP and CBV Scales

Scale conceptualization. Cyberbullying behavior refers to an

aggressive or harmful behavior directed toward an individual

or a group, carried out through electronic technologies (Belsey,

2006; Patchin & Hinduja, 2006). The definition of cyberbul-

lying behavior was used to guide the development of two

separate cyberbullying behavior scales, that is, CBP and CBV.

The first scale defined CBP as ‘‘aggressive or harmful behavior

directed towards an individual or a group using any form of

electronic communications technology, such as the internet

or mobile phones.’’ The second scale defined CBV as ‘‘being

the object of aggressive or harmful behavior by others using

any form of electronic communications device.’’ Each scale

consisted of three subscales including verbal/written bullying,

visual/sexual bullying, and social exclusion.

In CBP, verbal/written bullying was defined as sending

angry, rude, or vulgar online messages or saying mean things

using electronic communication with the intent to harm some-

one. Visual/sexual bullying meant sending or posting visually/

sexually incriminating things such as private or humiliating

pictures/videos to embarrass someone. Social exclusion was

defined as excluding someone from an online group activity

or social community with the intent to harm someone. In CBV,

verbal/written bullying referred to being sent angry, rude, or

Lee et al. 457

vulgar online messages or having mean things said to or about

you by others who are trying to hurt you. Visual/sexual bully-

ing referred to being sent visually/sexually incriminating things

such as private or humiliating pictures/videos by others trying

to hurt you. Social exclusion was defined as being excluded

from an online group activity or social community by someone

who wanted to make you feel left out.

Scale design. For the purpose of scale development, we were

initially guided in developing items for CBP and CBV by

measurements used in the studies by Hunt, Peters, and Rapee

(2012) and Cassidy, Jackson, and Brown (2009). For ease of

scoring and interpretation, both CBP and CBV items were

designed to be assessed using a 5-point Likert-type response

scale where 1 ¼ not at all and 5 ¼ very often. In subsequent

analyses, these responses were treated as equal-appearing

intervals (Nunnally & Bernstein, 1994). For the CBP subscales,

higher total scores indicated greater tendencies to carry out

aggressive or harmful behavior aimed at an individual or a

group by using any form of electronic communications tech-

nology. For the CBV subscales, higher total scores revealed

greater experience having been the object of aggressive or

harmful behavior from an individual or a group using any form

of electronic communications technology.

Furthermore, simple wording was considered important to

minimize respondents’ burden in reading and understanding

the scales. In order to consider the readability level of the CBP

and CBV scales, the Flesch-Kincaid reading score was assessed

(Kincaid, Fishburne, Rogers, & Chissom, 1975). The CBP and

CBV scales reported 9.7 U.S. grade level, indicating that aver-

age 10th-grade students (a 16-year-old) are able to read and

understand the items and instruction of these scales. Thus, the

readability level of the CBP and CBV scales is adequate for

emerging adults. Also, some reverse formatted items were

included to reduce the probability of acquiescence, affirma-

tion or agreement biases (DeVellis, 2003), and the respondents’

tendencies to drift into a form of autopilot, where their

responses are based more on a pattern they have somehow

slipped into.

Content validity. Eight expert panelists were invited to refine the

item pools and to critique the fit between proposed items and

their intended construct definitions. An initial set of items was

presented to four professors who had conducted research on

bullying or relevant behaviors and to four doctoral students

who had conducted bullying-related fieldwork or bullying stud-

ies. The experts were provided with construct definitions and

asked to rate the extent to which the content of each rated item

matched the target definition on a scale from 1 ¼ not at all to

5 ¼ very well. Most provided detailed comments on the word-

ing and intent of proposed items. Ratings were based on broad

construct definitions, with two measures (CBP and CBV). The

mean content validation indices were computed by averaging

panelists’ evaluations across all proposed scale items.

The following three methods were employed to revise the

items, constructs, or definitions based on panelists’ feedback.

First, all items that mean scored below 3.5 were immediately

removed. Nine items in the original CBP and CBV pools received

mean ratings of less than 3.5. Second, items that mean scored in

the 3.5–4.0 range were carefully evaluated using qualitative find-

ings from the panelists and were then deleted, revised, or retained.

The panelists described some suggestions or comments when

they had concerns regarding the items. They illustrated that ‘‘item

26 and 34 are very similar to item 25 . . . sound like all the same to

me . . . smaller number of items. As it is, in empirical terms,

redundant,’’ or ‘‘I think there is overlap across some items includ-

ing #25, #26, and #34. What about using a question out of the

three.’’ Of the 21 items mean scored in the 3.5–4.0 range, 2 items

were deleted because all expert panelists reported that these items

were almost identical to other items.

Lastly, all qualitative data provided by the panelists were

assessed and items were flagged for removal (pending subse-

quent psychometric analysis), revised, or retained accordingly.

In addition to considering item revision, constructs were also

revised in accordance with panelists’ open-ended feedback.

The preliminary item pool consisted of four constructs, that

is, verbal/written bullying, visual bullying, sexual bullying, and

social exclusion. However, the majority of panelists indicated

that some items intended for visual bullying and sexual bully-

ing overlapped across the two proposed constructs. Two pane-

lists suggested two subscales including visual bullying and

sexual bullying could be condensed to a new construct:

Designing multidimensional instruments is good for psychometric

properties . . . but, in fact, there is overlap across your constructs.

I mean most items in visual bullying and sexual bullying are similar

to . . . also, sexual bullying behaviors in cyber space could be

occurred (sic) by using visual images such as mean photos or pic-

tures. In my opinion, combining those two constructs allows your

survey to get a succinct design . . . reduce major problems with

non-response and random measurement error (with subjects paying

too little attention).

Based on panelists’ feedback, the initial total pool of 90 items

was reduced to 79. Eleven items were removed, 36 items were

revised, and 43 items were retained from the initial CBP and

CBV item pools.

Sampling

A purposive sample of all undergraduate students enrolled in

social work, criminology and criminal justice, and other social

science disciplines at a major public university in the south-

eastern United States was drawn for validation analyses.

Undergraduate students aged 18–25 who use electronic com-

munication technologies such as the Internet, cell phones, and

e-mail were targeted for participation. This population was

very familiar with online activities, which made it a logical

target for examining the psychometric properties of the cyber-

bullying behavior scales. The majority of participants had con-

sistent access to electronic devices. Approximately 98% of the

participants used laptop computers and 99% of those used

458 Research on Social Work Practice 27(4)

mobile phones daily (see Table 1). Very often, the participants

utilized online services for sending e-mails (M ¼ 4.07, SD ¼ 1.01, range from 1 to 5), sending text messages (M ¼ 4.30,

SD ¼ 1.25, range from 1 to 5), and using social networking

services (SNSs; M ¼ 4.26, SD ¼ 1.18, range from 1 to 5). In

addition, the participants were likely to understand the defini-

tion and nature of cyberbullying behaviors because they took

at least one course relevant to bullying in the fields of social

science. As most prior studies used children or adolescents as

the target population for bullying and school violence examina-

tion instruments (Hunt, Peters, & Rapee, 2012), a university-

based sample was important to developing a valid cyberbullying

behavior scale for emerging adults. While we believe the cur-

rent sample is appropriate to an understanding of cyberbullying

potential among young adults, our findings are sample depen-

dent, and their generalizability to others is unknown. The uni-

versity’s Institutional Review Board approved the study. All

respondents participated voluntarily and anonymously.

Data Collection Instrument

Respondents involved in the data collection of the cyberbully-

ing behavior scales were given two separate measures, namely,

CBP and CBV. The CBP scale had 41 items consisting of four

domains: 11 items on verbal/written bullying, 9 items on visual

bullying, 11 items on sexual bullying, and 10 items on social

exclusion. The CBV scale had 38 items consisting of 11 items

on verbal/written bullying, 10 items on visual bullying, 10 items

on sexual bullying, and 7 items on social exclusion. Both CBP

and CBV items were designed to be measured using a 5-point

Likert-type scale, ranging from 1¼ not at all to 5¼ very often.

To assist in characterizing the sample, the survey instruments

included demographic questions. All respondents were asked

for information on their age, gender, ethnicity, educational sta-

tus, major, academic performance, and mother education level.

Two standardized scales capturing constructs that should

theoretically relate to the concept of cyberbullying were chosen

for inclusion in the initial validation study of the two cyberbul-

lying behavior scales, that is, the Aggression Questionnaire

(AQ) and the Multidimensional Peer-Victimization Scale (MPVS).

The AQ was included to address the construct validity of CBP.

The AQ measured aggressive behavior using a questionnaire

developed by Buss and Perry (1992) as the most relevant mea-

sure of aggressive behavior. The AQ consisted of 29 items

measured on a 5-point Likert-type scale, ranging from 1 ¼ strongly disagree to 5¼ strongly agree. It reported satisfactory

internal reliability (a ¼ .89) (Buss & Perry, 1992). The internal

reliability of the AQ in this study was also found to be satisfac-

tory (a ¼ .91). Aggressive behavior is expected to correlate

positively with CBP (Hay et al., 2010). To address the evidence

of construct validity of CBV, the MPVS (Mynard & Joseph,

2000) was also included in this study. The MPVS consisted

of 16 items measured on a 3-point Likert-type scale from

1 ¼ not at all to 3 ¼ more than once. The internal reliability

of each subscale was reported to be satisfactory, which can

be seen as follows: physical victimization (a ¼ .85), verbal

victimization (a ¼ .75), social manipulation (a ¼ .77), and

attacks on property (a ¼ .73; Mynard & Joseph, 2000). In this

study, the a coefficients of internal consistency reliability were

also found to be satisfactory: .81 for physical victimization,

.79 for verbal victimization, .76 for social manipulation, and

.77 for attacks on property. Peer-victimization is expected to

correlate positively with CBV (Menesini et al., 2011; Mishna,

Saini, & Solomon, 2009; Slonje & Smith, 2008). If these con-

structs were found to correlate in the expected direction with

scores on the CBP and CBV, this would provide evidence for

the convergent validity of the CBP and CBV scales.

Data Collection Method

Both in-person and online surveys were utilized to collect

responses for this study. During the classroom survey, students

were provided with a cover sheet outlining the purpose of the

study and their rights. Students were advised that completing

Table 1. Respondent Demographics.

N (%) M (SD) Range

Gender Female 177 (61.9) Male 109 (38.1)

Age 20.92 (1.54) 18–25 Ethnicity

White 198 (69.5) Black 27 (9.5) Hispanic/Latino 36 (12.6) Other 24 (8.4)

Major Social work 92 (32.4) Criminology 130 (45.8) Other 62 (21.8)

Current status Freshmen 6 (2.1) Sophomore 35 (12.3) Junior 151 (53.0) Senior 93 (32.6)

GPA 3.29 (0.47) 2.00–4.00 Mother education level

Less than high school graduate

9 (3.2)

High school graduate or GED 64 (22.9) Some college 70 (25.0) College graduate 137 (48.9)

Using laptop computer Yes 275 (97.5) No 7 (2.5)

Using mobile phone Yes 278 (98.6) No 4 (1.4)

Sending e-mails 4.07 (1.01) 1–5 Sending text messages 4.30 (1.25) 1–5 Using SNS (e.g., Facebook and

Twitter) 4.26 (1.18) 1–5

Note. M ¼ mean; SD ¼ standard deviation; SNS ¼ social networking service; GPA ¼ grade point average; GED ¼ general equivalency diploma. N ¼ 286.

Lee et al. 459

the survey constituted voluntary agreement to participate. For

the online survey, we approached students by e-mail about the

nature of the study. Those who were interested were directed

to access the survey through an embedded e-mail link. Online

data collected through Qualtrics was separately maintained

from in-person data collected within classrooms. A variable

during data entry was created to code whether the survey was

completed online or in-person, allowing the researchers to

track these distinct methods over the course of the data analy-

sis. All data collected both online and in-person were ulti-

mately combined into a single data file, which was retained

in a password protected file.

Data Management and Item Changes Summary

The total number of people in the sampling frame was 450,

and 345 students answered the survey. The response rate was

approximately 76.7%. Of 345 responses received (308 by in-

person survey; 37 by online survey), we excluded 47 cases due

to extreme outliers or excessive patterns of missing data. When

a respondent omitted an entire subscale, the cases were deleted

as unusable. Also, 12 cases were removed because they did not

meet inclusion criteria (i.e., participants’ age are not between

18 and 25). Finally, responses from 286 undergraduate students

enrolled in social work, criminology and criminal justice, and

other social science disciplines were retained for analysis.

Following data cleaning, missing values were replaced

using the expectation-maximization (EM) method of imputa-

tion (Hill, 1997). Examination of missing data was conducted

with SPSS Missing Values 20. The impact of missing data

across the 79 proposed items of the CBP and CBV was assessed

by examining the number of cases missing per item. Of the

79 items, 26 items had missingness, ranging from 0.3% (n ¼ 1)

to 0.6% (n ¼ 2). All missing data fell below the exclusionary

criterion of 5%, and the impact of missing data was extremely

minimal. Little’s missing completely at random (MCAR) test

revealed that data were not MCAR (w2 ¼ 1181.66, df ¼ 839,

p < .05). Thus, we used the EM method in the replacement

of missing values for proposed CBP and CBV items. This

method took greater advantage of the structure in the data com-

pared with single imputation methods and uses a broad range of

variables in the replacement of missing values (Kline, 2011).

After imputing missing values, frequency distribution was

tested by investigating skewness and kurtosis. Items with the

absolute values of skewness >3 are described as extremely

skewed, and those with the absolute values of kurtosis >10

indicate problems, which can be considered as nonnormal

distribution (Abell, Springer, & Kamata, 2009; Kline, 2011).

Based on this guideline, 32 items were with extreme skewness

(5.73–10.48) or kurtosis (29.65–121.14) indices and were dis-

carded from the final analyses, which were conducted with

LISREL 9.1 and SPSS 20.

Following examination of any errors in data entry, missing

values, and frequency distributions, 47 items, all normally distrib-

uted in relation to the criterion identified earlier, were retained

for the final item pool. The CBP had 20 items, consisting of

9 items on verbal/written bullying, 5 items on visual/sexual

bullying, and 6 items on social exclusion. The CBV had

27 items, consisting of 10 items on verbal/written bullying,

10 items on visual/sexual bullying, and 7 items on social

exclusion. The full items of the CBP and CBV measures are

shown in Appendix A.

Results

Preliminary Analysis

Of in-person and online surveys distributed, the final sample

was 286 undergraduate students. As detailed in Table 1,

respondents were 62% female and 70% White. The mean age

of undergraduate students was 20.92 years (SD ¼ 1.54) with

a range of 18–25. The majority of respondents were enrolled

in Social work (32%) or criminology (46%), and nearly 53% of those were juniors. Respondents reported a mean GPA of

3.29 points (SD ¼ .47). Approximately 49% of respondents’

mothers had graduated from college, 25% reported completing

some college, and 23% reported completing only high school.

Normality was tested by calculating means, standard devia-

tion, skewness, and kurtosis (see Table 2). Kline (2011) indi-

cated that the absolute values of skewness should be less

than 3.0, and those of kurtosis should be less than 10.0 to be

supposed as acceptable normality. According to this rule, all

items were normally distributed and multivariate normality

was confirmed. Total mean scores of the CBP and CBV scales

after validation, as detailed subsequently, were 1.28 (SD¼ .59)

and 1.37 (SD ¼ .65), respectively. Similarly, after validation,

subscales of the CBP and CBV mean scores were (1) 1.30

(SD ¼ .59) for verbal/written perpetration and 1.57 (SD ¼ .79) for verbal/written victimization; (2) 1.20 (SD ¼ .51) for

visual/sexual perpetration and 1.27 (SD ¼ .62) for visual/

sexual victimization; and (3) 1.31 (SD ¼ .68) for social

exclusion perpetration and 1.21 (SD ¼ .52) for social exclu-

sion victimization. The observed means and standard devia-

tions of the CBP and CBV scales indicated that the majority

of respondents in this sample have rarely perpetrated or been

victimized by cyberbullying.

Reliability

Analyses of internal consistency were conducted with Cron-

bach’s a coefficients for each hypothesized subscale and

stratified a coefficients for the global scales of the CBP and

CBV. Investigation of alpha-if-item-deleted statistics for each

CBP and CBV subscale identified that items would be accepta-

ble (.70 < a < .95). Table 3 provides the final scale and subscale

reliabilities for the proposed cyberbullying behavior scales.

Two stratified a coefficients for the global CBP and CBV

scales exceeded .90 (The stratified a ¼ .93 for the global CBP;

the stratified a ¼ .95 for the global CBV). Only one reliability

coefficient (visual/sexual perpetration, a ¼ .73) might be clas-

sified as marginally acceptable. The remaining reliability coef-

ficients were very good. Three reported reliability coefficients

ranging from .80 to .89 and four ranging from .90 to .95.

460 Research on Social Work Practice 27(4)

Standard errors of measurement (SEMs) showed the unstan-

dardized unreliability of test scores by suggesting a confidence

interval of observed scores (Harvill, 1991). This quantity was

computed by using the standard deviation of observed scores

and the coefficient a. One standard for such analyses proposes

that SEM should ideally be less than or equal to 5% of the

possible scale score range (Springer, Abell, & Nugent, 2002).

Because the CBP and CBV items ranged from 1 to 5, desirable

SEMs should be less than .2, and the results indicated all scales

and subscales met this criterion.

Factorial Validity

To further investigate the multidimensional structures of the

CBP and CBV, confirmatory factor analyses (CFAs) were con-

ducted with LISREL 9.1 (see Figure 1). CBP had 210 observa-

tions and 43 parameters, and CBV had 378 observations and

57 parameters. Both CBP and CBV had fewer estimable para-

meters than observations which resulted in an overidentified

model. It was computed to look for model fit of the observed

CBP and CBV item responses as they were predicted by the

data in relation to each latent scale. Target criteria were w2/df

ratios < 3, comparative fit index (CFI) and Tucker-Lewis index

(TLI) > .90, root mean square error of approximation (RMSEA)

< .08, and standardized root mean square residual (SRMR)

< .10 (Kline, 2011). These criteria indicated a model-data fit

approaching satisfactory within some statisticians’ suggested

cut-off points (Hu & Bentler, 1999; McDonald & Ho, 2002).

Table 4 shows model fits of the CBP and CBV scales.

Table 2. Corrected Item-Total Correlation Coefficients, Means, Standard Deviations, Skewness, and Kurtosis for the CBP and CBV Scales.

CBP CBV

Item ITC M SD Skewness Kurtosis Item ITC M SD Skewness Kurtosis

Verbal/written 1.30 .59 2.31 6.11 Verbal/written 1.57 .79 1.58 2.66 1 .56 1.62 .76 1.38 2.55 21 .60 2.05 .81 .47 .18 2 .61 1.58 .81 1.54 2.64 22 .71 1.67 .82 1.13 1.05 3 .66 1.20 .52 2.93 9.13 23 .69 1.53 .79 1.47 1.72 4 .65 1.24 .56 2.36 6.11 24 .73 1.49 .76 1.56 2.07 5 .68 1.20 .52 2.89 9.04 25 .74 1.66 .86 1.46 2.30 6 .69 1.37 .60 1.49 1.62 26 .70 1.50 .81 1.76 2.98 7 .65 1.17 .47 2.95 9.01 27 .76 1.42 .77 1.97 3.80 8 .72 1.20 .55 2.47 7.31 28 .59 1.35 .72 2.32 5.81 9 .62 1.15 .49 2.81 7.54 29 .69 1.26 .59 2.36 5.07

30 .72 1.79 .92 1.25 1.65 Visual/sexual 1.20 .51 2.64 6.86 Visual/sexual 1.27 .62 2.38 7.37 10 .50 1.20 .49 2.49 5.38 31 .55 1.32 .62 2.02 3.82 11 .51 1.20 .54 2.76 6.84 32 .56 1.15 .46 2.98 9.31 12 .54 1.19 .48 2.56 6.52 33 .60 1.27 .61 2.48 6.64 13 .69 1.25 .55 2.40 5.85 34 .66 1.33 .66 2.44 7.23 14 .48 1.18 .47 2.97 9.71 35 .62 1.40 .72 2.12 5.09

36 .68 1.21 .59 2.51 8.85 37 .72 1.17 .51 2.62 9.32 38 .73 1.24 .65 2.43 9.03 39 .67 1.24 .64 2.14 8.76 40 .68 1.34 .72 2.09 5.69

Social exclusion 1.31 .68 2.45 7.25 Social exclusion 1.21 .52 2.56 7.34 15 .70 1.34 .71 2.46 6.76 41 .59 1.27 .57 2.27 5.08 16 .67 1.48 .83 2.02 4.28 42 .64 1.33 .61 1.97 3.79 17 .67 1.26 .70 2.37 8.67 43 .51 1.21 .51 2.69 8.04 18 .70 1.22 .59 2.70 8.73 44 .55 1.24 .56 2.56 7.04 19 .68 1.21 .57 2.86 9.08 45 .60 1.14 .46 2.79 8.93 20 .60 1.35 .67 2.28 5.96 46 .54 1.14 .44 2.84 9.34

47 .56 1.14 .46 2.83 9.19

Note. ITC ¼ item-total correlation; M ¼ mean; SD ¼ standard deviation; CBP ¼ cyberbullying perpetration; CBV ¼ cyberbullying victimization.

Table 3. Reliability and SEM for the CBP and CBV Scales.

Scale Subscale n of Items Reliability (a) SD SEM

CBP Global 20 .93a .37 .11 Verbal/written 9 .86 .41 .15 Visual/sexual 5 .73 .33 .17 Social exclusion 6 .87 .53 .19

CBV Global 27 .95a .43 .10 Verbal/written 10 .92 .59 .17 Visual/sexual 10 .89 .44 .15 Social exclusion 7 .91 .41 .13

Note. SD ¼ standard deviation; SEM ¼ standard error of the mean; CBP ¼ cyberbullying perpetration; CBV ¼ cyberbullying victimization. a Stratified a.

Lee et al. 461

We conducted the two second-order CFA model structures

of the 20-item CBP and the 27-item CBV scales. Results for the

second-order models of CBP and CBV identified problems

with the exception of RMSEA (.10 and .08, respectively) and

SRMR (.08 and .08, respectively). Next, estimation of modifi-

cation indices (MI) permitting error covariances consistent

with the proposed scale structures led to respecified models for

the CBP and CBV scales. Error covariances were permitted to

correlate between items within each distinct factor when they

were associated with significant changes in model fit (Hu &

Bentler, 1999). In all cases, covariances were only permitted

when the identified items shared content consistent with the

construct definition for the target latent indicator. Inspection

of MI resulted in allowing 7 error covariances on the CBP scale

and 16 on the CBV scale. In the CBP, correlating five error var-

iances on the verbal/written perpetration subscale was judged

conceptually consistent with the proposed model because those

items related specifically to mean messages respondents had

sent. Correlating one error variance on the visual/sexual perpe-

tration subscale appeared to make substantive sense because

the items related specifically to harmful pictures/videos respon-

dents had posted. Correlating one error variance on the social

exclusion perpetration subscale was similarly permitted because

the items related specifically to blocking online community

respondents to make someone feel left out. In the CBV, corre-

lating six error variances on the verbal/written victimization

subscale was judged conceptually consistent with the proposed

model because those items related specifically to mean mes-

sages respondents had received. Similarly, correlating six error

variances on the visual/sexual victimization subscale appeared

to make substantive sense because those items related specifi-

cally to received pictures and sexual things by someone. Corre-

lating five error variances on the social exclusion victimization

subscale appeared to make substantive sense because those

items related specifically to blocked online community partic-

ipation by individuals and/or groups.

After conducting the respecified models with error covar-

iances, resulting coefficients provided better model fit in each

scale. Results for the respecified models of CBP and CBV met

target criteria of good model fit with w2/df (ratio ¼ 1.97 and

2.86, respectively), CFI (.95 and .97, respectively), TLI (.94

and .95, respectively), RMSEA (.08 and .08, respectively), and

SRMR (.06 and .07, respectively). All the fit indices supported

the indicated multidimensional structures of the CBP and CBV

scales. The final CBP and CBV item pools in the second-order

forms were specified as the models best describing the under-

lying factor structures of the cyberbullying behavior scales.

Table 5 shows reliability estimates, residual variance for

each item, squared multiple correlations, and parameter esti-

mates for the CBP and CBV models. The results of reliability

and residual variance of each indicator and factor showed evi-

dence of internal consistency and factor structure. All support

model fit in that none are > |2.0| (Brown, 2006). As can be seen

from the squared multiple correlations, the CBP model was

best able to explain verbal/written perpetration (90%) and

visual/sexual perpetration (77%) followed by social exclusion

(46%). The CBV model was best able to explain visual/sexual

victimization (87%) and least able to explain verbal/written

victimization (61%) and social exclusion (33%). The parameter

estimates for the CBP and CBV models reported all the items

have factor loadings > .50.

Convergent Validity

Convergent validity was established by assessing the CBP and

CBV scales against theoretically related variables or constructs

from the literature. It was hypothesized that each of the CBP

Figure 1. The model of the cyberbullying behavior scales. Note. CBP ¼ cyberbullying perpetration; CBV ¼ cyberbullying victimization.

Table 4. Confirmatory Factor Analyses of the CBP and CBV Scales.

Scale Model w2/df CFI TLI RMSEA SRMR

CBP Second order Initial 4.39 .83 .81 .10 .08 Respecifieda 1.97 .95 .94 .08 .06

CBV Second order Initial 3.53 .88 .83 .08 .08 Respecifiedb 2.86 .97 .95 .08 .07

Note. CFI ¼ comparative fit index; TLI ¼ Tucker-Lewis index; RMSEA ¼ root mean square error of approximation; SRMR ¼ standardized root mean square residual; CBP¼ cyberbullying perpetration; CBV¼ cyberbullying victimization. aPermitting seven error covariances. bPermitting 16 error covariances.

462 Research on Social Work Practice 27(4)

and CBV subscales would individually correlate with other

measures that had been established in the literature. Evi-

dence of convergent validity was found based on the statis-

tical and practical significance of the relationships between

the CBP and the CBV subscales and their hypothesized cor-

relates as expressed through bivariate correlations and their

associated squared coefficients, representing effect size

(Abell et al., 2009). More specifically, evidence of conver-

gent validity is established if the CBP and CBV subscales

strongly correlate with variables that are theoretically

believed to be related to the bullying behaviors. Two standar-

dized and previously validated measures including the AQ

and the MPVS were used for examining convergent validity

(see Table 6).

We hypothesized that the AQ would correlate positively

with the CBP global and subscales. As we expected, there are

significantly positive correlations between the AQ and the CBP

global and subscales (r ¼ .37 for the global, r ¼ .38 for verbal/

written perpetration, r ¼ .34 for visual/sexual perpetration, and

r¼ .24 for social exclusion perpetration). Effect sizes assessing

the level of these relationships were evaluated as r2 statistics

and ranged from .06 for social exclusion to .14 for the global

CBP and verbal/written perpetration. We also hypothesized

that the CBV global and subscale scores would be positively

correlated with the global MPVS scores. All correlations with

the global CBV and three subscales demonstrated statistical

significance in the hypothesized directions. In other words, the

MPVS was positively correlated with the CBV global and

Table 5. Standardized Parameter Estimates, Reliability, Squared Multiple Correlations, and Standardized Residual Variance for Confirmatory Factor Analytic Model of the CBP and CBV Scales.

CBP CBV

Item b Alpha-if-Item Deleted R2 Residual Variance Item b Alpha-if-Item Deleted R2 Residual Variance

Verbal/written .98 a ¼ .86 .90 .03 Verbal/written .78 a ¼ .92 .61 .39 1 .52 .86 .27 .73 21 .67 .91 .45 .55 2 .56 .85 .31 .69 22 .74 .91 .54 .46 3 .67 .84 .45 .55 23 .69 .91 .48 .52 4 .71 .84 .50 .50 24 .74 .91 .55 .45 5 .72 .84 .52 .48 25 .76 .90 .58 .43 6 .68 .85 .46 .55 26 .75 .91 .57 .43 7 .66 .85 .44 .56 27 .82 .90 .67 .36 8 .73 .84 .54 .47 28 .59 .91 .35 .65 9 .66 .85 .43 .57 29 .70 .91 .48 .52

30 .77 .90 .59 .42 Visual/sexual .89 a ¼ .73 .77 .23 Visual/sexual .93 a ¼ .89 .87 .17 10 .54 .69 .29 .71 31 .56 .89 .32 .68 11 .54 .70 .30 .70 32 .60 .88 .36 .64 12 .65 .67 .42 .58 33 .62 .88 .38 .62 13 .69 .70 .47 .53 34 .65 .88 .42 .58 14 .58 .70 .33 .67 35 .63 .88 .40 .60

36 .75 .88 .56 .44 37 .79 .87 .63 .37 38 .78 .87 .61 .39 39 .71 .87 .51 .49 40 .66 .88 .43 .57

Social exclusion .68 a ¼ .87 .46 .54 Social exclusion .57 a ¼ .91 .33 .67 15 .65 .83 .43 .57 41 .61 .90 .37 .63 16 .58 .86 .33 .67 42 .60 .91 .36 .65 17 .64 .85 .41 .60 43 .83 .89 .68 .32 18 .89 .83 .81 .19 44 .84 .88 .70 .30 19 .90 .84 .81 .19 45 .90 .89 .81 .19 20 .66 .85 .43 .57 46 .81 .89 .66 .34

47 .81 .89 .65 .35

Note. CBP ¼ cyberbullying perpetration; CBV ¼ cyberbullying victimization. All reported estimates were significant at a ¼ .05 level.

Table 6. Convergent Validity for the CBP and CBV Scales.

AQ MPVS

CBP r r2 CBV r r2

Global .37** .14 Global .31** .10 Verbal/written .38** .14 Verbal/written .30** .09 Visual/sexual .34** .12 Visual/sexual .28** .08 Social exclusion .24** .06 Social exclusion .21** .04

Note. AQ ¼ aggression questionnaire; MPVS ¼ multidimensional peer victimi- zation scale; CBP ¼ cyberbullying perpetration; CBV ¼ cyberbullying victimization. **p < .01.

Lee et al. 463

subscales (r¼ .31 for the global, r¼ .30 for verbal/written vic-

timization, r ¼ .28 for visual/sexual victimization, and r ¼ .21

for social exclusion victimization). Effect sizes were generally

small, ranging from .04 to .10.

Discussion

Despite dramatically escalating the cyberbullying studies,

cyberbullying behaviors including perpetration and victimiza-

tion for emerging adults have not been systematically measured

in the literature. This study aimed to develop and validate

measurements to fill this knowledge gap. The cyberbullying

behavior scales have several strengths psychometrically that

would lend support to their usefulness among emerging

adults who have experienced cyberbullying behaviors. The

findings of the CBP and CBV scales reveal strong evidence

of psychometric properties. Content validity was well estab-

lished by ratings from eight expert panelists. Reliability

coefficients and SEM provided strong evidence of internal

consistency and factor structure, respectively. The stratified

a scores for the CBP and CBV global scales and Cronbach’s

a scores for each subscale remained strong indicators of

reliability. The results of CFAs demonstrated good model fit

along the underlying factor structures of the CBP and CBV

measures. All convergent validity indicators were significantly

correlated with their respective CBP and CBV global and sub-

scales, even though effect sizes associated with standardized

scales were modest. The results of the convergent validity indi-

cated more closely related scales measured what they were

intended to measure.

The CBP and CBV scales provide valid and reliable struc-

tures of measurements for use among university students as

well as other bullying program developers and implementers.

The CBP and CBV scales demonstrated strong evidence of

encouraging psychometric properties with the multidimen-

sional structures. In addition, our conceptualization and psy-

chometric performance of the multidimensional structures are

useful for discerning not only emerging adults’ CBP but also

their CBV at the same time.

In brief, the CBP and CBV scales have solid potential for

use in both clinical work and future research. These scales

could be useful for university counselors as they devise com-

prehensive assessment programs or interventions for students

who have experienced CBP or CBV. More recently, some uni-

versity students have been cyberbullied from others through

Internet, mobile phone, and SNSs including Facebook and

Twitter (Schenk & Fremouw, 2012; Walker, Sockman, &

Koehn, 2011). Often, those who have been bullied in school

could lead to more severe behaviors, such as mental health dis-

orders, substance abuse, and suicide. Thus, university counse-

lors need to assess cyberbullying behaviors for students’

safety both in and out campus.

Furthermore, the issue of CBP and CBV behaviors is of con-

cern for social workers. Although cyberbullying has received

extensive attention in the media and among the public, there

is a discouraging dearth of adequate cyberbullying prevention

and intervention strategies (Mishna, Cook, Saini, Wu, &

MacFadden, 2011). A valid and reliable assessment of cyber-

bullying makes it possible for social workers not only to

develop and implement cyberbullying prevention programs but

also to educate people in community with regard to the poten-

tial risks associated with cyberbullying behaviors. In this sense,

the CBP and CBV scales for emerging adults could provide one

step toward creating adequate tools that social workers might

use to guide clients understanding of the cyberbullying phe-

nomena and its consequences.

In addition to clinical work, the CBP and CBV scales could

be useful in various studies examining cyberbullying behaviors

among emerging adults because a majority of cyberbullying

instruments have been utilized for children and adolescence

so far (Ang & Goh, 2010; Menesini et al., 2011; Mishna, Cook,

Gadalla, Daciuk, & Solomon, 2010; Slonje & Smith, 2008;

Wright, Burnham, Inman, & Ogorchock, 2009). It could be also

helpful for researchers in the study of diverse effects of cyber-

bullying behaviors among emerging adults. For instance, by

using the CBP and CBV scales, researchers can explore the

relationship between CBP and CBV within the same person,

examine the comparative study between conventional bullying

and cyberbullying, and investigate predictors of cyberbullying

behaviors and long-term effects of cyberbullying behaviors.

Taken together, program development and implementation for

emerging adult groups experiencing cyberbullying behaviors

could now be tested with the CBP and CBV scales in both clin-

ical and academic areas.

Although this initial validation shows promising results,

there remain several limitations in the available evidence. First,

given that the reported findings are only based on undergradu-

ate students in a major public university in the southeastern

United States, replicating the findings among emerging adults

in other contexts is necessary to enhance generalizability. In

addition, participants were not randomly sampled. When com-

pared to a probability sampling, the nonprobability sampling

method is more likely to threaten the generalizability of the

findings. Thus, the generalizability of the findings to other

emerging adult populations should be considered with caution.

Finally, although other options might have been pursued regard-

ing our decisions on handling nonnormality in the initial item

pool, we focused exclusively on items meeting criteria for

normally distributed responses when developing these CBP

and CBV scales. Addressing nonnormality by other means

could conceivably have yielded different final item-scale

compositions.

The CBP and CBV scales are very promising instruments

for assessing cyberbullying behaviors among emerging adults

for use in both research and clinical settings. The initial vali-

dation of the CBP and CBV scales have demonstrated strong

psychometric properties for measuring multidimensional aspects

of both perpetration and victimization. Using the CBP and CBV

scales in future research and applications in clinical fields might

enhance confidence in researchers’ and clinicians’ capability

to better understand and effectively respond to both antece-

dents and consequences of cyberbullying experience.

464 Research on Social Work Practice 27(4)

Appendix A

CyberBullying Perpetration (CBP) and CyberBullying Victimization (CBV) Scales

CyberBullying Perpetration (CBP) Scale

CyberBullying Victimization (CBV) Scale

Drawing from your own experiences, please circle the answers that fits best, where:

1 ¼ Not at all 2 ¼ Rarely 3 ¼ Sometimes 4 ¼ Often 5 ¼ Very often

Verbal/written perpetration

1. I have sent someone mean text messages on the mobile phone to harm the person. 1 2 3 4 5 2. I have said mean things about someone on instant messenger or in chat rooms with intent to upset the person. 1 2 3 4 5 3. I have sent someone mean e-mails with intent to harm the person. 1 2 3 4 5 4. I have posted hurtful messages on Facebook or Twitter to damage the person’s reputation. 1 2 3 4 5 5. I have attempted with intent to harm another person by sending threatening statements via e-mail or text message. 1 2 3 4 5 6. I have never said mean things about someone to their friends on instant messengers or in chat rooms to damage the person’s

relationship. 1 2 3 4 5

7. I have spread rumors about someone online to damage the person’s reputation. 1 2 3 4 5 8. I have sent someone insulting online messages repeatedly. 1 2 3 4 5 9. I have said mean things about someone on websites repeatedly to embarrass the person. 1 2 3 4 5

Visual/sexual perpetration

10. I have posted embarrassing pictures or videos of someone online without their permission to damage the person’s reputation. 1 2 3 4 5 11. I have posted humiliating pictures or videos of someone on websites to embarrass the person. 1 2 3 4 5 12. I have sent never sexually explicit things to someone via e-mail or text message to embarrass the person. 1 2 3 4 5 13. I have teased someone about his/her appearance online to emotionally harm the person. 1 2 3 4 5 14. I have made sexual jokes about someone online to damage the person’s reputation. 1 2 3 4 5

Social exclusion perpetration

15. I have blocked someone in a chat room to harm the person. 1 2 3 4 5 16. I have blocked someone on an instant messenger to upset the person. 1 2 3 4 5 17. I have rejected someone’s request playing online games together to harm the person. 1 2 3 4 5 18. I have excluded someone from online community groups to make them feel left out. 1 2 3 4 5 19. I have never excluded someone from online group activities to make them feel left out. 1 2 3 4 5 20. I have ignored someone’s comments on social community online to embarrass the person. 1 2 3 4 5

Verbal/written victimization

1. I have received mean text messages on the mobile phone which made me uncomfortable. 1 2 3 4 5 2. Someone has said mean things about me on instant messengers or in chat rooms to upset me. 1 2 3 4 5 3. Someone has posted hurtful messages about me on Facebook or Twitter to damage my reputation. 1 2 3 4 5 4. I have been sent threatening statements via e-mail or text message which made me insecure. 1 2 3 4 5 5. Someone has never said mean things about me to my friends on instant messengers or in chat rooms to damage my

relationship. 1 2 3 4 5

6. People have spread rumors about me online to damage my reputation. 1 2 3 4 5 7. I have received insulting online messages from someone repeatedly. 1 2 3 4 5 8. I have continued to receive mean text messages or e-mails even after I have asked the sender to stop. 1 2 3 4 5 9. People have said mean things about me on websites repeatedly to embarrass the person. 1 2 3 4 5 10. I have received intentional messages from someone which made me upset. 1 2 3 4 5

(continued)

Lee et al. 465

Declaration of Conflicting Interests

The authors declared no potential conflicts of interest with respect to

the research, authorship, and/or publication of this article.

Funding

The authors received no financial support for the research, authorship,

and/or publication of this article.

References

Abell, N., Springer, D. W., & Kamata, A. (2009). Developing and

validating rapid assessment instruments. New York, NY: Oxford

University Press.

Ang, R. P., & Goh, D. H. (2010). Cyberbullying among adolescents:

The role of affective and cognitive empathy and gender. Child Psy-

chiatry and Human Development, 41, 387–397.

Arnett, J. J. (2000). Emerging adulthood: A theory of development

from the late teens through the twenties. American Psychologist,

55, 469–480.

Arnett, J. J. (2007). Emerging adulthood: What is it, and what is it

good for? Child Development Perspectives, 1, 68–73.

Bellis, M. D. (2002). Developmental traumatology: A contributory

mechanism for alcohol and substance use disorders. Psychoneur-

oendocrinology, 27, 155–170.

Belsey, B. (2006). Cyber bullying: An emerging threat to the always

on generation. Retrieved from http://www.cyberbullying.ca/pdf/

Cyberbullying_Article_by_Bill_Belsey.pdf

Bond, L., Carlin, J. B., Thomas, L., Rubin, K., & Patton, G. (2001).

Does bullying cause emotional problem? A prospective study of

young teenagers. British Medical Journal, 323, 480–484.

Brown, T. A. (2006). Confirmatory factor analysis for applied

research. New York, NY: Guilford Press.

Buss, A. H., & Perry, M. (1992). The aggression questionnaire. Jour-

nal of Personality and Social Psychology, 63, 452–459.

Cassidy, W., Jackson, M., & Brown, K. N. (2009). Sticks and stones

can break my bones, but how can pixels hurt me? Student’s experi-

ences with cyber-bullying. School Psychology International, 30,

383–402.

Chapell, M. S., Hasselman, S. L., Kitchin, T., Lomon, S. N., MacIver,

K. W., & Sarullo, P. L. (2006). Bullying in elementary school, high

school, and university. Adolescence, 41, 633–648.

DeVellis, R. E. (2003). Scale development: Theory and applications

(2nd ed.). Newbury Park, CA: Sage.

Erdur-Baker, Ö., & Kavsut, F. (2007). Cyber bullying: A new face of

peer bullying. Eurasian Journal of Educational Research, 27,

31–42.

Harvill, L. M. (1991). Standard error of measurement. Educational

Measurement: Issues and Practice, 10, 33–41.

Hay, C., & Meldrum, R. (2010). Bullying victimization and adolescent

self-harm: Testing hypotheses from general strain theory. Journal

of Youth Adolescence, 39, 446–459.

Hay, C., Meldrum, R., & Mann, K. (2010). Traditional bullying, cyber

bullying, and deviance: A general strain theory approach. Journal

of Contemporary Criminal Justice, 26, 130–147. doi:10.1177/

1043986209359557

Hazler, R. J., Miller, D. L., Carney, J. V., & Green, S. (2001). Adult

recognition of school bullying situations. Educational Research,

43, 133–146.

Hemphill, S. A., Kotevski, A., Tollit, M., Smith, R., Herrenkohl, T. I.,

Toumbourou, J. W., & Catalano, R. F. (2012). Longitudinal pre-

dictors of cyber and traditional bullying perpetration in Austra-

lian secondary school students. Journal of Adolescent Health,

51, 59–65.

Hill, M. (1997). SPSS missing values analysis 7.5. Chicago, IL: SPSS

Inc.

Hinduja, S., & Patchin, J. W. (2007). Offline consequences of online

victimization. Journal of School Violence, 6, 89–112.

Appendix A (continued)

Visual/sexual victimization

11. Someone has posted embarrassing pictures or videos of me online without my permission to damage my reputation. 1 2 3 4 5 12. Someone has sent private pictures or videos of mine on websites without my permission to upset me. 1 2 3 4 5 13. People have posted humiliating pictures or videos of mine on websites to embarrass me. 1 2 3 4 5 14. I have never received sexually explicit things from someone via e-mail or text message which embarrassed me. 1 2 3 4 5 15. I have received unwanted sexual suggestions from someone in chat rooms which embarrassed me. 1 2 3 4 5 16. People have made sexual jokes about me online to damage my reputation. 1 2 3 4 5 17. People have attempted to humiliate me by posting sexual comments or photos on Facebook or Twitter. 1 2 3 4 5 18. People have spread sexual rumors about me online to damage my reputation. 1 2 3 4 5 19. I have been sent sexually explicit things from someone via e-mail or text message repeatedly which made me uncomfortable. 1 2 3 4 5 20. Someone has teased me about my appearance online repeatedly to upset me. 1 2 3 4 5

Social exclusion victimization

21. I have been blocked in a chat room by other people who want to make me angry. 1 2 3 4 5 22. Someone has blocked me on an instant messenger to upset me. 1 2 3 4 5 23. I have been excluded from online community groups which made me feel left out. 1 2 3 4 5 24. I have never been excluded from online group activities which made me feel left out. 1 2 3 4 5 25. People have cooperatively excluded me from online community groups to make me feel left out. 1 2 3 4 5 26. Someone has led members of the online community in excluding me to make me feel left out. 1 2 3 4 5 27. I have been excluded from online group activity or social community online repeatedly which made me feel left out. 1 2 3 4 5

466 Research on Social Work Practice 27(4)

Hinduja, S., & Patchin, J. W. (2008). Cyberbullying: An exploratory

analysis of factors related to offending and victimization. Deviant

Behavior, 29, 129–156.

Hu, L. T., & Bentler, P. M. (1999). Cutoff criteria for fit indexes in

covariance structure analysis: Conventional criteria versus new

alternatives. Structural Equation Modeling, 6, 1–55.

Hunt, C., Peters, L., & Rapee, R. M. (2012). Development of a mea-

sure of the experience of being bullied in youth. Psychological

Assessment, 24, 156–165.

Juvonen, J., Nishina, A., & Graham, S. (2000). Peer harassment,

psychological adjustment, and school functioning in early ado-

lescence. Journal of Educational Psychology, 92, 349–359.

doi:10.1037/0022-0663.92.2.349

Kincaid, J. P., Fishburne, R. P., Rogers, R. L., & Chissom, B. S.

(1975). Derivation of new readability formulas (Automated Read-

ability Index, Fog Count and Flesch Reading Ease Formula) for

Navy enlisted personnel (Research Branch Report 8–75). Milli-

ngton, TN: Naval Technical Training, U. S. Naval Air Station.

Kline, R. B. (2011). Principles and practice of structural equation

modeling (3rd ed.). New York, NY: The Guilford Press.

Klomek, A. B., Marrocco, F., Kleinman, M., Schonfeld, I. S., &

Gould, M. S. (2007). Bullying, depression, and suicidality in ado-

lescents. Journal of the American Academy of Child & Adolescent

Psychiatry, 46, 40–49. doi:10.1097/01.chi.0000242237.84925.18

Kowalski, R., Limber, S., & Agatston, P. (2008). Cyber bullying: Bul-

lying in the digital age. Malden, MA: Blackwell Publishers.

McDonald, R. P., & Ho, M.-H. R. (2002). Principles and practice in

reporting statistical equation analyses. Psychological Methods, 7,

64–82.

Menesini, E., Nocentini, A., & Calussi, P. (2011). The measurement of

cyber bullying: Dimensional structure and relative item severity

and discrimination. Cyberpsychology, Behavior, and Social Net-

working, 14, 267–274.

Merch, G. (2009). Parental mediation, online activities and cyber-bul-

lying. Cyberpsychology, Behavior, and Social Networking, 12,

387–393.

Mishna, F., Cook, C., Gadalla, T., Daciuk, J., & Solomon, S. (2010).

Cyber bullying behaviors among middle and high school students.

The American Journal of Orthopsychiatry, 80, 362–374.

Mishna, F., Cook, C., Saini, M., Wu, M., & MacFadden, R. (2011).

Interventions to prevent and reduce cyber abuse of youth: A sys-

tematic review. Research on Social Work Practice, 2, 5–14.

Mishna, F., Saini, M., & Solomon, S. (2009). Ongoing and online:

Children and youth’s perceptions of cyber bullying. Children and

Youth Services Review, 31, 1222–1228.

Mynard, H., & Joseph, S. (2000). Development of the multidimen-

sional peer-victimization scale. Aggressive Behavior, 26, 169–178.

Nansel, T. R., Overpeck, M., Pilla, R. S., Ruan, W. J., Simons-Morton,

B., & Scheidt, P. (2001). Bullying behaviors among US youth: Pre-

valence and association with psychosocial adjustment. Journal of

the American Medical Association, 285, 2094–2100.

Nunnally, J. C., & Bernstein, I. H. (1994). Psychometric theory (3rd

ed.). New York, NY: McGraw-Hill.

O’Connell, P., Pepler, D. J., & Craig, W. (1999). Peer involvement in

bullying: Insights and challenges for intervention. Journal of Ado-

lescence, 22, 437–452.

Olweus, D. (1993). Bullying at school: What we know and what we

can do. Oxford, England: Blackwell Publishers.

Olweus, D. (2012). Cyber bullying: An overrated phenomenon? Eur-

opean Journal of Developmental Psychology, 9, 520–538.

Patchin, J. W., & Hinduja, S. (2006). Bullies move beyond the school-

yard: A preliminary look at cyber bullying. Youth Violence and

Juvenile Justice, 4, 148–169.

Patchin, J. W., & Hinduja, S. (2010). Cyberbullying and self-esteem.

Journal of School Health, 80, 614–621.

Patchin, J. W., & Hinduja, S. (2011). Traditional and nontraditional

bullying among youth: A test of general strain theory. Youth and

Society, 43, 727–751.

Raskauskas, J., & Stoltz, A. D. (2007). Involvement in traditional and

electronic bullying among adolescents. Developmental Psychol-

ogy, 43, 564–575.

Schenk, A. M., & Fremouw, W. J. (2012). Prevalence, psychologi-

cal impact, and coping of cyberbully victims among college

students. Journal of School Violence, 11, 21–37. doi:10.1080/

15388220.2011.630310

Slonje, R., & Smith, P. K. (2008). Cyber bullying: Another main

type of bullying? Scandinavian Journal of Psychology, 49,

147–154.

Smith, P. K., Mahdavi, J., Carvalho, M., Fisher, S., Russell, S., & Tip-

pett, N. (2008). Cyber bullying: Its nature and impact in secondary

school pupils. Journal of Child Psychology and Psychiatry, 49,

376–385.

Springer, D., Abell, N., & Nugent, W. R. (2002). Creating and validat-

ing rapid assessment instruments for practice and research: Part 2.

Research on Social Work Practice, 12, 768–795.

Stewart, A., Livingston, M., & Dennison, S. (2008). Transitions and

turning points: Examining the links between child maltreatment

and juvenile offending. Child Abuse & Neglect, 32, 51–66.

Walker, C. M., Sockman, B. R., & Koehn, S. (2011). An exploratory

study of cyberbullying with undergraduate university students.

TechTrends, 55, 31–38.

Wang, J., Ronald, J. I., & Tonja, R. N. (2009). School bullying among

adolescents in the United States: Physical, verbal, relational, and

cyber. Journal of Adolescent Health, 45, 368–375.

Wright, V. H., Burnham, J. J., Inman, C. T., & Ogorchock, H. N.

(2009). Cyberbullying: Using virtual scenarios to educate and raise

awareness. Journal of Computing in Teacher Education, 26,

35–42.

Ybarra, M. L., Diener-West, M., & Leaf, P. J. (2007). Examining

the overlap in internet harassment and school bullying: Implica-

tions for school intervention. Journal of Adolescent Health, 41,

S42–S50.

Lee et al. 467

<< /ASCII85EncodePages false /AllowTransparency false /AutoPositionEPSFiles true /AutoRotatePages /None /Binding /Left /CalGrayProfile (Gray Gamma 2.2) /CalRGBProfile (sRGB IEC61966-2.1) /CalCMYKProfile (U.S. Web Coated \050SWOP\051 v2) /sRGBProfile (sRGB IEC61966-2.1) /CannotEmbedFontPolicy /Warning /CompatibilityLevel 1.3 /CompressObjects /Off /CompressPages true /ConvertImagesToIndexed true /PassThroughJPEGImages false /CreateJobTicket false /DefaultRenderingIntent /Default /DetectBlends true /DetectCurves 0.1000 /ColorConversionStrategy /LeaveColorUnchanged /DoThumbnails false /EmbedAllFonts true /EmbedOpenType false /ParseICCProfilesInComments true /EmbedJobOptions true /DSCReportingLevel 0 /EmitDSCWarnings false /EndPage -1 /ImageMemory 1048576 /LockDistillerParams true /MaxSubsetPct 100 /Optimize true /OPM 1 /ParseDSCComments true /ParseDSCCommentsForDocInfo true /PreserveCopyPage true /PreserveDICMYKValues true /PreserveEPSInfo true /PreserveFlatness false /PreserveHalftoneInfo false /PreserveOPIComments false /PreserveOverprintSettings true /StartPage 1 /SubsetFonts true /TransferFunctionInfo /Apply /UCRandBGInfo /Remove /UsePrologue false /ColorSettingsFile () /AlwaysEmbed [ true ] /NeverEmbed [ true ] /AntiAliasColorImages false /CropColorImages false /ColorImageMinResolution 266 /ColorImageMinResolutionPolicy /OK /DownsampleColorImages true /ColorImageDownsampleType /Average /ColorImageResolution 175 /ColorImageDepth -1 /ColorImageMinDownsampleDepth 1 /ColorImageDownsampleThreshold 1.50286 /EncodeColorImages true /ColorImageFilter /DCTEncode /AutoFilterColorImages true /ColorImageAutoFilterStrategy /JPEG /ColorACSImageDict << /QFactor 0.40 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /ColorImageDict << /QFactor 0.76 /HSamples [2 1 1 2] /VSamples [2 1 1 2] >> /JPEG2000ColorACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000ColorImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasGrayImages false /CropGrayImages false /GrayImageMinResolution 266 /GrayImageMinResolutionPolicy /OK /DownsampleGrayImages true /GrayImageDownsampleType /Average /GrayImageResolution 175 /GrayImageDepth -1 /GrayImageMinDownsampleDepth 2 /GrayImageDownsampleThreshold 1.50286 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict << /QFactor 0.40 /HSamples [1 1 1 1] /VSamples [1 1 1 1] >> /GrayImageDict << /QFactor 0.76 /HSamples [2 1 1 2] /VSamples [2 1 1 2] >> /JPEG2000GrayACSImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /JPEG2000GrayImageDict << /TileWidth 256 /TileHeight 256 /Quality 30 >> /AntiAliasMonoImages false /CropMonoImages false /MonoImageMinResolution 900 /MonoImageMinResolutionPolicy /OK /DownsampleMonoImages true /MonoImageDownsampleType /Average /MonoImageResolution 175 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50286 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict << /K -1 >> /AllowPSXObjects false /CheckCompliance [ /None ] /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox false /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile (U.S. Web Coated \050SWOP\051 v2) /PDFXOutputConditionIdentifier (CGATS TR 001) /PDFXOutputCondition () /PDFXRegistryName (http://www.color.org) /PDFXTrapped /Unknown /CreateJDFFile false /Description << /ENU <FEFF005500730065002000740068006500730065002000530061006700650020007300740061006e0064006100720064002000730065007400740069006e0067007300200066006f00720020006300720065006100740069006e006700200077006500620020005000440046002000660069006c00650073002e002000540068006500730065002000730065007400740069006e0067007300200063006f006e006600690067007500720065006400200066006f00720020004100630072006f006200610074002000760037002e0030002e00200043007200650061007400650064002000620079002000540072006f00790020004f00740073002000610074002000530061006700650020005500530020006f006e002000310031002f00310030002f0032003000300036002e000d000d003200300030005000500049002f003600300030005000500049002f004a0050004500470020004d0065006400690075006d002f00430043004900540054002000470072006f0075007000200034> >> /Namespace [ (Adobe) (Common) (1.0) ] /OtherNamespaces [ << /AsReaderSpreads false /CropImagesToFrames true /ErrorControl /WarnAndContinue /FlattenerIgnoreSpreadOverrides false /IncludeGuidesGrids false /IncludeNonPrinting false /IncludeSlug false /Namespace [ (Adobe) (InDesign) (4.0) ] /OmitPlacedBitmaps false /OmitPlacedEPS false /OmitPlacedPDF false /SimulateOverprint /Legacy >> << /AllowImageBreaks true /AllowTableBreaks true /ExpandPage false /HonorBaseURL true /HonorRolloverEffect false /IgnoreHTMLPageBreaks false /IncludeHeaderFooter false /MarginOffset [ 0 0 0 0 ] /MetadataAuthor () /MetadataKeywords () /MetadataSubject () /MetadataTitle () /MetricPageSize [ 0 0 ] /MetricUnit /inch /MobileCompatible 0 /Namespace [ (Adobe) (GoLive) (8.0) ] /OpenZoomToHTMLFontSize false /PageOrientation /Portrait /RemoveBackground false /ShrinkContent true /TreatColorsAs /MainMonitorColors /UseEmbeddedProfiles false /UseHTMLTitleAsMetadata true >> << /AddBleedMarks false /AddColorBars false /AddCropMarks false /AddPageInfo false /AddRegMarks false /BleedOffset [ 9 9 9 9 ] /ConvertColors /ConvertToRGB /DestinationProfileName (sRGB IEC61966-2.1) /DestinationProfileSelector /UseName /Downsample16BitImages true /FlattenerPreset << /ClipComplexRegions true /ConvertStrokesToOutlines false /ConvertTextToOutlines false /GradientResolution 300 /LineArtTextResolution 1200 /PresetName ([High Resolution]) /PresetSelector /HighResolution /RasterVectorBalance 1 >> /FormElements true /GenerateStructure false /IncludeBookmarks false /IncludeHyperlinks false /IncludeInteractive false /IncludeLayers false /IncludeProfiles true /MarksOffset 9 /MarksWeight 0.125000 /MultimediaHandling /UseObjectSettings /Namespace [ (Adobe) (CreativeSuite) (2.0) ] /PDFXOutputIntentProfileSelector /DocumentCMYK /PageMarksFile /RomanDefault /PreserveEditing true /UntaggedCMYKHandling /UseDocumentProfile /UntaggedRGBHandling /UseDocumentProfile /UseDocumentBleed false >> ] /SyntheticBoldness 1.000000 >> setdistillerparams << /HWResolution [288 288] /PageSize [612.000 792.000] >> setpagedevice