w10dq

Assessing the perceived prevalence of research fraud among faculty at research-intensive universities in the USA Michael D. Reisiga, Kristy Holtfretera, and Marcus E. Berzofskyb

aSchool of Criminology and Criminal Justice, Arizona State University, Phoenix, AZ, USA; bDivision of Statistics and Data Science, RTI International, Research Triangle Park, NC, USA

ABSTRACT Survey-based studies on research fraud often feature narrow operationalizations of misbehavior and use limited samples. Such factors potentially hinder the development of strategies aimed at reducing the frequency of wrongdoing among research- ers. This study asked full-time faculty members in the natural, social, and applied sciences how frequently six types of research fraud (i.e., data fabrication, plagiarism, data falsification, author- ship fraud, publication fraud, and grant fraud) occur in their field of study. These data come from mail and online surveys that were administered to a stratified random sample of tenured and tenure-track faculty members (N = 613) at the top 100 research universities in the United States. Factor-analytic modeling demonstrated that the survey items load on the hypothesized latent constructs and also confirmed the presence of a second- order factor. A specific type of authorship fraud – gift authorship – was perceived to be the most prevalent overall. The least com- mon fraud was a form of data fabrication (i.e., creating data from a study that was never actually conducted). The results were largely consistent with previous studies indicating that serious forms of fraud like data fabrication are relatively rare. Future survey-based studies should pay careful attention to the multi- dimensional nature of research fraud.

KEYWORDS Research misconduct; deviant behavior; research integrity; fraud

1. Introduction

Research fraud is defined as the wrongful use of deception during the course of research to advance one’s self-interest. Importantly, fraudulent behavior must be intentional, or as some have described, it involves “an element of calculated deception” (Cattano and Turk 2007, 470). Included under this broad heading are many acts, some of which traditionally have been deemed research misconduct (i.e., data fabrication, plagiarism, and data falsification; Office of Research Integrity 2019). But the wrongful and self-interested use of deception by researchers includes other behaviors too. For example, publica- tion fraud, such as “gift” authorship (e.g., awarding authorship credit to an undeserving senior colleague who can reciprocate the good will at a later

CONTACT Michael D. Reisig [email protected] School of Criminology and Criminal Justice, Arizona State University, 411 N. Central Avenue, Phoenix, AZ 85004, USA

ACCOUNTABILITY IN RESEARCH, 2020, VOL. 27, NO. 7, 457–475 https://doi.org/10.1080/08989621.2020.1772060

© 2020 Informa UK Limited, trading as Taylor & Francis Group

date; Jones and McCullough 2015) and “ghost” authorship (e.g., not attribut- ing authorship credit to a coauthor who has a conflict of interest; Bosch and Ross 2012; Warren 2018), fall within the purview of research fraud (also see Claxton 2005; Coats 2009; Teixeria da Silva and Dobránski 2016). The detrimental after-effects of research fraud, such as unfairly rewarding deviant researchers, misguiding professional practices, and adversely affecting public confidence in science, justify the continued systematic investigation with an aim toward reducing its frequency.

Several studies have attempted to estimate the prevalence of researchers’ misbehavior (Fanelli 2009; Fiedler and Schwarz 2016; John, Loewenstein, and Prelec 2012; Pupovac and Fanelli 2015). While estimates of how much research fraud takes place vary from study-to-study, a consistent patterns has been observed: serious infractions, such as data fabrication, are infre- quent relative to other violations.1 This finding generally holds for self-report studies (Martinson, Anderson, and De Vries 2005; also see Tijdink, Verbeke, and Smulders 2014), studies using perceived prevalence measures (Broome et al. 2005; Hopp and Hoover 2019; Pryor, Habermann, and Broome 2007), and focus group studies (Buljan, Barać, and Marušić 2018; De Vries, Anderson, and Martinson 2006). There are a variety of methodological issues with using survey data to study research fraud that need to be addressed.2

Over the past two decades, survey-based studies on research fraud have drawn samples from different populations, including grant recipients from individual funding agencies (Martinson, Anderson, and De Vries 2005), membership directories from certain professional organizations (Fiedler and Schwarz 2016; Ranstam et al. 2000), authors of articles appearing in particular journals (Baerlocher et al. 2010), or other sources that are limited to specific fields of study (Hopp and Hoover 2019; John, Loewenstein, and Prelec 2012; Tijdink, Verbeke, and Smulders 2014). While such studies have provided valuable information about research fraud in different scientific fields, this body of knowledge has limits in terms of population validity. For example, it is unknown whether the findings for biostatisticians generalize to sociologists, engineers, and physicists. Scientific fields can be quite different in terms of opportunities to commit fraud, the risk of apprehension, conduct norms, and rewards associated with successfully deceiving others, to name just a few (see Haven et al. 2019; Kalichman, Sweet, and Plemmons 2015). Research fraud studies that use more inclusive samples (e.g., participants representing multiple scientific fields) will help increase the ability of such studies to make inferences about broader groups of researchers when com- pared to studies that use more limited samples.

Although it is not uncommon for studies to investigate fraudulent research practices simultaneously (see, e.g., Broome et al. 2005), the development of multi-item scales that capture the complex nature of different fraud types (e.g., plagiarism and grant fraud) have yet to be developed. Accordingly,

458 M. D. REISIG ET AL.

researchers continue to use single item measures and combined scales con- sisting of survey items reflecting disparate misbehaviors. Both practices have drawbacks. For starters both of these approaches likely fail to capture the breadth of any type of research fraud. Accordingly, the findings that these studies yield will be incomplete and potentially misleading. There is a clear need to construct a multidimensional research fraud scale. The development of such a scale will not only address the concerns associated with existing measurement practices, but will also provide researchers with a measure to compare the perceived prevalence of research fraud and its many constitutive elements across studies, between scientific fields, and over time.

This study investigated the perceived prevalence of various types of research fraud among faculty in the social (e.g., economics, psychology, and political science), applied (e.g., health care, public health, and engineer- ing), and natural (e.g., biology, physics, and astronomy) sciences at research- intensive universities in the United States. Two specific research objectives were pursued. First, a multidimensional model of research fraud was con- structed. Second, the perceived prevalence of different types of research fraud was assessed. Here, the analyses sought to determine whether serious forms of fraud, such as data fabrication, were perceived to be less common. Analyses also assessed whether the perceived prevalence of research fraud varied across scientific fields.

2. Method

2.1. Procedures

The process of constructing the study sample involved two steps. First, the top 100 research universities in the United States were identified using Phillips et al.’s (2013) systematic ranking. This specific evaluative assessment was selected because it used nine different measures of performance (e.g., federal research expenditures, total research spending, National Academy of Science members, and number of post-docs). Second, a stratified-random sampling process was used to select tenured and tenure-track faculty mem- bers in the social, applied, and natural sciences.3 A total of 6,000 individuals were selected (2,000 from each scientific field). A balanced sample allocation was used to allow for similar precision in estimates from each scientific field.

This study used a mixed-mode survey strategy that entailed administering both mail and online surveys. This approach was adopted because faculty members are not always easily reached via their university mail address. So including online surveys was partially done to reduce potential coverage error. The mixed-mode strategy also helped control the financial cost of data collection (Dillman, Smyth, and Christian 2014). The mail survey involved sending paper surveys to 2,000 selected individuals’ university

ACCOUNTABILITY IN RESEARCH 459

addresses via the U.S. Postal Service. This was done over the course of two four-month periods – the winter of 2016–17 and the spring of 2017. The mail survey was administered by the authors and a team of graduate students. The online survey was administered over four periods, beginning in October 2016 and ending in March 2017. Invitations to 4,000 potential participants were sent via e-mail. An experienced private research firm carried out the online survey. The governing institutional review board approved research protocols prior to the commencement of data collection.

Several steps were taken to encourage participation. These steps were tailored to each mode of data collection. Individuals who were selected to participate in the online survey received personalized e-mails from a member of the research team that described the study, provided confidentiality assur- ances, and politely requested their participation. Two personalized reminder e-mails were sent in each of the following weeks to nonrespondents. A total of 311 online surveys were submitted. Individuals selected to participate in the mail survey received an attractive, signed, multi-color paper survey in a personally addressed university envelope. The front cover of the survey explained the objective of the study and pledged confidentiality. A postage pre-paid return envelope accompanied each mail survey. A total of three waves were sent for each of the two mail surveys. A note card politely requesting participation in the study was sent to nonrespondents. A total of 302 mail surveys were returned. For the analysis, the data from the online and mail surveys were pooled. The overall response rate was approximately 10.6%.4 The combined sample consisted of 613 tenured and tenure-track faculty members from the social, natural, and applied sciences representing all 100 of the most research-intensive universities in the United States.5

2.2. Sample description

Participant ages ranged from 27 to 88 years (median = 56.00, mean = 55.43, SD = 13.13). A majority of the sample (69.3%) was male (30.7% female). The racial/ethnic breakdown of the sample was as follows: 82.5% white, 6.5% Asian, 3.9% Hispanic, 2.1% African-American, 0.2% Native American/ American Indian, and 4.7% self-reported “other.” In terms of nationality, 90% of the sample were U.S. citizens. With regard to faculty rank, 24.3% were Assistant Professors, 24.8% were Associate Professors, 38.3% were Professors, and 12.6% were Distinguished/Endowed Professors. The distribution of participants across scientific fields was 36.5% natural sciences, 34.6% social sciences, and 28.9% applied sciences. The number of years since participants completed their doctoral degree ranged from 1 to 54 years (median = 22.00, mean = 22.60, SD = 12.87). Finally, the median number of refereed publica- tions was 44 (mean = 68.98, SD = 75.60, range from 0 to 650). The sample was diverse in terms of rank, professional experience, and publication record.

460 M. D. REISIG ET AL.

Because population estimates for tenured and tenure-track faculty in the natural, social, and applied sciences at the top 100 research universities were not available, the true representativeness of the study sample could not be assessed. Therefore, to gauge how well the sample reflected the population, two sets of data from the Integrated Postsecondary Education Data System (IPEDS) were used based on the fall 2017 Human Resources Survey. Comparisons focused on three demographic characteristics (i.e., sex, race, and academic rank) that were common across the three data files. The first comparison group consisted of faculty at the rank of Assistant Professor, Associate Professor, and Professor at the 100 research-intensive universities where study participants were employed. Importantly, this group included faculty from all fields of study (e.g., literature, law, and music; National Center for Education Statistics 2019) as IPEDS does not allow data to be restricted by field. The second comparison group consisted of all full-time faculty at the rank of Assistant Professor, Associate Professor, and Professor who were employed at degree-granting postse- condary institutions in the United States (National Center for Education Statistics 2018). The 100 research-intensive universities represented in the study sample made-up only a small portion of the institutions included in this database and, therefore, demonstrate how the sample compares to the larger post-secondary universe. Despite the differences in faculty composi- tion across the three groups, Table 1 shows that the distributions for sex, race, and academic rank were largely consistent. Any differences between the sample and population distributions were in line with Li and Koedel (2017) who found that minorities (except Asians) and females were under- represented among faculty in the sciences, while individuals at the rank of Professor were overrepresented. These findings indicated that the

Table 1. Sample comparisons. Study Sample IPEDS: Same Universitiesa IPEDS: Postsecondary Institutionsb

Sex Male 69.3% 62.9% 57.3% Female 30.7 37.1 42.7

Race White 82.5 74.6 77.4 Nonwhite 17.5 25.4 22.6

Academic Rank Professor 50.9c 40.5 35.4 Associate Professor 24.8 27.5 30.5 Assistant Professor 24.3 32.0 34.1

IPEDS = Integrated Postsecondary Education Data System aNational Center for Education Statistics (2019) Integrated postsecondary education data system. Use the

data. Available at: https://nces.ed.gov/ipeds/use-the-data bNational Center for Education Statistics (2018) Digest of education statistics, Table 315.20. Available at:

https://nces.ed.gov/programs/digest/d18/tables/dt18_315.20.asp cIncludes participants who self-reported the rank of Professor and Distinguished/Endowed Professor.

ACCOUNTABILITY IN RESEARCH 461

procedures used in this study generated a sample representative of the target population.

To further assess the representativeness of the sample, a comparison of weighted and unweighted estimates and standard deviations was conducted. The sample was stratified by scientific fields, which by design entailed an unequal probability of selection across the social, natural, and applied sciences. Accordingly, survey weights were constructed to determine if more accurate inferences to the population of faculty at the 100 universities included in the sample would be ascertained through weighted estimates.6

Once the weights were constructed, the weighted mean scores and corre- sponding standard deviations were calculated and compared to the unweighted estimates. The estimates for the weighted data were similar to the unweighted data. Therefore, consistent with Bollen et al.’s (2016) recom- mendations for whether survey weights in model-based analyses are needed, it was determined that the unweighted analyses produced accurate estimates without inflating the variances in the manner that an analysis based on survey weights does. Moreover, these findings helped confirm the conclusion that the sample was representative of the population reducing the likelihood of bias in the estimates.7

2.3. Measures

The task of constructing the multidimensional research fraud scale began with a search for published studies on research misconduct, scientific fraud, and the like. These studies provided both an overview of the many different types of research fraud and provided an initial pool of survey items to work with (see Anderson, Louis, and Earle 1994; Broome et al. 2005; De Vries, Anderson, and Martinson 2006; Martinson, Anderson, and De Vries 2005; Montgomerie and Birkhead 2005; Tijdink, Verbeke, and Smulders 2014). After reviewing the available survey items, the authors discarded some items because they were deemed to be too broad.8 Next, the remaining survey items were grouped by fraud type. At this stage, the authors revised many of the existing items in an attempt to improve their clarity and precision. Finally, several items were written by the authors in an attempt to capture the complexity of each hypothesized dimension of research fraud. In all, 36 items reflecting six types of research fraud were available for analysis. Ten of these items were not used in this study because they were deemed to be redundant with other items, too vague or unclear, too specific to a particular scientific field, or failed to perform as expected in preliminary tests. In the end, 26 survey items were ultimately used to reflect the hypothesized dimen- sions of research fraud.

As noted at the outset, the concept of research fraud includes misbehaviors that fall under the heading of what the Office of Research Integrity (ORI)

462 M. D. REISIG ET AL.

refers to as research misconduct (i.e., data fabrication, data falsification, and plagiarism). The term data fabrication implies “making up data or results and recording or reporting them” (ORI 2019). This dimension of fraud was operationalized using five survey items reflecting instances of using fictitious data. One such item read, “Fabricating data so that a desired outcome is found.” Data falsification involves changing or deleting data for fraudulent purposes (“Altering values so that a desired outcome is found”). Finally, the definition of plagiarism not only includes traditional forms of stealing others’ words, ideas, and research findings (“Using another author’s exact language without giving appropriate credit”), but also includes instances of textual recycling (or self-plagiarism; e.g., “Publishing a previously published study under a different title at another journal”; see Bruton 2014; Loui 2002; Moskovitz 2019; Rohrich and Sullivan 2009). The scale used in this study captured both forms of plagiarism.

The literature search also produced survey items reflecting three other forms of research fraud. Authorship fraud entails misbehavior when allocat- ing authorship credit. Examples of this form of fraud include gift authorship (“Accepting authorship credit on a paper without making a substantive contribution”) and ghost authorship (“Not giving authorship credit to some- one who made a substantive contribution”). Publication fraud included items that reflect deliberate deception during the publishing process. Specific forms of fraud under this heading include “shotgunning” (Rogers 1999; “Submitting a paper for publication that is under review at another journal”). Finally, grant fraud is defined as the attempt by grant recipients to “deceive the government about their spending of award money” (Grants.gov 2020). This type of deception can take the form of misusing funds (e.g., “Using grant funds to cover personal expenses”) or lying about the how the awarded financial resources will be used (e.g., “Applying for grants to do work that is already done”).9

Prior to responding to the research fraud items, participants were asked to indicate “how frequently” they believed each type of misbehavior occurred in their field of study by using a closed-ended response set ranging from “never” (coded 1) to “often” (coded 4). The six dimensions of research fraud were operationalized as additive scales. Each scale was adjusted by dividing scores by the number of items used in the measure, which allowed the means to be interpreted using the four-point response option. The alpha estimates for each scale either exceeded or approached the traditional .70 threshold, indicating sufficient levels of internal consistency. The survey items and corresponding scales are presented in Table 2.

ACCOUNTABILITY IN RESEARCH 463

3. Results

3.1. Multidimensional model of research fraud

One objective of this study was to develop a multidimensional model of research fraud. Accordingly, the 26 survey items that were hypothesized to reflect the six dimensions of research fraud were entered into a second-order confirmatory factor model (see Table 3). Mean- and variance-adjusted weighted least squares (or WLSMV) estimation was used because of the ordinal nature of the survey items (Beauducel and Herzberg 2006). The factor-analytic model was estimated using Mplus 6.11 (Muthén and Muthén, Los Angeles, CA). The first order was comprised of measurement models, one for each dimension of research fraud. As expected, the standar- dized factor loadings exceeded conventional cut-points (> .50), and the corresponding test statistics were statistically significant. The results for the second order also conformed to expectations. Put simply, each of the

Table 2. Perceived prevalence of research fraud. Scale/Survey items Mean SD N

Data fabrication (α = .855) 1.92 .49 592 1 Fabricating data so that a desired outcome is found 2.01 .59 604 2 Fabricating parts of a grant proposal to be more competitive 2.12 .69 599 3 Adding fictitious data to a real data set to provide additional statistical validity 1.87 .59 593 4 Fabricating results from a pilot study to appear attractive to a funding agency 1.96 .64 593 5 Creating data from a study that was never actually conducted 1.67 .56 595 Data falsification (α = .812) 2.12 .58 593 6 Deleting data so that a desired outcome is found 2.24 .72 599 7 Altering values so that a desired outcome is found 2.03 .65 596 8 Reporting research results that are known to be inaccurate 2.09 .67 597 Plagiarism (α = .690) 2.09 .50 604 9 Using another author’s exact language without giving appropriate credit 2.38 .70 609 10 Presenting another study’s tables or figures without giving appropriate credit 2.01 .63 606 11 Publishing a previously published study under a different title at another journal 2.01 .73 606 12 Publishing a previously published study under a different title in another language 1.97 .70 604 Authorship fraud (α = .686) 2.58 .55 599 13 Accepting authorship credit on a paper without making a substantive contribution 3.07 .82 604 14 Not giving authorship credit to someone who made a substantive contribution 2.47 .76 602 15 Arranging authorship order in a way that doesn’t reflect each author’s contribution 2.86 .80 601 16 Not accepting authorship credit on a paper after making a substantive contribution 1.92 .70 599 Publication fraud (α = .740) 2.04 .52 592 17 Failing to disclose conflicts of interest when publishing a journal article 2.29 .71 596 18 Submitting a paper for publication that is under review at another journal 2.01 .68 598 19 Not publishing results under pressure from a funding source 2.03 .73 594 20 Deliberately not mentioning a funding source when publishing a study 1.86 .65 597 Grant fraud (α = .814) 2.34 .56 585 21 Using grant funds to cover personal expenses 2.09 .71 590 22 Charging a grant for work that was not performed 2.17 .76 588 23 Submitting a false financial statement to a funding agency 1.91 .63 586 24 Using grant funds to attend a conference and then not, or barely, showing up 2.45 .78 591 25 Applying for grants to do work that is already done 2.63 .91 591 26 Using funds from one source to pay for personnel working on an unrelated project 2.81 .82 595

Closed-ended response options for survey items included “never” (= 1), “seldom” (= 2), “sometimes” (= 3), and “often” (= 4).

464 M. D. REISIG ET AL.

first-order factors (e.g., data fabrication and publication fraud) loaded on the second-order construct – which was termed research fraud. Finally, tests indicated that the overall model fit the data sufficiently well: root mean square error of approximation (RMSEA) = .077 (90% CI = .072 to .081); comparative fit index (CFI) = .937; and Tucker-Lewis index (TLI) = .930 (Brown 2006). After establishing the factor validity of the multidimensional research fraud scale, the focus shifted to assessing the perceived prevalence of research fraud.

Table 3. Second-order confirmatory factor model. First Order

Item Data

Fabrication Data

Falsification Plagiarism Authorship

Fraud Publication

Fraud Grant Fraud

1 .871 2 .807 3 .878 4 .847 5 .747 6 .824 7 .898 8 .804 9 .672 10 .703 11 .626 12 .711 13 .660 14 .771 15 .663 16 .528 17 .732 18 .739 19 .654 20 .748 21 .724 22 .765 23 .861 24 .704 25 .623 26 .652

Second Order

Factor Research Fraud

Data Fabrication .954 Data Falsification .957 Plagiarism .786 Authorship Fraud .747 Publication Fraud .827 Grant Fraud .823

Coefficients are standardized estimates; all corresponding p-values are significant at the .001 level (two- tailed test). Model fit statistics: χ2 Test of Model Fit = 1345.578, p < .05; Comparative Fit Index = .937, Tucker Lewis Index = .930, Root Mean Square Error of Approximation = .077 (90% CI .072 to .081).

ACCOUNTABILITY IN RESEARCH 465

3.2. The perceived prevalence of research fraud

The mean scores for the additive scales indicated that participants perceived that authorship fraud was the most prevalent type of research fraud in their field of study (see Table 2). Within this specific category, gift authorship (“Accepting authorship credit on a paper without making a substantive contribution”) was perceived by participants to be most widespread. Importantly, gift authorship was perceived to be the most pervasive of all the specific forms of research fraud listed in Table 2. Another relatively prevalent form of authorship fraud was the unethical distribution of author- ship credit (“Arranging authorship order in a way that doesn’t reflect each author’s contribution”). The process of establishing authorship order can be contentious. Indeed, Smith et al. (2019) found that some researchers char- acterized this process as a “blood sport.” One of the least prevalent forms of fraud (both within the authorship fraud category and across categories) involved “[n]ot accepting authorship credit on a paper after making a substantive contribution.” In sum, authorship fraud was the most pre- valent type of research fraud in the eyes of participants. However, within this general category, heterogeneity in perceived frequency was observed.

The least prevalent form of research fraud as perceived by participants was data fabrication. The rarest form of data fabrication involved “creating data from a study that was never conducted.” This specific misbehavior was perceived to be the least prevalent of all forms of fraud listed in Table 2. Of the five forms of fabrication represented by the survey items, participants reported that “[f]abricating parts of a grant proposal to be more competitive” was the most common. Overall, the results from this portion of the analyses were pretty consistent with extant research employing different methodolo- gical strategies and samples from diverse populations in that the most serious form of misbehavior among researchers – data fabrication – was perceived to be the least prevalent (see, e.g., Hopp and Hoover 2019).

The results showed that the perceived prevalence of plagiarism, data falsification, publication fraud, and grant fraud fell in between data fabrica- tion and publication fraud. For plagiarism, the mean scores indicated that “[u]sing another author’s exact language without giving appropriate credit” was the most prevalent form of fraud in this category, whereas “publishing a previously published study under a different title in another language” was perceived by sample members to be the least frequent violation of this type. Slightly less within-group variation was apparent for the data falsification items. The most prevalent type of falsification was believed to be “[d]eleting data so that a desired outcome is found,” whereas “[a]ltering values so that a desired outcome is found” was perceived to be the least frequent type of falsification.

466 M. D. REISIG ET AL.

Turning to grant fraud, the most pervasive violation in the eyes of sample members involved “[u]sing funds from one source to pay for personnel working on an unrelated project.” The least common entailed “[s]ubmitting a false financial statement to a funding agency.”10 When it came to publica- tion fraud, participants perceived “[f]ailing to disclose conflicts of interest when publishing a journal article” to be the most prevalent type under this heading, whereas “[d]eliberately not mentioning a funding source when publishing a study” was perceived to be the least common.

One important take away from the descriptive findings presented in Table 2 was that within-group variation in mean scores was observed across the specific types of fraud under each of the six dimensional heading. Additionally, differences were also observed between scales (e.g., compare authorship fraud with plagiarism). The heterogeneity in scores both within- and between-scales highlights the utility of constructing and using composite scales to study research fraud.

3.3. Perceptions of research fraud by scientific field

The question of whether perceptions of research fraud prevalence vary across scientific fields was also addressed. Because the fraud scales were correlated with one another (see Appendix), differences across fields were assessed using one-way MANOVA. The results for the unweighted data indicated that perceptions of one type of research fraud varied across scientific fields, while others did not. More specifically, when it came to data fabrication, data falsification, plagiarism, authorship fraud, and grant fraud, differences across fields were not observed. The perceived prevalence of these forms of fraud were invariant across participants from the social, natural, and applied sciences. However, the prevalence of publication fraud was distinguishable by scientific field. The Bonferroni multiple comparison tests showed that the mean score for publication fraud was significantly lower in the natural sciences when compared to the applied sciences. Overall, the results in Table 4 demonstrated that the perceived prevalence of research fraud was largely consistent across scientific fields.11

4. Discussion

This study developed a multidimensional model of the perceived prevalence of research fraud. The factor-analytic results showed that six different forms of fraud loaded on the same higher-order latent construct, indicating that they represent a common factor – research fraud. After establishing the factor validity of the model, prevalence estimates were presented. Overall, the results were largely consistent with extant research showing that data fabrication is less frequent than other fraudulent behaviors. Only one

ACCOUNTABILITY IN RESEARCH 467

significant difference (i.e., publication fraud) across scientific fields was observed.

Although the scale developed in this study was used to capture perceptions of research fraud, the items could easily be retooled to measure self-reported fraudulent behavior. Of course, such a scale should feature a specific recall time frame. Various lengths of time have been used previously in self-report studies of research behavior (e.g., three years, Godescharle et al. 2018), and it is not uncommon for studies to query participants about involvement in research fraud throughout their career (see Baerlocher et al. 2010). To date, a standard recall time frame – one that balances the need to capture rare events such as serious research misconduct against the threat of memory decay – has yet to be established. Additionally, it will also be necessary to determine which type of response option yields the best information. Researchers may decide that binary responses (e.g., “yes” or “no”), closed- ended ordinal responses (e.g., “never,” “rarely,” “sometimes,” and “fre- quently”), or asking participants to report the actual number of times they engaged in a particular behavior are most optimal given their specific research objectives. Moving forward, it will be essential that appropriate recall time frames and response options are identified and used to help ensure accurate prevalence estimates of research fraud derived from self- report data (see Bhandari and Wagner 2006).12

Table 4. One-way MANOVA for perceived prevalence of research fraud by scientific field. Applied Sciences Natural Sciences Social Sciences

Variable Mean SD Mean SD Mean SD

Data fabrication 1.958 .498 1.920 .502 1.914 .467

F – ratio .416 (p =.660) Data falsification

2.138 .601 2.140 .578 2.109 .565 F – ratio .172 (p =.842)

Plagiarism 2.142 .512 2.081 .547 2.084 .421

F – ratio .820 (p =.441) Authorship fraud

2.640 .545 2.603 .546 2.516 .561 F – ratio 2.427 (p =.089)

Publication frauda

2.154 .583 1.979 .505 2.042 .477 F – ratio 5.279 (p =.005)

Grant fraud 2.393 .569 2.342 .556 2.316 .536

F – ratio .870 (p =.419) Roy’s largest root .026 (p =.025)

N 162 212 190 aResults from the Bonferroni multiple comparison test indicated that the difference between the natural

sciences and applied sciences was statistically significant (p =.004).

468 M. D. REISIG ET AL.

This study demonstrated that the multidimensional nature of research fraud and that behaviors representing each dimension vary considerably in terms of their perceived prevalence. Accordingly, future studies should not only consider evaluating the perceived prevalence of research fraud using multi-item scales, but also provide disaggregated estimates so that within- scale variation can be observed. Future research should also consider other potential dimensions of research fraud, such as falsifying information about one’s professional record in biographical statements, curriculum vitae, web pages, and elsewhere (see Broome et al. 2005). Studies that identify additional dimensions of research fraud should work to operationalize such concepts as multi-item scales with good measurement properties.

4.1. Limitations

Two limitations of the study need to be addressed. First, the response rates for the online and mail surveys were lower than anticipated, despite using many best practices for motivating participation (e.g., personalized commu- nication with all sampled individuals and follow-up contact encouraging participation). It is likely that the response rate was adversely affected by timing. More specifically, during the run up to the 2016 Presidential election, there was considerable media coverage of the cyberattack against the Democratic National Committee’s computer network. According to media accounts, this attack may have been initiated by spear-phishing e-mails (i.e., e-mail that appears to be from a trusted source that contains a link or attachment capable of deploying malware; Nakashima 2016). The extent to which this event adversely influenced participation in the online survey cannot be determined. However, research shows that low response rates do not necessarily cause nonresponse bias (Groves and Peytcheva 2008). And, as noted, some of the distributions for sample characteristics were roughly consistent with broader populations of full-time faculty at U.S. postsecondary institutions. Additionally, the unweighted results were similar to those that were observed when using survey weights. While the exact level of bias resulting from the low response rates is unknown, the available evidence suggests that it is likely negligible.

The second limitation concerns the omission of faculty members from the humanities (e.g., religious studies), formal sciences (e.g., mathematics), and other academic fields that do not regularly employ the scientific method. Although data fabrication and data falsification may be largely found in the natural, social, and applied sciences, some forms of research fraud (e.g., plagiarism and authorship fraud) also involve faculty members from other academic fields. Accordingly, the results from this study are limited in that they do not account for all fields where such violations occur. Future studies interested in forms of research fraud that do not directly involve the misuse

ACCOUNTABILITY IN RESEARCH 469

of the scientific method should include researchers from these other aca- demic fields.

4.2. Conclusion

Over the past few decades, growing empirical attention has focused on the nature and prevalence of researchers’ misbehavior – whether referred to as research misconduct, questionable research practices, or research fraud. The deleterious consequences associated with such deviance and the investment in research by the U.S. government necessitate continued inquiry. In fiscal 2018, the federal government allocated a reported 176.8 billion USD for research and development – 37 billion USD of which went to the National Institutes of Health and 7.8 billion USD of which went to the National Science Foundation (Science News Staff 2018). To safeguard this investment and protect the public image of science, continued empirical inquiry into research fraud and the development of sound prevention strategies should remain among the core objectives of the research integrity movement.

Notes

1. Judgments regarding the relative seriousness of different forms of research fraud can be somewhat subjective. However, one study provides some direction in this regard. Bouter et al. (2016) found that data fabrication ranked highest among attendees of four World Conferences on Research Integrity in terms of the negative effect it has on trust among scientists.

2. Article retractions and case summaries from the Office of Research Integrity (ORI) are two additional data sources that are used to study research fraud. Both approaches focus on known accounts of misbehavior – either instances of fraud published in refereed journals (see, e.g., Fang, Steen, and Casadevall 2012; Grieneisen and Zhang 2012; Steen 2011) or cases where a finding of misconduct was issued by the federal government (Davis, Riske-Morris, and Diaz 2007; Wright, Titus, and Cornelison 2008). Although both data sources have strengths, neither is able to shed much (if any) light on fraudulent behavior that goes undetected (i.e., the “dark figure” of research fraud; see Hesselmann, Wienefoet, and Reinhart 2014), nor do these studies provide informa- tion about compliant researchers.

3. Data fabrication and data falsification involve the misuse of the scientific method. These two misbehaviors were of interest in this study. Accordingly, academic fields that do not regularly employ the scientific method were not included (e.g., mathematics and music).

4. Response rates were calculated after taking into account bad e-mail addresses (n = 139) and incorrect mail addresses (n = 60). The response rate for the mail survey (15.6%) was higher than the online survey (8.1%). Although there are advantages to adminis- tering questionnaires online, such as relatively quick response times, research has shown that web-based surveys yield lower response rates when compared to postal survey methods (see, e.g., Sebo et al. 2017).

470 M. D. REISIG ET AL.

5. Missing survey data was handled using similar response pattern imputation. This procedure is available in PRELIS 2.3 (Scientific Software International, Chicago, IL).

6. A three-step process was followed to develop survey weights. First, the design-based weights, which were equal to the inverse probability of selection (Levy and Lemeshow 1999), were calculated. In this case, 2,000 faculty members from the target population in the social, natural, and applied sciences were randomly selected. Each set of 2,000 faculty were then randomly assigned to a recruitment method with two-thirds being assigned to e-mail recruitment (web survey) and one-third being assigned to mail recruitment (paper survey). Second, a weighting class adjustment was used within each scientific field and recruitment method combination to correct the survey weights for nonresponse (Gary 2007). Third, a composite estimator was applied to combine the online and mail respondents (Hartley 1962). The composite estimator was calculated using the probability of assignment to a recruitment method. After these steps, the survey weights of the sample respondents represented the target population.

7. Data from the survey have been used in prior investigations of the perceived causes of research misconduct (Holtfreter et al. 2020) and scholars’ preferences for dealing with research misconduct (Pratt et al. 2019).

8. An example of an item that was perceived to be too broad by the authors included items such as “[d]isagreements about authorship” or “[f]alsifying data,” both of which are included in the Scientific Misconduct Questionnaire – Revised (Broome et al. 2005, 274). The study’s authors concluded that these items (and other items like it) conflate different misbehaviors.

9. While these different types of research fraud are treated as if they were mutually exclusive, in actuality some overlap existed. For example, using fictitious data to make a grant proposal more competitive could technically be considered both data fabrication and grant fraud.

10. Two of the items from the grant fraud scale were pulled from Montgomerie and Birkhead's (2005) Scientific Misconduct Questionnaire (i.e., “Using grant funds to attend a conference and then not, or barely, showing up” and “Applying for grants to do work that is already done”).

11. Differences across scientific fields were also assessed using a combined 26-item research fraud scale (summated scale; Cronbach’s α =.926). The results from the one- way ANOVA model did not reveal statistically significant differences between the social, applied, and natural sciences (F-ratio = 1.526, p =.218).

12. Studies of self-reported criminal victimization and criminal involvement (both of which are often rare events) regularly use one year recall periods (see, e.g., Khade, Wang, and Decker 2018; Wolfe 2015). But whether such a recall period would capture sufficient variation in fraudulent behavior among researchers has yet to be determined.

Acknowledgments

The opinions, findings, and conclusions or recommendations expressed in this publication are those of the authors, and do not necessarily reflect those of the Department of Health and Human Services. A portion of these findings were presented at the 6th World Congress on Research Integrity in Hong Kong, June 2019. The authors would like to thank Katelyn Golladay, Ryan Mays, Susan Metosky, Travis Pratt, and Natasha Pusch for their valuable assistance.

ACCOUNTABILITY IN RESEARCH 471

Disclosure statement

No potential conflict of interest was reported by the authors.

Funding

This work was supported by the U.S. Department of Health and Human Services, Office of Research Integrity [grant numbers ORIIR150018-01-00 and ORIIR160028-04-00].

References

Anderson, M., K. Louis, and J. Earle. 1994. “Disciplinary and Departmental Effects on Observations of Faculty and Graduate Student Misconduct.” Journal of Higher Education 65 (3): 331–350. doi:10.1080/00221546.1994.11778504.

Baerlocher, M., J. O’Brien, M. Newton, T. Gautam, and J. Noble. 2010. “Data Integrity, Reliability and Fraud in Medical Research.” European Journal of Internal Medicine 21 (1): 40–45. doi:10.1016/j.ejim.2009.11.002.

Beauducel, A., and A. Herzberg. 2006. “On the Performance of Maximum Likelihood versus Means and Variance Adjusted Weighted Least Squares Estimation in CFA.” Structural Equation Modeling 13 (2): 186–203. doi:10.1207/s15328007sem1302_2.

Bhandari, A., and T. Wagner. 2006. “Self-reported Utilization of Health Care Services: Improving Measurement and Accuracy.” Medical Care Research and Review 63 (2): 217–235. doi:10.1177/1077558705285298.

Bollen, K., P. Biemer, A. Karr, S. Tueller, and M. Berzofsky. 2016. “Are Survey Weights Needed? A Review of Diagnostic Tests in Regression Analysis.” Annual Review of Statistics and Its Application 3 (1): 375–392. doi:10.1146/annurev-statistics-011516-012958.

Bosch, X., and J. Ross. 2012. “Ghostwriting: Research Misconduct, Plagiarism, or Fool’s Gold?” The American Journal of Medicine 125 (4): 324–326. doi:10.1016/j. amjmed.2011.07.015.

Bouter, L., J. Tijdink, N. Axelsen, B. Martinson, and G. Ter Riet. 2016. “Ranking Major and Minor Research Misbehaviors: Results from a Survey among Participants of Four World Conferences on Research Integrity.” Research Integrity and Peer Review 1 (1): 17. doi:10.1186/s41073-016-0024-5.

Broome, M., E. Pryor, B. Habermann, L. Pulley, and H. Kincaid. 2005. “The Scientific Misconduct Questionnaire—Revised (SMQ-R): Validation and Psychometric Testing.” Accountability in Research 12 (4): 263–280. doi:10.1080/08989620500440253.

Brown, T. 2006. Confirmatory Factor Analysis for Applied Research. New York, NY: Guilford Press.

Bruton, S. 2014. “Self-plagiarism and Textual Recycling: Legitimate Forms of Research Misconduct.” Accountability in Research 21 (3): 176–197. doi:10.1080/ 08989621.2014.848071.

Buljan, I., L. Barać, and A. Marušić. 2018. “How Researchers Perceive Research Misconduct in Biomedicine and How They Would Prevent It: A Qualitative Study in a Small Scientific Community.” Accountability in Research 25 (4): 220–238. doi:10.1080/ 08989621.2018.1463162.

Cattano, V., and J. Turk. 2007. “Fraud and Misconduct in Scientific Research: A Definition and Procedures for Investigation.” Medicine and Law 26 (3): 465–476.

Claxton, L. 2005. “Scientific Authorship, Part 2: History, Recurring Issues, Practices, and Guidelines.” Mutation Research 589 (1): 31–45. doi:10.1016/j.mrrev.2004.07.002.

472 M. D. REISIG ET AL.

Coats, A. 2009. “Ethical Authorship and Publishing.” International Journal of Cardiology 131 (2): 149–150. doi:10.1016/j.ijcard.2008.11.048.

Davis, M., M. Riske-Morris, and S. Diaz. 2007. “Causal Factors Implicated in Research Misconduct: Evidence from ORI Case Files.” Science and Engineering Ethics 13 (4): 395–414. doi:10.1007/s11948-007-9045-2.

De Vries, R., M. Anderson, and B. Martinson. 2006. “Normal Misbehavior: Scientists Talk about the Ethics of Research.” Journal of Empirical Research on Human Research Ethics 1 (1): 43–50. doi:10.1525/jer.2006.1.1.43.

Dillman, D., J. Smyth, and L. Christian. 2014. Internet, Phone, Mail, and Mixed-Mode Surveys: The Tailored Design Method. 4th ed. Hoboken, NJ: Wiley.

Fanelli, D. 2009. “How Many Scientists Fabricate and Falsity Research? A Systematic Review and Meta-analysis of Survey Data.” PLoS One 4 (5): e5738. doi:10.1371/journal. pone.0005738.

Fang, F., R. Steen, and A. Casadevall. 2012. “Misconduct Accounts for the Majority of Retracted Scientific Publications.” Proceedings of the National Academy of Sciences 109 (42): 17028–17033. doi:10.1073/pnas.1212247109.

Fiedler, K., and N. Schwarz. 2016. “Questionable Research Practices Revisited.” Social Psychological and Personality Science 7 (1): 45–52. doi:10.1177/1948550615612150.

Gary, P. 2007. “Adjusting for Nonresponse in Surveys.” In Higher Education: Handbook of Theory and Research, edited by J. Smart, 411–449. Dordrecht: Springer.

Godescharle, S., S. Fieuws, B. Nemery, and K. Dierickx. 2018. “Scientists Still Behaving Badly? A Survey within Industry and Universities.” Science and Engineering Ethics 24 (6): 1697–1717. doi:10.1007/s11948-017-9957-4.

Grants.gov. 2020. “Grant Fraud and Scams.” Accessed 1 April 2020. https://www.grants.gov Grieneisen, M., and M. Zhang. 2012. “A Comprehensive Survey of Retracted Articles from

the Scholarly Literature.” PLoS One 7 (10): e44118. doi:10.1371/journal.pone.0044118. Groves, R., and E. Peytcheva. 2008. “The Impact of Nonresponse Rates on Nonresponse Bias:

A Meta-Analysis.” Public Opinion Quarterly 72 (2): 167–189. doi:10.1093/poq/nfn011. Hartley, H. 1962. “Multiple Frame Surveys.” Proceedings of the Social Statistics Section,

American Statistical Association 19 (6): 203–206. Haven, T., J. Tijdink, B. Martinson, and L. Bouter. 2019. “Perceptions of Research Integrity

Climate Differ between Academic Ranks and Disciplinary Fields: Results from a Survey among Academic Researchers in Amsterdam.” PLoS One 14 (1): e0210588. doi:10.1371/ journal.pone.0210599.

Hesselmann, F., V. Wienefoet, and M. Reinhart. 2014. “Measuring Scientific Misconduct— Lessons from Criminology.” Publications 2 (3): 61–70. doi:10.3390/publications2030061.

Holtfreter, K., M. Reisig, T. Pratt, and R. Mays. 2020. “The Perceived Causes of Research Misconduct among Faculty Members in the Natural, Social, and Applied Sciences.” Studies in Higher Education. Advance online publication. doi:10.1080/03075079.2019.1593352.

Hopp, C., and G. Hoover. 2019. “What Crisis? Management Researchers’ Experiences with and Views of Scholarly Misconduct.” Science and Engineering Ethics 25 (5): 1549–1588. doi:10.1007/s11948-018-0079-4.

John, L., G. Loewenstein, and D. Prelec. 2012. “Measuring the Prevalence of Questionable Research Practices with Incentives for Truth Telling.” Psychological Science 23 (5): 524–532. doi:10.1177/0956797611430953.

Jones, J., and L. McCullough. 2015. “Is a Gift Authorship Really a Grift Authorship?” Journal of Vascular Surgery 61 (4): 1092–1093. doi:10.1016/j.jvs.2015.02.006.

Kalichman, M., M. Sweet, and D. Plemmons. 2015. “Standards of Scientific Conduct: Disciplinary Differences.” Science and Engineering Ethics 21 (5): 1085–1093. doi:10.1007/ s11948-014-9594-0.

ACCOUNTABILITY IN RESEARCH 473

Khade, N., X. Wang, and S. Decker. 2018. “Examining the Link between Childhood Physical Abuse and Risk for Violence Victimization in Youth and Young Adulthood in China.” Journal of Interpersonal Violence: 088626051879400. Advance online publication. doi:10.1177/0886260518794002.

Levy, P., and S. Lemeshow. 1999. Sampling of Populations: Methods and Applications. 3rd ed. New York, NY: Wiley.

Li, D., and C. Koedel. 2017. “Representation and Salary Gaps by Race-ethnicity and Gender at Selective Public Universities.” Educational Researcher 46 (7): 343–354. doi:10.3102/ 0013189X17726535.

Loui, M. 2002. “Seven Ways to Plagiarize: Handling Real Allegations of Research Misconduct.” Science and Engineering Ethics 8 (4): 529–539. doi:10.1007/s11948-002- 0005-6.

Martinson, B., M. Anderson, and R. De Vries. 2005. “Scientists Behaving Badly.” Nature 435 (9): 735–737. doi:10.1038/435737a.

Montgomerie, B., and T. Birkhead. 2005. “A Beginner’s Guide to Scientific Misconduct.” ISBE Newsletter 17 (1): 16–24.

Moskovitz, C. 2019. “Text Recycling in Scientific Writing.” Science and Engineering Ethics 25 (3): 813–851. doi:10.1007/s11948-018-0060-2.

Nakashima, E. 2016. “Russian Government Hackers Penetrated DNC, Stole Opposition Research on Trump.” The New York Times, June 14. http://www.nytimes.com

National Center for Education Statistics. 2018. “Digest of Education Statistics, Table 315.20.” Accessed 25 August 2019. https://nces.ed.gov/programs/digest/d18/tables/dt18_315.20.asp

National Center for Education Statistics. 2019. “Integrated Postsecondary Education Data System. Use the Data.” Accessed 25 August 2019. https://nces.ed.gov/ipeds/use-the-data

Office of Research Integrity. 2019. “Definition of Research Misconduct.” Accessed 9 July 2019. https://ori.hhs.gov

Phillips, B., J. Lombardi, C. Abbey, and D. Craig. 2013. The Top American Research Universities: 2013 Annual Report. Tempe, AZ: Center for Measuring University Performance.

Pratt, T., M. Reisig, K. Holtfreter, and K. Golladay. 2019. “Scholars’ Preferred Solutions for Research Misconduct: Results from a Survey of Faculty Members at America’s Top 100 Research Universities.” Ethics and Behavior 29 (7): 510–530. doi:10.1080/ 10508422.2019.1612748.

Pryor, E., B. Habermann, and M. Broome. 2007. “Scientific Misconduct from the Perspective of Research Coordinators: A National Survey.” Journal of Medical Ethics 33 (6): 365–369. doi:10.1136/jme.2006.016394.

Pupovac, V., and D. Fanelli. 2015. “Scientists Admitting to Plagiarism: A Meta-analysis of Surveys.” Science and Engineering Ethics 21 (5): 1331–1352. doi:10.1007/s11948-014-9600-6.

Ranstam, J., M. Buyse, S. George, S. Evans, N. Geller, B. Scherrer, E. Lesaffre, G. Murray, L. Edler, J. Hutton, et al. 2000. “Fraud in Medical Research: An International Survey of Biostatisticians.” Controlled Clinical Trials 21 (5): 415–427. doi:10.1016/s0197-2456(00) 00069-6.

Rogers, L. 1999. “Salami Slicing, Shotgunning, and the Ethics of Authorship.” American Journal of Roentgenology 173 (2): 265. doi:10.2214/ajr.173.2.10430115.

Rohrich, R., and D. Sullivan. 2009. “Plagiarism and Dual Publication: Review of the Issues and Policy Statement.” Plastic and Reconstructive Surgery 124 (4): 1333–1339. doi:10.1097/ PRS.0b013e3181b59d42.

Science News Staff. 2018. “Trump, Congress Approve Largest U.S. Research Spending Increase in a Decade.” Science, March 23. Accessed 30 July 2019. https://www.science mag.org

474 M. D. REISIG ET AL.

Sebo, P., H. Maisonneuve, B. Cerutti, J. Fournier, N. Senn, and D. Haller. 2017. “Rates, Delays, and Completeness of General Practitioners’ Responses to A Postal versus Web- based Survey: A Randomized Trial.” Journal of Medical Internet Research 19 (3): e83. doi:10.2196/jmir.6308.

Smith, E., B. Williams-Jones, Z. Master, V. Lariviére, C. Sugimoto, A. Paul-Hus, M. Shi, E. Diller, K. Caudle, and D. Resnik. 2019. “Researchers’ Perceptions of Ethical Authorship Distribution in Collaborative Research Teams.” Science and Engineering Ethics. Advance online publication. doi:10.1007/s11948-019-00113-3.

Steen, R. 2011. “Retractions in the Scientific Literature: Do Authors Deliberately Commit Research Fraud?” Journal of Medical Ethics 37 (2): 113–117. doi:10.1136/jme.2010.038125.

Teixeria da Silva, J., and J. Dobránski. 2016. “Multiple Authorship in Scientific Manuscripts: Ethical Challenges, Ghost and Guest/Gift Authorship, and the Cultural/Disciplinary Perspective.” Science and Engineering Ethics 22 (5): 1457–1472. doi:10.1007/s11948-015- 9716-3.

Tijdink, J., R. Verbeke, and Y. Smulders. 2014. “Publication Pressure and Scientific Misconduct in Medical Scientists.” Journal of Empirical Research on Human Research Ethics 9 (5): 64–71. doi:10.1177/1556264614552421.

Warren, M. 2018. “Ghost Authorship Haunts Industry-funded Clinical Trials.” Nature, October 9. Accessed 6 April 2018. https://www.nature.com

Wolfe, S. 2015. “Low Self-control, Gender, Race, and Offending in Late Life.” Psychology, Crime and Law 21 (5): 426–451. doi:10.1080/1068316X.2014.989169.

Wright, D., S. Titus, and J. Cornelison. 2008. “Mentoring and Research Misconduct: An Analysis of Research Mentoring in Closed ORI Cases.” Science and Engineering Ethics 14 (3): 323–336. doi:10.1007/s11948-008-9074-5.

Appendix. Pearson’s correlation coefficients

Variables 1. 2. 3. 4. 5. 6.

1. Data fabrication 1.00

2. Data falsification .79* 1.00

3. Plagiarism .54* .50* 1.00

4. Authorship fraud .45* .48* .47* 1.00

5. Publication fraud .58* .48* .58* .51* 1.00

6. Grant fraud .63* .61* .45* .51* .53* 1.00

*p < .001 (two-tailed test).

ACCOUNTABILITY IN RESEARCH 475

Copyright of Accountability in Research: Policies & Quality Assurance is the property of Taylor & Francis Ltd and its content may not be copied or emailed to multiple sites or posted to a listserv without the copyright holder's express written permission. However, users may print, download, or email articles for individual use.