Bridging the Gap

Industrial and Organizational Psychology, 11(2), pp 220–266 June 2018. Copyright © 2018 Society for Industrial and Organizational Psychology. doi:10.1017/iop.2018.8

Commentaries

A Systems View of the Scientist–Practitioner Gap

Jeffrey Olenick, Ross Walker, Jacob Bradburn, and Richard P. DeShon Michigan State University

We commend Rotolo et al. (2018) for introducing a new lens for viewing the well-known gap between industrial and organizational (I-O) psychology research and human resource (HR) practices in organizations. However, Ro- tolo et al.’s characterization of practitioner behavior as “anti I-O” suggests a particularly negative view of scientific research among some HR practition- ers. The label implies that some HR practitioners are intentionally ignoring or actively resisting academic research. More likely, the behavior stems from a passive indifference to academia, which may be the appropriate attitude for some practitioners to adopt when a great deal of academic research is too slow, too theoretical, and too cryptically communicated to be useful in ap- plied settings. We agree with Rotolo et al. when they say, “we are a discipline that is not geared for being cutting edge” (p. 182), and we appreciate their recommendations for addressing this lack of relevance. However, most rec- ommendations in this broader discussion do not address the foundational problem within our field: a systemic mismatch between the incentives of practitioners and academics. To support this point, we briefly describe a ty- pology of I-O psychologists as well as the varying contexts and incentives that drive their behavior. We then close with our own recommendations for how academia can improve its relevance to practitioners and close the gap. These changes are not easy, but we agree with Rotolo and colleagues that if any field can address such foundational problems, it is ours.

A Typology of Organizational Psychologists Rotolo et al. (2018) characterize organizational psychologists as “scientist– practitioners.” This popular term represents a specific location on a research–practice continuum at which attention to research and application are carefully balanced. This may represent a macro property of our field, but,

Jeffrey Olenick, Michigan State University; Ross Walker, Michigan State University; Jacob Bradburn, Michigan State University; Richard P. DeShon, Michigan State University.

Correspondence concerning this article should be addressed to Jeffrey Olenick, Depart- ment of Psychology, Michigan State University, 316 Physics Rd., East Lansing, MI 48824. E-mail: [email protected]

220

https://www.cambridge.org/core/terms. https://doi.org/10.1017/iop.2018.8 Downloaded from https://www.cambridge.org/core. IP address: 107.77.201.177, on 19 Oct 2020 at 15:29:54, subject to the Cambridge Core terms of use, available at

systems view of the scientist–practitioner gap 221

in our experience, it does not adequately capture the diversity of our field at the micro level (i.e., the specific contexts in which individual researchers and practitioners operate). Instead, expanding upon a typology briefly described by Rupp and Beal (2007), we see at least four distinct patterns in our I-O psychology colleagues: pure scientists, scientist–practitioners, practitioner– scientists, and pure practitioners.

The Pure Scientist Pure scientists in I-O psychology are traditional academics who want to un- derstand and explain the causes of human behavior within the workplace and organizations. They are incentivized primarily to publish in prestigious journals, earn tenure, and obtain grants to support their programs of re- search. They likely serve as editors or associate editors for one or more research-oriented journals. To these individuals, advancing theory and pur- suing “knowledge for knowledge’s sake” are legitimate endeavors. Typically, they are not focused on, nor rewarded for, researching topics addressing emerging HR trends, developing tools useful for practitioners, or expound- ing on the practical implications of their research.

The Scientist–Practitioner Scientist–practitioners are also predominantly, though not exclusively, found in universities. They are researchers, first and foremost, but they care deeply about having real-world impact and often partner with HR profession- als and/or I-O psychology colleagues in organizations to develop and im- plement evidence-based HR products and procedures. These individuals can still build successful academic careers through high-quality publica- tions on practically oriented topics. However, because they are employed in universities, they are subject to the same incentive system as pure sci- entists, and those requirements can create tension with their applied goals. Scientist–practitioners located in applied research institutions typically con- duct practice-oriented research, and although they may periodically publish in academic journals, they are not constrained by the same demands as uni- versity academics.

The Practitioner–Scientist Practitioner–scientists typically live in large for-profit organizations, but they can also be found in large external consulting firms, large nonprofit or- ganizations, and a variety of state and federal government roles. They pri- marily focus on adding value to organizations by advising on human re- source strategies and implementing research-based practices. However, they are personally motivated to contribute expertise to the collective knowl- edge pool through publications, either in academic journals and books.

https://www.cambridge.org/core/terms. https://doi.org/10.1017/iop.2018.8 Downloaded from https://www.cambridge.org/core. IP address: 107.77.201.177, on 19 Oct 2020 at 15:29:54, subject to the Cambridge Core terms of use, available at

222 jeffrey olenick et al.

Additionally, they may collaborate with academics on research projects. They tend to invest extra-role efforts to maintain familiarity with current research and strive to implement best practices consistent with research findings.

The Pure Practitioner Pure practitioners are usually employed by the same organizations as practitioner–scientists. They are driven almost exclusively by organizational needs. They are either uninterested in or too constrained by time and re- sources to read academic research, but they may learn from practitioner- focused outlets such as Harvard Business Review. These individuals may feel that pure science is disconnected from the realities they face every day in real organizations. They may focus more on trending topics and on signaling their value to organizational leaders than on the evidentiary support of their work. They tend to rely on a network of vendors to provide best practices rather than recent research findings.

Systemic Pressures As Garman (2011) notes regarding academia and practice, “within the two contexts, success is defined very differently” (p. 130). Pure scien- tists and scientist–practitioners are driven by publication standards im- posed by journals and the requirements of their university tenure systems, whereas pure practitioners and practitioner–scientists must ultimately add and demonstrate value to organizations. Thus, although the personal mo- tivations of I-O psychologists are numerous, the structural incentives are bimodal. Earnestly honoring both sides, as the scientist–practitioners and practitioner–scientists desire, often requires additional work for the same, or even less, reward. Many commentaries on the science–practice divide have been written from both sides; however, no amount of finger wagging in either direction will engender meaningful change. The gap between research and practice is the inevitable result of the disconnected incentive structures that scientists and practitioners face. Any changes that do not address this fun- damental incongruity will continue to allow “anti I-O” practices to survive. For true change to occur, the incentive structure of the field must change.

Other Barriers to the Scientist–Practitioner Ideal In addition to differing incentive systems, the scientist–practitioner gap is exacerbated by two characteristics of academia that we believe Rotolo et al. (2018) underemphasize: (a) an overreliance on theory and theoretical contri- bution and (b) a mismatch between publication timelines and organizational interests. These factors often prevent scientists from doing the research on emerging and frontier topics that Rotolo et al. espouse.

https://www.cambridge.org/core/terms. https://doi.org/10.1017/iop.2018.8 Downloaded from https://www.cambridge.org/core. IP address: 107.77.201.177, on 19 Oct 2020 at 15:29:54, subject to the Cambridge Core terms of use, available at

systems view of the scientist–practitioner gap 223

Overreliance on Theory Rotolo et al. stress the importance of theoretical alignment on emerging top- ics. Although this may be desirable in and of itself, academic overreliance on theory often prevents research on practical and emerging topics. The increasing valuation of “theoretical contribution” in organizational science is undeniable. For example, Cucina and Moriarty (2015) showed how two of our top journals, Journal of Applied Psychology and Personnel Psychology, have transformed from practice-oriented to strongly theory-oriented outlets over the past few decades and in ways not seen in other prominent outlets (e.g., American Psychologist and Psychological Bulletin, as well as scientific gold standards such as Nature and Science). Hambrick (2007) speculated that this trend in management science stems from academic insecurity, which it- self may have arisen from external criticism of the academic sophistication of business schools.

Theory is undeniably useful (e.g., for explaining, predicting, synthesiz- ing, and preventing rash conclusions based on anomalous findings; Miller, 2007) but the pendulum has, perhaps, swung too far toward deduction. Re- quiring that all published research be firmly grounded in existing theory and make a substantial theoretical contribution inhibits publication of impor- tant, frontier, and rigorous but atheoretical findings. Under such conditions, research on emerging topics and new technologies that practitioners want can be prohibitively risky, particularly for assistant professors seeking tenure. Allowing rigorous but atheoretical research into our established literature may allow a more rapid response to the needs of practitioners and provide the basis for the development of future theory.

Temporal Mismatch Rotolo et al. (2018) lament a difference in the speed of science and prac- tice. Namely, academia progresses at a much slower rate than practice de- mands, and we echo this concern. It can take years for even simple research to go through theoretical conception, data collection, analysis, manuscript writing, and potentially multiple rounds of revisions. Often, reviewers try to remake papers in their preferred image, requiring increasingly extensive and time-consuming changes, which can result in three or more revisions at a single journal with no guarantee of acceptance. The review process it- self can significantly slow the time it takes for research to reach publication. Other work has shown that publication time across science has been consis- tent over the last 30 years but that social science publication times lag behind the natural sciences (Powell, 2016). This is particularly problematic in I-O psychology because those slower publication times are not improving, while at the same time the organizations we study are moving at an ever-faster rate. This pace of change exacerbates the science–practice gap as science falls ever

https://www.cambridge.org/core/terms. https://doi.org/10.1017/iop.2018.8 Downloaded from https://www.cambridge.org/core. IP address: 107.77.201.177, on 19 Oct 2020 at 15:29:54, subject to the Cambridge Core terms of use, available at

224 jeffrey olenick et al.

further behind. In the time required to research and publish a study on a new topic, an organization may try multiple approaches to the same prob- lem, discard the approaches that do not work, and keep the ones that do (or appear to). In addition, this slow publication system only leads to results in the aggregate. Thus, we may know that a procedure does not work well for organizations in general but have little evidence on how it works in specific organizations. This lack of contextualization further incentivizes practition- ers to experiment independently.

We believe that caution is warranted as well: Science should be slower than “popular” advances to ensure rigor. Science strives for cumulative the- ory building rather than anecdotal evidence, and the former necessarily takes more time. However, whereas hastening research could hypothetically com- promise quality, the current system is far from this danger.

Recommendations We recommend three key changes to the academic system that can help close the gap between science and practice. We limit our recommendations to academia for two reasons: (a) Incentives in academia are more homoge- nous than those in industry and are, therefore, more conducive to broad pre- scriptions, and (b) we are not practitioners, so constructively criticizing their idiosyncratic systems could be irresponsible. However, we encourage prac- titioners to take a deeper look at their side of the divide and consider similar ways to restructure their incentives.

1. Reduce Our Reliance on Theory Although it is perfectly natural for academics to be more interested in ex- plaining phenomena and for practitioners to be more interested in predicting, research need not stay on the theoretical end of the spectrum. Not all studies need to be directly tied to a specific organizational concern; there is room for basic psychological research in our field. However, allowing academic inse- curity to drive our research questions is unlikely to yield meaningful cumula- tion. Our recommendation aligns with those of Hambrick (2007) and Miller (2007). Journals should swing the pendulum back toward the center by re- warding practically significant studies with high probabilities of stimulating meaningful future research and loosening requirements for novel theoreti- cal contribution. Any well-executed study that moves toward better under- standing of consequential phenomena in organizations deserves a chance at publication, even without a significant theoretical contribution.

2. Improve the Speed of Our Pipelines to Production Expediting publication of quality research will help academia meet practi- tioners’ needs more effectively. Although the average time from acceptance

https://www.cambridge.org/core/terms. https://doi.org/10.1017/iop.2018.8 Downloaded from https://www.cambridge.org/core. IP address: 107.77.201.177, on 19 Oct 2020 at 15:29:54, subject to the Cambridge Core terms of use, available at

systems view of the scientist–practitioner gap 225

to publication has fallen due to technology (Powell, 2016), the transition from physical mail to electronic correspondence has not decreased review times. Journals should seek ways in which the same technological advance- ments that improve time from acceptance to publication would shorten the review process itself.

We recommend that reviewers and editors return to the predominant role they played in earlier days of our science. That is, they should func- tion primarily as gatekeepers of academic rigor, weeding out poor science and moving quality work toward publication as quickly and with as lit- tle tampering as possible. Subjective judgments of quality and contribu- tion are unavoidable and often desirable, but remaking articles through drawn-out revision processes is often counterproductive. This is especially true when the primary purpose of an article is data analysis and presen- tation. In those cases, the primary judgment made by reviewers and ed- itors should regard the rigor of the study design and appropriateness of the analyses. If the study satisfies those criteria and has either theoreti- cal or practical utility, the article should be moved rapidly toward publica- tion.

Finally, although busy schedules may hinder tight turnarounds both for authors and reviewers, limiting the scope of revisions is a reasonable compromise. We recommend that editors avoid multiple rounds of re- visions, which can add additional months, or even years, to the review process. In particular, action editors should strive to make clear, swift de- cisions on a submission’s potential contribution and simply reject sub- missions that require too much alteration. Powell (2016) describes a rela- tively new open access journal in biomedical science called eLife that has adopted a strategy of either reviewing submissions quickly or rejecting them. The strategy includes quick initial decisions, single rounds of revi- sion whenever possible, and limited requests for additional analyses (i.e., two months maximum). The current impact factor for eLife is 7.725, which is not particularly high for biomedical sciences but sizeable by our stan- dards.

Allowing any well-executed study into public knowledge quickly could have several positive effects on our science. For example, it would likely reduce the so called “file-drawer” effect where valid data collections never see publication, either because journals reject them or because they are never submitted. Expediting parts of the pub- lication process without compromising on quality will allow for the greater and faster cumulation of knowledge in our field. It will also allow academia to more nimbly respond to the needs of real organi- zations by providing them with sound science in a relatively timely manner.

https://www.cambridge.org/core/terms. https://doi.org/10.1017/iop.2018.8 Downloaded from https://www.cambridge.org/core. IP address: 107.77.201.177, on 19 Oct 2020 at 15:29:54, subject to the Cambridge Core terms of use, available at

226 jeffrey olenick et al.

3. Change Incentive Structures to Reward a Wider Range of Work Academic incentives must change to encourage the cutting-edge research that Rotolo et al. (2018) say is lacking. A recent symposium published in Perspectives on Psychological Science on measuring merit in academia speaks directly to this point. Specifically, Lubart and Mouchiroud (2017) propose that psychology more generally should move beyond the common measures of merit, such as the popular h-index (Hirsch, 2005), to include measures of transmission, originality, usefulness, and generativity. For example, they view usefulness as developing “valuable new tools or practices in an applied setting” (Hirsch, 2005, p. 1160). Their examples include publishing in practi- tioner magazines, recommendation reports for practitioners, citations in the media, and invited talks at practitioner conferences. Under our field’s usual incentive structure, these outlets are not explicitly encouraged or rewarded.

Beyond individual metrics of success, expanding and altering the calcu- lation of journal impact factors could facilitate our first two recommenda- tions. Right now, journals are only incentivized to publish articles with high probabilities of citation. This exacerbates the file-drawer effect by limiting the range of article types. We cannot expect meaningful changes in publica- tion without changes to macro incentives as well.

Expanding how our field defines meritorious contributions is the only way to bridge the science–practice gap in a meaningful way. We are under no illusion that this represents an easy change to make. There are overrid- ing pressures on I-O psychology departments (and our partners in manage- ment, HR, and other related fields) from the larger university systems within which they exist. However, as evidenced by the broader discussion regard- ing merit in the psychological sciences, growing awareness may help bring positive change to the field of which I-O psychology is a part (e.g., Lubart & Mouchiroud, 2017). Although it will be difficult, this systemic change could free scientist–practitioners to move back toward the center of the science– practice continuum and do the kind of work that would help our field bridge the gap (e.g., designing practical tools and translating science in practitioner- friendly outlets).

Conclusion As technology accelerates and forces organizations to adapt, the gap between science and practice will likely widen without systemic changes and allow “anti I-O” practices to proliferate. We are cautious not to paint a picture of gloom for the future of our field, as we do believe that our science can ad- dress the problems at hand and remain relevant for organizations far into the future. However, remaining relevant requires that we, as a field, be willing to take a realistic and hard look at ourselves and be proactive about making the changes that such self-reflection suggests. The time to start making those

https://www.cambridge.org/core/terms. https://doi.org/10.1017/iop.2018.8 Downloaded from https://www.cambridge.org/core. IP address: 107.77.201.177, on 19 Oct 2020 at 15:29:54, subject to the Cambridge Core terms of use, available at

what curbs frontier research? 227

changes is now. We hope that our perspective and recommendations can contribute to a much larger conversation about practical ways to ensure a bright future for I-O psychology.

References Cucina, J. M., & Moriarty, K. O. (2015). A historical look at theory in industrial-organizational psy-

chology journals. The Industrial-Organizational Psychologist, 53(1), 57–70. Garman, A. N. (2011). Shooting for the moon: How academicians could make management research

even less irrelevant. Journal of Business Psychology, 26(2), 129–133. Hambrick, D. C. (2007). The field of management’s devotion to theory: Too much of a good thing?

Academy of Management Journal, 50(6), 1346–1352. Hirsch, J. (2005). An index to quantify an individual’s research output. Proceedings of the National

Academy of Sciences, USA, 102, 16569–16572. Lubart, T., & Mouchiroud, C. (2017). TOUGH: The measure of merit in psychological science. Per-

spectives on Psychological Science, 12(6), 1159–1161. Miller, D. (2007). Paradigm prison, or in praise of atheoretic research. Strategic Organization, 5(2),

177–184. Powell, K. (2016). Does it take too long to publish research? Nature, 530, 148–151. Rotolo, C.T., Church, A. H., Adler, S., Smither, J.W., Colquitt, A., Shull, A.C., … Foster, G. (2018).

Putting an end to bad talent management: A call to action for the field of I-O psychology. Indus- trial and Organizational Psychology: Perspectives on Science and Practice, 11(2), 176–219.

Rupp, D. E., & Beal, D. (2007). Checking in with the scientist–practitioner model: How are we doing. The Industrial-Organizational Psychologist, 45(1), 35–40.

What Curbs Frontiers Research? A Reaction to Rotolo et al.’s Article

Edna Rabenu and Aharon Tziner Netanya Academic College

Rotolo et al. (2018) decry the rise in use of trendy, simplistic human resource management (HRM) procedures and practices such as talent management, regardless of any solid scientific basis culled from relevant disciplines such as industrial and organizational (I-O) psychology. Furthermore, they observe a propagating spirit of anti-I-O psychology that has recently emerged and that should provoke our concern. What has ignited and fueled this reality? Correctly noted, Rotolo et al. indicate that I-O psychology academics have, over the years, lost touch with the actual preoccupying needs of managers

Edna Rabenu, Schools of Behavioral Studies and Business Administration, Netanya Aca- demic College; Aharon Tziner, Schools of Behavioral Studies and Business Administration, Netanya Academic College.

Correspondence concerning this article should be addressed to Aharon Tziner, Schools of Behavioral Studies and Business Administration, Netanya Academic College, 1, University St., Netanya 42365, Israel. E-mail: [email protected]

https://www.cambridge.org/core/terms. https://doi.org/10.1017/iop.2018.8 Downloaded from https://www.cambridge.org/core. IP address: 107.77.201.177, on 19 Oct 2020 at 15:29:54, subject to the Cambridge Core terms of use, available at