Discussion Post

Comment

494 www.thelancet.com Vol 393 February 9, 2019

Funders should evaluate projects, not people Since the Cold War, public and private funding agencies have relied on peer review to guide their investments in science.1 This process allows funders to make informed bets when allocating resources to advance their goals. In The Lancet, Holly Witteman and colleagues2 suggest that a traditional, project-based grant review process outperforms the increasingly popular people-not- project funding schemes. Their analysis leverages a natural experiment created by the decision of the Canadian Institutes of Health Research to divide its funding into Project and Foundation grant programmes in 2014. This change in policy enabled the authors to compare traditional peer review procedures evaluating both the project and the scientist with those focused either primarily on the project or on the scientist. When the peer review focused on identifying the best scientific ideas (Project programme), the proposals from male and female investigators were funded in roughly equal proportions. By contrast, when the review focused on assessing the scientist (Foundation programme), male investigators were an astounding 1·4 times more likely to receive funding than female investigators, a disparity that persisted after adjustment for age and domain of research.

Witteman and colleagues adeptly consider potential causes for the documented gender disparity in the scientist-focused funding programme, and make a compelling case for two factors. One factor is an influence of individual bias, in which peer reviewers’ subjective evaluations are affected by explicit or implicit (unconscious) gender bias. Gender bias is a preference for or unequal treatment of one gender over another. Most people, scientists included, more readily link men with science in implicit association tests measuring reaction times,3 and prefer men when asked to evaluate resumes that are identical except for the gender conveyed by the name.4 Evaluations are most likely to be affected by implicit bias when they are generated under conditions that increase the cognitive load associated with decision making, such as time pressure,5 scarcity of information, or high stakes, all of which might occur during peer review. The second factor is systemic bias, which refers to the accrual of advantage to male scientists, based on differences in funding, service load, the burden of sexual harassment, and other gender- based disparities. Consider the hypothetical case of a matched pair of equally talented scientists, John and Jane, whose ideas and research plans are equally likely to advance knowledge and yield a cure for Alzheimer’s disease. Given the uncertainty inherent in fundamental research, a rational investment strategy would be to fund John and Jane at equal levels. But suppose that implicit bias allows John to get more grants than Jane in the first few rounds of peer review. This funding would enable John to hire an extra postdoctoral researcher or two, who will publish a few additional papers, which will make John more competitive for the next round of funding. In this way, an advantage (bias) that initially exists only in the mind of reviewers can create a real- world disparity of opportunity and achievement that compounds over time. If the proportions of successful applications were like those in the people-based funding mechanism of the Canadian Institutes of Health Research (12·7% for male vs 8·8% for female scientists, after adjustment for age and domain of research), then the compounded advantage enjoyed by John would result in a highly skewed investment in his research (figure), even though Jane’s research programme is equally likely to yield a cure that could

See Articles page 531

For #LancetWomen see https://www.thelancet.com/

lancet-women

Cu m

ul at

iv e

gr an

ts fu

nd ed

(p re

di ct

ed )

Rounds of applications (n)

Jane

John People focused funding programme Project focused funding programme

Figure: Compounding of advantage to male scientists

Comment

www.thelancet.com Vol 393 February 9, 2019 495

benefit multitudes of people. In the project-based funding programme of the Canadian Institutes of Health Research, the difference in the percentage of successful male and female applicants (12·9% vs 12·1%, after adjustment for age and domain of research) was not statistically significant; nevertheless, even small differences have the potential to yield substantial disparities when compounded over time (figure).6,7

People-based funding mechanisms have been gaining popularity8 as a way to provide scientists with more freedom to innovate. Yet, Witteman and colleagues’ findings show that this approach is highly compromised by the influence of individual or systemic bias, or both. To achieve their scientific and clinical goals, funders would do better using project-based instead of people- based grant review to guide research investments, and by designing new methods to promote innovation across the broader research community. That said, even project-focused reviews typically include assessment of the investigator as a component of the review process, and this is likely to contribute to the consistent finding across countries, disciplines, and career stages that male scientists receive more research funding than their female counterparts. A potential solution suggested by Witteman and colleagues is to adjust review scores to account for gender biases. We propose an alternative, which leverages the documented benefits of anonymised or double-blind review for mitigating the influence of biases related to gender and other factors:9,10 assess the investigator only after double-blind review of the proposed research is complete; and reduce the assessment of the investigator to a binary judgment of whether or not the investigator has the expertise and resources needed to do the proposed research.

Similar approaches could be used to improve scientific judgments and decisions throughout the practice of science. Numerous studies indicate that peer review of scientific articles is similarly compromised by biases, as are decisions about hiring, start-up resources,11

assignment of laboratory space,12 and hiring advisory board members for biotech companies.13,14 As a result, even if Jane and John were to receive equal funding, society is less likely to benefit from the cure that Jane’s research has the potential to produce. We all stand to benefit by optimising the return on investment in science. This will require rethinking many, if not all, aspects of the practice of science to reduce the influence of individual and systemic bias about scientists and refocus assessments on the promise of the research.

*Jennifer L Raymond, Miriam B Goodman Department of Neurobiology (JLR) and Molecular and Cellular Physiology (MBG), Stanford University, Stanford, CA 94305, USA [email protected]

We declare no competing interests.

1 Baldwin M. Scientific autonomy, public accountability, and the rise of “peer review” in the Cold War United States. ISIS 2018; 109: 538–58.

2 Witteman H, Hendricks M, Straus S, Tannenbaum C. Are gender gaps due to evaluations of the applicant or the science? A natural experiment at a national funding agency. Lancet 2019; 393: 531–40.

3 Nosek BA, Smyth FL, Sriram N, et al. National differences in gender–science stereotypes predict national sex differences in science and math achievement. Proc Natl Acad Sci USA 2009, 106: 10593–97.

4 Moss-Racusin CA, Dovidio JF, Brescoll VL, Graham MJ, Handelsman J. Science faculty’s subtle gender biases favor male students. Proc Natl Acad Sci USA 2012, 109: 16474–79.

5 Dijker AJ, Koomen W. Stereotyping and attitudinal effects under time pressure. Eur J Soc Psychol 1996; 26: 61–74.

6 Martell R, Lane D, Emrich C. Male-female differences: a computer simulation. Am Psychologist 1996; 51: 157–58.

7 Ridgeway CL. Framed by gender: how gender inequality persists in the modern world. Oxford: Oxford University Press, 2011.

8 Ioannadis JPA. Fund people not projects. Nature 2011; 477: 529–31. 9 Largent EA, Snodgrass RT. Blind peer review by academic journals. In:

Robertson CT, Kesselheim AS, eds. Blinding as a solution to bias: strengthening biomedical science, forensic science, and law. Cambridge: Academic Press, 2016; 75–95.

10 Irish Research Council Gender Strategy and Actions. http://research.ie/ assets/uploads/2018/08/04108-IRC-Gender-flyer-proof03-single.pdf (accessed Jan 15, 2019).

11 Sege R, Linley Nykiel-Bub L, Selk S. Sex differences in institutional support for junior biomedical researchers. JAMA 2015, 314: 1175–77.

12 Massachusetts Institute of Technology. A study on the status of women faculty in science at MIT, 1999. http://web.mit.edu/fnl/women/women.pdf (accessed Jan 16, 2019).

13 McCook A. Women in biotechnology: barred from the boardroom. Nature 2013; 495: 25–27.

14 Ding WW, Murray F, Stuart TE. From bench to board: gender differences in university scientists’ participation in corporate scientific advisory boards. Acad Manag J 2013; 56: 1443–64.

Preventing the tower from toppling for women in surgery In a rich qualitative analysis of interviews with women who left surgical training in Australia, Rhea Liang and colleagues report in The Lancet1 their study that applied insights from feminist and social theories to

illuminate how various factors interact to disadvantage women. They persuasively argue that various stresses accumulate like a tower of stacked blocks. Eventually, an individual’s tower can reach a height that it will

See Articles page 541

For #LancetWomen see https://www.thelancet.com/ lancet-women