Prposal essay
Changing the Rules of the Game: How Do We Measure Success in Social Media? Aisling M. Hogan, MD, FRCS(Gen)1 Desmond C. Winter, MD, FRCSI, FRCS(Gen)1
1Department of Surgery, St Vincent’s University Hospital, Dublin, Ireland
Clin Colon Rectal Surg 2017;30:259–263.
Address for correspondence Desmond C. Winter, MD, FRCSI, FRCS(Gen), Department of Surgery, St Vincent’s University Hospital, Elm Park, Dublin 4, D04 T6F4, Ireland (e-mail: deswinter@svcpc.ie; des.winter@gmail.com).
Just as children compete to build better sandcastles on the beach, the social comparison theory, popularized by Festinger in 1954,1 describes the need of individuals to gain accurate self-evaluation by comparing themselves to others. Surgeons, physicians, and academics are prime ex- amples of groups that highlight this intrinsic human need. Perhaps, it is fueled by an inherent competitive drive but comparison to peers fosters and nurtures the much-de- scribed stereotypical surgical ego.2 In few other groups, is mutual comparison sought with such vigor, from impact factors and H indices to “results” and “outcomes.”
Technological developments have always influenced scientific interactions. Many communication revolutions have been overcome from the birth of the postal service in the mid-19th century to the induction of the Computer Science Network (an embryonic form of the modern day internet) in 1981 and the most recent meteoric rise of
“social” media in the past decade. Its widespread availability renders it an extremely powerful tool for communication and interaction among the global scientific community. Generic availability of novel scientific data on “free” sites could and will revolutionize the practice of surgery on a global scale.
The power of social media is currently more than ever predicted and the implications of instant dissemination of surgical literature in the global sphere are vast. It has been demonstrated that highly “tweeted” articles are more likely to become highly cited.3 Thus, social media drives interested parties to a more conventional knowledge source (in search of full text articles on journal Web sites). A majority of journals are utilizing this tool to increase international readership and penetration. It appears then that gauge of twitter activity surrounding any scientific publication could be used by publishers as a predictor of future effect on
Keywords
► social media ► Twitter ► altmetrics ► Publons
Abstract Ours will be the generation proud to say we shifted the sands of educational deserts by open access and proliferation, seeding of data sharing, and watering grassroots research in resource-compromised environments. Universal “social” media is defining features of modern professional life that provide powerful modes of knowledge acquisition/sharing to that end. Altmetric and other measurements stratify academic communications according to this alternate, online media presence (not academic penetrance). Are they meaningless, self-absorbed integers, or reliable yardsticks of scientific and educational prowess? Far beyond this trite, patronizing question from the minds of outdated, terrified technophobes, the real impact of “social” media is not narcissistic solipsism. Instant dissemination of contemporary surgical controversies on a truly global level drives improved (or at least reflective) health care for all. While a numerical assignment of value according to views, “likes,” impressions, or “retweets” may seem meaningless to cynical, established academics, the impetus for universal improvement is self-evident. Electronic data and opinion sharing may not balance the inequity between low- and high-income countries, but it keeps it in perspective. The best way to shift desert sands is to blow on them constantly.
Issue Theme Hot Topics: Social Media and Surgery; Guest Editor: Kyle G. Cologne, MD, FACS, FASCRS
Copyright © 2017 by Thieme Medical Publishers, Inc., 333 Seventh Avenue, New York, NY 10001, USA. Tel: +1(212) 584-4662.
DOI https://doi.org/ 10.1055/s-0037-1604254. ISSN 1531-0043.
259
T h is
d o cu
m e n t w
a s
d o w
n lo
a d e d f o r
p e rs
o n a l u
se o
n ly
. U
n a u th
o ri ze
d d
is tr
ib u tio
n is
s tr
ic tly
p ro
h ib
ite d .
journal impact factor. Perhaps, dissemination of bite size messages on social media sites will replace the conventional peer-reviewed process and publications will be informally and pre-emptively selected based on popular vote? This remains to be seen, but it is clear a new system of evaluation is needed to judge success in the digital age, where social media has completely changed the rules of the game.
History
Surgeons and scientists are experts in balancing personal and professional applications of social media.4,5 The process of peer review has evolved, and although it still has relevance in its traditional form, the second process (that of online review by the broader community) which frequently occurs within hours of publication carries far more weight. The concept of open communication in science is not new.6 The acclaimed sociological historian Merton described the “norms” of scientific communication as Communism, Uni- versalism, Disinterestedness, and Organized Scepticism or “CUDOS” in 1979.7 It could be argued that Facebook, Twitter, Blogging, Instagram, and Symplur epitomize the pillars of Mertonian communication. The main barrier to information acquisition in the developing surgical community is mone- tary, but in the case of social media, content is freely available to screen devices with internet or cellular access. Seventy- eight percent of Twitter users take place on mobile devices. Even in the poorest communities, a majority of professionals have access to smartphones and Wi-Fi, thus revolutionizing the process of continuous professional development through medical education. This reduces the need for international travel to conferences for engagement with contemporary surgical practice. Geographic, temporal, and monetary boundaries are removed. Expert discourse and interchange serve as an “online journal club” and critique of published literature is available almost instantaneously.8
The sands of scientific writing are shifting so fast that the landslide is almost perceptible. The relevance of impact factor (popular since the 17th century) is being undermined by the outdated, manipulable, and time-lagged nature of the metric. The metric is calculated yearly, based on a yearly average number of citations compared with recent publica- tions. Higher impact factors have a greater relative impor- tance than smaller ones. Examples of impact factors are The New England Journal of Medicine: 59.5, Diseases of the Colon and Rectum: 3.7, and Clinics in Colon and Rectal Surgery: 0.714.
Arguments against the use of the impact factor as a metric include that there are no viable alternatives and are simply the hollow sophistry of a threatened elite. Flawed as these surrogate markers of academia may be, they did provide a common language for mutual rating, albeit one that was self- serving, quasi-mathematical, and clumsy. Traditionally, sim- plistic comparisons could be made and conclusions drawn. Journals were weighted, authors bore a public value, and we all spoke the same language. Journal publications form only one peak in the mountain range of scientific communication. Every article is reviewed and critiqued instantaneously and
opinions and criticisms are widely available (unlike the traditional model of anonymous peer review, where a small subset of reviewers determine the fate of an article—where the authors are not anonyms and thus bias or personal opinion may play a role in this process). In this new form of peer review (after online publication), the authors receive invaluable feedback, both positive and negative from a much wider audience. Will open access and a more transparent review process improve the quality of surgical publishing? Not necessarily, but there is wisdom in a crowd-shared approach. Knowledge was a fiercely protected resource in the 20th century. The 21st century will see a balancing of knowledge sharing on a global scale. It should be for the better.
Immediacy
Only a minute percentage of scientific communication oc- curs via an article medium and journals face being somewhat outdated before they emerge from the printing press. There are often significant delays from the time of data gathering to meeting/podium presentation to article submission, review, revision, and finally publication. These delays are being targeted by a new generation of medical professionals who grew up in a digital age where instantaneous access to free information is the new norm. The older cohorts of surgeons are being usurped by a new generation of communicators. Surgical literacy is evolving at a rapid pace. International communication is instantaneous and ubiquitous. Seventy- seven percent of all twitter accounts are held outside the United States. It is not unusual for a photograph of “the perfect specimen” to be tweeted and commented on from multiple continents before ever it is seen by a pathologist. This echoes international political reporting where events are presented with virtually no lag time, thus encouraging the need for immediate receipt of all news stories—either political or surgical. Measuring academic success in this era represents a new challenge. If an academic idea or statement is seen and “retweeted” by a million individuals worldwide, what weight should this have in comparison to a peer reviewed first or senior author publication (that may only be read by the authors and the editor, according to a study done on article in the 1980s).9
Social Media and Continuous Professional Development
There is an emerging need for evolution in the awarding of points for continuous professional development. Time spent debating current contentious issues in the public domain should and will be rewarded by the relevant bodies. It is at least equivalent (and usually educationally superior) to participation in conference discourse or publication of scien- tific communication. Never have educational resources for medical students and doctors in training been so contem- porary.10 Comparison of international practice has long been recognized as vital in the process of audit and this has been simplified by the popularization of social media. One
Clinics in Colon and Rectal Surgery Vol. 30 No. 4/2017
Changing the Rules of the Game Hogan, Winter260
T h is
d o cu
m e n t w
a s
d o w
n lo
a d e d f o r
p e rs
o n a l u
se o
n ly
. U
n a u th
o ri ze
d d
is tr
ib u tio
n is
s tr
ic tly
p ro
h ib
ite d .
hundred million users login to Twitter daily. There are 320 million monthly active users and 500 million tweets are sent per day. Thus, the impact of this mode of communication on global surgical practice from high- to low-income areas and back is immeasurable.
The responsibility of the boundaries of ethics on social media is yet to be defined within the surgical community.11
The question of specific consent of sharing of photography has yet to be explored. Although an individual specimen is unlikely to allow identification of an individual by any physical attributes, there are definite concerns about indi- viduals forming temporal relations and thus connecting a social media photograph to a specific patient.
It is impossible to ascertain whether the impetus for such social media activity is the drive to educate and foster meaningful discussion or (like a majority of Facebook activ- ity) a cry for endorsement and appreciation. Perhaps, open- ing and virtual chairing of international discourse on controversial topics should be awarded with a point system of sorts. There is no doubt but that following specialty leaders proves educational and economic of time and a majority of practice changing publications can be succinctly presented in 140 characters. While traditionalists voice concerns that many nuances are lost in brief communica- tions of scientific findings, in truth, a majority of readers draw conclusions based on abstracts alone. Furthermore, the subsequent traditional communications or letters to the editor do not receive nearly the same attention as the original article. The popularization of postcard style visual abstracts or infographics allow assimilation of multiple articles within several seconds. While this undoubtedly has a positive effect on surgical education, there is no single figure, ratio or coefficient that can easily quantify it. All that we can con- clude is that simple visual presentation of important surgical findings will disseminate widely, stimulate educational dis- cussion, drive readers to the source content, and up every- one’s game.
Impact in the First World
Impact factor is not a surrogate marker of penetration nor does it attempt to be. The present generation of university students has grown up in an era of instant gratification. An immediate “like” or a “retweet” is held in far higher regard than a rise in impact factor (up to 2 years later). Universities must embrace this shift in perception and begin to recognize and reward achievement appropriately. Otherwise, reliance on traditional appraisal methods will lead to inaccurate quantification of academic throughput. Real-time assess- ment of global impact of scientific output is, however, not a simple concept. There are various intricacies that need formalization before this process could be instituted—defini- tion of what constitutes a scientific communication, poten- tial weighting of authors based on past publication history, and valuation of a retweet, a like or a Web site hit.
Altmetrics12 is a relatively new science but has potential to offer guidance as to where to begin in the weighting or validation of social media use. It tracks articles (referenced
by a hyperlink) on various social media sites and a manually maintained list of Blogs. In time, the science of Altmetrics may be expanded to enable detailed and comprehensive analysis of individual researchers, bloggers, vloggers, Face- book, and Twitter users and to quantify their contribution to the wider scientific community. Conceived in 2010, it has attempted to address the quantification of success in social media. It analyses data of articles from supported publishers only and attempts to apply a weighting system where media felt to be further reaching are weighted heavier. It makes no effort to comment on or quantify the quality of the data itself, but its aim is to assess and place a numerical value on the “reach” of the article in the media. Articles cited in conven- tional news stories are given a weighting of eight times that of those cited on twitter. In this regard, the Altmetrics is outdated just as it becomes relevant. It is doubtless that the most prolific journal readers and “citers” spend far more time doing just that than on conventional mainstream news, so it is time for evolution.13
Definition of success is more complex than ever. Should there be a formula to incorporate the well-validated concepts of impact factor and H index with the emerging Altmetric value and number of social media followers? The concept of professional ranking leads to many unanswered questions.14
Does a following from a field leader equate with a follow from a friend? Should we consider a retweet to be a citation? And if so, every retweet? Although surgical journals are widely available, they are rarely accessed by patients or the lay community. This is not the case with social media. It has been suggested that access be tiered with certain content available to medical professionals only. However, this is at odds with one of the great strengths of social media—its universal availability, and thus, with Mertonian rules of scientific communication.
Valid questions have been asked regarding how appro- priate the use of generic social media sites is and whether there should be a separate platform for instant communica- tion within the medical community. In that instance, strati- fication of success would be easier (quantifying Symplur views). It is likely that definition of such a value would require intensive collaborative assessment by mathemati- cians and statisticians but, undoubtedly, we will be pre- sented with a formula to assess overall contribution to the worldwide scientific community within this decade. This will enable robust comparison of institutions, individuals, journals, and articles.
Publons and others have eased the scientific community into methods of scoring their online academic performance (i.e., journal referee tasks). Similarly, we have tried to score authors and journals for a long time without a good system. Several Web sites attempt to link the surgical community through publications and thus rank individual authors, often assigning a seemingly arbitrary number to each.15 However, the multiplicity of values assigned to every academic (on various sites) renders all of them meaningless. The merit in having a single value assigned to an author or journal is undisputed and allows indulgence in the generic need for social comparison.
Clinics in Colon and Rectal Surgery Vol. 30 No. 4/2017
Changing the Rules of the Game Hogan, Winter 261
T h is
d o cu
m e n t w
a s
d o w
n lo
a d e d f o r
p e rs
o n a l u
se o
n ly
. U
n a u th
o ri ze
d d
is tr
ib u tio
n is
s tr
ic tly
p ro
h ib
ite d .
Impact in the Developing World
Dissemination of surgical knowledge in the first world is easy. Access to institutional journal subscriptions, physical presence of highly trained experts, and easy access to well- renowned teaching institutions enable education. However, the global surgical community is far more diverse than this, and many that practice outside of large institutions may not have the financial resources to maintain such widespread access and license to read full text journal articles. Any medium that is universally available to support those making decisions at the cold face with no access to other educational resources cannot be underestimated. The immediacy of social media means that it is perfectly placed to provide instantaneous continuous professional development to those with few or no other resources. Publishers have a stranglehold on contemporary surgical research and innova- tion, and perhaps, the very best measure of success would be quantification of availability in otherwise educationally de- prived regions.
Ideally, all surgical communications would be open access for several reasons—removal of publishing biases, equity of access to education, homogenization, and optimization of practice. However, this is unlikely to occur in the multi- million dollar publishing empire. It is clear, however, that the rules of the game are changing. As first world surgeons, we have an obligation to ensure the diffusion of contemporary knowledge to our colleagues practicing in the developing world and contribute to universal equity in health care. Since social media drives those with easy access to journal Web sites and article downloads, one option is to use the revenue generated by publishers to make this content available in third world countries. This potential decrease in knowledge gradient would be the ultimate measure of success of social media and would represent a landmark in history as the antithesis of the 19th century resource stripping of the “third world” by the “first world” (as they were known in the self- indulgent, self-scored days of empires—narcissistic solipsism was alive long before Facebook). It is an opportunity to make reparation for previous injustices.
Weighting
The current system of peer review has been in place since the 17th century and it is universally acknowledged that change is necessary. Traditional markers of success in scientific publishing (citation index) refer to one author’s reference of another’s work. This includes an author’s references of their own previously published work. Conventional online publication allows for relative universalization of the review process. It may be that dissemination beyond traditional academic circles enables less biased and thus more mean- ingful review. However, instead of adopting a simplistic view of discarding the old system and adopting a new one, it is perhaps advisable to direct efforts toward finding practical ways of incorporating the best properties of our old system into the new one. Several questions remain unanswered: How can we accurately track informal usage such as reading,
sharing, and annotation (vs. just retweeting a link)? Is there a method of differentiating between articles’ popularity on social networking sites due to their “tabloid” value versus those which are discussed due to genuine scientific merit?
Paradoxically, the potential for instant dissemination of data internationally could lead to greater reluctance of researchers to communicate ideas or results. Scientific meet- ings are awash with bloggers and unpublished data in the form of infographics frequently appear on social media sites before a speaker has reached his concluding slide. Subse- quently, conference organizers often require attendees to obtain permission from speakers before blogging or tweet- ing. Preserving the integrity of science, and by extension society, in the era of Twitter and Facebook represents a significant challenge. In embracing technological progress, it is crucial to appraise new developments in communication with the same critical eye that is applied to laboratory data. Opportunity for expert discourse in real time enables open and unchartered critique of data. The opinions of experi- enced researchers and clinical practitioners are invaluable in guiding a trainee in interpretation of published work and drawing conclusions regarding it.
The introduction of a widely adopted weighting system of social media use would enable universal quality control. The Oxford grading system for levels of evidence (on the basis of study quality, imprecision, indirectness) endeavors to allow the scientific reader to immediately categorize findings prior to forming an idea on its quality. The mere publication of data in a peer-reviewed journal is perceived to ensure a minimal robustness and allows stratification of evidence. The concept of impact factor takes this one level further. However, in data or opinion presented on social media, there is no “filter.” Robust scientific findings of eminent leaders are as readily available to the general public as opinions of the entirely unqualified. The initiated recipient of informa- tion can easily distinguish these “levels of evidence,” but, with availability to the wider community, differentiation may be less clear. Thus, a simple figure (akin to the H index of conventional publication) would immediately allow the social media user to quantify the importance the data should be afforded.
The Present Is Already the Past
Universal social media is a defining feature of this generation of surgeons and academics. To its credit, it has embraced technology and used it to advantage but will this affect practice.16,17 Audit is a cornerstone of surgical development, but it is difficult to apply it to social media use because there is a merging of social and professional life with unclear margins. Quality assessment of information disseminated on social media is challenging to validate.18,19 To classify social media users into various categories, a complex, com- posite scoring system that belies the beauty of the internet would be required. Breaking down barriers to information sharing should be humanity’s collective goal, not building sandcastle monuments to our achievement. For now, success continues to have many definitions. These are likely to
Clinics in Colon and Rectal Surgery Vol. 30 No. 4/2017
Changing the Rules of the Game Hogan, Winter262
T h is
d o cu
m e n t w
a s
d o w
n lo
a d e d f o r
p e rs
o n a l u
se o
n ly
. U
n a u th
o ri ze
d d
is tr
ib u tio
n is
s tr
ic tly
p ro
h ib
ite d .
continue to evolve as the world grows more accustomed to new ways of knowledge sharing.
References 1 Allport GW. “The historical background of social psychology”. In:
Lindzey G, Aronson E, eds. The Handbook of Social Psychology. Vol. 5. New York: McGraw Hill; 1985
2 Fisher E, Youngs R, Hussain M, Fishman J. Service delivery: subspecialisation, emergencies, sharps injuries, personality traits and the ENT surgeon. J Laryngol Otol 2016;130(06):511
3 Eysenbach G. Can tweets predict citations? Metrics of social impact based on Twitter and correlation with traditional metrics of scientific impact. J Med Internet Res 2011;13(04):e123
4 McCartney M. How much of a social media profile can doctors have? BMJ 2012;344:e440
5 Ralston MR, O’Neill S, Wigmore SJ, Harrison EM. An exploration of the use of social media by surgical colleges. Int J Surg 2014; 12(12):1420–1427
6 Cook-Deegan R. The science commons in health research: struc- ture, function, and value. J Technol Transf 2007;32:133–156
7 Merton R. The Sociology of Science: Theoretical and Empirical Investigations. Chicago, IL: University of Chicago Press; 1979. ISBN: 978-0226520926
8 Chetlen AL, Dell CM, Solberg AO, et al. Another time, another space: the evolution of the virtual journal club. Acad Radiol 2017; 24(03):273–285
9 Hamilton DP. Publishing by – and for? – the Numbers. Science 1990;250:1331–1332
10 Reames BN, Sheetz KH, Englesbe MJ, Waits SA. Evaluating the use of Twitter to enhance the educational experience of a medical school surgery clerkship. J Surg Educ 2016;73(01): 73–78
11 Adams SA, Van Veghel D, Dekker L. Developing a research agenda on ethical issues related to using social media in healthcare. Camb Q Healthc Ethics 2015;24(03):293–302
12 Watson R. What are altmetrics and why would anyone be inter- ested? Nurs Open 2016;3(03):124
13 Scarlat MM, Mavrogenis AF, Pećina M, Niculescu M. Impact and alternative metrics for medical publishing: our experience with international orthopaedics. Int Orthop 2015;39(08):1459– 1464
14 Wexner SD, Petrucci AM, Brady RR, Ennis-O’Connor M, Fitzgerald JE, Mayol J. Social media in colorectal surgery. Colorectal Dis 2017;19(02):105–114
15 Memon AR. ResearchGate and impact factor: a step further on predatory journals. J Pak Med Assoc 2017;67(01):148–149
16 Davies N, Murphy DG, van Rij S, Woo HH, Lawrentschuk N. Online and social media presence of Australian and New Zealand urol- ogists. BJU Int 2015;116(06):984–989
17 McDonald JJ, Bisset C, Coleman MG, Speake D, Brady RR. Con- temporary use of social media by consultant colorectal surgeons. Colorectal Dis 2015;17(02):165–171
18 Park S, Oh HK, Park G, et al. The source and credibility of colorectal cancer information on Twitter. Medicine (Baltimore) 2016; 95(07):e2775
19 O’Neill S, Brady RR. Colorectal smartphone apps: opportunities and risks. Colorectal Dis 2012;14(09):e530–e534
Clinics in Colon and Rectal Surgery Vol. 30 No. 4/2017
Changing the Rules of the Game Hogan, Winter 263
T h is
d o cu
m e n t w
a s
d o w
n lo
a d e d f o r
p e rs
o n a l u
se o
n ly
. U
n a u th
o ri ze
d d
is tr
ib u tio
n is
s tr
ic tly
p ro
h ib
ite d .